oversight

DOD Business Systems Modernization: Continued Investment in Key Accounting Systems Needs to be Justified

Published by the Government Accountability Office on 2003-03-28.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

             United States General Accounting Office

GAO          Report to the Chairman, Subcommittee
             on National Security, Emerging Threats,
             and International Relations, Committee
             on Government Reform, House of
             Representatives
March 2003
             DOD BUSINESS
             SYSTEMS
             MODERNIZATION
             Continued Investment
             in Key Accounting
             Systems Needs to be
             Justified




GAO-03-465
             a
                                               March 2003


                                               DOD BUSINESS SYSTEMS
                                               MODERNIZATION

Highlights of GAO-03-465, a report to the      Continued Investment in Key Accounting
Chairman, Subcommittee on National
Security, Emerging Threats, and                Systems Needs to be Justified
International Relations, Committee on
Government Reform, House of
Representatives




The Department of Defense’s                    DOD estimated that it had 1,731 business systems for its day-to-day operations
(DOD) long-standing financial                  as of October 2002. As GAO previously reported, these systems have evolved
management and business systems                over time into the overly complex, error prone, duplicative, stovepiped
modernization problems result in a             environment that exists today. To support the operation, maintenance, and
lack of information needed to make             modernization of its business systems, the department requested approximately
sound decisions, hinder the                    $18 billion for fiscal year 2003. Funding is only part of the solution to improving
efficiency of operations, and leave            DOD’s current system environment. A key ingredient to success is effectively
the department vulnerable to fraud,            managing and overseeing these investments.
waste, and abuse. Such problems
led us in 1995 to put financial
                                               DOD has invested approximately $316 million in four key Defense Finance and
management and business systems
modernization at DOD on our list               Accounting Service (DFAS) projects. However, DOD has not demonstrated that
of high-risk areas in the federal              this substantial investment will markedly improve DOD financial management
government, a designation that                 information needed for decision-making and financial reporting purposes. In
continues today. GAO was asked                 fact, the DOD Comptroller terminated one project in December 2002, after an
to (1) provide information on the              investment of over $126 million, citing poor program performance and
number and cost of DOD’s current               increasing costs. Continued investment in the other three projects has not been
business systems and (2) determine             justified because requisite analyses of the costs, benefits, and risks of each one
if DOD is effectively managing and             do not reflect cost increases and/or schedule delays.
overseeing selected accounting
system investments.
                                               Cost Increases and Schedule Delays for the DFAS Projects Reviewed
                                               Dollars in millions
                                               DFAS system project                                    Cost increase          Schedule slippage
This report recommends that the                Corporate Database/Warehouse                                     $41                     4 years
Secretary of Defense limit future              Defense Procurement Payment                                     $274                     4 years
investments by the Defense                     Defense Departmental Reporting                                  $118                     5 years
Finance and Accounting Service                 Defense Standard Disbursing                                      N/A
                                                                                                                      a
                                                                                                                                        4 years
(DFAS) in three ongoing system
                                               Source: GAO based upon information provided by DFAS
projects we reviewed until each                a
                                                   Defense Standard Disbursing System began in 1997; however, a cost estimate was not
has been adequately justified. The
report also recommends that all                developed until September 2000 and this estimate has not been updated.
other DFAS information
technology projects be evaluated to            DOD oversight of the four DFAS projects has not been effective. Collectively,
ensure they are being implemented              DFAS, the DOD Comptroller, and the DOD Chief Information Officer share
at acceptable costs and within                 investment management responsibility for these four projects. However, these
reasonable time frames.                        DOD oversight entities have not questioned the impact of the cost increases and
                                               schedule delays and allowed the projects to proceed absent the requisite
DOD concurred with our                         analytical justification.
recommendations and described
efforts to address them.

www.gao.gov/cgi-bin/getrpt?GAO-03-465.

To view the full report, including the scope
and methodology, click on the link above.
For more information, contact Gregory Kutz,
(202) 512-9095 (kutzg@gao.gov) or
Randolph Hite (202) 512-3439
(hiter@gao.gov).
Contents



Letter                                                                                                                   1
                             Results in Brief                                                                            2
                             Background                                                                                  3
                             DOD Is Investing Billions of Dollars Annually to Operate, Maintain,
                               and Modernize Its Business Systems                                                       11
                             Investment Management and Oversight of Key DFAS Accounting
                               Systems Has Not Been Effective                                                           16
                             Conclusions                                                                                32
                             Recommendations for Executive Action                                                       33
                             Agency Comments and Our Evaluation                                                         34


Appendixes
              Appendix I:    Scope and Methodology                                                                      36
             Appendix II:    Comments From the Under Secretary of Defense                                               39
             Appendix III:   GAO Contacts and Staff Acknowledgments                                                     41


Tables                       Table 1: DFAS Fiscal Year 2002 Financial Operations                                         4
                             Table 2: Reported DOD Business Systems by Functional Area                                  12
                             Table 3: Proposed Allocation of DOD’s Fiscal Year 2003 IT
                                      Budget                                                                            15
                             Table 4: Reported Cost Increases and Schedule Delays for the Four
                                      DFAS Projects Reviewed                                                            17
                             Table 5: Reported Investment in DPPS, DDRS, and DSDS                                       22


Figures                      Figure 1: Intended Interrelationship of the Four DFAS Projects                              7
                             Figure 2: Distribution of DOD’s Fiscal Year 2003 $26 Billion IT
                                       Budget Request (dollars in billions)                                             14
                             Figure 3: DCD/DCW Schedule Slippages and Cost Increases                                    20
                             Figure 4: DPPS Schedule Slippages and Cost Increases                                       23
                             Figure 5: DDRS Cost Increases and Schedule Delays                                          25
                             Figure 6: DSDS Schedule Delays                                                             27
                             Figure 7: ITIM Stages of Maturity                                                          37


                              This is a work of the U.S. Government and is not subject to copyright protection in the
                              United States. It may be reproduced and distributed in its entirety without further
                              permission from GAO. It may contain copyrighted graphics, images or other materials.
                              Permission from the copyright holder may be necessary should you wish to reproduce
                              copyrighted materials separately from GAO’s product.




                             Page i                                   GAO-03-465 DOD Business Systems Modernization
A
United States General Accounting Office
Washington, D.C. 20548



                                    March 28, 2003                                                                                  Leter




                                    The Honorable Christopher Shays
                                    Chairman
                                    Subcommittee on National Defense, Emerging Threats,
                                      and International Relations
                                    Committee on Government Reform
                                    House of Representatives

                                    Dear Mr. Chairman:

                                    The Department of Defense’s (DOD) long-standing financial management
                                    and business systems1 modernization problems result in a lack of
                                    information needed to make sound decisions, hinder the efficiency of
                                    operations, and leave the department vulnerable to fraud, waste, and
                                    abuse. Such problems led us in 1995 to put financial management and
                                    business systems modernization at DOD on our list of high-risk areas2 in
                                    the federal government, a designation that continues today.3 DOD’s
                                    financial management problems are the result of challenges in the area of
                                    human capital, processes (internal controls), and its inability to effectively
                                    modernize its business systems.

                                    The President has made financial management and the use of technology
                                    integral to his fiscal year 2002 Management Agenda for making the federal
                                    government more focused on results. The President’s Management
                                    Agenda states, “Without accurate and timely information it is not possible
                                    to accomplish the President’s agenda to secure the best performance and
                                    the highest measure of accountability for the American people.”
                                    Additionally, in September 2002, the Secretary of Defense identified the
                                    modernization of DOD’s financial management and business operations as
                                    one of his top 10 priorities.




                                    1
                                      Business systems include those that are used for areas such as civilian personnel, finance,
                                    health, logistics, military personnel, procurement, and transportation, with the common
                                    element being the generation or use of financial data.
                                    2
                                      U.S. General Accounting Office, High-Risk Series: An Overview, GAO-HR-95-263
                                    (Washington, D.C.: February 1995).
                                    3
                                      U.S. General Accounting Office, High-Risk Series: An Update, GAO-03-119 (Washington,
                                    D.C.: January 2003).




                                    Page 1                                    GAO-03-465 DOD Business Systems Modernization
                   This report responds to your request for information about DOD’s current
                   and planned business systems environment and its management of certain
                   key accounting system projects. As agreed with your office, our objectives
                   were to (1) identify the number of existing business systems and the
                   estimated cost to operate, maintain, and modernize systems and (2)
                   determine if DOD is effectively overseeing selected business system
                   investments made by the Defense Finance and Accounting Service
                   (DFAS)—the centralized accounting agency for DOD.

                   To evaluate the effectiveness of information technology (IT) investment
                   management and oversight practices, we selected four DFAS system
                   acquisition projects as case studies. We selected these projects because all
                   four were intended to benefit the entire department by addressing DOD’s
                   financial management weaknesses in the areas of data accuracy, financial
                   reporting, and problem disbursements. In reviewing these four system
                   projects, we relied on documentation, including cost estimates, provided
                   by DFAS. We did not verify the accuracy and completeness of the cost
                   information provided by DFAS. Our work was performed from November
                   2001 to January 2003 in accordance with U.S. generally accepted
                   government auditing standards. Details on our scope and methodology are
                   included in appendix I. We requested comments on a draft of this report
                   from the Secretary of Defense or his designee. Written comments from the
                   Under Secretary of Defense (Comptroller) are reprinted in appendix II.



Results in Brief   As part of its ongoing business systems modernization program, DOD is
                   creating a repository of information about its existing systems
                   environment. As of October 2002, DOD reported that its current business
                   system environment consisted of 1,731 DOD systems and system
                   acquisition projects. DOD officials said they believe the inventory is fairly
                   comprehensive, given the in-depth work the department has undertaken
                   over the past 18 months to develop this information, but acknowledge that
                   it likely does not include all systems. More importantly, as we testified
                   before this Subcommittee,4 these systems are acknowledged by DOD to be
                   error prone, duplicative, and stovepiped. To support its existing business
                   systems environment, the department requested approximately $18 billion
                   for fiscal year 2003. However, funding alone is not the solution to


                   4
                     U.S. General Accounting Office, DOD Financial Management: Important Steps
                   Underway But Reform Will Require a Long-term Commitment, GAO-02-784T (Washington,
                   D. C.: June 4, 2002).




                   Page 2                              GAO-03-465 DOD Business Systems Modernization
             improving DOD’s current systems environment. A key to success is
             effectively managing and overseeing its investments in systems.

             DOD has not effectively managed and overseen its planned investment of
             over $1 billion in four DFAS system modernization efforts. Specifically, one
             project did not have an economic analysis, and the other three did not have
             approved economic analyses that reflected the fact that expected project
             costs had increased, while in some cases the benefits had decreased. One
             project’s estimated cost had increased by as much as $274 million, while
             the schedule slipped by almost 4 years. Such analyses provide the requisite
             justification for decision makers to use in determining whether to invest
             additional resources in anticipation of receiving commensurate benefits
             and mission value. For each of these projects, DOD oversight entities—
             DFAS, the DOD Comptroller, and the DOD Chief Information Officer
             (CIO)—could not provide documentation that indicated they questioned
             the impact of the cost increases and schedule delays, and allowed the
             projects to proceed in the absence of the requisite analytical justification.
             For example, in one case, they allowed a $270 million project to proceed
             without an economic analysis. In another case, they allowed a project to
             continue despite known concerns with the validity of the project’s
             economic analysis. After spending over $126 million as of September 2002,
             the DOD Comptroller terminated the latter project in December 2002,
             citing poor program performance and increasing costs.

             This report recommends that the Secretary of Defense limit future
             investments in the remaining three DFAS system projects until such
             investments have been adequately justified. Additionally, we are
             recommending that all remaining DFAS IT projects be evaluated to ensure
             they are being implemented at acceptable cost and within reasonable time
             frames. In its comments on a draft of this report, DOD agreed with our
             recommendations and briefly outlined its actions for addressing them.



Background   DFAS, as DOD’s central accounting agency, is responsible for recording and
             processing accounting transactions; paying vendors, contractors, and
             military and civilian employees; preparing reports used by DOD managers
             and by the Congress; and preparing DOD-wide and service-specific
             financial statements required by the Chief Financial Officers Act.
             Organizationally, DFAS is under the direction of the Under Secretary of
             Defense (Comptroller). Table 1 illustrates the enormous scope and
             importance of DFAS’s reported fiscal year 2002 financial operations.




             Page 3                             GAO-03-465 DOD Business Systems Modernization
Table 1: DFAS Fiscal Year 2002 Financial Operations

Type of activity                                                        Volume of activity
Accounting transactions                                                 124 million
Disbursements made                                                      $346.6 billion
Invoices paid                                                           11.2 million
Military and civilian employees, retirees, and annuitants paid          5.7 million
Active DOD appropriations                                               267
Source: DFAS


DFAS’s fiscal year 2003 IT budgetary request was approximately $494
million.5 Of that amount, $353 million relates to the operation and
maintenance of existing DFAS systems and the remaining $141 million is
for the modernization of systems. The purpose of each DFAS project we
reviewed is highlighted below.

DFAS Corporate Database/DFAS Corporate Warehouse (DCD/DCW). DCD
and DCW were originally separate initiatives. DCD was initiated in October
1998, and was to be the single DFAS database, meaning it was to contain all
DOD financial information required by DFAS systems and would be the
central point for all shared data within DFAS. To accomplish this goal,
DCD would crosswalk6 detailed transaction data from nonstandard finance
and feeder7 systems into a standard format. Further, once the department
implemented standard systems, the need to perform these crosswalks
would be eliminated. In February 2001, the project’s scope was revised
after DFAS realized that crosswalks of detail transaction data were
cumbersome and cost prohibitive. DFAS is planning to crosswalk detailed
transaction data only when information from multiple systems must be
aggregated to satisfy a cross-service need such as the working capital fund
activities.



5
    DOD Information Technology Fiscal Year 2003 Budget Estimate, February 2002.
6
  Translate information so that systems that define data differently (i.e., use different data
schemes) can understand each other and communicate accurately.
7
  Feeder systems are outside the direct control of DFAS. These are the systems used by
DOD’s various functional areas such as acquisition, logistics, and personnel. DOD has
estimated that 80 percent of the department financial management data comes from the
feeder systems controlled by the military services and defense agencies.




Page 4                                     GAO-03-465 DOD Business Systems Modernization
DCW was initiated in July 2000 to provide a historical database to store and
manage official DFAS information for analysis and generation of
operational reports and queries. In November 2000, the DFAS CIO
combined DCD/DCW into one program. In March 2001, DCD/DCW was
designated as a major automated information system.8

Defense Procurement Payment System (DPPS). DFAS determined the need
for DPPS in April 1995. DPPS was intended to be the standard, automated
information system for contract and vendor pay authorization and
addressing deficiencies associated with overpayments, negative
unliquidated obligations,9 and unmatched disbursements10—all of which
are long-standing problems in DOD. DPPS also was to incrementally
replace eight contract and vendor systems. In October 1995, the DFAS
Director approved proceeding with defining and evaluating the feasibility
of alternative concepts and assessing the relative merits of these concepts.
In November 1996, the Office of the Assistant Secretary of Defense
(Command, Control, Communications, and Intelligence)—DOD’s CIO—
designated DPPS a major automated information system. DFAS awarded a
contract in June 1998 for the acquisition of a system that was intended to
address DOD’s contract and vendor pay deficiencies.

Defense Standard Disbursing System (DSDS). Disbursing activities for
DOD are largely accomplished through systems that were designed 15-20
years ago. In 1997, DFAS launched DSDS to be the single, standard DFAS
automated information system for collecting, processing, recording, and
reporting disbursement data and transactions for the military services and
defense agencies. These disbursing functions are currently being provided


8
  Major automated information systems are defined as IT projects with (1) program costs in
any single year that exceed $32 million, (2) total program costs that exceed $126 million, or
(3) total life-cycle costs that exceed $378 million. The life-cycle cost is the total cost to the
government for an information system over its expected useful life and includes the costs to
acquire, operate, maintain, and dispose of the system. DOD Regulation 5000.2-R,
Mandatory Procedures for Major Defense Acquisition Programs and Major Automated
Information System Acquisition Programs, specifies mandatory policies and procedures
for major acquisitions. The policy also specifies that the DOD CIO is the milestone decision
authority, responsible for program approval, for all major automated information systems,
such as DCD/DCW and DPPS.
9
  Negative unliquidated obligations occur when recorded disbursements exceed recorded
obligations, indicating that expenditures may exceed amounts obligated.
10
  Unmatched disbursements occur when a disbursement cannot be matched to an
obligation.




Page 5                                      GAO-03-465 DOD Business Systems Modernization
by multiple automated information systems and manual activities at
various DFAS locations.

Defense Departmental Reporting System (DDRS). In April 1997, DFAS
initiated DDRS to be the standardized departmental reporting system.
DDRS has two phases. The first phase—DDRS-AFS (Audited Financial
Statements)—is intended to be a departmentwide financial reporting
system. The second phase—DDRS-Budgetary—is intended to establish a
departmentwide budgetary reporting system. Among other things, DDRS is
intended to reduce the number of departmental reporting systems and
standardize departmental general ledger processes.

These four projects are part of the DFAS Corporate Information
Infrastructure (DCII) program. According to DFAS, DCII is intended to
facilitate cross-functional, integrated processes; promote standardized data
and reporting; facilitate standardized business practices; reduce cost of
operations; and provide timely information for decision making. Figure 1
depicts a high-level view of the interrelationships among these four system
projects.




Page 6                            GAO-03-465 DOD Business Systems Modernization
Figure 1: Intended Interrelationship of the Four DFAS Projects

                                  Financial feeder a and accounting and finance systems




                                                                       DCD / DCW
                     DCD is intended to facilitate the sharing of all DOD financial data among systems and functions, applications, and users.
                     DCW is intended to serve as a shared data warehouse that provides uniform, centralized information to DOD's systems.




                       DDRS                                               DPPS                                                   DSDS
           DDRS is intended to use data from               DPPS uses data from procurement                        DSDS is intended to collect, process,
                                                                           b
           the DCD and DCW to prepare                      systems, EC/EDI transactions,                          record, and report DOD's disbursement
           standard financial and budgetary                electronic document management,                        data and transactions, and serve as
           reports.                                        and source data entry (all obtained                    the standard vendor and contractor
                                                           from DCD) to calculate contract                        payment system.
                                                           and vendor payments.




                     Reports go to                                                                                             Payment to
                       end user                                                                                            contractors/vendors
Source: GAO.

                                                      a
                                                        Feeder systems are outside the direct control of DFAS. These are the systems used by DOD’s
                                                      various functional areas such as acquisition, logistics, and personnel. DOD has estimated that 80
                                                      percent of the department’s financial management data comes from the feeder systems controlled by
                                                      the military services and defense agencies.
                                                      b
                                                        Electronic commerce/electronic data interchange. Electronic commerce is the interchange and
                                                      processing of information using electronic techniques for accomplishing business. Electronic data
                                                      interchange is the computer-to-computer exchange of business data in a standardized format between
                                                      entities.




                                                      Page 7                                           GAO-03-465 DOD Business Systems Modernization
Overview of DFAS and DOD   DOD and DFAS have an established acquisition management and oversight
System Acquisition         process for acquiring, operating, and maintaining business systems. Among
                           other things, this process requires project managers to provide cost,
Management and Oversight   schedule, and performance data to the DFAS Chief Information
Process                    Officers/Business Integration Executive (CIO/BIE) Council—DFAS’s IT
                           investment board—prior to scheduled milestone11 reviews. These
                           milestones are intended to be decision points for determining whether a
                           project should continue in the current phase of the system life-cycle,
                           proceed to the next phase, be modified, or be terminated. The results of
                           these reviews are to be set forth in a system decision memorandum which
                           is to be signed by the milestone decision authority. The milestone decision
                           authority for DSDS and DDRS is the Director, DFAS. The DOD CIO is the
                           milestone decision authority for DCD/DCW and DPPS.



Prior Reviews Have         We and the DOD Inspector General have continued to report on a variety of
Identified Problems With   long-standing management problems for modernizing DOD’s IT systems.
                           Three recent system endeavors that have fallen short of their intended
DOD’s Management and       goals illustrate these problems. They are the Standard Procurement
Oversight of System        System, the Defense Travel System, and the Defense Joint Accounting
Acquisitions               System. These efforts were aimed at improving the department’s financial
                           management and related business operations. Significant resources—in
                           terms of dollars, time, and people—have been invested in these three
                           efforts.




                           11
                             DFAS’s system life-cycle process is consistent with DOD’s Defense Acquisition System
                           guidance, which has three milestones: Milestone A or Concept and Technology
                           Development, Milestone B or System Development and Demonstration, and Milestone C or
                           Production and Deployment. The Defense Acquisition System guidance was revised in
                           October 2000. Prior to this, Milestone A was Milestone 0, Milestone B was Milestone I/II,
                           and Milestone C was Milestone III.




                           Page 8                                  GAO-03-465 DOD Business Systems Modernization
Standard Procurement System (SPS). In November 1994, DOD began the
SPS program to acquire and deploy a single automated system to perform
all contract management-related functions within DOD’s procurement
process for all DOD organizations and activities. The laudable goal of SPS
was to replace 76 existing procurement systems with a single departmental
system. DOD estimated that SPS had a life-cycle cost of approximately $3
billion over a 10-year period. According to DOD, SPS was to support about
43,000 users at over 1,000 sites worldwide and was to interface with key
financial management functions, such as payment processing.
Additionally, SPS was intended to replace the contract administration
functions currently performed by the Mechanization of Contract
Administration Services, a system implemented in 1968. Our July 2001
report12 and February 2002 testimony13 identified weaknesses in the
department’s management of its investment in SPS. Specifically:

• The department had not economically justified its investment in the
  program because its latest (January 2000) analysis of costs and benefits
  was not credible. Further, this analysis showed that the system, as
  defined, was not a cost-beneficial investment.

• The department had not effectively addressed the inherent risks
  associated with investing in a program as large and lengthy as SPS
  because it had not divided the program into incremental investment
  decisions that coincided with incremental releases of system
  capabilities.

• Although the department committed to fully implementing the system
  by March 31, 2000, this target date had slipped by over 3 ½ years to
  September 30, 2003, and program officials have recently stated that this
  date will also not be met.




12
  U.S. General Accounting Office, DOD Systems Modernization: Continued Investment in
the Standard Procurement System Has Not Been Justified, GAO-01-682 (Washington, D.C.:
July 31, 2001).
13
  U.S. General Accounting Office, DOD’s Standard Procurement System: Continued
Investment Has Yet to Be Justified, GAO-02-392T (Washington, D.C.: Feb. 7, 2002).




Page 9                                 GAO-03-465 DOD Business Systems Modernization
Defense Travel System (DTS). In July 2002,14 the DOD Inspector General
raised concerns that DTS remained a program at high risk of not being an
effective solution in streamlining the DOD travel management process.
The report stated that “The Defense Travel System was being substantially
developed without the requisite requirements, cost, performance, and
schedule documents and analyses needed as the foundation for assessing
the effectiveness of the system and its return on investment.” The report
further noted there was increased risk that the $114.8 million and 6 years of
effort already invested will not fully realize all goals to reengineer
temporary duty travel, make better use of IT, and provide an integrated
travel system. Additionally, the DOD Inspector General reported that DTS
was to cost approximately $491.9 million (approximately 87 percent more
than the original contract cost of $263.7 million) and DOD estimates that
deployment will not be completed until fiscal year 2006, approximately 4
years behind schedule.

Defense Joint Accounting System (DJAS). In 1997, DOD selected DJAS15 to
be one of three general fund accounting systems. The other two general
fund systems were the Standard Accounting and Reporting System and the
Standard Accounting and Budgetary Reporting System. As originally
envisioned, DJAS would perform the accounting for the Army and the Air
Force as well as the DOD transportation and security assistance areas.
Subsequently, in February 1998, DFAS decided that the Air Force could
withdraw from using DJAS, because either the Air Force processes or the
DJAS processes would need significant reengineering to permit use of a
joint accounting system. As a result, the Air Force started its own general
fund accounting system—General Fund and Finance System—which
resulted in the development of a fourth general fund accounting system.

In June 2000, the DOD Inspector General reported16 that DFAS was
developing DJAS at an estimated life-cycle cost of about $700 million


14
  Department of Defense Office of the Inspector General, Allegations to the Defense Hotline
on the Management of the Defense Travel System, Report No. D-2002-124 (Arlington, Va.:
July 1, 2002).
15
  The original name of the system was the Corps of Engineers Financial Management
System (CEFMS). After it was determined that CEFMS could be modified to satisfy Army
customers and had the potential for supporting the Defense Working Capital Funds, DFAS
selected CEFMS to meet the DJAS requirements.
16
 Department of Defense Office of the Inspector General, Acquisition of the Defense Joint
Accounting System, Report No. D-2000-151 (Arlington, Va.: June 16, 2000).




Page 10                                  GAO-03-465 DOD Business Systems Modernization
                         without demonstrating that the program was the most cost-effective
                         alternative for providing a portion of DOD’s general fund accounting. More
                         specifically, the report stated that DFAS had not developed a complete or
                         fully supportable feasibility study, analysis of alternatives, economic
                         analysis, acquisition program baseline, or performance measures, and had
                         not reengineered business processes.



DOD Is Investing         As part of its ongoing business systems modernization program, and
                         consistent with our past recommendation,17 DOD is creating a repository of
Billions of Dollars      information about its existing systems environment. As of October 2002,
Annually to Operate,     DOD reported that its current business systems environment consisted of
                         1,731 systems and system acquisition projects. In particular, DOD reported
Maintain, and            that it had 374 systems to support civilian and military personnel matters,
Modernize Its Business   335 systems to perform finance and accounting functions, and 310 systems
Systems                  that produce information for management decision making. Table 2
                         presents the composition of DOD business systems by functional area.




                         17
                           U.S. General Accounting Office, Financial Management: DOD Improvement Plan Needs
                         Strategic Focus, GAO-01-764 (Washington, D.C.: Aug. 17, 2001).




                         Page 11                              GAO-03-465 DOD Business Systems Modernization
Table 2: Reported DOD Business Systems by Functional Area

Functional area                                                            Army          Navy        Air Force        DFAS            DLA        Other           Total
Personnel                                                                     266           49               13          19               0          27           374
Finance and accounting                                                         79           61               27         131              9           28           335
Management information                                                        156           40               50          14               4          46           310
Inventory                                                                      98           53               40            7              7          17           222
Acquisition                                                                    18           10               22            0              5          19             74
Budget formulation                                                             25           18               10            5              0          10             68
Cost                                                                           19           29                 8           0              1            4            61
Logistics                                                                      12             6              22            3              7            5            55
National defense property management                                            5           12               25            1              2            1            46
Travel                                                                          9           13                 3           2              0            5            32
Real property management                                                       17             4                6           0             0             1            28
Time and attendance                                                             3           14                 2           2             3             1            25
Budget execution                                                                6             4                2           7              0            3            22
Personal property management                                                    3             7                7           0              0            4            21
Procurement                                                                     7             5                1           0              3            4            20
Vendor payment                                                                  3             3                1           7              0            4            18
Transportation                                                                  5             1                4           0              0            2            12
Other functions combined                                                       12             7                6           3             0             9            37
Total                                                                         743          336              249         201             41          190        1,760a
Source: DOD Business Modernization Systems Integration Office.
                                                                 a
                                                                  There are 29 reported duplications within the DOD database (e.g., systems shown in multiple
                                                                 functional areas). Taking this duplication into account provides the reported 1,731 business systems.


                                                                 As we have previously reported,18 these numerous systems have evolved
                                                                 into the overly complex and error prone operation that exists today,
                                                                 including (1) little standardization across DOD components, (2) multiple
                                                                 systems performing the same tasks, (3) the same data stored in multiple
                                                                 systems, (4) manual data entry into multiple systems, and (5) a large
                                                                 number of data translations and interfaces that combine to exacerbate
                                                                 problems with data integrity. The department has recognized the
                                                                 uncontrolled proliferation of systems and the need to eliminate as many
                                                                 systems as possible and integrate and standardize those that remain. In


                                                                 18
                                                                   U.S. General Accounting Office, DOD Financial Management: Important Steps
                                                                 Underway But Reform Will Require a Long-term Commitment, GAO-02-784T (Washington,
                                                                 D.C.: June 4, 2002).




                                                                 Page 12                                       GAO-03-465 DOD Business Systems Modernization
fact, three of the four DFAS projects we reviewed were intended to reduce
the number of systems or eliminate a portion of different systems that
perform the same function. For example,

• DPPS was intended to consolidate eight contract and vendor pay
  systems;

• DDRS is intended to reduce the number of departmental reporting
  systems from seven to one; and

• DSDS is intended to eliminate four different disbursing systems.

Similarly, DTS is intended to be the DOD-wide travel system. According to
data reported by DOD, currently there are 32 travel systems operating
within the department.

For fiscal year 2003, DOD has requested approximately $26 billion in IT
funding to support a wide range of military operations as well as DOD
business system operations. As shown in figure 2, the $26 billion is spread
across the military services and defense agencies. Each receives its own
funding for IT investments.




Page 13                           GAO-03-465 DOD Business Systems Modernization
Figure 2: Distribution of DOD’s Fiscal Year 2003 $26 Billion IT Budget Request
(dollars in billions)
                                                                              a
                                                                   3% DLA $0.9 Billion


                                                                   3% Tri-Care $0.9 Billion
                                            14%
                 24%                   All other DOD
              Air Force                componentsb
              $6.4 Billion              $3.7 Billion

                                                15%
                                                      c
                                               DISA
              21%                            $3.9 Billion
              Navy
           $5.5 Billion          20%
                                Army
                              $5.2 Billion



    Source: GAO analysis based on DOD's fiscal year 2003 IT budget request.


a
  Defense Logistics Agency is DOD’s logistics manager for all consumable and some repair items; its
primary business function is providing supply support to sustain military operations and readiness.
b
    Other DOD components include entities such as DFAS and the Defense Commissary Agency.
c
 Defense Information Systems Agency provides DOD and other organizations a wide range of
information services such as data processing, telecommunication services, and database
management.


The $26 billion supports three categories of IT—business systems, business
systems infrastructure, and national security systems (NSS)—the first two
of which comprise the 1,731 business systems. DOD defines these three
categories as follows:

• Business systems—used to record the events associated with DOD’s
  functional areas. Such areas include finance, logistics, personnel, and
  transportation.

• Business systems infrastructure—represents the costs associated with
  the operations of the department’s business systems. Such costs would
  include transmission lines, network management, and information
  security.

• National Security System (NSS)—intelligence systems, cryptologic
  activities related to national security, military command and control



Page 14                                                      GAO-03-465 DOD Business Systems Modernization
     systems, and equipment that is an integral part of a weapon or weapons
     system, or is critical to the direct fulfillment of military or intelligence
     mission.

As shown in table 3, approximately $18 billion—the nearly $5.2 billion for
business systems and the $12.8 billion for business systems
infrastructure—relates to the operation, maintenance, and modernization
of DOD’s 1,731 business systems.



Table 3: Proposed Allocation of DOD’s Fiscal Year 2003 IT Budget

Dollars in millions
                                                                         Business
                                                 Business                  system
Component                                         systems           infrastructure    NSS         Total
Air Force                                               $578               $3,178    $2,674     $6,430
Navy                                                   1,397                2,674     1,444      5,515
Army                                                   1,122                2,223     1,823      5,168
DISA                                                       45               3,190      648       3,884
Tricare                                                   444                 452       15         911
DLA                                                       434                 407       24         865
Other DOD components                                   1,135                  676     1,843      3,653
Total                                                $5,155               $12,800    $8,471    $26,426
Source: GAO analysis based on DOD’s fiscal year 2003 IT budget request


As we have reported,19 while DOD plans to invest billions of dollars in
modernizing its financial management and other business support systems,
it does not yet have an overall blueprint—or enterprise architecture—in
place to guide and direct these investments. Our review of practices at
leading organizations showed they were able to provide reasonable
assurance that their business systems addressed corporate—rather than
individual business units—objectives by using enterprise architectures to
guide and constrain investments.20


19
  U.S. General Accounting Office, Information Technology: Architecture Needed to Guide
Modernization of DOD’s Financial Operations, GAO-01-525 (Washington, D.C.: May 17,
2001).
20
  U.S. General Accounting Office, Defense Acquisitions: DOD Faces Challenges in
Implementing Best Practices, GAO-02-469T (Washington, D.C.: Feb. 27, 2002).




Page 15                                                   GAO-03-465 DOD Business Systems Modernization
                         Consistent with our recommendation,21 DOD is now working to develop a
                         financial management enterprise architecture, which is a positive step.
                         Further, Section 1004 of the National Defense Authorization Act for Fiscal
                         Year 200322 directs DOD to develop an enterprise architecture not later than
                         May 1, 2003, and that a transition plan accompany the architecture that
                         delineates how the architecture will be implemented. The act also directs
                         that we provide an assessment to the congressional defense committees as
                         to whether DOD has complied with the provisions of Section 1004.



Investment               DOD management and oversight authorities for the four case study
                         projects are DFAS, the DOD Comptroller, and the DOD CIO. They
Management and           permitted each project to proceed despite the absence of the requisite
Oversight of Key DFAS    analysis to demonstrate that the projects will produce value commensurate
                         with the costs being incurred. For example, an economic analysis has yet
Accounting Systems       to be prepared for DCD/DCW and the other three projects did not have
Has Not Been Effective   economic analyses that reflected the fact that project costs, schedules,
                         and/or expected benefits had changed materially. Table 4 highlights these
                         cost increases and schedule delays.




                         21
                           U.S. General Accounting Office, Information Technology: Architecture Needed to Guide
                         Modernization of DOD’s Financial Operations, GAO-01-525 (Washington, D.C.: May 17,
                         2001).
                         22
                              P.L. 107-314, December 2, 2002.




                         Page 16                                GAO-03-465 DOD Business Systems Modernization
Table 4: Reported Cost Increases and Schedule Delays for the Four DFAS Projects
Reviewed

Dollars in millions
                                                     Original planned    Current planned
                                                     date (fiscal year)  date (fiscal year)
                         Original cost Current cost of full operational of full operational
System                       estimate     estimate           capabilitya         capability
DCD/DCWb                             $229               $270                2001                    2005
DPPS                                 $278               $552                2002                    2006
DDRS                                 $ 52               $170                1999                    2004
DSDS                                $151c               $151                2002                    2006
Total                                $710              $1,143
Source: GAO based upon information provided by DFAS.
a
    Full operational capability means the system is deployed and operating at all intended locations.
b
 When DFAS initiated the DCW in July 2000, a full operational capability date was not established.
The current full operational capability date applies to both DCD and DCW since they were combined
into one program in November 2000.
c
 DSDS began in 1997; however, a cost estimate was not developed until September 2000, and this
estimate has not been updated.


In the case of DPPS, the estimated costs had increased by $274 million and
the schedule had slipped by almost 4 years. In December 2002, following
our discussions with DOD Comptroller officials, the DOD Comptroller
terminated DPPS after 7 years of effort and an investment of over $126
million. In making this decision, the DOD Comptroller noted that the
project was being terminated due to poor program performance and
increasing costs.




Page 17                                                GAO-03-465 DOD Business Systems Modernization
                         The Clinger-Cohen Act of 1996 and Office of Management and Budget
                         (OMB) guidance provide an effective framework for IT investment
                         management.23 They emphasize the need to have investment management
                         processes and information to help ensure that IT projects are being
                         implemented at acceptable costs and within reasonable and expected time
                         frames and that they are contributing to tangible, observable improvements
                         in mission performance. DOD policy also reflects these investment
                         principles by requiring that investments be justified by an economic
                         analysis.24 More specifically, the policy states that the economic analysis is
                         to reflect both the life-cycle cost and benefit estimates, including a return-
                         on-investment calculation, to demonstrate that the proposed investment is
                         economically justified before it is made.



DCD/DCW Has Not Been     After 4 years of effort and an investment of approximately $93 million,
Economically Justified   DOD has yet to economically justify that its investment in DCD/DCW will
                         result in tangible improvement in DOD financial management operations.
                         Consistent with the Clinger-Cohen Act, DOD and DFAS systems acquisition
                         guidance25 requires that certain documentation be prepared at each
                         milestone within the system life-cycle. This documentation is intended to
                         provide relevant information for management oversight and in making
                         decisions as to whether the investment of resources is cost beneficial.

                         A key piece of information—the economic analysis—was never completed
                         for the DCD/DCW project. In May 2000, the Director, DFAS, granted
                         approval to continue with development of DCD with a condition that a cost
                         benefit analysis be completed by June 2000. DFAS completed a draft cost
                         benefit analysis for DCD in October 2000. This document was not finalized
                         and in November 2000, DCD/DCW were combined into one program. Since
                         that time, DCD/DCW has continued without a valid, well-supported


                         23
                           Clinger-Cohen Act of 1996, P.L. 104-106, Div. E, 110 Stat. 679, February 10, 1996 (originally
                         known as the Information Technology Management Reform Act of 1996) and OMB Circular
                         A-130, Management of Federal Information Resources (Nov. 30, 2000).
                         24
                           DOD Regulation 5000.2-R, Mandatory Procedures for Major Defense Acquisition
                         Programs and Major Automated Information System Acquisition Programs (Apr. 5,
                         2002).
                         25
                           DOD Regulation 5000.2-R, Mandatory Procedures for Major Defense Acquisition
                         Programs and Major Automated Information System Acquisition Programs (Apr. 5, 2002)
                         and DFAS 8000.1-R, Part C, DFAS Information Technology Life Cycle Management Policy
                         (May 3, 2002).




                         Page 18                                    GAO-03-465 DOD Business Systems Modernization
economic justification to support continued investment in DCD/DCW.
DCD project management officials stated that the economic analysis has
not been finalized because they were unable to agree on how to compute
the return on investment and demonstrate that benefits exceeded costs.

In March 2001, DCD/DCW was designated a Major Automated Information
System, and as such, DOD’s Office of Program Analysis and Evaluation
(PA&E) is required to assess the economic analysis and provide any
recommendations to the DOD CIO. However, after approximately 2 years,
the economic analysis still has not been developed and PA&E officials
stated that it did not anticipate receiving the economic analysis until May
2003. At the same time, as highlighted in figure 3, the cost and schedule of
this project have continued to increase over the years.




Page 19                           GAO-03-465 DOD Business Systems Modernization
Figure 3: DCD/DCW Schedule Slippages and Cost Increases
                                                                                                                           FOC schedule



        Cost estimates (dollars in millions)                                                                                      2005
                                                                                                                    4th
        $300                                                                                                     quarter
                                                                                                                   FY05
                                                                                                                   DCD/
                                                                                                                   DCW
                                                                                                        $270                      2004
                                                                                                       Million
        $250                                                                                            DCD/
                                                                                                        DCW
                                                                      $229.1
                                                                      Million
                                                                       total    $115.9                                            2003
                                                                                Million
                                                                                 DCW
        $200


                                                                                                                                  2002


        $150

                                                                                                                                  2001

                                                                                $113.2
        $100                                                                    Million
                                                                                 DCD
                                                                                              1st                                         DCW
                                                                                          quarter                                 2000    initiated
                                                                                            FY01                                          7-00
                                                                                            DCD

         $50
                                                                                                                                  1999


                                                                                                                                  1998    DCD
           $0                                                                                                                             initiated
                    Project initiated         Milestone 0      Milestone I      Milestone II             Current                          10-98
                      DCD 10-98                DCD 5-99         DCD 5-00         DCD 5-00               DCD/DCW
                       DCW 7-00               DCW 8-00         DCW (none)       DCW (none)                12-02

                   Cost estimate

                   Full operational capability (FOC) schedule
Source: GAO based on information provided by DFAS.




                                                            Page 20                                 GAO-03-465 DOD Business Systems Modernization
                              Additionally, the planned functionality of DCD has been drastically reduced
                              since the original concept was set forth. Originally, DCD was to contain all
                              DOD financial information required by DFAS systems, making it the central
                              point for all shared data within DFAS. To accomplish this goal, DCD was to
                              crosswalk26 detailed transactions from nonstandard finance and feeder
                              systems into a standard format, pending the acquisition and
                              implementation of standard feeder systems. In February 2001, the scope of
                              the DCD project was revised after DFAS realized, through testing of Air
                              Force detailed transactions from feeder systems, that the planned
                              crosswalks were cumbersome and cost prohibitive. Currently, DFAS is
                              planning to crosswalk detailed transaction data only when information
                              from multiple systems must be aggregated to satisfy a cross-service need
                              such as the working capital fund activities. This will result in the originally
                              envisioned capability not being provided. Additionally, DCD/DCW will
                              continue to rely on the error-plagued data in the feeder systems and will not
                              produce financial records that are traceable to transaction-level data.
                              According to the DOD Inspector General, DCD was a high-risk effort
                              because there was no assurance that DCD and other financial management
                              systems would standardize DOD business processes; reduce the number of
                              finance, accounting, and feeder systems; reduce costs; and produce
                              accurate and auditable financial information.27

                              Until the economic analysis is finalized, DOD does not know if its
                              investment in DCD/DCW is justified and the decision to move to the next
                              milestone will continue to be delayed. Nevertheless, DOD continues to
                              spend funds to perform tasks in anticipation of milestone approval being
                              received. In fiscal year 2002, according to DFAS officials, approximately
                              $36 million was spent on DCD/DCW.



Economic Justification for    DOD had developed an economic analysis for each of the remaining three
the Other Three Projects Is   projects. However, these analyses had not been updated to reflect
                              schedule delays, cost increases, and changes in scope that have occurred—
Not Current
                              each of which has an impact on the projected benefits that were originally



                              26
                                Translate information so that systems that define data differently can understand each
                              other and communicate accurately.
                              27
                                 Department of Defense Office of the Inspector General, Development of the Defense
                              Finance and Accounting Service Corporate Database and Other Financial Management
                              Systems, Report No. D-2002-014 (Arlington, Va.: Nov. 7, 2001).




                              Page 21                                  GAO-03-465 DOD Business Systems Modernization
justified. Nevertheless, as shown in table 5, investment in each project
continues.



Table 5: Reported Investment in DPPS, DDRS, and DSDS

Dollars in millions
                         Date of the economic             Fiscal year 2002    Total investment as
System                   analysis                              investment      of September 2002
DPPS                     February 1998                                 $20                $126.5
DDRS                     October 1998                                  $10                 $57.5
DSDS                     September 2000                                 $7                   $39
Total                                                                  $37                  $223
Source: GAO based on information provided by DFAS


The investment of resources in a system project should be conditional
upon analytical justification that the proposed investment will produce
commensurate value. As called for in OMB guidance,28 analyses of
investment costs, benefits, and risks should be (1) updated throughout a
project’s life cycle to reflect material changes in project scope and
estimates and (2) used as a basis for ongoing investment selection and
control decisions. To do less presents the risk of continued investment in
projects on the basis of outdated and invalid economic justification.

In the case of DPPS, PA&E questioned the validity of the economic analysis
developed by DFAS. Since DPPS is classified as a major automated
information system, the economic analysis is to be reviewed by PA&E. In
its May 1998 assessment of the economic analysis, PA&E questioned areas
such as the validity of the estimated savings and the ability to implement
DPPS within the original estimated cost and schedule. According to DOD
officials, these issues were resolved, but they could not provide any
documentation to substantiate their position. The DOD CIO subsequently
granted permission to continue the project.

Over the years, as shown in figure 4, the DPPS effort has been marked by
significant increases in cost and schedule delays.




28
     OMB Circular A-130 (Nov. 30, 2000).




Page 22                                             GAO-03-465 DOD Business Systems Modernization
Figure 4: DPPS Schedule Slippages and Cost Increases
                                                                                                             FOC schedule

                                                                                                     12-05
                                                                                                                    2005



                                                                                                                    2004



                                                                                                                    2003


         Cost estimates (dollars in millions)                                                                       2002
         $600                                                                     4-02

                                                                                                 $552 M             2001

         $500
                                                                                                                    2000

         $400
                                                                                                                    1999


         $300
                                                                                                                    1998
                                                                            $278 M

         $200                                                                                                       1997



         $100                                                                                                       1996


                                                                                                                    1995    Project
            $0                                                                                                              initiated
                          Project initiated          Milestone 0       Milestone I/II           12-02                       4-95
                               4-95                     10-95              6-98

                    Cost estimate

                    Full operational capability (FOC) schedule
Source: GAO based on information provided by DFAS.



                                                      The original full operational capability date of April 2002 slipped to
                                                      December 2005—a delay of almost 4 years—with the estimated cost almost
                                                      doubling to $552 million. In December 2002, following our discussion with
                                                      DOD Comptroller officials of DPPS cost increases and schedule slippages,
                                                      the DOD Comptroller terminated DPPS. In making this decision, the DOD




                                                      Page 23                            GAO-03-465 DOD Business Systems Modernization
Comptroller noted that the project was being terminated due to poor
program performance and increasing costs.

With regard to DDRS, the economic analysis used to justify this initiative
was developed in October 1998—over 4 years ago. At that time, it was
estimated that DDRS would cost $111 million and be fully operational by
April 2000. However, based upon information provided by DFAS, and as
shown in figure 5, DDRS has experienced increased cost and schedule
delays. However, the economic analysis has not been updated to reflect the
known changes in the project’s costs and schedule.




Page 24                          GAO-03-465 DOD Business Systems Modernization
Figure 5: DDRS Cost Increases and Schedule Delays
                                                                                                         FOC schedule


                                                                                                                2004
                                                                                                  7-04



                                                                                                                2003

                                                                              4-03


                                                                                                                2002
        Cost estimates (dollars in millions)
         $200

                                                                                                                2001
                                                                                             $170 M

         $150
                                                                                                                2000

                                                              4-00

                                                         $111 M          $112 M
         $100
                                                                                                                1999
                                        6-99



                                                                                                                1998
          $50                      $52 M



                                                                                                                1997
                                                                                                                        Project
            $0                                                                                                          initiated
                         Project initiated           Milestone 0     Milestone I         Milestone II                   4-97
                              4-97                      11-98           4-02                12-02


                     Cost estimate

                     Full operational capability (FOC) schedule
Source: GAO based on information provided by DFAS.




                                                       Page 25                       GAO-03-465 DOD Business Systems Modernization
Moreover, the intended capability of DDRS as originally envisioned has
been reduced. For example, DDRS is no longer intended to provide the
capability to build an audit trail so that financial data can be tracked back
to its transaction-based support, as originally planned. The Federal
Financial Management Improvement Act of 199629 requires that agency
financial management systems comply with federal financial management
systems requirements, applicable federal accounting standards, and the
U.S. Government Standard General Ledger at the transaction level.
Systems meeting these requirements should be able to produce auditable
financial statements and otherwise have audit trail capability. However,
DDRS system users will have to rely on the audit trail capabilities of feeder
systems in order to trace individual transactions to their source
documents. As we have previously reported,30 the data from the feeder
systems, which are outside the control of DFAS and provide approximately
80 percent of the data that DOD needs for financial reporting purposes, are
not reliable. Additionally, until DCD is operational, DDRS will be receiving
data from the feeder systems in order to prepare the department’s financial
reports on the results of its operations. Therefore, DOD’s financial reports
produced by DDRS will (1) continue to be incomplete and inaccurate and
thus not useful for decision-making purposes and (2) remain unable to
withstand the scrutiny of a financial audit.

For DSDS, an economic analysis was prepared in September 2000.
However, it has not been updated to reflect material changes in the project.
For example, as shown in figure 6, the full operational capability (FOC)
date31 at the time the economic analysis was prepared was February 2003.
However, according to information provided by DFAS, the current FOC
date is December 2005—a schedule slippage of almost 3 years.32 Such
delays postpone the delivery of promised benefits. DFAS has stated that
the cost information is being updated to support a Milestone C decision,
which they anticipate will occur in early fiscal year 2004.



29
     P.L. 104-208, Div. A, 110 Stat. 3009-389, Sept. 30, 1996.
30
  U.S. General Accounting Office, DOD Financial Management: Important Steps
Underway But Reform Will Require a Long-term Commitment, GAO-02-784T (Washington,
D. C.: June 4, 2002).
31
  The full operational capability date represents the date that a system will be operating at
all intended locations.
32
     In March 1998, DFAS estimated the FOC date would be February 2002.




Page 26                                        GAO-03-465 DOD Business Systems Modernization
Figure 6: DSDS Schedule Delays
                                                                                                                            FOC schedule

                                                                                                                    12-05
                                                                                                                                   2005




                                                                                                                                   2004

       Cost estimates (dollars in millions)
       $250
                                                                                                                                   2003


                                                                                 2-03            2-03

       $200                                                                                                                        2002

                                                       2-02

                                                                                                                                   2001

       $150
                                                                            $151 M          $151 M

                                                                                                                                   2000


       $100
                                                                                                                                   1999




        $50                                                                                                                        1998



                                                                                                                                   1997
                                                                                                                                           Project
          $0                                                                                                                               initiated
                      Project initiated         Milestone 0             Milestone I     Milestone II            12-02                      2-97
                           2-97                     3-98                   12-00            3-01

                  Cost estimate

                  Full operational capability (FOC) schedule
Source: GAO based on information provided by DFAS.



                                                              Additionally, DSDS delivery of promised benefits depends upon the
                                                              DCD/DCW being implemented on time. However, as previously discussed,
                                                              DCD/DCW implementation has been fraught with difficulties, which has a
                                                              corresponding adverse effect on DSDS schedule delays. For example,



                                                              Page 27                                  GAO-03-465 DOD Business Systems Modernization
                           DCD/DCW project management officials are in the process of addressing
                           102 requests for requirement changes. According to the DCD/DCW
                           program manager, the date for resolving these changes and approving the
                           Operational Requirements Document is November 2003. Until this process
                           is completed, affected systems integration testing for other DCD/DCW
                           dependent systems, such as DSDS, cannot be finalized. Further, according
                           to DFAS officials, the continued operation of existing legacy systems may
                           result in an increase to the DSDS life-cycle cost estimate by approximately
                           $14 million for each 6-month delay. This would quickly erode the savings of
                           $171 million that DFAS estimated in September 2000, and reconfirmed in
                           January 2003.

                           Without an updated economic analysis to justify continued investment in
                           DDRS and DSDS, DOD does not have reasonable assurance that continued
                           investment will result in commensurate improvement in the financial
                           management operations of the department.



DOD Oversight of DFAS IT   DOD’s oversight over the four DFAS projects we reviewed has been
Projects Has Not Been      ineffective. Investment management responsibility for the four projects
                           rests with DFAS, the DOD Comptroller, and the DOD CIO. In discharging
Effective
                           this responsibility, each has allowed project investments to continue year
                           after year, even though the projects have been marked by cost increases,
                           schedule slippages, and capability changes. As a result, DOD has invested
                           approximately $316 million in the four projects without adequately
                           knowing if these efforts will resolve some of DOD’s financial management
                           difficulties—the rationale upon which each initiative was undertaken. In
                           fact, as previously noted, after an investment of over $126 million and 7
                           years of effort, the DOD Comptroller terminated DPPS in December 2002.

                           GAO’s Information Technology Investment Management (ITIM) maturity
                           framework33 defines critical processes pertaining to IT investment
                           management and oversight. Among other things these processes provide
                           for establishing investment decision-making bodies responsible for
                           selecting and controlling IT investments by (1) understanding, for example,
                           each project’s expected return on investment and associated costs,
                           schedule, and performance commitments, (2) regularly determining each


                           33
                             U.S. General Accounting Office, Information Technology Investment Management: A
                           Framework for Assessing and Improving Process Maturity (Exposure Draft), GAO/AIMD-
                           10.1.23 (Washington, D.C.: May 2000).




                           Page 28                              GAO-03-465 DOD Business Systems Modernization
project’s progress toward these expectations and commitments, and (3)
taking corrective actions to address deviations. Additionally, the Clinger-
Cohen Act and OMB guidance similarly emphasize the need to have
investment management processes and information to help ensure that IT
projects are being implemented at acceptable costs and within reasonable
and expected time frames and that they are contributing to tangible,
observable improvements in mission performance (i.e., that projects are
meeting the cost, schedule, and performance commitments upon which
their approval was justified).

Organizationally, within DOD, the Comptroller has overall management
and oversight responsibility for DFAS’s activities—including system
investments. However, DOD Comptroller officials told us that they were
unaware of the cost increases and schedule slippages on the projects until
we brought them to their attention. Further, these officials said that they
do not review DFAS’s system investments to ensure that they are meeting
cost, schedule, and performance commitments, stating that DFAS is
responsible for ensuring that projects stay on target in terms of cost,
schedule, and performance. Additionally, they told us that their review is
limited to a review of budgetary information and budget exhibits, and that
they compare the current year budget request to the previous year’s request
to determine if any significant funding increases are being requested for the
coming fiscal year. If the budget request is generally consistent from year
to year, they said that they do not raise questions about the project.
According to these officials, the review of DFAS’s fiscal year 2003 budget
did not result in the identification of issues that warranted further review.

While the DOD Comptroller is the responsible authority for DFAS
activities, DFAS is also responsible for ensuring that its proposed
investments will result in systems that are implemented at acceptable costs
and within reasonable and expected time frames. To fulfill this
responsibility, DFAS established the CIO/BIE Council to oversee system
investments. As outlined in the CIO/BIE Council charter, members of the
council are responsible for, among other things, advising the Leadership
Council—DFAS’s senior decision-making body—on IT investment
decisions. The CIO/BIE Council membership includes representatives of
DFAS’s business lines, such as accounting services and commercial pay, as
well as IT management.

In order to assure that the roles, responsibilities, and authorities of the IT
investment board are well defined and that board processes are clear, the
ITIM Framework states that an IT investment process guide should be



Page 29                             GAO-03-465 DOD Business Systems Modernization
created to direct IT investment board operations. While DFAS has
endeavored to give the CIO/BIE a role in the acquisition management and
oversight process, it has not provided clear, consistent guidance to
describe that role and the associated operating procedure. Though the
council charter does mention the CIO/BIE Council’s responsibilities, it does
not adequately describe them, address the council’s authority, or describe
how the council is to fulfill its responsibilities. The DFAS 8000 series also
addresses CIO/BIE responsibilities (DFAS 8000.1-R, Part C). However, the
8000 series does not describe how the CIO/BIE is expected to execute its
responsibilities, including providing corporate oversight and reviewing
capital budget proposals. The lack of clear definition of responsibilities
and authority limits the council’s ability to effectively perform oversight-
related activities. For the four IT investment projects we reviewed, we
found no evidence that the CIO/BIE effectively monitored the cost,
schedule, or performance goals of the four projects.

As previously noted, the DOD CIO is responsible for overseeing major
automated information systems. As such, this office is responsible for
ensuring that the investments being made in DCD/DCW and DPPS are
justified. However, the DOD CIO did not effectively exercise this authority.
In regard to DPPS, the DOD CIO was designated the milestone decision
authority in November 1996. While DOD CIO officials told us that they
were aware of the problems with DPPS, they were unable to provide any
documentation that indicated they had raised concerns with the DPPS
effort.

DCD/DCW was not brought under the purview of the DOD CIO until March
2001— approximately 2½ years after the project began. DOD CIO officials
expressed concerns about the viability of DCD/DCW and questioned
DFAS’s decision to move forward absent an economic analysis. However,
they were unable to provide us with documentation that indicated they had
carried out their oversight responsibilities and independently determined
whether DCD/DCW was a viable investment.




Page 30                            GAO-03-465 DOD Business Systems Modernization
According to DOD CIO officials, despite being the milestone decision
authority for major projects, they have little practical authority in
influencing component agency IT projects. As such, they said they try to
work with the program managers to ensure that all of the required
documentation for passing the next milestone is prepared, but the
department’s culture, which rests organizational authority and funding
control with the components, precludes them from exercising effective IT
investment oversight. The comments of the DOD CIO officials support the
fact that the current stovepiped, parochial management of DOD’s IT
investments has led to the previously discussed proliferation of business
systems. As we previously reported,34 DOD’s organizational structure and
embedded culture have made it difficult to implement departmentwide
oversight or visibility over information resources.




34
  U.S. General Accounting Office, Defense IRM: Poor Implementation of Management
Controls Has Put the Migration Strategy at Risk, GAO/AIMD-98-5 (Washington, D.C.: Oct.
20, 1997).




Page 31                                GAO-03-465 DOD Business Systems Modernization
              Similarly, we recently reported35 that DOD does not yet have the
              departmental investment governance structure and process controls
              needed to adequately align ongoing investments with DOD’s architectural
              goals and direction. Instead, DOD continues to allow its component
              organizations to make their own investment decisions, following different
              approaches and criteria. We reported that this stovepiped decision-making
              process has contributed to the department’s current complex, error prone
              environment of over 1,700 systems. In particular, DOD has not yet
              established and applied common investment criteria to its ongoing IT
              system projects using a hierarchy of investment review and funding
              decision-making bodies, each composed of representatives from across the
              department. DOD also has not yet conducted a comprehensive review of its
              ongoing IT investments to ensure that they are consistent with its
              architecture development efforts. Until it does these things, DOD will
              likely continue to lack effective control over the billions of dollars it is
              currently spending on IT projects. To address this problem we
              recommended that DOD establish a series of investment review boards,
              each responsible and accountable for selecting and controlling investments
              that meet defined threshold criteria, and each composed of the appropriate
              level of executive representatives, depending on the threshold criteria,
              from across the department. We also reiterated our open
              recommendations governing limitations in business system investments
              pending development of the architecture.36



Conclusions   DOD is investing billions of dollars annually in hundreds of systems that
              perform the same function spread across numerous DOD components. As
              we have previously reported, this proliferation of systems has resulted in
              part because DOD’s embedded culture and parochial operations have
              permitted each of the military services and DOD agencies to manage and
              oversee their IT investments apart from one another. It has also occurred
              because DOD has not effectively managed its investments in IT business
              systems, as our past work and the DOD Inspector General work have
              demonstrated. As a result, DOD runs a high risk that hundreds of millions

              35
                U.S. General Accounting Office, DOD Business Systems Modernization: Improvements
              to Enterprise Architecture Development and Implementation Efforts Needed, GAO-03-458
              (Washington, D.C.: Feb. 28, 2003).
              36
                U.S. General Accounting Office, Information Technology: Architecture Needed to Guide
              Modernization of DOD’s Financial Operations, GAO-01-525 (Washington, D.C.: May 17,
              2001).




              Page 32                                GAO-03-465 DOD Business Systems Modernization
                      of dollars will continue to be invested annually in modernization efforts
                      that will not result in improvements in the department’s operations.

                      In each of the four system projects we discuss in the report, DOD has
                      invested millions of dollars without economically justifying its investments,
                      in large part because those entities responsible for managing and
                      overseeing these investments have not required such justification despite
                      schedule slippages, cost overruns, and reductions in planned capability.
                      Urgent need for effective investment control is exemplified by DPPS—$126
                      million for a terminated project. More vigorous oversight of DPPS could
                      have precluded the substantial investment in this failed effort. Until it has
                      effective investment management and oversight, DOD will not have
                      reasonable assurance that its continued investment in the remaining three
                      projects discussed in this report, as well as its other system projects, are
                      justified.



Recommendations for   We recommend that the Secretary of Defense direct the Under Secretary of
                      Defense (Comptroller) to limit funding in the DFAS Corporate Database/
Executive Action      Corporate Warehouse, the Defense Standard Disbursing System, and the
                      Defense Departmental Reporting System until the DOD Comptroller, in
                      collaboration with the Assistant Secretary of Defense (Command, Control,
                      Communications & Intelligence), and the Director, Program Analysis and
                      Evaluation, demonstrates on the basis of credible analysis and data that
                      continued investment in these three projects will produce benefits that
                      exceed costs.

                      We further recommend that the Secretary of Defense, in light of the
                      department’s ongoing efforts to modernize its business systems, direct the
                      Under Secretary of Defense (Comptroller) to evaluate all remaining DFAS
                      IT projects and ensure that each project is being implemented at
                      acceptable costs, within reasonable time frames, and is contributing to
                      tangible, observable improvements in mission performance.




                      Page 33                            GAO-03-465 DOD Business Systems Modernization
Agency Comments and   DOD provided written comments on a draft of this report. DOD concurred
                      with our recommendations and identified actions it planned to take to
Our Evaluation        ensure that future investments in DFAS’s systems are justified. For
                      example, the Under Secretary of Defense (Comptroller) noted that the
                      review of DCD/DCW, DDRS, and DSDS would be completed by June 15,
                      2003. Additionally, the Under Secretary of Defense (Comptroller) stated
                      that all systems would be reviewed as part of the department’s effort to
                      establish a financial management enterprise architecture governance
                      structure. As discussed in our February 2003 report,37 the governance
                      structure is intended to provide DOD the means to gain control over its IT
                      investments. However, as noted in our report, we have not verified or
                      evaluated the extent to which the planned governance structure will
                      address our recommendation. DOD comments are reprinted in appendix
                      II.


                      As agreed with your office, unless you announce the contents of this report
                      earlier, we will not distribute this report until 30 days from its date. At that
                      time, we will send copies to the Chairman and Ranking Minority Member,
                      Senate Committee on Armed Services; Chairman and Ranking Minority
                      Member, Senate Appropriations Subcommittee on Defense; Chairman and
                      Ranking Minority Member, House Armed Services Committee; Chairman
                      and Ranking Minority Member, House Appropriations Subcommittee on
                      Defense; Chairman and Ranking Minority Member, Senate Committee on
                      Governmental Affairs; Chairman and Ranking Minority Member, House
                      Committee on Government Reform; the Director, Office of Management
                      and Budget; the Under Secretary of Defense (Comptroller); the Assistant
                      Secretary of Defense (Command, Control, Communications &
                      Intelligence); and the Director, Defense Finance and Accounting Service.
                      Copies of this report will be made available to others upon request. The
                      report will also be available on GAO's Web site at http://www.gao.gov.

                      If you or your staff have any questions on matters discussed in this report,
                      please contact Gregory D. Kutz at (202) 512-9505 or kutzg@gao.gov or




                      37
                        U.S. General Accounting Office, DOD Business Systems Modernization: Improvements
                      to Enterprise Architecture Development and Implementation Efforts Needed, GAO-03-458
                      (Washington, D.C.: Feb. 28, 2003).




                      Page 34                               GAO-03-465 DOD Business Systems Modernization
Randolph C. Hite at (202) 512-3439 or hiter@gao.gov. Major contributors to
this report are acknowledged in appendix III.




Gregory D. Kutz
Director, Financial Management
and Assurance




Randolph C. Hite
Director, Information Technology Architecture and Systems Issues




Page 35                          GAO-03-465 DOD Business Systems Modernization
Appendix I

Scope and Methodology                                                                                  AA
                                                                                                        ppp
                                                                                                          ep
                                                                                                           ned
                                                                                                             n
                                                                                                             x
                                                                                                             id
                                                                                                              e
                                                                                                              x
                                                                                                              Iis




             To obtain an overview of DOD’s current business systems environment we
             met with representatives of the then Financial Management Modernization
             Program Office1 to obtain information on the number of systems that are
             part of the current systems environment. We also reviewed DOD’s $26
             billion fiscal year 2003 IT budget request to determine what portion of the
             budget relates to DOD business systems. Additionally, we reviewed the IT
             budget to determine the reported operations, maintenance, development,
             and infrastructure costs for DOD’s business systems.

             To determine if DOD was effectively managing and overseeing its IT
             investments, we focused on the four system projects previously noted. To
             assist us in our evaluation, we used our Information Technology
             Investment Management (ITIM) framework. The ITIM identifies critical
             processes for successful IT investment and organizes these processes into
             a framework of increasingly mature stages. We focused on the Stage 2
             critical processes of IT project oversight and IT investment board practices
             based on DFAS’s self assessment that it was at Stage 2. Figure 7 shows
             ITIM’s five stages of maturity.




             1
                 This office has been renamed the Business Modernization Systems Integration Office.




             Page 36                                    GAO-03-465 DOD Business Systems Modernization
Appendix I
Scope and Methodology




Figure 7: ITIM Stages of Maturity

                       Maturity stages                      Critical processes

                                                  Investment process benchmarking
                              Stage 5             IT-driven strategic business change
                         Leveraging IT for
                        strategic outcomes

                                                  Post-implementation reviews and feedback
                         Stage 4                  Portfolio performance evaluation and improvement
                 Improving the investment         Systems and technology sucession management
                         process

                                                  Authority alignment of IT investment boards
                      Stage 3                     Portfolio selection criteria definition
               Developing a complete              Investment analysis
                                                  Portfolio development
                investment portfolio              Portfolio performance oversight

                                                  IT investment board operation
                    Stage 2                       IT project oversight
           Building the investment                IT project and system identification
                                                  Business needs identification for IT projects
                  foundation                      Proposal selection

                                                  IT spending without disciplined
              Stage 1                             investment processes
        Creating investment
            awareness


Source: GAO.



In addition, we also evaluated DOD’s and DFAS’s guidance on systems
acquisition, as it relates to life-cycle management and milestones for
proceeding to the next phase of the system acquisition process.2 To verify
application of the critical processes and practices, we selected projects
that (1) were in different life-cycle phases of systems development
(2) required oversight by a DOD authority outside of the DOD Comptroller,
such as the Office of the Assistant Secretary of Defense (Command,
Control, Communications & Intelligence)—DOD’s CIO, and (3) supported
different DFAS business areas such as disbursements and departmental


2
 DOD Regulation 5000.2-R, Mandatory Procedures for Major Defense Acquisition
Programs and Major Automated Information System Acquisition Programs (Apr. 5,
2002), DOD Instruction 5000.2, Operation of the Defense Acquisition System (Apr. 5, 2002),
DFAS 8000.1-R, Part C, DFAS Life Cycle Management Policy (May 3, 2002), and DFAS 8000-
1-R, Information Systems Life Cycle Management, policies for 1996, 1997, and 1998.




Page 37                                      GAO-03-465 DOD Business Systems Modernization
Appendix I
Scope and Methodology




reporting. For these four projects we reviewed documentation, such as
mission needs statements, acquisition program baseline updates, and
project management plans. According to DOD, it provided estimates for
DCD/DCW and DDRS in constant dollars and DPPS and DSDS in escalated
dollars. We also reviewed and analyzed charters and meeting minutes of
the DFAS investment oversight boards and working groups. To supplement
our document reviews, we interviewed senior DFAS officials in the CIO and
Systems Integration Offices, as well as the program managers for the four
projects. We also met with officials in the offices of the DOD Comptroller
and DOD CIO to obtain an understanding of their specific duties and
responsibilities in approving, reviewing, and overseeing investments in the
four DFAS systems modernization projects.

We conducted our work at DFAS Headquarters; the Office of the Under
Secretary of Defense (Comptroller); the Office of the Secretary of Defense
Program Analysis and Evaluation; and the Office of the Assistant Secretary
of Defense (Command, Control, Communications & Intelligence) from
November 2001 through January 2003, in accordance with U.S. generally
accepted government auditing standards. We did not verify the accuracy
and completeness of the cost information provided by DFAS for the four
projects we reviewed. We requested comments on a draft of this report
from the Secretary of Defense or his designee. We received written
comments on a draft of this report from the Under Secretary of Defense
(Comptroller), which are reprinted in appendix II.




Page 38                           GAO-03-465 DOD Business Systems Modernization
Appendix II

Comments From the Under Secretary of
Defense                                                               Appendx
                                                                            Ii




              Page 39     GAO-03-465 DOD Business Systems Modernization
Appendix II
Comments From the Under Secretary of
Defense




Page 40                                GAO-03-465 DOD Business Systems Modernization
Appendix III

GAO Contacts and Staff Acknowledgments                                                           Appendx
                                                                                                       iI




GAO Contacts      Darby Smith, (202) 512-7803
                  Jenniffer Wilson, (202) 512-9192



Acknowledgments   In addition to the individuals named above, key contributors to this report
                  included Beatrice Alff, Joseph Cruz, Francine DelVecchio, Lester Diamond,
                  Jason Kelly, J. Christopher Martin, Stacey Smith, and Robert Wagner.




(192043)          Page 41                            GAO-03-465 DOD Business Systems Modernization
GAO’s Mission            The General Accounting Office, the audit, evaluation and investigative arm of
                         Congress, exists to support Congress in meeting its constitutional responsibilities
                         and to help improve the performance and accountability of the federal government
                         for the American people. GAO examines the use of public funds; evaluates federal
                         programs and policies; and provides analyses, recommendations, and other
                         assistance to help Congress make informed oversight, policy, and funding
                         decisions. GAO’s commitment to good government is reflected in its core values of
                         accountability, integrity, and reliability.


Obtaining Copies of      The fastest and easiest way to obtain copies of GAO documents at no cost is
                         through the Internet. GAO’s Web site (www.gao.gov) contains abstracts and full-
GAO Reports and          text files of current reports and testimony and an expanding archive of older
                         products. The Web site features a search engine to help you locate documents
Testimony                using key words and phrases. You can print these documents in their entirety,
                         including charts and other graphics.
                         Each day, GAO issues a list of newly released reports, testimony, and
                         correspondence. GAO posts this list, known as “Today’s Reports,” on its Web site
                         daily. The list contains links to the full-text document files. To have GAO e-mail this
                         list to you every afternoon, go to www.gao.gov and select “Subscribe to GAO
                         Mailing Lists” under “Order GAO Products” heading.


Order by Mail or Phone   The first copy of each printed report is free. Additional copies are $2 each. A check
                         or money order should be made out to the Superintendent of Documents. GAO
                         also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a single
                         address are discounted 25 percent. Orders should be sent to:
                         U.S. General Accounting Office
                         441 G Street NW, Room LM
                         Washington, D.C. 20548
                         To order by Phone:     Voice: (202) 512-6000
                                                TDD: (202) 512-2537
                                                Fax: (202) 512-6061


To Report Fraud,         Contact:
                         Web site: www.gao.gov/fraudnet/fraudnet.htm
Waste, and Abuse in      E-mail: fraudnet@gao.gov
Federal Programs         Automated answering system: (800) 424-5454 or (202) 512-7470



Public Affairs           Jeff Nelligan, Managing Director, NelliganJ@gao.gov (202) 512-4800
                         U.S. General Accounting Office, 441 G Street NW, Room 7149
                         Washington, D.C. 20548
United States                  Presorted Standard
General Accounting Office      Postage & Fees Paid
Washington, D.C. 20548-0001           GAO
                                 Permit No. GI00
Official Business
Penalty for Private Use $300
Address Service Requested