oversight

Defense Computers: LSSC Needs to Confront Significant Year 2000 Issues

Published by the Government Accountability Office on 1997-09-26.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                  United States General Accounting Office

GAO               Report to the Director, Army Logistics
                  Systems Support Center



September 1997
                  DEFENSE
                  COMPUTERS
                  LSSC Needs to
                  Confront Significant
                  Year 2000 Issues




GAO/AIMD-97-149
      United States
GAO   General Accounting Office
      Washington, D.C. 20548

      Accounting and Information
      Management Division

      B-277732

      September 26, 1997

      Mr. Michael A. Whitelaw
      Director, Logistics Systems Support Center
      1222 Spruce Street
      St. Louis, Missouri 63103

      Dear Mr. Whitelaw:

      On August 22, 1997, we briefed you on the results of our review to date of
      the Logistics Systems Support Center’s (LSSC) program for solving the
      Commodity Command Standard System (CCSS) Year 2000 computer system
      problem.1 The problem results from the inability of computer programs at
      the year 2000 to interpret the correct century from a recorded or
      calculated date having only two digits to indicate the year. Unless
      corrected, this problem could cause the CCSS system to malfunction or
      produce incorrect information when the year 2000 is encountered during
      automated data processing. Since CCSS supports the Army’s wholesale
      logistics supply management business area, which procures supplies and
      equipment totaling over $23 billion annually for Army forces around the
      world, LSSC’s inability to ensure that CCSS is Year 2000 compliant could
      result in a loss of operational support that would be widespread, costly,
      and potentially debilitating to important Army and other DOD agency
      missions.

      Our briefing was based on work we performed as part of our review of the
      Department of Defense’s (DOD) Year 2000 computer systems effort for the
      Chairman, Senate Governmental Affairs Committee; the Chairman and
      Ranking Minority Member of the Subcommittee on Government
      Management, Information and Technology, House Committee on
      Government Reform and Oversight; and the Honorable Thomas M. Davis,
      III, House of Representatives. During our review, we concentrated on
      determining (1) the status of LSSC’s efforts to correct CCSS Year 2000 system
      problems and (2) the appropriateness of LSSC’s strategy and actions for
      ensuring that CCSS Year 2000 issues will be successfully addressed. This
      letter summarizes the concerns we raised and provides recommendations
      for addressing these issues.




      1
       Although CCSS is considered to be one system, LSSC reports that it actually consists of 561
      subsystems and supporting applications which contain about 10.2 million lines of code—about
      5.5 million of which are being reported by LSSC as potentially requiring renovation.



      Page 1                                                  GAO/AIMD-97-149 Army LSSC Year 2000
                   B-277732




                   The Year 2000 problem is one of the most comprehensive and complex
Results in Brief   information management projects ever faced by LSSC. If not successfully
                   completed, the procurement of weapon systems and their spare parts,
                   accounting for the sales of Army equipment and services to allies, and the
                   financial management of $9 billion of inventory could be disrupted. As a
                   result, it could be extremely difficult to efficiently and effectively equip
                   and sustain the Army’s forces around the world.

                   To its credit, LSSC has completed several actions to address the CCSS Year
                   2000 problem. For example, LSSC has inventoried software lines of code,
                   established a Year 2000 task force, and acquired automated software
                   assessment tools. In addition, a Year 2000 project manager and
                   management staff have been designated, a project charter and schedule
                   were developed, and supplementary contractor support was acquired to
                   assist with assessment tasks. Further, regularly scheduled quarterly
                   meetings are held by Army Materiel Command (AMC) headquarters to
                   report LSSC Year 2000 status. These steps are compatible with DOD’s
                   suggested approach and consistent with those found in our five-phased
                   approach for planning, managing, and evaluating Year 2000 projects.

                   Although LSSC commenced its Year 2000 project over a year ago, there are
                   several critical issues facing LSSC that, if not completely addressed, may
                   result in the failure of CCSS to successfully operate at the year 2000.
                   Specifically, LSSC has yet to completely address (1) competing workload
                   priorities and staffing issues, (2) the appropriate mix and scheduling of
                   needed testing data and expertise as well as the development of test plans,
                   (3) the scope and substance of written interface agreements with system
                   interface partners to ensure that CCSS subsystems will be capable of
                   exchanging data at the year 2000, and (4) contingency plan development to
                   help assure that Army’s missions will be accomplished if CCSS is not fully
                   available to users by the year 2000. LSSC’s risk of failure is increased
                   because the agency has not attained the level of software development and
                   maintenance maturity that would provide the foundation needed for
                   successful management of large-scale projects such as the Year 2000
                   initiative. Because CCSS is used to support military readiness, these critical
                   elements must be resolved and aggressively pursued to enable LSSC to
                   successfully achieve a Year 2000 compliant environment prior to the year
                   2000.




                   Page 2                                     GAO/AIMD-97-149 Army LSSC Year 2000
              B-277732




              In conducting our review, we compared LSSC’s progress in planning and
Scope and     managing its Year 2000 project to our Year 2000 Assessment Guide.2 We
Methodology   also reviewed DOD’s Year 2000 Management Plan,3 Department of the Army
              and Army Materiel Command (AMC) Year 2000 guidance,4 and private
              industry Year 2000 guidance. We focused our review on Year 2000 work
              performed by (1) LSSC—the designer, developer, and maintainer of CCSS,
              and (2) AMC—the Army major command responsible for promulgating Year
              2000 policy and guidance and providing assistance to its major
              subordinate commands and central design activities.

              To determine the status of LSSC’s Year 2000 project and the
              appropriateness of its strategy and actions for ensuring successful
              completion, we interviewed LSSC’s Year 2000 Project Manager, Project
              Officer, and Focal Point who are responsible for project management,
              direction, and reporting. We also interviewed the AMC Year 2000 project
              team and the AMC Year 2000 Logistics Systems Chair. We obtained and
              analyzed Year 2000 guidance as well as documentation from the CCSS
              Configuration Control Board, AMC quarterly progress reviews, and AMC
              Year 2000 Logistics Task Force meetings to determine Year 2000 plans,
              strategy, and status for each of the five phases. We obtained and discussed
              software change schedules, workload statistics, and testing procedures
              and testing resource information with LSSC’s Quality Assurance Division
              Chief. We discussed software change procedures with the Technical Data
              Systems Division Chief and impact and workload issues with the Asset
              Management Systems Chief. To compare LSSC’s workload with its available
              staff resources, we obtained and discussed staffing information with the
              agency’s Resources Management Director, Budget and Manpower Division
              Chief, and the Business Information Systems Director. To determine cost
              estimates for the project, we interviewed the Year 2000 Project Manager
              and obtained and analyzed LSSC documents pertaining to cost. We also
              discussed LSSC’s software maturity capability and efforts to improve its
              maturity level with the Year 2000 Project Manager and the computer
              specialist tasked with improving LSSC’s software maturity capability.

              We conducted our work primarily at the Logistics Systems Support Center
              in St. Louis, Missouri, and at the U.S. Army Materiel Command in
              Alexandria, Virginia. Our audit work was performed from December 1996

              2
                Year 2000 Computing Crisis: An Assessment Guide (Exposure Draft) (GAO/AIMD-10.1.14,
              February 1997).
              3
               Department of Defense Year 2000 Management Plan (Version 1.0, April 1997).
              4
                Army Project Change of Century Action Plan (Revision I, October 1996) and the U.S. Army Materiel
              Command Year 2000 Action Plan (May 1997).



              Page 3                                                  GAO/AIMD-97-149 Army LSSC Year 2000
             B-277732




             through August 1997 in accordance with generally accepted government
             auditing standards.

             The Department of Defense provided written comments on a draft of this
             report. These comments are discussed in the “Agency Comments and Our
             Evaluation” section and are reprinted in appendix I.


             LSSC is one of several central design activities (CDAs) for the Army Materiel
Background   Command (AMC).5 LSSC’s major responsibility is to design, develop, deploy,
             integrate, and maintain the Commodity Command Standard System (CCSS),
             a standard automated wholesale logistics system supporting AMC and other
             Army and DOD organizations. CCSS performs stock control, supply
             management, cataloging, provisioning, procurement, maintenance,
             security assistance, and financial management over an inventory of supply
             items for these organizations. It is the business automation core of AMC’s
             commodity commands and is linked to other Army logistics systems, such
             as the Continuing Balance System-Expanded (CBSX).6 CCSS’ financial
             module also provides general accounting, inventory accounting, billing
             support, and general ledger and financial reports for both reimbursable
             and non-reimbursable issues. As one of the world’s largest integrated
             business systems, CCSS comprises 561 separate subsystems that contain
             10.2 million lines of program code in about 5000 programs. These
             subsystems work collectively to process an annual procurement budget
             for supplies and equipment of over $23 billion.

             The Year 2000 problem is rooted in the way dates are recorded and
             computed in automated information systems. For the past several
             decades, systems have typically used two digits to represent the year, such
             as “97” representing 1997, in order to conserve electronic data storage and
             reduce operating costs. However, with this two-digit format, the year 2000
             is indistinguishable from 1900, as is 2001 indistinguishable from 1901. As a
             result of this ambiguity, system or application programs that use dates to
             perform calculations, comparisons, or sorting may generate incorrect
             results when working with years after 1999.




             5
             AMC is headquartered in Alexandria, Virginia, and is responsible for developing, buying, and
             maintaining equipment and supplies for U.S. soldiers and allies worldwide.
             6
              The Continuing Balance System-Expanded (CBSX) is the Army’s central logistics system for reporting
             the types, quantities, and locations of equipment, generally major end items such as helicopters and
             tanks.



             Page 4                                                   GAO/AIMD-97-149 Army LSSC Year 2000
B-277732




LSSC  staff recognized the significance of this anomaly in 1991, and at an
estimated cost of $5.9 million, recommended making all CCSS date fields
Year 2000 compliant in accordance with Army Regulation 25-9.7 The
regulation specified that date fields were to be 8 positions in length
(YYYYMMDD) including two century positions to be populated with
normal values of “19” or “20.” LSSC has requested funding for the
recommended changes every year since 1991; however, funding was
denied because CCSS was a legacy system designated for replacement by
other systems under the DOD Corporate Information Management (CIM)
initiative. By 1994, however, it became apparent that emergency system
changes would be necessary to allow certain CCSS subsystems to continue
forecasting requirements beyond 1999. LSSC reports that, since 1994, it has
renovated at least 3.8 million lines of code to accommodate the year 2000.
However, initial funding for completing the CCSS Year 2000 effort, now
estimated at over $12 million, was not approved until January 1997.

The impact of the year 2000 on CCSS is substantial since CCSS is heavily date
dependent. Date fields are used in nearly all CCSS subsystems, files,
databases, and data used for status accounting, computations, forecasting,
financial accounting, and requisition processing. Consequently, faulty turn
of the century date processing would significantly impair the Army’s
ability to order, manage, sell, and account for commodities such as
ammunition, communications, and electronics. In turn, through its other
logistics systems connections, it could also impair the Army’s ability to
track and manage major end items such as aircraft, missiles, and tanks, as
well as the many thousands of repair parts that support them.

Because CCSS is the Army’s wholesale logistics system, a loss of CCSS
operational support to AMC and other DOD agencies poses a serious threat
to overall mission capability. For example, if dates are not processed
accurately in CCSS applications that support inventory management and
requisition processing, items ordered on or after January 1, 2000, could be
identified as 99-year old excess inventory and become candidates for
disposal. The cost of such faulty date processing would be great
considering the (1) cost of the inventory item, (2) administrative costs
involved in requisitioning, shipping, handling, and accounting for the item
in the various financial, inventory, and transportation subsystems, and
(3) costs associated with designating the item as excess inventory for
disposal and the subsequent physical disposal of the item. Such an
occurrence could severely impair overall military readiness since the
necessary items would not be available for the soldier in the field. More

7
 Army Data Management and Standards Program, AR 25-9, effective October 24, 1989.



Page 5                                                 GAO/AIMD-97-149 Army LSSC Year 2000
    B-277732




    importantly, soldiers and military civilians may not be able to properly
    maintain or replace weapon systems components, which could result in
    injury or death. Also, military equipment maintenance and overhaul
    facilities could be temporarily closed for lack of spare parts.

    In February 1997, we published the Year 2000 Computing Crisis: An
    Assessment Guide, which addresses common issues affecting most federal
    agencies and presents a structured approach and checklist to aid in the
    planning, managing, and implementing of Year 2000 projects. The guide
    describes five phases —supported by program and project management
    activities —with each phase representing a major Year 2000 program
    activity or segment. The guidance draws heavily on the work of the Best
    Practices Subcommittee of the Interagency Year 2000 Committee, and
    incorporates guidance and practices identified by leading organizations in
    the information technology industry. The five phases are consistent with
    those prescribed by DOD in its Year 2000 Management Plan. The phases and
    a description of each phase follows:

•   Awareness—Define the Year 2000 problem and gain executive-level
    support and sponsorship. Establish a Year 2000 program team and develop
    an overall strategy. Ensure that everyone in the organization is fully aware
    of the issue.
•   Assessment—Assess the Year 2000 impact on the enterprise. Identify
    core business areas and processes, inventory and analyze systems
    supporting the core business areas, and rank their conversion or
    replacement. Develop contingency plans to handle data exchange issues,
    lack of data, and bad data. Identify and secure the necessary resources.
•   Renovation—Convert, replace, or eliminate selected platforms,
    applications, databases, and utilities. Modify interfaces.
•   Validation—Test, verify, and validate converted or replaced platforms,
    applications, databases, and utilities. Test the performance, functionality,
    and integration of converted or replaced platforms, applications,
    databases, utilities, and interfaces in an operational environment.
•   Implementation—Implement converted or replaced platforms,
    applications, databases, utilities, and interfaces. Implement data exchange
    contingency plans, if necessary.

    In addition to following the five phases described above, the Year 2000
    program should also be planned and managed as a single large information
    system development effort. Agencies should promulgate and enforce good
    management practices on the program and project levels.




    Page 6                                    GAO/AIMD-97-149 Army LSSC Year 2000
                    B-277732




                    LSSC is the Army component responsible for applying the Year 2000
Current Status of   five-phased resolution process to CCSS. As such, in July 1996, LSSC initiated
LSSC’s Year 2000    a project to address CCSS Year 2000 processing issues. As of July 1997, LSSC
Project             has completed a number of activities associated with the awareness and
                    assessment phases of the process, including identifying its inventory,
                    establishing a Year 2000 project team, and assessing the date impact on
                    CCSS’ 10.2 million lines of code. LSSC has identified that as much as 54
                    percent or 5.5 million lines of code may be impacted by the year 2000 due
                    to the fact that entire applications may need to be corrected to
                    accommodate the date change. LSSC officials stated that they still need to
                    determine how specific code will be changed in affected applications. LSSC
                    also reported that an additional 3.8 million lines of code have already been
                    renovated but still need to undergo integrated and regression testing. LSSC
                    plans to implement the Year 2000-compliant CCSS by November 1998 at a
                    cost of over $12 million.

                    Prior to receiving funding in January 1997, the Year 2000 project remained
                    in the awareness phase. During the awareness phase, LSSC completed tasks
                    such as assembling technical and functional representatives into a Year
                    2000 task force, evaluating automated software assessment tools, and
                    identifying the number of software lines of code. Once the project was
                    officially funded and entered the assessment phase, LSSC officials
                    appointed the project manager and management staff. Also, the Year 2000
                    project team prepared a project charter and schedule, secured contractor
                    support to assist with assessment tasks, and began to determine the date
                    impact on CCSS program code. As project activity proceeded, project staff
                    routinely reported Year 2000 progress to the AMC Deputy Commanding
                    General, AMC Year 2000 Logistics Task Force, Communications-Electronics
                    Command (CECOM) Year 2000 Project Office, and the CCSS Configuration
                    Control Board.

                    To support project management, LSSC’s Year 2000 project manager drafted
                    a plan which initially did not conform to DOD’s recommended Year 2000
                    five-phased approach, although the plan did identify some tasks typically
                    associated with Year 2000 projects. For example, the plan included such
                    tasks as beginning risk assessment and contingency plan development,
                    providing assessment tool training, conducting an inventory of CCSS
                    applications, and obtaining contractor support for date impact assessment.
                    As a result of our concerns that the plan did not clearly specify or identify
                    key Year 2000 phases and associated tasks, LSSC’s Year 2000 project
                    manager later revised the plan in an attempt to better identify phases and
                    tasks in accordance with DOD’s five-phased approach.



                    Page 7                                     GAO/AIMD-97-149 Army LSSC Year 2000
B-277732




In addition to assessing the lines of code for CCSS, LSSC reported that it had
cataloged the applications, modules, functional areas served, and
languages used. LSSC had also determined that all the source code for CCSS
was available and matched production code. In addition, LSSC had acquired
automated assessment tools to help identify affected and obsolete code
and trained LSSC staff to use these tools for the assessment, renovation,
and validation phases.

Since the exchange of data with other systems through external interfaces
creates the potential to introduce or propagate errors from one system to
another, LSSC identified 57 other systems which interface with CCSS and is
in the process of confirming data exchange requirements with the external
system owners. LSSC also developed a standard memorandum of
agreement (MOA) to document coordination of data exchange
requirements.8 Since CCSS and its interfacing partners plan to use
procedural code9 and sliding window techniques10 to correct the Year 2000
problem, any date formats exchanged would be properly converted
through internal program coding changes rather than changes to date
formats.

As part of its assessment of the level of date impact on CCSS, LSSC assessed
the risk of not preparing CCSS for the year 2000. LSSC reported that CCSS, as
a whole, is not Year 2000 compliant and that a catastrophic failure of the
Army wholesale logistics mission would occur if CCSS is not made
compliant. LSSC further reported that no known commercial or government
replacements exist for CCSS functionality and that renovation of existing
CCSS code was essential to mitigate the risk of failure.


In May 1997, LSSC was still addressing the assessment phase activities of
identifying a renovation strategy and developing a validation strategy and
schedule for testing. According to DOD’s Year 2000 Management Plan and

8
 LSSC’s approach to data exchange and its proposal to the external system owners consists of keeping
date formats in the current two-digit year configuration in accordance with existing military data
formatting standards. CCSS and nearly all DOD logistics systems conform with military data
formatting standards which require year dates to be input, stored, and transmitted in a two-digit
configuration. While use of the four-digit year field is now the preferred standard for system interfaces,
DOD recognizes that a two-digit year field is an acceptable configuration when accompanied by signed
MOAs documenting the data exchange requirements of the interfacing partners.
9
 Procedural code is code which derives the correct century based on the two-digit year (e.g., any year
smaller than year 50 is a 2000 date, and any year 50 or larger is a 1900 date).
10
 Sliding windows are similar to procedural code in that they derive the correct century based on the
two-digit year, but the numeric constant used to determine the century changes each year. Using the
procedural code example above, in the current year, 50 or larger would be a 1900 date, while next year,
51 or larger would be a 1900 date.



Page 8                                                     GAO/AIMD-97-149 Army LSSC Year 2000
                         B-277732




                         AMC’s Year 2000 Action Plan, the validation strategy should identify the
                         general time frames for the validation of all information resources and
                         include consideration of hardware concerns such as availability of
                         processing cycles and storage as well as human resource issues. In
                         addition, efforts were ongoing to contract with a vendor to perform
                         automated code correction on some CCSS subsystems.


                         To its credit, LSSC recognized the problems inherent in the century date
LSSC Must Address        change and began seeking funding to address Year 2000 issues years ago.
Key Year 2000 Issues     However, although some progress has been made, several key project
to Increase Chances      management actions associated with the assessment phase have not been
                         completed. As a result, LSSC is not presently well-positioned to move
of Success               forward to the more difficult phases of renovation, validation, and
                         implementation in the Year 2000 process—phases that industry experts
                         estimate could consume as much as three-fourths of Year 2000 project
                         time and resources. LSSC still needs to take a number of actions to increase
                         its chances of success, including (1) managing competing workload
                         priorities, (2) planning for testing, (3) clarifying and coordinating written
                         systems interface agreements, and (4) developing a contingency plan. To
                         increase its chances of successfully managing its Year 2000 program, LSSC
                         will also need to institutionalize a repeatable software change process that
                         can be used from project to project. If these areas are not addressed soon,
                         LSSC could find itself limited in its ability to meet the turn of century date.
                         Given the prominence of date processing in CCSS and its central mission of
                         sustaining the soldier in the field, LSSC cannot afford to delay any longer,
                         and needs to demonstrate that it will perform, all the key actions
                         associated with sound Year 2000 planning and management.


LSSC Should Initiate     In 1991, the Software Engineering Institute (SEI)11 introduced the
Actions to Improve Its   Capability Maturity Model (CMM) to assist organizations in assessing the
Software Capability      maturity level of their software development and maintenance processes.
                         In general, software process maturity serves as an indicator of the likely
Maturity                 range of cost, schedule, and quality of results that can be expected to be
                         achieved by projects within a software organization. Our Year 2000
                         Assessment Guide points out that few activities within federal agencies
                         operate above CMM level 1, and as a result, organizations lack the basic
                         policies, tools, and practices necessary to successfully manage large-scale
                         efforts. A CMM level 1 is the lowest level and is characterized by a software

                         11
                          The Software Engineering Institute (SEI) is a federally funded research and development center
                         sponsored by DOD and operated by Carnegie Mellon University in Pittsburgh, Pennsylvania.



                         Page 9                                                  GAO/AIMD-97-149 Army LSSC Year 2000
                             B-277732




                             process that is ad hoc, and occasionally even chaotic. Few processes are
                             defined and success depends on individual effort. We have recommended
                             that federal agency information technology organizations be at least a CMM
                             level 2 which is characterized by an established software development
                             process discipline that is repeatable from project to project.

                             In 1994, LSSC’s software development process was assessed by a team of
                             LSSC and SEI-licensed contract staff. Using the CMM methodology, the team
                             determined that LSSC should be ranked at a level 1 maturity. The
                             assessment results concluded that LSSC lacked the basic software
                             management practices necessary for repeatable software project success.
                             The team also indicated that level 2 maturity could be attainable with a
                             modest effort. Accordingly, the assessment team made recommendations
                             that, if implemented, could provide the basis for LSSC’s attainment of a
                             level 2 maturity. Based on the team’s assessment, LSSC developed an action
                             plan to address the identified deficiencies. However, according to LSSC
                             officials, the action plan was never implemented due to the reassignment
                             of LSSC assessment staff, agency staff reductions, and lack of funding.

                             After a period of nearly 3 years, LSSC resurrected the CMM assessment
                             project in December 1996 to, once again, review the assessment team’s
                             findings and recommendations and to propose follow-on actions to
                             address the deficiencies. The review concluded that a project management
                             system which would allow LSSC to better plan, estimate, and track software
                             projects on an enterprise-wide basis was essential for LSSC to mature to a
                             CMM level 2. While LSSC has an automated project management system
                             under development, a member of the LSSC review said the system is
                             inadequate because it is unable to track all software projects and may not
                             address all level 2 requirements. This information was presented to LSSC’s
                             Executive Steering Council in March 1997. However, at the time of our
                             review, LSSC had made little progress in correcting the software process
                             deficiencies and was still ranked at CMM level 1. Until LSSC moves on to the
                             next CMM level, its ability to contend with the later stages of the Year 2000
                             effort will be constrained. Recently, LSSC officials informed us of their
                             intent to obtain a CMM level 2 certification following completion of the Year
                             2000 project.


Actions Needed to Mitigate   In addition to lacking a mature software development and maintenance
Impact of Competing          process, LSSC now has 42 percent fewer staff available to make the needed
Workload Priorities          renovations to CCSS than it had in fiscal year 1990. Moreover, since fiscal
                             year 1990, LSSC’s workload has increased, showing a notable jump in fiscal



                             Page 10                                    GAO/AIMD-97-149 Army LSSC Year 2000
B-277732




years 1997 and 1998—the 2 years when the majority of Year 2000 actions
need to be performed to enable agencies to have a realistic chance of
meeting the turn of century date time frame. At the same time that its staff
is decreasing and its workload is increasing, LSSC continues to be tasked
with other software projects by the Lead AMC Integration Support Office
(LAISO). Despite these indicators of potential problems, LSSC has only
recently begun to take the steps necessary to augment its staff with
contract support for the renovation phase and has yet to fully resolve
staffing issues concerning the development of test plans. In addition, LSSC
has not prioritized its software project schedule to provide the structure
needed to keep the Year 2000 project on schedule and within cost
estimates. Until these issues are addressed, they pose unnecessary risk to
the success of LSSC’s Year 2000 project.

As of June 1997, LSSC reported that it had devoted 7 of its 315 total staff to
the Year 2000 project full-time. While four contract support staff had been
retained to train LSSC staff to use the automated software assessment tools
and help with impact assessment, these contractor staff have since been
released. As of August 1997, no contract staff were on board to augment
LSSC staff during the renovation phase, although steps were underway to
obtain additional contract support and to obtain an automated code
correction solution.12

Also, LSSC reported that staff would be tasked to work exclusively on the
Year 2000 renovation phase after completing an ongoing major systems
change project related to a Base Realignment and Closure (BRAC)
decision.13 LSSC officials stated that as the BRAC-related renovation begins
to diminish in September 1997, both LSSC and contract staff would be
transferred to the Year 2000 project. While we do not question the
appropriateness of performing the BRAC-related work prior to Year 2000
work, we are concerned that LSSC’s Year 2000 project approach does not
provide for alternatives should the BRAC target completion schedule slip
and the subsequent LSSC staff and contractors not become available.


12
 LSSC is planning to utilize the services of a private contractor that provides an automated code
conversion service to organizations seeking an automated method for correcting Common Business
Oriented Language (COBOL) software code. Under this arrangement, LSSC will provide its software
code to the contractor’s off-site facility where Year 2000 windowing logic is inserted using the
contractor’s proprietary automated software. The converted code will then be returned to LSSC for
compiling and testing.
13
  The BRAC decision entails reducing the size of DOD and saving money by closing or consolidating
DOD facilities. Since one of AMC’s major subordinate commands (MSC), located in St. Louis, Missouri,
was dissolved and its functions moved to other MSCs as a result of the 1995 BRAC decision, it has
been necessary for LSSC to develop conversion programs to extract aviation and troop support
information and transfer the data to the MSCs now responsible for those functions.



Page 11                                                 GAO/AIMD-97-149 Army LSSC Year 2000
                           B-277732




                           Further, an examination of CCSS software release schedules since fiscal
                           year 1990 shows that the number of projects has increased as much as
                           five-fold. At the same time as the majority of CCSS Year 2000 actions are to
                           be performed, LSSC’s schedule calls for 10 software change projects to be
                           fielded in fiscal year 1997 and 8 in fiscal year 1998. These projects range in
                           terms of complexity and magnitude from routine systems maintenance,
                           which may require minimal effort, to the Year 2000 and BRAC projects,
                           which call for comprehensive changes in many of CCSS’ subsystems. In past
                           years, LSSC routinely accomplished two to four software change projects a
                           year. This significant increase in workload will undoubtedly impact the
                           CCSS Year 2000 project schedule for several reasons. First, LAISO, the
                           workload manager for CCSS, has not ensured that competing projects do
                           not adversely affect LSSC’s ability to complete the Year 2000 effort.
                           Prioritization of projects could result in the postponement or cancellation
                           of some of the competing projects. Second, LSSC has little historical
                           experience dealing with a workload of this magnitude which is
                           compounded by a workforce that has diminished significantly in recent
                           years.


LSSC Should Be Planning    Prior to commencing the validation (testing) phase of its Year 2000 effort,
for the Validation Phase   LSSC needs to fully address two key issues regarding its testing
                           requirements and capabilities. Specifically, LSSC should be planning now to
                           (1) assure that enough staff with the appropriate background and
                           experience are available to develop Year 2000 test data and transactions
                           and to review test results and (2) assess whether enough time has been
                           scheduled to perform Year 2000 testing. Without planning how it will
                           address these issues now, LSSC is increasing the risk that CCSS will not be
                           fully validated in time for the change of century date.

                           According to AMC’s Year 2000 Action Plan, many agencies will need to
                           establish test environments which are specific to future date testing and
                           which have no possibility of corrupting or destroying production data.14
                           Since the current CCSS test files do not contain the necessary Year 2000 test
                           conditions and data, LSSC will need to establish Year 2000-specific test files
                           to certify that CCSS is Year 2000 compliant. Such test data and transactions
                           are typically designed by functional staff knowledgeable of the CCSS
                           business processes. These staff review the testing results to ensure that
                           Year 2000 software changes have processed data correctly and that other

                           14
                             Production data are existing data and transactions processed by or generated from everyday,
                           operational use of a computer system. Preparing and using production data for testing does not require
                           knowledge of the internal logic of the software. As a result, production data may not test all the
                           functions or logic paths desired and could produce inconclusive results.



                           Page 12                                                  GAO/AIMD-97-149 Army LSSC Year 2000
B-277732




data and processes have not been inadvertently changed during testing.
According to a LSSC official and LSSC staffing statistics, however, there are
fewer functional staff now available to identify the date fields in the test
transactions or test data needed to ensure that CCSS business processes are
not adversely affected by the Year 2000 software changes. Also, LSSC
officials stated that they expect the availability of these staff to continue to
decrease over the next few years as staff retire and agency staff reductions
continue.

Further, LSSC is not allowing enough time for Year 2000 testing. While LSSC
officials assert that the complexity and scope of the Year 2000 project is
about the same as the BRAC project, LSSC’s June 1997 systems change
release schedule calls for far less time to test Year 2000 changes than it
does for BRAC changes. For example, BRAC testing began in February 1997
and is scheduled for completion in September 1997. Year 2000 testing is
scheduled to begin in September 1998 and end almost 8 weeks later in
November 1998. A LSSC official acknowledged that the amount of time
scheduled for Year 2000 testing is insufficient, but stated that the schedule
will be revised once ongoing negotiations to acquire an automated code
correction service are resolved. The official also stated that he fully
expects the Year 2000 test schedule to greatly increase beyond the
currently scheduled 8 weeks but that the increased test time should be
offset by the reduced renovation time expected to be garnered by using
the automated code correction service. Although LSSC believes that the
automated code correction service should provide increased Year 2000
testing time, it could not provide documented analysis to support this
conclusion.

While LSSC believes it can increase its testing time without increasing the
overall Year 2000 project time, we are not as confident given LSSC’s CMM
level 1 ranking. Trying to compensate for unrealistic time schedules by
either shortening earlier phases of a software change project or by
lengthening overall project time is characteristic of level 1 organizations.
Until LSSC realistically assesses its testing requirements, capabilities, and
time schedules, effective Year 2000 project management will become
increasingly difficult to achieve, and LSSC will increase the risk that it may
be unable to meet the demand imposed by Year 2000 testing.




Page 13                                      GAO/AIMD-97-149 Army LSSC Year 2000
                            B-277732




Interface Agreements Lack   CCSS’ ability to successfully operate at the year 2000 hinges on the proper
Basic Information for       and timely exchange of data with other systems, both within the Army and
Effective Management and    with external Defense components. It is critically important during the
                            Year 2000 effort that agencies ensure that interfacing systems have the
Implementation              ability to exchange data throughout the transition period and protect
                            against the potential for introducing and propagating errors from one
                            organization to another and ensure that interfacing systems have the
                            ability to exchange data through the transition period. This potential
                            problem may be mitigated through formal agreements between interface
                            partners that describe the method of interface and assign responsibility for
                            accommodating the exchange of data. Both the DOD Year 2000
                            Management Plan and AMC Year 2000 Action Plan place responsibility on
                            component heads or their designated Year 2000 contact points to
                            document and obtain system interface agreements in the form of
                            memorandums of agreement (MOA) or the equivalent. Further, to help
                            assure that interfaces continue to properly exchange data after systems
                            are renovated for the year 2000, AMC has issued minimum MOA
                            documentation requirements designed to produce consistency, assign
                            accountability, and recognize a level of detail necessary for effective
                            interface renovation among data exchange partners.

                            While LSSC has developed MOAs to document interface specifics between
                            CCSS and its interfacing systems and is in the process of finalizing those
                            agreements with system owners, nearly all the MOAs lack basic information
                            necessary for effective management and implementation of the interfaces.
                            According to AMC Year 2000 guidance and the accompanying requirements
                            of the standard MOA, the agreements are to specify the (1) points of contact
                            for reporting progress and coordinating schedules and (2) date the
                            agreement becomes effective. To successfully implement interface
                            changes, these agreements should also communicate the type, form, and
                            frequency of transactions exchanged, the windowing technique that is
                            being used at each end of the interface, and the review process for
                            monitoring interface renovation progress and reconciling differences.
                            However, our review disclosed that 39 of 41 MOAs that LSSC had finalized as
                            of July 1997 failed to fully follow AMC’s guidance or include other
                            information necessary to ensure that LSSC can successfully communicate
                            with interface partners. Our Year 2000 Assessment Guide stresses the
                            importance of adequately addressing interface and data exchange issues.
                            Without such information, the MOAs do not serve to communicate and
                            coordinate the actions designed to help assure that Year 2000 changes are
                            made properly and promptly by LSSC and its interfacing partners.




                            Page 14                                   GAO/AIMD-97-149 Army LSSC Year 2000
                        B-277732




                        Timely and complete information on all systems interfaces that may be
                        affected by Year 2000 changes is essential to the success of the LSSC Year
                        2000 compliance program. The amount of work required to coordinate the
                        data being exchanged between systems must be known as early as
                        possible and documented in written MOAs so that LSSC may complete
                        renovation schedules, allocate resources, plan testing, and schedule
                        implementation.


Year 2000 Contingency   The year 2000 represents a great potential for operational failure to CCSS
Plan Needed for CCSS    that could adversely impact core business processes as well as those of
                        entities that depend on the CCSS system for information. To mitigate this
                        risk of failure, our Year 2000 Assessment Guide, DOD’s Year 2000
                        Management Plan, and the Army’s Project Change of Century Action Plan
                        suggest that agencies perform risk assessments and prepare realistic
                        contingency plans that identify alternatives to ensure the continuity of
                        core business processes in the event of operational failure. These
                        alternatives could include performing automated functions manually or
                        using the processing services of contractors.

                        While LSSC has taken the first steps toward development of a contingency
                        plan by assessing the level of risk to each business area that could be
                        affected by processing errors and by determining how that risk can be
                        mitigated or reduced, at the completion of our review, LSSC had not yet
                        developed a contingency plan. Further, despite explicit guidance from DOD
                        and the Army to develop contingency plans should Year 2000 corrections
                        to CCSS not be completed in time, LSSC officials stated that no contingency
                        plan would be developed for CCSS. They maintained that AMC does not
                        require a contingency plan for CCSS because CCSS is not scheduled for
                        replacement prior to the advent of the year 2000. While AMC’s Year 2000
                        Action Plan states that contingency plan development is only required for
                        replacement systems and implies that all other systems are exempt, the
                        AMC plan also states that the guidance, policy, and responsibilities
                        identified in the Army’s Project Change of Century Action Plan are
                        mandatory and are the basis of the AMC plan.

                        Nevertheless, despite LSSC’s and AMC’s position that a contingency plan is
                        not needed for CCSS because the system is not being replaced prior to the
                        year 2000, the system still risks unanticipated operational failure. Without
                        a contingency plan that identifies specific actions to be taken if CCSS fails
                        at the year 2000, the procurement of weapon systems and their spare
                        parts, accounting for the sale of Army equipment and services to allies,



                        Page 15                                     GAO/AIMD-97-149 Army LSSC Year 2000
                      B-277732




                      and the financial management of $9 billion of inventory could be
                      disrupted. As a result, the Army could be unable to efficiently and
                      effectively equip and sustain its forces around the world. Given the
                      dangers associated with an operational failure of this magnitude, LSSC
                      needs the protection provided by good contingency planning to ensure
                      that options are available if CCSS is not able to operate at the year 2000.
                      Recently, LSSC officials stated that they have begun preparing an initial
                      contingency plan, which they estimate will be completed by September 30,
                      1997.


                      If CCSS cannot correctly process dates on and after January 1, 2000,
Conclusions           military equipment, such as tanks, artillery, aircraft, missiles, munitions,
                      trucks, electronics, and other supporting materials for the soldier, in all
                      likelihood, will not be ordered, stored, transported, issued, paid for, or
                      maintained. Mobilization plans and contingencies would be significantly
                      impaired if materiel is delayed. However, LSSC has yet to resolve several
                      critical problems associated with the assessment phase to ensure that
                      (1) systems are adequately tested, (2) contingency plans are developed,
                      and (3) interface partners are fully aware of LSSC’s Year 2000 plans.
                      Furthermore, during the same time that LSSC is addressing the Year 2000
                      issue, the agency is also working to implement considerably more
                      software projects than it has in the past. This unprecedented workload is
                      compounded by a reduced staff level and LSSC’s basic lack of a mature
                      software development and maintenance process. Together, these factors
                      raise the risk level of the Year 2000 project beyond what is normally
                      expected of a software modification effort of this magnitude. Until these
                      problems are resolved, LSSC is not well-positioned to move forward into
                      the more time-consuming phases of renovation, validation, and
                      implementation. As a result, we believe LSSC will find it increasingly
                      difficult to prepare CCSS in time for the arrival of the year 2000.


                      We recommend that you:
Recommendations
                  •   Act to improve LSSC’s software development process that will provide the
                      basis for achieving CMM level 2 maturity.
                  •   Immediately assess the impact of competing workload and staffing
                      demands on the CCSS Year 2000 project. Based on this assessment,
                      consider (1) canceling or deferring less critical software projects until
                      after the Year 2000 project is substantially completed and (2) augmenting




                      Page 16                                    GAO/AIMD-97-149 Army LSSC Year 2000
                         B-277732




                         the Year 2000 project with staff having the necessary skills to ensure
                         timely completion of the project.
                     •   Ensure that LSSC has the capability to complete the testing of all CCSS
                         subsystems and programs. Specifically, LSSC should (1) determine test
                         requirements, (2) identify the testing staff needed, (3) finalize Year 2000
                         test plans describing how the testing staff will be acquired and scheduled
                         for developing Year 2000 compliant test scenarios and data, and (4) revise
                         the Year 2000 test schedule to assure that enough time is available to meet
                         Army-mandated deadlines for Year 2000 implementation.
                     •   Ensure that written interface agreements describe the method of data
                         exchange between interfacing systems, name the entity responsible for
                         performing the system interface modification, and state the completion
                         date.
                     •   Develop a contingency plan that includes specific actions for ensuring that
                         the Army’s logistic functions continue to operate at appropriate levels if all
                         or part of CCSS fails to work at the year 2000.


                         In written comments on a draft of this report, the Office of the Under
Agency Comments          Secretary of Defense (Acquisition and Technology) concurred with all of
and Our Evaluation       our recommendations to improve the Army’s LSSC Year 2000 program.
                         Specifically, DOD agreed that a contingency plan would be developed by
                         September 30, 1997, to ensure continuity of operations if all or part of CCSS
                         fails to operate by the year 2000.

                         DOD also outlined a number of actions that have recently been initiated
                         that are aimed at reducing and prioritizing LSSC’s current workload, and
                         increasing staff with the necessary skills to help ensure the timely
                         completion of the Year 2000 project. In addition, DOD pointed to several
                         actions, both taken and planned, to improve its capability to complete
                         Year 2000 testing of CCSS subsystems and programs. While we have not
                         reviewed LSSC’s latest actions, if properly implemented, we believe they
                         could help resolve the workload and testing issues we identified.

                         In concurring with our recommendation regarding the need to initiate
                         actions to improve LSSC’s software development process, DOD recognized
                         the value of achieving a CMM level 2 maturity and agreed that LSSC does not
                         have all configuration management procedures in place to reach CMM level
                         2 at this time. However, DOD stated that LSSC’s history indicates that it can
                         accomplish large projects successfully and that LSSC will meet the
                         mandated dates for the BRAC and Year 2000 projects without achieving CMM
                         level 2. After completion of these projects, LSSC plans to resume its efforts



                         Page 17                                     GAO/AIMD-97-149 Army LSSC Year 2000
B-277732




to achieve a CMM level 2 maturity. We believe LSSC’s position comes at some
risk. The discipline derived from reaching a CMM level 2 maturity can
greatly enhance LSSC’s ability to address the Year 2000 challenge. This
higher level of maturity is key to reducing the risk of schedule slippage,
cost overruns, and poor software quality. As our report states, we have
recommended that information technology organizations be at least a CMM
level 2 to successfully manage large-scale projects such as the Year 2000
project. Our Year 2000 Assessment Guide provides interim actions that
level 1 organizations can take prior to the year 2000 to minimize the risk of
failure, such as training staff on proven industry system development and
program management practices and soliciting assistance from
organizational entities experienced in performing or managing major
software conversions. LSSC could benefit from these interim actions.

In concurring with our recommendation on strengthening written
interface agreements, DOD stated that LSSC will formalize MOAs between
interface partners. It also agreed to include specific detailed information in
MOAs, but only when appropriate. As our report stated, we believe that, at a
minimum, MOAs should also contain essential information for effective
management of system interfaces, such as the type, form, and frequency of
transactions exchanged, the windowing technique to be used, and the
review process for monitoring interface renovation progress and
reconciling differences. This additional information would help to ensure
that interface partners are sufficiently prepared to handle unforeseen
problems that may occur and to plan for contingencies. The full text of
DOD’s comments is provided in appendix I.



This report contains recommendations to you. Within 60 days of the date
of this letter, we would appreciate receiving a written statement on actions
taken to address these recommendations.

We appreciate the courtesy and cooperation extended to our audit team by
LSSC officials and staff. We are providing copies of this letter to the
Chairmen and Ranking Minority Members of the Senate Committee on
Governmental Affairs; the Subcommittee on Oversight of Government
Management, Restructuring and the District of Columbia, Senate
Committee on Governmental Affairs; and the Subcommittee on
Government Management, Information and Technology, House Committee
on Government Reform and Oversight; the Honorable Thomas M. Davis,
III, House of Representatives; the Secretary of Defense; the Deputy
Secretary of Defense; the Acting Under Secretary of Defense (Acquisition



Page 18                                    GAO/AIMD-97-149 Army LSSC Year 2000
B-277732




and Technology); the Acting Under Secretary of Defense (Comptroller);
the Acting Assistant Secretary of Defense (Command, Control,
Communications and Intelligence); the Secretary of the Army;
Commanders of the Army Materiel Command and Communications-
Electronics Command; the Director of the Office of Management and
Budget; and other interested parties. Copies will be made available to
others upon request.


If you have any questions on matters discussed in this letter, please call me
at (202) 512-6240, or John B. Stephenson, Assistant Director, at
(202) 512-6225. Major contributors to this report are listed in appendix II.

Sincerely yours,




Jack L. Brock, Jr.
Director, Defense Information and
  Financial Management Systems




Page 19                                    GAO/AIMD-97-149 Army LSSC Year 2000
Contents



Letter                                                                                        1


Appendix I                                                                                   22

Comments From the
Department of
Defense
Appendix II                                                                                  27

Major Contributors to
This Report




                        Abbreviations

                        AMC       Army Materiel Command
                        BRAC      Base Realignment and Closure
                        CBSX      Continuing Balance System-Expanded
                        CCSS      Commodity Command Standard System
                        CDA       central design activities
                        CECOM     Communications-Electronics Command
                        CIM       Corporate Information Management
                        CMM       Capability Maturity Model
                        COBOL     Common Business Oriented Language
                        DOD       Department of Defense
                        LAISO     Lead AMC Integration Support Office
                        LSSC      Logistics Systems Support Center
                        MOA       memorandum of agreement
                        MSC       major subordinate command
                        SEI       Software Engineering Institute


                        Page 20                              GAO/AIMD-97-149 Army LSSC Year 2000
Page 21   GAO/AIMD-97-149 Army LSSC Year 2000
Appendix I

Comments From the Department of Defense


Note: GAO comment
supplementing those in
the report text appears at
the end of this appendix.




See comment 1.




                             Page 22   GAO/AIMD-97-149 Army LSSC Year 2000
Appendix I
Comments From the Department of Defense




Page 23                                   GAO/AIMD-97-149 Army LSSC Year 2000
Appendix I
Comments From the Department of Defense




Page 24                                   GAO/AIMD-97-149 Army LSSC Year 2000
Appendix I
Comments From the Department of Defense




Page 25                                   GAO/AIMD-97-149 Army LSSC Year 2000
              Appendix I
              Comments From the Department of Defense




              The following is GAO’s comment on the Department of Defense’s letter
              dated September 12, 1997.


              1. DOD provided a number of clarifications to the report that we have
GAO Comment   incorporated as appropriate.




              Page 26                                   GAO/AIMD-97-149 Army LSSC Year 2000
Appendix II

Major Contributors to This Report


                       Ronald B. Bageant, Assistant Director
Accounting and         Carl M. Urie, Technical Advisor
Information            Brenda A. James, Senior Information Systems Analyst
Management Division,   Cristina T. Chaplain, Communications Analyst

Washington, D.C.
                       Denice M. Millett, Evaluator-in-Charge
Kansas City Field      Michael W. Buell, Staff Member
Office




(511627)               Page 27                                  GAO/AIMD-97-149 Army LSSC Year 2000
Ordering Information

The first copy of each GAO report and testimony is free.
Additional copies are $2 each. Orders should be sent to the
following address, accompanied by a check or money order
made out to the Superintendent of Documents, when
necessary. VISA and MasterCard credit cards are accepted, also.
Orders for 100 or more copies to be mailed to a single address
are discounted 25 percent.

Orders by mail:

U.S. General Accounting Office
P.O. Box 37050
Washington, DC 20013

or visit:

Room 1100
700 4th St. NW (corner of 4th and G Sts. NW)
U.S. General Accounting Office
Washington, DC

Orders may also be placed by calling (202) 512-6000
or by using fax number (202) 512-6061, or TDD (202) 512-2537.

Each day, GAO issues a list of newly available reports and
testimony. To receive facsimile copies of the daily list or any
list from the past 30 days, please call (202) 512-6000 using a
touchtone phone. A recorded menu will provide information on
how to obtain these lists.

For information on how to access GAO reports on the INTERNET,
send an e-mail message with "info" in the body to:

info@www.gao.gov

or visit GAO’s World Wide Web Home Page at:

http://www.gao.gov




PRINTED ON    RECYCLED PAPER
United States                       Bulk Rate
General Accounting Office      Postage & Fees Paid
Washington, D.C. 20548-0001           GAO
                                 Permit No. G100
Official Business
Penalty for Private Use $300

Address Correction Requested