Year 2000 Computing Crisis: An Assessment Guide--Exposure Draft (Superseded by AIMD-10.1.14)

Published by the Government Accountability Office on 1997-03-01.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                      United States General Accounting Office   " '

GAO                   Accounting
                      Managementand Information

February    1997
            1997      Year 2000 Computing Crisis:
                      An Assessment Guide

                      Exposure Draft

          The year 2000 is not rocket science, but it is the largestprojectever to
          be undertaken by the IT organization. The complexity of the projectis
          not in the solution but ratherin the size and scope of the project itself.
          This means that the year 2000 requires"world class"project

                        /I~~~~~~     ~~~Kevin                        Schick
                                                              Gartner Group
                                                              April 16, 1996

GAO/AIMD-10.1.14 Year 2000 Computing Crisis

At 12:01 on New Year's morning of the year 2000, many computer systems worldwide could
malfunction or produce incorrect information simply because the date has changed. Unless
corrected, the impact of these failures could be widespread and costly. For example:

*   IRS' tax systems could be unable to process returns, which in turn could jeopardize the
    collection of revenue and the entire tax processing system.

*   Payments to veterans with service-connected disabilities could be severely delayed because
    Veterans Affairs' compensation mand pension system either halts or produces checks that are so
    erroneous that the system must be shut down and the checks processed manually.

*   Social Security Administration's disability insurance process could experience major
    disruptions because the interface with various state systems fails, thereby causing delays and
    interruptions in disability payments to citizens.

*   Federal systems used to track student education loans could produce erroneous information on
    loan status, such as indicating that an unpaid loan had been satisfied.

The year 2000 problem is rooted in the way dates are recorded and computed in many computer
systems. For the past several decades, systems have typically used two digits to represent the year,
such as "97" representing 1997, in order to conserve on electronic data storage and reduce
operating costs. With this two-digit format, however, the year 2000 is indistinguishable from
1900, 2001 from 1901, and so on. As a result of this ambiguity, system or application programs
that use dates to perform calculations, comparisons, or sorting may generate incorrect results when
working with years after 1999.

Many government computer systems were originally designed and developed 20 to 25 years ago,
are poorly documented, and use a wide variety of computer languages--many of which are old or
obsolete. The systems consist of tens or hundreds of computer programs, each with thousands,
tens of thousands, or even millions of lines of code, which must be examined for date problems.
Moreover, the government's computer systems, like private sector systems, have numerous
components--hardware, software stored in read-only-memory, operating systems, communications
applications, and database software--that are affected by the date problem. Correcting the problem
and achieving year 2000 compliance--defined as the ability of information systems to accurately
process date data from, into, and between the twentieth and twenty-first centuries, including leap
year calculations--will not be easy.

Every federal agency is at risk of widespread system failures. Because converting systems to a
4-digit year will be a massive undertaking for large systems, agencies must start now to address
this problem. They need to identify their inventories of mission-critical computer systems, develop
conversion strategies and plans, and dedicate sufficient resources to converting and adequately
testing their computer systems and programs before January 1, 2000.

This guide provides a framework and a checklist for assessing the readiness of federal agencies to
achieve year 2000 compliance. It provides information on the scope of the challenge, and offers a

GAO/AIMD-10.1.14 Year 2000 Computing Crisis                                                          2
structured approach for reviewing the adequacy of agency planning and management of the year
2000 program.

Because each agency is different, there is no single, cookie cutter approach for solving the year
2000 problem. Some agencies are highly centralized, while others operate in a highly decentralized
information resource environment. This guide addresses issues that will be common to most year
2000 programs; however, each agency must tailor its year 2000 program in response to its unique

The guide is divided into five phases supported by program and project management activities:

        *   Awareness
        *   Assessment
        *   Renovation
        *   Validation
        *   Implementation

An electronic version of this guide is available from GAO's World Wide Web server at the
following Internet address: http:l/www.gao.gov/. If you have any questions about the guide or the
year 2000 process outlined here, please contact us, or Mirko J. Dolak, Technical Assistant
Director, at (202) 512-6362. We can also be reached by e-mail at willemssenj.aimd@gao.gov,
franklinw.aimd@gao.gov, and dolakm.aimd@gao.gov.

Joel C. Willemssen                             William S. Franklin
Director                                       Director
Information Resources Management               Information Systems Methods and Support

GAO/AIMD-10.1.14 Year 2000 Computing Crisis                                                         3

Year 2000 Conversion Model: Structured Approach and
Rigorous Project Management Can Decrease Risks






Program and Project Management

Year 2000 Program Assessment Checklist

Selected Year 2000 Resources


GAO/AIMD-10.1.14 Year 2000 Computing Crisis           4
Year 2000 Conversion Model: Structured Approach and
Rigorous Program Management Can Decrease Risks

The year 2000 date conversion poses a global challenge to the information technology industry.
Every organization, whether federal or private, must ensure that its information systems are fully
year 2000 compliant well before December 31, 1999. While the year 2000 problem is not
technically challenging, it is massive and complex. For many agencies, the year 2000 conversion
program will be the largest project ever to be managed and implemented by their information
resource management organizations.

 This guide presents a structured approach and a checklist to aid federal agencies in planning,
 managing, and evaluating their year 2000 programs. The guide draws heavily on the work of the
-Best Practices Subcommitteee of the Interagency Year 2000 Committee, and incorporates guidance
 and practices identified by leading organizations in the information technology industry.

The guide describes five phases--supported by program and project management--with each phase
representing a major year 2000 program activity or segment.

                                                    Year 2000 Conversion Model

                                                                Awarenes                             Define the year 2000 problem and gain executive level
                                                                Awareness                            support and sponsorship. Establish year 2000 program
                                               'K               _team                                     and develop an overall strategy. Ensure that
                                                                       every.one..in. the..organi~za.tion. is.ful~lyaware. of!the issue .
                                                   Assessment                                   Assess the year 2000 impact on the enterprise. Identify
                                                   J^~                                        j     ~~core
                                                                                                      business areas and processes, inventory and
                                                    :   :                                          : :.....analyze
                                                                                                          systems supporting the core business areas,
                                                                                                and prioritize their conversion or replacement.
                                                                                                Develop contingency plans to handle data exchange

                           .   i~~~                                                             issues, lack of data, and bad data. Identify and secure
                                                                                             g ~~the~~~
                                                                                                    necessary resources.
       Program &
        Project                  Renovation                Convert, replace, or eliminate selected platforms,
                 PManagj~emnc~t                       .~ ~applications, databases, and utilities. Modify
      Managementj                                          interfaces.
   :: : :: ! :    ~    ............. ..............................................................................................
                                                     Validation                                      Test, verify, and validate converted or replaced
                                                                                                     platforms, applications, databases, and utilities. Test
                                                                                                     the performance, functionality, and integration of
                                                                                                     converted or replaced platforms, applications,
                                                                                                     databases, utilities, and interfaces in an operational

                                             Implementation                                          Implement converted or replaced platforms,
                                                                                                     applications, databases, utilities, and interfaces.
                                                                                                     Implement data exchange contingency plans, if

   Plan and manage the year 2000 program as a single large information system development
   effort. Promulgate and enforce good management practices on the program and project levels.

GAO/AIMD-10.1.14 Year 2000 Computing Crisis                                                                                                                    5
Immovable Deadline and Fixed Schedule

Time is running out. Renovation work should be done by the end of 1998 or early 1999 to allow
sufficient time for validation and implementation. However, agencies are just beginning to address
the year 2000 problem. A 1996 congressional survey of 24 agencies has showed that only 9
agencies had developed a year 2000 plan, and that only 7 agencies had estimated the year 2000
program costs., These agencies must act quickly and start their year 2000 programs now.

                                         Year 2000 Schedule





                           A                          Vldto

    Offce Sptmbr 7,19976.

                          LII  A                   ~~~Implementation

 Year 2000 Computer Software Conversions: Surrmary of Oversight Findings and Recommendations
(House Report 104-857), Committee on Government Reform and Oversight, U. S. Govermnment Printing
Office, September 27, 1996.

GAOIAIMD-10.1.14 Year 2000 Computing Crisis                                                        6
1.0 Awareness

As agencies begin to deal with the year 2000 issue, it is essential that executive management be
fully aware of the year 2000 problem and its potential impact on the enterprise and its customers.
It is the responsibility of the chief information officer to provide the leadership in defining and
explaining the importance of achieving year 2000 compliance, selecting the overall approach for
structuring the agency's year 2000 program, assessing the adequacy of the existing information
resource management infrastructure to adequately support the year 2000 efforts, and mobilizing
needed resources.

g                                                                                                 .:ยข|

1.1. Define the year 2000 problem and its potential impact on the enterprise

      Developing andpublishing a high-level assessment of the year 2000 issue provides
      executive management and staffwith a high-level overview of the potentialimpact of the
      year 2000 problem on the enterprise.

1i.2. Conduct a year 2000 awareness campaign

      A year 2000 awareness campaign is an importantfirststep to raisethe awarenessof
      executive management and line staffabout the potentialimpact of the year 2000 problem
      on the agency's operations.

1.3. Assess the adequacy of the agency's program management capabilities, including

      1   policies, guidelines, and processes for program and project management, configuration
          management, quality assurance, and risk management
      1   staffing levels and skill mix

      The ability to successfully manage the year-2000programwill depend on the degree to
      which the agency has institutionalizedkey system development and programmanagement
      practices and on experience in  itsmanaging large-scalesoftware conversion or system
      development efforts. With only afew a ctivities withinfederal agencies operatingabove

GAO/AIMD-10.1.14 Year 2000 Computing Crisis                                                           7
        level 1 on the_Software Engineering Institute's CapabilityMaturity Model,2 most
        information resource management organizations lack the basicpolicies, tools, and
        practicesnecessary to successfully manage a large-scaleyear-2000program. While there
        may not be enough time to achieve a higher maturity level, agencies shouldassess, and
        upgrade, if needed, their information resource management capabilities. Agencies should
        consider the establishment of an enterprise competency center to provide trainingand to
       foster adherenceto proven industry system development and programmanagement
       practices. Agencies also need to considersoliciting assistancefrom organizationalentities
        experienced in performing or managing major software conversions.

1.4.   Develop and document a high-level year 2000 strategy which addresses

       A high-level year 2000 strategy provides the agency's executive management with a
       roadmapfor achievingyear 2000 compliance. The strategy should discuss key year 2000
       issues, including the program'smanagement structure, programmetrics and reporting
       requirements, the mix of enterprise-widesolutions, and provide initial cost and schedule

1.5.   Obtain and formalize executive management support through issuance of

       *     year 2000 policy directive
       *     year 2000 program charter

       The management supportfor the agency's year2000 strategy should be formalized by the
       issuance of a year 2000 policy directive, and/oryear 2000 programcharter. Without such
       support, information resource managers may not be able to mobilize adequate resources to
       implement the strategy and to interactwith other organizationsand interfaced data

1.6.   Establish year 2000 executive management council

       A committee or a council needs to be establishedwithin the agency to continually
       coordinatewith the programmaticand functionalarea managers on prioritiesand
       potential mission impact if certainprocesses and systems malfunction. A processfor quick
       conflict resolution on prioritiesbetween programmaticand functional areas is also

1.7. Appoint a year 2000 program manager and establish an agency-level year 2000 program

       It is essential that agencies appointa year 2000 program managerand establish an
       agency-level program office to manage and coordinate the enterprise'syear 2000 program
       activities. The solutions of the year 2000 problem extend beyond simple software
       conversion, hardwareupgrades, and databaserestructuring. The problem--and the
       solutions--involve a wide range of dependencies among information systems; the need to

2 "ProcessMaturity Profile of the Software Community, 1996 Update," Software Engineering Institute,
April 1996.

GAO/AIMD-10.1.14 Year 2000 Computing Crisis                                                           8
     centrally develop or acquire conversion and validation standards, inspection, conversion,
     and testing tools; the need to coordinate the conversion of cross-boundaryinformation
     systems and their components; the need to establishpriorities;and the need to reallocate
     resourcesas needed.

1.8. Identify technical and management points of contact in core business areas

     A year 2000 program should not be viewed as a system development or maintenance effort
     managedby the information resourcemanagement organization, but ratheras an
     enterprise-wideeffort requiringthe input and cooperation of all organizationalunits.
     Thus, it is important that the technicaland management staffof the core business areas
     work closely with the year 2000 project teams in the assessment and testing process.

GAO/AIMD-10.1.14 Year 2000 Computing Crisis                                                      9
2.0 Assessment

Federal agencies may not have enough resources, skill, or time to convert or replace all of their
information systems. Agencies must determine what systems are mission-critical and must be
converted or replaced, what systems support important functions and should be converted or
replaced, and what systems support marginal functions, and may be converted or replaced later.

The year 2000 problem is not just an information technology problem, but is primarily a business
problem. Thus, the process of identifying and ranking information systems should not be limited to
a simple inventory of applications and platforms, but must include assessments of the impact of
information systems' failures on the agency's core business areas and processes.

The assessment should also include systems using information technology which operate outside
the traditional information resource area, including building infrastructure systems and telephone
switching equipment.

2.1. Define year 2000 compliance

2.2. Focus on core business areas and processes and develop a year 2000 assessment document

    Information systems are not created equal. Systems supportingmission-criticalbusiness
    processes are clearly more important than systems supporting mission supportfunctions--
    usually administrative--althoughthese are necessaryfunctions. Afocus on core business

GAO/AIMD-10.1.14 Year 2000 Computing Crisis                                                          10
    areasandprocesses is essential to the task of assessing the impact of the year 2000 problem
    on the enterpriseandfor establishingthe prioritiesfor the year 2000 program.

2.3. Assess the severity of an impact of potential year 2000-induced failures

     An assessment of the severity of year 2000 failure needs to be done for each core business
     area and associatedprocesses.

2.4. Conduct an enterprise-wide inventory of information systems for each business area

    An enterprise-wide inventory of information systems and their components provides the
    necessaryfoundationfor year 2000 programplanning. A thorough inventory ensures that
    all systems are identified and linked to a specific business area orprocess, and that all
    enterprise-wide, cross-boundarysystems are considered.

2.5. Use inventory data to develop a comprehensive automated system portfolio and identify, for
     each system

      *   links to core business areas or processes
      *   platforms, languages, and database management systems
      *   operating system software and utilities
      *   telecommunications
      *   internal and external interfaces
      *   owners
      *   the availability and adequacy of source code and associated documentation

2.6. Analyze portfolio and identify for each system

      *   non-repairable items (lack of source code or documentation)
      *   conversion or replacement resources required for each platform, application, database
          management systems, archive, utility, or interface

2.7. Prioritize system conversions and replacements

    An agency must determineprioritiesfor system conversion and replacement by ranking
    based on key factors, such as business impact and the anticipatedfailure date. An agency
    also needs to identify applications, databases, archives, and interfaces that cannot be
    converted because of resource and time constraints.

2.8. Establish year 2000 project teams for business areas and major systems

     Multi-disciplinaryproject teams consisting of domain experts in relevantfunctionalareas,
     system and software specialists, operationalanalysis specialists, and contract specialists
     need to be established with explicit objectives and time schedules. Access to legal advice is
     also a necessity.

GAO/AIMD-10.1.14 Year 2000 Computing Crisis
2.9. Develop year 2000 program plan, including

      *       schedules for all tasks and phases of the year 2000 program
      *       master conversion and replacement schedule, including identification of systems and
              their components
      *       assessment and selection of outsourcing options
      *       assignment of conversion, or replacement projects to year 2000 project teams
      *       risk assessment
      *       contingency plans for all systems

2.10. Identify, prioritize, and mobilize needed resources

      Achieving year 2000 compliance will require significant investment in two vital resources--
      money and people. Accordingly, agencies will need to make informed choices about
      information technology prioritieswithin their organization by assessing the costs, benefits,
      and risks of competing projects. In some instances, agencies may have to defer or cancel
      new system development efforts and reprogram the freed resources to achieve year 2000

2.11. Develop validation strategies and testing plans for all converted or replaced systems and
      their components. Identify and acquire automated test tools and develop test scripts.

      The testing and validation of the converted or replaced systems will require a phased
      approach. For example, an approachdeveloped by IBM includesfour phases:

          *    Phase I--unit testing-focuses onfunctional and compliance testing of a single
               applicationor software module.
          *    PhaseII--integration testing--test the integrationof related software modules and
          *    Phase Ill--system testing--test all of the integratedcomponents of an information
          *    PhaseIV--acceptance testing--test the information system with live operationaldata.

     Regardless of the selected validation and testing strategy, the scope of the testing and
     validation effort will require carefulplanning and use of automated tools, including test
     case analyzers and test data libraries.

2.12. Define requirements for year 2000 test facility

      Agencies may have to acquire a year 2000 test facility to provide an adequate testing
      environment and to avoid potential contamination or interference with the operation of
      production systems.

2.13. Identify and acquire year 2000 tools

      Agencies should identify and acquireyear 2000 tools to facilitatethe conversion and
      testing processes.

GAO/AIMD-10.1.14 Year 2000 Computing Crisis                                                         12
2.14. Address implementation schedule issues, including

      *   the identification and selection of conversion facilities
      *   time needed to put converted systems into production
      *   the conversion of backup and archival data

2.15. Address interface and data exchange issues, including

      *   the development of a model showing the internal and external dependency links between
          enterprise core business areas, processes, and information systems
      *   the notification of all outside data exchange entities
      *   the need for data bridges and filters
      *   contingency plans if no data are received from an external source
      *   validation process for incoming external data
      *   contingency plans for invalid data

2.16. Develop contingency plans for critical systems and activities

     Agencies shoulddevelop realisticcontingency plans--includingthe development and
     activation of manual or contractprocedures--toensure the continuity of its core business

2.17. Identify year 2000 vulnerable systems and processes operating outside the information
      resource management area

      Identify and assess year 2000 vulnerable systems andprocesses outside the information
      resource management area, including telephone and network switching equipment, and
      building infrastructuresystems. Develop a separateplanfor their renovation.

GAO/AIMD-10.1.14 Year 2000 Computing Crisis                                                       13
3.0 Renovation

The renovation--conversion, replacement, or retirement--phase involves making and documenting
software and hardware changes, developing replacement systems, and decommissioning eliminated
systems. Renovation involves conversion of an existing application; replacement deals with the
development of a new application; elimination focuses on the retirement or decommissioning of an
existing application or system component. In all three cases, the process must also consider the
complex interdependencies among applications, hardware platforms, databases, and the internal
and external interfaces.

All changes to the information systems and their components must be made under configuration
management to ensure that changes are adequately documented and coordinated throughout the
agency. Equally important is the need for each agency to assess dependencies and to communicate
all changes to the information systems to internal and external users.

3.1. Convert selected applications, databases, archives, and related system components

    In converting applicationsystems, consider changes in operatingsystems, compilers,
    utilities, domain-specificprogramproducts, and commercial database management

3.2. Develop data bridges and filters

    Ensure thlean
               all internaland external
                                    xeatdata sources meet the year 2000 date standardsof the
    converted or replaced systems. Develop bridges orfilters to convert non-conforming data.

 .33.Replace selected applications, platforms, database management systems, operating systems,
     compilers, utilities, and other commercial off-the-shelf (COTS) software

    Ensure that replacementproducts are year 2000 compliant, including their ability to
    properly handle the leap year adjustments. Direct contractspecialistand legal staffto
    review contracts and warranties.

GAO/AIMD-10.1.14 Year 2000 Computing Crisis                                                      14
3.4. Document code and system changes

    Implement and use configuration managementprocedures to ensure that all changes to
    information systems and their components are properly documented and managed.

3.5. Schedule unit, integration, and system tests

    Schedule unit, integration, and system tests following the conversion of individual
    applicationand software modules. Coordinatescheduling with otherproject teams to
    ensure that all components--includingdata bridges orfilters--areavailablefor testing.

3.6. Eliminate selected applications, platforms, database management systems, operating
     systems, utilities, and COTS software

    Prepareto eliminate replacedapplications,platforms, databasemanagement systems,
    operatingsystems, utilities, and COTS software upon the successful completion of

3.7. Communicate changes to information systems to all internal and external users

    Communicate changes to the agency's information systems and components, and
    specifically all changes to dateformatsfordata exchangedwith other systems or external
    organizations. Document changes through the configurationmanagementprocess.

3.8. Track the conversion and replacement process and collect project metrics

    Track the conversion and replacementprojects and collect and use project metrics to
    manage cost and schedule.

3.9. Share information among year 2000 projects and disseminate lessons learned and best

    Ensure that project staffs understandthe need to collect and disseminate information on
    lessons learnedand bestpractices. Develop disseminationstrategy and tools, such as
    intranetweb sites and newsletters.

GAO/AIMD-10.1.14 Year 2000 Computing Crisis                                                   15
4.0 Validation

We expect that agencies may need over a year to adequately validate and test converted or replaced
systems for year 2000 compliance, and that the testing and validation process may consume over
half of the year 2000 program resources and budget. The length of the validation and test phase
and its cost are driven by the complexity inherent in the year 2000 problem. Agencies must not
only test year 2000 compliance of individual applications, but also the complex interactions
between scores of converted or replaced computer platforms, operating systems, utilities,
applications, databases, and interfaces. Moreover, in some instances, agencies may not be able to
shut down their production systems for testing, and may thus have to operate parallel systems
implemented on a year 2000 test facility.

All converted or replaced system components must be thoroughly validated and tested to (1)
uncover errors introduced during the renovation phase, (2) validate year 2000 compliance, and (3)
verify operational readiness. The testing should account for application, database
interdependencies, and interfaces. The testing should take place in a realistic test environment. A
year 2000 test facility may be required to ensure adequate testing of licensed software and
converted applications while preventing the contamination or the corruption of operational
information systems and related databases. Agencies should assess their testing procedures and
tools to ensure that all converted system components meet quality standards and are year 2000

4.1. For each converted or replaced application or system component, develop and document test
     and compliance plans and schedules

     Establish a compliance validationprocess. Most suppliers of COTS software do not
     disclose their source code or the internal logic of theirproducts, therefore, testing should
     be complemented by a careful review of warrantiesand.orguarantees.

4.2. Develop a strategy for managing the testing of contractor-converted systems

     In many instances, the agency will contractforthe conversion of selected systems and their
     components. The contractconversion must be closely managed to ensure that the
     contractorfollows the agency's year 2000 conversion standards. In addition, the agency
     must ensure that the contractor-convertedsystems are adequately tested.

GAO/AIMD-10.1.14 Year 2000 Computing Crisis                                                         16
4.3. Implement year 2000 test facility

     Testing the converted or replacedsystems and their componentsfor year 2000 compliance
     will likely requirean isolatedtestfacility capable of simulatingyear 2000 requirements.
     The testfacility should provide sufficient disk storagefor large test databasesand multiple
     versions of the applicationsoftware.

4.4. Implement automated test tools and test scripts

     The use of computer-aidedsoftware testing tools and test scripts has the potential to
     significantly reduce the testing and validation burden. Test management tools may help in
     the preparationand management of test data, in the automation of the comparison of test
     results, in scheduling and incidenttracking, and in managing test documentation.

4.5. Perform unit, integration, and system testing

     Using a phased approach,perform unit, integration, and system testing. Use selected
     testing techniques to ensure that the converted or replaced systems and accompanying
     components arefunctionally correct andyear 2000 compliant. The testing should include
     regression,performance, stress, andforwardand backward time testing.

4.6. Define, collect, and use test metrics to manage the testing and validation process

4.7. Initiate acceptance testing

     Acceptance testing is thefinal stage of the multiphase testing and validationprocess.
     During this phase, the entire information system--including data interfaces--istested with
     operationaldata. In general, we expect that the acceptancetesting will be done on the year
     2000 testfacility with duplicate databases to avoid risk to the production systems and the
     potentialcontamination of data.

GAO/AIMD-10.1.14 Year 2000 Computing Crisis                                                      17
5.0 Implementation

Implementation of year 2000 compliant systems and their components requires extensive
integration and acceptance testing to ensure that all converted or replaced system components
perform adequately in a heterogeneous operating environment. Because of the scope and
complexity of the year 2000 conversion changes, integration, acceptance, and implementation will
likely be a lengthy and costly process.

Once converted or replaced and subsequently tested, year 2000 compliant applications and system
components must be implemented. Since not all system components will be converted or replaced
simultaneously, agencies may be expected to operate in a heterogeneous computing environment
comprised of a mix of year 2000 compliant and non-compliant applications and system
components. The reintegration of the year 2000 compliant applications and components into the
agency's production environment must be carefully coordinated to account for system
interdependencies. Parallel processing--where the old and the converted systems are run
concurrently--may be needed to reduce risk.

5.1. Define transition environment and procedures

     The transitionfrom the current environment to year 2000 compliant systems will be difficult
     and complex. First,some key components of the agency systems--year 2000 compliant
     databases,operating systems, utilities, and other COTS products--may not be available
     until late 1998 or early 1999. Second, externaldata suppliers may not plan to complete
     their conversion and testing until 1999. Third, the testing, validation, and correction
     processes may take much of 1999. Fourth, replacementsystems may not be readyfor
     testing until late 1999. As a result, agencies may be forced to operate--at leastfor a time--
     parallelsystems and databases.

5.2. Develop implementation schedule

     The year 2000 implementation schedule must not only deal with uncertaintiescommon to all
     large system development efforts, but also should indicate all major milestones and the
     criticalpathfor the completion of the year 2000 program.

GAO/AIMD-10.1.14 Year 2000 ComdoutineCrisis                                                        18
5.3. Resolve data exchange issues and interagency concerns, including ensuring that

      *    all outside data exchange entities are notified
      *    data bridges and filters are ready to handle non-conforming data
      *    contingency plans and procedures are in place if data are not received from an external
      *    contingency plans and procedures are in place if invalid data are received from an
           external source
      *    the validation process is in place for incoming external data

    All data issues and interagencyconcerns must be resolvedpriorto acceptance testing and
    implementation. Bridges andfilters should be in place to handle non-conforming data
    receivedfrom external sources, and contingency plans and proceduresshould be in place to
    handle no data or bad data situations.

5.4. Deal with database and archive conversion

     Because the conversion of large databasesfrom 2-digit to 4-digit yearfields is a time
     consuming effort, agencies may consider off-site conversion alternatives.

5.5. Complete acceptance testing

     In general, formal testing uncovers about 80-90 percent of software errors, with the
     remaining 10-20 percent of errors discovered during operations. Acceptance testing should
     be completed no later than Fall of 1999, to allow sufficient time for the correction of
     software errorsdiscoveredfollowing implementation.

5.6. Develop contingency plans

     Unlike routine system development or maintenance efforts where schedule slippagesare
     non-fatal--and common--the year 2000 program must be completed on time. Agencies
     should develop realistic contingency plans--includingthe development and activation of
     manual or contractprocedures--to ensure the continuity of their core businessprocesses.

5.7. Update or develop disaster recovery plans

     All year 2000 compliant systems--including the converted and replaced systems and related
     databases--shouldhave disasterrecovery plansfor the restorationof operationsand data
     in case of extended outage, sabotage, or naturaldisaster.

5.8. Implement converted and replaced systems

     Reintegrate the converted and replacedsystems and relateddatabasesinto the production

GAO/AIMD-10.1.14 Year 2000 Computing Crisis                                                      19
Program and Project Management

The year 2000 program is likely the largest and most complex system conversion effort ever
undertaken by many federal agencies. It requires the disciplined and coordinated application of
scarce resources to an enterprise-wide system conversion effort that must be completed by a fixed
date. To succeed, agencies must manage the year 2000 program as a large system development

A. Establish year 2000 program management structure

    *    appoint a year 2000 program manager and establish a year 2000 program team
    *    identify technical and management representatives from each core business area

    The agency's year 2000 program--headedby a program manager--shouldbe adequately
    staffed to ensure the successful completion of the assessment phase. In addition to technical
    skills, the program staff should be able to track the cost and schedule for individualyear
    2000 projects, and to coordinate the agency's year 2000 activities with other organizations.

B. Based on the assessment of the agency's program management capability performed during the
    awareness phase, ensure that necessary enterprise-wide program management policies and
    procedures are in place, including

    *    configuration management
    *    quality assurance
    *    risk management
    *    project scheduling and tracking
    *    metrics
    *    budgeting

    Agencies may consider establishingan enterprise-level competency center to train staff and
    tofoster the use of proven industry system development andprogrammanagement practices.

C. Monitor year 2000 projects, and ensure that projects follow required policies and procedures
   for configuration management, project scheduling and tracking, and metrics.

    Agencies may considersubjecting their year 2000 program to an independent verification
    and validation effort. This verification and validation may be performed by the agency's
    quality assurancestaff complemented by internalauditors.

GAO/AIMD-10.1.14 Year 2000 Computing Crisis                                                         20
Year 2000 Program Assessment Checklist

Agency Year 2000 Program Phase or Activity

ol Awareness                                      O Validation
O Assessment                                      O Implementation
O Renovation                                      O Program Management


o    Has the agency defined and documented the potential impact of the year 2000 problem?

O    Has the agency conducted a year 2000 awareness campaign?

o    Has the agency assessed the adequacy of its program management policies, capabilities, and
     practices, including configuration management, program and project management, and
     quality assurance?

o    Has the agency developed and documented a year 2000 strategy?

O    Is the year 2000 strategy supported by executive management?

       The agency has

       D3 year2000 policy directive
       0 year2000 program charter

O    Has the agency established an executive management council or committee to guide the year
     2000 program?

O    Has a program manager been appointed and a year 2000 program office been established
     and staffed?

o    Has the agency identified technical and management points of contacts in core business


O    Has the agency defined year 2000 compliance?

0    Has the agency identified core business areas and processes and assessed the potential
     impact of year 2000-induced failures for each area and process?

GAO/AIMD-10.1.14 Year 2000 Computing Crisis                                                   21
O     Has the agency assessed the severity of the impact of potential year 2000-induced failures
      for each core business areas and associated processes?

U    Has the agency conducted a comprehensive enterprise-wide inventory of its information

       The agency has

       Q system inventory listing components and interfacesfor each system
       Q comprehensive plan to identify and eliminate obsolete code

O    Has the agency developed a comprehensive automated system portfolio?

       The agency's portfolio identifies

        O links to core business areasorprocesses
        O platforms, languages, databasemanagement systems
        O operatingsystem software and utilities
        O telecommunications
        O internaland external interfaces
        0 owners
        O the availabilityand adequacy of source code & associateddocumentation

E Has the agency analyzed its system portfolio and identified for each system

        O non-repairableitems (lack of source code or documentation)
        O conversion or replacement resourcesrequiredfor each platform, application,
          database management system, archive, utility, or interface

El Has the agency prioritized its system conversion and replacement program?

    The agency'sprioritizationprocess includes

        O ranking by business impact
        0 ranking by anticipatedfailure date
        O identificationof applications,databases, archives, and interfaces that cannot be
          convenrted because of resource and time constraints

 l Has the agency established year 2000 project teams for business areas and major systems?

GAO/AIMD-10.1.14 Year 2000 Computing Crisis                                                        22
U Has the agency developed a year 2000 program plan?

    The agency'sprogram plan includes

        O     schedulesfor all tasks andphases
        O     master conversion and replacement schedule
          l   assessment and selection of outsourcingoptions
        0     assignment of conversion or replacementprojects to project teams
        O     risk assessment
        0     contingency plansfor all systems

El Has the agency identified and mobilized required resources and capabilities?

E Has the agency developed validation strategies and testing plans for all converted or replaced
  systems and their components?

E Has the agency analyzed and identified requirements for a year 2000 test facility?

E Has the agency identified and acquired year 2000 tools?

E Has the agency considered implementation scheduling issues?

    The agency's programplan addresses

        O where conversion will take place (data center or off-site location)
        O time needed to place converted systems into production
        O conversion of backup or archived data

El Has the agency addressed interface and data exchange issues?

    The agency has

        D analyzed dependencies on data provided by other organizations
        O contactedall entities with whom it exchanges data
        D identified the needfor data bridges orfilters
        D made contingency plans if no data are receivedfrom external sources
        O made plans to determine that incoming data are valid
        D developed contingency plans to handle invalid data

0   Has the agency developed contingency plans for critical systems and activities?

GAO/AIMD-10.1.14 Year 2000 Computing Crisis                                                        23
Q Does the impact assessment document identify year 2000 vulnerable systems and processes
    outside the traditional information resource management area that may affect the agency's

    The assessment document addressesthe impact of potentialyear 2000 inducedfailure of

        D telecommunication systems, including telephone and data networks switching
        C building infrastructure


El Is the agency meeting its budget and schedule in the conversion of targeted applications,
   platforms, databases, archives, or interfaces?

E   Is the agency meeting its budget and schedule in developing bridges and filters to handle non-
    conforming data?

E Is the agency meeting its budget: and schedule in the replacement of targeted applications and
  system components?

 ) Is the agency documenting all code and system modifications and using configuration
   management to control changes'?

E Is the agency scheduling unit, integration, and system tests?

O Is the agency meeting its budget and schedule in eliminating targeted applications and system

C Is the agency communicating the changes to its information systems to all internal and external

CL Is the agency tracking the conversion and replacement process and collecting and using project
   metrics to manage the conversion and replacement process?

L   Is the agency sharing information among year 2000 projects?

    The agency is disseminating

        Cl "lessons learned"
        Q bestpractices


C Has the agency developed and documented test and validation plans for each converted or
  replaced application or system component?

GAO/AIMD-10.1.14 Year 2000 Computing Crisis                                                      24
o   Has the agency developed and documented a strategy for testing contractor-converted or
    replaced applications or system components?

El Has the agency implemented a year 2000 test facility?

O Has the agency implemented automated test tools and scripts?

O Has the agency performed unit, integration, and system tests on each converted or replaced

    The agency's testing procedures include the following types of tests

          ) regression
        O performance
        C stress
        O forward and backward time

Elo Is the agency tracking the testing and validation process and collecting and using test metrics to
    manage the testing activities?

ElO Has the agency initiated acceptance tests?


C Has the agency defined its transition environment and procedures?

E Has the agency developed and documented a schedule for the implementation of all converted
  or replaced applications and system components?

E Has the agency resolved data exchange issues and interagency concerns?

E Has the agency dealt with database and archive conversion?

O Has the agency completed acceptance testing?

E Has the agency developed contingency plans?

O Has the agency updated or developed disaster recovery plans?

E Has the agency reintegrate the converted and replaced systems and related databases into the
  production environment?

GAO/AIMD-10.1.14 Year 2000 Computing Crisis                                                         25
Program and Project Managemnent

El Has the agency established a year 2000 program management structure?

    The agency has

        0   appointeda year 2000 program managerand established a year 2000 program team
        0   identified technical and management representativesfrom each core business area

El Based on the assessment of its program management capabilities, has the agency developed
   and implemented policies, guidelines, and procedures to manage a major program?

    The agency's policies, guidelines, and process include

        E configurationmanagement
        O quality assurance
        O risk management
        D project scheduling and tracking
        E1 metrics
        C budgeting

El Is the agency monitoring the year 2000 program to ensure that projects are following required
   policies and procedures for configuration management, project scheduling and tracking, and

GAO/AIMD-10.1.14 Year 2000 Computing Crisis                                                    26
Selected Year 2000 Resources

There are many readily accessible sources of useful information on the year 2000 problem, with
many government and industry organizations establishing year 2000 web sites. These sites provide
information about year 2000 compliant products, tools, and practices.

Best Practices

The Program Manager's Guide to Software Acquisition Best Practices (Version 1.1), Software
Acquisition Best Practice Initiative, Department of Defense (Undated).

A framework of bestpracticesfocusedon effective project management, software defect
detection, andproject risk reduction. The guide and several companion publicationsare
available at http://spmn.comZ.

Key Practices of the Capability Maturity Model. Version 1.1, Software Engineering Institute,
Carnegie Mellon University, February 1993.

A set of key practicesforplanning, engineering, and managing software development and
maintenance. Discussed practices include configuration management, software quality
assurance, project tracking and oversight, andprojectplanning. More information on the
capability model may be found at http://www.sei cmu.edu/technology/cmm.htmL

Year 2000 Interagencv Committee Best Practices, Year 2000 Interagency Committee, Best
Practices Subcommittee, 1997 (draft)

A compendium of best practicesfocused on a year 2000programpresented in aframework of
awareness, assessment, renovation, validation, and implementation phases. Available at

The Year 2000 and 2-Digit Dates: A Guide for Planning and Implementation, 6th edition,
International Business Machines Corporation, September 1996.

The guide provides information on the cause and scope of using dates representedby 2-digit
years, problems with programs using 2-digit-yeardata, the best technique for reformatting the
year-date notations, migrationstrategies to a year 2000-ready environment, testing techniques,
and a list of IBM tools. Available at http:l//www.software.ibnmcomlyear2000/resource.html.

Selected Year 2000 Web Sites

Federal Year 2000 Web Sites

O Year 2000 Interagency Committee

O Armnny

GAO/AIMD-10.1.14 Year 2000 Computing Crisis                                                    27
D Air Force
  http:/rinfosphere.safb.afanil/-j wid/fadl/worldly2k.htm

O Navy

O Marine Corps

O Defense Information Systems Agency

General Year 2000 Sites

O The Year 2000 Information Center

U Year 2000 Technical Audit Center

0   Year 2000 Information Network
    http://webidirect. com/%7Embsprog/y2kcon.html

O The Year 2000 Resource

o   CIO Year 2000 Resource Center

0   The National Bulletin Board For Year 2000

Year 2000 Products. Tools, and Patches

C Defense Information Systems Agency Tools

0   Air Force Software Technology Support Center

D Army Tools
  http.:/www.army.mil/army-y2kltools/tools- l-.htm

0 RighTime PC Patches

GAO/AIMD-10.1.14 Year 2000 Computing Crisis                 28

The definitions in this glossary were developed by the project staff or were drawn from other
sources, including the Computer Dictionary: The Comprehensive Standard For Business, School,
Library, and Home, Microsoft Press, Washington, DC, 1991; The year 2000 Resource Book,
Management Support Technology Corp., Framingham, Massachusetts, 1996; The Year 2000 and
2-Digit Dates: A Guide for Planning and Implementation, 5th Edition, International Business
Machines Corporation, 1996; and the "Free On-line Dictionary of Computing," Denis Howe,
1996. <http://wombat.doc.ic.ac.uk/> (November 11, 1996).

Application            A computer program designed to help people perform a certain type of
                       work. Depending on the work for which it was designed, an application
                       can manipulate text, numbers, graphics, or a combination of these

Architecture           A description of all functional activities to be performed to achieve the
                       desired mission, the system elements needed to perform the functions, and
                       the designation of performance levels of those system elements. An
                       architecture also includes information on the technologies, interfaces, and
                       location of functions and is considered an evolving description of an
                       approach to achieving a desired mission.

Business                A description of the systems, databases, and interactions between
architecture            systems and databases that will be needed to fulfill business requirements.

Business area           A grouping of business functions and processes focused on the production
                        of specific outputs.

Business function       A group of logically related tasks that are performed together to
                        accomplish an objective.

Business plan           An action plan that the enterprise will follow on a short-term and/or long-
                        term basis. It specifies the strategic and tactical objectives of the company
                        over a period of time. The plan, therefore, is time dependent; it will
                        change with the enterprise. Although a business plan is usually written in
                        a style unique to a specific enterprise, it should concisely describe "what"
                        is planned, "why" it is planned, "when" it will be implemented, by "who,"
                        and "how" it will be gauged. The architects of the plan are typically the
                        principals of the enterprise.

Component               A single resource with defined characteristics. The component concept is
                        used in defining precise specifications for testing the validity of various
                        resources. These components are also defined by their relationship to
                        other components.

GAO/AIMD-10.1.14 Year 2000 Computing Crisis                                                           29
Configuration         The continuous control of changes made to a system's hardware,
management            software, and documentation throughout the development and operational
                      life of the system.

Contingency plan      A plan for responding to the loss of system use due to a disaster such as a
                      flood, fire, computer virus, or major software failure. The plan contains
                      procedures for emergency response, backup, and post-disaster recovery.

Conversion            The process of making changes to databases or source code.

Database              An aggregation of data; a file consisting of a number of records or tables,
                      each of which is constructed of files of a particular type, together with a
                      collection of operations that facilitate searching, sorting, recombination,
                      and similar operations.

Data dictionary       A set of data descriptions that can be shared by several applications.

Debug                 With software, to detect, locate, and correct logical or syntactical errors in
                      a computer program.

Defect                A problem or "bug", that if not removed, could cause a program to either
                      produce erroneous results or otherwise fail.

Information           A description of the enterprise in terms of its business activity,
architecture          business information, and their interaction.

Infrastructure        The computer and communication hardware, software, databases, people,
                      and policies supporting the enterprise's information management

Integration testing   Testing to determine that the related information system components
                      perform to specification

Interface             A boundary across which two systems communicate. An interface might
                      be a hardware connector used to link to other devices, or it might be a
                      convention used to allow communication between two software systems.

Inventory             In the context of a year 2000 program, the process of determining the
                      components that comprise the agency's systems portfolio. The inventory
                      should include all applications, databases, files, and related system
                      components that will require inspection to locate date data and related date

Line of code          A single computer program command, declaration, or instruction.
                      Program size is often measured in lines of code.

GAO/AIMD-10.1.14 Year 2000 Computing Crisis                                                      30
Metrics               Means by which software engineers measure and predict aspects of
                      processes, resources, and products that are relevant to the software
                      engineering activity.

Mission-critical      A system supporting a core business activity or process.

Object code           The machine code generated by a source code language processor such as
                      an assembler or compiler. A file of object code may be immediately
                      executable or it may require linking with other object code files, e.g.
                      libraries, to produce a complete executable program.

Operating system      The software which schedules tasks, allocates storage, handles the
                      interface to peripheral hardware, and presents a default interface to the
                      user when no application program is running.

Outsourcing           Paying another company to provide services which an organization might
                      otherwise have performed itself, e.g. software development.

Parallel processing   The simultaneous use of more than one computer to solve a problem.

Platform              The foundation technology of a computer system. Typically, a specific
                      combination of hardware and operating system.

Portfolio             In the context of the year 2000 program, an inventory--preferably
                      automated--of an agency's information systems and their components
                      grouped by business areas.

Production            The system environment where the agency performs its routine
environment           information processing activities.

Quality assurance     All the planned and systematic actions necessary to provide adequate
                      confidence that a product or service will satisfy given requirements for

Regression testing    Selective retesting to detect faults introduced during modification of a

Risk assessment       A continuous process performed during all phases of system development
                      to provide an estimate of the damage, loss, or harm that could result from
                      a failure to successfully develop individual system components.

Risk management       A management approach designed to reduce risks inherent to system

Source code           The form in which a computer program is written by the programmer.
                      Source code is written in a programming language which is then compiled
                      into object code or machine code or executed by an interpreter.

GAO/AIMD-10.1.14 Year 2000 Computing Crisis                                                       31
Standard              In computing, a set of detailed technical guidelines used as a means of
                      establishing uniformity in an area of hardware or software development.

Strategic IRM plan    A long-term, high-level plan that defines a systematic way of how the
                      agency will use information technology to effectively accomplish the
                      agency's missions, goals, and objectives.

Strategic plan        A long-term, high-level plan that identifies broad business goals and
                      provides a roadmap for their achievement.

System testing        Testing to determine that the results generated by the enterprise's
                      information systems and their components are accurate and the systems
                      perform to specification.

Test                  The process of exercising a product to identify differences between
                      expected and actual behavior.

Test facility         A computer system isolated from the production environment dedicated to
                      the testing and validation of applications and system components.

Unit testing          Testing to determine that individual program modules perform to

Utilities             Computer programs designed to perform maintenance work on the system
                      or on system components--for example, a storage backup program, a disk
                      or file recovery program, or a resource editor.

Validation            The process of evaluating a system or component during or at the end of
                      the development process to determine whether it satisfies specified

Year 2000 compliant   Information systems able to accurately process date data--including, but
                      not limited to, calculating, comparing, and sequencing--from, into, and
                      between the twentieth and twenty-first centuries, including leap year

Year 2000 problem     The potential problems and its variations that might be encountered in any
                      level of computer hardware and software from microcode to application
                      programs, files, and databases that need to correctly interpret year-date
                      data represented in 2-digit-year format.

GAO/AIMD-10.1.14, Year 2000 Computing Crisis                                                     32