United States General Accounting Office Accounting and Information Management Division GAO September 1997 Year 2000 Computing Crisis: An Assessment Guide GAO/AIMD-10.1.14 The year 2000 is not rocket science, but it is the largest project ever to be undertaken by the IT organization. The complexity of the project is not in the solution but rather in the size and scope of the project itself. This means that the year 2000 requires “world class” project management. Kevin Schick Gartner Group April 16, 1996 GAO/AIMD-10.1.14 Year 2000 Computing Crisis 1 Preface ______________________________________________________________________________ At 12:01 on New Year's morning of the year 2000, many computer systems worldwide could malfunction or produce incorrect information simply because the date has changed. Unless corrected, the impact of these failures could be widespread and costly. For example: • IRS' tax systems could be unable to process returns, which in turn could jeopardize the collection of revenue and the entire tax processing system. • Payments to veterans with service-connected disabilities could be severely delayed because Veterans Affairs' compensation and pension system either halts or produces checks that are so erroneous that the system must be shut down and the checks processed manually. • The Social Security Administration's disability insurance process could experience major disruptions because the interface with various state systems fails, thereby causing delays and interruptions in disability payments to citizens. • Federal systems used to track student education loans could produce erroneous information on loan status, such as indicating that an unpaid loan had been satisfied. The Year 2000 problem is rooted in the way dates are recorded and computed in many computer systems. For the past several decades, systems have typically used two digits to represent the year, such as "97" representing 1997, in order to conserve on electronic data storage and reduce operating costs. With this two-digit format, however, the Year 2000 is indistinguishable from 1900, 2001 from 1901, and so on. As a result of this ambiguity, system or application programs that use dates to perform calculations, comparisons, or sorting may generate incorrect results when working with years after 1999. Many government computer systems were originally designed and developed 20 to 25 years ago, are poorly documented, and use a wide variety of computer languages--many of which are old or obsolete. The systems consist of tens or hundreds of computer programs, each with thousands, tens of thousands, or even millions of lines of code, which must be examined for date problems. Moreover, the government's computer systems, like private sector systems, have numerous components--hardware, software stored in read-only-memory, operating systems, communications applications, and database software--that are affected by the date problem. Correcting the problem and achieving Year 2000 compliance--defined as the ability of information systems to accurately process date data from, into, and between the twentieth and twenty-first centuries, including leap year calculations--will not be easy. Every federal agency is at risk of widespread system failures. Because converting systems to a 4-digit year will be a massive undertaking for large systems, agencies must start now to renovate their systems. They need to identify their inventories of mission-critical computer systems, develop conversion strategies and plans, and dedicate sufficient resources to converting and adequately testing their computer systems and programs before January 1, 2000. This guide provides a framework and a checklist for assessing the readiness of federal agencies to achieve Year 2000 compliance. It provides information on the scope of the challenge and offers a GAO/AIMD-10.1.14 Year 2000 Computing Crisis 2 structured approach for reviewing the adequacy of agency planning and management of the Year 2000 program. Because each agency is different, there is no single, cookie cutter approach for solving the Year 2000 problem. Some agencies are highly centralized, while others operate in a highly decentralized information resource environment. This guide addresses issues that will be common to most Year 2000 programs; however, each agency must tailor its Year 2000 program in response to its unique needs. The guide is divided into five phases supported by program and project management activities: • Awareness • Assessment • Renovation • Validation • Implementation An electronic version of this guide is available from GAO’s World Wide Web server at the following Internet address: <http://www.gao.gov>. If you have any questions about the guide or the Year 2000 process outlined here, please contact us, or Mirko J. Dolak, Technical Assistant Director, at (202) 512-6362. We can also be reached by e-mail at firstname.lastname@example.org, email@example.com, and firstname.lastname@example.org. Joel C. Willemssen William S. Franklin Director Director Information Resources Management Information Systems Methods and Support GAO/AIMD-10.1.14 Year 2000 Computing Crisis 3 Contents ______________________________________________________________________________ Preface 2 ______________________________________________________________________________ Year 2000 Conversion Model: Structured Approach and 5 Rigorous Project Management Can Decrease Risks ______________________________________________________________________________ Awareness 7 ______________________________________________________________________________ Assessment 10 ______________________________________________________________________________ Renovation 14 ______________________________________________________________________________ Validation 16 ______________________________________________________________________________ Implementation 18 ______________________________________________________________________________ Program and Project Management 20 ______________________________________________________________________________ Year 2000 Program Assessment Checklist 22 ______________________________________________________________________________ Selected Year 2000 Resources 28 ________________________________________________________________________ Glossary 30 GAO/AIMD-10.1.14 Year 2000 Computing Crisis 4 Year 2000 Conversion Model: Structured Approach and Rigorous Program Management Can Decrease Risks ______________________________________________________________________________ The Year 2000 date conversion poses a global challenge to the information technology industry. Every organization, whether federal or private, must ensure that its information systems are fully Year 2000 compliant well before December 31, 1999. While the Year 2000 problem is not technically challenging, it is massive and complex. For many agencies, the Year 2000 conversion program will be the largest project ever to be managed and implemented by their information resource management organizations. This guide presents a structured approach and a checklist to aid federal agencies in planning, managing, and evaluating their Year 2000 programs. The guide draws on the work of the CIO Council Subcommittee on Year 2000 and incorporates guidance and practices identified by leading organizations in the information technology industry. The guide describes five phases--supported by program and project management--with each phase representing a major Year 2000 program activity or segment. Year 2000 Conversion Model Define the Year 2000 problem and gain executive level Awareness support and sponsorship. Establish Year 2000 program team and develop an overall strategy. Ensure that everyone in the organization is fully aware of the issue. Assess the Year 2000 impact on the enterprise. Assessment Identify core business areas and processes, inventory and analyze systems supporting the core business areas, and prioritize their conversion or replacement. Develop contingency plans to handle data exchange issues, lack of data, and bad data. Identify and secure the necessary resources. Program & Convert, replace, or eliminate selected platforms, Project Renovation applications, databases, and utilities. Modify Management interfaces. Test, verify, and validate converted or replaced Validation platforms, applications, databases, and utilities. Test the performance, functionality, and integration of converted or replaced platforms, applications, databases, utilities, and interfaces in an operational environment. Implementation Implement converted or replaced platforms, applications, databases, utilities, and interfaces. Implement data exchange contingency plans, if necessary. Plan and manage the Year 2000 program as a single large information system development effort. Promulgate and enforce good management practices on the program and project levels. GAO/AIMD-10.1.14 Year 2000 Computing Crisis 5 Immovable Deadline and Fixed Schedule ______________________________________________________________________________ Time is running out. Renovation work should be done by mid-1998 to allow sufficient time for validation and implementation. However, only half of the 24 departments and major agencies reported that they completed the assessment phase by the end of August 1997. The 12 agencies that have not completed the assessment phase account for about 70 percent of the estimated federal cost of achieving Year 2000 compliance. These agencies must act quickly to complete the assessment phase and begin to renovate their mission-critical systems now. Year 2000 Schedule J F M A M J 1996 J Awareness A S O N D J F M A Assessment M 1997 J J A S O N D Renovation J F M A M J 1998 J A S O N D J F Validation M A & M J Implementation 1999 J A S O N D GAO/AIMD-10.1.14 Year 2000 Computing Crisis 6 1.0 Awareness ______________________________________________________________________________ It is essential that executive management be fully aware of the Year 2000 problem and its potential impact on the enterprise and its customers. It is the responsibility of the chief information officer to provide the leadership in defining and explaining the importance of achieving Year 2000 compliance, selecting the overall approach for structuring the agency’s Year 2000 program, assessing the adequacy of the existing information resource management infrastructure to adequately support the Year 2000 efforts, and mobilizing needed resources. Key Processes 1.1. Define the Year 2000 problem and its potential impact on the enterprise 1.2. Conduct Year 2000 awareness campaign 1.3. Assess the adequacy of the agency’s program management capabilities 1.4. Develop Year 2000 strategy 1.5. Obtain support from executive management 1.6. Establish Year 2000 executive management council 1.7. Appoint Year 2000 program manager and establish a Year 2000 program office 1.8. Identify technical and management contacts in core business areas 1.1. Define the Year 2000 problem and its potential impact on the enterprise Developing and publishing a high-level assessment of the Year 2000 issue provides executive management and staff with a high-level overview of the potential impact of the Year 2000 problem on the enterprise. 1.2. Conduct a Year 2000 awareness campaign A Year 2000 awareness campaign is an important first step to raise the awareness of executive management and line staff about the potential impact of the Year 2000 problem on the agency’s operations. 1.3. Assess the adequacy of the agency’s program management capabilities, including • policies, guidelines, and processes for program and project management, configuration management, quality assurance, and risk management • staffing levels and skill mix The ability to successfully manage the Year 2000 program will depend on the degree to which the agency has institutionalized key system development and program management practices and on its experience in managing large-scale software conversion or system development efforts. With only a few activities within federal agencies operating above level 1 on the Software Engineering Institute’s Capability Maturity Model,1 most 1 “ProcessMaturity Profile of the Software Community, 1996 Update,” Software Engineering Institute, April 1996. GAO/AIMD-10.1.14 Year 2000 Computing Crisis 7 information resource management organizations lack the basic policies, tools, and practices necessary to successfully manage a large-scale Year 2000 program. While there may not be enough time to achieve a higher maturity level, agencies should assess, and upgrade, if needed, their information resource management capabilities. Agencies should consider the establishment of an enterprise competency center to provide training and to foster adherence to proven industry system development and program management practices. Agencies also need to consider soliciting assistance from organizational entities experienced in performing or managing major software conversions. 1.4. Develop and document a high-level Year 2000 strategy A high-level Year 2000 strategy provides the agency’s executive management with a roadmap for achieving Year 2000 compliance. The strategy should discuss key Year 2000 issues, including the program’s management structure, program metrics and reporting requirements, the mix of enterprise-wide solutions, and initial cost and schedule estimates. 1.5. Obtain and formalize executive management support through issuance of • Year 2000 policy directive • Year 2000 program charter The management support for the agency’s Year 2000 strategy should be formalized by the issuance of a Year 2000 policy directive and/or Year 2000 program charter. Without such support, information resource managers may not be able to mobilize adequate resources to implement the strategy and to interact with other organizations and data sources. 1.6. Establish Year 2000 executive management council A committee or a council needs to be established within the agency to continually coordinate with the programmatic and functional area managers on priorities and potential mission impact if certain processes and systems malfunction. A process for quick conflict resolution on priorities between programmatic and functional areas is also needed. 1.7. Appoint a Year 2000 program manager and establish an agency-level Year 2000 program office It is essential that agencies appoint a Year 2000 program manager and establish an agency-level program office to manage and coordinate the enterprise’s Year 2000 program activities. The solutions of the Year 2000 problem extend beyond simple software conversion, hardware upgrades, and database restructuring. The problem--and the solutions--involve a wide range of dependencies among information systems; the need to centrally develop or acquire conversion and validation standards, inspection, conversion, and testing tools; the need to coordinate the conversion of cross-boundary information systems and their components; the need to establish priorities; and the need to reallocate resources as needed. GAO/AIMD-10.1.14 Year 2000 Computing Crisis 8 1.8. Identify technical and management points of contact in core business areas A Year 2000 program should not be viewed as a system development or maintenance effort managed by the information resource management organization, but rather as an enterprise-wide effort requiring the input and cooperation of all organizational units. Thus, it is important that the technical and management staff of the core business areas work closely with the Year 2000 project teams in the assessment and testing process. GAO/AIMD-10.1.14 Year 2000 Computing Crisis 9 2.0 Assessment ______________________________________________________________________________ Federal agencies may not have enough resources, skill, or time to convert or replace all of their information systems. Agencies must determine what systems are mission-critical and must be converted or replaced, what systems support important functions and should be converted or replaced, and what systems support marginal functions, and may be converted or replaced later. The Year 2000 problem is not just an information technology problem, but is primarily a business problem. Thus, the process of identifying and ranking information systems should not be limited to a simple inventory of applications and platforms, but must include assessments of the impact of information systems’ failures on the agency’s core business areas and processes. The assessment should also include systems using information technology which operate outside the traditional information resource area, including building infrastructure systems and telephone switching equipment. Key Processes 2.1. Define Year 2000 compliance 2.2. Focus on core business areas and processes and develop a Year 2000 assessment document 2.3. Assess the severity of an impact of Year 2000-induced failures 2.4. Conduct enterprise-wide inventory of information systems for each business area 2.5. Develop a comprehensive automated system portfolio 2.6. Analyze system portfolio 2.7. Prioritize systems and components to be converted or replaced 2.8. Establish Year 2000 project teams for business areas and major systems 2.9. Develop Year 2000 program plan 2.10. Identify, prioritize, and mobilize needed resources 2.11. Develop validation strategies, testing plans, and scripts 2.12. Define requirements for Year 2000 test facility 2.13. Identify and acquire Year 2000 tools 2.14. Address implementation schedule issues 2.15. Address interface and data exchange issues 2.16. Initiate the development of contingency plans for mission-critical systems 2.17. Identify Year 2000 vulnerable systems and processes operating outside the information resource management area 2.1. Define Year 2000 compliance 2.2. Focus on core business areas and processes and develop a Year 2000 assessment document Information systems are not created equal. Systems supporting mission-critical business processes are clearly more important than systems supporting mission support functions-- usually administrative--although these are necessary functions. A focus on core business GAO/AIMD-10.1.14 Year 2000 Computing Crisis 10 areas and processes is essential to the task of assessing the impact of the Year 2000 problem on the enterprise and for establishing the priorities for the Year 2000 program. 2.3. Assess the severity of an impact of potential Year 2000-induced failures An assessment of the severity of Year 2000 failure needs to be done for each core business area and associated processes. 2.4. Conduct an enterprise-wide inventory of information systems for each business area An enterprise-wide inventory of information systems and their components provides the necessary foundation for Year 2000 program planning. A thorough inventory ensures that all systems are identified and linked to a specific business area or process, and that all enterprise-wide, cross-boundary systems are considered. 2.5. Use inventory data to develop a comprehensive automated system portfolio and identify, for each system • links to core business areas or processes • platforms, languages, and database management systems • operating system software and utilities • telecommunications • internal and external interfaces • owners • the availability and adequacy of source code and associated documentation 2.6. Analyze portfolio and identify for each system • non-repairable items (lack of source code or documentation) • conversion or replacement resources required for each platform, application, database management system, archive, utility, or interface 2.7. Prioritize system conversions and replacements An agency must determine priorities for system conversion and replacement by ranking based on key factors, such as business impact and the anticipated failure date. An agency also needs to identify applications, databases, archives, and interfaces that cannot be converted because of resource and time constraints. 2.8. Establish Year 2000 project teams for business areas and major systems Multi-disciplinary project teams consisting of domain experts in relevant functional areas, system and software specialists, operational analysis specialists, and contract specialists need to be established with explicit objectives and time schedules. Access to legal advice is also a necessity. GAO/AIMD-10.1.14 Year 2000 Computing Crisis 11 2.9. Develop Year 2000 program plan, including • schedules for all tasks and phases of the Year 2000 program • master conversion and replacement schedule, including identification of systems and their components • assessment and selection of outsourcing options • assignment of conversion or replacement projects to Year 2000 project teams • risk assessment • contingency plans for all systems 2.10. Identify, prioritize, and mobilize needed resources Achieving Year 2000 compliance will require significant investment in two vital resources--money and people. Accordingly, agencies will need to make informed choices about information technology priorities within their organization by assessing the costs, benefits, and risks of competing projects. In some instances, agencies may have to defer or cancel new system development efforts and reprogram the freed resources to achieve Year 2000 compliance. 2.11. Develop validation strategies and testing plans for all converted or replaced systems and their components. Identify and acquire automated test tools and develop test scripts. The testing and validation of the converted or replaced systems will require a phased approach. For example, an approach developed by IBM includes four phases: • Phase I--unit testing--focus on functional and compliance testing of a single application or software module. • Phase II--integration testing--test the integration of related software modules and applications. • Phase III--system testing--test all of the integrated components of an information system. • Phase IV--acceptance testing--test the information system with live operational data. Regardless of the selected validation and testing strategy, the scope of the testing and validation effort will require careful planning and use of automated tools, including test case analyzers and test data libraries. 2.12. Define requirements for Year 2000 test facility Agencies may have to acquire a Year 2000 test facility to provide an adequate testing environment and to avoid potential contamination or interference with the operation of production systems. 2.13. Identify and acquire Year 2000 tools Agencies should identify and acquire Year 2000 tools to facilitate the conversion and testing processes. GAO/AIMD-10.1.14 Year 2000 Computing Crisis 12 2.14. Address implementation schedule issues, including • the identification and selection of conversion facilities • time needed to put converted systems into production • the conversion of backup and archival data 2.15. Address interface and data exchange issues, including • the development of a model showing the internal and external dependency links between enterprise core business areas, processes, and information systems • the notification of all outside data exchange entities • the need for data bridges and filters • contingency plans if no data are received from an external source • validation process for incoming external data • contingency plans for invalid data 2.16. Initiate the development of contingency plans for mission-critical systems. Agencies should initiate the development of realistic contingency plans--including the development of manual and contract procedures--to ensure the continuity of core business processes. 2.17. Identify Year 2000 vulnerable systems and processes operating outside the information resource management area Identify and assess Year 2000 vulnerable systems and processes outside the information resource management area, including telephone and network switching equipment, and building infrastructure systems. Develop a separate plan for their renovation. GAO/AIMD-10.1.14 Year 2000 Computing Crisis 13 3.0 Renovation ______________________________________________________________________________ The renovation--conversion, replacement, or retirement--phase involves making and documenting software and hardware changes, developing replacement systems, and decommissioning eliminated systems. Renovation involves conversion of an existing application; replacement deals with the development of a new application; elimination focuses on the retirement or decommissioning of an existing application or system component. In all three cases, the process must also consider the complex interdependencies among applications, hardware platforms, databases, and the internal and external interfaces. All changes to the information systems and their components must be made under configuration management to ensure that changes are adequately documented and coordinated throughout the agency. Equally important is the need for each agency to assess dependencies and to communicate all changes to the information systems to internal and external users. Key Processes 3.1. Convert selected applications, databases, archives, and related system components 3.2. Develop data bridges and filters 3.3. Replace selected applications and related system components 3.4. Document code and system changes 3.5. Schedule unit, integration, and system tests 3.6. Retire selected applications and related system components 3.7. Communicate changes to information systems to internal and external users 3.8. Track conversion and replacement process, collect project metrics 3.9. Share information among Year 2000 projects, including lessons learned and best practices 3.1. Convert selected applications, databases, archives, and related system components In converting application systems, consider changes in operating systems, compilers, utilities, domain-specific program products, and commercial database management systems. 3.2. Develop data bridges and filters Ensure that all internal and external data sources meet the Year 2000 date standards of the converted or replaced systems. Develop bridges or filters to convert non-conforming data. 3.3. Replace selected applications, platforms, database management systems, operating systems, compilers, utilities, and other commercial off-the-shelf (COTS) software Ensure that replacement products are Year 2000 compliant, including their ability to properly handle the leap year adjustments. Direct contract specialist and legal staff to review contracts and warranties. GAO/AIMD-10.1.14 Year 2000 Computing Crisis 14 3.4. Document code and system changes Implement and use configuration management procedures to ensure that all changes to information systems and their components are properly documented and managed. 3.5. Schedule unit, integration, and system tests Schedule unit, integration, and system tests following the conversion of individual application and software modules. Coordinate scheduling with other project teams to ensure that all components--including data bridges or filters--are available for testing. 3.6. Retire selected applications, platforms, database management systems, operating systems, utilities, and COTS software Prepare to retire replaced applications, platforms, database management systems, operating systems, utilities, and COTS software upon the successful completion of acceptance testing. 3.7. Communicate changes to information systems to all internal and external users Communicate changes to the agency’s information systems and components, and specifically all changes to date formats for data exchanged with other systems or external organizations. Document changes through the configuration management process. 3.8. Track the conversion and replacement process and collect project metrics Track the conversion and replacement projects and collect and use project metrics to manage cost and schedule. 3.9. Share information among Year 2000 projects and disseminate lessons learned and best practices Ensure that project staffs understand the need to collect and disseminate information on lessons learned and best practices. Develop dissemination strategy and tools, such as intranet web sites and newsletters. GAO/AIMD-10.1.14 Year 2000 Computing Crisis 15 4.0 Validation ______________________________________________________________________________ We expect that agencies may need over a year to adequately validate and test converted or replaced mission-critical systems for Year 2000 compliance, and that the testing and validation process may consume over half of the Year 2000 program resources and budget. The length of the validation and test phase and its cost are driven by the complexity inherent in the Year 2000 problem. Agencies must not only test Year 2000 compliance of individual applications, but also the complex interactions between scores of converted or replaced computer platforms, operating systems, utilities, applications, databases, and interfaces. Moreover, in some instances, agencies may not be able to shut down their production systems for testing, and may thus have to operate parallel systems implemented on a Year 2000 test facility. All converted or replaced system components must be thoroughly validated and tested to (1) uncover errors introduced during the renovation phase, (2) validate Year 2000 compliance, and (3) verify operational readiness. The testing should account for application, database interdependencies, and interfaces. The testing should take place in a realistic test environment. A Year 2000 test facility may be required to ensure adequate testing of licensed software and converted applications while preventing the contamination or the corruption of operational information systems and related databases. Agencies should assess their testing procedures and tools to ensure that all converted system components meet quality standards and are Year 2000 compliant. Key Processes 4.1. Develop and document test and compliance plans and schedules 4.2. Develop strategy for managing the testing of contractor-converted systems 4.3. Implement Year 2000 test facility 4.4. Implement automated test tools and test scripts 4.5. Perform unit, integration, and system testing 4.6. Define, collect, and use test metrics to manage the testing and validation process 4.7. Initiate acceptance testing 4.1. For each converted or replaced application or system component, develop and document test and compliance plans and schedules Establish a compliance validation process. Most suppliers of COTS software do not disclose their source code or the internal logic of their products; therefore, testing should be complemented by a careful review of warranties and/or guarantees. 4.2. Develop a strategy for managing the testing of contractor-converted systems In many instances, the agency will contract for the conversion of selected systems and their components. The contract conversion must be closely managed to ensure that the contractor follows the agency’s Year 2000 conversion standards. In addition, the agency must ensure that the contractor-converted systems are adequately tested. GAO/AIMD-10.1.14 Year 2000 Computing Crisis 16 4.3. Implement Year 2000 test facility Testing the converted or replaced systems and their components for Year 2000 compliance will likely require an isolated test facility capable of simulating Year 2000 requirements. The test facility should provide sufficient disk storage for large test databases and multiple versions of the application software. 4.4. Implement automated test tools and test scripts The use of computer-aided software testing tools and test scripts has the potential to significantly reduce the testing and validation burden. Test management tools may help in the preparation and management of test data, in the automation of the comparison of test results, in scheduling and incident tracking, and in managing test documentation. 4.5. Perform unit, integration, and system testing Using a phased approach, perform unit, integration, and system testing. Use selected testing techniques to ensure that the converted or replaced systems and accompanying components are functionally correct and Year 2000 compliant. The testing should include regression, performance, stress, and forward and backward time testing. 4.6. Define, collect, and use test metrics to manage the testing and validation process 4.7. Initiate acceptance testing Acceptance testing is the final stage of the multiphase testing and validation process. During this phase, the entire information system--including data interfaces--is tested with operational data. In general, acceptance testing should be done on the Year 2000 test facility with duplicate databases to avoid risk to the production systems and the potential contamination of data. GAO/AIMD-10.1.14 Year 2000 Computing Crisis 17 5.0 Implementation ______________________________________________________________________________ Implementation of Year 2000 compliant systems and their components requires extensive integration and acceptance testing to ensure that all converted or replaced system components perform adequately in a heterogeneous operating environment. Because of the scope and complexity of the Year 2000 conversion changes, integration, acceptance, and implementation will likely be a lengthy and costly process. Once converted or replaced and subsequently tested, Year 2000 compliant applications and system components must be implemented. Since not all system components will be converted or replaced simultaneously, agencies may be expected to operate in a heterogeneous computing environment comprised of a mix of Year 2000 compliant and non-compliant applications and system components. The reintegration of the Year 2000 compliant applications and components into the agency’s production environment must be carefully coordinated to account for system interdependencies. Parallel processing--where the old and the converted systems are run concurrently--may be needed to reduce risk. Key Processes 5.1. Define transition environment and procedures 5.2. Develop implementation schedule 5.3. Resolve data exchange issues and interagency concerns 5.4. Deal with database and archive conversion 5.5. Complete acceptance testing 5.6. Implement contingency plans 5.7. Update or develop disaster recover plans 5.8. Implement converted and replaced systems 5.1. Define transition environment and procedures The transition from the current environment to Year 2000 compliant systems will be difficult and complex. First, some key components of the agency systems--Year 2000 compliant databases, operating systems, utilities, and other COTS products--may not be available until late 1998 or early 1999. Second, external data suppliers may not plan to complete their conversion and testing until 1999. Third, the testing, validation, and correction processes may take much of 1999. Fourth, replacement systems may not be ready for testing until late 1999. As a result, agencies may be forced to operate--at least for a time-- parallel systems and databases. 5.2. Develop implementation schedule The Year 2000 implementation schedule must not only deal with uncertainties common to all large system development efforts, but also should indicate all major milestones and the critical path for the completion of the Year 2000 program. GAO/AIMD-10.1.14 Year 2000 Computing Crisis 18 5.3. Resolve data exchange issues and interagency concerns, including ensuring that • all outside data exchange entities are notified • data bridges and filters are ready to handle non-conforming data • contingency plans and procedures are in place if data are not received from an external source • contingency plans and procedures are in place if invalid data are received from an external source • the validation process is in place for incoming external data All data issues and interagency concerns should be resolved prior to acceptance testing and implementation. Bridges and filters should be in place to handle non-conforming data received from external sources, and contingency plans and procedures should be in place to handle no data or bad data situations. 5.4. Deal with database and archive conversion Because the conversion of large databases from 2-digit to 4-digit year fields is a time consuming effort, agencies may consider off-site conversion alternatives. 5.5. Complete acceptance testing In general, formal testing uncovers about 80-90 percent of software errors, with the remaining 10-20 percent of errors discovered during operations. Acceptance testing should be completed no later than Fall of 1999, to allow sufficient time for the correction of software errors discovered following implementation. 5.6. Implement contingency plans as necessary Implement contingency plans to ensure support for business functions and processes that may be interrupted by the failure to achieve Year 2000 compliance of a specific mission- critical system. 5.7. Update or develop disaster recovery plans All Year 2000 compliant systems--including the converted and replaced systems and related databases--should have disaster recovery plans for the restoration of operations and data in case of extended outage, sabotage, or natural disaster. 5.8. Implement converted and replaced systems Reintegrate the converted and replaced systems and related databases into the production environment. GAO/AIMD-10.1.14 Year 2000 Computing Crisis 19 Program and Project Management ______________________________________________________________________________ The Year 2000 program is likely the largest and most complex system conversion effort ever undertaken by many federal agencies. It requires the disciplined and coordinated application of scarce resources to an enterprise-wide system conversion effort that must be completed by a fixed date. To succeed, agencies must manage the Year 2000 program as a large system development effort. Key Processes A. Establish Year 2000 program management structure B. Develop and implement needed policies, guidelines and procedures to manage a major program C. Implement program management processes and tools A. Establish Year 2000 program management structure • appoint a Year 2000 program manager and establish a Year 2000 program team • identify technical and management representatives from each core business area The agency’s Year 2000 program--headed by a program manager--should be adequately staffed to ensure the successful completion of the assessment phase. In addition to technical skills, the program staff should be able to track the cost and schedule for individual Year 2000 projects, and to coordinate the agency’s Year 2000 activities with other organizations. B. Develop and implement needed policies, guidelines and procedures to manage a major program Based on the assessment of the agency’s program management capability performed during the awareness phase, ensure that necessary enterprise-wide program management policies and procedures are in place, including • configuration management • quality assurance • risk management • project scheduling and tracking • metrics • budgeting Agencies may consider establishing an enterprise-level competency center to train staff and to foster the use of proven industry system development and program management practices. GAO/AIMD-10.1.14 Year 2000 Computing Crisis 20 C. Implement program management processes and tools Monitor Year 2000 projects, and ensure that projects follow required policies and procedures for configuration management, project scheduling and tracking, and metrics. Agencies may consider subjecting their Year 2000 program to an independent verification and validation effort. This verification and validation may be performed by the agency’s quality assurance staff complemented by internal auditors. GAO/AIMD-10.1.14 Year 2000 Computing Crisis 21 Year 2000 Program Assessment Checklist ______________________________________________________________________________ Agency Year 2000 Program Phase or Activity ❑ Awareness ❑ Validation ❑ Assessment ❑ Implementation ❑ Renovation ❑ Program Management ______________________________________________________________________________ Awareness ❑ Has the agency defined and documented the potential impact of the Year 2000 problem? ❑ Has the agency conducted a Year 2000 awareness campaign? ❑ Has the agency assessed the adequacy of its program management policies, capabilities, and practices, including configuration management, program and project management, and quality assurance? ❑ Has the agency developed and documented a Year 2000 strategy? ❑ Is the Year 2000 strategy supported by executive management? The agency has ❑ Year 2000 policy directive ❑ Year 2000 program charter ❑ Has the agency established an executive management council or committee to guide the Year 2000 program? ❑ Has a program manager been appointed and a Year 2000 program office been established and staffed? ❑ Has the agency identified technical and management points of contacts in core business areas? Assessment ❑ Has the agency defined Year 2000 compliance? ❑ Has the agency identified core business areas and processes? ❑ Has the agency assessed the severity of potential impact of Year 2000-induced failures for core business areas and processes? GAO/AIMD-10.1.14 Year 2000 Computing Crisis 22 ❑ Has the agency conducted a comprehensive enterprise-wide inventory of its information systems? The agency has ❑ system inventory listing components and interfaces for each system ❑ comprehensive plan to identify and eliminate obsolete code ❑ Has the agency developed a comprehensive automated system portfolio? The agency’s portfolio identifies ❑ links to core business areas or processes ❑ platforms, languages, and database management systems ❑ operating system software and utilities ❑ telecommunications ❑ internal and external interfaces ❑ owners ❑ the availability and adequacy of source code and associated documentation ❑ Has the agency analyzed its system portfolio and identified for each system ❑ non-repairable items (lack of source code or documentation) ❑ conversion or replacement resources required for each platform, application, database management system, archive, utility, or interface ❑ Has the agency prioritized its system conversion and replacement program? The agency’s prioritization process includes ❑ ranking by business impact ❑ ranking by anticipated failure date ❑ identification of applications, databases, archives, and interfaces that cannot be converted because of resource and time constraints ❑ Has the agency established Year 2000 project teams for business areas and major systems? ❑ Has the agency developed a Year 2000 program plan? The agency’s program plan includes ❑ schedules for all tasks and phases ❑ master conversion and replacement schedule ❑ assessment and selection of outsourcing options ❑ assignment of conversion or replacement projects to project teams ❑ risk assessment ❑ contingency plans for all systems GAO/AIMD-10.1.14 Year 2000 Computing Crisis 23 ❑ Has the agency identified and mobilized required resources and capabilities? ❑ Has the agency developed validation strategies and testing plans for all converted or replaced systems and their components? ❑ Has the agency analyzed and identified requirements for a Year 2000 test facility? ❑ Has the agency identified and acquired Year 2000 tools? ❑ Has the agency considered implementation scheduling issues? The agency’s program plan addresses ❑ where conversion will take place (data center or off-site location) ❑ time needed to place converted systems into production ❑ conversion of backup or archived data ❑ Has the agency addressed interface and data exchange issues? The agency has ❑ analyzed dependencies on data provided by other organizations ❑ contacted all entities with whom it exchanges data ❑ identified the need for data bridges or filters ❑ made contingency plans if no data are received from external sources ❑ made plans to determine that incoming data are valid ❑ developed contingency plans to handle invalid data ❑ Has the agency initiated the development of contingency plans for critical systems? ❏ Does the impact assessment document identify Year 2000 vulnerable systems and processes outside the traditional information resource management area that may affect the agency’s operations? The assessment document addresses the impact of potential Year 2000 induced failure of ❑ telecommunication systems, including telephone and data networks switching equipment ❑ building infrastructure Renovation ❑ Is the agency meeting its budget and schedule in the conversion of targeted applications, platforms, databases, archives, or interfaces? ❑ Is the agency meeting its budget and schedule in developing bridges and filters to handle non- conforming data? GAO/AIMD-10.1.14 Year 2000 Computing Crisis 24 ❑ Is the agency meeting its budget and schedule in the replacement of targeted applications and system components? ❑ Is the agency documenting all code and system modifications and using configuration management to control changes? ❑ Is the agency scheduling unit, integration, and system tests? ❑ Is the agency meeting its budget and schedule in eliminating targeted applications and system components? ❑ Is the agency communicating the changes to its information systems to all internal and external users? ❑ Is the agency tracking the conversion and replacement process and collecting and using project metrics to manage the conversion and replacement process? ❑ Is the agency sharing information among Year 2000 projects? The agency is disseminating ❑ “lessons learned” ❑ best practices Validation ❑ Has the agency developed and documented test and validation plans for each converted or replaced application or system component? ❑ Has the agency developed and documented a strategy for testing contractor-converted or replaced applications or system components? ❑ Has the agency implemented a Year 2000 test facility? ❑ Has the agency implemented automated test tools and scripts? ❑ Has the agency performed unit, integration, and system tests on each converted or replaced component The agency’s testing procedures include the following types of tests ❑ regression ❑ performance ❑ stress ❑ forward and backward time GAO/AIMD-10.1.14 Year 2000 Computing Crisis 25 ❑ Is the agency tracking the testing and validation process and collecting and using test metrics to manage the testing activities? ❑ Has the agency initiated acceptance tests? Implementation ❑ Has the agency defined its transition environment and procedures? ❑ Has the agency developed and documented a schedule for the implementation of all converted or replaced applications and system components? ❑ Has the agency resolved data exchange issues and interagency concerns? ❑ Has the agency dealt with database and archive conversion? ❑ Has the agency completed acceptance testing? ❑ Has the agency implemented contingency plans? ❑ Has the agency updated or developed disaster recovery plans? ❑ Has the agency reintegrated the converted and replaced systems and related databases into the production environment? Program and Project Management ❑ Has the agency established a Year 2000 program management structure? The agency has ❑ appointed a Year 2000 program manager and established a Year 2000 program team ❑ identified technical and management representatives from each core business area ❑ Based on the assessment of its program management capabilities, has the agency developed and implemented policies, guidelines, and procedures to manage a major program? The agency’s policies, guidelines, and procedures include ❑ configuration management ❑ quality assurance ❑ risk management ❑ project scheduling and tracking ❑ metrics ❑ budgeting GAO/AIMD-10.1.14 Year 2000 Computing Crisis 26 ❑ Is the agency monitoring the Year 2000 program to ensure that projects are following required policies and procedures for configuration management, project scheduling and tracking, and metrics? GAO/AIMD-10.1.14 Year 2000 Computing Crisis 27 Selected Year 2000 Resources ______________________________________________________________________________ There are many readily accessible sources of useful information on the Year 2000 problem, with many government and industry organizations establishing Year 2000 web sites. These sites provide information about Year 2000 compliant products, tools, and practices. Best Practices The Program Manager’s Guide to Software Acquisition Best Practices (Version 1.1), Software Acquisition Best Practice Initiative, Department of Defense (Undated). A framework of best practices focused on effective project management, software defect detection, and project risk reduction. The guide and several companion publications are available at <http://spmn.com/>. Key Practices of the Capability Maturity Model, Version 1.1, Software Engineering Institute, Carnegie Mellon University, February 1993. A set of key practices for planning, engineering, and managing software development. Discussed practices include configuration management, software quality assurance, project tracking and oversight, and project planning. More information on the capability model may be found at <http://www.sei.cmu.edu/technology/cmm.html>. CIO Council Subcommittee on Year 2000 Best Practices, CIO Council Subcommittee on Year 2000, April 1997. A compendium of best practices focused on a Year 2000 program presented in a framework of awareness, assessment, renovation, validation, and implementation phases. Available at <http://www.itpolicy.gsa.gov/mks/yr2000/y207best.htm>. The Year 2000 and 2-Digit Dates: A Guide for Planning and Implementation, International Business Machines Corporation, April 1997. The guide provides information on the cause and scope of using dates represented by 2-digit years, problems with programs using 2-digit-year data, the best technique for reformatting the year-date notations, migration strategies to a Year 2000-ready environment, testing techniques, and a list of IBM tools. Available at <http://ppdbooks.pok.ibm.com/cgi- bin/bookmgr/bookmgr.cmd/books/y2kpaper/contents>. Selected Year 2000 Web Sites Federal Year 2000 Web Sites ❏ CIO Council Subcommittee on Year 2000 <http://www.itpolicy.gsa.gov/mks/yr2000/y201toc1.htm> GAO/AIMD-10.1.14 Year 2000 Computing Crisis 28 ❏ Army <http://imabbs.army.mil/army-y2k/> ❏ Air Force <http://infosphere.safb.af.mil/~jwid/fadl/world/y2k.htm> ❏ Navy <http://www.doncio.navy.mil/y2k/year2000.htm> ❏ Marine Corps <http://issb-www1.mqg.usmc.mil/year2000/> ❏ Defense Information Systems Agency <http://www.disa.mil/cio/y2k/cioosd.html> General Year 2000 Sites ❏ The Year 2000 Information Center <http://www.year2000.com> ❏ Year 2000 Technical Audit Center <http://www.auditserve.com/> ❏ Year 2000 Information Network <http://web.idirect.com/%7Embsprog/y2kcon.html> ❏ The Year 2000 Resource <http://www.deweerd.org/year2000/> ❏ CIO Year 2000 Resource Center <http://www.cio.com/forums/year2k.html> Year 2000 Products, Tools, and Patches ❏ Defense Information Systems Agency Tools <http://www.mitre.org/research/y2k/docs/TOOLS_CAT.html> ❏ Air Force Software Technology Support Center <http://www.stsc.hill.af.mil/RENG/index.html/> ❏ Army Tools <http://www.army.mil/army-y2k/tools/tools~1.htm> ❏ RighTime PC Patches <http://www.RighTime.com/> GAO/AIMD-10.1.14 Year 2000 Computing Crisis 29 Glossary ______________________________________________________________________________ The definitions in this glossary were developed by the project staff or were drawn from other sources, including the Computer Dictionary: The Comprehensive Standard For Business, School, Library, and Home, Microsoft Press, Washington, DC, 1991; The Year 2000 Resource Book, Management Support Technology Corp., Framingham, Massachusetts, 1996; The Year 2000 and 2-Digit Dates: A Guide for Planning and Implementation, International Business Machines Corporation, 1997; Denis Howe’s “Free On-line Dictionary of Computing,” at <http://wombat.doc.ic.ac.uk/>; and the Gartner Group’s “IT Glossary” at <http://gartner5.gartnerweb.com/gartner/itglossary/dlist.html>. Application A computer program designed to help people perform a certain type of work. Depending on the work for which it was designed, an application can manipulate text, numbers, graphics, or a combination of these elements. Architecture A description of all functional activities to be performed to achieve the desired mission, the system elements needed to perform the functions, and the designation of performance levels of those system elements. An architecture also includes information on the technologies, interfaces, and location of functions and is considered an evolving description of an approach to achieving a desired mission. Business A description of the systems, databases, and interactions between architecture systems and databases that will be needed to fulfill business requirements. Business area A grouping of business functions and processes focused on the production of specific outputs. Business function A group of logically related tasks that are performed together to accomplish an objective. Business plan An action plan that the enterprise will follow on a short-term and/or long- term basis. It specifies the strategic and tactical objectives of the company over a period of time. The plan, therefore, is time dependent; it will change with the enterprise. Although a business plan is usually written in a style unique to a specific enterprise, it should concisely describe "what" is planned, "why" it is planned, "when" it will be implemented, by "who," and "how" it will be gauged. The architects of the plan are typically the principals of the enterprise. Component A single resource with defined characteristics. The component concept is used in defining precise specifications for testing the validity of various resources. These components are also defined by their relationship to other components. GAO/AIMD-10.1.14 Year 2000 Computing Crisis 30 Configuration The continuous control of changes made to a system's hardware, management software, and documentation throughout the development and operational life of the system. Contingency plan In the context of the Year 2000 program, a plan for responding to the loss of a system due to a Year 2000 problem. In general, a contingency plan describes the steps the enterprise would take-- including the activation of manual or contract processes--to ensure the continuity of its core business processes in the event of a Year 2000-induced system failure. Conversion The process of making changes to databases or source code. Database An aggregation of data; a file consisting of a number of records or tables, each of which is constructed of files of a particular type, together with a collection of operations that facilitate searching, sorting, recombination, and similar operations. Data dictionary A repository of information about data, such as its meaning, relationships to other data, origin, usage and format. The dictionary assists company management, database administrators, systems analysts and application programmers in effectively planning, controlling, and evaluating the collection, storage and use of data. A data dictionary manages data categories such as aliases, data elements, data records, data structures, data stores, data models, data flows, relationships, processes, functions, dynamics, size, frequency, resource consumption, and other user-defined attributes. Debug With software, to detect, locate, and correct logical or syntactical errors in a computer program. Defect A problem or “bug” that if not removed, could cause a program to either produce erroneous results or otherwise fail. Information A description of the enterprise in terms of its business activity, architecture business information, and their interaction. Infrastructure The computer and communication hardware, software, databases, people, and policies supporting the enterprise’s information management functions. Integration testing Testing to determine that the related information system components perform to specification. Interface A boundary across which two systems communicate. An interface might be a hardware connector used to link to other devices, or it might be a convention used to allow communication between two software systems. GAO/AIMD-10.1.14 Year 2000 Computing Crisis 31 Inventory In the context of a Year 2000 program, the process of determining the components that comprise the agency’s systems portfolio. The inventory should include all applications, databases, files, and related system components that will require inspection to locate date data and related date computations. Line of code A single computer program command, declaration, or instruction. Program size is often measured in lines of code. Metrics Means by which software engineers measure and predict aspects of processes, resources, and products that are relevant to the software engineering activity. Mission-critical A system supporting a core business activity or process. system Object code The machine code generated by a source code language processor such as an assembler or compiler. A file of object code may be immediately executable or it may require linking with other object code files, e.g. libraries, to produce a complete executable program. Operating system The software which schedules tasks, allocates storage, handles the interface to peripheral hardware, and presents a default interface to the user when no application program is running. Outsourcing Paying another company to provide services which an organization might otherwise have performed itself, e.g. software development. Parallel processing The simultaneous use of more than one computer to solve a problem. Platform The foundation technology of a computer system. Typically, a specific combination of hardware and operating system. Portfolio In the context of the Year 2000 program, an inventory--preferably automated--of an agency’s information systems and their components grouped by business areas. Production The system environment where the agency performs its routine environment information processing activities. Quality assurance All the planned and systematic actions necessary to provide adequate confidence that a product or service will satisfy given requirements for quality. Regression testing Selective retesting to detect faults introduced during modification of a system. GAO/AIMD-10.1.14 Year 2000 Computing Crisis 32 Risk assessment A continuous process performed during all phases of system development to provide an estimate of the damage, loss, or harm that could result from a failure to successfully develop individual system components. Risk management A management approach designed to reduce risks inherent to system development. Source code The form in which a computer program is written by the programmer. Source code is written in a programming language which is then compiled into object code or machine code or executed by an interpreter. Standard In computing, a set of detailed technical guidelines used as a means of establishing uniformity in an area of hardware or software development. Strategic IRM plan A long-term, high-level plan that defines a systematic way of how the agency will use information technology to effectively accomplish the agency's missions, goals, and objectives. Strategic plan A long-term, high-level plan that identifies broad business goals and provides a roadmap for their achievement. System testing Testing to determine that the results generated by the enterprise’s information systems and their components are accurate and the systems perform to specification. Test The process of exercising a product to identify differences between expected and actual behavior. Test facility A computer system isolated from the production environment dedicated to the testing and validation of applications and system components. Unit testing Testing to determine that individual program modules perform to specification. Utilities Computer programs designed to perform maintenance work on the system or on system components--for example, a storage backup program, a disk or file recovery program, or a resource editor. Validation The process of evaluating a system or component during or at the end of the development process to determine whether it satisfies specified requirements. Year 2000 compliant "Year 2000 compliant . . . means, with respect to information technology, that the information technology accurately processes date/time data (including, but not limited to, calculating, comparing, and sequencing) from, into, and between the twentieth and twenty-first centuries, and the years 1999 and 2000 and leap year calculations, to the extent that other information technology, used in combination with the information GAO/AIMD-10.1.14 Year 2000 Computing Crisis 33 technology being acquired, properly exchanges date/time data with it." (48 CFR Part 39.002) Year 2000 problem The potential problems and its variations that might be encountered in any level of computer hardware and software from microcode to application programs, files, and databases that need to correctly interpret year-date data represented in 2-digit-year format. (511428) GAO/AIMD-10.1.14 Year 2000 Computing Crisis 34
Year 2000 Computing Crisis: An Assessment Guide (Supersedes 158206)
Published by the Government Accountability Office on 1997-09-01.
Below is a raw (and likely hideous) rendition of the original report. (PDF)