oversight

Year 2000 Computing Challenge: SBA Needs to Strengthen Systems Testing to Ensure Readiness

Published by the Government Accountability Office on 1999-08-27.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                  United States General Accounting Office

GAO               Report to the Committee on Small
                  Business, U.S. Senate



August 1999
                  YEAR 2000
                  COMPUTING
                  CHALLENGE

                  SBA Needs to
                  Strengthen Systems
                  Testing to Ensure
                  Readiness




GAO/AIMD-99-265
United States General Accounting Office                                           Accounting and Information
Washington, D.C. 20548                                                                 Management Division



                                    B-283376                                                                Letter

                                    August 27, 1999

                                    The Honorable Christopher S. Bond
                                    Chairman
                                    The Honorable John F. Kerry
                                    Ranking Minority Member
                                    Committee on Small Business
                                    United States Senate

                                    The Small Business Administration (SBA) relies on automated information
                                    systems to fulfill its mission to provide assistance to small businesses and
                                    victims of disaster. To prevent disruptions of assistance caused by the Year
                                    2000 (Y2K) computing problem, SBA has repaired and tested its
                                    mission-critical information systems and acted to address its external data
                                    exchanges. However, further actions are needed for the agency to have
                                    reasonable assurance that its systems are Y2K ready.

                                    Because of the importance of the agency’s Y2K program, you requested that
                                    we verify (1) that SBA has completed repair and replacement work for its
                                    mission-critical systems and (2) the extent to which data exchanges with
                                    outside entities have been addressed. This report provides a summary of
                                    information presented at our July 19, 1999, briefing to your staff and
                                    presents recommendations we are making to SBA to assist it in addressing
                                    its Y2K testing weaknesses. We performed our work from February
                                    through mid-July 1999 in accordance with generally accepted government
                                    auditing standards, and incorporated SBA’s comments into our briefing
                                    before delivering it to your staff. The briefing slides are presented in
                                    appendix I, and our objectives, scope, and methodology in appendix II.



Results in Brief                    SBA completed systems renovations and unit testing for its 42
                                    mission-critical systems, and implemented the renovated systems by the
                                    Office of Management and Budget’s (OMB) October 1, 1998, deadline.
                                    Since then, SBA performed integration testing and system acceptance
                                    testing to ensure the renovated mission-critical systems will operate
                                    correctly after the turn of the century--those tests were completed at the
                                    end of May 1999. Despite these efforts, however, weaknesses inSBA’s Y2K
                                    testing increase the risk that its mission-critical systems are not yet Y2K
                                    ready. First, SBA’s integration tests are incomplete because key business
                                    processes were not specifically tested, and tests were not tracked to verify



                   Leter            Page 1                                 GAO/AIMD-99-265 SBA Year 2000 Readiness
                     B-283376




                     that all key business processes were actually tested. Second, system
                     acceptance tests are incomplete because users did not review test plans,
                     guidance, procedures, or data, or participate in the tests. Third, testing of
                     SBA’s mission-critical systems has not been independently validated, and
                     these systems have not been certified as being Y2K ready.

                     SBA has been addressing the Y2K problem for its external data exchanges.
                     It has reported testing or reaching agreements with 38 of its 58 external
                     data exchange partners, including its largest, which accounts for about
                     90 percent of SBA’s external data exchange activity. SBA is continuing to
                     pursue testing or agreements with its remaining partners.

                     In addition, SBA’s approach to end-to-end testing is not adequate because it
                     (1) does not ensure that software supporting key business areas or
                     functions has been tested and (2) leaves many potential issues, such as
                     interoperability of internal and external systems, untested. Because of
                     these testing weaknesses, SBA lacks reasonable assurance that its systems
                     will function correctly and adequately support its key business areas and
                     functions beyond 1999.



Background           SBA provides financial, technical, and management assistance to the
                     nation’s small businesses as well as offering disaster recovery assistance to
                     individuals, families, and businesses. In fiscal year 1997, SBA provided
                     $10.9 billion in new loan guarantees and $2.4 billion in new venture capital,
                     and approved more than 49,000 disaster loans totaling $1.1 billion. If not
                     adequately addressed, the Y2K computing problem poses significant risks
                     to SBA’s ability to provide these services to more than 490,000 small
                     businesses nationally.

                     SBA has been working to ensure that the Y2K problem does not disrupt its
                     ability to provide services to businesses and victims of disaster. Under the
                     leadership of its Office of the Chief Information Officer, SBA has identified
                     42 mission-critical systems and 58 external data exchanges to be renovated
                     and tested before 2000. SBA estimates the cost of its Y2K program will be
                     $10.6 million for fiscal years 1996 through 2000.

                     To help agencies reduce the risk of potential Y2K-induced disruptions to
                     their operations, we prepared the Year 2000 Computing Crisis: An




             Leter   Page 2                                  GAO/AIMD-99-265 SBA Year 2000 Readiness
                           B-283376




                           Assessment Guide1 and the Year 2000 Computing Crisis: A Testing Guide.2
                           The assessment guide identifies five phases that cover key steps necessary
                           to complete Y2K work, and the testing guide identifies a complete Y2K
                           testing approach that complements the five phases described in the
                           assessment guide.



Mission-Critical           SBA has completed renovation and unit testing of its mission-critical
                           systems. However, we found weaknesses in SBA’s Y2K integration testing,
Systems Have Been          system acceptance testing, and independent verification and validation
Renovated But Testing      (IV&V). As a result, SBA does not yet have reasonable assurance that its
                           systems are Y2K ready.
Processes Are Weak

Mission-Critical Systems   SBA has performed repair work and unit testing for its 42 mission-critical
Have Been Renovated        systems. This included developing date conversion software so the 2,032
                           software programs that constitute the 42 mission-critical systems would
                           not need to be renovated individually. SBA performed unit tests of those
                           software programs, and the test results were reviewed and validated by
                           SBA programmers and users. In addition, the Office of Inspector General’s
                           (OIG) technical contractor reviewed the results of about 30 unit tests.
                           Renovations and unit tests were completed and software programs were
                           placed into production by OMB’s October 1, 1998, deadline. SBA also
                           completed the migration of its systems to a Y2K-compliant hardware
                           environment in late June 1999.


Integration Testing Is     Our Y2K guidance recommends performing integration testing to verify
Incomplete                 that software programs, when combined, work together as intended.
                           Typically, this includes identifying key business processes and testing the
                           software that supports those processes, including developing specific test
                           cases, data, and identifying expected outcomes for each key business
                           process. In accordance with our guidance, SBA developed an integration
                           test plan for its mission-critical systems, which identified key business
                           processes to be tested, such as approving loans, increasing or decreasing
                           loan amounts, and canceling and reinstating loans.


                           1
                            Year 2000 Computing Crisis: An Assessment Guide (GAO/AIMD-10.1.14, September 1997).
                           2
                            Year 2000 Computing Crisis: A Testing Guide (GAO/AIMD-10.1.21, November 1998).




                           Page 3                                           GAO/AIMD-99-265 SBA Year 2000 Readiness
                               B-283376




                               However, SBA did not follow its test plan in that it did not develop specific
                               test cases to test most of its key business processes. Instead, SBA used
                               (1) all transaction data for one day to test daily, weekly, monthly, and some
                               quarterly, semiannual, and annual processing and reports and (2) some test
                               scripts and manually-entered transactions. In addition, SBA did not track
                               its integration test procedures, cases, and results to verify that all key
                               business processes were actually tested. According to SBA’s Y2K program
                               manager, testing systems with actual transactions using typical production
                               processes (i.e., daily, weekly, monthly, etc., processing) provides adequate
                               assurance that key business processes are Y2K ready because SBA uses
                               most of those processes on a daily basis. However, SBA could not provide
                               information on which of its business processes were tested or the extent of
                               such testing because it did not plan, conduct, or track its tests and testing
                               procedures to ensure sufficient testing of key business processes. As a
                               result, SBA lacks reasonable assurance that all systems supporting its key
                               business processes have been adequately tested and will function beyond
                               1999.


System Acceptance Testing      Our Y2K guidance states that user participation in system acceptance
Is Incomplete                  testing is essential to ensure that systems perform correctly and are
                               acceptable to users. User involvement should include participation in
                               (1) reviews of test plans, guidance, procedures and data, (2) systems tests,
                               and (3) validating that the systems perform correctly. SBA combined
                               system acceptance testing of its mission-critical systems with its
                               integration tests by having SBA users review integration test results for
                               validation and acceptance. However, except for tests of 3 of the
                               42 mission-critical systems, user groups did not review test plans,
                               guidance, procedures, and data, and did not participate in the tests. SBA’s
                               Y2K program manager told us that he believes that user review and
                               validation of integration test results provides adequate assurance that
                               systems performed correctly. However, without participation in all test
                               processes, SBA does not have reasonable assurance that systems have
                               been adequately tested and will perform correctly beyond 1999.



Independent Verification       Our Y2K guidance recommends engaging an independent quality assurance
and Validation Is Incomplete   or IV&V group to ensure adequate testing and validation of mission-critical
                               systems. Typically, these groups review test plans, procedures, and data;
                               witness test execution; and certify that systems are Y2K ready. SBA’s OIG,
                               with assistance from its technical contractor, has been reviewing SBA’s
                               Y2K work, including its test plans and procedures. However, this review



                               Page 4                                 GAO/AIMD-99-265 SBA Year 2000 Readiness
                        B-283376




                        does not represent a complete IV&V as recommended in our guidance
                        because neither the OIG nor its technical contractor are validating the
                        systems testing or certifying that SBA’s mission-critical systems are Y2K
                        ready. As a result, SBA lacks reasonable assurance that testing has been
                        adequately planned, conducted, and documented to ensure that its systems
                        will be Y2K ready.



SBA Continues To        SBA is continuing to address the Y2K problem for its external data
                        exchanges. It has identified 58 external data exchange partners and has
Address External Data   reported testing or reaching testing agreements with 38 data exchange
Exchanges               partners, including its largest, which accounts for about 90 percent of SBA’s
                        external data exchange activity. SBA is actively pursuing testing or
                        agreements with its remaining data exchange partners, and has
                        followed-up with letters and telephone calls. SBA has not established a
                        completion date for its data exchange work since finalizing testing
                        agreements is contingent upon the cooperation of its data exchange
                        partners.



End-to-End Testing Is   Our Y2K guidance recommends end-to-end testing to verify that a set of
                        systems, including those external to the organization, interoperate as
Incomplete              intended to support key business areas or functions. According to SBA
                        officials, the agency performed some end-to-end testing because it (1) used
                        typical transactions processed from inception to completion, (2) tested
                        internal data exchanges as part of its integration testing, and (3) tested
                        external data exchanges separately. However, SBA has not ensured that
                        key business functions, such as those supporting SBA’s loan servicing
                        function, have been tested. In addition, SBA’s approach to integration
                        testing leaves many potential issues, such as the interoperability of internal
                        and external systems, untested. As a result, SBA lacks reasonable
                        assurance that the systems will work together to support SBA’s key
                        business functions beyond 1999.



Recommendations         To address the weaknesses in its Y2K software testing, we recommend that
                        SBA’s Administrator take the following actions, as stated in our July 19,
                        1999, briefing:




                        Page 5                                  GAO/AIMD-99-265 SBA Year 2000 Readiness
                  B-283376




                  • ensure that SBA’s key business processes have been adequately tested
                    and validated, and develop specific test scripts and cases to test all key
                    business processes that were not adequately tested;
                  • ensure user participation in all phases of additional testing, including
                    development of test scripts and cases, and validation that systems
                    perform correctly;
                  • obtain independent quality assurance or IV&V of the testing and
                    validation processes to certify that each mission-critical system is Y2K
                    ready; and
                  • plan and conduct end-to-end testing with data exchange partners for
                    each key business area or function.



Agency Comments   SBA officials provided comments on a draft of this briefing, and we
                  incorporated those comments as appropriate. SBA officials stated that
                  they are implementing our recommendations and ensuring the completion
                  of adequate Y2K readiness testing. In this regard, SBA officials told us that
                  they are currently working with users to document the extent of business
                  process testing and, if needed, SBA’s technical and user staff will develop
                  additional test cases to ensure that all business processes have been tested
                  adequately. Also, SBA users will participate in all phases of any additional
                  testing. To address the weaknesses in its IV&V efforts, SBA is working with
                  the OIG’s technical contractor to identify additional work needed to review
                  testing and validation processes and results. Finally, SBA plans to conduct
                  additional end-to-end testing with its internal and external data exchange
                  partners to ensure that its key business functions are Y2K ready.


                  We are sending copies of this report to The Honorable Aida Alvarez,
                  Administrator of the Small Business Administration; The Honorable John
                  Koskinen, Chairman of the President’s Council on Year 2000 Conversion;
                  and The Honorable Jacob Lew, Director of the Office of Management and
                  Budget. Copies will be made available to others upon request.




                  Page 6                                 GAO/AIMD-99-265 SBA Year 2000 Readiness
B-283376




Should you or your staff have any questions concerning this report, please
contact me at (202) 512-6253. I can also be reached by e-mail at
willemssenj.aimd@gao.gov. Other points of contact and key contributors
are listed in appendix III.




Joel C. Willemssen
Director, Civil Agencies Information Systems




Page 7                                GAO/AIMD-99-265 SBA Year 2000 Readiness
Contents



Letter                                                                                            1


Appendix I                                                                                       10
Briefing on Small
Business
Administration Year
2000 Readiness

Appendix II                                                                                      41
Objectives, Scope, and
Methodology

Appendix III                                                                                     42
GAO Contacts and
Staff
Acknowledgements




                         Abbreviations

                         IV&V     independent verification and validation
                         OIG      Office of Inspector General
                         OMB      Office of Management and Budget
                         SBA      Small Business Administration
                         Y2K      year 2000



                         Page 8                              GAO/AIMD-99-265 SBA Year 2000 Readiness
Page 9   GAO/AIMD-99-265 SBA Year 2000 Readiness
Appendix I

Briefing on Small Business Administration
Year 2000 Readiness                                                                Appenx
                                                                                        Idi




               Accounting and Information Management
               Division


             Small Business Administration
             Year 2000 Readiness



             Briefing to the Senate Small
             Business Committee


             July 19, 1999




                             Page 10         GAO/AIMD-99-265 SBA Year 2000 Readiness
             Appendix I
             Briefing on Small Business Administration
             Year 2000 Readiness




Overview


•   Background
•   Objectives, scope, and methodology
•   Mission-critical systems renovated but testing is
    incomplete
• Small Business Administration (SBA) continuing to
  address external data exchanges
•   Other observations
•   Conclusions
•   Recommendations

                                                                                  2




             Page 11                                     GAO/AIMD-99-265 SBA Year 2000 Readiness
               Appendix I
               Briefing on Small Business Administration
               Year 2000 Readiness




Background


•   SBA provides financial, technical, and management
    assistance to small businesses
•   23 million small businesses employ more than 50
    percent of the private workforce
•   In fiscal year 1997, SBA provided
    •   $10.9 billion in new loan guarantees
    •   $2.4 billion in venture capital
•   SBA helps businesses and families recover from
    disasters. In fiscal year 1997, it approved more
    than 49,000 disaster loans totaling $1.1 billion
                                                                                    3




               Page 12                                     GAO/AIMD-99-265 SBA Year 2000 Readiness
               Appendix I
               Briefing on Small Business Administration
               Year 2000 Readiness




Background (cont’ d.)


•   SBA has identified 42 mission-critical systems
    •   10 loan accounting
    •   6 loan origination
    •   16 loan servicing
    •   10 administrative
•   SBA has identified 58 external data exchanges
    •   38 federal agencies
    •   20 commercial entities

                                                                                    4




               Page 13                                     GAO/AIMD-99-265 SBA Year 2000 Readiness
               Appendix I
               Briefing on Small Business Administration
               Year 2000 Readiness




Background (cont’ d.)


•   SBA’ s Year 2000 (Y2K) program
    •   is being led by the chief information officer,
        who is also responsible for coordinating Y2K
        efforts of the offices of the chief financial officer
        and disaster assistance
    •   is being reviewed by the Office of Inspector
        General (OIG), assisted by a technical contractor
•   SBA estimates its Y2K program costs will be
    $10.6 million for fiscal years 1996 through 2000


                                                                                    5




               Page 14                                     GAO/AIMD-99-265 SBA Year 2000 Readiness
                Appendix I
                Briefing on Small Business Administration
                Year 2000 Readiness




Objectives


•      Verify that SBA has completed repair and
       replacement work for its mission-critical systems
•      Verify the extent to which data exchanges with
       outside entities have been addressed




                                                                                     6




Lte
  rt            Page 15                                     GAO/AIMD-99-265 SBA Year 2000 Readiness
            Appendix I
            Briefing on Small Business Administration
            Year 2000 Readiness




Scope and Methodology


To verify that SBA has completed repair and
replacement work, we
 • reviewed SBA’ s guidance for repairing its mission-
   critical systems and the date conversion software
   used by 39 of 42 of those systems
 • reviewed about 100 unit test plans and results, and
   technical and user validation of test results
 • reviewed integration and system acceptance test plan,
   61 test results, and technical and user validations
 • discussed SBA’ s repair and replacement work with
   SBA, technical contractor, and OIG officials
                                                                                 7




            Page 16                                     GAO/AIMD-99-265 SBA Year 2000 Readiness
             Appendix I
             Briefing on Small Business Administration
             Year 2000 Readiness




Scope and Methodology (cont’ d.)


•   To verify the extent to which SBA has addressed
    external data exchanges, we
     • reviewed SBA’ s inventory of data exchanges
     • reviewed letters sent by SBA to its data exchange
       partners
     • reviewed results of testing with SBA’ s largest data
       exchange partner
     • discussed data exchanges with SBA, technical
       contractor, and OIG officials
•   We performed our work from February through June
    1999 in accordance with generally accepted
    government auditing standards
                                                                                  8




             Page 17                                     GAO/AIMD-99-265 SBA Year 2000 Readiness
              Appendix I
              Briefing on Small Business Administration
              Year 2000 Readiness




Systems Renovated But Testing is Incomplete--Repair
Work Performed

•   SBA has performed repair work on its 42 mission-
    critical systems
    •   all mission-critical systems software was
        renovated to address Y2K issues
    •   no replacements were made to mission-critical
        software components
•   SBA completed migrating to a Y2K-compliant
    hardware environment at the end of June 1999



                                                                                   9




              Page 18                                     GAO/AIMD-99-265 SBA Year 2000 Readiness
              Appendix I
              Briefing on Small Business Administration
              Year 2000 Readiness




Systems Renovated But Testing is Incomplete--Unit
Testing Performed

•   Performed unit testing to verify that software
    modules work as intended. Unit test results were
    •   validated by SBA programmers and technical
        contractor
    •   reviewed, validated, and accepted by users
•   OIG’ s technical contractor reviewed the results of
    about 30 unit tests
•   Renovations and unit testing were completed and
    software modules placed into production by
    OMB’ s October 1, 1998, deadline
                                                                                   10




              Page 19                                     GAO/AIMD-99-265 SBA Year 2000 Readiness
            Appendix I
            Briefing on Small Business Administration
            Year 2000 Readiness




Systems Renovated But Testing is Incomplete--
Testing Weaknesses Increase Risks

•   Although SBA completed renovation and unit testing,
    Y2K work for its mission-critical systems is not yet
    complete
     • Integration testing is incomplete
     • Systems acceptance testing is incomplete
     • Independent quality assurance (QA) or
       independent verification and validation (IV&V) is
       incomplete
•   As a result, SBA does not yet have reasonable
    assurance that systems are Y2K ready


                                                                                 11




            Page 20                                     GAO/AIMD-99-265 SBA Year 2000 Readiness
               Appendix I
               Briefing on Small Business Administration
               Year 2000 Readiness




Systems Renovated But Testing is Incomplete--
Testing Guidance

Testing recommended in GAO Y2K guidance
•   Integration test--verify that software modules,
    when combined, work together to meet specified
    requirements
•   System acceptance test--verify that completed
    systems are acceptable to users
•   Engage a QA or an IV&V group to
    •   ensure that tests are complete and accurate and
        test standards are met
    •   certify, in writing, that systems are Y2K ready
                                                                                    12




               Page 21                                     GAO/AIMD-99-265 SBA Year 2000 Readiness
             Appendix I
             Briefing on Small Business Administration
             Year 2000 Readiness




Systems Renovated But Testing is Incomplete--
Integration Testing Guidance

GAO Y2K guidance for integration testing
recommends
•   identifying business processes and developing
    corresponding test cases, data, and expected
    outcomes to test software that supports business
    processes
•   tracking test procedures, cases, and expected
    outcomes to business processes to ensure proper
    testing



                                                                                  13




             Page 22                                     GAO/AIMD-99-265 SBA Year 2000 Readiness
              Appendix I
              Briefing on Small Business Administration
              Year 2000 Readiness




Systems Renovated But Testing is Incomplete--
Integration Test Performed

SBA performed integration testing of its mission-
critical systems
•   Developed a single test plan to provide guidelines
    for testing every date-related path in the systems
    defined as mission-critical
•   SBA’ s test plan identified key business processes to
    be tested, including:
    •   loan approvals
    •   increase or decrease in loan amounts
    •   cancellation or reinstatement of loans
                                                                                   14




              Page 23                                     GAO/AIMD-99-265 SBA Year 2000 Readiness
               Appendix I
               Briefing on Small Business Administration
               Year 2000 Readiness




Systems Renovated But Testing is Incomplete--
Integration Test Performed (cont’ d.)

•   Integration tests were performed using
    •   all transactions for 1 day as data to test daily
        weekly, monthly, and some quarterly, semi-
        annual, and annual processing and reports
    •   test scripts and manually entered transactions
•   Tests were completed by the end of May 1999
•   Test results
    •   have been reviewed and validated by SBA
        programmers and technical contractor
    •   are being reviewed for validation and acceptance
        by users
                                                                                    15




               Page 24                                     GAO/AIMD-99-265 SBA Year 2000 Readiness
             Appendix I
             Briefing on Small Business Administration
             Year 2000 Readiness




Systems Renovated But Testing is Incomplete--
Integration Testing Weaknesses

SBA did not ensure that its key business processes
were tested
•   SBA did not carry out its test plan--specific test
    cases were not developed for most key business
    processes
•   SBA did not track integration test procedures,
    cases, and expected outcomes to verify that all
    key business processes were actually tested




                                                                                  16




             Page 25                                     GAO/AIMD-99-265 SBA Year 2000 Readiness
             Appendix I
             Briefing on Small Business Administration
             Year 2000 Readiness




Systems Renovated But Testing is Incomplete--
Integration Testing Weaknesses (cont’ d.)

•   An SBA official told us they believe testing
    systems with actual transactions provided adequate
    assurance that key business processes are Y2K-
    ready because SBA uses most of the processes on a
    daily basis to do its work
•   However, SBA could not provide information on
    which business processes have been tested or the
    extent of such testing
•   As a result, SBA lacks reasonable assurance that all
    systems supporting key business processes have
    been tested and will function beyond 1999
                                                                                  17




             Page 26                                     GAO/AIMD-99-265 SBA Year 2000 Readiness
              Appendix I
              Briefing on Small Business Administration
              Year 2000 Readiness




Systems Renovated But Testing is Incomplete--
System Acceptance Test Guidance

GAO guidance states that user groups’ participation
in all aspects of system acceptance testing is
essential to ensuring that systems perform
correctly. User groups should
•   review test plans and guidance
•   review test procedures and data
•   participate in the tests
•   acknowledge that systems perform correctly


                                                                                   18




              Page 27                                     GAO/AIMD-99-265 SBA Year 2000 Readiness
              Appendix I
              Briefing on Small Business Administration
              Year 2000 Readiness




Systems Renovated But Testing is Incomplete--
System Acceptance Test Performed

•   SBA combined system acceptance testing with
    integration testing of its mission-critical systems
•   Users are reviewing test results for validation and
    acceptance




                                                                                   19




              Page 28                                     GAO/AIMD-99-265 SBA Year 2000 Readiness
               Appendix I
               Briefing on Small Business Administration
               Year 2000 Readiness




Systems Renovated But Testing is Incomplete--
System Acceptance Testing Weaknesses

•   Except for tests of 3 systems, user groups did not
    •   review test plans or guidance
    •   review test procedures and data
    •   participate in tests
•   An SBA official told us that they believe user
    review and validation of test results provided
    adequate assurance that systems perform correctly




                                                                                    20




               Page 29                                     GAO/AIMD-99-265 SBA Year 2000 Readiness
             Appendix I
             Briefing on Small Business Administration
             Year 2000 Readiness




Systems Renovated But Testing is Incomplete--
System Acceptance Testing Weaknesses (cont’ d.)

•   However, without participation in all test
    processes, SBA does not have assurance that
    systems have been adequately tested and
    perform correctly
•   As a result, SBA lacks reasonable assurance that
    the systems will perform correctly beyond 1999




                                                                                  21




             Page 30                                     GAO/AIMD-99-265 SBA Year 2000 Readiness
             Appendix I
             Briefing on Small Business Administration
             Year 2000 Readiness




Systems Renovated But Testing is Incomplete--
IV&V Guidance

GAO guidance recommends engaging an
independent QA group or IV&V group to ensure that
mission-critical systems are adequately tested and
validated. Typically, these groups
•   review test plans
•   review test procedures and data
•   witness test execution
•   certify that systems are Y2K-ready


                                                                                  22




             Page 31                                     GAO/AIMD-99-265 SBA Year 2000 Readiness
             Appendix I
             Briefing on Small Business Administration
             Year 2000 Readiness




Systems Renovated But Testing is Incomplete--
IV&V Testing Weaknesses

•   SBA’ s work is being reviewed by its OIG, with
    technical assistance from a contractor
•   However, the review is not a complete IV&V
     • the OIG or its contractor did not witness
       integration and system acceptance tests
     • the OIG or its contractor are not validating or
       certifying that SBA systems are Y2K ready
•   As a result, SBA lacks reasonable assurance that
    testing has been adequately planned, conducted, and
    documented to help ensure that its systems will be
    Y2K ready
                                                                                  23




             Page 32                                     GAO/AIMD-99-265 SBA Year 2000 Readiness
              Appendix I
              Briefing on Small Business Administration
              Year 2000 Readiness




SBA Continuing to Address External Data
Exchanges

•   Sent letters to most data exchange partners
    beginning in 1998 to establish testing agreements
•   Reported testing or making testing agreements with
    38 of 58 data exchange partners
    •   28 federal agencies
    •   10 commercial entities
•   Tested exchanges with the largest data exchange
    partner, who accounts for about 90 percent of SBA’ s
    external data exchange activity


                                                                                   24




              Page 33                                     GAO/AIMD-99-265 SBA Year 2000 Readiness
              Appendix I
              Briefing on Small Business Administration
              Year 2000 Readiness




SBA Continuing to Address External Data
Exchanges (cont’ d.)

•   Pursuing testing or agreements with remaining
    external data exchange partners
    •   Made follow-up telephone calls to non-
        responding partners
    •   Sent additional letters in early 1999
•   SBA has not established a final completion date for
    data exchange work; finalizing testing agreements
    is contingent upon the cooperation of partners



                                                                                   25




              Page 34                                     GAO/AIMD-99-265 SBA Year 2000 Readiness
             Appendix I
             Briefing on Small Business Administration
             Year 2000 Readiness




Other Observations--SBA Performing Some End-to-
End Testing

•   GAO Y2K guidance recommends end-to-end testing
    to verify that a set of systems, including those
    external to the organization, interoperate as intended
    to support core business areas or functions
•   According to officials, SBA performed some end-to-
    end testing
     • using typical transactions processed from inception
       to completion during integration testing
     • testing internal data exchanges as part of its
       transaction processing during integration testing
     • testing external data exchanges separately
                                                                                  26




             Page 35                                     GAO/AIMD-99-265 SBA Year 2000 Readiness
              Appendix I
              Briefing on Small Business Administration
              Year 2000 Readiness




Other Observations--End-to-End Testing Not Yet
Complete

•   SBA’ s testing approach did not ensure that software
    supporting core business areas or functions, such as
    loan servicing, was tested
•   In addition, SBA’ s approach, using actual transactions,
    was not adequate because it leaves many potential
    issues, such as interoperability of internal and external
    applications, untested
•   As a result, SBA lacks reasonable assurance that the
    systems will work together to support SBA’ s core
    business functions beyond 1999

                                                                                   27




              Page 36                                     GAO/AIMD-99-265 SBA Year 2000 Readiness
             Appendix I
             Briefing on Small Business Administration
             Year 2000 Readiness




Conclusions


Because of weaknesses in its Y2K software testing,
and despite completing software repairs and
performing integration and acceptance testing of its
mission-critical systems, SBA lacks reasonable
assurance that
•   software supporting key business processes has
    been tested and will function beyond 1999;
•   systems will perform correctly beyond 1999;



                                                                                  28




             Page 37                                     GAO/AIMD-99-265 SBA Year 2000 Readiness
             Appendix I
             Briefing on Small Business Administration
             Year 2000 Readiness




Conclusions (cont’ d.)



•   testing was adequately planned, conducted, and
    documented to help ensure that its systems will
    be Y2K ready; and
•   its core business functions will be adequately
    supported and operate properly beyond 1999.




                                                                                  29




             Page 38                                     GAO/AIMD-99-265 SBA Year 2000 Readiness
             Appendix I
             Briefing on Small Business Administration
             Year 2000 Readiness




Recommendations


We therefore recommend that SBA continue to
conduct additional Y2K testing during 1999 to
eliminate test deficiencies and reduce risks of Y2K
failures. Specifically, SBA should
•   ensure that (1) its business processes have been
    adequately tested and validated, and (2) develop
    specific scripts and cases to test all key business
    processes that were not adequately tested;




                                                                                  30




             Page 39                                     GAO/AIMD-99-265 SBA Year 2000 Readiness
             Appendix I
             Briefing on Small Business Administration
             Year 2000 Readiness




Recommendations (cont’ d.)


•   ensure user participation in all phases of
    additional testing, including development of test
    scripts and cases, and validation that systems
    perform correctly;
•   obtain independent QA or IV&V of the testing
    and validation processes and results to certify
    that each mission-critical system is Y2K ready;
•   plan and conduct end-to-end testing, with data
    exchange partners, for each core business area
    or function.

                                                                                  31




             Page 40                                     GAO/AIMD-99-265 SBA Year 2000 Readiness
Appendix II

Objectives, Scope, and Methodology                                                           Appe
                                                                                                nIx
                                                                                                  Idi




              As requested by the Senate Committee on Small Business, our objectives
              were to verify (1) that SBA has completed repair and replacement work for
              its mission-critical systems and (2) the extent to which data exchanges
              with outside entities have been addressed.

              To meet our first objective, we reviewed SBA’s guidance for repairing its
              mission-critical systems, as well as the date conversion software program
              written to address the Y2K problem for most of those systems. We
              reviewed about 100 unit test plans and results, and technical and user
              validation of those test results. We reviewed SBA’s integration and system
              acceptance test plan, 61 test results, and technical and user validation of
              those results. We discussed SBA’s repair and replacement work, and its
              Y2K testing methodologies and results with SBA and its technical
              contractor, OIG officials, and the OIG’s technical contractor. To meet our
              second objective, we reviewed SBA’s inventory of external data exchanges.
              We also reviewed letters sent by SBA to its external data exchange
              partners, as well as the results of testing with SBA’s largest data exchange
              partner. We discussed data exchanges with SBA and OIG officials, and the
              OIG’s technical contractor.

              We performed our work at SBA’s headquarters in Washington, D.C., and at
              SBA’s Office of the Chief Financial Officer in Denver, Colorado. We
              requested and received comments on a draft of our briefing from SBA and
              incorporated those comments as appropriate. We performed our work
              from February through mid-July 1999 in accordance with generally
              accepted government auditing standards.




              Page 41                                GAO/AIMD-99-265 SBA Year 2000 Readiness
Appendix III

GAO Contacts and Staff Acknowledgements                                                             AppeInx
                                                                                                          Idi




GAO Contacts          David G. Gill (202) 512-6250




Acknowledgements      In addition to the individual named above, Martin E. Caulk, Mirko J. Dolak,
                      Phoebe Furey, and Elizabeth A. Roach, made key contributions to this
                      report.




(511784)       Lte
                 rt   Page 42                               GAO/AIMD-99-265 SBA Year 2000 Readiness
Ordering Information

The first copy of each GAO report and testimony is free.
Additional copies are $2 each. Orders should be sent to the
following address, accompanied by a check or money order made
out to the Superintendent of Documents, when necessary, VISA and
MasterCard credit cards are accepted, also.

Orders for 100 or more copies to be mailed to a single address are
discounted 25 percent.

Orders by mail:

U.S. General Accounting Office
P.O. Box 37050
Washington, DC 20013

or visit:

Room 1100
700 4th St. NW (corner of 4th and G Sts. NW)
U.S. General Accounting Office
Washington, DC

Orders may also be placed by calling (202) 512-6000
or by using fax number (202) 512-6061, or TDD (202) 512-2537.

Each day, GAO issues a list of newly available reports and
testimony. To receive facsimile copies of the daily list or any list
from the past 30 days, please call (202) 512-6000 using a touchtone
phone. A recorded menu will provide information on how to obtain
these lists.

For information on how to access GAO reports on the INTERNET,
send an e-mail message with “info” in the body to:

info@www.gao.gov

or visit GAO’s World Wide Web Home Page at:

http://www.gao.gov
United States                       Bulk Rate
General Accounting Office      Postage & Fees Paid
Washington, D.C. 20548-0001           GAO
                                 Permit No. GI00
Official Business
Penalty for Private Use $300

Address Correction Requested