oversight

Year 2000 Computing Challenge: Financial Management Service Has Established Effective Year 2000 Testing Controls

Published by the Government Accountability Office on 1999-10-29.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                 United States General Accounting Office

GAO              Report to the Chairman, Subcommittee
                 on Oversight, Committee on Ways and
                 Means, House of Representatives


October 1999
                 YEAR 2000
                 COMPUTING
                 CHALLENGE

                 Financial Management
                 Service Has
                 Established Effective
                 Year 2000 Testing
                 Controls




GAO/AIMD-00-24
Contents



Letter                                                                                   3


Appendixes   Appendix I: Briefing to House Committee on Ways and Means,
               Subcommittee on Oversight                                               10
             Appendix II:   Objectives, Scope, and Methodology                         64




             Page 1                          GAO/AIMD-00-24 FMS Year 2000 Testing Controls
Page 2   GAO/AIMD-00-24 FMS Year 2000 Testing Controls
United States General Accounting Office                                           Accounting and Information
Washington, D.C. 20548                                                                 Management Division



                                    B-282428                                                                         Leter




                                    October 29, 1999

                                    The Honorable Amo Houghton
                                    Chairman, Subcommittee on Oversight
                                    Committee on Ways and Means
                                    House of Representatives

                                    Dear Mr. Chairman:

                                    The Treasury Department’s Financial Management Service (FMS) annually
                                    disburses social benefit and other payments and collects revenue which in
                                    the aggregate is now about $2.7 trillion. FMS also manages and oversees
                                    the federal government’s central accounting and reporting systems that
                                    generate vital financial information used by congressional and executive
                                    agency decisionmakers. Consequently, it is essential for FMS’ mission-
                                    critical computer systems to operate correctly at and beyond
                                    January 1, 2000.

                                    At your request, we reviewed FMS’ Year 2000 program to determine
                                    whether FMS is (1) effectively managing its Year 2000 testing and (2) taking
                                    adequate steps to mitigate the Year 2000 risks associated with four mission-
                                    critical systems that were not implemented by the Office of Management
                                    and Budget’s (OMB) March 1999 deadline. On September 22, 1999, we
                                    briefed FMS’ Chief Information Officer (CIO) on our work results and later
                                    obtained FMS’ comments on this report. The CIO agreed with our results
                                    and conclusions. On October 4, 1999, we provided this briefing to your
                                    office. This report summarizes the information presented at that briefing.
                                    The briefing slides are included in appendix I and details of our scope and
                                    methodology are in appendix II. Our work was performed from February
                                    1999 through October 1999, in accordance with generally accepted
                                    government auditing standards.



Results in Brief                    FMS has established effective Year 2000 test management controls for its
                                    six most mission-critical systems. For instance, FMS developed test
                                    guidance, defined compliance criteria, and defined test roles and
                                    responsibilities. Together, these and other controls provided the
                                    infrastructure needed for planning, executing and reporting Year 2000 test
                                    activities, including system acceptance and end-to-end testing.




                                    Page 3                            GAO/AIMD-00-24 FMS Year 2000Testing Controls
B-282428




In line with our Year 2000 test guide,1 which is widely accepted and used in
government and private industry, FMS also engaged an Independent
Verification and Validation (IV&V) contractor to ensure that testing was
complete and thorough. We reviewed this contractor’s work and found that
(1) its scope was consistent with our Year 2000 test guide and (2) the
contractor identified no material problems with system acceptance testing
of five of FMS’ six most critical systems. 2 We also found that although the
IV&V contractor did not review the sixth system, 3 FMS took steps to gain
reasonable assurance that Year 2000 testing for this system was effectively
managed.

Further, FMS has established effective management controls in performing
its portion of selected Year 2000 end-to-end tests. Specifically, FMS
satisfied the end-to-end testing key processes defined in our guidance for
three critical test events. These events focused on three of FMS’ most
important core business functions−Social Security payments,
Supplemental Security Income (SSI) payments, and IRS tax refund
payments. The tests included FMS processing payment files from the Social
Security Administration (SSA) and Internal Revenue Service (IRS), printing
checks, and transmitting electronic payment files to Federal Reserve
Banks.

As of October 1, 1999, FMS reported that it had implemented two of the
four systems4 that did not meet the March 31, 1999, OMB-imposed deadline
for implementation. For the remaining two, FMS reported that it has
(1) renovated and tested both, (2) implemented both at two of five sites,
and (3) plans to complete implementation in early November 1999. In
addition, FMS has prepared and plans to test system contingency plans for
these late systems as well as its other mission-critical systems.




1
  Year 2000 Computing Crisis: A Testing Guide (GAO/AIMD-10.1.21, issued as an exposure
draft in June 1998; issued in final in November 1998).
2
 Social Security Administration (SSA) Payments, Supplemental Security Income (SSI)
Payments, Internal Revenue Service (IRS) Payments, STAR, and Government On-line
Accounting Link System (GOALS).
3
    Electronic Federal Tax Payment System (EFTPS).
4
 None of the four were among FMS’ six most mission-critical systems. Instead, FMS ranked
the four to be lower priority mission-critical systems.




Page 4                                   GAO/AIMD-00-24 FMS Year 2000Testing Controls
             B-282428




Background   FMS, a bureau of the Department of the Treasury, is the federal
             government’s financial manager. In this capacity, FMS has three primary
             functions: disburser, collector, and accountant of financial information.

             As a disburser for most federal agencies, FMS processed in fiscal year 1998
             over 860 million disbursements totaling over $1 trillion. These covered a
             wide variety of expenses, including social security and veterans benefit
             payments, IRS tax refunds, federal employee salaries, and vendor billings.

             As a collections agent, FMS is responsible for administering the world’s
             largest collections system. In fiscal year 1998, the government collected
             over $1.7 trillion from individual and corporate income tax deposits,
             customs duties, loan repayments, fines, and proceeds from leases, among
             other sources. FMS relies on a network of about 11,000 financial
             institutions to help collect these revenues.

             As an accountant, FMS operates and maintains the federal government’s
             central accounting and reporting systems to reconcile and keep track of the
             federal government’s assets, liabilities, receipts, and disbursements.
             Financial and budget execution information from these central systems is
             used by FMS to publish financial reports that are used by the Congress,
             OMB, and others who make financial decisions on behalf of the U.S.
             government.

             To accomplish many of these functions, FMS relies on six systems it
             considers its most mission critical:

             • The Social Security Administration (SSA) Payments system validates
               payment certification against payment file totals, performs edit
               checking, and generates and releases old-age and survivor social
               security payments.
             • The Supplemental Security Income (SSI) Payments system validates
               payment certification against payment file totals, performs edit
               checking, and generates and releases SSI payments.
             • The Internal Revenue Service (IRS) Payments system validates payment
               certification against payment file totals, performs edit checking, and
               generates and releases IRS tax refund payments.
             • The Electronic Federal Tax Payment System (EFTPS) collects, deposits,
               and accounts for taxes withheld by employers from individuals’ wages.
             • The Government On-line Accounting Link System (GOALS) is a
               commercial timesharing service comprised of 18 subsystems that




             Page 5                            GAO/AIMD-00-24 FMS Year 2000Testing Controls
                         B-282428




                           collect, edit, and communicate accounting and financial data to and
                           from federal program agency users.
                         • STAR maintains the Treasury’s central accounting system by aggregating
                           all transactions relating to the receipt and disbursement of government
                           funds.

                         Because of FMS’ heavy reliance on these systems, complete and thorough
                         testing is essential to provide reasonable assurance that they process dates
                         correctly and will not jeopardize FMS’ ability to perform core business
                         functions during and after transition to a Year 2000 computing
                         environment. Our Year 2000 test guide describes a structured and
                         disciplined approach for managing Year 2000 test activities.



FMS Established an       Establishing an effective organizational infrastructure for Year 2000 testing
                         provides the foundation for planning, execution, and reporting on each
Effective Year 2000      incremental phase of Year 2000 testing activities, including system
Testing Organizational   acceptance testing and end-to-end testing. FMS has established the 11
                         organizational infrastructure key processes that our test guide defines. For
Infrastructure           example, FMS (1) designated program- and project-level test managers for
                         its mission-critical systems, (2) developed and issued organizational Year
                         2000 test guidance, (3) defined Year 2000 compliance criteria, (4) defined
                         the test organization and its components’ roles and responsibilities,
                         (5) defined test facilities and Year 2000 reporting requirements, and
                         (6) employed a process for ensuring the Year 2000 compliance of vendor-
                         supported products and services.

                         In addition, FMS engaged an IV&V contractor to provide third-party
                         assurance that its testing of 22 of its most mission-critical systems was
                         performed effectively (i.e., that it met process and product standards). We
                         found that the IV&V contractor’s scope of work, as specified in the contract
                         between FMS and the contractor, was consistent with our test guide and
                         that the IV&V contractor performed according to the scope of work.




                         Page 6                             GAO/AIMD-00-24 FMS Year 2000Testing Controls
                         B-282428




FMS Employed             As specified in our test guide, system acceptance testing (SAT) verifies that
                         the entire system performs as intended. To determine how well FMS
Effective Management     managed SAT, we (1) selected the six mission-critical systems that FMS
Controls in Performing   identified as being the most important to supporting FMS’ central payment,
                         collections, and accounting functions and (2) determined whether the
Systems Acceptance       selected systems’ testing had been independently verified and validated
Testing                  and, if so, reviewed the results of the IV&V contractor’s work.

                         FMS’ IV&V contractor found no material problems with the SAT of five of
                         these systems (SSA Payments, SSI Payments, IRS Payments, STAR, and
                         GOALS) and concluded that FMS had effectively managed SAT. FMS did
                         not subject EFTPS to IV&V because the two commercial banks that operate
                         and maintain the system were subject to Year 2000 examinations by a
                         federal banking regulator−the Office of the Comptroller of the Currency
                         (OCC).

                         Nevertheless, FMS took other steps to ensure that SAT for EFTPS was
                         managed effectively. For example, FMS reviewed the two banks’ testing
                         progress monthly. FMS also required the banks to submit documentation
                         certifying the system’s Year 2000 compliance. In addition, OCC agreed to
                         review the banks’ progress on EFTPS during the regulator’s Year 2000
                         examinations and report any concerns to FMS. According to FMS, as of
                         October 1, 1999, OCC had performed several on-site Year 2000 reviews at
                         each bank, reported that both had made satisfactory progress, and raised
                         no issues to FMS.5



FMS Employed             End-to-end testing verifies that a set of interrelated systems, which
                         collectively support an organizational core business area or function,
Effective Management     interoperate properly in an operational environment. These interrelated
Controls in Performing   systems include not only those owned and managed by the organization but
                         also the external systems with which the organization interfaces, as well as
Its Portion of End-to-   the supporting telecommunications infrastructures.
End Test Events

                         5
                          We reviewed the Year 2000 oversight efforts of OCC and the other federal depository
                         institution regulators and found that they had developed and issued detailed Year 2000
                         guidelines for the institutions and performed extensive, periodic on-site examinations of
                         banks’ and other depository institutions’ Year 2000 efforts (e.g., see Year 2000 Computing
                         Crisis: Federal Depository Institution Regulators Are Making Progress, But Challenges
                         Remain (GAO/T-AIMD-98-305, Sept. 17, 1998)).




                         Page 7                                    GAO/AIMD-00-24 FMS Year 2000Testing Controls
                       B-282428




                       In its management of its portion of end-to-end test events for three critical
                       business functions (Social Security payments, SSI payments, and IRS tax
                       refund payments), FMS satisfied the end-to-end testing key processes
                       specified in our guide. For example, FMS worked with its test partners to
                       define the boundaries of these end-to-end tests, secured the commitment of
                       data exchange partners, used interorganizational test teams, prepared test
                       procedures and data, defined the expected results of each test, and
                       documented the test results. In addition, FMS confirmed the Year 2000
                       compliance of its vendor-supported telecommunications and
                       infrastructure.



FMS Is Reporting       OMB’s Year 2000 guidance, as amended in January 1998, requires that all
                       mission-critical systems be renovated, tested, and implemented by
Progress on Its Late   March 31, 1999, in order to allow enough time for agencies to ensure that
Mission-Critical       systems are running smoothly and to plan for unexpected failures. On that
                       date, FMS reported that seven mission-critical systems had not yet been
Systems                implemented. By June 1999, FMS reported that it had implemented three of
                       these systems. As of October 1, 1999, FMS reported that it had
                       implemented two of the four remaining systems. With respect to the
                       remaining two systems, both of which FMS ranked as lower priority
                       mission-critical systems, FMS reported that it had (1) renovated and tested
                       both, (2) implemented both at two of five sites, and (3) planned to complete
                       implementation in early November 1999. In addition, FMS reports that it
                       had prepared and planned to test system contingency plans for these late
                       systems as well as its other mission-critical systems.



Conclusion             FMS has effectively managed the Year 2000 testing of its most critical
                       payment, collection, and accounting systems. While this does not
                       guarantee that Year 2000-induced disruptions will not occur, it should
                       significantly reduce FMS’ risk of internal system failures.

                       We are sending copies of this report to Representative William Coyne,
                       Ranking Minority Member of your Subcommittee; Representatives Bill
                       Archer, Chairman, and Charles Rangel, Ranking Minority Member, House
                       Committee on Ways and Means; Senators William Roth, Chairman, and
                       Daniel P. Moynihan, Ranking Minority Member, Senate Committee on
                       Finance; Senators Fred Thompson, Chairman, and Joseph Lieberman,
                       Ranking Minority Member, Senate Committee on Governmental Affairs;
                       Representatives Dan Burton, Chairman, and Henry Waxman, Ranking



                       Page 8                             GAO/AIMD-00-24 FMS Year 2000Testing Controls
B-282428




Minority Member, House Committee on Government Reform; and
Representatives Steven Horn, Chairman, and Jim Turner, Ranking Minority
Member, Subcommittee on Government Management, Information and
Technology, House Committee on Government Reform.

We are also sending copies to the Honorable Lawrence H. Summers,
Secretary of the Treasury; the Honorable Richard Gregg, Commissioner,
Financial Management Service; the Honorable Kenneth S. Apfel,
Commissioner, Social Security Administration; the Honorable
Charles O. Rossotti, Commissioner of Internal Revenue; the Honorable
John Koskinen, Chair, the President’s Council on Year 2000 Conversion; and
the Honorable Jacob Lew, Director, Office of Management and Budget. We
will send copies to others upon request.

If you have any questions, please contact me or Gary Mountjoy, Assistant
Director, at (202) 512-6240 or via e-mail at hiter.aimd@gao.gov or
mountjoyg.aimd@gao.gov. Other major contributors to this work were
Bernard Anderson, Timothy Hopkins, Richard Hung, and Sabine Paul.

Sincerely yours,




Randolph C. Hite
Associate Director
Governmentwide and Defense
 Information Systems




Page 9                            GAO/AIMD-00-24 FMS Year 2000Testing Controls
Appendix I

Briefing to House Committee on Ways and
Means, Subcommittee on Oversight                                                     Appendx
                                                                                           Ii




                        Accounting and Information
                        Management Division


                Treasury Department’s Financial
                 Management Service (FMS) Has
                 Established Effective Year 2000
                        Testing Controls

             Briefing to House Committee on Ways and
                  Means, Subcommittee on Oversight
                           October 4, 1999



                                                                      1




                        Page 10           GAO/AIMD-00-24 FMS Year 2000Testing Controls
                Appendix I
                Briefing to House Committee on Ways and
                Means, Subcommittee on Oversight




                 Briefing Overview


•   Objectives, Scope, and Methodology
•   Results in Brief
•   Background
•   Key Findings
•   Conclusions




                                                                                      2




                Page 11                                   GAO/AIMD-00-24 FMS Year 2000Testing Controls
                  Appendix I
                  Briefing to House Committee on Ways and
                  Means, Subcommittee on Oversight




                  Objectives, Scope and Methodology

The Subcommittee asked us to determine whether
• FMS is effectively managing its Year 2000 testing.
  Specifically, has FMS
   • implemented an effective organizational infrastructure for
     Year 2000 testing,
   • employed effective management controls in performing
     system acceptance testing (SAT) of selected systems, and
   • employed effective management controls in performing
     selected end-to-end test events.


• FMS is adequately mitigating the Year 2000 risks
  associated with four mission-critical systems that did
  not meet OMB’s 3/31/99, implementation deadline.
                                                                                        3




                  Page 12                                   GAO/AIMD-00-24 FMS Year 2000Testing Controls
                        Appendix I
                        Briefing to House Committee on Ways and
                        Means, Subcommittee on Oversight




                        Objectives, Scope and Methodology
                        (cont’d)
To address the first objective, we used applicable criteria
from our test guide, Year 2000 Computing Crisis: A
Testing Guide (GAO/AIMD-10.1.21, November 1998):

                        Applicable GAO Year 2000 Testing Guide Criteria



   Testing       •Assign test management authority and responsibility; define
Infrastructure   compliance criteria; secure test resources; issue test guidance

   System        •Schedule and plan tests; prepare test procedures and data; define
 Acceptance
                 exit criteria; execute tests and document results; correct defects
   Testing

                 •Define test boundaries; schedule and plan tests; prepare test
  End-to-End     procedures and data; define exit criteria; execute tests; document
   Testing       results; correct defects

                                                                                              4




                        Page 13                                   GAO/AIMD-00-24 FMS Year 2000Testing Controls
                Appendix I
                Briefing to House Committee on Ways and
                Means, Subcommittee on Oversight




                Objectives, Scope and Methodology
                (cont’d)
Objective 1(a): To assess FMS’ Year 2000 testing
  organizational infrastructure, we

• analyzed the institutional management structures and
  controls (organizations, policies, guidance,
  standards) used by FMS to perform Year 2000
  testing and

• compared them to the 11 key processes in our guide
  to identify any variances, their causes, and impacts.


                                                                                      5




                Page 14                                   GAO/AIMD-00-24 FMS Year 2000Testing Controls
               Appendix I
               Briefing to House Committee on Ways and
               Means, Subcommittee on Oversight




                Objectives, Scope and Methodology
                (cont’d)
Objective 1(b): To evaluate the management of
  selected systems’ testing, we
• determined whether the selected systems’ testing
  had been independently verified and validated (IV&V)
  and, if so, we reviewed FMS’ IV&V agent’s plans and
  results.
• reviewed, for any selected system whose testing was
  not independently verified and validated, the
  management control and oversight steps that FMS
  took to assure itself that the system had been
  adequately tested.

                                                                                     6




               Page 15                                   GAO/AIMD-00-24 FMS Year 2000Testing Controls
                Appendix I
                Briefing to House Committee on Ways and
                Means, Subcommittee on Oversight




                 Objectives, Scope, and Methodology
                 (cont’d)
• We selected the six mission-critical systems identified
  by FMS as being the most crucial to supporting FMS’
  central payment, collections, and accounting
  functions. Testing of all but one was reviewed by
  FMS’ IV&V agent.




                                                                                      7




                Page 16                                   GAO/AIMD-00-24 FMS Year 2000Testing Controls
                                                          Appendix I
                                                          Briefing to House Committee on Ways and
                                                          Means, Subcommittee on Oversight




                                                            Objectives, Scope, and Methodology
                                                            (cont’d)

S y ste m                               D e s c r ip t io n
S S A P aym ents                        S S A P a y m e n ts i s u se d t o is su e o v e r 5 0 0 m il li o n p a y m e n t s (ro u g h l y 6 0 p e rc e n t o f F M S ’ p a y m e n ts )
                                        re p r e se n t in g a b o u t $ 3 3 6 b il li o n a n n u a ll y . T h i s s y s te m v a li d a te s p a y m e n t c e rti fi c a ti o n a g a in s t
                                        p a y m e n t fi le t o ta l s , p e r fo rm s e d it c h e c k i n g , a n d g e n e ra t e s a n d r e le a s e s o ld a g e a n d su rv i v o r S o c ia l
                                        S e c u r it y p a y m e n t s .

S SI P a y m e n t s                    S S I P a y m e n t s i s u se d t o iss u e a b o u t 8 0 m il lio n p a y m e n ts (ro u g h l y 1 0 p e rc e n t o f F M S ’ p a y m e n ts )
                                        re p r e se n t in g a b o u t $ 2 4 b il li o n a n n u a ll y . T h i s s y s te m v a li d a te s p a y m e n t c e rti fi c a ti o n a g a in s t
                                        p a y m e n t f il e to t a ls , p e r fo rm s e d i t c h e c k i n g , a n d g e n e ra t e s a n d r e le a s e s su p p l e m e n t a l s e c u ri ty
                                        in c o m e (S S I) p a y m e n t s.

IR S P a y m e n t s                    In f is c a l y e a r 1 9 9 8 , I R S P a y m e n ts w a s u s e d to i ss u e 9 1 m il li o n p a y m e n t s (a b o u t 1 0 p e rc e n t o f
                                        F M S ’ p a y m e n t s) t o ta l in g a b o u t $ 1 3 7 b il li o n . T h is s y ste m v a li d a te s p a y m e n t c e rti fi c a ti o n a g a in s t
                                        p a y m e n t fi le t o ta l s , p e rfo r m s e d it c h e c k in g , a n d g e n e ra te s a n d re l e a s e s I R S t a x re f u n d p a y m e n t s.

E l e c tr o n ic F e d e r a l T a x   I n fis c a l y e a r 1 9 9 8 , E FT P S w a s u s e d t o c o ll e c t a b o u t $ 1 .1 tr il li o n o r n e a rl y 6 0 % o f t h e
P a y m e n t S y ste m                 g o v e rn m e n t ’s t o ta l c o ll e c t io n s. T h is sy s t e m c o l le c t s, d e p o si ts a n d a c c o u n ts f o r t a x e s w ith h e ld b y
(E F T P S )                            e m p l o y e rs f ro m i n d iv i d u a ls ' w a g e s. E F T P S is r e p la c i n g th e c u r re n t m a n u a l p a p e r- b a se d F e d e ra l
                                        T a x D e p o si t/ T re a su r y T a x & L o a n S y ste m .

G o v e rn m e n t O n -li n e          G O A L S is a c o m m e rc i a l t im e sh a ri n g se rv i c e c o m p ris e d o f 1 8 su b - sy s te m s t h a t c o l le c t , e d it a n d
A c c o u n ti n g L i n k S y s te m   tra n s m i t d a t a to a n d f ro m F e d e r a l p r o g ra m a g e n c y (F P A ) u s e rs . It s u p p o rts th e g a t h e ri n g o f
(G O A L S )                            m a n d a to ry F P A a c c o u n ti n g d a ta t h a t is u t il iz e d t o sa t isf y s ta t u to r y re p o rt in g re q u i re m e n ts (e .g .,
                                        M o n t h ly T re a s u ry S ta t e m e n t , U n i te d S t a te s G o v e r n m e n t C o n s o li d a te d F i n a n c ia l S ta t e m e n t ).

STAR                                    S T A R is th e a u t o m a t e d sy st e m t h a t m a in t a in s t h e T re a su r y 's c e n tr a l a c c o u n ti n g sy s te m b y
                                        a g g re g a tin g a ll t ra n s a c ti o n s re l a ti n g to t h e re c e i p t a n d d i s b u rs e m e n t o f g o v e rn m e n t f u n d s .




                                                                                                                                                                                               8




                                                          Page 17                                                                      GAO/AIMD-00-24 FMS Year 2000Testing Controls
                Appendix I
                Briefing to House Committee on Ways and
                Means, Subcommittee on Oversight




                 Objectives, Scope, and Methodology
                 (cont’d)
Objective 1(c): To assess the management of selected
  end-to-end test events, we
• selected three completed test events (SSA
  Payments, SSI Payments, and IRS Payments) due to
  their impact on the public,
• analyzed the management structures and controls
  (organizations, policies, guidance, standards) used
  by FMS to manage and perform end-to-end testing
  for these events and compared them to the 11 key
  processes in our guide to identify any variances, their
  causes, and impacts, and

                                                                                      9




                Page 18                                   GAO/AIMD-00-24 FMS Year 2000Testing Controls
               Appendix I
               Briefing to House Committee on Ways and
               Means, Subcommittee on Oversight




               Objectives, Scope, and Methodology
               (cont’d)

• did not analyze the management structures and
  controls used by the other end-to-end test
  participants (SSA, IRS, and FRB).




                                                                                    10




               Page 19                                   GAO/AIMD-00-24 FMS Year 2000Testing Controls
                 Appendix I
                 Briefing to House Committee on Ways and
                 Means, Subcommittee on Oversight




                 Objectives, Scope, and Methodology
                 (cont’d)
Objective 2: To assess risk mitigation efforts for the four
  systems that missed OMB’s implementation
  milestone, we
• determined the current status of each system,
• determined each system’s purpose and mission
  significance,
• analyzed plans and schedules for completing each
  system’s outstanding Year 2000 activities, and
• analyzed FMS’ plans and activities for identifying and
  managing each system’s risks and assessed
  progress in implementing risk mitigation strategies.

                                                                                      11




                 Page 20                                   GAO/AIMD-00-24 FMS Year 2000Testing Controls
                Appendix I
                Briefing to House Committee on Ways and
                Means, Subcommittee on Oversight




                Objectives, Scope, and Methodology
                (cont’d)

• We coordinated our work with the Treasury IG, who is
  assessing the effectiveness of FMS’ Year 2000
  business continuity and contingency planning.

• We performed our work from February through
  September 1999 in accordance with generally
  accepted government auditing standards.




                                                                                     12




                Page 21                                   GAO/AIMD-00-24 FMS Year 2000Testing Controls
                  Appendix I
                  Briefing to House Committee on Ways and
                  Means, Subcommittee on Oversight




                   Results in Brief
• Objective 1: FMS is effectively managing its Year
  2000 testing for its most critical payment, collection,
  and accounting systems:
   – FMS has established the Year 2000 testing organizational
     infrastructure key processes specified in our test guide;
   – FMS has implemented an IV&V process which is consistent
     with our guide, and the IV&V agent followed this process in
     performing its work;
   – FMS’ IV&V agent found no material problems with system
     acceptance testing of SSA Payments, SSI Payments, IRS
     Payments; STAR; and GOALS and concluded that this
     testing was managed effectively;
   – FMS took steps to provide itself assurance that Year 2000
     testing for EFTPS had been effectively managed; and
                                                                                       13




                  Page 22                                   GAO/AIMD-00-24 FMS Year 2000Testing Controls
                  Appendix I
                  Briefing to House Committee on Ways and
                  Means, Subcommittee on Oversight




                  Results in Brief (cont’d)
   – FMS has satisfied our end-to-end testing key processes on
     three test events--Social Security payments, Supplemental
     Security Income payments, and IRS income tax refunds.


• Objective 2: As of September 1, 1999, FMS had
  implemented two of the four late systems. FMS had
  also (1) renovated and tested the remaining two late
  systems, (2) implemented them at two of five sites,
  and (3) planned to complete their implementation in
  October 1999. In addition, FMS prepared and
  planned to test system contingency plans for these
  late systems as well as its other mission-critical
  systems.
                                                                                       14




                  Page 23                                   GAO/AIMD-00-24 FMS Year 2000Testing Controls
                   Appendix I
                   Briefing to House Committee on Ways and
                   Means, Subcommittee on Oversight




                   Background
• The Financial Management Service, a bureau of the
  Treasury Department, is the federal government’s
  financial manager.
• In this capacity, FMS has three primary functions:

   – central disburser,
   – collections agent, and
   – accountant/reporter of financial information.




                                                                                        15




                   Page 24                                   GAO/AIMD-00-24 FMS Year 2000Testing Controls
                    Appendix I
                    Briefing to House Committee on Ways and
                    Means, Subcommittee on Oversight




                    Background (cont’d)
• As a “central disburser,” FMS makes disbursements
  for most federal agencies.

    – For fiscal year 1998, FMS reported processing over
      860 million disbursements totaling over $1 trillion for a wide
      variety of expenses, including Social Security and veterans
      benefit payments, IRS tax refunds, federal employee
.     salaries, and vendor billings.




                                                                                         16




                    Page 25                                   GAO/AIMD-00-24 FMS Year 2000Testing Controls
                   Appendix I
                   Briefing to House Committee on Ways and
                   Means, Subcommittee on Oversight




                    Background (cont’d)
• As a “collections agent,” FMS is also responsible for
  administering the world’s largest collections system.

    – In fiscal year 1998, the government collected over
      $1.7 trillion from sources such as individual and corporate
      income tax deposits, customs duties, loan repayments, fines,
      and proceeds from leases.
    – FMS relies on a network of about 11,000 financial institutions
.     to help collect these revenues.




                                                                                        17




                   Page 26                                   GAO/AIMD-00-24 FMS Year 2000Testing Controls
                   Appendix I
                   Briefing to House Committee on Ways and
                   Means, Subcommittee on Oversight




                    Background (cont’d)
• As an “accountant,” FMS operates and maintains the
  federal government’s central accounting and
  reporting systems to reconcile and keep track of the
  federal government’s assets, liabilities, receipts, and
  disbursements.

    – Financial and budget execution information from these
.     central systems is used by FMS to publish financial reports
      that are used by the Congress, OMB, and others who make
      financial decisions on behalf of the U.S. government.




                                                                                        18




                   Page 27                                   GAO/AIMD-00-24 FMS Year 2000Testing Controls
                  Appendix I
                  Briefing to House Committee on Ways and
                  Means, Subcommittee on Oversight




                   Background (cont’d)
• To provide its services, FMS relies on a wide array of
  geographically dispersed information systems. For
  example,

   – FMS has data centers at six regional centers that support its
     payment functions.
   – FMS also uses a network of contractors and Federal
     Reserve Banks to help carry out its other financial
     management responsibilities.




                                                                                       19




                  Page 28                                   GAO/AIMD-00-24 FMS Year 2000Testing Controls
                             Appendix I
                             Briefing to House Committee on Ways and
                             Means, Subcommittee on Oversight




                              Objective 1(a):
                              Test Organizational Infrastructure Key
                              Processes Satisfied

                                    G A O K ey Processes                                         Satisfied? (Y/N )
1.   Assign Y ear 2 000 te st m a nag em e nt authority and respo nsib ility                            Y
2.   De fin e Yea r 2 000 com plia nce c riteria                                                        Y
3.   D e velo p org aniz atio nal Y ear 20 00 test and ev alu atio n m aster plan (TE M P )             Y
4.   En gag e the qu ality a ssu ranc e/ verifica tion and valida tion g rou p                          Y
5.   D e fine a nd sec ure test bud gets                                                                Y
6.   Estab lish n ew or au gm ent ex isting test e nviron m en ts and sched ule th eir use              Y
7.   D eve lop an d issu e org aniz atio nal Y ear 20 00 test guid anc e                                Y
8.   E sta blish pro cesses and info rm a tion so urc es to su ppo rt testers an d activitie s          Y
9.   Pro vid e for en suring Y e ar 200 0 co m plianc e o f ven dor-sup ported produ cts an d           Y
     serv ice s
1 0. Establish pro cesses a nd m e trics for rep ortin g test a ctiv ity an d pro gress                 Y
1 1. E stab lish a libra ry of test tools                                                               Y




                                                                                                                     20




                             Page 29                                                    GAO/AIMD-00-24 FMS Year 2000Testing Controls
                Appendix I
                Briefing to House Committee on Ways and
                Means, Subcommittee on Oversight




                 Objective 1(a):
                 Test Organizational Infrastructure Key
                 Process 1 Satisfied
• Year 2000 test management authority, responsibility,
  and accountability should be assigned at both the
  program and project levels.

• FMS designated two test managers for its mission-
  critical IT systems. These managers are responsible
  for coordinating and overseeing testing across
  platforms to ensure critical testing is performed on a
  priority basis, as well as to manage the best use of
  available resources. Automated Information System
  Project Managers are responsible for testing at the
  project level.
                                                                                     21




                Page 30                                   GAO/AIMD-00-24 FMS Year 2000Testing Controls
                  Appendix I
                  Briefing to House Committee on Ways and
                  Means, Subcommittee on Oversight




                   Objective 1(a):
                   Test Organizational Infrastructure Key
                   Process 2 Satisfied

• Year 2000 compliance criteria should be defined.

• FMS defined Year 2000 compliance criteria in its
  Year 2000 Testing Guidance. For example,
  according to the criteria,

   – “date-based functionality must behave consistently for dates
     prior to, during, and after Year 2000. Manipulations of date
     data need to be reliable/correct only over the range of dates
     that an application was designed to process.”


                                                                                       22




                  Page 31                                   GAO/AIMD-00-24 FMS Year 2000Testing Controls
                  Appendix I
                  Briefing to House Committee on Ways and
                  Means, Subcommittee on Oversight




                   Objective 1(a):
                   Test Organizational Infrastructure Key
                   Process 3 Satisfied
• An organizational Year 2000 test and evaluation
  master plan should be developed.
• FMS defined its Year 2000 test and evaluation
  master plan in several documents. For example,
  FMS
   – defined the test organization and its components’ roles and
     responsibilities in its Year 2000 Compliance Methodology
     Document;
   – developed a master schedule of high-level test activities for
     each system/project in its Schedule for Year 2000 Platform
     Availability
   – defined its test facilities and Year 2000 testing reporting
     requirements in its Year 2000 Testing Guide.
                                                                                       23




                  Page 32                                   GAO/AIMD-00-24 FMS Year 2000Testing Controls
                  Appendix I
                  Briefing to House Committee on Ways and
                  Means, Subcommittee on Oversight




                  Objective 1(a):
                  Test Organizational Infrastructure Key
                  Process 4 Satisfied
• The quality assurance/verification and validation
  group should be engaged.

• On August 24, 1998, FMS engaged a contractor to
  conduct IV&V. The contractor’s scope included 22
  high priority systems. The IV&V contractor’s role was
  to
   – review and, if necessary, provide assistance in the
     development of test plans;
   – evaluate established test criteria, including systems
     acceptance tests (SAT) and associated test data, to ensure
     they are comprehensive;

                                                                                       24




                  Page 33                                   GAO/AIMD-00-24 FMS Year 2000Testing Controls
                Appendix I
                Briefing to House Committee on Ways and
                Means, Subcommittee on Oversight




                 Objective 1(a):
                 Test Organizational Infrastructure Key
                 Process 4 Satisfied (cont’d)
– act as an independent observer during the test and
  certification processes; and
– evaluate and validate test results prior to certification.




                                                                                     25




                Page 34                                   GAO/AIMD-00-24 FMS Year 2000Testing Controls
                Appendix I
                Briefing to House Committee on Ways and
                Means, Subcommittee on Oversight




                Objective 1(a):
                Test Organizational Infrastructure Key
                Process 5 Satisfied

• Test budgets should be defined and secured.

• FMS defined and secured a Year 2000 program
  budget through the century date change. FMS
  budgeted about $45 million to convert and test its
  systems from FY 1997 through FY 2000.




                                                                                     26




                Page 35                                   GAO/AIMD-00-24 FMS Year 2000Testing Controls
               Appendix I
               Briefing to House Committee on Ways and
               Means, Subcommittee on Oversight




                Objective 1(a):
                Test Organizational Infrastructure Key
                Process 6 Satisfied

• New test environments should be established or
  existing ones should be augmented.

• FMS established two Year 2000 test environments for
  its mainframe systems--one in which HourGlass 2000
  software is used to simulate advancement of the
  operating system date, and the other in which the
  operating system date is actually advanced. It also
  established test environments for its mid-level and
  personal computer systems.

                                                                                    27




               Page 36                                   GAO/AIMD-00-24 FMS Year 2000Testing Controls
               Appendix I
               Briefing to House Committee on Ways and
               Means, Subcommittee on Oversight




                Objective 1(a):
                Test Organizational Infrastructure
                Key Process 7 Satisfied

• Organizational Year 2000 test guidance should be
  developed and issued.

• FMS developed and issued organizational Year 2000
  test guidance via its March 1999 Year 2000 Testing
  Guidance. FMS used our testing guide to develop its
  guidance.




                                                                                    28




               Page 37                                   GAO/AIMD-00-24 FMS Year 2000Testing Controls
                Appendix I
                Briefing to House Committee on Ways and
                Means, Subcommittee on Oversight




                Objective 1(a):
                Test Organizational Infrastructure Key
                Process 8 Satisfied

• Processes (e.g., configuration management, risk
  management, etc.) and information sources (e.g.,
  intranet web site containing Year 2000 test
  requirements, lessons learned data base, etc.) to
  support testers and test activities should be
  established.

• FMS established processes and information sources
  for configuration management, Year 2000
  certification, risk management, and quality
  assurance.
                                                                                     29




                Page 38                                   GAO/AIMD-00-24 FMS Year 2000Testing Controls
                Appendix I
                Briefing to House Committee on Ways and
                Means, Subcommittee on Oversight




                Objective 1(a):
                Test Organizational Infrastructure Key
                Process 9 Satisfied
• Year 2000 compliance of vendor-supported products
  and services should be ensured.

• FMS ensured that vendors’ products and services
  (including hardware, operating systems software and
  utilities, application software, telecommunications
  equipment and lines) for its internal systems were
  compliant by (1) obtaining vendor certification of its
  products’ and services’ Year 2000 compliance and
  (2) validating vendors’ certifications through testing.



                                                                                     30




                Page 39                                   GAO/AIMD-00-24 FMS Year 2000Testing Controls
                 Appendix I
                 Briefing to House Committee on Ways and
                 Means, Subcommittee on Oversight




                Objective 1(a):
                Test Organizational Infrastructure Key
                Process 9 Satisfied
• For systems operated by contractors, FMS required
  the contractors to attest to the Year 2000 compliance
  of their vendors’ products and services as part of their
  certification documentation.




                                                                                      31




                 Page 40                                   GAO/AIMD-00-24 FMS Year 2000Testing Controls
                Appendix I
                Briefing to House Committee on Ways and
                Means, Subcommittee on Oversight




                 Objective 1(a):
                 Test Organizational Infrastructure
                 Key Process 10 Satisfied

• Processes and metrics for reporting test activity and
  progress should be established.

• FMS’ Year 2000 Special Project Office established
  processes and metrics for reporting testing activity
  and progress. For example, project managers submit
  monthly reports to the project office that detail,
  among other things, whether system renovation,
  testing and implementation milestones have been
  completed and the number of lines of code tested.

                                                                                     32




                Page 41                                   GAO/AIMD-00-24 FMS Year 2000Testing Controls
                  Appendix I
                  Briefing to House Committee on Ways and
                  Means, Subcommittee on Oversight




                   Objective 1(a):
                   Test Organizational Infrastructure
                   Key Process 11 Satisfied

• A library of test tools should be established.

• FMS established a library of test tools. It includes
   – HourGlass 2000 used to simulate advancement of the
     system operating date;
   – CA-Endevor for configuration management;
   – CA-Datamacs/II which assists in creating test data sets.




                                                                                       33




                  Page 42                                   GAO/AIMD-00-24 FMS Year 2000Testing Controls
                   Appendix I
                   Briefing to House Committee on Ways and
                   Means, Subcommittee on Oversight




                   Objective 1(b):
                   IV&V Agent Found That Acceptance Testing
                   on Five Systems Was Managed Effectively

• Our test guide recommends establishing IV&V for test
  activities as a key process for developing an effective
  testing infrastructure. Through IV&V of testing, an
  independent third party group generally reviews test plans
  and procedures, observes execution of the tests, and
  reviews test results to ensure that test criteria (guidance,
  plans, standards) are satisfied.

• FMS’ Year 2000 Testing Guidance and its IV&V contract
  statement of work specify the contractor’s responsibilities
  and define an IV&V process that is consistent with our
  guide.
                                                                                        34




                   Page 43                                   GAO/AIMD-00-24 FMS Year 2000Testing Controls
                    Appendix I
                    Briefing to House Committee on Ways and
                    Means, Subcommittee on Oversight




                    Objective 1(b):
                    IV&V Agent Found That Acceptance Testing
                    on Five Systems Was Managed Effectively
• FMS employed IV&V on five of our selected systems:
  SSA, SSI, and IRS Payments, GOALS, and STAR.*

• We found that the IV&V agent followed FMS’ IV&V
  requirements and did not report material problems.

• The IV&V agent concluded that FMS had effectively
  managed acceptance testing, including testing of
  interfaces.
  -------------------------
   *IV&V was also employed on 17 other FMS mission-critical systems.


                                                                                         35




                    Page 44                                   GAO/AIMD-00-24 FMS Year 2000Testing Controls
                Appendix I
                Briefing to House Committee on Ways and
                Means, Subcommittee on Oversight




                 Objective 1(b):
                 For EFTPS, FMS Took Other Steps to
                 Ensure SAT Was Managed Effectively

• FMS did not select EFTPS for IV&V because the two
  financial agents (commercial banks) who operate and
  maintain the system are examined by a federal
  banking regulator (the Office of the Comptroller of the
  Currency (OCC)) pursuant to Federal Financial
  Institutions Examination Council Year 2000 guidance.

• Nevertheless, FMS obtained and reviewed the
  agents’ testing progress via monthly meetings with
  the two banks.


                                                                                     36




                Page 45                                   GAO/AIMD-00-24 FMS Year 2000Testing Controls
                Appendix I
                Briefing to House Committee on Ways and
                Means, Subcommittee on Oversight




                Objective 1(b):
                For EFTPS, FMS Took Other Steps to
                Ensure SAT Was Managed Effectively
• OCC agreed to review the agents’ progress on
  EFTPS during the regulator’s Year 2000
  examinations and report any concerns to FMS.
  According to FMS, OCC has performed several on-
  site Year 2000 reviews, reported that both agents
  have made satisfactory progress, and has raised no
  issues to FMS.
• FMS also required the agents to complete FMS’
  process for certification. This process includes
  having bank senior executives attest in writing that
  EFTPS is Year 2000 compliant and was successfully
  and comprehensively tested.
                                                                                     37




                Page 46                                   GAO/AIMD-00-24 FMS Year 2000Testing Controls
                Appendix I
                Briefing to House Committee on Ways and
                Means, Subcommittee on Oversight




                Objective 1(b):
                For EFTPS, FMS Took Other Steps to
                Ensure SAT Was Managed Effectively
• The banks submitted their certification documentation
  to FMS in March 1999, and it has been reviewed and
  approved by FMS.




                                                                                     38




                Page 47                                   GAO/AIMD-00-24 FMS Year 2000Testing Controls
                            Appendix I
                            Briefing to House Committee on Ways and
                            Means, Subcommittee on Oversight




                             Objective 1(c):
                             End-to-End Testing Key Processes
                             Satisfied for FMS’ Portion of the Tests


                            GAO Key Processes                                    Satisfied? (Y/N)

1. Define the system boundaries of the end-to-end test(s)                               Y
2. Secure the commitment of key data exchange partners                                  Y
3. Establish an interorganizational end-to-end test team                                Y
4. Confirm Year 2000 compliance of vendor-supported telecommunications and              Y
   other infrastructure(s)
5. Schedule and plan the end-to-end test(s)                                             Y
6. Prepare end-to-end test procedures and data                                          Y
7. Define end-to-end test exit criteria                                                 Y
8. Execute end-to-end test(s)                                                           Y
9. Document end-to-end test results                                                     Y
10.Correct defects                                                                      Y
11.Ensure that end-to-end test exit criteria are met                                    Y




                                                                                                    39




                            Page 48                                   GAO/AIMD-00-24 FMS Year 2000Testing Controls
                  Appendix I
                  Briefing to House Committee on Ways and
                  Means, Subcommittee on Oversight




                   Objective 1(c): End-to-End Testing Key
                   Processes Satisfied for FMS’ Portion of
                   the Tests--Key Process 1
• The system boundaries of end-to-end tests should be
  defined.
• FMS worked with its test partners to define the
  boundaries of their end-to-end tests. For example,
   – for the SSA and SSI payment core business functions, the
     partners agreed that the test would encompass the following
     business subfunctions: (1) SSA transmitting payment
     requests to FMS, (2) FMS processing the requests and then
     printing checks with post-Jan. 1, 2000, dates or transmitting
     electronic payment files to the Federal Reserve Banks
     (FRBs), (3) for the latter, FRBs processing direct deposit
     files and transmitting them electronically to commercial
     banks; and (4) FMS transmitting its processing results to
     SSA.
                                                                                       40




                  Page 49                                   GAO/AIMD-00-24 FMS Year 2000Testing Controls
                  Appendix I
                  Briefing to House Committee on Ways and
                  Means, Subcommittee on Oversight




                   Objective 1(c): End-to-End Testing Key
                   Processes Satisfied for FMS’ Portion of
                   the Tests--Key Process 1 (cont’d)
– for the IRS refund payment core business function, the
  partners agreed that the test would encompass the following
  business subfunctions: FMS (1) processing IRS payment
  requests and (2) transmitting electronic payment files to an
  FRB.*




 ---------------------
  *This test did not include IRS’ generation and transmission
  of payment request files to FMS. IRS officials stated that
  IRS plans to test this function with FMS but has not yet
  established a test date.
                                                                                       41




                  Page 50                                   GAO/AIMD-00-24 FMS Year 2000Testing Controls
                Appendix I
                Briefing to House Committee on Ways and
                Means, Subcommittee on Oversight




                Objective 1(c): End-to-End Testing Key
                Processes Satisfied for FMS’ Portion of
                the Tests--Key Process 2
• Commitment of key data exchange partners should
  be secured.

• FMS secured SSA, IRS, and the Federal Reserve’s
  commitment to participate in the end-to-end tests that
  were planned and successfully conducted.




                                                                                     42




                Page 51                                   GAO/AIMD-00-24 FMS Year 2000Testing Controls
                Appendix I
                Briefing to House Committee on Ways and
                Means, Subcommittee on Oversight




                          Objective 1(c): End-to-End Testing
                          Key Processes Satisfied for FMS’
                          Portion of the Tests--Key Process 3
• An interorganizational end-to-end test team should
  be established.

• FMS and other test participants (SSA, IRS, Federal
  Reserve) built upon their existing working
  relationships and interorganizational teams to assign
  and share roles and responsibilities for the planning,
  execution, and reporting of the end-to-end tests.
  Under this arrangement, each participant was
  responsible for, among other things, ensuring that
  exit criteria were met for its portion of the test.

                                                                                     43




                Page 52                                   GAO/AIMD-00-24 FMS Year 2000Testing Controls
               Appendix I
               Briefing to House Committee on Ways and
               Means, Subcommittee on Oversight




                Objective 1(c): End-to-End Testing Key
                Processes Satisfied for FMS’ Portion of
                the Tests--Key Process 4
• Year 2000 compliance of vendor-supported
  telecommunications and other infrastructure should
  be confirmed.

• Consistent with the shared leadership approach,
  FMS confirmed Year 2000 compliance of its vendor-
  supported telecommunications and other
  infrastructure via FMS’ certification process.




                                                                                    44




               Page 53                                   GAO/AIMD-00-24 FMS Year 2000Testing Controls
                 Appendix I
                 Briefing to House Committee on Ways and
                 Means, Subcommittee on Oversight




                 Objective 1(c): End-to-End Testing Key
                 Processes Satisfied for FMS’ Portion of
                 the Tests--Key Process 5
• End-to-end tests should be scheduled and planned.

• For the three end-to-end test events, FMS defined
  and documented test schedules, data to be used,
  anticipated results, interfaces to be tested, roles and
  responsibilities for performing key tasks, etc.




                                                                                      45




                 Page 54                                   GAO/AIMD-00-24 FMS Year 2000Testing Controls
                Appendix I
                Briefing to House Committee on Ways and
                Means, Subcommittee on Oversight




                 Objective 1(c): End-to-End Testing Key
                 Processes Satisfied for FMS’ Portion of
                 the Tests--Key Process 6
• End-to-end test procedures and data should be
  prepared.

• In addition to test plans, FMS developed test
  procedures or “scripts” that detailed the steps to be
  followed and the functions to be tested during the
  end-to-end test events. SSA and IRS prepared and
  provided the test data to FMS.




                                                                                     46




                Page 55                                   GAO/AIMD-00-24 FMS Year 2000Testing Controls
                Appendix I
                Briefing to House Committee on Ways and
                Means, Subcommittee on Oversight




                 Objective 1(c): End-to-End Testing Key
                 Processes Satisfied for FMS’ Portion of
                 the Tests--Key Process 7
• End-to-end test exit criteria should be defined.

• For each end-to-end test, FMS defined exit criteria as
  100% success. FMS also documented the expected
  results of the test.




                                                                                     47




                Page 56                                   GAO/AIMD-00-24 FMS Year 2000Testing Controls
               Appendix I
               Briefing to House Committee on Ways and
               Means, Subcommittee on Oversight




                Objective 1(c): End-to-End Testing Key
                Processes Satisfied for FMS’ Portion of
                the Tests--Key Process 8
• End-to-end tests should be executed.

• End-to-end tests for SSA, SSI, and IRS payments
  were conducted between June 1998 and December
  1998 in accordance with the test plans.




                                                                                    48




               Page 57                                   GAO/AIMD-00-24 FMS Year 2000Testing Controls
               Appendix I
               Briefing to House Committee on Ways and
               Means, Subcommittee on Oversight




                Objective 1(c): End-to-End Testing Key
                Processes Satisfied for FMS’ Portion of
                the Tests--Key Process 9
• End-to-end test results should be documented.

• FMS’ Year 2000 Testing Guidance requires end-to-
  end test results to be documented and to be reported
  to management via monthly status reports.
  Accordingly, FMS documented the test results along
  with expected results.




                                                                                    49




               Page 58                                   GAO/AIMD-00-24 FMS Year 2000Testing Controls
                Appendix I
                Briefing to House Committee on Ways and
                Means, Subcommittee on Oversight




                 Objective 1(c): End-to-End Testing Key
                 Processes Satisfied for FMS’ Portion of
                 the Tests--Key Process 10
• Defects identified during tests should be corrected.

• No defects were identified during end-to-end tests for
  IRS Payments.
• Defects were identified and corrected during end-to-
  end tests for SSA and SSI Payments.




                                                                                     50




                Page 59                                   GAO/AIMD-00-24 FMS Year 2000Testing Controls
                Appendix I
                Briefing to House Committee on Ways and
                Means, Subcommittee on Oversight




                Objective 1(c): End-to-End Testing Key
                Processes Satisfied for FMS’ Portion of
                the Tests--Key Process 11
• Organization should ensure end-to-end test exit
  criteria are met.

• According to FMS, it ensured exit criteria were met
  on its segments of the tests by having test
  participants verify whether FMS file formats, test
  results, and input/output data met exit criteria.




                                                                                     51




                Page 60                                   GAO/AIMD-00-24 FMS Year 2000Testing Controls
              Appendix I
              Briefing to House Committee on Ways and
              Means, Subcommittee on Oversight




              Objective 2: Two Remaining Late Systems
              Are to Be Implemented in October 1999

• OMB’s guidance, as amended in January 1998,
  requires that agencies complete implementation of
  their mission-critical systems by March 31, 1999.

• As of March 31, 1999, FMS reported that seven
  mission-critical systems had not been implemented.
  At the time of the Subcommittee’s request, FMS
  reported that it had implemented three of these
  systems, leaving four to be completed.




                                                                                   52




              Page 61                                   GAO/AIMD-00-24 FMS Year 2000Testing Controls
                 Appendix I
                 Briefing to House Committee on Ways and
                 Means, Subcommittee on Oversight




                 Objective 2: Two Remaining Late
                 Systems Are to Be Implemented in
                 October 1999 (cont’d)

• As of September 1, 1999, FMS reported that it had
  implemented two of the four late systems. For the two
  remaining systems, FMS reported that it had
  (1) renovated and tested the systems, (2) implemented
  them at two of five sites, and (3) planned to complete
  their implementation in October 1999. In addition, FMS
  prepared and planned to test in the fall system
  contingency plans for these late systems as well as its
  other mission-critical systems .




                                                                                      53




                 Page 62                                   GAO/AIMD-00-24 FMS Year 2000Testing Controls
               Appendix I
               Briefing to House Committee on Ways and
               Means, Subcommittee on Oversight




                Conclusion

• FMS has effectively managed the Year 2000 testing of
  its most critical payment, collection, and accounting
  systems.




                                                                                    54




               Page 63                                   GAO/AIMD-00-24 FMS Year 2000Testing Controls
Appendix II

Objectives, Scope, and Methodology                                                                      Appendx
                                                                                                              Ii




              Our objectives were to determine whether FMS is (1) effectively managing
              its Year 2000 testing and (2) taking adequate steps to mitigate the Year 2000
              risks associated with four mission-critical systems that were not
              implemented by OMB’s March 1999 deadline.

              To address the first objective, we assessed whether FMS had
              (1) implemented an effective organizational infrastructure for Year 2000
              testing, (2) employed effective management controls in performing system
              acceptance testing of selected systems, and (3) employed effective
              management controls in performing selected end-to-end tests.

              To assess organizational infrastructure, we analyzed FMS’ institutional
              management structure and controls (organizations, policies, guidance, and
              standards) used to perform Year 2000 testing. We compared these
              structures and controls against the 11 key processes in our Year 2000 test
              guidance1 to identify variances, their causes, and impacts.

              To evaluate the management of the selected systems’ acceptance testing,
              we first selected six key systems to review. These systems were selected
              because they are the most mission-critical systems that support FMS’ three
              central functions (payments, collections, and accounting). For payments,
              we selected the three systems that process the largest dollar volume and
              process payment transactions related to public financial well-being.2 For
              collections, we selected the system that collects the vast majority of the
              government’s revenue.3 For accounting, we selected the two systems that,
              among other things, are central to FMS meeting its statutory mandate of
              preparing an annual consolidated financial statement for the federal
              government.4 FMS officials agreed that our selected systems were its most
              important systems.




              1
                Year 2000 Computing Crisis: A Testing Guide (GAO/AIMD-10.1.21, issued as an exposure
              draft in June 1998; issued in final in November 1998).
              2
               According to FMS, these systems−the Social Security Administration (SSA) Payments,
              Supplemental Security Income (SSI) Payments, and Internal Revenue Service (IRS)
              Payments systems−issue annual disbursements totaling about $497 billion.
              3
               Electronic Federal Tax Payment System (EFTPS) processed the collection of tax receipts
              totaling about $1.1 trillion in 1998.
              4
                  These systems are the Government On-line Accounting Link System (GOALS) and STAR.




              Page 64                                  GAO/AIMD-00-24 FMS Year 2000Testing Controls
                   Appendix II
                   Objectives, Scope, and Methodology




                   We then determined whether the selected systems’ testing had been
                   independently verified and validated and, if so, we compared the IV&V
                   contractor’s scope of work (as specified in the contract between FMS and
                   the contractor) to our guidance and then compared the actual work to
                   FMS’ IV&V requirements to ensure that the contractor’s work was complete
                   and thorough. This was the case for SSA, SSI, and IRS payments; GOALS;
                   and STAR. For the system whose testing was not independently verified
                   and validated (EFTPS), we reviewed the management control and
                   oversight steps that FMS took to assure itself that the system had been
                   adequately tested.

                   To assess the management of selected end-to-end tests, we selected three
                   completed test events (SSA payments, SSI payments, and IRS tax refunds)
                   pertaining to FMS’ core business functions that are essential to its ability to
                   meet its mission goals. We then analyzed the management structures and
                   controls that FMS used to manage and perform end-to-end testing for these
                   events and compared them to the 11 key processes in our Year 2000 test
                   guidance to identify any variances, their causes, and impacts. We did not
                   analyze the management structures and controls used by the other end-to-
                   end test participants (SSA, IRS, and the Federal Reserve).

                   To assess the risk mitigation efforts for the four mission-critical systems
                   that missed OMB’s March 31, 1999, implementation date, we (1) determined
                   the current status of each system, (2) determined each system’s purpose
                   and mission significance, (3) analyzed FMS’ plans and schedules for
                   completing outstanding Year 2000 activities, and (4) analyzed FMS’ plans
                   and activities for identifying and managing each system’s risks and
                   assessed progress in implementing risk mitigation strategies.

                   We coordinated our work with the Department of the Treasury’s Office of
                   the Inspector General, which is conducting a concurrent review of FMS’
                   Year 2000 business continuity and contingency planning.

                   We conducted our work at the Financial Management Service in
                   Washington, D.C., and Hyattsville, Maryland. We performed our work from
                   February 1999 through October 1999 in accordance with generally
                   accepted government auditing standards.




(511138)   Leter   Page 65                              GAO/AIMD-00-24 FMS Year 2000Testing Controls
Ordering Information

The first copy of each GAO report and testimony is free.
Additional copies are $2 each. Orders should be sent to the
following address, accompanied by a check or money order made
out to the Superintendent of Documents, when necessary, VISA and
MasterCard credit cards are accepted, also.

Orders for 100 or more copies to be mailed to a single address are
discounted 25 percent.

Orders by mail:

U.S. General Accounting Office
P.O. Box 37050
Washington, DC 20013

or visit:

Room 1100
700 4th St. NW (corner of 4th and G Sts. NW)
U.S. General Accounting Office
Washington, DC

Orders may also be placed by calling (202) 512-6000
or by using fax number (202) 512-6061, or TDD (202) 512-2537.

Each day, GAO issues a list of newly available reports and
testimony. To receive facsimile copies of the daily list or any list
from the past 30 days, please call (202) 512-6000 using a touchtone
phone. A recorded menu will provide information on how to obtain
these lists.

For information on how to access GAO reports on the INTERNET,
send an e-mail message with “info” in the body to:

info@www.gao.gov

or visit GAO’s World Wide Web Home Page at:

http://www.gao.gov
United States                       Bulk Rate
General Accounting Office      Postage & Fees Paid
Washington, D.C. 20548-0001           GAO
                                 Permit No. GI00
Official Business
Penalty for Private Use $300

Address Correction Requested