Financial Management: Defining Requirements Is Crucial to System 90's Success

Published by the Government Accountability Office on 1990-09-05.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                        IJnited States General Accounting   Office

  GAO                   Report to the Secretary of the Treasury

: Nt~pl.twrtwr 1!I!)0
                        Defining Requirements
                        Is Crucial to
                        System 90’s Success



                   United States
GAO                General Accounting Office
                   Washington, D.C. 20548

                   Accounting and Financial
                   Management Division


                   September 5, 1990

                   The Honorable Nicholas F. Brady
                   The Secretary of the Treasury

                   Dear Mr. Secretary:

                   This letter provides the results of our review of the Financial Manage-
                   ment Service’s efforts to initiate and define the requirements for System
                   90, a long-term strategy for modernizing and integrating its financial
                   management activities, and System 90’s first application, the Payments,
                   Claims, and Enhanced Reconciliation (PACER)system. We undertook our
                   review at an early stage in the development of System 90 and PACER,
                   prior to the start of the detailed design effort, so that the Service could
                   address any identified concerns before they adversely affected the
                   system’s implementation and operation.

                   As envisioned by the Service, System 90 could offer significant benefits
                   to the Department of the Treasury, other federal agencies, and the recip-
                   ients of federal payments, primarily by improving the timeliness and
                   security of the Service’s operations through increased use of telecommu-
                   nications and easier access to data. Efforts such as this are important to
                   help the government modernize its financial management systems.
                   While we support the development of System 90, it is important that the
                   Service follow federal system development guidelines to reduce the
                   associated risk and to ensure development of a system that meets man-
                   agement’s needs in a cost-effective manner and that contains adequate

                   We found that although the Service performed many of the analyses
Results in Brief   stipulated in federal system development guidance, the Service has not
                   (1) clearly defined the system capabilities that will be developed during
                   phase 1 of System 90’s development or (2) adequately analyzed the
                   related costs and benefits. The uncertainty associated with not clearly
                   defining the project’s scope is compounded because the Service has not
                   yet finished refining and clarifying the PACERfunctional and internal
                   control requirements. We believe that to help avoid misunderstandings,
                   oversights, and potentially expensive future design alterations, the Ser-
                   vice must complete PACER'Sfunctional and internal control requirements
                   before the contractor begins developing the detailed system design.

                   Page1                                               GAO/AFMD-90435
                                                                                   System 90

                         The Financial Management Service is a bureau of the Department of the
Background               Treasury whose mission is to promote the financial integrity of the U. S.
                         government through sound money management on the public’s behalf.
                         The Service’s major responsibilities include issuing federal payments,
                         reconciling payments issued with checks cashed, and resolving claims
                         resulting from reports of government checks being forged, lost, stolen,
                         or destroyed.

                         The Service’s seven regional financial centers issue 86 percent of all fed-
                         eral government payments, including Social Security and veterans’ bene-
                         fits, vendor payments, federal salary and retirement payments, and tax
                         refunds. Over 800 federal civilian agency offices send payment requests
                         to the regional financial centers for processing. During fiscal year 1989,
                         the Service issued 769 million payments valued at over $1 trillion in the
                         form of checks and electronic funds transfers and resolved about
                         1.7 million check claims.

Plans for a New System   In developing System 90, the Service is undertaking a major effort to
                         improve its operations with new computer hardware and software and
                         the increased use of data telecommunications. By reducing reliance on
                         cumbersome paper-based procedures and improving data accessibility,
                         System 90 is to improve the efficiency of the Service’s operations and
                         service to the public.

                         Phase 1 of System 90, which is to be fully implemented by the mid-
                         1990s is to include the (1) acquisition of new hardware, (2) develop-
                         ment of the PACERsystem, and (3) integration of hardware and software
                         with a telecommunications network linking regional financial centers
                         with Service headquarters, federal agencies, and the Federal Reserve
                         System. During phase 2, the Service anticipates adding other applica-
                         tions, such as central accounting, to the System 90 hardware and tele-
                         communications processing environment. However, other than including
                         an add-on capability in the System 90 design, the Service has not devel-
                         oped specific plans or funding requirements for phase 2.

                         PACERwill replace many of the Service’s existing processes for verifying
                         agency payment requests, issuing checks, reconciling payments issued
                         with checks cashed and electronic fund transfers executed, and
                         resolving claims. The Service believes that the PACERsystem, as the first
                         application of System 90, will substantially improve the existing

                         Page2                                               GAO/AFMD-90-86
                                                                                         System 90

                          processes and benefit not only the Service, but also other federal agen-
                          cies and the recipients of federal payments. For example, the new
                          system is to

                      . ease system maintenance by replacing the Service’s incompatible com-
                        puter hardware and nonstandard software with one uniform hardware
                        and software architecture;
                      . increase the timeliness and efficiency of payment processing by
                        replacing paper and magnetic tape submissions with electronic
                      9 improve security by implementing electronic certification and message
                        authentication capabilities that will reduce the Service’s reliance on
                        handwritten signatures of agency certifying officials; and
                      l reduce the time associated with claims resolution from weeks to days
                        primarily by providing easier access to information on whether checks
                        have been cashed and other claims-related information.

Current Status of         Federal guidance divides system development efforts into consecutive
Development Efforts       stages: initiation, definition, detailed design, programming, testing, and
                          operation. For PACER,the Service has completed the initiation stage,
                          when existing deficiencies and general needs are identified and alterna-
                          tive approaches considered, and is nearing the end of the definition
                          stage, when system functions and performance, security, interface, and
                          data requirements are defined.

                          The Service is currently evaluating contract proposals for the acquisi-
                          tion of the System 90 hardware, development of a detailed PACERdesign,
                          and subsequent programming, testing, and implementation. Service offi-
                          cials plan to award a contract by September 1990. According to the
                          request for proposals (RF?), development of the first application, PACER,
                          is to be completed within 18 months. If the contract is awarded as
                          expected, the Service plans to begin implementing an operational system
                          at the regional financial centers in April 1993. Completing the imple-
                          mentation at all centers is expected to take more than a year.

                          According to the RFP,the System 90 hardware will be purchased on a
                          firm-fixed-price basis, while development of the PACERapplication
                          software will be priced on a cost-plus-award-fee basis. This means that
                          the contract will stipulate specific prices for hardware purchased but
                          only estimates, subject to change, for software development. The needed


                        telecommunications services are to be provided by the Federal Telecom-
                        munications System (FTS) 2000, a newly implemented system that pro-
                        vides the federal government with long-distance telecommunications
                        services under a General Services Administration contract.

                        The Service is also evaluating proposals for a second contract for an
                        “independent verification and validation” of System 90 to ensure that
                        System 90 and PACERmeet the Service’s objectives. The contract is to be
                        awarded under an agreement with the General Service Administration’s
                        Federal Systems Integration and Management Center, which also
                        assisted the Service in defining System 90 and PACERrequirements.

                        The objectives of our review were to determine whether the Service’s
Objectives, Scope,and   initial efforts to develop System 90 and PACERwere in conformance with
Methodology             federal guidance and how any lack of conformance we identified
                        increased the risk that system development and implementation efforts
                        would not be successful. Specifically, we wanted to determine if the Ser-
                        vice had performed initiation and definition stage tasks in accordance
                        with guidance developed by the Department of Commerce’s National
                        Institute of Standards and Technology (NET). NEST(formerly the National
                        Bureau of Standards) is authorized by the Brooks Act (Public Law 89-
                        306 or 40 U.S.C. 759, as amended) and the Computer Security Act of
                        1987 (Public Law 100-235) to set standards and provide guidance to
                        agencies to improve the use of federal automated information systems
                        and to assure the cost-effective security and privacy of the sensitive
                        data that they contain.

                        We also considered guidance issued by the Office of Management and
                        Budget (OMB), GAO,the President’s Council on Management Improve-
                        ment, and the President’s Council on Integrity and Efficiency. This gui-
                        dance provides more detailed explanations of how federal agencies
                        should develop and evaluate security and internal control requirements
                        and plan for new systems. Appendix I lists specific documents, including
                        NIST Federal Information Processing Standards (FIPS),pertinent to our

                        Because the System 90 and PACERdesign will not be operational for sev-
                        eral years, we did not attempt to predict whether the new system will be
                        completed according to schedule, operate as envisioned, or contain ade-
                        quate internal controls. In addition, our objectives did not include an
                        assessment of the RFP'Stechnical specifications for hardware, including

                        Page4                                              GAO/AFMD-SO-35

                           whether the project was biased toward a specific vendor’s equipment, or
                           an assessment of the Service’s proposal evaluation process.

                           To assess the Service’s conformance with federal system development
                           guidance, we examined documentation such as the System 90 and PACER
                           RFPand related analyses and project plans. To clarify our understanding
                           of the System 90 and PACERdevelopment effort, we interviewed appro-
                           priate officials in the Service and in Treasury’s Office of the Inspector
                           General and Office of Information Resources Management. To determine
                           how System 90 related to the Service’s overall financial management
                           improvement plans, we discussed the Service’s Financial Management
                           Systems Five Year Plan with Treasury and OMBofficials.

                           Our work was conducted between October 1989 and May 1990 at the
                           Service’s headquarters in Washington, D.C., and Hyattsville, Maryland.
                           Our review was performed in accordance with generally accepted gov-
                           ernment auditing standards. The Department of the Treasury provided
                           written comments on a draft of this report. These comments are
                           presented in appendix III.

                           According to NISTguidance, the initiation stage consists of the identifica-
Initiation Stage           tion of a need by the user organization, an assessment of risks to assets
Analyses Are               and resources, and an exploration of alternative solutions, including a
Incomplete and Poorly      comparison of the expected costs and benefits. After management
                           selects the best alternative, the project continues through the remaining
Documented                 stages of system development. Treasury and Service procedures reflect
                           NET'S guidance on the initiation stage analyses. Although the Service
                           performed most of the initiation stage analyses specified by NIST, (1) the
                           resulting documents were inconsistent in describing the planned capabil-
                           ities for phase 1 of System 90 and (2) the related costs and benefits were
                           not fully analyzed. For this reason, it is difficult to determine how
                           System 90 will operate, what benefits will be realized from phase 1, or
                           the system’s governmentwide cost.

Inconsistencies in         The key documents that the Service prepared do not clearly or consis-
Documentation of Planned   tently describe the planned capabilities for phase 1 of System 90. When
                           a system is developed in multiple phases, it is important to clearly
Capabilities               define which capabilities are to be developed first and which are being
                           considered for development in a later phase. We could not determine
                           from the available documentation if several capabilities mentioned in
                           the September 1987 requirements analysis, June 1988 project plan, and

                           Page6                                               GAO/AFMD-90-N

                             August 1989 cost/benefit analysis were still to be included in the System
                             90 and PACERdesign. We discussed these capabilities with the Service
                             officials to determine which had been deferred, which had been deleted,
                             and which were still planned for development during phase 1. Examples
                             of capabilities that were not clearly or consistently described are dis-
                             cussed below.

                         l A digital imaging system’ to facilitate retrieval of check images is dis-
                           cussed in the requirements analysis, project plan, and cost/benefit anal-
                           ysis. Service officials and the RFPstated that this capability will not be
                           developed until phase 2.
                         . The requirements analysis and cost/benefit analysis describe three data
                           bases (cash management, credit administration, and claimant profile)
                           that are to be accessible to federal agencies as part of System 90. A Ser-
                           vice official told us that the first two data bases will not be developed
                           until phase 2 and that the third may never be developed.
                         . The requirements analysis, project plan, cost/benefit analysis, and RFP
                           state that agency accounting services, which the Service currently pro-
                           vides to some federal agencies via other systems, will be incorporated
                           under System 90. However, these documents do not clearly state when
                           this capability will be developed. Service officials told us that agency
                           accounting services will be incorporated into or accessed through
                           System 90 after PACERdevelopment.
                         9 The ability to “warehouse,” or store, and precisely time vendor pay-
                           ments is to result in significant cash management savings, according to
                           key planning documents and Service officials. However, this capability
                           is not described in the PACERdata flow diagrams that are included in the
                           RIT. Service officials told us that they still plan to include this capability
                           in the PACERdesign during phase 1 but need to define more specific
                           payment-timing requirements.

Costs and Benefits Not       The Service’s System 90 cost/benefit analysis omits certain costs and
Fully Analyzed               does not clearly match others with specific system features which will
                             result in benefits. Consequently, the analysis provides limited informa-
                             tion for determining which aspects of the System 90 and PACERdesign
                             are likely to be the most cost beneficial.

                             ‘The RFP defines a digital imaging system as “an automated system designed to produce and convert
                             hardcopy images into digital images that can be stored on high density media, displayed on high
                             resolution workstations, accessedand distributed by communications networks, and otherwise
                             manipulated by a computer system.”

                             Page6                                                               GAO/AFlKD-3O-SSSystemBO
According to FIPS64, the purpose of a cost/benefit analysis is to “pro-
vide managers, users, designers, and auditors with adequate cost and
benefit information to analyze alternative approaches.” FIPS64 also rec-
ommends that system developers perform sensitivity analyses to assess
the extent to which costs and benefits will vary if factors such as
system requirements, workload mix, and equipment configuration

The Service developed a cost/benefit analysis in 1987 and revised and
updated it in August 1989. This analysis cites net benefits of $146 mil-
lion over a lo-year system life (system development and operation from
fiscal years 1990 to 1999). These savings are reported to be $28 million2
when converted to constant dollars (to eliminate increases due to infla-
tion) and discounted to present value (to accommodate the time-value of
money).3 According to this analysis, benefits will begin to accrue to the
Service, the U. S. Treasury, and other federal agencies in fiscal year
1994, after PACERimplementation. The primary benefits expected over
the system’s life (through 1999) are reduced claims processing, the elim-
ination of early payments, the replacement of microfilmed check images
with digital check images, and reduced maintenance costs.

Although the analysis includes both nonrecurring and recurring benefits
on a governmentwide basis, costs included are generally limited to those
that will be incurred by the Service. In addition, these data are not com-
plete or readily verifiable. We identified potentially significant costs
that had been omitted, and we could not determine how significant cost
and benefit figures had been developed because few supporting calcula-
tions were available.

We attempted to assess the reasonableness of some costs presented by
comparing them to estimates contained in planning documents and in a
February 1989 alternatives analysis prepared by a Service contractor.
This analysis, which primarily addresses alternatives related to the geo-
graphical distribution of processing and data, provides detailed esti-
mates of many of the costs that are to be included in a cost/benefit
analysis. Based on this comparison and on discussions with Service offi-

2We identified and corrected errors in the present value factors applied. This increased net benefits
by about $10 million to $38 million.
3Present value is the estimated current worth of future benefits It is determined by estimating the
return that could be realized if the funds were used for the best available investment instead of for
the alternative being considered. In accordance with OMB Circular A-94, “Discount Rates to Be Used
ln Evaluating Time-Distributed Costs and Benefits,” the Service used a lo-percent discount rate to
determine the present value of figures presented in the System 90 cost/benefit analysis.

Page7                                                                   GAO/AFMD-90-85

                                    cials, we identified the following potentially significant costs that were
                                    not included in the System 90 cost/benefit analysis:

                                l recurring telecommunications costs, which the alternatives analysis esti-
                                  mates at about $7 million per year for the Service alone;
                                l telecommunications costs that will be incurred by the federal agencies
                                  who transmit data to the Service (no estimate was readily available);
                                . site and facilities costs, estimated to total about $8 million through fiscal
                                  year 1993 according to a 1989 Service facilities financial plan;
                                . contractor assistance with system definition and verification at a cost of
                                  $4.2 million; and
                                l costs associated with the creation and storage of digital check images
                                  (no estimate was readily available).

                                    System 90 project officials agreed that the cost/benefit analysis was
                                    incomplete. They explained that the analysis was prepared primarily to
                                    support the Service’s request for a delegation of procurement authority
                                    from the General Services Administration and that, for this reason, they
                                    intended to include only costs that the Service would incur for the
                                    System 90 contract.

Relation BetweenSpecificCosts       The System 90 cost/benefit analysis describes an array of new capabili-
and Benefits Unclear                ties that are to speed payment and claims processing, facilitate access to
                                    information, and increase system security and reliability. However,
                                    because supporting calculations and sensitivity analyses were incom-
                                    plete, the relationship between specific system features and expected
                                    benefits is not apparent. For example, the analysis does not

                                . clearly differentiate between the costs and benefits attributable to those
                                  PACERrequirements currently under development and planned for
                                  funding and those that may be added or
                                l assess the impact of possible alterations in (1) system design, such as
                                  the elimination or deferral of a planned feature like payment ware-
                                  housing, (2) configuration, such as a reduction in the number of regional
                                  financial centers, or (3) development and implementation plans, such as
                                  the deferral of major system segments in order to implement others
                                  sooner than currently planned.

                                    Because it has not clearly matched specific system features and antici-
                                    pated benefits, the Service does not have a basis for readily assessing
                                    the impact of any modifications to the system’s design or implementa-
                                    tion plans that may be proposed during the remainder of the develop-
                                    ment effort. For example, in response to a request by OMB,the Service is

                                    Page8                                                GAO/AFMD30-36
                   assessing the effect of reducing the number of regional financial centers,
                   an option that could have been considered earlier and, if selected, incor-
                   porated into the System 90 design definition. In addition, a clearer pres-
                   entation of costs and related benefits would provide the Service and
                   others, such as Treasury’s Office of Information Resources Management
                   and OMB,a basis for monitoring the project’s progress and success.

                   At the close of our review in May 1990, the System 90 software design
Definition of      team was still defining PACERfunctional and data requirements in an
F’unctionaland     effort to ensure the completeness and accuracy of this documentation,
Internal Control   which will be used by the contractor to develop a detailed system
                   design. Team members and project officials told us that they were not
Requirements       including a comprehensive analysis of security and internal control
Incomplete         requirements as part of this effort because the Service plans to review
                   control needs during a later stage of the system’s development.

                   Guidance provided by NET, OMB,and GAOstates that detailed definitions
                   of functional and internal control requirements should be completed
                   before a project’s detailed design stage begins. These requirements are
                   to provide a basis for mutual understanding between users and
                   designers regarding system functions, performance requirements, data
                   inputs and outputs, data characteristics, failure contingencies, equip-
                   ment, interfaces, security, and control requirements. A primary respon-
                   sibility for ensuring that requirements are complete and sufficiently
                   detailed lies with those who will use the new system. It is most cost
                   beneficial to fully define and verify functional and control requirements
                   as early as possible because alterations become more difficult as the
                   development effort progresses.

                   Service officials have recognized that a complete list of PACERfunctional
                   and internal control requirements will be needed not only to provide
                   direction to system designers, but also to facilitate a planned indepen-
                   dent “verification and validation” review of the System QO/PAcERdesign.
                   This planned review is to include development of a “requirements trace-
                   ability matrix” as a tool to ensure that all system requirements are met.
                   The matrix is to list and describe all system requirements, including
                   their source and objective. System 90 project officials told us that
                   internal control requirements would be included in the list. An indepen-
                   dent contractor is then to use this list to trace requirements to the
                   system’s design and, ultimately, to the operational system. At the close
                   of our review, the Service had not yet awarded a contract for the inde-
                   pendent verification and validation effort.

                   Page9                                               GAO/AFMD9O-S6
                                                                                  System 90

Service Efforts to Fully   Although the Service included narrative, data flow diagrams, and a data
Define System Are in       dictionary outlining PACERrequirements in the System 90 RFP,issued on
                           March 16,1989, the Service has continued to modify and supplement
Progress                   these requirements. In an effort to eliminate any errors, omissions, or
                           ambiguities in the documentation prior to contract award and the subse-
                           quent detailed design stage, a group of about 26 Service personnel,
                           referred to as the Software Design Team, has been analyzing various
                           aspects of the PACERdesign. Deficiencies in the requirements definition
                           could impede the contractor’s ability to develop PACERwithin the 18-
                           month period envisioned by the Service. Specifically, the team is exam-
                           ining assumptions, further defining specific requirements, identifying
                           missing data elements, examining the relationship between automated
                           and manual procedures, raising concerns regarding the feasibility of cer-
                           tain system objectives, and substantially expanding the design

                           Based on discussions with team members and an examination of their
                           expanded documentation, we determined that the team has identified or
                           further defined

                           the need for additional data to support some of the processes outlined in
                           the RFP;
                           errors in the RFTdescription of processes, data elements, field sizes, and
                           related items;
                           more detailed requirements for reporting and transmitting accounting
                           data; and
                           specific requirements of certain laws and OMBcirculars and the precise
                           meaning of terms used in the RW, such as “warehousing.”

                           The System 90 Project Manager told us that he thought the Service
                           would be able to finish refining the functional requirements before a
                           contract is awarded and detailed design begins. However, others said
                           that a completion date was difficult to estimate because the detailing of
                           some areas may be more complex than currently envisioned. Also, the
                           design team plans to distribute copies of the refined requirements to
                           appropriate user groups for review and comment, a task for which addi-
                           tional time will be needed. At the close of our review in May 1990, the
                           Service officials told us that the design team had finished refining 6 of
                           10 design segments being reviewed.


Internal Control          Although the System 90 RIT and PACERfunctional requirements empha-
Requirements Not          size the importance of security and internal controls, the Service has not
                          systematically analyzed the need for specific controls over individual
Systematically Analyzed   PACERprocesses. NIST,OMB,GAO,and the President’s Councils on Manage-
                          ment Improvement and Integrity and Efficiency have issued guidance
                          directing agencies to systematically analyze control needs, establish con-
                          trol objectives, and develop specific control techniques for meeting these
                          objectives. All have recognized the importance of assessing control
                          needs in the early stages of system development to avoid the often more
                          expensive route of adding control features at a later design stage or cor-
                          recting control weaknesses after the system has been implemented.
                          While noting that there is no formal methodology for easily identifying
                          needed controls in systems under development, their guidance generally
                          calls for (1) managers (system users) to establish specific control objec-
                          tives based on the results of a risk analysis during a project’s definition
                          stage and (2) system developers to identify the best techniques for
                          meeting these objectives during the detailed design stage. According to
                          FIPS73, control objectives should be specific enough so that tests that
                          will tell whether the requirement is satisfied can be designed.

                          Documentation associated with the Service’s planning of System 90 and
                          PACERindicates that system developers and potential users did consider
                          potential threats to the new systems and the data that would be
                          processed. However, with the exception of access to files, the documen-
                          tation does not indicate that these threats were (1) analyzed to deter-
                          mine their impact on individual PACERprocesses, (2) ranked according to
                          the level of risk they posed, or (3) translated into control objectives.

                          The System 90 RFP,including the PACERfunctional requirements, speci-
                          fies a number of very general security and internal control objectives,
                          such as “compliance with government security standards” and “effec-
                          tive production job controls to minimize human error and damage.”
                          However, in most cases, these objectives have not been refined and
                          restated as they apply to individual PACERprocesses. As a result, they do
                          not provide adequate criteria against which the system design can be
                          tested, as recommended in FIF%73.

                          On the other hand, the PACERdata flow diagrams and the accompanying
                          narrative describe in varying detail a number of specific application
                          controls (such as edits and balancing requirements) associated with spe-
                          cific process segments. However, control objectives for most of these
                          segments are either absent or not clearly stated; as a result, it is not
                          possible to determine whether the techniques specified are needed or

                          Page11                                              GAO/AFMBQO-85
                                                                                         System SO

                     adequate. Software design team members told us that they are including
                     some application controls, such as edits and control totals, in the PACER
                     requirements because they consider this to be “good accounting and
                     system design.” However, they said that they are not comprehensively
                     evaluating the system’s internal control needs because this will be han-
                     dled as part of an independent verification and validation effort at a
                     later stage in the system’s development,

                     Appendix II provides an example of control objectives that the Service
                     developed for the Electronic Certification System that it has been pilot
                     testing since late 1987. We believe that analyses at this level of detail
                     are valuable to ensure that a new system design addresses all significant
                     system vulnerabilities.

                     The Service has not included System 90 or PACERin the Financial Man-
System SO/PACER      agement Systems Five Year Plan required under OMBCircular A-127,
Excluded From Five   “Financial Management Systems.” The Circular directs agencies to
Year Financial       develop and submit to OMBa plan estimating specific milestones, obliga-
                     tions, and outlays associated with financial management system
Management Plan      improvements. An official from OMB'SFinancial Management Policy
                     Office stated that these plans assist OMBin overseeing system improve-
                     ment activities and in making funding decisions.

                     The omission of System 90 from the Service’s plan is significant com-
                     pared to other reported system improvement efforts. The 1989 Service
                     plan that Treasury submitted to OMBlisted six projects estimated to cost
                     about $11 million during fiscal years 1990 through 1994. The Service
                     estimates that the System go/PACEReffort could cost about five times
                     that much for the same period.

                     In System 90, the Service has undertaken a major long-term system
Conclusions          development effort to improve and integrate its financial management
                     systems. We support this effort and emphasize the importance of fol-
                     lowing federal system development procedures to help ensure the new
                     system’s success.

                     The Service has not yet clearly and completely defined the capabilities
                     that it plans to develop during the first phase of this effort as part of
                     PACER.A complete and carefully reviewed set of functional and internal
                     control requirements for phase 1 will be needed to develop the system’s
                     detailed design and to assess development progress. Attempting to

                     Page12                                              GAO/AFMD-9046System90
                    develop a detailed design before functional and control requirements
                    have been fully defined will result in additional costs and unnecessary
                    delays if design questions must be resolved or the initial design must be
                    modified to correct oversights resulting from an incomplete require-
                    ments definition. Since the PACERsoftware is being developed on a Cost-
                    Plus-Award-Fee basis, the Service will bear the risk of incurring
                    increased costs if such delays occur. In addition, inadequately defined
                    requirements increase the risk that the system will not (1) fulfill the
                    Service’s system development objectives or (2) contain adequate
                    internal controls.

                    In particular, the Service needs a more structured approach to evalu-
                    ating PACER'sinternal control requirements prior to development of the
                    detailed system design. Federal guidance stresses the importance of
                    identifying internal control objectives as an integral step during the
                    early stages of system development.

                    Because significant costs were omitted from the Service’s System 90
                    cost/benefit analysis, PACERcould be much more expensive on a govern-
                    mentwide basis than the analysis indicates. Also, since supporting calcu-
                    lations and sensitivity analyses do not provide information on the value
                    of specific system capabilities, the Service does not now have a firm
                    basis for determining the effects of potential system design alterations
                    or for measuring the new system’s success in achieving the Service’s

                    In order to ensure that System 90 and PACERprovide the capabilities that
Recommendations     the Financial Management Service needs in the most cost-beneficial
                    manner, we recommend that you direct the System 90 project develop-
                    ment team to

                  . complete its detailed definition of PACERfunctional and internal control
                    requirements before beginning development of a detailed system design,
                    including a comprehensive analysis of PACERinternal control require-
                    ments based on a risk analysis and the development of detailed control
                    objectives so that developers can identify appropriate control tech-
                    niques and reviewers can assess the adequacy of those techniques, and
                  . revise and update its analysis of costs and benefits associated with
                    System 90 and PACERin accordance with FIPS64 to determine how much
                    PACERwill cost the federal government as a whole and to determine
                    which system capabilities will be the most valuable and, therefore, the
                    most important to implement.

                    Page13                                             GAO/AFMD-9046

                      In addition, we recommend that you direct the Financial Management
                      Service to include System 90 in future 5-year plans as required by OMB
                      Circular A-l 27.

                      Treasury generally agreed with our recommendations regarding (1) the
Agency Comments and   importance of further refining the PACERfunctional requirements and
Our Evaluation        (2) the need to update the System 90 cost/benefit analysis. Treasury’s
                      Fiscal Assistant Secretary stated that at the time of our review, the
                      department had already planned to complete a refined version of the
                      PACERfunctional and data requirements document, including internal
                      control objectives. He stated that this version would be completed prior
                      to contract award. He also indicated that the Service would update its
                      analysis of System 90 costs and benefits in accordance with FIPS64.

                      However, the Fiscal Assistant Secretary reported that the updated cost/
                      benefit analysis would not include special purpose sensitivity analyses
                      because the PACERdesign does not lend itself to separation into compo-
                      nent parts. According to FIB 64, a sensitivity analysis describes the
                      extent to which costs and benefits are sensitive to changes in key fac-
                      tors, such as the length of system life; the volume, mix, or pattern of
                      workload; the requirements; and the configuration of equipment and
                      software. As discussed in our report, we believe that some of these
                      assumptions, as well as specific requirements planned for PACER,could
                      be assessed individually. For example, the Service could assess the costs
                      and benefits of developing the capability to receive and process elec-
                      tronically certified and transmitted payment requests prior to acquiring
                      new hardware or implementing other PACERfeatures, Treasury is
                      already pilot testing electronic payment requests on a limited basis, and
                      expansion of this concept may be more cost-effective than continuing
                      the current paper-based procedures at most agencies until PACERis

                      While Treasury officials are responsible for determining which specific
                      cost/benefit relationships merit detailed analysis, sensitivity analyses of
                      the most critical factors could make the overall cost/benefit analysis
                      more meaningful. Regardless of Treasury’s decision on this matter, the
                      updated cost/benefit analysis needs to clearly identify the govern-
                      mentwide costs that are likely to be incurred in order to achieve the
                      benefits cited.

                      Treasury disagreed with our recommendation that System 90 be
                      included in future 5-year plans required by OMBCircular A-l 27, citing

                      Page14                                              GAO/AFMDW-SS
    the Service’s belief that System 90 is not covered by that circular. Cir-
    cular A-127 broadly defines financial management systems as “the total
    of agency financial systems, both manual and automated, for planning,
    budget formulation and execution, program and administrative
    accounting, and audit; as well as all other systems for recording and
    classifying financial data and reporting financial management informa-
    tion, including purchasing, property, inventory, etc.” We believe that
    this definition clearly covers System 9O/pAcrz. By supporting govern-
    mentwide payment, claims, and reconciliation operations, System 90/
    PACERwill perform several of the most significant federal financial man-
    agement functions.

    This report contains recommendations to you. The head of a federal
    agency is required by 31 U.K. 720 to submit a written statement on
    actions taken on these recommendations to the Senate Committee on
    Governmental Affairs and the House Committee on Government Opera-
    tions no later than 60 days after the date of the report and to the House
    and Senate Committees on Appropriations with the agency’s first
    request for appropriations made more than 60 days after the date of the

    We are sending copies of this report to the Commissioner, Financial
    Management Service; the Director, Office of Management and Budget;
    and other interested parties. Please contact me at (202) 276-9464 if you
    have any questions concerning this report. Major contributors to this
    report are listed in appendix IV.

    Sincerely yours,

4              “I          9
    Jeffrey C. Steinhoff
    Director, Financial Management Systems
      and Audit Oversight

    Page16                                             GAO/AFMD-90-85

Appendix I
System Development
Guidance Pertinent to
System 90 Review
Appendix II
Sample Control
Objective Developed
for the Financial
Management Service’s
Pilot Electronic
Certification System
Appendix III
Comments From the
Department of the
Appendix IV                                         30
Major Contributors to
This Report

                        Page16   GAO/AFMD&MS   System90


    FIPS       Federal Information Processing Standards
    Iv3        Federal Telecommunications System
    GAO        General Accounting Office
    NIST       National Institute of Standards and Technology
    OMB        Office of Management and Budget
    PACER      Payments, Claims, and Enhanced Reconciliation
    RFP        request for proposals

    Page17                                             GAO/-    System 66
Appendix I                                                                                     #

SystemDevelopmentGuidancePertinent to                                                               ’
System90 Review

                         Guidelines for Documentation of Computer Programs and Automated
National Institute of    Data Systems, FIPS38, February 161976.
Standards and
Technology (formerly     Provides basic guidance for the preparation of 10 document types that
                         are used in the development of computer software.
the National Bureau of
Standards)               Guidelines for Documentation of Computer Programs and Automated
                         Data Systems for the Initiation Phase, FIPS64, August 1, 1979.

                         Provides guidance in determining the content and extent of documenta-
                         tion needed for the initiation stage of the software life cycle.

                         Guideline for Automatic Data Processing Risk Analysis, FIPS66,
                         August 1,1979.

                         Provides a method of quantifying the impact of potential threats on
                         organizations supported by automatic data processing.

                         Guidelines for Security of Computer Applications, FIPS73, June 30,

                         Describes the different security objectives for a computer application,
                         explains the control measures that can be used, and identifies the deci-
                         sions that should be made at each stage in the life cycle of a sensitive
                         computer application.

                         Guide to Auditing for Controls and Security: A System Development Life
                         Cycle Approach, Zella G. Ruthberg, National Bureau of Standards Spe-
                         cial Publication 600-163, April 1988.

                         Addresses auditing the system development life cycle process for an
                         automated information system to ensure that controls and security are
                         designed and built into the system. (Co-sponsored by the President’s
                         Council on Integrity and Efficiency.)

                         Standards For Internal Controls In The Federal Government, 1983.
General Accounting
Office                   Contains the Comptroller General’s internal control standards to be fol-
                         lowed by executive agencies in establishing and maintaining systems of
             Y           internal control as required by the Federal Managers’ Financial Integrity
                         Act of 1982 (31 USC. 3612 (b)).

                         Page18                                              GAO/AFMD-SO&S
                       Appendix I
                       SyaWm Development Guidance Pertinent to
                       System 99 Review

                       Critical Factors in Developing Automated Accounting and Financial
                       Management Systems, January 1987.

                       Presents 14 major factors that are critical in developing automated
                       accounting and financial management systems including systems meth-
                       odology, functional requirements, documentation, and equipment

                       Information Systems: Agencies Overlook Security Controls During
                       Development, GAO/IMTEGFB-11and 1 lS, May 1988.

                       Examines nine federal civilian agencies that were neither meeting fed-
                       eral criteria nor using good system development practices for providing
                       reasonable assurance that appropriate security controls were being suc-
                       cessfully incorporated into their automated information systems.
                       Presents guidance on how to assess information security for systems in
                       development and the potential effects of not appropriately addressing
                       this area.

                       Federal Information Resources Management Regulations
General Services
Administration         Prescribe requirements for the acquisition and management of informa-
                       tion resources, including security for automated information systems
                       under development.

                       Internal Control Systems, OMBCircular A-123, August 4, 1986.
Office of Management
and Budget -           Prescribes the policies and procedures for executive agencies to follow
                       in establishing, maintaining, evaluating, improving, and reporting on
                       internal controls in their program and administrative activities,

                       Financial Management Systems, OMBCircular A-127, December 19, 1984.

                       Prescribes policies and procedures for executive agencies to follow in
                       developing, operating, evaluating, and reporting on financial manage-
                       ment systems.

                       Management of Federal Information Resources, OMBCircular A-130,
                       December 24, 1985.

                       Page 19                                            GAO/AFMD-90-85 System 99
                               Appendix I
                               System Development Guhbnce Pertlnent to
                               System 90 Review

                               Establishes policy for the management of federal information resources
                               as well as procedures for information system security.

                       Model Framework for Management Control Over Automated Informa-
President’s council on tion Systems 9          Jmuaw      l988

Improvement and        Provides guidance to federal managers for better understanding control
                       requirements directed by the Federal Managers’ Financial Integrity Act
President’s Council on of 1982 (Public Law 97-26631 U.S.C. 66a), the Privacy Act of 1974
Integrity and          (Public Law 93-679,6 U.S.C. 662a), and OMBCirculars A-123, A-127, and

                               Page 20                                           GAO/AFMD90-96 System 90
A@endix II

SampleControl ObjectiveDevelopedfor the
FInmcial ManagementService’sPilot Electronic
Certification System
                  Control objectives should be precise enough to allow’( 1) system devel-
                  opers to design related control techniques and (2) reviewers to deter-
                  mine that the designed techniques satisfy the stated objectives. We
                  believe that the following set of objectives illustrates the level of preci-
                  sion needed to accomplish these purposes. It was adapted from the con-
                  trol objectives developed for the Service’s Electronic Certification

                  CONTROL OBJECTIVE: Provide separation of payment schedule
                  processing functions so that no single individual will be able to initiate
                  and conceal errors or deliberately perform illegal actions.

              l Agency actions to certify a payment schedule should require the knowl-
                edge of two individuals, the security administrator and the certifying
              . Only the agency’s data entry operator should be permitted to enter a
                payment schedule for certification.
              . Only the agency’s certifying officer should be permitted to certify and
                transmit a payment schedule to a regional financial center.

                  Page 2 1                                              GAO/AF‘MB9W35 System fW
Appendix III

CommentsFrom the Department of
the Treasury

Note: GAO comments
supplementing those in the
report text appear at the
end of this appendix.                                              DEPARTMENT         OF THE         TREASURY

                                                                                   July      13,     1990

                              Dear     Mr.      Steinhoff:

                             We appreciate          receiving,        in advance,         your     draft                      report   on the
                             Financial       Management        Service’s      efforts         regarding                         System   90.     By
                             completing        this    review     at an early         stage      in the                       project,     prior    to
                             the    award    of the      System     90 contract,          the    Service                        has gained
                             valuable      insight,

                             Each of the           recommendations                    in your         report   has          been addressed                 in
                             the     following        paragraphs               and      selected         comments           regarding    your
                             findings          are provided        in          the      enclosure.

                             Relative        to the      first        recommendation               in your        draft       report,        we had
                             already       planned       to go the            extra       yard.        A refined          version        of the
See comment 1                Payments,         Claims       and Enhanced              Reconciliation              (PACER)         Functional
                             and Data        Requirements             (FDR)       document,          that     included          a third        level
                             of detail,           was being         prepared          prior      to your        review.           The document
                             will     capture        the    internal          control        objectives           in a discrete
                             section.          Prior     to the         award       of the       System       90 contract,            the
                             document        will      be completed,              printed,         and distributed                for    review
                             and approval            by all       signatories             of the       previous         version.

                             As to your               second         recommendation,                the      Financial           Management
                             Service           (FMS)        initiated            System       90 as an absolute                    necessity         due    to
                             the     need        to replace             old     hardware          whose        maintenance            costs       and
See comment 2.               operational                risks       were      escalating.               We do,         however,         plan      to
                             update        our       original           analysis          of System            90 PACER costs               and
                             benefits            that       was performed               to support             our     request        for    a
                             Delegation              of Procurement                 Authority           from       the      General       Services
                             Administration                   (GSA).          This      update        will      not       include       any special
                             purpose          sensitivity               analysis          because          the     PAC&        design       does     not
                             lend      itself          to separation                into      component            parts.         While      this
                             document           was prepared                using       FMS standards,                 we will        examine
                             Federal          Information               Processing            Standards            (FIPS)        64 to determine
                             that     we are            in accord           with      that      standard.

                             FMS staff        has discussed          the   third       recommendation          with     your
                             auditors       on several        occasions.           FMS believes         System      90 does                          m
See comment 3                fall     under     the coverage         of OMB Circular            A-127.       We are willing                                to
                             meet with        your  staff      and again         review      your    arguments       to
                             understand        the  rationale          for   your     conclusion.

                                     Page 22                                                                                    GAO/AFMD-90-86 System 90
                              APW*      In
                              Cbnmenta From the Department of
                              the Treasury

                   Page       2 -   Mr.     Jeffrey       Steinhoff

                  As a general            comment,     we had hoped               your       report       would     be more
                  balanced.            As you know,        FMS very           successfully             used      a new approach
                  to the       development         of requirements                definition           for     PACER,      the
                  Joint      Application        Design        (JAI31 methodology.                     FMS was the          first
See comment   4   Government           agency   to use this             interactive             design       technique           to
                  define       functional       and data          requirements.                 The requ i rements               for
                  our    new PACER System            were       gathered           in a 5-month            period      rather
                  than     the     12 to 18 months           estimated            for      an endeavor          of the        scope
                  and complexity             of System       90.       Our investments                  in the      JAD
                  methodology           had very     positive          returns          as substantiated               by written
                  comments        received      from    the      vendor        community            (see     item     1 of the

                  The benefits              ofthe     JAD process      that   accrued      to FMS were
                  increased          user   satisfaction         and confidence        and a cohesive
                  requirements           document         that contained      the   shared    goals   and
                  objectives          of all      affected     groups.

                  Thank     you      for     your      support        of   System       90   and   for   identifying           your


                                                                           Gerald     Murphy
                                                                 Fiscal     Assistant       Secretary

                  Mr.    Jeffrey       C. Steinhoff
                  Director,         Financial       Management
                      Systems       and Audit       Oversight
                  General        Accounting      Office


                  cc:     Commissioner,                 FMS
                          Assistant              Secretary        (Management)

                              Page 22                                                                           GAO/AFMD-90-85 System fJ0
                            AQQeluux        m
                            Comment8 From the Department of
                            the lkeafmq

                                                FMS Response            To    QAO Draft          Report

                        Response          to specific            statements           within        the    Draft       Report         are
                        keyed    to       the  page        on    which     they       appear.

See comment 5.     1.                                                                      References          to    “detai     led
                        design”         and     “clearly         defined        system.”

                                  The Payments           Claims         and Enhanced            Reconciliation
                                  (PACER)        Functional           and Data         Requirements            contained          two
                                  levels       of detail          when the          document        was released            with
                                  the     System       90 Request           for     Proposal        on March         13,    1989.
                                  At the       highest        level,        the     PACER functions             were
                                  described.            At the        second        level,      the     processes         within
                                  each     function         were      delineated.            The Financial            Management
                                  Service        feels      that      this      information           was more than
                                  adequate         for   the      purpose         of responding            to the RFP.
                                  This     was substantiated                  by comments           received        from      the
                                  vendors        on the       Request         for     Comments        document        released
                                  prior      to the      finalization               of the      RFP.       (See     Exhibit         1.)

                                  Subsequent        to the    release               of the      RFP, the         Software
                                  Design     Team was formed           to           acquire        in-depth        knowledge              of
                                  PACER and as a by-product                         to create          the   third      level        of
                                  detail     for    the  FDR.       This            effort      will       be completed             by
                                  July   31,     1990.     The document                    will   then      undergo       the
                                  review     and approval         process.

                                  The approach           taken     with   the    FDR is entirely          consistent
                                  with     traditional         design     methodology,        where      by you
                                  proceed         from   high    level    design     to detailed         design.     The
                                  information          in the      FDR was more than          sufficient         for  the
                                  System       90 contractor           to produce      a high      level    or system
                                  design,         and subsequently          the detailed        design.

See comment   6.   2.                                                        System      90     Phases.

                                  We recognized              the   need      for     a reasonable                approach         to the
                                  future,          We chose        to undertake               only      that       degree       of work
                                  that      could      be managed          within        our      resource           limits.          We
                                  recognized           that      we could        size       future        applications              so
                                  that      the    System        90 platform           i.e..        the      hardware,          systems
                                  software,          security        , and intelligent                    front        end
                                  processor          could       be built        to accommodate                  new
                                  applications.                Phase     I of the          system         90 initiative
                                  includes         hardware,         systems         software,            security           and the
                                  PACER application                software          as well          as the        System        90
                                  utility        that     will     enable        users        to reach           other       FMS
                                  applications.                Phase     II application                 initiatives             have
                                  not     yet    been     defined        in detail            and will           probably         be
                                  given       a different          name.         We are         not     now working             on
                                  Phase       2 applications.               Instead           we are         concentrating             on

                            Page 24                                                                                   GAO/AFMD-O0-M System 90
See comment   7.     3.

                              Following          14 Joint        Application          Design       (JAD)     sessions
                              which       averaged        four    days      each,     FMS produced           a two level
                              design        document        that    described          7 PACER functions              and
                              detailed         each     of the      189 processes           within       those
                              functions.            The documentation               from     these     JAD sessions,
                              two volumes,             was prepared          by a contractor             working       with
                              a facilitator,              the    FMS user        community,         major     Federal
                              Program        Agencies,         Federal       Reserve       Board,      Financial
                              Institutions,             FMS Regional           Financial        Center      Directors
                              and staff,          as well        as FMS project,            program        and
                              technical         staff.

See comment 6.      4,
                              The Service              is actually           evaluating         contractor           proposals
                              for     acquisition             of the       System       90 hardware,          system
                              software,           security,           and development             of a general            PACER
                              design.           Only      after       the    general       PACER design           is
                              completed           will      the     contractor          prepare      a detailed           Pacer
                              design.           Contractor            proposals         must    also     address        the
                              detailed          design,         build,       test     and implementation                phases
                              of PACER.

See comment   9.    5.

                              The major             tasks      of the      FEDSIM      (GSA)    contract         are:
                              requirements’                tracking,         risk    analysis,        review        ot all
                              System      90        contractor         deliverables,           development            of an
                              independent              Pacer      test     plan,     conduct      of the       PACER
                              preliminary              acceptance          testing,       and test        analysis.

See comment   10.   6.

                              The      digital      imaging        system     is still        in the      prototype
                              stage.         An RFP for          a pilot      system       iS being       developed           by
                              the     Federal      Reserve         Bank of Boston.              Once this        contract
                               is awarded        and the         pilot    system       is built        and
                              successfully           tested,         an RFP for        a production           system
                              will     be produced.              When the       production         systems       are
                              tested       and ready         for     use,   the capability            to utilize
                              digital        images      in PACER will            be implemented.

See comment   11.   7.

                              The      cash       management        and credit         administration          data
                              basee       are       not   included       in Phase        I of System       90.      The
                              “claimant             prof i le”     as originally          conceived     has been
                              abandoned             after     being    reviewed        under      terms of the
                              Privacy            Act.     A “claims        history”       wi I I, however,        be
                              included            in the      PACER data         base.

                          Page 26                                                                          GAO/AIWlbWM             System 90
                            Appendix III
                            Comments From the Department of

See comment 12.       8.        Agency        accounting          services        are pow available             using       an
                                off-the-shelf             package.            The package        is the      American
                                Management           System’s         “Federal       Financial         System.”         System
                                90 will        provide       access         to this      package       as part      of the
                                System        90 ( PACER 1 deve I opment.                  This    does     not   involve
                                application           development.               Access     will    be provided           by
                                routing         requests       from      client      agencies       through      the
                                System        90 front       end processor’s,               menu to the
                                appropriate           computer         on which        the    accounting
                                application           resides.

See comment 13.       9.

                                The abi 1 ity         to “warehouse”            payments         has always          been   a
                                part      of the      PACER design.             The capability              is described
                                 in Volume         I of the         FDR, page       70,,            “Benefits       of
                                Pacer       for    FPA’s,”          The “payment-timing               requirements”
                                 refer      to the      calculation          of the      amount       of time        the
                                fully       designed        PACER system          will     require       to produce         a
                                particular          payment.           Since    the    system       in not      yet
                                designed,         the    payment-timing             requirements           are not
See comment 14.       10.

                                The “alterations”               the    report       refers     to were        not
                                considered         because        of the      direction        given       in JAD 1 by
                                the     Extended       Planninq        Board      (EPB)      of FMS.         The EPB
                                defined       System      90 and PACER and stated                    that     seven
                                Regional        Financial         Centers       were      to be assumed.              PACER
                                was consciously             defined       as an integrated               data     system:
                                not     capable      of being        separated          into   component          parts.

See comment 15.       11.

                                “Team     members       and    project       officials”         who     made    the
                                statement        that     security        and    internal       control
                                requirements         were     not    included       must  have   misunderstood
                                the     question.       As was observed             on page    19 of the      draft
Now on p. 13.                   report,        there  are     such     requirements       in various       sections
                                of the       FDR, but     they     are    not   specific     enough    to be

See comment 16.       12.

                                The IV&V contractor            will     both     develop         and use an
                                automated       requirements        traceability              matrix    in
                                conjunction         with   FMS, to ensure           that       all   requirements
                                are    implemented.


                            Page 26                                                                       GAO/APMBSO-g6 System BO
                                       APpe*     nl
                                       Comment8 From the Lkpartment of

See comment 17.               13.          The third      level        of detail         has been        added      to   the    FDR.
                                           No requirements             have  been        changed.

See comment 18.               14.
                                           An example          of the    kind      of “errors”       found         by the
                                           Software         Design    Team is shown            in Exhibit          2.   The
                                           characterization            seems       harsh,     since    the        word  “errors”
                                           could      be so widely         interpreted.

See comment 19.               15.

                                           The IV&V contractor               has a task            to do an initial              risk
                                           analysis      on PACER high             level       design        and a final         risk
                                           analysis      once      PACER is implemented.                       The initial         risk
                                           analysis      will      help    FMS to further                define      security        and
                                           control     objectives          before        the     detailed         design      is
                                           started.        The fact        that      the     IV&V contractor               is
                                           performing         this    task     probably          led     to the misstatements
Now on p, 10 and p. 13.                    contained       on page        15 and the           last      sentence        of the
                                           second     complete        paragraph          on page         19.

See comment 20.               16.

                                           FMS does   not    believe      System    90 falls      under  the     coverage
                                           of OMB circular         A-127.      We have   frequently      briefed        the
                                           Department     of Treasury’s,         OIRM,   Budget      and Procurement
                                           offices,   as well        as,  OMB’s   Management        and Budget      staff
                                           regarding    System       90.

See comment 21.               17.
                                    “Recommendations.           ”

                                           An updated          cost/benefit         analysis     and the   new refined
                                           version       of the        FDR containing        the  third  level     of detail
                                           should      significantly          allay      the concerns    addressed      in
                                           your    draft       report.


                                        Page27                                                                           GAO/AFMLX90-85    System 90
               Appendix III
               Commenta From the Department of
               the Treasury

               The following are GAO'Scomments on the Department of the Treasury’s
               letter dated July 13, 1990.

               1. Discussed in “Agency Comments and Our Evaluation” section of the
GAO Comments   report.

               2. Discussed in “Agency Comments and Our Evaluation” section of the

               3. Discussed in “Agency Comments and Our Evaluation” section of the

               4. No change to report needed. Baaed on discussions with several session
               participants and a review of documentation from the sessions, we deter-
               mined that the Joint Application Design methodology apparently suc-
               ceeded in involving users early in the requirements definition effort and
               in documenting the initial set of functional and data requirements.
               Exhibit 1 of the enclosure has not been included in our report. Copies
               are available from GAOfor any interested party.

               6. No change to report needed. Exhibit 1 of the enclosure has not been
               included. Copies are available from GAOfor any interested party.

               6. No change to report needed.

               7. See comment 4.

               8. No change to report needed.

               9. No change to report needed.

               10. No change to report needed.

               11. No change to report needed.

               12. No change to report needed.

               13. Before this segment of the system can be designed, the Service’s
               managers must determine the specific payment-timing capabilities

               Page 28                                            GAO/AFMD-90-85
    Appendix III
    Comments From the Department of

    14. Discussed in “Agency Comments and Our Evaluation” section of the

    16. We discussed this point extensively with software design team mem-
    bers. We have modified the first reference to this issue to clarify our
    concern that internal control requirements are not being comprehen-
    sively analyzed.

    16. No change to report needed. Although the contractor is to develop a
    requirements traceability matrix, the Service is ultimately responsible
    for identifying the requirements to be included in this matrix.

    17. No change to report needed.

    18. No change to report needed. Exhibit 2 showed that several lines of
    text were deleted because they were extraneous, not incorrect. Because
    the software design team had not completed refining all the design seg-
    ments at the close of our review, we did not assess the magnitude of
    errors identified. Therefore, we cannot determine whether this exhibit is
    typical of other modifications. Exhibit 2 is not included in our report,
    but copies of it are available from GAOfor any interested party.

    19. According to NISTguidance, a risk analysis should be conducted
    during the initiation stage prior to developing internal control objec-
    tives, Although the contractor will assist in this effort, it is ultimately
    the Service’s responsibility to develop internal control objectives. As of
    mid-July 1990, the Service had not awarded a contract for the indepen-
    dent verification and validation effort. Therefore, we do not believe the
    report contains misstatements in this regard.

    20. Discussed in “Agency Comments and Our Evaluation” section of the

    2 1. No change to report needed.

    Page 29                                              GAO/tiFWHM5 System99
Appendix IV

Major Contributors to This Repoport

                         John C. Martin, Assistant Director, (202) 276-9481
Accounting and           Jean L. Boltz, Auditor-in-Charge
IF’inanciti Management   Lisa him, Evaluator
Division, Washington,

(9014Bt3)                Page 30                                              GAO/AFMD-90-86 System 90
llt~~ut~st,s for cwpies of (;A0 reports   shtdtl   tw sthnt. t,o:
I   /Jnited States
    General Accounting Office
    Wanhington, D.C. 20848
    Official Business
    Penalty for Private Use $300