IJnited States General Accounting Office GAO Report to the Secretary of the Treasury : Nt~pl.twrtwr 1!I!)0 FINANCIAL MANAGEMENT Defining Requirements Is Crucial to System 90’s Success $4 142137 GAO/AFMD-90-86 United States GAO General Accounting Office Washington, D.C. 20548 Accounting and Financial Management Division B-240003 September 5, 1990 The Honorable Nicholas F. Brady The Secretary of the Treasury Dear Mr. Secretary: This letter provides the results of our review of the Financial Manage- ment Service’s efforts to initiate and define the requirements for System 90, a long-term strategy for modernizing and integrating its financial management activities, and System 90’s first application, the Payments, Claims, and Enhanced Reconciliation (PACER)system. We undertook our review at an early stage in the development of System 90 and PACER, prior to the start of the detailed design effort, so that the Service could address any identified concerns before they adversely affected the system’s implementation and operation. As envisioned by the Service, System 90 could offer significant benefits to the Department of the Treasury, other federal agencies, and the recip- ients of federal payments, primarily by improving the timeliness and security of the Service’s operations through increased use of telecommu- nications and easier access to data. Efforts such as this are important to help the government modernize its financial management systems. While we support the development of System 90, it is important that the Service follow federal system development guidelines to reduce the associated risk and to ensure development of a system that meets man- agement’s needs in a cost-effective manner and that contains adequate controls. We found that although the Service performed many of the analyses Results in Brief stipulated in federal system development guidance, the Service has not (1) clearly defined the system capabilities that will be developed during phase 1 of System 90’s development or (2) adequately analyzed the related costs and benefits. The uncertainty associated with not clearly defining the project’s scope is compounded because the Service has not yet finished refining and clarifying the PACERfunctional and internal control requirements. We believe that to help avoid misunderstandings, oversights, and potentially expensive future design alterations, the Ser- vice must complete PACER'Sfunctional and internal control requirements before the contractor begins developing the detailed system design. Page1 GAO/AFMD-90435 System 90 . B-240003 The Financial Management Service is a bureau of the Department of the Background Treasury whose mission is to promote the financial integrity of the U. S. government through sound money management on the public’s behalf. The Service’s major responsibilities include issuing federal payments, reconciling payments issued with checks cashed, and resolving claims resulting from reports of government checks being forged, lost, stolen, or destroyed. The Service’s seven regional financial centers issue 86 percent of all fed- eral government payments, including Social Security and veterans’ bene- fits, vendor payments, federal salary and retirement payments, and tax refunds. Over 800 federal civilian agency offices send payment requests to the regional financial centers for processing. During fiscal year 1989, the Service issued 769 million payments valued at over $1 trillion in the form of checks and electronic funds transfers and resolved about 1.7 million check claims. Plans for a New System In developing System 90, the Service is undertaking a major effort to improve its operations with new computer hardware and software and the increased use of data telecommunications. By reducing reliance on cumbersome paper-based procedures and improving data accessibility, System 90 is to improve the efficiency of the Service’s operations and service to the public. Phase 1 of System 90, which is to be fully implemented by the mid- 1990s is to include the (1) acquisition of new hardware, (2) develop- ment of the PACERsystem, and (3) integration of hardware and software with a telecommunications network linking regional financial centers with Service headquarters, federal agencies, and the Federal Reserve System. During phase 2, the Service anticipates adding other applica- tions, such as central accounting, to the System 90 hardware and tele- communications processing environment. However, other than including an add-on capability in the System 90 design, the Service has not devel- oped specific plans or funding requirements for phase 2. PACERwill replace many of the Service’s existing processes for verifying agency payment requests, issuing checks, reconciling payments issued with checks cashed and electronic fund transfers executed, and resolving claims. The Service believes that the PACERsystem, as the first application of System 90, will substantially improve the existing Page2 GAO/AFMD-90-86 System 90 B24ooo3 processes and benefit not only the Service, but also other federal agen- cies and the recipients of federal payments. For example, the new system is to . ease system maintenance by replacing the Service’s incompatible com- puter hardware and nonstandard software with one uniform hardware and software architecture; . increase the timeliness and efficiency of payment processing by replacing paper and magnetic tape submissions with electronic transmissions; 9 improve security by implementing electronic certification and message authentication capabilities that will reduce the Service’s reliance on handwritten signatures of agency certifying officials; and l reduce the time associated with claims resolution from weeks to days primarily by providing easier access to information on whether checks have been cashed and other claims-related information. Current Status of Federal guidance divides system development efforts into consecutive Development Efforts stages: initiation, definition, detailed design, programming, testing, and operation. For PACER,the Service has completed the initiation stage, when existing deficiencies and general needs are identified and alterna- tive approaches considered, and is nearing the end of the definition stage, when system functions and performance, security, interface, and data requirements are defined. The Service is currently evaluating contract proposals for the acquisi- tion of the System 90 hardware, development of a detailed PACERdesign, and subsequent programming, testing, and implementation. Service offi- cials plan to award a contract by September 1990. According to the request for proposals (RF?), development of the first application, PACER, is to be completed within 18 months. If the contract is awarded as expected, the Service plans to begin implementing an operational system at the regional financial centers in April 1993. Completing the imple- mentation at all centers is expected to take more than a year. According to the RFP,the System 90 hardware will be purchased on a firm-fixed-price basis, while development of the PACERapplication software will be priced on a cost-plus-award-fee basis. This means that the contract will stipulate specific prices for hardware purchased but only estimates, subject to change, for software development. The needed Page3 B240003 telecommunications services are to be provided by the Federal Telecom- munications System (FTS) 2000, a newly implemented system that pro- vides the federal government with long-distance telecommunications services under a General Services Administration contract. The Service is also evaluating proposals for a second contract for an “independent verification and validation” of System 90 to ensure that System 90 and PACERmeet the Service’s objectives. The contract is to be awarded under an agreement with the General Service Administration’s Federal Systems Integration and Management Center, which also assisted the Service in defining System 90 and PACERrequirements. The objectives of our review were to determine whether the Service’s Objectives, Scope,and initial efforts to develop System 90 and PACERwere in conformance with Methodology federal guidance and how any lack of conformance we identified increased the risk that system development and implementation efforts would not be successful. Specifically, we wanted to determine if the Ser- vice had performed initiation and definition stage tasks in accordance with guidance developed by the Department of Commerce’s National Institute of Standards and Technology (NET). NEST(formerly the National Bureau of Standards) is authorized by the Brooks Act (Public Law 89- 306 or 40 U.S.C. 759, as amended) and the Computer Security Act of 1987 (Public Law 100-235) to set standards and provide guidance to agencies to improve the use of federal automated information systems and to assure the cost-effective security and privacy of the sensitive data that they contain. We also considered guidance issued by the Office of Management and Budget (OMB), GAO,the President’s Council on Management Improve- ment, and the President’s Council on Integrity and Efficiency. This gui- dance provides more detailed explanations of how federal agencies should develop and evaluate security and internal control requirements and plan for new systems. Appendix I lists specific documents, including NIST Federal Information Processing Standards (FIPS),pertinent to our review. Because the System 90 and PACERdesign will not be operational for sev- eral years, we did not attempt to predict whether the new system will be completed according to schedule, operate as envisioned, or contain ade- quate internal controls. In addition, our objectives did not include an assessment of the RFP'Stechnical specifications for hardware, including Page4 GAO/AFMD-SO-35 System90 B-240003 whether the project was biased toward a specific vendor’s equipment, or an assessment of the Service’s proposal evaluation process. To assess the Service’s conformance with federal system development guidance, we examined documentation such as the System 90 and PACER RFPand related analyses and project plans. To clarify our understanding of the System 90 and PACERdevelopment effort, we interviewed appro- priate officials in the Service and in Treasury’s Office of the Inspector General and Office of Information Resources Management. To determine how System 90 related to the Service’s overall financial management improvement plans, we discussed the Service’s Financial Management Systems Five Year Plan with Treasury and OMBofficials. Our work was conducted between October 1989 and May 1990 at the Service’s headquarters in Washington, D.C., and Hyattsville, Maryland. Our review was performed in accordance with generally accepted gov- ernment auditing standards. The Department of the Treasury provided written comments on a draft of this report. These comments are presented in appendix III. According to NISTguidance, the initiation stage consists of the identifica- Initiation Stage tion of a need by the user organization, an assessment of risks to assets Analyses Are and resources, and an exploration of alternative solutions, including a Incomplete and Poorly comparison of the expected costs and benefits. After management selects the best alternative, the project continues through the remaining Documented stages of system development. Treasury and Service procedures reflect NET'S guidance on the initiation stage analyses. Although the Service performed most of the initiation stage analyses specified by NIST, (1) the resulting documents were inconsistent in describing the planned capabil- ities for phase 1 of System 90 and (2) the related costs and benefits were not fully analyzed. For this reason, it is difficult to determine how System 90 will operate, what benefits will be realized from phase 1, or the system’s governmentwide cost. Inconsistencies in The key documents that the Service prepared do not clearly or consis- Documentation of Planned tently describe the planned capabilities for phase 1 of System 90. When a system is developed in multiple phases, it is important to clearly Capabilities define which capabilities are to be developed first and which are being II considered for development in a later phase. We could not determine from the available documentation if several capabilities mentioned in the September 1987 requirements analysis, June 1988 project plan, and Page6 GAO/AFMD-90-N System90 B-240003 August 1989 cost/benefit analysis were still to be included in the System 90 and PACERdesign. We discussed these capabilities with the Service officials to determine which had been deferred, which had been deleted, and which were still planned for development during phase 1. Examples of capabilities that were not clearly or consistently described are dis- cussed below. l A digital imaging system’ to facilitate retrieval of check images is dis- cussed in the requirements analysis, project plan, and cost/benefit anal- ysis. Service officials and the RFPstated that this capability will not be developed until phase 2. . The requirements analysis and cost/benefit analysis describe three data bases (cash management, credit administration, and claimant profile) that are to be accessible to federal agencies as part of System 90. A Ser- vice official told us that the first two data bases will not be developed until phase 2 and that the third may never be developed. . The requirements analysis, project plan, cost/benefit analysis, and RFP state that agency accounting services, which the Service currently pro- vides to some federal agencies via other systems, will be incorporated under System 90. However, these documents do not clearly state when this capability will be developed. Service officials told us that agency accounting services will be incorporated into or accessed through System 90 after PACERdevelopment. 9 The ability to “warehouse,” or store, and precisely time vendor pay- ments is to result in significant cash management savings, according to key planning documents and Service officials. However, this capability is not described in the PACERdata flow diagrams that are included in the RIT. Service officials told us that they still plan to include this capability in the PACERdesign during phase 1 but need to define more specific payment-timing requirements. Costs and Benefits Not The Service’s System 90 cost/benefit analysis omits certain costs and Fully Analyzed does not clearly match others with specific system features which will result in benefits. Consequently, the analysis provides limited informa- tion for determining which aspects of the System 90 and PACERdesign are likely to be the most cost beneficial. ‘The RFP defines a digital imaging system as “an automated system designed to produce and convert hardcopy images into digital images that can be stored on high density media, displayed on high resolution workstations, accessedand distributed by communications networks, and otherwise manipulated by a computer system.” Page6 GAO/AFlKD-3O-SSSystemBO According to FIPS64, the purpose of a cost/benefit analysis is to “pro- vide managers, users, designers, and auditors with adequate cost and benefit information to analyze alternative approaches.” FIPS64 also rec- ommends that system developers perform sensitivity analyses to assess the extent to which costs and benefits will vary if factors such as system requirements, workload mix, and equipment configuration change. The Service developed a cost/benefit analysis in 1987 and revised and updated it in August 1989. This analysis cites net benefits of $146 mil- lion over a lo-year system life (system development and operation from fiscal years 1990 to 1999). These savings are reported to be $28 million2 when converted to constant dollars (to eliminate increases due to infla- tion) and discounted to present value (to accommodate the time-value of money).3 According to this analysis, benefits will begin to accrue to the Service, the U. S. Treasury, and other federal agencies in fiscal year 1994, after PACERimplementation. The primary benefits expected over the system’s life (through 1999) are reduced claims processing, the elim- ination of early payments, the replacement of microfilmed check images with digital check images, and reduced maintenance costs. Although the analysis includes both nonrecurring and recurring benefits on a governmentwide basis, costs included are generally limited to those that will be incurred by the Service. In addition, these data are not com- plete or readily verifiable. We identified potentially significant costs that had been omitted, and we could not determine how significant cost and benefit figures had been developed because few supporting calcula- tions were available. We attempted to assess the reasonableness of some costs presented by comparing them to estimates contained in planning documents and in a February 1989 alternatives analysis prepared by a Service contractor. This analysis, which primarily addresses alternatives related to the geo- graphical distribution of processing and data, provides detailed esti- mates of many of the costs that are to be included in a cost/benefit analysis. Based on this comparison and on discussions with Service offi- 2We identified and corrected errors in the present value factors applied. This increased net benefits by about $10 million to $38 million. 3Present value is the estimated current worth of future benefits It is determined by estimating the return that could be realized if the funds were used for the best available investment instead of for the alternative being considered. In accordance with OMB Circular A-94, “Discount Rates to Be Used ln Evaluating Time-Distributed Costs and Benefits,” the Service used a lo-percent discount rate to determine the present value of figures presented in the System 90 cost/benefit analysis. Page7 GAO/AFMD-90-85 System90 B-240003 cials, we identified the following potentially significant costs that were not included in the System 90 cost/benefit analysis: l recurring telecommunications costs, which the alternatives analysis esti- mates at about $7 million per year for the Service alone; l telecommunications costs that will be incurred by the federal agencies who transmit data to the Service (no estimate was readily available); . site and facilities costs, estimated to total about $8 million through fiscal year 1993 according to a 1989 Service facilities financial plan; . contractor assistance with system definition and verification at a cost of $4.2 million; and l costs associated with the creation and storage of digital check images (no estimate was readily available). System 90 project officials agreed that the cost/benefit analysis was incomplete. They explained that the analysis was prepared primarily to support the Service’s request for a delegation of procurement authority from the General Services Administration and that, for this reason, they intended to include only costs that the Service would incur for the System 90 contract. Relation BetweenSpecificCosts The System 90 cost/benefit analysis describes an array of new capabili- and Benefits Unclear ties that are to speed payment and claims processing, facilitate access to information, and increase system security and reliability. However, because supporting calculations and sensitivity analyses were incom- plete, the relationship between specific system features and expected benefits is not apparent. For example, the analysis does not . clearly differentiate between the costs and benefits attributable to those PACERrequirements currently under development and planned for funding and those that may be added or l assess the impact of possible alterations in (1) system design, such as the elimination or deferral of a planned feature like payment ware- housing, (2) configuration, such as a reduction in the number of regional financial centers, or (3) development and implementation plans, such as the deferral of major system segments in order to implement others sooner than currently planned. Because it has not clearly matched specific system features and antici- pated benefits, the Service does not have a basis for readily assessing the impact of any modifications to the system’s design or implementa- tion plans that may be proposed during the remainder of the develop- ment effort. For example, in response to a request by OMB,the Service is Page8 GAO/AFMD30-36 System33 assessing the effect of reducing the number of regional financial centers, an option that could have been considered earlier and, if selected, incor- porated into the System 90 design definition. In addition, a clearer pres- entation of costs and related benefits would provide the Service and others, such as Treasury’s Office of Information Resources Management and OMB,a basis for monitoring the project’s progress and success. At the close of our review in May 1990, the System 90 software design Definition of team was still defining PACERfunctional and data requirements in an F’unctionaland effort to ensure the completeness and accuracy of this documentation, Internal Control which will be used by the contractor to develop a detailed system design. Team members and project officials told us that they were not Requirements including a comprehensive analysis of security and internal control Incomplete requirements as part of this effort because the Service plans to review control needs during a later stage of the system’s development. Guidance provided by NET, OMB,and GAOstates that detailed definitions of functional and internal control requirements should be completed before a project’s detailed design stage begins. These requirements are to provide a basis for mutual understanding between users and designers regarding system functions, performance requirements, data inputs and outputs, data characteristics, failure contingencies, equip- ment, interfaces, security, and control requirements. A primary respon- sibility for ensuring that requirements are complete and sufficiently detailed lies with those who will use the new system. It is most cost beneficial to fully define and verify functional and control requirements as early as possible because alterations become more difficult as the development effort progresses. Service officials have recognized that a complete list of PACERfunctional and internal control requirements will be needed not only to provide direction to system designers, but also to facilitate a planned indepen- dent “verification and validation” review of the System QO/PAcERdesign. This planned review is to include development of a “requirements trace- ability matrix” as a tool to ensure that all system requirements are met. The matrix is to list and describe all system requirements, including their source and objective. System 90 project officials told us that internal control requirements would be included in the list. An indepen- dent contractor is then to use this list to trace requirements to the system’s design and, ultimately, to the operational system. At the close of our review, the Service had not yet awarded a contract for the inde- pendent verification and validation effort. Page9 GAO/AFMD9O-S6 System 90 I E24ooo3 Service Efforts to Fully Although the Service included narrative, data flow diagrams, and a data Define System Are in dictionary outlining PACERrequirements in the System 90 RFP,issued on March 16,1989, the Service has continued to modify and supplement Progress these requirements. In an effort to eliminate any errors, omissions, or ambiguities in the documentation prior to contract award and the subse- quent detailed design stage, a group of about 26 Service personnel, referred to as the Software Design Team, has been analyzing various aspects of the PACERdesign. Deficiencies in the requirements definition could impede the contractor’s ability to develop PACERwithin the 18- month period envisioned by the Service. Specifically, the team is exam- ining assumptions, further defining specific requirements, identifying missing data elements, examining the relationship between automated and manual procedures, raising concerns regarding the feasibility of cer- tain system objectives, and substantially expanding the design documentation. Based on discussions with team members and an examination of their expanded documentation, we determined that the team has identified or further defined the need for additional data to support some of the processes outlined in the RFP; errors in the RFTdescription of processes, data elements, field sizes, and related items; more detailed requirements for reporting and transmitting accounting data; and specific requirements of certain laws and OMBcirculars and the precise meaning of terms used in the RW, such as “warehousing.” The System 90 Project Manager told us that he thought the Service would be able to finish refining the functional requirements before a contract is awarded and detailed design begins. However, others said that a completion date was difficult to estimate because the detailing of some areas may be more complex than currently envisioned. Also, the design team plans to distribute copies of the refined requirements to appropriate user groups for review and comment, a task for which addi- tional time will be needed. At the close of our review in May 1990, the Service officials told us that the design team had finished refining 6 of 10 design segments being reviewed. Page10 5240003 Internal Control Although the System 90 RIT and PACERfunctional requirements empha- Requirements Not size the importance of security and internal controls, the Service has not systematically analyzed the need for specific controls over individual Systematically Analyzed PACERprocesses. NIST,OMB,GAO,and the President’s Councils on Manage- ment Improvement and Integrity and Efficiency have issued guidance directing agencies to systematically analyze control needs, establish con- trol objectives, and develop specific control techniques for meeting these objectives. All have recognized the importance of assessing control needs in the early stages of system development to avoid the often more expensive route of adding control features at a later design stage or cor- recting control weaknesses after the system has been implemented. While noting that there is no formal methodology for easily identifying needed controls in systems under development, their guidance generally calls for (1) managers (system users) to establish specific control objec- tives based on the results of a risk analysis during a project’s definition stage and (2) system developers to identify the best techniques for meeting these objectives during the detailed design stage. According to FIPS73, control objectives should be specific enough so that tests that will tell whether the requirement is satisfied can be designed. Documentation associated with the Service’s planning of System 90 and PACERindicates that system developers and potential users did consider potential threats to the new systems and the data that would be processed. However, with the exception of access to files, the documen- tation does not indicate that these threats were (1) analyzed to deter- mine their impact on individual PACERprocesses, (2) ranked according to the level of risk they posed, or (3) translated into control objectives. The System 90 RFP,including the PACERfunctional requirements, speci- fies a number of very general security and internal control objectives, such as “compliance with government security standards” and “effec- tive production job controls to minimize human error and damage.” However, in most cases, these objectives have not been refined and restated as they apply to individual PACERprocesses. As a result, they do not provide adequate criteria against which the system design can be tested, as recommended in FIF%73. On the other hand, the PACERdata flow diagrams and the accompanying narrative describe in varying detail a number of specific application controls (such as edits and balancing requirements) associated with spe- cific process segments. However, control objectives for most of these segments are either absent or not clearly stated; as a result, it is not possible to determine whether the techniques specified are needed or Page11 GAO/AFMBQO-85 System SO B24ooo3 adequate. Software design team members told us that they are including some application controls, such as edits and control totals, in the PACER requirements because they consider this to be “good accounting and system design.” However, they said that they are not comprehensively evaluating the system’s internal control needs because this will be han- dled as part of an independent verification and validation effort at a later stage in the system’s development, Appendix II provides an example of control objectives that the Service developed for the Electronic Certification System that it has been pilot testing since late 1987. We believe that analyses at this level of detail are valuable to ensure that a new system design addresses all significant system vulnerabilities. The Service has not included System 90 or PACERin the Financial Man- System SO/PACER agement Systems Five Year Plan required under OMBCircular A-127, Excluded From Five “Financial Management Systems.” The Circular directs agencies to Year Financial develop and submit to OMBa plan estimating specific milestones, obliga- tions, and outlays associated with financial management system Management Plan improvements. An official from OMB'SFinancial Management Policy Office stated that these plans assist OMBin overseeing system improve- ment activities and in making funding decisions. The omission of System 90 from the Service’s plan is significant com- pared to other reported system improvement efforts. The 1989 Service plan that Treasury submitted to OMBlisted six projects estimated to cost about $11 million during fiscal years 1990 through 1994. The Service estimates that the System go/PACEReffort could cost about five times that much for the same period. In System 90, the Service has undertaken a major long-term system Conclusions development effort to improve and integrate its financial management systems. We support this effort and emphasize the importance of fol- lowing federal system development procedures to help ensure the new system’s success. The Service has not yet clearly and completely defined the capabilities that it plans to develop during the first phase of this effort as part of PACER.A complete and carefully reviewed set of functional and internal control requirements for phase 1 will be needed to develop the system’s detailed design and to assess development progress. Attempting to Page12 GAO/AFMD-9046System90 develop a detailed design before functional and control requirements have been fully defined will result in additional costs and unnecessary delays if design questions must be resolved or the initial design must be modified to correct oversights resulting from an incomplete require- ments definition. Since the PACERsoftware is being developed on a Cost- Plus-Award-Fee basis, the Service will bear the risk of incurring increased costs if such delays occur. In addition, inadequately defined requirements increase the risk that the system will not (1) fulfill the Service’s system development objectives or (2) contain adequate internal controls. In particular, the Service needs a more structured approach to evalu- ating PACER'sinternal control requirements prior to development of the detailed system design. Federal guidance stresses the importance of identifying internal control objectives as an integral step during the early stages of system development. Because significant costs were omitted from the Service’s System 90 cost/benefit analysis, PACERcould be much more expensive on a govern- mentwide basis than the analysis indicates. Also, since supporting calcu- lations and sensitivity analyses do not provide information on the value of specific system capabilities, the Service does not now have a firm basis for determining the effects of potential system design alterations or for measuring the new system’s success in achieving the Service’s objectives. In order to ensure that System 90 and PACERprovide the capabilities that Recommendations the Financial Management Service needs in the most cost-beneficial manner, we recommend that you direct the System 90 project develop- ment team to . complete its detailed definition of PACERfunctional and internal control requirements before beginning development of a detailed system design, including a comprehensive analysis of PACERinternal control require- ments based on a risk analysis and the development of detailed control objectives so that developers can identify appropriate control tech- niques and reviewers can assess the adequacy of those techniques, and . revise and update its analysis of costs and benefits associated with System 90 and PACERin accordance with FIPS64 to determine how much PACERwill cost the federal government as a whole and to determine which system capabilities will be the most valuable and, therefore, the most important to implement. Page13 GAO/AFMD-9046 System90 B-240003 In addition, we recommend that you direct the Financial Management Service to include System 90 in future 5-year plans as required by OMB Circular A-l 27. Treasury generally agreed with our recommendations regarding (1) the Agency Comments and importance of further refining the PACERfunctional requirements and Our Evaluation (2) the need to update the System 90 cost/benefit analysis. Treasury’s Fiscal Assistant Secretary stated that at the time of our review, the department had already planned to complete a refined version of the PACERfunctional and data requirements document, including internal control objectives. He stated that this version would be completed prior to contract award. He also indicated that the Service would update its analysis of System 90 costs and benefits in accordance with FIPS64. However, the Fiscal Assistant Secretary reported that the updated cost/ benefit analysis would not include special purpose sensitivity analyses because the PACERdesign does not lend itself to separation into compo- nent parts. According to FIB 64, a sensitivity analysis describes the extent to which costs and benefits are sensitive to changes in key fac- tors, such as the length of system life; the volume, mix, or pattern of workload; the requirements; and the configuration of equipment and software. As discussed in our report, we believe that some of these assumptions, as well as specific requirements planned for PACER,could be assessed individually. For example, the Service could assess the costs and benefits of developing the capability to receive and process elec- tronically certified and transmitted payment requests prior to acquiring new hardware or implementing other PACERfeatures, Treasury is already pilot testing electronic payment requests on a limited basis, and expansion of this concept may be more cost-effective than continuing the current paper-based procedures at most agencies until PACERis operational. While Treasury officials are responsible for determining which specific cost/benefit relationships merit detailed analysis, sensitivity analyses of the most critical factors could make the overall cost/benefit analysis more meaningful. Regardless of Treasury’s decision on this matter, the updated cost/benefit analysis needs to clearly identify the govern- mentwide costs that are likely to be incurred in order to achieve the benefits cited. Treasury disagreed with our recommendation that System 90 be included in future 5-year plans required by OMBCircular A-l 27, citing Page14 GAO/AFMDW-SS System90 the Service’s belief that System 90 is not covered by that circular. Cir- cular A-127 broadly defines financial management systems as “the total of agency financial systems, both manual and automated, for planning, budget formulation and execution, program and administrative accounting, and audit; as well as all other systems for recording and classifying financial data and reporting financial management informa- tion, including purchasing, property, inventory, etc.” We believe that this definition clearly covers System 9O/pAcrz. By supporting govern- mentwide payment, claims, and reconciliation operations, System 90/ PACERwill perform several of the most significant federal financial man- agement functions. This report contains recommendations to you. The head of a federal agency is required by 31 U.K. 720 to submit a written statement on actions taken on these recommendations to the Senate Committee on Governmental Affairs and the House Committee on Government Opera- tions no later than 60 days after the date of the report and to the House and Senate Committees on Appropriations with the agency’s first request for appropriations made more than 60 days after the date of the report. We are sending copies of this report to the Commissioner, Financial Management Service; the Director, Office of Management and Budget; and other interested parties. Please contact me at (202) 276-9464 if you have any questions concerning this report. Major contributors to this report are listed in appendix IV. Sincerely yours, , c? 4 “I 9 Jeffrey C. Steinhoff Director, Financial Management Systems and Audit Oversight Page16 GAO/AFMD-90-85 Syetem90 * Contents Letter Appendix I System Development Guidance Pertinent to System 90 Review Appendix II Sample Control Objective Developed for the Financial Management Service’s Pilot Electronic Certification System Appendix III Comments From the Department of the Treasury Appendix IV 30 Major Contributors to This Report Page16 GAO/AFMD&MS System90 L Content8 Abbreviations FIPS Federal Information Processing Standards Iv3 Federal Telecommunications System GAO General Accounting Office NIST National Institute of Standards and Technology OMB Office of Management and Budget PACER Payments, Claims, and Enhanced Reconciliation RFP request for proposals Page17 GAO/- System 66 Appendix I # SystemDevelopmentGuidancePertinent to ’ System90 Review Guidelines for Documentation of Computer Programs and Automated National Institute of Data Systems, FIPS38, February 161976. Standards and Technology (formerly Provides basic guidance for the preparation of 10 document types that are used in the development of computer software. the National Bureau of Standards) Guidelines for Documentation of Computer Programs and Automated Data Systems for the Initiation Phase, FIPS64, August 1, 1979. Provides guidance in determining the content and extent of documenta- tion needed for the initiation stage of the software life cycle. Guideline for Automatic Data Processing Risk Analysis, FIPS66, August 1,1979. Provides a method of quantifying the impact of potential threats on organizations supported by automatic data processing. Guidelines for Security of Computer Applications, FIPS73, June 30, 1980. Describes the different security objectives for a computer application, explains the control measures that can be used, and identifies the deci- sions that should be made at each stage in the life cycle of a sensitive computer application. Guide to Auditing for Controls and Security: A System Development Life Cycle Approach, Zella G. Ruthberg, National Bureau of Standards Spe- cial Publication 600-163, April 1988. Addresses auditing the system development life cycle process for an automated information system to ensure that controls and security are designed and built into the system. (Co-sponsored by the President’s Council on Integrity and Efficiency.) Standards For Internal Controls In The Federal Government, 1983. General Accounting Office Contains the Comptroller General’s internal control standards to be fol- lowed by executive agencies in establishing and maintaining systems of Y internal control as required by the Federal Managers’ Financial Integrity Act of 1982 (31 USC. 3612 (b)). Page18 GAO/AFMD-SO&S System90 Appendix I SyaWm Development Guidance Pertinent to System 99 Review Critical Factors in Developing Automated Accounting and Financial Management Systems, January 1987. Presents 14 major factors that are critical in developing automated accounting and financial management systems including systems meth- odology, functional requirements, documentation, and equipment acquisition. Information Systems: Agencies Overlook Security Controls During Development, GAO/IMTEGFB-11and 1 lS, May 1988. Examines nine federal civilian agencies that were neither meeting fed- eral criteria nor using good system development practices for providing reasonable assurance that appropriate security controls were being suc- cessfully incorporated into their automated information systems. Presents guidance on how to assess information security for systems in development and the potential effects of not appropriately addressing this area. Federal Information Resources Management Regulations General Services Administration Prescribe requirements for the acquisition and management of informa- tion resources, including security for automated information systems under development. Internal Control Systems, OMBCircular A-123, August 4, 1986. Office of Management and Budget - Prescribes the policies and procedures for executive agencies to follow in establishing, maintaining, evaluating, improving, and reporting on internal controls in their program and administrative activities, Financial Management Systems, OMBCircular A-127, December 19, 1984. Prescribes policies and procedures for executive agencies to follow in developing, operating, evaluating, and reporting on financial manage- ment systems. Management of Federal Information Resources, OMBCircular A-130, December 24, 1985. Page 19 GAO/AFMD-90-85 System 99 Appendix I System Development Guhbnce Pertlnent to System 90 Review Establishes policy for the management of federal information resources as well as procedures for information system security. Model Framework for Management Control Over Automated Informa- President’s council on tion Systems 9 Jmuaw l988 Management Improvement and Provides guidance to federal managers for better understanding control requirements directed by the Federal Managers’ Financial Integrity Act President’s Council on of 1982 (Public Law 97-26631 U.S.C. 66a), the Privacy Act of 1974 Integrity and (Public Law 93-679,6 U.S.C. 662a), and OMBCirculars A-123, A-127, and A-130. Efficiency Page 20 GAO/AFMD90-96 System 90 A@endix II SampleControl ObjectiveDevelopedfor the FInmcial ManagementService’sPilot Electronic Certification System Control objectives should be precise enough to allow’( 1) system devel- opers to design related control techniques and (2) reviewers to deter- mine that the designed techniques satisfy the stated objectives. We believe that the following set of objectives illustrates the level of preci- sion needed to accomplish these purposes. It was adapted from the con- trol objectives developed for the Service’s Electronic Certification System. CONTROL OBJECTIVE: Provide separation of payment schedule processing functions so that no single individual will be able to initiate and conceal errors or deliberately perform illegal actions. l Agency actions to certify a payment schedule should require the knowl- edge of two individuals, the security administrator and the certifying officer. . Only the agency’s data entry operator should be permitted to enter a payment schedule for certification. . Only the agency’s certifying officer should be permitted to certify and transmit a payment schedule to a regional financial center. Page 2 1 GAO/AF‘MB9W35 System fW Appendix III CommentsFrom the Department of the Treasury Note: GAO comments supplementing those in the report text appear at the end of this appendix. DEPARTMENT OF THE TREASURY WASnlNGTON July 13, 1990 Dear Mr. Steinhoff: We appreciate receiving, in advance, your draft report on the Financial Management Service’s efforts regarding System 90. By completing this review at an early stage in the project, prior to the award of the System 90 contract, the Service has gained valuable insight, Each of the recommendations in your report has been addressed in the following paragraphs and selected comments regarding your findings are provided in the enclosure. Relative to the first recommendation in your draft report, we had already planned to go the extra yard. A refined version of the See comment 1 Payments, Claims and Enhanced Reconciliation (PACER) Functional and Data Requirements (FDR) document, that included a third level of detail, was being prepared prior to your review. The document will capture the internal control objectives in a discrete section. Prior to the award of the System 90 contract, the document will be completed, printed, and distributed for review and approval by all signatories of the previous version. As to your second recommendation, the Financial Management Service (FMS) initiated System 90 as an absolute necessity due to the need to replace old hardware whose maintenance costs and See comment 2. operational risks were escalating. We do, however, plan to update our original analysis of System 90 PACER costs and benefits that was performed to support our request for a Delegation of Procurement Authority from the General Services Administration (GSA). This update will not include any special purpose sensitivity analysis because the PAC& design does not lend itself to separation into component parts. While this document was prepared using FMS standards, we will examine Federal Information Processing Standards (FIPS) 64 to determine that we are in accord with that standard. FMS staff has discussed the third recommendation with your auditors on several occasions. FMS believes System 90 does m See comment 3 fall under the coverage of OMB Circular A-127. We are willing to meet with your staff and again review your arguments to understand the rationale for your conclusion. Page 22 GAO/AFMD-90-86 System 90 APW* In Cbnmenta From the Department of the Treasury Page 2 - Mr. Jeffrey Steinhoff As a general comment, we had hoped your report would be more balanced. As you know, FMS very successfully used a new approach to the development of requirements definition for PACER, the Joint Application Design (JAI31 methodology. FMS was the first See comment 4 Government agency to use this interactive design technique to define functional and data requirements. The requ i rements for our new PACER System were gathered in a 5-month period rather than the 12 to 18 months estimated for an endeavor of the scope and complexity of System 90. Our investments in the JAD methodology had very positive returns as substantiated by written comments received from the vendor community (see item 1 of the enclosure). The benefits ofthe JAD process that accrued to FMS were increased user satisfaction and confidence and a cohesive requirements document that contained the shared goals and objectives of all affected groups. Thank you for your support of System 90 and for identifying your concerns. Sincerely, Gerald Murphy Fiscal Assistant Secretary Mr. Jeffrey C. Steinhoff Director, Financial Management Systems and Audit Oversight General Accounting Office Enclosure cc: Commissioner, FMS Assistant Secretary (Management) Page 22 GAO/AFMD-90-85 System fJ0 AQQeluux m Comment8 From the Department of the lkeafmq FMS Response To QAO Draft Report Response to specific statements within the Draft Report are keyed to the page on which they appear. See comment 5. 1. References to “detai led design” and “clearly defined system.” The Payments Claims and Enhanced Reconciliation (PACER) Functional and Data Requirements contained two levels of detail when the document was released with the System 90 Request for Proposal on March 13, 1989. At the highest level, the PACER functions were described. At the second level, the processes within each function were delineated. The Financial Management Service feels that this information was more than adequate for the purpose of responding to the RFP. This was substantiated by comments received from the vendors on the Request for Comments document released prior to the finalization of the RFP. (See Exhibit 1.) Subsequent to the release of the RFP, the Software Design Team was formed to acquire in-depth knowledge of PACER and as a by-product to create the third level of detail for the FDR. This effort will be completed by July 31, 1990. The document will then undergo the review and approval process. The approach taken with the FDR is entirely consistent with traditional design methodology, where by you proceed from high level design to detailed design. The information in the FDR was more than sufficient for the System 90 contractor to produce a high level or system design, and subsequently the detailed design. See comment 6. 2. System 90 Phases. We recognized the need for a reasonable approach to the future, We chose to undertake only that degree of work that could be managed within our resource limits. We recognized that we could size future applications so that the System 90 platform i.e.. the hardware, systems software, security , and intelligent front end processor could be built to accommodate new applications. Phase I of the system 90 initiative includes hardware, systems software, security and the PACER application software as well as the System 90 utility that will enable users to reach other FMS applications. Phase II application initiatives have not yet been defined in detail and will probably be given a different name. We are not now working on Phase 2 applications. Instead we are concentrating on PACER. Page 24 GAO/AFMD-O0-M System 90 See comment 7. 3. Following 14 Joint Application Design (JAD) sessions which averaged four days each, FMS produced a two level design document that described 7 PACER functions and detailed each of the 189 processes within those functions. The documentation from these JAD sessions, two volumes, was prepared by a contractor working with a facilitator, the FMS user community, major Federal Program Agencies, Federal Reserve Board, Financial Institutions, FMS Regional Financial Center Directors and staff, as well as FMS project, program and technical staff. See comment 6. 4, The Service is actually evaluating contractor proposals for acquisition of the System 90 hardware, system software, security, and development of a general PACER design. Only after the general PACER design is completed will the contractor prepare a detailed Pacer design. Contractor proposals must also address the detailed design, build, test and implementation phases of PACER. See comment 9. 5. The major tasks of the FEDSIM (GSA) contract are: requirements’ tracking, risk analysis, review ot all System 90 contractor deliverables, development of an independent Pacer test plan, conduct of the PACER preliminary acceptance testing, and test analysis. See comment 10. 6. The digital imaging system is still in the prototype stage. An RFP for a pilot system iS being developed by the Federal Reserve Bank of Boston. Once this contract is awarded and the pilot system is built and successfully tested, an RFP for a production system will be produced. When the production systems are tested and ready for use, the capability to utilize digital images in PACER will be implemented. See comment 11. 7. The cash management and credit administration data basee are not included in Phase I of System 90. The “claimant prof i le” as originally conceived has been abandoned after being reviewed under terms of the Privacy Act. A “claims history” wi I I, however, be included in the PACER data base. Page 26 GAO/AIWlbWM System 90 Appendix III Comments From the Department of theTreaswy See comment 12. 8. Agency accounting services are pow available using an off-the-shelf package. The package is the American Management System’s “Federal Financial System.” System 90 will provide access to this package as part of the System 90 ( PACER 1 deve I opment. This does not involve application development. Access will be provided by routing requests from client agencies through the System 90 front end processor’s, menu to the appropriate computer on which the accounting application resides. See comment 13. 9. The abi 1 ity to “warehouse” payments has always been a part of the PACER design. The capability is described in Volume I of the FDR, page 70, 18.104.22.168, “Benefits of Pacer for FPA’s,” The “payment-timing requirements” refer to the calculation of the amount of time the fully designed PACER system will require to produce a particular payment. Since the system in not yet designed, the payment-timing requirements are not known. See comment 14. 10. The “alterations” the report refers to were not considered because of the direction given in JAD 1 by the Extended Planninq Board (EPB) of FMS. The EPB defined System 90 and PACER and stated that seven Regional Financial Centers were to be assumed. PACER was consciously defined as an integrated data system: not capable of being separated into component parts. See comment 15. 11. “Team members and project officials” who made the statement that security and internal control requirements were not included must have misunderstood the question. As was observed on page 19 of the draft Now on p. 13. report, there are such requirements in various sections of the FDR, but they are not specific enough to be tested. See comment 16. 12. The IV&V contractor will both develop and use an automated requirements traceability matrix in conjunction with FMS, to ensure that all requirements are implemented. Y Page 26 GAO/APMBSO-g6 System BO APpe* nl Comment8 From the Lkpartment of theTreamry See comment 17. 13. The third level of detail has been added to the FDR. No requirements have been changed. See comment 18. 14. An example of the kind of “errors” found by the Software Design Team is shown in Exhibit 2. The characterization seems harsh, since the word “errors” could be so widely interpreted. See comment 19. 15. The IV&V contractor has a task to do an initial risk analysis on PACER high level design and a final risk analysis once PACER is implemented. The initial risk analysis will help FMS to further define security and control objectives before the detailed design is started. The fact that the IV&V contractor is performing this task probably led to the misstatements Now on p, 10 and p. 13. contained on page 15 and the last sentence of the second complete paragraph on page 19. See comment 20. 16. FMS does not believe System 90 falls under the coverage of OMB circular A-127. We have frequently briefed the Department of Treasury’s, OIRM, Budget and Procurement offices, as well as, OMB’s Management and Budget staff regarding System 90. See comment 21. 17. “Recommendations. ” An updated cost/benefit analysis and the new refined version of the FDR containing the third level of detail should significantly allay the concerns addressed in your draft report. Y Page27 GAO/AFMLX90-85 System 90 Appendix III Commenta From the Department of the Treasury The following are GAO'Scomments on the Department of the Treasury’s letter dated July 13, 1990. 1. Discussed in “Agency Comments and Our Evaluation” section of the GAO Comments report. 2. Discussed in “Agency Comments and Our Evaluation” section of the report. 3. Discussed in “Agency Comments and Our Evaluation” section of the report, 4. No change to report needed. Baaed on discussions with several session participants and a review of documentation from the sessions, we deter- mined that the Joint Application Design methodology apparently suc- ceeded in involving users early in the requirements definition effort and in documenting the initial set of functional and data requirements. Exhibit 1 of the enclosure has not been included in our report. Copies are available from GAOfor any interested party. 6. No change to report needed. Exhibit 1 of the enclosure has not been included. Copies are available from GAOfor any interested party. 6. No change to report needed. 7. See comment 4. 8. No change to report needed. 9. No change to report needed. 10. No change to report needed. 11. No change to report needed. 12. No change to report needed. 13. Before this segment of the system can be designed, the Service’s managers must determine the specific payment-timing capabilities needed. Page 28 GAO/AFMD-90-85 System90 . Appendix III Comments From the Department of theTreasnry 14. Discussed in “Agency Comments and Our Evaluation” section of the report. 16. We discussed this point extensively with software design team mem- bers. We have modified the first reference to this issue to clarify our concern that internal control requirements are not being comprehen- sively analyzed. 16. No change to report needed. Although the contractor is to develop a requirements traceability matrix, the Service is ultimately responsible for identifying the requirements to be included in this matrix. 17. No change to report needed. 18. No change to report needed. Exhibit 2 showed that several lines of text were deleted because they were extraneous, not incorrect. Because the software design team had not completed refining all the design seg- ments at the close of our review, we did not assess the magnitude of errors identified. Therefore, we cannot determine whether this exhibit is typical of other modifications. Exhibit 2 is not included in our report, but copies of it are available from GAOfor any interested party. 19. According to NISTguidance, a risk analysis should be conducted during the initiation stage prior to developing internal control objec- tives, Although the contractor will assist in this effort, it is ultimately the Service’s responsibility to develop internal control objectives. As of mid-July 1990, the Service had not awarded a contract for the indepen- dent verification and validation effort. Therefore, we do not believe the report contains misstatements in this regard. 20. Discussed in “Agency Comments and Our Evaluation” section of the report. 2 1. No change to report needed. Page 29 GAO/tiFWHM5 System99 Appendix IV Major Contributors to This Repoport 1 John C. Martin, Assistant Director, (202) 276-9481 Accounting and Jean L. Boltz, Auditor-in-Charge IF’inanciti Management Lisa him, Evaluator Division, Washington, DC. (9014Bt3) Page 30 GAO/AFMD-90-86 System 90 llt~~ut~st,s for cwpies of (;A0 reports shtdtl tw sthnt. t,o: I /Jnited States General Accounting Office Wanhington, D.C. 20848 Official Business Penalty for Private Use $300
Financial Management: Defining Requirements Is Crucial to System 90's Success
Published by the Government Accountability Office on 1990-09-05.
Below is a raw (and likely hideous) rendition of the original report. (PDF)