*. 2 ,I I 1Jnited States General Accounting Office &A0 Report to the Chairman, Committee on Government Operations, House of ’ Representatives ’ RETIREMENT SYSTEM Concerns About OPM’s FERS Automated Processing System Procurement 8 141179 t united states ‘GAO General Accounting Office Waehington, D.C. 20548 Information Management and Technology Division B-238930 April 4,199O The Honorable John Conyers, Jr. Chairman, Committee on Government Operations House of Representatives Dear Mr. Chairman: This report respondsto your December11,1989, request that we review a planned $64 million procurement by the Office of Personnel Manage- ment (OPM). OPM plans to develop an automated system to support the Federal Employees Retirement System (FER@and buy hardware compat- ible with International BusinessMachines, Inc. (IBM) equipment for this new system and to enhancethe Civil Service Retirement System.l You asked that we determine whether (1) the request for proposals contains unnecessarily restrictive specifications and conditions and (2) whether OPM received any outside assistancein developing the requirements and the request for proposals. Details on our objectives, scope,and method- ology are included in appendix I. At the time of our review, OPM was in the proposal evaluation phase of this procurement. Information related to the competition, including the number of offerors, is procurement sensitive and thus is not discussedin this report. We found that the technical specifications in the FERS request for pro- Results in Brief posals were not unduly restrictive. We also found that OPM did not act improperly in obtaining assistancefrom two outside sourcesin develop- ing the request for proposals. However, in seeking to reduce project risk, OPM required prospective vendors to meet six experience requirements on astrict go/no go basis.2We believe that OPM'S objective of reducing risk was proper but that the go/no go basis for evaluation may have unnecessarily restrictedcompetition for the procurement. Further, even after complaints by potential offerors that the go/no go requirements ‘OPM currently manages two federal retirement systems: PEW and the Civil Service Retirement Sys- tem. PRRS covers over 900,090 federal employees including 9,000 annuitants, and is supported by an automated/manual system. The Civil Service Retirement ‘System covers nearly 2 million federal employees and over 2 million annuit#ts and is supported by an,IBM-compatible automated system. ‘In the request for proposals, the experience requirements are located in Section L.32 and the go/no go evaluation approach is described in Section M. Under a go/no go evaluation, vendors who fail to meet one or more of the requirements are eliminated from the procurement competition. Page 1 GAO/IMTJ3GBO-46 Concerns about F’ERSProcurement B229929 were overly restrictive, OPM did not adequately evaluate the effect of these requirements on potential competition, Given the magnitude of this procurement and the concernsraised by potential offerors, we believe it would be prudent for OPMto determine if a less restrictive approach, such as awarding offerors points for their experience, would meet the agency’sminimum needs. We recommendthat the OPMDirector not award the FERS Automated Recommendations processing System contract until OPM has: reviewed the validity of vendor concernsregarding the impact of these requirements on competition; and determined whether OPM’S method of evaluating them on a go/no go basis is neededor whether a less restrictive method, such as using a weighted point system to consider vendors’ experience, could ade- quately meet OPM’S needs. If OPMdetermines that the go/no go experience requirements unnecessa- rily restricted competition, we recommendthat the OPMDirector ensure that the request for proposals is revised and that the procurement is reinitiated. On June 6, 1986, the FERS Act becamelaw, creating a new retirement Background plan for federal employees.FERS covers most of the 820,000 federal employeeshired after December31, 1983, and around 60,000 employees who transferred to FERS from the Civil Service Retirement System dur- ing the 1987 open season.OPM administers both the FERSBasic Benefit Plan and the Civil Service Retirement System. Soon after I?EP~was enacted,the OPM Director determined that an auto- mated system was neededto support processingclaims and benefits for future PERS annuitants. In June 1989, after a FERS system conceptual design was developed, OPM issued a request for proposals to hire the ser- vices of a contractor for 8 years to: l design, develop and implement the FERSautomated processingsystem; . acquire IBM-compatiblehardware and software to support development and operation of the FERS system and to upgrade operation of the pre- sent Civil Service Retirement System; and Page 2 GAO/IMTEG9045 Concerns about FEW Procurement B-228920 . provide data-processingservices as required by OPMfor FERSand the Civil Service Retirement System. The FERSsystem project, which is estimated to cost $64 million, is the largest single automated data-processingproject that OPMhas undertaken. In developing its request for proposals for the FERSsystem, OPMsought OPM’s Mandatory to reduce project risk by requiring vendors to meet six mandatory Experience experience requirements. OPMfelt that these requirements would ensure Requirements May that only experienced vendors could participate. One of these require- ments, for example, states that offerors must demonstrate that they Have Unduly have successfully completed at least two large-scalesystems integration Restricted Competition projects, involving both the installation of a mainframe computer and design and development of large-scaletransaction processingsystems. (Seeapp. II for a complete list of these requirements.) OPMalso decided to evaluate each vendor’s experience on a go/no go basis, which means that unless a company can meet all six requirements, it will not be con- sidered further for the contract. According to the FERSproject manager, these restrictions were neededto limit competition only to offerors who have credible depth and breadth of experience on similar systems devel- opment projects. After the request for proposals was issued in June 1989, two vendors questioned the mandatory experience requirements, stating that they appeared to be too restrictive and suggestingthat if these requirements were not changed,the vendors would not compete.OPMresponded by stating: “We have reviewed our rationale for constructing the solicitation with the high hur- dle factors as they now appear.3We believe the experiences represented by the high hurdle factors are essential to achieve the objectives of this project in a timely, effi- cient and effective way. Moreover, we have no reason to believe that these provi- sions will preclude a competitive procurement.” OPMmade minor changesto its experience requirements in responseto the vendors’ comments,but did not changethe go/no go evaluation method. 30PMsometimes refersto its mandatoryexperience requirements as“high hurdles.” Page 3 GAO/IMTJW9O-43Concerns about FERSProcurement I o&s decision to evaluate its experience requirements on a strict go/no go basis restricted the competition to firms that could meet all those requirements. We agree with OPM’S objective of minimizing project risk. However, we are not aware of any reason why OPM could not have ade- quately addressedthis issue with a less restrictive evaluation approach, for instance, one that would have given vendors the opportunity to demonstrate equivalent or even superior experience as a result of simi- lar or more difficult undertakings. OPMstaff said that they did not refer vendor complaints about the experience criteria and the go/no go evaluation to OPM'SCompetition Advocate4becausethey viewed the complaints as technical issues,not issuesrelated to competition. OPM staff said that they believed the pro- curement would be competitive, despite vendor misgivings about the mandatory experience requirements. This belief was basedprimarily on the number of vendors who visited the reading room (72), attended the pre-proposal conference(32), and visited the FFXSprocessingsite in Pennsylvania (8). We believe that OPM'Smethod of estimating potential vendor interest was inadequate and that OPMshould have taken greater steps to assure that the request for proposals would promote maximum competition. In particular, we believe OPM erred in concluding that the vendors’ concerns about the experience requirements were technical issuesthat would not deter competition. Both complaints specifically challenged the restric- tiveness of the go/no go experience criteria and both clearly indicated the vendors, who had spent considerable resourcespreparing for the competition, would not participate further if the requirements were not changed. We believe it was unreasonable for OPMnot to have recognizedthe ven- dors’ concernsas evidencethat these experience requirements may have been unnecessarily restrictive. Moreover, given these clear challengesto the restrictiveness of the solicitation and threatened withdrawals by vendors, OPMstaff should have been alerted that the competition may be unduly limited. Attendance by potential offerors at specific OPM events and locations is questionable support for the agency’sconclusion that its 4The Competition in Contracting Act requires an executive agency to designate a Competition Advo- cate who is responsible for “challenging barriers to and promoting full and open competition.” 41 USC. 418 (1984). Page 4 GAO/IMTJZG9046 Concerns about FER!SProcurement B228920 request for proposals was appropriately competitive. Further, attend- ance at events is not relevant to determining whether the go/no go eval- uation approach was necessary. Our review of the technical specifications in the FEN request for propos- Technical als did not reveal any requirements that posed an unnecessarily Specifications Did Not restricted competition. The request for proposals contained functional Unnecessarily Restrict specifications, which, according to the Federal Information Resources ManagementRegulation Part 201-30.013-1,are the most desirable for Competition maximizing competition. Also, the procurement did not restrict equip- ment to any specific type, such as a particular size computer that only one manufacturer could supply. Further, vendors said that they believed the technical specifications were competitive and did not favor any particular vendor. Finally, while the technical specifications limit the procurement to IBM-compatiblecomputers and peripheral equipment, OPMproperly justified this restriction basedon agency needsand a study that estimated it would cost over $21 million to convert software to a non-IBM-compatibleformat. OPMproperly obtained help from two outside contractors-American OPM’s Use of ManagementSystems,Inc. and Compuware Corporation-in developing Contractors to Help the FERSrequest for proposals. OPMcontracted with American Manage- Develop the FERS ment Systems, a consulting firm, to perform a variety of services, including writing and reviewing portions of the request for proposals. Request for Proposals The American ManagementSystems staff helped write the two sections Was Proper of the request for proposals that contain the experience requirements and the evaluation criteria. Becauseof the contractor’s extensive involvement in developing and reviewing the document, OPMproperly precluded American ManagementSystems from competing on future FERSsystem contracts. OPMalso obtained help from on-site Compuware personnel in writing the hardware technical specifications in the request for proposals. OPMhas an ongoing contract with Compuware to provide software and hardware support services for the Civil Service Retirement System. Becausethe assistancewas limited to compiling information on the amount and size of computers neededto run FERS,OPMproperly determined that Com- puware would not have an unfair competitive advantage and did not prohibit the firm from competing in the procurement. Page 5 GAO/lMTEGaO-46 ConcermJabout FEW3Procurement Is-228920 We conducted our review from December1989 through March 1990 at OPM headquarters in Washington, DC., and at vendor locations in the Washington, D.C. metropolitan area. Our review was conducted in accordancewith generally acceptedgovernment auditing standards. In accordancewith your wishes, we did not obtain official agency com- ments on a draft of this report; however, we discussedour review with OPMofficials and have incorporated their views where appropriate. As arranged with your office, unless you publicly announceits contents earlier, we plan no further distribution of this report until 30 days after the date of this letter. At that time, we will send copiesto the Chairman of the SenateCommittee on Governmental Affairs; the Director, OPM; other interested parties; and will make copies available to others upon request. This report was prepared under the direction of Jack L. Brock, Jr., Director, Government Information and Financial Management Issues,who can be reached at (202) 276-3196.Other major contributors are listed in appendix III. Sincerely yours, Ralph V. Carlone Assistant Comptroller General Page 6 GAO/IMTEGaO-4S Concerns about FERSProcurement Page 7 GAO/lMTEG9O-46 Concerns about FEW Procurement Letter Appendix I Objectives, Scope,and Methodology Appendix II Vendor Experience Criteria Appendix III Major Contributors to This Report Abbreviations ADP Automated Data Processing FAPS FERSAutomated ProcessingSystem FERS Federal Employees Retirement System GAO General Accounting Office IBM International BusinessMachines IMTEC Information Managementand Technology Division OPM Office of PersonnelManagement Page8 GAO/IMTEC90-46 Concerns about F’ElRSProcurement , Page a GAO/IMTEC90-45 Concerns about FERSProcurement Appendix I Objectives, Scope,and Methodology On December11,1989, the Chairman, HouseCommittee on Government Operations, asked that we review a $64 million procurement by OPM to (1) develop an automated system that would support FERS and (2) acquire IBM-compatiblecomputer hardware for both the new FERS sys- tem and the Civil Service Retirement System. Our specific objectives for this assignment were to determine if (1) the procurement contained unnecessarily restrictive specifications and conditions, and (2) OPM received assistancefrom any outside sourcesin developing the require- ments and the request for proposals for the procurement. To determine if the procurement contained unnecessarily restrictive specifications and conditions, we reviewed the provisions of the Compe- tition in Contracting Act, the Federal Acquisition Regulation, and the Federal Information ResourcesManagementRegulation regarding com- petition and vendor experience requirements. Further, we discussed with private industry officials any requirements they believed were overly restrictive. We also interviewed OPM officials to determine their rationale for having restrictive requirements, examined OPM documents regarding the types of restrictions neededfor the procurement, reviewed information from Datapro’ to determine if the requirements were overly restrictive, and assessedOPM’S software conversion study, which concluded that the FERS system should be IBM-compatible. To determine what assistanceOPM received from outside sources,we interviewed FERS Design Division staff, who developed the FERSsystem request for proposals. Additionally, we reviewed OPM documents related to the FXRS system development project including the conceptual design for FERS, draft statement of work for the FERSsystem, FERS system pro- ject summary, consulting contract with American ManagementSystems, and FERS system request for proposals. We interviewed American Man- agementSystems and Compuware personnel to determine what assis- tance they gave to OPM in developing the request for proposals. Our review was conducted at OPM Headquarters in Washington, D.C., and at vendor locations in the Washington, D.C. metropolitan area. Our review was conducted from December1989 through March 1990. We performed our work in accordancewith generally acceptedgovernment auditing standards. ‘Datapro is a trade publication that provides detailed information on computen, peripheral equip ment, and software. Page 10 GAO/IMTEGBO~ Concerns about FJ3RSProcurement Appendix II Vendor Ekperience Criteria Section L.32 of the FERS Automated ProcessingSystem (FAFS) request for proposals contains six experience criteria that vendors must meet on a go/no go basis. The criteria presented below are taken verbatim from the FAF%request for proposals. Criterion #l: Defined Benefit Programs. Minimum of two (2) major projects successfully completed by the Offeror and/or subcontractor(s) in support of the development, implementation and/or improvement of public or private sector defined benefit pension programs (each covering a minimum of 16,000 employees) similar in character and complexity to the Federal Employees Retirement System. Projects should show an in- depth understanding of these programs and the ability to apply that understanding to the development of new systems. It is desirable that the prime contractor have defined benefits experience. Criterion #2: Applications Development. Minimum of two (2) application development projects successfully completed by the Offeror and/or sub- contractor(s) which are similar in size, scopeand complexity to FAFB, where “size, scope,and complexity” are represented by full life-cycle development, large scale on-line business-typetransaction processing application, at least 200,000 lines of custom COBOLcode (or its equiva- lent if a 4GL is employed), use of data managementsoftware to support a 260,000+ record data base and use of the methodologies and tools pro- posed for FAPS. It is desirable that the prime contractor have this experience. Criterion #3: Large ScaleSystemsIntegration. Minimum of two (2) suc- cessful projects completed by the Offeror which included substantially the samemajor tasks as FAPS - installation of a mainframe-based plat- form and design and development of one or more large scale transaction processing applic?&ns integral to the operations of the client organiza- tion - in which the Offeror had primary responsibility for the integra- tion of all major technology components.A mainframe-based platform in this context is defined as one whose characteristics (size, capacity, throughput, volume, etc.) are similar in size, scopeand complexity to the proposed equipment. Projects where a subcontractor served as the prime contractor may not be used to meet the requirements of this criterion. Criterion #4: Large ScaleData Conversion. Minimum of two (2) projects successfully completed by the Offeror and/or subcontractor(s) which demonstrate the ability to convert large, nonstandard, complex files con- taining either manual or automated/manual records, where large is at Page 11 GAO/lMTEC9O4 Concerns about FERSProcurement Appendix II Vendor Experience Criterh least 260,000 records. Projects should show the conversion time expended and should demonstrate an understanding of the technical and management complexities of the effort. Criterion #6: Federal Agencies and Programs. Minimum of two (2) major projects successfully completed by the Offeror and/or subcontractor(s) which demonstrate: (1) an in-depth understanding and ability to comply with Federal policies for accounting and financial management,informa- tion resourcesmanagement,and ADP; and (2) an understanding of the environment in which new Federal programs are implemented. Criterion #6: Installation and Upgrade of Computing Facilities. Minimum of two (2) major projects successfully completed by the Offeror and/or subcontractor in which a similar technical platform was installed. The platform should encompasssystems software and utilities as well as a large mainframe host and associatedperipherals. As used in this con- text, a large mainframe host is defined as one whose characteristics (size, capacity, throughput, volume, etc.) are similar in size, scopeand complexity to the proposed equipment. It is desirable to cite projects in which the initial platform was upgraded in a manner similar to that envisioned for FAP~. Psge 12 GAO/IMTEG9&46 Concems about FERSProcurement Appendix III ‘Major Contributors to This Report Information Frank W. Deffer, Evaluator-in-Charge Management and Peter C. Wade,Staff Evaluator Matthew D. Ryan, Staff Evaluator Technology Division, Washington, DC. Jerold D. Cohen,Assistant General Counsel Office of the General John A. Carter, Senior Attorney Counsel (aloaM) Page 18 GAO/MTEWM4 Concmmedent m -t .,. ,, . ,/ ,,, ,,, ,“,,? ,,,, 88, ,,* 1/11 First,-(3ass Mail Post;kgt5 & Ftws Paid GAO Permit No. GlOO
Retirement System: Concerns About OPM's FERS Automated Processing System Procurement
Published by the Government Accountability Office on 1990-04-04.
Below is a raw (and likely hideous) rendition of the original report. (PDF)