oversight

Retirement System: Concerns About OPM's FERS Automated Processing System Procurement

Published by the Government Accountability Office on 1990-04-04.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

*.                                                                  2       ,I
                                                                            I
      1Jnited States   General   Accounting   Office

&A0   Report to the Chairman, Committee on
      Government Operations, House of
                                                                        ’
      Representatives        ’



      RETIREMENT
      SYSTEM
      Concerns About OPM’s
      FERS Automated
      Processing System
      Procurement

                                                                8




                                                       141179
    t



                   united
                     states
‘GAO               General Accounting Office
                   Waehington, D.C. 20548

                   Information Management and
                   Technology Division

                   B-238930
                   April 4,199O

                   The Honorable John Conyers, Jr.
                   Chairman, Committee on
                     Government Operations
                   House of Representatives
                   Dear Mr. Chairman:

                   This report respondsto your December11,1989, request that we review
                   a planned $64 million procurement by the Office of Personnel Manage-
                   ment (OPM). OPM plans to develop an automated system to support the
                   Federal Employees Retirement System (FER@and buy hardware compat-
                   ible with International BusinessMachines, Inc. (IBM) equipment for this
                   new system and to enhancethe Civil Service Retirement System.l You
                   asked that we determine whether (1) the request for proposals contains
                   unnecessarily restrictive specifications and conditions and (2) whether
                   OPM received any outside assistancein developing the requirements and
                   the request for proposals. Details on our objectives, scope,and method-
                   ology are included in appendix I.
                   At the time of our review, OPM was in the proposal evaluation phase of
                   this procurement. Information related to the competition, including the
                   number of offerors, is procurement sensitive and thus is not discussedin
                   this report.

                   We found that the technical specifications in the FERS request for pro-
Results in Brief   posals were not unduly restrictive. We also found that OPM did not act
                   improperly in obtaining assistancefrom two outside sourcesin develop-
                   ing the request for proposals. However, in seeking to reduce project risk,
                   OPM required prospective vendors to meet six experience requirements
                   on astrict go/no go basis.2We believe that OPM'S objective of reducing
                   risk was proper but that the go/no go basis for evaluation may have
                   unnecessarily restrictedcompetition for the procurement. Further, even
                   after complaints by potential offerors that the go/no go requirements

                   ‘OPM currently manages two federal retirement systems: PEW and the Civil Service Retirement Sys-
                   tem. PRRS covers over 900,090 federal employees including 9,000 annuitants, and is supported by an
                   automated/manual system. The Civil Service Retirement ‘System covers nearly 2 million federal
                   employees and over 2 million annuit#ts and is supported by an,IBM-compatible automated system.

                   ‘In the request for proposals, the experience requirements are located in Section L.32 and the go/no
                   go evaluation approach is described in Section M. Under a go/no go evaluation, vendors who fail to
                   meet one or more of the requirements are eliminated from the procurement competition.



                   Page 1                                     GAO/IMTJ3GBO-46      Concerns about F’ERSProcurement
                      B229929




                      were overly restrictive, OPM did not adequately evaluate the effect of
                      these requirements on potential competition,

                      Given the magnitude of this procurement and the concernsraised by
                      potential offerors, we believe it would be prudent for OPMto determine if
                      a less restrictive approach, such as awarding offerors points for their
                      experience, would meet the agency’sminimum needs.

                      We recommendthat the OPMDirector not award the FERS Automated
Recommendations       processing System contract until OPM has:
                      reviewed the validity of vendor concernsregarding the impact of these
                      requirements on competition; and
                      determined whether OPM’S   method of evaluating them on a go/no go
                      basis is neededor whether a less restrictive method, such as using a
                      weighted point system to consider vendors’ experience, could ade-
                      quately meet OPM’S  needs.

                      If OPMdetermines that the go/no go experience requirements unnecessa-
                      rily restricted competition, we recommendthat the OPMDirector ensure
                      that the request for proposals is revised and that the procurement is
                      reinitiated.

                      On June 6, 1986, the FERS  Act becamelaw, creating a new retirement
Background            plan for federal employees.FERS  covers most of the 820,000 federal
                      employeeshired after December31, 1983, and around 60,000 employees
                      who transferred to FERS  from the Civil Service Retirement System dur-
                      ing the 1987 open season.OPM administers both the FERSBasic Benefit
                      Plan and the Civil Service Retirement System.
                      Soon after I?EP~was enacted,the OPM Director determined that an auto-
                      mated system was neededto support processingclaims and benefits for
                      future PERS annuitants. In June 1989, after a FERS
                                                                       system conceptual
                      design was developed, OPM issued a request for proposals to hire the ser-
                      vices of a contractor for 8 years to:

                  l design, develop and implement the FERSautomated processingsystem;
                  . acquire IBM-compatiblehardware and software to support development
                    and operation of the FERS system and to upgrade operation of the pre-
                    sent Civil Service Retirement System; and



                      Page 2                         GAO/IMTEG9045   Concerns about FEW Procurement
                           B-228920




                         . provide data-processingservices as required by OPMfor FERSand the
                           Civil Service Retirement System.
                           The FERSsystem project, which is estimated to cost $64 million, is the
                           largest single automated data-processingproject that OPMhas
                           undertaken.

                           In developing its request for proposals for the FERSsystem, OPMsought
OPM’s Mandatory            to reduce project risk by requiring vendors to meet six mandatory
Experience                 experience requirements. OPMfelt that these requirements would ensure
Requirements May           that only experienced vendors could participate. One of these require-
                           ments, for example, states that offerors must demonstrate that they
Have Unduly                have successfully completed at least two large-scalesystems integration
Restricted Competition     projects, involving both the installation of a mainframe computer and
                           design and development of large-scaletransaction processingsystems.
                           (Seeapp. II for a complete list of these requirements.) OPMalso decided
                           to evaluate each vendor’s experience on a go/no go basis, which means
                           that unless a company can meet all six requirements, it will not be con-
                           sidered further for the contract. According to the FERSproject manager,
                           these restrictions were neededto limit competition only to offerors who
                           have credible depth and breadth of experience on similar systems devel-
                           opment projects.
                           After the request for proposals was issued in June 1989, two vendors
                           questioned the mandatory experience requirements, stating that they
                           appeared to be too restrictive and suggestingthat if these requirements
                           were not changed,the vendors would not compete.OPMresponded by
                           stating:
                           “We have reviewed our rationale for constructing the solicitation with the high hur-
                           dle factors as they now appear.3We believe the experiences represented by the high
                           hurdle factors are essential to achieve the objectives of this project in a timely, effi-
                           cient and effective way. Moreover, we have no reason to believe that these provi-
                           sions will preclude a competitive procurement.”

                           OPMmade minor   changesto its experience requirements in responseto
                           the vendors’ comments,but did not changethe go/no go evaluation
                           method.



                           30PMsometimes
                                       refersto its mandatoryexperience
                                                                      requirements
                                                                                as“high hurdles.”


                           Page 3                                GAO/IMTJW9O-43Concerns about FERSProcurement
                                                                                               I




o&s decision to evaluate its experience requirements on a strict go/no
go basis restricted the competition to firms that could meet all those
requirements. We agree with OPM’S objective of minimizing project risk.
However, we are not aware of any reason why OPM could not have ade-
quately addressedthis issue with a less restrictive evaluation approach,
for instance, one that would have given vendors the opportunity to
demonstrate equivalent or even superior experience as a result of simi-
lar or more difficult undertakings.
OPMstaff said that they did not refer vendor complaints about the
experience criteria and the go/no go evaluation to OPM'SCompetition
Advocate4becausethey viewed the complaints as technical issues,not
issuesrelated to competition. OPM staff said that they believed the pro-
curement would be competitive, despite vendor misgivings about the
mandatory experience requirements. This belief was basedprimarily on
the number of vendors who visited the reading room (72), attended the
pre-proposal conference(32), and visited the FFXSprocessingsite in
Pennsylvania (8).
We believe that OPM'Smethod of estimating potential vendor interest
was inadequate and that OPMshould have taken greater steps to assure
that the request for proposals would promote maximum competition. In
particular, we believe OPM erred in concluding that the vendors’ concerns
about the experience requirements were technical issuesthat would not
deter competition. Both complaints specifically challenged the restric-
tiveness of the go/no go experience criteria and both clearly indicated
the vendors, who had spent considerable resourcespreparing for the
competition, would not participate further if the requirements were not
changed.
We believe it was unreasonable for OPMnot to have recognizedthe ven-
dors’ concernsas evidencethat these experience requirements may have
been unnecessarily restrictive. Moreover, given these clear challengesto
the restrictiveness of the solicitation and threatened withdrawals by
vendors, OPMstaff should have been alerted that the competition may be
unduly limited. Attendance by potential offerors at specific OPM events
and locations is questionable support for the agency’sconclusion that its



4The Competition in Contracting Act requires an executive agency to designate a Competition Advo-
cate who is responsible for “challenging barriers to and promoting full and open competition.” 41
USC. 418 (1984).



Page 4                                    GAO/IMTJZG9046 Concerns about FER!SProcurement
                         B228920




                         request for proposals was appropriately competitive. Further, attend-
                         ance at events is not relevant to determining whether the go/no go eval-
                         uation approach was necessary.

                         Our review of the technical specifications in the FEN request for propos-
Technical                als did not reveal any requirements that posed an unnecessarily
Specifications Did Not   restricted competition. The request for proposals contained functional
Unnecessarily Restrict   specifications, which, according to the Federal Information Resources
                         ManagementRegulation Part 201-30.013-1,are the most desirable for
Competition              maximizing competition. Also, the procurement did not restrict equip-
                         ment to any specific type, such as a particular size computer that only
                         one manufacturer could supply. Further, vendors said that they
                         believed the technical specifications were competitive and did not favor
                         any particular vendor. Finally, while the technical specifications limit
                         the procurement to IBM-compatiblecomputers and peripheral equipment,
                         OPMproperly justified this restriction basedon agency needsand a study
                         that estimated it would cost over $21 million to convert software to a
                         non-IBM-compatibleformat.

                         OPMproperly   obtained help from two outside contractors-American
OPM’s Use of             ManagementSystems,Inc. and Compuware Corporation-in developing
Contractors to Help      the FERSrequest for proposals. OPMcontracted with American Manage-
Develop the FERS         ment Systems, a consulting firm, to perform a variety of services,
                         including writing and reviewing portions of the request for proposals.
Request for Proposals    The American ManagementSystems staff helped write the two sections
Was Proper               of the request for proposals that contain the experience requirements
                         and the evaluation criteria. Becauseof the contractor’s extensive
                         involvement in developing and reviewing the document, OPMproperly
                         precluded American ManagementSystems from competing on future
                         FERSsystem contracts.

                         OPMalso obtained help from   on-site Compuware personnel in writing the
                         hardware technical specifications in the request for proposals. OPMhas
                         an ongoing contract with Compuware to provide software and hardware
                         support services for the Civil Service Retirement System. Becausethe
                         assistancewas limited to compiling information on the amount and size
                         of computers neededto run FERS,OPMproperly determined that Com-
                         puware would not have an unfair competitive advantage and did not
                         prohibit the firm from competing in the procurement.




                         Page 5                         GAO/lMTEGaO-46
                                                                    ConcermJabout FEW3Procurement
Is-228920




We conducted our review from December1989 through March 1990 at
OPM headquarters in  Washington, DC., and at vendor locations in the
Washington, D.C. metropolitan area. Our review was conducted in
accordancewith generally acceptedgovernment auditing standards. In
accordancewith your wishes, we did not obtain official agency com-
ments on a draft of this report; however, we discussedour review with
OPMofficials and have incorporated their views where appropriate.

As arranged with your office, unless you publicly announceits contents
earlier, we plan no further distribution of this report until 30 days after
the date of this letter. At that time, we will send copiesto the Chairman
of the SenateCommittee on Governmental Affairs; the Director, OPM;
other interested parties; and will make copies available to others upon
request. This report was prepared under the direction of Jack L. Brock,
Jr., Director, Government Information and Financial Management
Issues,who can be reached at (202) 276-3196.Other major contributors
are listed in appendix III.
Sincerely yours,




Ralph V. Carlone
Assistant Comptroller General




Page 6                          GAO/IMTEGaO-4S
                                            Concerns about FERSProcurement
Page 7   GAO/lMTEG9O-46   Concerns about FEW Procurement
Letter
Appendix I
Objectives, Scope,and
Methodology
Appendix II
Vendor Experience
Criteria
Appendix III
Major Contributors to
This Report




                        Abbreviations

                        ADP       Automated Data Processing
                        FAPS      FERSAutomated ProcessingSystem
                        FERS      Federal Employees Retirement System
                        GAO       General Accounting Office
                        IBM       International BusinessMachines
                        IMTEC     Information Managementand Technology Division
                        OPM       Office of PersonnelManagement


                        Page8                        GAO/IMTEC90-46 Concerns about F’ElRSProcurement
,




    Page a   GAO/IMTEC90-45 Concerns about FERSProcurement
Appendix I

Objectives, Scope,and Methodology


               On December11,1989, the Chairman, HouseCommittee on Government
               Operations, asked that we review a $64 million procurement by OPM to
               (1) develop an automated system that would support FERS and (2)
               acquire IBM-compatiblecomputer hardware for both the new FERS      sys-
               tem and the Civil Service Retirement System. Our specific objectives for
               this assignment were to determine if (1) the procurement contained
               unnecessarily restrictive specifications and conditions, and (2) OPM
               received assistancefrom any outside sourcesin developing the require-
               ments and the request for proposals for the procurement.
               To determine if the procurement contained unnecessarily restrictive
               specifications and conditions, we reviewed the provisions of the Compe-
               tition in Contracting Act, the Federal Acquisition Regulation, and the
               Federal Information ResourcesManagementRegulation regarding com-
               petition and vendor experience requirements. Further, we discussed
               with private industry officials any requirements they believed were
               overly restrictive. We also interviewed OPM officials to determine their
               rationale for having restrictive requirements, examined OPM documents
               regarding the types of restrictions neededfor the procurement,
               reviewed information from Datapro’ to determine if the requirements
               were overly restrictive, and assessedOPM’S software conversion study,
               which concluded that the FERS   system should be IBM-compatible.
               To determine what assistanceOPM received from outside sources,we
               interviewed FERS   Design Division staff, who developed the FERSsystem
                request for proposals. Additionally, we reviewed OPM documents related
               to the FXRS system development project including the conceptual design
               for FERS, draft statement of work for the FERSsystem, FERS system pro-
               ject summary, consulting contract with American ManagementSystems,
                and FERS system request for proposals. We interviewed American Man-
               agementSystems and Compuware personnel to determine what assis-
               tance they gave to OPM in developing the request for proposals.
               Our review was conducted at OPM Headquarters in Washington, D.C.,
               and at vendor locations in the Washington, D.C. metropolitan area. Our
               review was conducted from December1989 through March 1990. We
               performed our work in accordancewith generally acceptedgovernment
               auditing standards.



               ‘Datapro is a trade publication that provides detailed information on computen, peripheral equip
               ment, and software.



               Page 10                                   GAO/IMTEGBO~ Concerns about FJ3RSProcurement
Appendix II

Vendor Ekperience Criteria


               Section L.32 of the FERS Automated ProcessingSystem (FAFS) request for
               proposals contains six experience criteria that vendors must meet on a
               go/no go basis. The criteria presented below are taken verbatim from
               the FAF%request for proposals.
               Criterion #l: Defined Benefit Programs. Minimum of two (2) major
               projects successfully completed by the Offeror and/or subcontractor(s)
               in support of the development, implementation and/or improvement of
               public or private sector defined benefit pension programs (each covering
               a minimum of 16,000 employees) similar in character and complexity to
               the Federal Employees Retirement System. Projects should show an in-
               depth understanding of these programs and the ability to apply that
               understanding to the development of new systems. It is desirable that
               the prime contractor have defined benefits experience.
               Criterion #2: Applications Development. Minimum of two (2) application
               development projects successfully completed by the Offeror and/or sub-
               contractor(s) which are similar in size, scopeand complexity to FAFB,
               where “size, scope,and complexity” are represented by full life-cycle
               development, large scale on-line business-typetransaction processing
               application, at least 200,000 lines of custom COBOLcode (or its equiva-
               lent if a 4GL is employed), use of data managementsoftware to support
               a 260,000+ record data base and use of the methodologies and tools pro-
               posed for FAPS. It is desirable that the prime contractor have this
               experience.
               Criterion #3: Large ScaleSystemsIntegration. Minimum of two (2) suc-
               cessful projects completed by the Offeror which included substantially
               the samemajor tasks as FAPS - installation of a mainframe-based plat-
               form and design and development of one or more large scale transaction
               processing applic?&ns integral to the operations of the client organiza-
               tion - in which the Offeror had primary responsibility for the integra-
               tion of all major technology components.A mainframe-based platform in
               this context is defined as one whose characteristics (size, capacity,
               throughput, volume, etc.) are similar in size, scopeand complexity to the
               proposed equipment. Projects where a subcontractor served as the
               prime contractor may not be used to meet the requirements of this
               criterion.
               Criterion #4: Large ScaleData Conversion. Minimum of two (2) projects
               successfully completed by the Offeror and/or subcontractor(s) which
               demonstrate the ability to convert large, nonstandard, complex files con-
               taining either manual or automated/manual records, where large is at


               Page 11                        GAO/lMTEC9O4 Concerns about FERSProcurement
Appendix II
Vendor Experience Criterh




least 260,000 records. Projects should show the conversion time
expended and should demonstrate an understanding of the technical and
management complexities of the effort.

Criterion #6: Federal Agencies and Programs. Minimum of two (2) major
projects successfully completed by the Offeror and/or subcontractor(s)
which demonstrate: (1) an in-depth understanding and ability to comply
with Federal policies for accounting and financial management,informa-
tion resourcesmanagement,and ADP; and (2) an understanding of the
environment in which new Federal programs are implemented.
Criterion #6: Installation and Upgrade of Computing Facilities. Minimum
of two (2) major projects successfully completed by the Offeror and/or
subcontractor in which a similar technical platform was installed. The
platform should encompasssystems software and utilities as well as a
large mainframe host and associatedperipherals. As used in this con-
text, a large mainframe host is defined as one whose characteristics
(size, capacity, throughput, volume, etc.) are similar in size, scopeand
complexity to the proposed equipment. It is desirable to cite projects in
which the initial platform was upgraded in a manner similar to that
envisioned for FAP~.




Psge 12                        GAO/IMTEG9&46 Concems about FERSProcurement
Appendix III

‘Major Contributors to This Report


Information             Frank W. Deffer, Evaluator-in-Charge
Management and          Peter C. Wade,Staff Evaluator
                        Matthew D. Ryan, Staff Evaluator
Technology Division,
Washington, DC.
                        Jerold D. Cohen,Assistant General Counsel
Office of the General   John A. Carter, Senior Attorney
Counsel




(aloaM)                 Page 18                       GAO/MTEWM4    Concmmedent m   -t
                                              .,.   ,,   .   ,/
,,,   ,,,   ,“,,?   ,,,,   88,   ,,*   1/11
  First,-(3ass Mail
Post;kgt5 & Ftws Paid
           GAO
  Permit    No. GlOO