oversight

National Laboratories: DOE Needs to Assess the Impact of Using Performance-Based Contracts

Published by the Government Accountability Office on 1999-05-07.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                  United States General Accounting Office

GAO               Report to the Committee on Science,
                  House of Representatives



May 1999
                  NATIONAL
                  LABORATORIES
                  DOE Needs to Assess
                  the Impact of Using
                  Performance-Based
                  Contracts




GAO/RCED-99-141
                       United States
GAO                    General Accounting Office
                       Washington, D.C. 20548

                       Resources, Community, and
                       Economic Development Division

                       B-282356

                       May 7, 1999

                       The Honorable F. James Sensenbrenner, Jr.
                       Chairman
                       The Honorable George E. Brown, Jr.
                       Ranking Minority Member
                       Committee on Science
                       House of Representatives

                       The Department of Energy (DOE) contracts with private companies and
                       educational institutions to manage and operate 18 of its 22 laboratories.
                       These are cost reimbursement contracts under which DOE pays all of its
                       contractors’ allowable costs. DOE can also provide a fee, or profit, to a
                       contractor for managing a laboratory. Responding to criticism that its
                       historical contracting practices were costly and inefficient, DOE switched
                       to performance-based contracts in 1994 as part of its contract reform
                       program. Use of these contracts allows DOE to structure each contract to
                       provide a clear statement of what needs to be accomplished—rather than
                       providing broad statements of work—and to rely on performance
                       measures to evaluate a contractor’s progress toward meeting its
                       objectives. An important feature of performance-based contracting is
                       providing incentives, including fees, to the contractor’s achievement of
                       objectives as a means of encouraging superior performance and lowering
                       costs. Concerned about the progress made to implement
                       performance-based contracting at the national laboratories, you asked us
                       to

                   •   assess the status of performance-based contracting in DOE’s national
                       laboratory contracts, and
                   •   identify efforts being made to determine the impact of performance-based
                       contracting.


                       DOE’s use of performance-based contracting for its laboratories is in a state
Results in Brief       of transition. While all laboratory contracts we examined had some
                       performance-based features, we found wide variance in the number of
                       performance measures and the types of fees negotiated. About half of the
                       18 laboratory contracts have performance fees to encourage superior
                       performance—a major goal of performance-based contracting. Most of the
                       remaining laboratory contracts are still based on DOE’s traditional fixed-fee
                       arrangement in which the fees are paid regardless of performance.




                       Page 1                            GAO/RCED-99-141 Performance-Based Contracting
             B-282356




             DOE has not evaluated the impact of performance-based contracting on its
             laboratory contractors and, as a result, does not know if this new form of
             contracting is achieving the intended results of improved performance and
             lower costs. Specifically, DOE has not determined whether giving higher
             fees to encourage superior performance by its laboratory contractors is
             advantageous to the government, although we recommended in 1994 that
             DOE develop criteria for measuring the costs and benefits to the
             government of using higher fees.1 Fees for the laboratories totaled over
             $100 million for fiscal year 1998. While the contractors were unable to cite
             measurable benefits achieved by switching to performance-based
             contracting, they support its goals. The main benefits from
             performance-based contracting cited by laboratory contractors was that it
             has helped DOE clarify what it expects from the contractors and that it has
             improved communication.


             DOE manages the largest laboratory system of its kind in the world. Since
Background   the early days of the World War II Manhattan Project, DOE’s laboratories
             have played a major role in maintaining U.S. leadership in research and
             development. DOE is responsible for ensuring that the laboratory
             system—with 22 laboratories in 14 states, a combined budget of over
             $10 billion a year, and a staff of about 60,000—is managed in an effective,
             efficient, and economical manner. DOE contracts with educational
             institutions and private sector organizations for the management and
             operation of 18 of its laboratories. (App. I lists DOE’s national laboratories.)
             The remaining four laboratories are staffed by federal employees.

             DOE pays its laboratory contractors all allowable costs. DOE can also pay
             contractors a separate fee, or profit, as compensation for operating the
             laboratories. Fees are based on the contract value and the technical
             complexity of the work to be performed at a laboratory, but also on the
             degree of financial liability or risk that a contractor is willing to assume.
             Under performance-based contracting principles, fees can include both a
             fixed amount and an amount that is linked to achieving performance
             objectives. One of DOE’s major goals in performance-based contracting is
             to develop performance objectives for each contractor that are specific,
             results-oriented, measurable, and reflect the most critical activities.




             1
              Energy Management: Modest Reforms Made in University of California Contracts, but Fees Are
             Substantially Higher (GAO/RCED-94-202, Aug. 25, 1994).



             Page 2                                      GAO/RCED-99-141 Performance-Based Contracting
                            B-282356




                            DOE’s implementation of performance-based contracting for its
Performance-Based           laboratories is in a state of transition. While most of its laboratory
Contracting at the          contracts contain some performance-based features, the contracts
National Laboratories       negotiated by DOE vary from contract to contract. For example, DOE is
                            incorporating performance-based features in all of its laboratory contracts,
Is an Evolving Process      although measures vary substantially in number, ranging from a low of 7 in
                            one laboratory contract to about 250 in another. Also, DOE has negotiated
                            performance fees in only 9 of its 18 laboratory contracts because the
                            remaining laboratories are still operating under DOE’s traditional approach
                            in which fees are not linked to performance. We found that similar
                            laboratories managed by similar contractors have different contracts. The
                            wide diversity of contract features reflects DOE’s philosophy of relying on
                            DOE field units to tailor contracts to local conditions and contractors’
                            preferences.


Developing the Right        Since introducing performance-based contracting in 1994, DOE and its
Performance Measures Is a   laboratory contractors have struggled to find the right mix of measures
Challenge                   that accurately and reliably capture the contractors’ performance.
                            According to DOE field staff, in the early years of contract reform, DOE
                            encouraged its field units to construct as many measures as they could,
                            but provided limited guidance on how to accomplish this task. As a result,
                            early attempts led to large numbers of performance measures. A large
                            number of measures diminishes the importance of any single measure,
                            whereas a small number results in measures that are too broad to be
                            meaningful. For example, a DOE field official told us,

                            “The original guidance from DOE Headquarters was to [develop
                            performance measures] as much as possible. Unfortunately, there was
                            inadequate guidance on how to do this. . . . The number of performance
                            measures . . . is too large. However, if we fail to cover an activity [with a
                            measure] the contractor may not give the attention needed to the
                            activity.”

                            DOE and its laboratories are still attempting to develop the right number of
                            measures. For example, we found that the number of performance
                            measures in the laboratory contracts we examined ranged from a low of 7
                            measures at the Ames Laboratory in Iowa to about 250 at the Idaho
                            National Engineering and Environmental Laboratory in Idaho.

                            DOE and its contractors are also working to develop measures that reliably
                            address the most important activities of the laboratories. According to a



                            Page 3                              GAO/RCED-99-141 Performance-Based Contracting
                          B-282356




                          field official, DOE’s early attempts at developing performance measures
                          resulted in contractors focusing only on those activities that were tied to
                          performance fees, while neglecting other important activities. Another DOE
                          site official stated,

                          “[P]erformance-based contracting tends to focus too much on the
                          monetary reward . . . and less on an analysis of performance. The incentive
                          at the labs should be [for] good science, not more dollars.”

                          Developing the right number and type of performance measures is an
                          evolving process between DOE and its contractors. Most DOE and
                          contractor representatives told us that they are making progress in finding
                          measures that accurately and reliably reflect performance, particularly in
                          management and operations activities. Measuring a contractor’s
                          performance in science and technology is more difficult. Science and
                          technology measures are broader in scope and typically rely on peer
                          reviews and a contractor’s self-assessment for evaluating performance.


Types of Fees Paid to     Although performance fees are a major feature of performance-based
Contractors Vary Widely   contracting, only 9 of the Department’s 18 laboratory contracts have them.
                          Nine of the remaining laboratory contracts operate under DOE’s traditional
                          fixed-fee arrangement, and one laboratory contract has no fee. Fixed fees
                          are earned regardless of performance and were commonly used before
                          DOE adopted performance-based contracting as its normal business
                          practice. Appendixes I and II summarize laboratory fee arrangements and
                          illustrate the wide variety of fee arrangements in use. In commenting on a
                          draft of this report, DOE said that by the end of calendar year 1999, the
                          majority of laboratory contracts that provide fees will have
                          performance-based fee structures.

                          Performance fees were introduced as a way of encouraging superior
                          performance and can include an incentive and an award fee. An incentive
                          fee is usually applied to activities for which progress can be accurately
                          measured, for example, cleaning up 40 barrels of toxic waste within a
                          prescribed period of time. An award fee is usually applied to tasks that are
                          harder to measure and require a more subjective judgment of
                          performance, for example, assessing a contractor’s attention to
                          community relations. Performance fees represent the amount of a
                          contractor’s total fee placed “at risk” since the fee that could be earned is
                          determined by how well the contractor performs.




                          Page 4                             GAO/RCED-99-141 Performance-Based Contracting
    B-282356




    As the following examples show, some laboratory contracts include both
    types of performance fees, while others rely solely on an incentive fee or
    an award fee. Still others have neither and use only fixed fees.

•   At the Sandia National Laboratories in New Mexico and the Oak Ridge
    National Laboratory in Tennessee, DOE negotiated fixed-fee contracts.
    Both of these laboratories are operated by subsidiaries of the Lockheed
    Martin Corporation—a for-profit company. DOE officials told us they were
    confident that incentive fees were not needed for these laboratories
    because the existing Lockheed Martin contractors’ performance is
    superior and introducing incentive fees might distract the contractors
    from performing all essential work.
•   At the Idaho National Engineering and Environmental Laboratory in Idaho,
    operated by Lockheed Martin Idaho Technologies Company, DOE uses a
    combination of fixed, incentive, and award fees. DOE officials told us that
    incentive fees were used because of the many different tasks that could be
    identified and measured, but that award fees were also needed to assess
    activities that required more subjective judgments.
•   At the Stanford Linear Accelerator Center in California, operated by
    Stanford University, DOE negotiated a no-fee contract, the only such
    arrangement in the laboratory system. According to DOE, the laboratory
    contractor does not want a fee for operating this laboratory because a fee
    would not motivate performance and may be a detriment to the conduct of
    outstanding science, which is the primary mission of this laboratory.
•   The Lawrence Berkeley National Laboratory and Lawrence Livermore
    National Laboratory in California and the Los Alamos National Laboratory
    in New Mexico are operated by the University of California. The contracts
    contain a fixed fee and an incentive fee for meeting expectations, plus
    another amount for exceeding expectations.

    A senior DOE official acknowledged the variability in laboratory contracts
    but said that imposing uniform practices throughout the laboratory system
    would not necessarily improve the overall performance and accountability
    of the contractors. According to DOE and laboratory officials, there are
    several reasons for the variability in the contracts. First, the laboratories
    engage in different activities with different levels of technical
    complexities. Second, some contractors are willing to assume greater
    financial risk or liability and thus expect a higher or different fee
    arrangement. Finally, DOE field officials who negotiate the contracts
    employ features that they believe are best suited for their particular
    circumstances. However, we found that similar laboratories operated by
    similar contractors have different fee arrangements. For example, both the



    Page 5                            GAO/RCED-99-141 Performance-Based Contracting
                      B-282356




                      Lawrence Berkeley and Argonne national laboratories have similar
                      research missions and are both managed by university contractors.
                      However, Lawrence Berkeley’s contractor, the University of California,
                      works under a fixed-fee plus performance fee arrangement, while
                      Argonne’s contractor, the University of Chicago, works under a
                      performance fee arrangement only.

                      We also found substantial variations in contracting philosophy among DOE
                      field officials. DOE relies on field units to negotiate its contracts, including
                      whether to use performance-based fees, and how performance objectives
                      and measures will be accomplished. Some of these officials told us that
                      performance fees are important motivators, while others said performance
                      fees can distract the contractor from other important work.

                      In commenting on a draft of this report, DOE provided us with additional
                      reasons for the variability in contracts, including the timing of when
                      contractors first converted to performance-based contracting, the nature
                      of the proposals received in competitive awards, and the negotiated terms
                      in contract extensions. In addition, DOE cited other motivations for
                      laboratory contractors, such as their reputations in the scientific
                      community and contract extensions.


                      DOE’s guidance states that the purpose of performance-based contracting is
The Impact of         to obtain better performance or lower costs or both. DOE has not analyzed
Performance-Based     the impact of performance-based contracting on its laboratory
Contracting Remains   contractors. As a result, it has not determined whether performance-based
                      contracting is achieving the intended objectives of reducing costs and
Unknown               improving performance.

                      DOE officials told us that the amounts of fees paid to laboratory
                      contractors have generally increased with the implementation of
                      performance-based contracting but that it is difficult to determine the
                      return on this investment since contractors are also assuming more risk or
                      liability for costs previously paid by DOE. Increased liabilities include costs
                      due to a failure to exercise prudent business judgment on the part of the
                      contractor’s managerial personnel. DOE has not analyzed the relative costs
                      and benefits to the government of using higher fees in performance-based
                      contracts. We previously recommended that DOE ensure that the fees paid
                      to contractors for incurring increased financial risks are cost-effective by
                      developing criteria for measuring the costs and benefits to the government




                      Page 6                              GAO/RCED-99-141 Performance-Based Contracting
    B-282356




    of this approach.2 DOE officials told us that while they have not conducted
    a comprehensive cost-benefit analysis of fees, they try to negotiate fees
    that make sense for individual contracts, taking into account the financial
    risks and incentives needed to motivate performance. Without such an
    overall analysis, however, it is difficult to determine the value to the
    government of the over $100 million spent on contractor fees for fiscal
    year 1998.

    Although DOE has not assessed the impact of performance-based
    contracting, limited reviews have found both progress and problems, as
    these examples show:

•   Since 1997, DOE’s Office of Inspector General has issued three reports on
    problems the Department had in implementing performance-based
    incentives at three facilities (one of which was a laboratory).3 Problems
    reported by the Inspector General included contracts with poorly
    developed performance measures and fees that were paid to contractors
    before agreement was reached on the performance incentives.
•   In 1997, DOE’s Office of Procurement issued a report on the use of
    performance-based incentives. The report noted that the use of incentives
    has been effective in directing contractors’ attention to performance
    outcomes and has improved communications concerning performance
    expectations. The report also noted that DOE field units are improving the
    quality of their contracts. However, the report pointed out that
    implementation was sometimes inconsistent and that performance
    objectives sometimes were overly focused on process milestones rather
    than on outcomes. DOE’s laboratories were not the focus of this review,
    however.4
•   Our July 1998 report on DOE’s performance-based incentive contracts
    noted that the Department had taken steps to correct many of the
    problems cited in the Inspector General’s reports, including issuing
    guidance, conducting training, and incorporating lessons learned into
    fiscal year 1998 contract incentives.5 We noted that although DOE

    2
     GAO/RCED-94-202, Aug. 25, 1994.
    3
     Inspection of the Performance Based Incentive Program at the Richland Operations Office
    (DOE/IG-0401, Mar. 10, 1997); Audit of the Contractor Incentive Programs at the Rocky Flats
    Environmental Technology Site (DOE/IG-0411, Aug. 13, 1997); and Inspection Report: The Fiscal Year
    1996 Performance Based Incentive Program at the Savannah River Operations Office (DOE-INS-O-98-03,
    May 1998).
    4
     Contract Reform Self Assessment Report, Office of Contract Reform and Privatization, DOE (Sept.
    1997).
    5
     Department of Energy: Lessons Learned Incorporated Into Performance-Based Incentive Contracts
    (GAO/RCED-98-223, July 15, 1998).



    Page 7                                       GAO/RCED-99-141 Performance-Based Contracting
              B-282356




              maintained that its performance-based incentives have been effective in
              achieving the desired end results, it had not been clear whether these
              successes were due to performance-based incentives or to an increased
              emphasis on program management.

              None of these assessments focused exclusively on laboratory contracts. In
              our discussions, DOE field staff generally credited performance-based
              contracting with improving their ability to set expectations for the
              Department’s laboratories, and several laboratory contractors concurred
              that this was a benefit. In addition, both DOE and laboratory officials cited
              improved communication as a benefit of performance-based contracting.
              Laboratory contractors also credited DOE for focusing its oversight on
              evaluating results and away from dwelling on strict compliance with DOE’s
              rules and regulations. In addition, contractors told us they have increased
              productivity and lowered costs, especially for the support and overhead
              functions. However, most of these officials also said that these advances
              were more the result of other initiatives, such as internal streamlining
              actions, than of performance-based contracting.


              DOE and its laboratory contractors told us that they are committed to
Conclusions   making performance-based contracting work effectively and that the
              contracts are including more specific and reliable performance measures.
              However, since DOE has not evaluated the impact of performance-based
              contracting on its laboratories—owing in part to the wide variance in fee
              arrangements—there is limited evidence on how performance fees ensure
              a high level of performance by contractors at lower cost. As a result, DOE
              cannot show how the higher fees it is paying to contractors under
              performance-based contracting are of value to the government and to the
              taxpayers.

              We previously recommended that the Secretary of Energy ensure that the
              fees paid to contractors for incurring increased financial risk are
              cost-effective by developing criteria for measuring the costs and benefits
              to the government of this approach. DOE did not implement our
              recommendation and has no plans to measure the overall costs and
              benefits of performance-based contracting for its laboratories. DOE
              officials maintain that performance-based contracting is working, but this
              is based on anecdotal evidence. Moreover, the fees DOE negotiates are
              based on its best judgment of what is needed to motivate contractors and
              to compensate them for increased risk, but DOE’s evidence is based
              primarily on non-laboratory contractors, and DOE has not quantified the



              Page 8                             GAO/RCED-99-141 Performance-Based Contracting
                  B-282356




                  value of the increased risk assumed by contractors under
                  performance-based conditions.


                  Because DOE does not know whether performance-based contracting is
Recommendation    improving performance at lower cost at its national laboratories and
                  because our previous recommendation to develop criteria for measuring
                  the costs and benefits of paying fees to contractors for incurring increased
                  financial risk was not implemented, we recommend that the Secretary of
                  Energy evaluate the costs and benefits from using performance-based
                  contracting at the national laboratories. While we recognize that each
                  laboratory contract is individually negotiated, DOE should nevertheless
                  ensure that the fees it provides to motivate contractors and to compensate
                  them for increased financial risk is based on an analysis of costs and
                  benefits. The need for this type of evaluation is consistent with the
                  principles of the Government Performance and Results Act of 1993 that
                  require agencies to measure outcomes against their goals.6


                  We provided a draft of this report to DOE for review and comment. DOE
Agency Comments   disagreed with our conclusion on the need for determining the costs and
                  benefits of the fees it has negotiated with its laboratory contractors. DOE
                  noted that its performance-based contracting experience is in transition
                  but that its evaluations show that performance-based contracting is
                  working. We acknowledge in our report that DOE’s evaluations of
                  performance-based contracting show promise, but we also point out that
                  these evaluations did not focus on the laboratories’ experiences with
                  performance-based contracting. Because of this limitation and because of
                  the higher fees being negotiated with the laboratories, we continue to
                  believe it is desirable for DOE to determine if its performance-based
                  contracting is improving performance at lower cost.

                  DOE also commented that the variability we found in performance-based
                  laboratory contracts reflects many different factors, including differences
                  in the scope of work, the type of contractor, and the experiences the
                  laboratories have with performance-based contracting features. Our report
                  described the reasons for the variability in laboratory contracts, and we
                  have included the additional reasons provided in DOE’s comments. We also
                  agree that DOE’s use of performance-based contracting is evolving and that
                  the variability we found in laboratory contracts (principally in

                  6
                   The Results Act applies to agencies as defined in 5 U.S.C. 306(f), which generally covers executive
                  departments, government corporations, and independent establishments.



                  Page 9                                         GAO/RCED-99-141 Performance-Based Contracting
B-282356




performance measures and fee arrangements) is in part due to an ongoing
learning process associated with the transition to performance-based
contracting.

DOE also raised a number of issues regarding the use of fees in its
laboratory contracts and strongly defended its use of performance fees.
We agree with many of DOE’s observations on the use of performance fees,
and we are not suggesting that DOE should abandon its performance-based
approach or that it should eliminate performance-based fees in its
laboratory contracts. It is also not our intent to show that
performance-based contracting should be abandoned if its impacts on the
laboratories cannot be measured. We do believe, however, that effective
implementation of performance-based contracting provisions is dependent
on the ability to support the fee amounts paid through a cost and benefit
analysis.

DOE also provided a number of clarifications that we have incorporated in
our report as appropriate. Appendix III includes the full text of DOE’s
comments and our response.


Our review was performed from September 1998 through April 1999 in
accordance with generally accepted government auditing standards. See
appendix IV for a description of our scope and methodology.

As arranged with your offices, unless you publicly announce its contents
earlier, we plan no further distribution of this report until 30 days after the
date of this letter. At that time, we will send copies to Bill Richardson,
Secretary of Energy, and Jacob J. Lew, Director, Office of Management
and Budget. We will make copies available to others on request.

If you or your staff have any questions about this report, please call me at
(202) 512-3841. Major contributors to this report were Gary R. Boss and
Tom Kingham.




Susan D. Kladiva
Associate Director, Energy,
  Resources, and Science Issues

Page 10                            GAO/RCED-99-141 Performance-Based Contracting
Page 11   GAO/RCED-99-141 Performance-Based Contracting
Contents



Letter                                                                                            1


Appendix I                                                                                       14
Contract Amount and
Fees Earned by DOE’s
Laboratory
Contractors in Fiscal
Year 1998
Appendix II                                                                                      16
Status of DOE
Laboratory Contracts
as of March 30, 1999
Appendix III                                                                                     18
Comments From the
Department of Energy
Appendix IV                                                                                      29
Scope and
Methodology




                        Abbreviations

                        DOE       Department of Energy
                        GAO       General Accounting Office
                        R&D       research and development


                        Page 12                        GAO/RCED-99-141 Performance-Based Contracting
Page 13   GAO/RCED-99-141 Performance-Based Contracting
Appendix I

Contract Amount and Fees Earned by DOE’s
Laboratory Contractors in Fiscal Year 1998


                                         Contract amount
Laboratory/contractor                          (millions)a   Fixed or base feeb    Performance feec        Total fee earned
Argonne National Laboratory/University
of Chicago                                         $466.9                                $3,425,000              $3,425,000
Brookhaven National
Laboratory/Brookhaven Science
Associates                                          385.9            3,574,000                                     3,574,000
Idaho National Engineering and
Environmental Laboratory/Lockheed
Martin Idaho Technologies Company                   578.7                                 9,848,000                9,848,000
Lawrence Berkeley National
Laboratory/University of California                 320.0              420,000            1,063,780                1,483,780
Lawrence Livermore National
Laboratory/University of California                1,100.0           1,680,000            4,482,000                6,162,000
Los Alamos National
Laboratory/University of California                1,345.0           2,100,000            5,550,000                7,650,000
Oak Ridge National
Laboratory/Lockheed Martin Energy
Research                                            475.6            7,220,000                                     7,220,000
Pacific Northwest Laboratory/Battelle
Memorial Institute                                  461.0            5,600,000                                     5,600,000
Sandia National Laboratories/Sandia
Corp. (Lockheed Martin)                            1,397.6          14,347,000                                   14,347,000
                                                                                                    d
Ames Laboratory/Iowa State University                25.4                                       TBD                      TBDd
Fermi National Accelerator
Laboratory/University Research
Associates, Inc.                                    279.6            2,750,000                                     2,750,000
National Renewable Energy
Laboratory/Midwest Research Institute               199.4            3,522,500            3,522,500                7,045,000
Princeton Plasma Physics
Laboratory/Princeton University                      59.4               10,000                                        10,000
Stanford Linear Accelerator
Center/Stanford University                          187.0
Thomas Jefferson National Accelerator
Facility/Southeastern University
Research Associates, Inc.                            70.2            1,874,633                                     1,874,633e
Bettis Atomic Power Laboratory/Bechtel
Group                                               305.1            8,686,000                                     8,686,000
Knolls Atomic Power Laboratory/KAPL,
Inc. (Lockheed Martin)                              265.0            7,300,000                                     7,300,000
Savannah River Technology
Centerf/Westinghouse Savannah River
Co.                                                1,248.0                               51,570,100              51,570,100

                                                                                                    (Table notes on next page)




                                         Page 14                             GAO/RCED-99-141 Performance-Based Contracting
Appendix I
Contract Amount and Fees Earned by DOE’s
Laboratory Contractors in Fiscal Year 1998




a
Contract amounts for fiscal year 1998 include some estimates.
b
  A base fee, used in a performance fee contract, is the part of the fee not at risk and is similar to a
fixed fee.
c
A performance fee can be either an award fee or an incentive fee or a combination.
d
 The fee is to be determined after DOE reviews of the contractor’s self-assessment and after the
issuance of final DOE reports.
e
The fee is considered a management allowance, which is similar to a fixed fee.
f
 The contract and fee amounts shown are for the entire Savannah River Site, including the
Savannah River Technology Center.

Source: GAO based on data from DOE’s headquarters and operations offices.




Page 15                                         GAO/RCED-99-141 Performance-Based Contracting
Appendix II

Status of DOE Laboratory Contracts as of
March 30, 1999


Laboratory/contractor                                       Type and status of contract
Argonne National Laboratory/University of Chicago           Incentive-fee contract. DOE plans to extend this contract 5 years, but is
                                                            renegotiating to make it consistent with the federal acquisition
                                                            regulations format and to incorporate all contract reform features,
                                                            including performance-based provisions.
Brookhaven National Laboratory/Brookhaven Science           Fixed-fee contract. The new contract was signed in January 1998 with a
Associates                                                  fixed fee through September 1998. DOE is still negotiating the contract
                                                            for fiscal year 1999. DOE plans to negotiate a performance fee.
Idaho National Engineering and Environmental                Incentive-fee contract. DOE plans to recompete this contract in fiscal
Laboratory/Lockheed Martin Idaho Technologies Company       year 1999. The current contractor, Lockheed Martin, announced it will
                                                            not bid.
Lawrence Berkeley National Laboratory/University of         Incentive-fee contract. The amount of the annual available fee remains
California                                                  the same for each year of the 5-year contract.
Lawrence Livermore National Laboratory/University of        Incentive-fee contract. The amount of the annual available fee remains
California                                                  the same for each year of the 5-year contract.
Los Alamos National Laboratory/University of California     Incentive-fee contract. The amount of the annual available fee remains
                                                            the same for each year of the 5-year contract.
Oak Ridge National Laboratory/Lockheed Martin Energy        Fixed-fee contract. DOE is recompeting this contract. DOE plans to
Research                                                    convert this contract to a performance fee. Lockheed-Martin announced
                                                            that it will not bid as a prime contractor on the new contract. The other
                                                            two contracts for this site—for environmental cleanup and
                                                            production—are performance-fee
Pacific Northwest Laboratory/Battelle Memorial Institute    Incentive-fee contract. DOE converted this contract from a fixed-fee to
                                                            an incentive-fee type and made available $7.1 million in potential fees
                                                            geared to incentives in four areas—science and technology excellence,
                                                            operational excellence, leadership and management, and community
                                                            relations.
Sandia National Laboratories/Sandia Corp. (Lockheed Martin) Fixed-fee contract. The contract that expired in September 1998 was
                                                            renegotiated and extended noncompetitively for 5 years. The new
                                                            contract remains a fixed-fee arrangement but now includes performance
                                                            objectives, measures, and criteria. DOE decided that the contractor’s
                                                            superior performance could be sustained with a fixed fee.
Ames Laboratory/Iowa State University                       Incentive-fee contract. DOE is renegotiating this contract and plans to
                                                            extend noncompetitively for 5 years. DOE plans to make the contract
                                                            consistent with the federal acquisition regulations format and to
                                                            incorporate all contract reform conditions, including performance-based
                                                            provisions.
Fermi National Accelerator Laboratory/University Research   Fixed-fee contract. DOE has not announced whether it will recompete or
Associates, Inc.                                            extend this contract. DOE rates the contractor’s performance as
                                                            outstanding.
National Renewable Energy/Midwest Research Institute        Award-fee contract. DOE recompeted this contract in 1998. The new
                                                            contract was effective on Oct. 1, 1998, and is fixed fee until March 1999,
                                                            at which time DOE intends to includes an award fee for the remainder of
                                                            the contract period.
Princeton Plasma Physics Laboratory/Princeton University    Fixed-fee contract. The contractor did not want any fee, but DOE
                                                            negotiated a small fee of $10,000.
                                                                                                                           (continued)




                                              Page 16                               GAO/RCED-99-141 Performance-Based Contracting
                                             Appendix II
                                             Status of DOE Laboratory Contracts as of
                                             March 30, 1999




Laboratory/contractor                                         Type and status of contract
Stanford Linear Accelerator Center/Stanford University        No-fee contract. The contract term ended on March 31, 1998, and was
                                                              extended noncompetitively on a month-by-month basis during
                                                              negotiations to incorporate performance-based incentives. The contract
                                                              was then extended noncompetitively for 5 years in January 1999. The
                                                              contract includes performance measures and expectations, but no fee.
Thomas Jefferson National Accelerator Facility/Southeastern   No fee contract (with management allowance). The contract is currently
University Research Associates, Inc                           being renegotiated so that it can be extended noncompetitively for 5
                                                              years. DOE plans the new contract to be a fixed-fee arrangement.
                                                              Objectives of the negotiations are to structure the contract to be
                                                              consistent with the federal acquisition regulations format and to
                                                              incorporate all contract reform conditions, including performance-based
                                                              features.
Bettis Atomic Power Laboratory/Bechtel Group                  Fixed-fee contract. The contract was recompeted in 1998. The new
                                                              contractor was selected (the Bechtel Group) but the incumbent
                                                              contractor, Westinghouse Electric Corporation, protested the award. The
                                                              existing contract extended non-competitively pending the result of a bid
                                                              protest to GAO. The bid protest was denied by GAO. The new contract
                                                              with Bechtel was effective February 1, 1999.
Knolls Atomic Power Laboratory/KAPL, Inc. (Lockheed           Fixed-fee contract.
Martin)
Savannah River Technology Center/Westinghouse Savannah Incentive-fee contract.
River Co.

                                             Source: GAO based on data from DOE’s operations offices.




                                             Page 17                                    GAO/RCED-99-141 Performance-Based Contracting
Appendix III

Comments From the Department of Energy


Note: GAO comments
supplementing those in the
report text appear at the
end of this appendix.




                             Page 18   GAO/RCED-99-141 Performance-Based Contracting
                 Appendix III
                 Comments From the Department of Energy




See comment 1.




See comment 2.




See comment 1.




Now on p. 1.
See comment 1.




See comment 3.



See comment 4.




                 Page 19                             GAO/RCED-99-141 Performance-Based Contracting
                 Appendix III
                 Comments From the Department of Energy




See comment 5.




See comment 6.




Now on p. 6.
See comment 7.




                 Page 20                             GAO/RCED-99-141 Performance-Based Contracting
                 Appendix III
                 Comments From the Department of Energy




See comment 8.




                 Page 21                             GAO/RCED-99-141 Performance-Based Contracting
                 Appendix III
                 Comments From the Department of Energy




See comment 9.




                 Page 22                             GAO/RCED-99-141 Performance-Based Contracting
                      Appendix III
                      Comments From the Department of Energy




Now on p. 4.
See comment 10.




Now on p. 4.
See comment 11.



Now on pp. 5 and 6.
See comment 12.




                      Page 23                             GAO/RCED-99-141 Performance-Based Contracting
                  Appendix III
                  Comments From the Department of Energy




Now on p. 6.
See comment 13.




Now on p. 8.
See comment 14.




                  Page 24                             GAO/RCED-99-141 Performance-Based Contracting
                 Appendix III
                 Comments From the Department of Energy




See comment 1.




                 Page 25                             GAO/RCED-99-141 Performance-Based Contracting
               Appendix III
               Comments From the Department of Energy




               The following are GAO’s comments on the Department of Energy’s letter
GAO Comments   dated April 22, 1999.

               1. We have made changes to the report as appropriate in response to DOE’s
               comments.

               2. Our wording is drawn from DOE’s guidance on performance-based
               contracting, and we have made changes to our report to reflect DOE’s
               comments. DOE recommends that its laboratory contracts contain
               performance-based features, which include clear expectations described
               in terms of results, not how the work is to be accomplished.

               3. As we stated in our report, DOE’s evaluations did not focus on the
               laboratory contractors, nor did these evaluations focus on the costs and
               benefits of performance-based contracting features, including the impact
               of fees.

               4. We recognize that one of the purposes of providing fees is to reflect the
               financial risk associated with work performance, and we make this point
               in our report. Our 1994 recommendation questioned the cost-benefit of the
               increased fees, regardless of whether they were related to performance or
               financial risk. We continue to believe that our recommendation is relevant
               because DOE has not evaluated the cost and benefit of the fees it is
               providing to laboratory contractors.

               5. We believe our wording adequately reflects the conditions discussed.
               Information on the laboratory fees and total contract costs is presented in
               appendix I.

               6. We have made changes to the report as appropriate in response to DOE’s
               comments on contract type. We stated in our report that DOE’s
               performance-based contracting is in a state of transition. We also stated
               that there are wide variations in performance measures and fee
               arrangements negotiated by DOE and its laboratory contractors. This
               material is presented as facts describing the conditions that presently
               exist. Our report also describes the reasons for the variability in laboratory
               contracts and includes most of the reasons given in DOE’s comments. We
               have made changes in the report to reflect these additional reasons for the
               variability in DOE’s laboratory contracts.

               7. Our statement that contract differences are the product of DOE’s relying
               on its field units to tailor contracts to local conditions is based on



               Page 26                             GAO/RCED-99-141 Performance-Based Contracting
Appendix III
Comments From the Department of Energy




interviews with numerous DOE field officials. This statement is not an
implied criticism of how DOE negotiates contracts. Also, we disagree with
DOE’s characterization that contractors’ preferences are “generally
irrelevant” when accounting for the variations that exist among laboratory
contractors. As DOE noted, contractors’ preferences are reflected in the
negotiation process. In our discussions with DOE field officials responsible
for negotiating contracts, laboratory contractors’ preferences on fees were
cited as a critical factor in determining fee structures.

8. Our report recognizes that developing the optimum number of
performance measures is a challenge, as reflected in the wide range of
performance measures in use even among similar laboratories. We are not
suggesting that any two contractors should have the same measures or the
same number of measures. Our point is that DOE continues to struggle with
finding the right number of measures. To further illustrate, the University
of California’s fiscal year 1998 contracts for its two weapons
laboratories—Lawrence Livermore and Los Alamos—contain 83 and 120
performance measures, respectively, even though these laboratories are
very similar in budget and scope. They are, however, managed by different
DOE field units.


9. Our purpose in including comments we received from DOE field units is
to illustrate the wide differences in philosophy about the use of fees to
motivate laboratory contractors. Several DOE field staff, as well as
contractors, told us that they strongly believe that providing fees does not
motivate contractors, including both for-profit and not-for-profit
contractors. Moreover, our statement that performance-based contracting
has tended to focus in some instances on monetary rewards at the expense
of good science was a frequent comment from both DOE field officials and
laboratory contractors. Thus, it is very important to identify the need for
monetary incentives where they are appropriate. Other motivations that
DOE cited for laboratory contractors, such as their reputations in the
scientific community and desire for contract extensions, were added to
our report. These differences in philosophy account for some of the
variation in contracts.

10. Our report reflects information provided directly from DOE field staff,
who we were advised by DOE headquarters were the proper source for this
information. The data in DOE’s comments are reflected in the appendixes
to our report. We have also revised our report to show that there are now
18 laboratory contractors, reflecting a recent change in how DOE defines its
laboratories.



Page 27                             GAO/RCED-99-141 Performance-Based Contracting
Appendix III
Comments From the Department of Energy




11. DOE field officials told us that performance fees are used to encourage
superior performance. Asserting that fees are used to link performance to
financial reward is self-evident in this context.

12. We agree with DOE that no single approach in contracting has proven to
be optimum, and we reflected this view in our report. Regarding the wide
variability in fee arrangements, we stated that there was very little
consistency among the contracts of similar laboratory contractors
conducting similar work. We also stated that local conditions influence the
variability in laboratory contracts.

13. Our wording was taken from DOE’s guidance on performance-based
contracting. As we state in our report, prior assessments of
performance-based contracting have not focused on laboratory
contractors. We also stated in our report that DOE believes that the results
from its assessments of performance-based contracting have been
positive. We believe it is a logical and desirable step for DOE to determine
whether performance-based contracting is improving performance at
lower cost in its national laboratories. Also, we are not suggesting that DOE
should abandon its performance-based approach or that it should
eliminate performance-based fees in its laboratory contracts. It is also not
our intent to show that performance-based contracting should be
abandoned if its impacts on the laboratories cannot be measured. We
believe that effective implementation of performance-based contracting
provisions is dependent on the ability to support the fee amounts paid
through a cost and benefit analysis. While it may appear intuitively
obvious that defining performance expectations and measuring results are
effective management tools, it is not intuitively obvious that the
government is receiving a reasonable return on its investments in fee
amounts for laboratory contractors. Likewise, while DOE commented that
increases in fees reflect, in part, the increased financial risks being borne
by contractors, no cost-benefit analysis quantifying this increased financial
risk has been completed; thus it is not possible to determine if the proper
level of fee is appropriate for the risk assumed.

14. We recognize that laboratory contractor fees are relatively small
percentages of the total contract amounts. However, these percentages,
which translated into $100 million in fees for fiscal year 1998, must be
considered in light of the fact that DOE’s laboratories are government
owned and that a laboratory contractor’s financial risk is limited.




Page 28                             GAO/RCED-99-141 Performance-Based Contracting
Appendix IV

Scope and Methodology


              To obtain information on the national laboratories’ contracts, we
              interviewed officials from the following laboratories: Sandia National
              Laboratories and Los Alamos National Laboratory in New Mexico;
              Lawrence Berkeley National Laboratory, Lawrence Livermore National
              Laboratory, and the Stanford Linear Accelerator in California; the National
              Renewable Energy Laboratory in Colorado; the Idaho National
              Engineering and Environmental Laboratory in Idaho; the Oak Ridge
              National Laboratory in Tennessee; and the Argonne National Laboratory in
              Illinois. We also spoke with laboratory officials in other locations to obtain
              cost and status information. We asked officials at these laboratories to
              comment on the impact of performance-based contracting on their
              operations.

              We also interviewed Department of Energy (DOE) officials responsible for
              overseeing these laboratories. These officials were from DOE’s operations
              offices in Albuquerque, New Mexico; Oakland, California; Oak Ridge,
              Tennessee; and Chicago, Illinois. We also interviewed DOE area and site
              office staff located at each of the operations offices we visited. To obtain a
              broader perspective, we interviewed DOE headquarters officials
              responsible for developing contracting policy.

              We conducted our review from September 1998 through April 1999 in
              accordance with generally accepted government auditing standards.




(141265)      Page 29                            GAO/RCED-99-141 Performance-Based Contracting
Ordering Information

The first copy of each GAO report and testimony is free.
Additional copies are $2 each. Orders should be sent to the
following address, accompanied by a check or money order
made out to the Superintendent of Documents, when
necessary. VISA and MasterCard credit cards are accepted, also.
Orders for 100 or more copies to be mailed to a single address
are discounted 25 percent.

Orders by mail:

U.S. General Accounting Office
P.O. Box 37050
Washington, DC 20013

or visit:

Room 1100
700 4th St. NW (corner of 4th and G Sts. NW)
U.S. General Accounting Office
Washington, DC

Orders may also be placed by calling (202) 512-6000
or by using fax number (202) 512-6061, or TDD (202) 512-2537.

Each day, GAO issues a list of newly available reports and
testimony. To receive facsimile copies of the daily list or any
list from the past 30 days, please call (202) 512-6000 using a
touchtone phone. A recorded menu will provide information on
how to obtain these lists.

For information on how to access GAO reports on the INTERNET,
send an e-mail message with "info" in the body to:

info@www.gao.gov

or visit GAO’s World Wide Web Home Page at:

http://www.gao.gov




PRINTED ON    RECYCLED PAPER
United States                       Bulk Rate
General Accounting Office      Postage & Fees Paid
Washington, D.C. 20548-0001           GAO
                                 Permit No. G100
Official Business
Penalty for Private Use $300

Address Correction Requested