United States General Accounting Office GAO Report to the Committee on Science, House of Representatives May 1999 NATIONAL LABORATORIES DOE Needs to Assess the Impact of Using Performance-Based Contracts GAO/RCED-99-141 United States GAO General Accounting Office Washington, D.C. 20548 Resources, Community, and Economic Development Division B-282356 May 7, 1999 The Honorable F. James Sensenbrenner, Jr. Chairman The Honorable George E. Brown, Jr. Ranking Minority Member Committee on Science House of Representatives The Department of Energy (DOE) contracts with private companies and educational institutions to manage and operate 18 of its 22 laboratories. These are cost reimbursement contracts under which DOE pays all of its contractors’ allowable costs. DOE can also provide a fee, or profit, to a contractor for managing a laboratory. Responding to criticism that its historical contracting practices were costly and inefficient, DOE switched to performance-based contracts in 1994 as part of its contract reform program. Use of these contracts allows DOE to structure each contract to provide a clear statement of what needs to be accomplished—rather than providing broad statements of work—and to rely on performance measures to evaluate a contractor’s progress toward meeting its objectives. An important feature of performance-based contracting is providing incentives, including fees, to the contractor’s achievement of objectives as a means of encouraging superior performance and lowering costs. Concerned about the progress made to implement performance-based contracting at the national laboratories, you asked us to • assess the status of performance-based contracting in DOE’s national laboratory contracts, and • identify efforts being made to determine the impact of performance-based contracting. DOE’s use of performance-based contracting for its laboratories is in a state Results in Brief of transition. While all laboratory contracts we examined had some performance-based features, we found wide variance in the number of performance measures and the types of fees negotiated. About half of the 18 laboratory contracts have performance fees to encourage superior performance—a major goal of performance-based contracting. Most of the remaining laboratory contracts are still based on DOE’s traditional fixed-fee arrangement in which the fees are paid regardless of performance. Page 1 GAO/RCED-99-141 Performance-Based Contracting B-282356 DOE has not evaluated the impact of performance-based contracting on its laboratory contractors and, as a result, does not know if this new form of contracting is achieving the intended results of improved performance and lower costs. Specifically, DOE has not determined whether giving higher fees to encourage superior performance by its laboratory contractors is advantageous to the government, although we recommended in 1994 that DOE develop criteria for measuring the costs and benefits to the government of using higher fees.1 Fees for the laboratories totaled over $100 million for fiscal year 1998. While the contractors were unable to cite measurable benefits achieved by switching to performance-based contracting, they support its goals. The main benefits from performance-based contracting cited by laboratory contractors was that it has helped DOE clarify what it expects from the contractors and that it has improved communication. DOE manages the largest laboratory system of its kind in the world. Since Background the early days of the World War II Manhattan Project, DOE’s laboratories have played a major role in maintaining U.S. leadership in research and development. DOE is responsible for ensuring that the laboratory system—with 22 laboratories in 14 states, a combined budget of over $10 billion a year, and a staff of about 60,000—is managed in an effective, efficient, and economical manner. DOE contracts with educational institutions and private sector organizations for the management and operation of 18 of its laboratories. (App. I lists DOE’s national laboratories.) The remaining four laboratories are staffed by federal employees. DOE pays its laboratory contractors all allowable costs. DOE can also pay contractors a separate fee, or profit, as compensation for operating the laboratories. Fees are based on the contract value and the technical complexity of the work to be performed at a laboratory, but also on the degree of financial liability or risk that a contractor is willing to assume. Under performance-based contracting principles, fees can include both a fixed amount and an amount that is linked to achieving performance objectives. One of DOE’s major goals in performance-based contracting is to develop performance objectives for each contractor that are specific, results-oriented, measurable, and reflect the most critical activities. 1 Energy Management: Modest Reforms Made in University of California Contracts, but Fees Are Substantially Higher (GAO/RCED-94-202, Aug. 25, 1994). Page 2 GAO/RCED-99-141 Performance-Based Contracting B-282356 DOE’s implementation of performance-based contracting for its Performance-Based laboratories is in a state of transition. While most of its laboratory Contracting at the contracts contain some performance-based features, the contracts National Laboratories negotiated by DOE vary from contract to contract. For example, DOE is incorporating performance-based features in all of its laboratory contracts, Is an Evolving Process although measures vary substantially in number, ranging from a low of 7 in one laboratory contract to about 250 in another. Also, DOE has negotiated performance fees in only 9 of its 18 laboratory contracts because the remaining laboratories are still operating under DOE’s traditional approach in which fees are not linked to performance. We found that similar laboratories managed by similar contractors have different contracts. The wide diversity of contract features reflects DOE’s philosophy of relying on DOE field units to tailor contracts to local conditions and contractors’ preferences. Developing the Right Since introducing performance-based contracting in 1994, DOE and its Performance Measures Is a laboratory contractors have struggled to find the right mix of measures Challenge that accurately and reliably capture the contractors’ performance. According to DOE field staff, in the early years of contract reform, DOE encouraged its field units to construct as many measures as they could, but provided limited guidance on how to accomplish this task. As a result, early attempts led to large numbers of performance measures. A large number of measures diminishes the importance of any single measure, whereas a small number results in measures that are too broad to be meaningful. For example, a DOE field official told us, “The original guidance from DOE Headquarters was to [develop performance measures] as much as possible. Unfortunately, there was inadequate guidance on how to do this. . . . The number of performance measures . . . is too large. However, if we fail to cover an activity [with a measure] the contractor may not give the attention needed to the activity.” DOE and its laboratories are still attempting to develop the right number of measures. For example, we found that the number of performance measures in the laboratory contracts we examined ranged from a low of 7 measures at the Ames Laboratory in Iowa to about 250 at the Idaho National Engineering and Environmental Laboratory in Idaho. DOE and its contractors are also working to develop measures that reliably address the most important activities of the laboratories. According to a Page 3 GAO/RCED-99-141 Performance-Based Contracting B-282356 field official, DOE’s early attempts at developing performance measures resulted in contractors focusing only on those activities that were tied to performance fees, while neglecting other important activities. Another DOE site official stated, “[P]erformance-based contracting tends to focus too much on the monetary reward . . . and less on an analysis of performance. The incentive at the labs should be [for] good science, not more dollars.” Developing the right number and type of performance measures is an evolving process between DOE and its contractors. Most DOE and contractor representatives told us that they are making progress in finding measures that accurately and reliably reflect performance, particularly in management and operations activities. Measuring a contractor’s performance in science and technology is more difficult. Science and technology measures are broader in scope and typically rely on peer reviews and a contractor’s self-assessment for evaluating performance. Types of Fees Paid to Although performance fees are a major feature of performance-based Contractors Vary Widely contracting, only 9 of the Department’s 18 laboratory contracts have them. Nine of the remaining laboratory contracts operate under DOE’s traditional fixed-fee arrangement, and one laboratory contract has no fee. Fixed fees are earned regardless of performance and were commonly used before DOE adopted performance-based contracting as its normal business practice. Appendixes I and II summarize laboratory fee arrangements and illustrate the wide variety of fee arrangements in use. In commenting on a draft of this report, DOE said that by the end of calendar year 1999, the majority of laboratory contracts that provide fees will have performance-based fee structures. Performance fees were introduced as a way of encouraging superior performance and can include an incentive and an award fee. An incentive fee is usually applied to activities for which progress can be accurately measured, for example, cleaning up 40 barrels of toxic waste within a prescribed period of time. An award fee is usually applied to tasks that are harder to measure and require a more subjective judgment of performance, for example, assessing a contractor’s attention to community relations. Performance fees represent the amount of a contractor’s total fee placed “at risk” since the fee that could be earned is determined by how well the contractor performs. Page 4 GAO/RCED-99-141 Performance-Based Contracting B-282356 As the following examples show, some laboratory contracts include both types of performance fees, while others rely solely on an incentive fee or an award fee. Still others have neither and use only fixed fees. • At the Sandia National Laboratories in New Mexico and the Oak Ridge National Laboratory in Tennessee, DOE negotiated fixed-fee contracts. Both of these laboratories are operated by subsidiaries of the Lockheed Martin Corporation—a for-profit company. DOE officials told us they were confident that incentive fees were not needed for these laboratories because the existing Lockheed Martin contractors’ performance is superior and introducing incentive fees might distract the contractors from performing all essential work. • At the Idaho National Engineering and Environmental Laboratory in Idaho, operated by Lockheed Martin Idaho Technologies Company, DOE uses a combination of fixed, incentive, and award fees. DOE officials told us that incentive fees were used because of the many different tasks that could be identified and measured, but that award fees were also needed to assess activities that required more subjective judgments. • At the Stanford Linear Accelerator Center in California, operated by Stanford University, DOE negotiated a no-fee contract, the only such arrangement in the laboratory system. According to DOE, the laboratory contractor does not want a fee for operating this laboratory because a fee would not motivate performance and may be a detriment to the conduct of outstanding science, which is the primary mission of this laboratory. • The Lawrence Berkeley National Laboratory and Lawrence Livermore National Laboratory in California and the Los Alamos National Laboratory in New Mexico are operated by the University of California. The contracts contain a fixed fee and an incentive fee for meeting expectations, plus another amount for exceeding expectations. A senior DOE official acknowledged the variability in laboratory contracts but said that imposing uniform practices throughout the laboratory system would not necessarily improve the overall performance and accountability of the contractors. According to DOE and laboratory officials, there are several reasons for the variability in the contracts. First, the laboratories engage in different activities with different levels of technical complexities. Second, some contractors are willing to assume greater financial risk or liability and thus expect a higher or different fee arrangement. Finally, DOE field officials who negotiate the contracts employ features that they believe are best suited for their particular circumstances. However, we found that similar laboratories operated by similar contractors have different fee arrangements. For example, both the Page 5 GAO/RCED-99-141 Performance-Based Contracting B-282356 Lawrence Berkeley and Argonne national laboratories have similar research missions and are both managed by university contractors. However, Lawrence Berkeley’s contractor, the University of California, works under a fixed-fee plus performance fee arrangement, while Argonne’s contractor, the University of Chicago, works under a performance fee arrangement only. We also found substantial variations in contracting philosophy among DOE field officials. DOE relies on field units to negotiate its contracts, including whether to use performance-based fees, and how performance objectives and measures will be accomplished. Some of these officials told us that performance fees are important motivators, while others said performance fees can distract the contractor from other important work. In commenting on a draft of this report, DOE provided us with additional reasons for the variability in contracts, including the timing of when contractors first converted to performance-based contracting, the nature of the proposals received in competitive awards, and the negotiated terms in contract extensions. In addition, DOE cited other motivations for laboratory contractors, such as their reputations in the scientific community and contract extensions. DOE’s guidance states that the purpose of performance-based contracting is The Impact of to obtain better performance or lower costs or both. DOE has not analyzed Performance-Based the impact of performance-based contracting on its laboratory Contracting Remains contractors. As a result, it has not determined whether performance-based contracting is achieving the intended objectives of reducing costs and Unknown improving performance. DOE officials told us that the amounts of fees paid to laboratory contractors have generally increased with the implementation of performance-based contracting but that it is difficult to determine the return on this investment since contractors are also assuming more risk or liability for costs previously paid by DOE. Increased liabilities include costs due to a failure to exercise prudent business judgment on the part of the contractor’s managerial personnel. DOE has not analyzed the relative costs and benefits to the government of using higher fees in performance-based contracts. We previously recommended that DOE ensure that the fees paid to contractors for incurring increased financial risks are cost-effective by developing criteria for measuring the costs and benefits to the government Page 6 GAO/RCED-99-141 Performance-Based Contracting B-282356 of this approach.2 DOE officials told us that while they have not conducted a comprehensive cost-benefit analysis of fees, they try to negotiate fees that make sense for individual contracts, taking into account the financial risks and incentives needed to motivate performance. Without such an overall analysis, however, it is difficult to determine the value to the government of the over $100 million spent on contractor fees for fiscal year 1998. Although DOE has not assessed the impact of performance-based contracting, limited reviews have found both progress and problems, as these examples show: • Since 1997, DOE’s Office of Inspector General has issued three reports on problems the Department had in implementing performance-based incentives at three facilities (one of which was a laboratory).3 Problems reported by the Inspector General included contracts with poorly developed performance measures and fees that were paid to contractors before agreement was reached on the performance incentives. • In 1997, DOE’s Office of Procurement issued a report on the use of performance-based incentives. The report noted that the use of incentives has been effective in directing contractors’ attention to performance outcomes and has improved communications concerning performance expectations. The report also noted that DOE field units are improving the quality of their contracts. However, the report pointed out that implementation was sometimes inconsistent and that performance objectives sometimes were overly focused on process milestones rather than on outcomes. DOE’s laboratories were not the focus of this review, however.4 • Our July 1998 report on DOE’s performance-based incentive contracts noted that the Department had taken steps to correct many of the problems cited in the Inspector General’s reports, including issuing guidance, conducting training, and incorporating lessons learned into fiscal year 1998 contract incentives.5 We noted that although DOE 2 GAO/RCED-94-202, Aug. 25, 1994. 3 Inspection of the Performance Based Incentive Program at the Richland Operations Office (DOE/IG-0401, Mar. 10, 1997); Audit of the Contractor Incentive Programs at the Rocky Flats Environmental Technology Site (DOE/IG-0411, Aug. 13, 1997); and Inspection Report: The Fiscal Year 1996 Performance Based Incentive Program at the Savannah River Operations Office (DOE-INS-O-98-03, May 1998). 4 Contract Reform Self Assessment Report, Office of Contract Reform and Privatization, DOE (Sept. 1997). 5 Department of Energy: Lessons Learned Incorporated Into Performance-Based Incentive Contracts (GAO/RCED-98-223, July 15, 1998). Page 7 GAO/RCED-99-141 Performance-Based Contracting B-282356 maintained that its performance-based incentives have been effective in achieving the desired end results, it had not been clear whether these successes were due to performance-based incentives or to an increased emphasis on program management. None of these assessments focused exclusively on laboratory contracts. In our discussions, DOE field staff generally credited performance-based contracting with improving their ability to set expectations for the Department’s laboratories, and several laboratory contractors concurred that this was a benefit. In addition, both DOE and laboratory officials cited improved communication as a benefit of performance-based contracting. Laboratory contractors also credited DOE for focusing its oversight on evaluating results and away from dwelling on strict compliance with DOE’s rules and regulations. In addition, contractors told us they have increased productivity and lowered costs, especially for the support and overhead functions. However, most of these officials also said that these advances were more the result of other initiatives, such as internal streamlining actions, than of performance-based contracting. DOE and its laboratory contractors told us that they are committed to Conclusions making performance-based contracting work effectively and that the contracts are including more specific and reliable performance measures. However, since DOE has not evaluated the impact of performance-based contracting on its laboratories—owing in part to the wide variance in fee arrangements—there is limited evidence on how performance fees ensure a high level of performance by contractors at lower cost. As a result, DOE cannot show how the higher fees it is paying to contractors under performance-based contracting are of value to the government and to the taxpayers. We previously recommended that the Secretary of Energy ensure that the fees paid to contractors for incurring increased financial risk are cost-effective by developing criteria for measuring the costs and benefits to the government of this approach. DOE did not implement our recommendation and has no plans to measure the overall costs and benefits of performance-based contracting for its laboratories. DOE officials maintain that performance-based contracting is working, but this is based on anecdotal evidence. Moreover, the fees DOE negotiates are based on its best judgment of what is needed to motivate contractors and to compensate them for increased risk, but DOE’s evidence is based primarily on non-laboratory contractors, and DOE has not quantified the Page 8 GAO/RCED-99-141 Performance-Based Contracting B-282356 value of the increased risk assumed by contractors under performance-based conditions. Because DOE does not know whether performance-based contracting is Recommendation improving performance at lower cost at its national laboratories and because our previous recommendation to develop criteria for measuring the costs and benefits of paying fees to contractors for incurring increased financial risk was not implemented, we recommend that the Secretary of Energy evaluate the costs and benefits from using performance-based contracting at the national laboratories. While we recognize that each laboratory contract is individually negotiated, DOE should nevertheless ensure that the fees it provides to motivate contractors and to compensate them for increased financial risk is based on an analysis of costs and benefits. The need for this type of evaluation is consistent with the principles of the Government Performance and Results Act of 1993 that require agencies to measure outcomes against their goals.6 We provided a draft of this report to DOE for review and comment. DOE Agency Comments disagreed with our conclusion on the need for determining the costs and benefits of the fees it has negotiated with its laboratory contractors. DOE noted that its performance-based contracting experience is in transition but that its evaluations show that performance-based contracting is working. We acknowledge in our report that DOE’s evaluations of performance-based contracting show promise, but we also point out that these evaluations did not focus on the laboratories’ experiences with performance-based contracting. Because of this limitation and because of the higher fees being negotiated with the laboratories, we continue to believe it is desirable for DOE to determine if its performance-based contracting is improving performance at lower cost. DOE also commented that the variability we found in performance-based laboratory contracts reflects many different factors, including differences in the scope of work, the type of contractor, and the experiences the laboratories have with performance-based contracting features. Our report described the reasons for the variability in laboratory contracts, and we have included the additional reasons provided in DOE’s comments. We also agree that DOE’s use of performance-based contracting is evolving and that the variability we found in laboratory contracts (principally in 6 The Results Act applies to agencies as defined in 5 U.S.C. 306(f), which generally covers executive departments, government corporations, and independent establishments. Page 9 GAO/RCED-99-141 Performance-Based Contracting B-282356 performance measures and fee arrangements) is in part due to an ongoing learning process associated with the transition to performance-based contracting. DOE also raised a number of issues regarding the use of fees in its laboratory contracts and strongly defended its use of performance fees. We agree with many of DOE’s observations on the use of performance fees, and we are not suggesting that DOE should abandon its performance-based approach or that it should eliminate performance-based fees in its laboratory contracts. It is also not our intent to show that performance-based contracting should be abandoned if its impacts on the laboratories cannot be measured. We do believe, however, that effective implementation of performance-based contracting provisions is dependent on the ability to support the fee amounts paid through a cost and benefit analysis. DOE also provided a number of clarifications that we have incorporated in our report as appropriate. Appendix III includes the full text of DOE’s comments and our response. Our review was performed from September 1998 through April 1999 in accordance with generally accepted government auditing standards. See appendix IV for a description of our scope and methodology. As arranged with your offices, unless you publicly announce its contents earlier, we plan no further distribution of this report until 30 days after the date of this letter. At that time, we will send copies to Bill Richardson, Secretary of Energy, and Jacob J. Lew, Director, Office of Management and Budget. We will make copies available to others on request. If you or your staff have any questions about this report, please call me at (202) 512-3841. Major contributors to this report were Gary R. Boss and Tom Kingham. Susan D. Kladiva Associate Director, Energy, Resources, and Science Issues Page 10 GAO/RCED-99-141 Performance-Based Contracting Page 11 GAO/RCED-99-141 Performance-Based Contracting Contents Letter 1 Appendix I 14 Contract Amount and Fees Earned by DOE’s Laboratory Contractors in Fiscal Year 1998 Appendix II 16 Status of DOE Laboratory Contracts as of March 30, 1999 Appendix III 18 Comments From the Department of Energy Appendix IV 29 Scope and Methodology Abbreviations DOE Department of Energy GAO General Accounting Office R&D research and development Page 12 GAO/RCED-99-141 Performance-Based Contracting Page 13 GAO/RCED-99-141 Performance-Based Contracting Appendix I Contract Amount and Fees Earned by DOE’s Laboratory Contractors in Fiscal Year 1998 Contract amount Laboratory/contractor (millions)a Fixed or base feeb Performance feec Total fee earned Argonne National Laboratory/University of Chicago $466.9 $3,425,000 $3,425,000 Brookhaven National Laboratory/Brookhaven Science Associates 385.9 3,574,000 3,574,000 Idaho National Engineering and Environmental Laboratory/Lockheed Martin Idaho Technologies Company 578.7 9,848,000 9,848,000 Lawrence Berkeley National Laboratory/University of California 320.0 420,000 1,063,780 1,483,780 Lawrence Livermore National Laboratory/University of California 1,100.0 1,680,000 4,482,000 6,162,000 Los Alamos National Laboratory/University of California 1,345.0 2,100,000 5,550,000 7,650,000 Oak Ridge National Laboratory/Lockheed Martin Energy Research 475.6 7,220,000 7,220,000 Pacific Northwest Laboratory/Battelle Memorial Institute 461.0 5,600,000 5,600,000 Sandia National Laboratories/Sandia Corp. (Lockheed Martin) 1,397.6 14,347,000 14,347,000 d Ames Laboratory/Iowa State University 25.4 TBD TBDd Fermi National Accelerator Laboratory/University Research Associates, Inc. 279.6 2,750,000 2,750,000 National Renewable Energy Laboratory/Midwest Research Institute 199.4 3,522,500 3,522,500 7,045,000 Princeton Plasma Physics Laboratory/Princeton University 59.4 10,000 10,000 Stanford Linear Accelerator Center/Stanford University 187.0 Thomas Jefferson National Accelerator Facility/Southeastern University Research Associates, Inc. 70.2 1,874,633 1,874,633e Bettis Atomic Power Laboratory/Bechtel Group 305.1 8,686,000 8,686,000 Knolls Atomic Power Laboratory/KAPL, Inc. (Lockheed Martin) 265.0 7,300,000 7,300,000 Savannah River Technology Centerf/Westinghouse Savannah River Co. 1,248.0 51,570,100 51,570,100 (Table notes on next page) Page 14 GAO/RCED-99-141 Performance-Based Contracting Appendix I Contract Amount and Fees Earned by DOE’s Laboratory Contractors in Fiscal Year 1998 a Contract amounts for fiscal year 1998 include some estimates. b A base fee, used in a performance fee contract, is the part of the fee not at risk and is similar to a fixed fee. c A performance fee can be either an award fee or an incentive fee or a combination. d The fee is to be determined after DOE reviews of the contractor’s self-assessment and after the issuance of final DOE reports. e The fee is considered a management allowance, which is similar to a fixed fee. f The contract and fee amounts shown are for the entire Savannah River Site, including the Savannah River Technology Center. Source: GAO based on data from DOE’s headquarters and operations offices. Page 15 GAO/RCED-99-141 Performance-Based Contracting Appendix II Status of DOE Laboratory Contracts as of March 30, 1999 Laboratory/contractor Type and status of contract Argonne National Laboratory/University of Chicago Incentive-fee contract. DOE plans to extend this contract 5 years, but is renegotiating to make it consistent with the federal acquisition regulations format and to incorporate all contract reform features, including performance-based provisions. Brookhaven National Laboratory/Brookhaven Science Fixed-fee contract. The new contract was signed in January 1998 with a Associates fixed fee through September 1998. DOE is still negotiating the contract for fiscal year 1999. DOE plans to negotiate a performance fee. Idaho National Engineering and Environmental Incentive-fee contract. DOE plans to recompete this contract in fiscal Laboratory/Lockheed Martin Idaho Technologies Company year 1999. The current contractor, Lockheed Martin, announced it will not bid. Lawrence Berkeley National Laboratory/University of Incentive-fee contract. The amount of the annual available fee remains California the same for each year of the 5-year contract. Lawrence Livermore National Laboratory/University of Incentive-fee contract. The amount of the annual available fee remains California the same for each year of the 5-year contract. Los Alamos National Laboratory/University of California Incentive-fee contract. The amount of the annual available fee remains the same for each year of the 5-year contract. Oak Ridge National Laboratory/Lockheed Martin Energy Fixed-fee contract. DOE is recompeting this contract. DOE plans to Research convert this contract to a performance fee. Lockheed-Martin announced that it will not bid as a prime contractor on the new contract. The other two contracts for this site—for environmental cleanup and production—are performance-fee Pacific Northwest Laboratory/Battelle Memorial Institute Incentive-fee contract. DOE converted this contract from a fixed-fee to an incentive-fee type and made available $7.1 million in potential fees geared to incentives in four areas—science and technology excellence, operational excellence, leadership and management, and community relations. Sandia National Laboratories/Sandia Corp. (Lockheed Martin) Fixed-fee contract. The contract that expired in September 1998 was renegotiated and extended noncompetitively for 5 years. The new contract remains a fixed-fee arrangement but now includes performance objectives, measures, and criteria. DOE decided that the contractor’s superior performance could be sustained with a fixed fee. Ames Laboratory/Iowa State University Incentive-fee contract. DOE is renegotiating this contract and plans to extend noncompetitively for 5 years. DOE plans to make the contract consistent with the federal acquisition regulations format and to incorporate all contract reform conditions, including performance-based provisions. Fermi National Accelerator Laboratory/University Research Fixed-fee contract. DOE has not announced whether it will recompete or Associates, Inc. extend this contract. DOE rates the contractor’s performance as outstanding. National Renewable Energy/Midwest Research Institute Award-fee contract. DOE recompeted this contract in 1998. The new contract was effective on Oct. 1, 1998, and is fixed fee until March 1999, at which time DOE intends to includes an award fee for the remainder of the contract period. Princeton Plasma Physics Laboratory/Princeton University Fixed-fee contract. The contractor did not want any fee, but DOE negotiated a small fee of $10,000. (continued) Page 16 GAO/RCED-99-141 Performance-Based Contracting Appendix II Status of DOE Laboratory Contracts as of March 30, 1999 Laboratory/contractor Type and status of contract Stanford Linear Accelerator Center/Stanford University No-fee contract. The contract term ended on March 31, 1998, and was extended noncompetitively on a month-by-month basis during negotiations to incorporate performance-based incentives. The contract was then extended noncompetitively for 5 years in January 1999. The contract includes performance measures and expectations, but no fee. Thomas Jefferson National Accelerator Facility/Southeastern No fee contract (with management allowance). The contract is currently University Research Associates, Inc being renegotiated so that it can be extended noncompetitively for 5 years. DOE plans the new contract to be a fixed-fee arrangement. Objectives of the negotiations are to structure the contract to be consistent with the federal acquisition regulations format and to incorporate all contract reform conditions, including performance-based features. Bettis Atomic Power Laboratory/Bechtel Group Fixed-fee contract. The contract was recompeted in 1998. The new contractor was selected (the Bechtel Group) but the incumbent contractor, Westinghouse Electric Corporation, protested the award. The existing contract extended non-competitively pending the result of a bid protest to GAO. The bid protest was denied by GAO. The new contract with Bechtel was effective February 1, 1999. Knolls Atomic Power Laboratory/KAPL, Inc. (Lockheed Fixed-fee contract. Martin) Savannah River Technology Center/Westinghouse Savannah Incentive-fee contract. River Co. Source: GAO based on data from DOE’s operations offices. Page 17 GAO/RCED-99-141 Performance-Based Contracting Appendix III Comments From the Department of Energy Note: GAO comments supplementing those in the report text appear at the end of this appendix. Page 18 GAO/RCED-99-141 Performance-Based Contracting Appendix III Comments From the Department of Energy See comment 1. See comment 2. See comment 1. Now on p. 1. See comment 1. See comment 3. See comment 4. Page 19 GAO/RCED-99-141 Performance-Based Contracting Appendix III Comments From the Department of Energy See comment 5. See comment 6. Now on p. 6. See comment 7. Page 20 GAO/RCED-99-141 Performance-Based Contracting Appendix III Comments From the Department of Energy See comment 8. Page 21 GAO/RCED-99-141 Performance-Based Contracting Appendix III Comments From the Department of Energy See comment 9. Page 22 GAO/RCED-99-141 Performance-Based Contracting Appendix III Comments From the Department of Energy Now on p. 4. See comment 10. Now on p. 4. See comment 11. Now on pp. 5 and 6. See comment 12. Page 23 GAO/RCED-99-141 Performance-Based Contracting Appendix III Comments From the Department of Energy Now on p. 6. See comment 13. Now on p. 8. See comment 14. Page 24 GAO/RCED-99-141 Performance-Based Contracting Appendix III Comments From the Department of Energy See comment 1. Page 25 GAO/RCED-99-141 Performance-Based Contracting Appendix III Comments From the Department of Energy The following are GAO’s comments on the Department of Energy’s letter GAO Comments dated April 22, 1999. 1. We have made changes to the report as appropriate in response to DOE’s comments. 2. Our wording is drawn from DOE’s guidance on performance-based contracting, and we have made changes to our report to reflect DOE’s comments. DOE recommends that its laboratory contracts contain performance-based features, which include clear expectations described in terms of results, not how the work is to be accomplished. 3. As we stated in our report, DOE’s evaluations did not focus on the laboratory contractors, nor did these evaluations focus on the costs and benefits of performance-based contracting features, including the impact of fees. 4. We recognize that one of the purposes of providing fees is to reflect the financial risk associated with work performance, and we make this point in our report. Our 1994 recommendation questioned the cost-benefit of the increased fees, regardless of whether they were related to performance or financial risk. We continue to believe that our recommendation is relevant because DOE has not evaluated the cost and benefit of the fees it is providing to laboratory contractors. 5. We believe our wording adequately reflects the conditions discussed. Information on the laboratory fees and total contract costs is presented in appendix I. 6. We have made changes to the report as appropriate in response to DOE’s comments on contract type. We stated in our report that DOE’s performance-based contracting is in a state of transition. We also stated that there are wide variations in performance measures and fee arrangements negotiated by DOE and its laboratory contractors. This material is presented as facts describing the conditions that presently exist. Our report also describes the reasons for the variability in laboratory contracts and includes most of the reasons given in DOE’s comments. We have made changes in the report to reflect these additional reasons for the variability in DOE’s laboratory contracts. 7. Our statement that contract differences are the product of DOE’s relying on its field units to tailor contracts to local conditions is based on Page 26 GAO/RCED-99-141 Performance-Based Contracting Appendix III Comments From the Department of Energy interviews with numerous DOE field officials. This statement is not an implied criticism of how DOE negotiates contracts. Also, we disagree with DOE’s characterization that contractors’ preferences are “generally irrelevant” when accounting for the variations that exist among laboratory contractors. As DOE noted, contractors’ preferences are reflected in the negotiation process. In our discussions with DOE field officials responsible for negotiating contracts, laboratory contractors’ preferences on fees were cited as a critical factor in determining fee structures. 8. Our report recognizes that developing the optimum number of performance measures is a challenge, as reflected in the wide range of performance measures in use even among similar laboratories. We are not suggesting that any two contractors should have the same measures or the same number of measures. Our point is that DOE continues to struggle with finding the right number of measures. To further illustrate, the University of California’s fiscal year 1998 contracts for its two weapons laboratories—Lawrence Livermore and Los Alamos—contain 83 and 120 performance measures, respectively, even though these laboratories are very similar in budget and scope. They are, however, managed by different DOE field units. 9. Our purpose in including comments we received from DOE field units is to illustrate the wide differences in philosophy about the use of fees to motivate laboratory contractors. Several DOE field staff, as well as contractors, told us that they strongly believe that providing fees does not motivate contractors, including both for-profit and not-for-profit contractors. Moreover, our statement that performance-based contracting has tended to focus in some instances on monetary rewards at the expense of good science was a frequent comment from both DOE field officials and laboratory contractors. Thus, it is very important to identify the need for monetary incentives where they are appropriate. Other motivations that DOE cited for laboratory contractors, such as their reputations in the scientific community and desire for contract extensions, were added to our report. These differences in philosophy account for some of the variation in contracts. 10. Our report reflects information provided directly from DOE field staff, who we were advised by DOE headquarters were the proper source for this information. The data in DOE’s comments are reflected in the appendixes to our report. We have also revised our report to show that there are now 18 laboratory contractors, reflecting a recent change in how DOE defines its laboratories. Page 27 GAO/RCED-99-141 Performance-Based Contracting Appendix III Comments From the Department of Energy 11. DOE field officials told us that performance fees are used to encourage superior performance. Asserting that fees are used to link performance to financial reward is self-evident in this context. 12. We agree with DOE that no single approach in contracting has proven to be optimum, and we reflected this view in our report. Regarding the wide variability in fee arrangements, we stated that there was very little consistency among the contracts of similar laboratory contractors conducting similar work. We also stated that local conditions influence the variability in laboratory contracts. 13. Our wording was taken from DOE’s guidance on performance-based contracting. As we state in our report, prior assessments of performance-based contracting have not focused on laboratory contractors. We also stated in our report that DOE believes that the results from its assessments of performance-based contracting have been positive. We believe it is a logical and desirable step for DOE to determine whether performance-based contracting is improving performance at lower cost in its national laboratories. Also, we are not suggesting that DOE should abandon its performance-based approach or that it should eliminate performance-based fees in its laboratory contracts. It is also not our intent to show that performance-based contracting should be abandoned if its impacts on the laboratories cannot be measured. We believe that effective implementation of performance-based contracting provisions is dependent on the ability to support the fee amounts paid through a cost and benefit analysis. While it may appear intuitively obvious that defining performance expectations and measuring results are effective management tools, it is not intuitively obvious that the government is receiving a reasonable return on its investments in fee amounts for laboratory contractors. Likewise, while DOE commented that increases in fees reflect, in part, the increased financial risks being borne by contractors, no cost-benefit analysis quantifying this increased financial risk has been completed; thus it is not possible to determine if the proper level of fee is appropriate for the risk assumed. 14. We recognize that laboratory contractor fees are relatively small percentages of the total contract amounts. However, these percentages, which translated into $100 million in fees for fiscal year 1998, must be considered in light of the fact that DOE’s laboratories are government owned and that a laboratory contractor’s financial risk is limited. Page 28 GAO/RCED-99-141 Performance-Based Contracting Appendix IV Scope and Methodology To obtain information on the national laboratories’ contracts, we interviewed officials from the following laboratories: Sandia National Laboratories and Los Alamos National Laboratory in New Mexico; Lawrence Berkeley National Laboratory, Lawrence Livermore National Laboratory, and the Stanford Linear Accelerator in California; the National Renewable Energy Laboratory in Colorado; the Idaho National Engineering and Environmental Laboratory in Idaho; the Oak Ridge National Laboratory in Tennessee; and the Argonne National Laboratory in Illinois. We also spoke with laboratory officials in other locations to obtain cost and status information. We asked officials at these laboratories to comment on the impact of performance-based contracting on their operations. We also interviewed Department of Energy (DOE) officials responsible for overseeing these laboratories. These officials were from DOE’s operations offices in Albuquerque, New Mexico; Oakland, California; Oak Ridge, Tennessee; and Chicago, Illinois. We also interviewed DOE area and site office staff located at each of the operations offices we visited. To obtain a broader perspective, we interviewed DOE headquarters officials responsible for developing contracting policy. We conducted our review from September 1998 through April 1999 in accordance with generally accepted government auditing standards. (141265) Page 29 GAO/RCED-99-141 Performance-Based Contracting Ordering Information The first copy of each GAO report and testimony is free. Additional copies are $2 each. Orders should be sent to the following address, accompanied by a check or money order made out to the Superintendent of Documents, when necessary. VISA and MasterCard credit cards are accepted, also. Orders for 100 or more copies to be mailed to a single address are discounted 25 percent. Orders by mail: U.S. General Accounting Office P.O. Box 37050 Washington, DC 20013 or visit: Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders may also be placed by calling (202) 512-6000 or by using fax number (202) 512-6061, or TDD (202) 512-2537. Each day, GAO issues a list of newly available reports and testimony. To receive facsimile copies of the daily list or any list from the past 30 days, please call (202) 512-6000 using a touchtone phone. A recorded menu will provide information on how to obtain these lists. For information on how to access GAO reports on the INTERNET, send an e-mail message with "info" in the body to: email@example.com or visit GAO’s World Wide Web Home Page at: http://www.gao.gov PRINTED ON RECYCLED PAPER United States Bulk Rate General Accounting Office Postage & Fees Paid Washington, D.C. 20548-0001 GAO Permit No. G100 Official Business Penalty for Private Use $300 Address Correction Requested
National Laboratories: DOE Needs to Assess the Impact of Using Performance-Based Contracts
Published by the Government Accountability Office on 1999-05-07.
Below is a raw (and likely hideous) rendition of the original report. (PDF)