oversight

Management of Contingency in the EarthScope Awards

Published by the National Science Foundation, Office of Inspector General on 2012-09-28.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                   National Science Foundation • Office of Inspector General
                   4201 Wilson Boulevard, Suite I-1135, Arlington, Virginia 22230


MEMORANDUM


DATE:                  September 28, 2012

TO:                    Ms. Martha Rubenstein
                       Office Head and Chief Financial Officer
                       Office of Budget, Finance, and Award Management
                       National Science Foundation

FROM:                  Dr. Brett M. Baker /s/
                       Assistant Inspector General for Audit

SUBJECT:               Audit of NSF’s Management of Contingency in the EarthScope
                       Awards, Report No. 12-2-010

Attached please find the final report on the subject audit. The report contains one finding
on NSF’s award, management, and oversight of contingency for the EarthScope project.

In accordance with Office of Management and Budget Circular A-50, Audit Followup,
please provide a written corrective action plan within 60 days to address the report
recommendations. This corrective action plan should detail specific actions and milestone
dates.

We appreciate the courtesies and assistance provided by so many NSF staff during the
review. If you have any questions, please contact Marie Maguire, Senior Audit Manager,
at (703) 292-5009.

Attachment

cc:    Clifford Gabriel                         G. P. Peterson
       Mary Santonastasso                       Allison Lerner
       Joanne Rom                               Susan Carnohan
       Gregory Anderson                         Marie Maguire
       Mark Coles                               Lisa Hansen
       Bart Bridwell                            Ken Lish
       Jeffrey Lupis                            Brittany DiChello
Audit of NSF’s Management of Contingency
         in the EarthScope Awards
          National Science Foundation
          Office of Inspector General

               September 28, 2012
                 OIG 12-2-010
Table of Contents

Introduction                                                    1

NSF’s Award, Management, and Oversight of Contingency for the   2
EarthScope Project Did Not Safeguard Federal Funds

Conclusion and Recommendations                                  6

Summary of Agency Response and OIG Comments                     7

OIG Contact and Staff Acknowledgements                          8

Appendices

   Appendix A: Agency’s Response                                9

   Appendix B: Objectives, Scope, and Methodology               12
Introduction

In recent years, NSF has instituted a policy of ensuring that its large facility construction
projects do not exceed their planned budgets by requiring a level of “contingency” costs
in the initial proposed budget. At the end of fiscal year (FY) 2011, NSF had 9 projects
with a total awarded amount of approximately $1.9 billion that included nearly $334
million (18 percent) for contingency. Project management, especially for projects of this
scale and complexity, requires a higher level of planning and risk management.
Proposal budgets create a basis upon which awardees can draw down funds over the
course of the award for specific cost items and serve as a tool for managing the
progress of the project.

Federal cost principles define how award funds may be budgeted and spent.
Among other things, OMB Circulars do not allow “[c]ontributions to a contingency
reserve or any similar provision made for events the occurrence of which cannot be
foretold with certainty as to time, intensity, or with an assurance of their happening.”
Audits of the Ocean Observatories Initiative (OOI), the Advanced Technology Solar
Telescope (ATST), and the National Ecological Observatory Network (NEON), three of
NSF’s large facility construction projects, revealed significant problems with proposed
budgets for these projects.

Auditors found that the proposed $386.4 million budget for OOI contained $88.1 million
in unallowable contingency funds. This finding was based on a lack of evidence to
support that the amounts budgeted were for events that could be “foretold with certainty
as to time, intensity, or an assurance of their happening.” Follow-up audit work which
examined how the contingency costs were estimated concluded that the contingency
amounts were not supportable as required by OMB, confirming the original finding that
the $88.1 million was unallowable. In the case of ATST, auditors found that the $298
million cost proposal contained $62.3 million in unallowable contingencies, and most
recently auditors identified $74.2 million in unallowable contingency costs budgeted in
the $433.8 million unauditable cost proposal for NEON. In total, audits have identified
over $224.6 million in unallowable contingency costs out of total proposed costs of over
$1.1 billion.

Because of the large dollar amounts associated with contingencies in NSF awards, the
risk posed by NSF’s current process of funding these costs, and the complexity of the
issue, we conducted an audit of the construction portion of the EarthScope-Acquisition,
Construction, and Facility Management project (EarthScope), a closed award to
examine NSF’s management and oversight of contingencies. This project was awarded
in 2003 for approximately $197.4 million, including an estimated $10.5 million for
contingency. EarthScope was comprised of four individual cooperative agreements
awarded to three separate entities.




                                              1
The following details the specifics of each award:

            SUMMARY OF EARTHSCOPE CONSTRUCTION AWARDS WE REVIEWED
               Awardee                            Project Name                   Total Awarded
                                                                                    Amount
 Incorporated Research              EarthScope Facility Office (Facility Office)   $ 4,295,263
 Institutions for Seismology (IRIS)
 Stanford University                San Andreas Fault Observatory at Depth           24,569,658
                                    (SAFOD)
 IRIS                               United States Seismic Array (USArray)            68,765,256
 UNAVCO, Inc.                       Plate Boundary Observatory (PBO)                 99,790,540
                                                                       Total     $197,420,717



NSF’s Award, Management, and Oversight of Contingency for the
EarthScope Project Did Not Safeguard Federal Funds

We found that the proposed contingencies for the EarthScope project were not
supported by cost data and were not compliant with the OMB cost principles. Once the
awards were made, the awardees had immediate access to the contingency funds. We
also found a weak management control environment that undermined NSF’s ability to
manage contingencies. As a result, NSF needs to implement additional internal
controls to reduce the risk of misuse or abuse of contingency funds.

    Unallowable Contingency Included in Each of the Four EarthScope Awards

Project officials included unallowable contingency in budget proposals at NSF’s
direction. The inclusion of contingency was unallowable based on OMB Circulars,
which state that “contributions to a contingency reserve or any similar provision made
for events the occurrence of which cannot be foretold with certainty as to time, intensity,
or with an assurance of their happening are unallowable.” That is, OMB requirements
do not allow estimated contingency for unforeseeable events, the occurrence of which is
uncertain and the effect cannot be measured precisely, to be placed on awards.
Rather, contingency estimates are only allowable for events, which can be foretold with
certainty as to time, intensity, or an assurance of their happening.

For the EarthScope project, NSF and the awardees used flat percentages to determine
the contingency amounts for each award. We were unable to find, and project officials
were unable to provide, any supporting evidence to show how these estimates were
calculated, therefore, there was no evidence that they meet the cost principle’s “with
certainty requirement”. Without that evidence, we concluded that the budgeted
contingencies were unallowable.

Based on our review of the Project Execution Plans and on information from NSF and
awardee officials, the estimated contingency amounts and percentages for the four
cooperative agreements comprising the EarthScope awards were as follows:

                                                 2
   •   Facility Office –The estimated contingency was $60,000 per year for 4 years, a
       total of $240,000 for the award. This was 4 percent of the total estimated project
       cost.

   •   SAFOD – The estimated contingency was approximately $1.5 million which was
       based on flat percentages of drilling cost for each of the project’s three
       construction phases: 8 percent for Phase I, 10 percent for Phase II, and 20
       percent for Phase III.

   •   PBO – The estimated contingency for the PBO award was $5.25 million, or 5.5
       percent of the total estimated project cost.

   •   USArray – The initial contingency for the USArray award was estimated at almost
       $3.5 million, or 5 percent of the total estimated project cost.

In 2011, NSF revised its contingency guidelines in its draft Planning, Use, and Oversight
of Contingency in the Construction of Large Facilities document. These draft guidelines
stated that NSF expects awardees to apply estimates of the likelihood of risk factors
occurring and their impact on the project budget, which should prevent awardees from
using flat percentages to estimate contingency. However, verifiable support sufficient to
meet OMB requirements is still necessary in order for the contingency to be allowable.

          Factors Contributing to the Award of Unallowable Contingency
We identified two factors in NSF’s process that contributed to the inclusion of
unallowable contingencies in the EarthScope award. First, NSF did not require Earth
Scope to submit Standard Form 424C (BUDGET INFORMATION—Construction
Programs). Form 424C displays budget line items for activities such as equipment,
demolition, engineering fees, and site work as well as contingencies; total allowable and
unallowable costs for each budget line item are also recorded on the Form. With this
form, contingent events that did not meet the certainty requirement of the cost principles
would be subtracted from the project’s total costs, leaving only allowable contingency
costs in the total allowable costs column. Those events that did not meet the certainty
requirement would be placed in the costs not allowable column. The form that NSF
required the awardees to submit did not include a line item for contingencies, nor did it
include columns for allowable and unallowable costs. Use of the Form 424C (or one
like it) would have enabled NSF to clearly identify contingency amounts, and to further
distinguish between amounts that were allowable versus amounts that were not.
The unallowable contingency (i.e., that for unforeseeable events) should be budgeted
for by NSF, but would not be placed in the award until the awardee demonstrates a
bona fide need supported by verifiable cost data.

It is worth noting that in 2006, NSF began withholding the annual contingency budget
from the awardees’ annual funding increments and centrally managing it. NSF elected
to keep the contingency funds within the Major Research Equipment and Facilities
Construction (MREFC) account. In most cases, NSF awarded the contingency amount
as separate funding increments to awardees after a change order had been submitted
and approved. However, this is not NSF’s current process. Presently there are no

                                            3
technical barriers to prevent awardees from drawing down the contingency funds at any
point in the project and using them for any purpose.

Second, NSF’s policies and procedural guidance were not sufficient to prevent
unallowable contingency (i.e., for unforeseeable events) from being placed into the
awards. During the EarthScope project, NSF and the awardees were relying on the
following separate internal guidance documents:

   •   Facilities Management & Oversight Guide ( July 31, 2003) replaced by the Large
       Facilities Manual (May 2007, revised September 2009)
   •   NSF Risk Management Guide (May 2004 draft)
   •   Definition and Use of Contingency Resources in NSF Facility Construction (May
       6, 2004 draft)
   •   Guidelines for Development of Project Execution Plans for Large Facilities
       (August 26, 2004 draft)
   •   Guidelines for Reviews of Large Facilities (September 29, 2005 draft)
   •   Guidelines for Planning and Managing the Major Research Equipment and
       Facilities Construction (MREFC) Account (November 22, 2005)
   •   Guidelines for Development of Internal Management Plans for Large Facilities
       (November 14, 2008 draft)

Some of these documents defined contingency differently and none of the contingency
definitions reflected the OMB cost principles, which distinguish between foreseeable
and unforeseeable events. For example, NSF’s Definition and Use of Contingency
Resources in NSF Facility Construction defined contingency as “the portion of the
project’s construction budget that is held in reserve to accommodate unknowns relating
to the requirements and the uncertainty that are within the scope of the project." At the
same time, the 2004 Risk Management Guide stated that “Contingency is the amount
added to an estimate to allow for items, conditions, or events for which the state,
occurrences, and/or effect is uncertain and that experience shows will likely result, in
aggregate, in additional costs.” This document also states that “Contingency usually
excludes items such as major scope changes…,” while the July 2003 Facilities
Management and Oversight Guide states that “Contingency is used for additional scope
and work to meet the necessary requirements of the project.” NSF’s definition of
contingency should be consistent with the OMB cost principles to ensure NSF’s
compliance with OMB requirements, and should be consistently articulated in NSF
guidance. A lack of clarity as to what constitutes contingency could undermine the
agency’s ability to oversee contingency funds.

       Weak Management Control Environment Undermined Ability to Manage
                                Contingencies

We found that problems with the control environment governing contingencies
undermined NSF’s (and our) ability to ensure that contingencies were used
appropriately. Importantly, we determined that two of the awardees (Stanford University
and UNAVCO), which expended nearly $7.9 million of contingency funds, did not
separately track contingency expenditures in their accounting systems. Thus, we could
not confirm that the awardees spent the contingency funds for items requested in the
                                            4
change orders. The inability to determine what the contingency funds were actually
used to purchase is particularly concerning because these two awardees spent
approximately 75 percent of the contingency funds associated with the EarthScope
project.

In addition, NSF lacked visibility over EarthScope’s expenditure of contingency funds
because its process, prior to centrally managing contingencies in FY 2006, permitted
the awardees to approve all change order requests for $250,000 or less to the MREFC
account or $100,000 or less to the operations and maintenance yearly cost without
NSF’s review or approval. We found that prior to FY 2006, awardees had approved all
9 of the existing contingency change orders totaling over $1 million. Eight of these
change orders totaling almost $960,000 were for changes related to the SAFOD project.
This approval process limited NSF’s ability to ensure that the requests and approval for
the use of contingency funds were appropriate.

Finally, our review of the EarthScope awards surfaced instances in which NSF
approved the use of contingency funds for matters that did not appear to represent the
materialization of contingent events. For example, the Facility Office project used
$47,271 to hire a publications coordinator. The USArray project used $728,875 to fund
an increase in the general and administrative rate, a large portion of which was due to
the awardee retaining space NSF told them it would not support, and another $399,433
for an off-site data archiving and distribution center.

By using contingency funds for expenses that do not appear to relate to contingent
events, there is a risk that there will not be sufficient funds available if true contingent
events occur. There is also a risk that project cost overruns will be obscured when
contingency funds are used for expenses that are not related to contingent events.




                                               5
Conclusion and Recommendations

The illustration below summarizes weaknesses related to NSF’s process for award,
management, and oversight of contingency in the EarthScope project.

                                                                        NSF's Process for
                                           Change Orders
          Proposal Budget                                             Approving Contingency
                                        Approved by Awardee
                                                                            Requests

• No Form 424C to identify        • Awardees could spend up     • Some contingency funds
  contingency costs or to           to $250,000 of MREFC          spent on non-contingent
  segregate unallowable             contingency funds without     costs.
  and allowable contingency         NSF approval before NSF     • Two awardees, which
  costs.                            began withholding the         expended 75 percent of
• No identification of specific     annual contingency            EarthScope contingency
  contingent events.                budget from the awardees'     funds, did not separately
• No verifiable cost data to        annual funding increments     track contingency
  support budgeted                  in 2006.                      expenditures, so NSF was
  contingency.                    • Awardee does not have to      unable to determine what
• Contingency amounts               show "bona fide" need for     was actually purchased
  determined by using flat          contingency supported by      with contingency funds.
  percentages.                      current, accurate, and      • The definition of
                                    complete cost data.           contingency in NSF's
                                                                  policies and procedures is
                                                                  not consistent with the
                                                                  OMB cost principles.

As shown, and discussed herein, NSF’s award, management, and oversight of
contingencies for the Earth Scope construction project did not comply with federal
requirements. NSF permitted the awardees to include unallowable contingency, which
was not based on adequate, accurate, current cost data, in its proposal budgets. Once
the awards were made, NSF permitted the awardees to spend contingency funds
without demonstrating a “bona fide” need that met federal requirements. These
management practices increased the risk of misuse and abuse of contingency funds.

Recommendations

We recommend that the NSF Chief Financial Officer take appropriate action to improve
its award, management and oversight of contingency funds by strengthening its
guidance, processes and internal controls. Among other things, the agency should:

   1. Require awardees to use Standard Form 424C or an equivalent form (which
      clearly identifies amounts needed for contingencies and distinguishes between
      allowable and unallowable costs) when submitting proposals for construction
      projects;

   2. Require awardees to support contingency estimates in budget proposals with
      adequate supporting cost data;

   3. Only include amounts for allowable contingencies in awards;


                                                     6
   4. Ensure that internal contingency policies and procedures reflect OMB cost
      principles for contingency; and

   5. Retain control over funds budgeted for unforeseeable events and release the
      funds only when the awardee demonstrates a bona fide need supported by
      verifiable cost data.


Summary of Agency Response and OIG Comments

NSF concurred with recommendation 1, and agreed to require awardees to use
Standard Form 424C or an equivalent form when submitting proposals for construction
projects. We consider NSF’s comments and planned action to be responsive to this
recommendation.

NSF’s position is that it is already in compliance with recommendation 2, but the agency
agreed to re-examine its procedures in this area. While NSF may have made
improvements in this area, during the EarthScope project, flat percentages, without
supporting documentation, were used to estimate contingencies. If the results of NSF’s
reexamination of its procedures require adequate supporting documentation of
estimates, then we consider this to be responsive to our recommendation.

NSF’s position is that it is already in compliance with recommendations 3 and 4. NSF
replied that OMB has explicitly stated to NSF that contingency budgeting for large
construction project is an essential aspect of managing allowable costs resulting from
foreseen, anticipated events. NSF states that it is allowable for contingency amounts to
be explicitly included in budget estimates to the extent that they are necessary to
improve the precision of those estimates. As such, according to NSF, contingency
amounts are to be included in the awards and to be expended during project execution.
However, we disagree with NSF on this position. Per the OMB cost principles,
“contributions to a contingency reserve or any similar provision made for events the
occurrence of which cannot be foretold are unallowable.” NSF’s current policies and
procedures allow items that are unforeseeable to be added to an award, thus it is our
position that NSF is not in compliance with the OMB cost principles or our
recommendations. The intent of these two recommendations is to allow NSF to budget
for all types of contingency (foreseeable and unforeseeable), but to not include in the
award the estimated amount for items which cannot be foretold with certainty as to time,
intensity, or with an assurance of their happening. Rather, NSF can provide funds to
the awardees after the unforeseen contingent events occurred and the awardees
provide information to support the amount requested. Therefore, we do not consider
NSF’s comments to be responsive to these two recommendations.

Finally, NSF disagrees with recommendation 5 to the extent that our intention is to
exclude the budgeting of contingency estimates for events “the occurrence of which

                                           7
cannot be foretold with certainty as to time, intensity, or with an assurance of their
happening.” Again, our intention is not to prevent NSF from budgeting for these events.
NSF should internally budget for these events, but not include the estimated amounts in
the award or make the funds available for the awardee to obtain through the cash draw
down process. Rather, NSF should maintain control over the budgeted contingency
funds and release the funds to the awardee after it has demonstrated a need for the
funds and provided support for the amount requested. Additionally, this step will act as
a stronger internal control over the appropriate expenditure of contingency funds.
Therefore, we do not consider NSF’s response to address this recommendation.

We look forward to receiving the Corrective Action Plan and working with NSF officials
to resolve the recommendations.

We have included NSF’s response to this report in its entirety as Appendix A.


OIG Contact and Staff Acknowledgements

Marie Maguire- Director of Performance Audits
(703) 292-5009 or mmaguire@nsf.gov

In addition to Ms. Maguire, Susan Carnohan, Lisa Hansen, Ken Lish, and Brittany
DiChello made key contributions to this report.




                                           8
Appendix A: Agency’s Response




                                9
10
11
Appendix B: Objectives, Scope, and Methodology

The objectives of this audit were to examine the sufficiency of the National Science
Foundation’s oversight of awardees’ expenditure of contingency and to assess whether
the awarded contingency funds were spent in accordance with the terms of the award.
Due to limited visibility of awardee expenditure data, we were unable to address the
latter objective. The scope of our audit focused on a single closed project. We
reviewed seven construction projects with contingency that were either closed or
nearing completion at the time of fieldwork. We selected the construction portion of the
EarthScope-Acquisition, Construction, and Facility Management project (EarthScope),
which consisted of four cooperative agreement awards for the following projects: the
Plate Boundary Observatory (PBO), the San Andreas Fault Observatory at Depth
(SAFOD), USArray, and the EarthScope Facility Office.

We conducted this performance audit from February 2011 to August 2012, at NSF in
Denver, Colorado and Arlington, Virginia. To answer our objectives, we obtained
information on the contingency from NSF’s eJacket reporting system, NSF officials, and
EarthScope personnel for the project period of September 1, 2003 through September
30, 2009. We gained an understanding of NSF’s oversight and management
responsibilities regarding how contingency funds were requested, awarded, and
managed.

To meet our objectives, we:

   •    Obtained and reviewed relevant laws, regulations, policies, and procedures,
   •    Obtained and reviewed prior Federal audits and reviews,
   •    Communicated with the NSF OIG Office of Investigations,
   •    Consulted with NSF OIG Legal Counsel,
   •    Interviewed NSF officials,
   •    Obtained and reviewed contingency change orders and supporting
        documentation for each of the four awards,
   •    Communicated with awardee officials, and
   •    Reviewed IRIS’ expenditure of contingency for the Facility Office and USArray
        projects to assess the appropriateness of usage.

We reviewed NSF’s compliance with applicable provisions of pertinent laws and
regulations including:

    •   2 CFR Part 220, Cost Principles for Educational Institutions (formerly OMB
        Circular A-21),
    •   2 CFR Part 230, Cost Principles for Non-profit Organizations (formerly OMB
        Circular A-122), and
    •   2 CFR Part 215, Uniform Administrative Requirements for Grants and
        Agreements with Institutions of Higher Education, Hospitals, and Other Non-
        Profit Organizations (formerly OMB Circular A-110).



                                           12
As discussed in this report, we found that NSF was not in compliance with these
regulations because it permitted inclusion of unallowable contingency in the awardees’
budgets, which is inconsistent with OMB’s cost principles. Specifically, NSF required
the awardees to set aside contingency funds that were not for events which can be
foretold with certainty as to time, intensity, or with an assurance of their happening, as
required by these regulations

We also obtained an understanding of the management controls over NSF’s process for
reviewing, managing, and overseeing contingency through interviews with NSF officials
and staff and by reviewing NSF policies and procedures. We identified internal control
deficiencies, which we discuss in this report. However, we did not identify any
instances of fraud, illegal acts, violations, or abuse.

During the course of this audit, the auditors relied on information and data received from
the EarthScope awardees and NSF officials in electronic format that had been entered
into a computer system or that resulted from computer processing. We did test the
reliability of the data by corroborating it between the awardee and NSF records. We
also tested the reliability of the awardees’ financial data by corroborating the results with
NSF’s Federal Financial Reports. We also obtained information from NSF’s eJacket,
which serves as a repository for all documents related to a specific proposal or award
from all of NSF’s central information systems.

We conducted this performance audit in accordance with generally accepted
government auditing standards. Those standards require that we plan and perform the
audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our
findings and conclusions based on our audit objectives. We believe that the evidence
obtained provides a reasonable basis for our findings and conclusions based on our first
audit objective to examine the sufficiency of NSF’s oversight of awardees’ expenditure
of contingency.

We held an exit conference with NSF officials on August 30, 2012.




                                             13