Alert Memo, NSF's Management of Cooperative Agreements

Published by the National Science Foundation, Office of Inspector General on 2012-09-28.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

            National Science Foundation   •   4201 Wilson Boulevard     •   Arlington, Virginia 22230
                                          Office of Inspector General


DATE:         September 28, 2012

TO:           Martha Rubenstein, Director
              Office Head and Chief Financial Officer,
              Office of Budget, Finance, and Award Management (BFA)

FROM:         Dr. Brett M. Baker /s/
              Assistant Inspector General for Audit

SUBJECT: NSF OIG Alert Memo, Report No. 12-6-001, NSF’s Management of Cooperative

This memo brings to your attention serious weaknesses in NSF’s cost surveillance measures for
awarding and managing cooperative agreements (CAs). These weaknesses are particularly troubling
since NSF currently has 685 open cooperative agreements, totaling nearly $11 billion. Thirty-eight of
NSF’s cooperative agreements are for over $50 million each and comprise $5.5 billion of the total
number of CAs 1. Among other things, NSF uses cooperative agreements to construct and fund the
operations and maintenance of large facility projects, including NSF’s federally funded research and
development centers.

A federal agency can use a cooperative agreement when entering into a relationship with a recipient
when the primary purpose of the relationship is to transfer a thing of value to carry out a public
purpose of support or stimulation, and substantial involvement between the federal agency and the
recipient when carrying out the agreement is expected. 2 A CA is not subject to the same rigor and
reporting mechanisms as a contract, nor does it have the same level of transparency over transactions
as a contract. Since NSF has chosen to use CAs for the construction, operation, and maintenance of
high-risk, high-dollar large facility projects, it is imperative that it exercise strong cost surveillance
controls over the lifecycle of such projects.

At the pre-award phase of such projects, such monitoring should include conducting audits of
awardees’ proposed budgets and accounting systems to ensure that awardees’ cost estimates are fair

  As of 8/23/12, NSF’s Awards Database listed 685 CAs, with obligated amounts totaling nearly $11
billion that do not have a final closed date. NSF does not consider 161 of these CAs totaling $2.2
billion as active primarily because they have passed their expiration date. However, they have some
level of outstanding administrative requirements needing to be addressed before they can be
considered closed.

    31 United States Code §3605
and reasonable and that the accounting system is adequate to bill the government properly. Although
they are not required by law or regulation, such audits provide essential information that NSF can use
to ensure that it funds only costs that are allowable and can be supported by adequate documentation.
Obtaining such information at the pre-award stage of high-risk, high-dollar CAs is especially important
as the proposed budget, once approved by NSF, creates the basis upon which awardees can draw down
advanced funds over the course of the award for specific cost items.

NSF does not regularly obtain such audits for its high risk, high-dollar projects. When the OIG
commissioned three, the findings were dramatic and underscored the need for such oversight. At our
request, over the last two years the Defense Contract Audit Agency (DCAA) audited the proposed
construction budgets for three of these non-competitive proposals valued at $1.1 billion and questioned
approximately $305 million (almost 28 percent) in unallowable or unsupported costs 3. All three of the
awardees’ proposals had significant unallowable contingency costs. In addition, two of the awardees’
proposals were initially found unacceptable for audit. After much work, one of these proposals was
ultimately audited; the auditors rendered an adverse opinion, finding that the proposal did not form an
acceptable basis for the negotiation of a fair and reasonable price. The third proposal, which was
submitted by an awardee found to have an inadequate accounting system, remains unaudited.
Inadequate proposals which contain large amounts of unallowable and unsupported costs undermine
NSF’s ability to serve as a proper steward of federal funds.

As we worked with NSF to resolve the DCAA audits, we identified serious weaknesses in NSF’s post-
award monitoring processes for high-risk projects that compound our concern that unallowable costs
could be charged to awards, thereby placing federal funds awarded under CAs at further risk. NSF
does not routinely obtain incurred cost submissions or audits of costs claimed on its largest CAs to
determine the allowability of direct and indirect costs claimed on federal awards. While not required
by law or regulation, such submissions and audits are essential tools for ensuring accountability in
high-risk, high-dollar projects. In their absence, unallowable costs charged to these awards may go
undetected because NSF lacks sufficient visibility over incurred costs. The failure to regularly obtain
incurred cost submissions also has a negative impact on our office’s ability to conduct incurred cost

NSF manages billions of dollars through cooperative agreements for major research facilities and
equipment. It stated in its FY 2013 budget request that “modern and effective research infrastructure is
critical to maintaining U.S. leadership in science and engineering.” Given the critical importance of
the projects funded through cooperative agreements and the billions of dollars at stake, it is vital that
NSF strengthen its end-to-end cost monitoring processes over high-risk CAs from the proposal stage to
close out.

Following is a summary of our specific concerns about NSF’s pre- and post-award processes for
Cooperative Agreements.

  Reports issued were Consortium of Ocean Leadership’s (COL) Ocean Observatories Initiative proposal in
September 2010 (OIG Report No. 10-1-012); Association of Universities for Research (AURA) in Astronomy’s
Advance Technology Solar Telescope proposal and AURA’s accounting system in March 2011 (OIG 11-1-001
and 11-1-010); and National Ecological Observatories Network’s (NEON) proposal in September 2011 (OIG
11-1-021). NEON’s subsequent proposal audit report was recently provided to NSF management and OIG will
issue this report in September 2012.

Pre-Award Process

NSF’s pre-award process includes a limited review of awardees’ cost estimates and budgets by a panel
of outside technical experts. Based on these reviews, panels provide reports to NSF that assess
whether, in the panel’s view, the project can be completed within the estimated cost and contingency,
in light of NSF’s no cost overrun policy.

The panel reviews do not reference or apply the OMB cost principles in their evaluations.
Consequently, panels do not review cost proposals for overstated costs with the same level of scrutiny
required in an audit. In fact, the final report from the panel reviewing one of NSF’s largest CAs, noted
that NSF policy does not require detailed, independent cost reviews and recommended that NSF
consider having such a review performed. Despite this recommendation, NSF approved the cost
estimates and made the award without an audit. We do not express an opinion on the overall merit of
the panels, but we concluded that the panels’ work on reviewing cost estimates is not sufficient, by
itself, to determine if the proposal represents a fair and reasonable basis for funding the project.

NSF has indicated that Grants Officers, not the panels, are required to assess compliance with OMB
cost principles. Our main concern is ensuring that, in high-risk, high-dollar projects, the agency is
taking proper steps to ensure that proposals provide an adequate basis for the negotiation of project
costs, and that potential recipients are capable of appropriately managing federal funds. Such steps
should include (1) obtaining proposal audits for large CAs prior to award to ensure that cost estimates
are reasonable; (2) obtaining audits of prospective awardees’ accounting systems and estimating
practices to determine whether these systems are capable of properly managing federal funds, and (3)
using Form 424C or an equivalent form or process which displays allowable and unallowable costs for
each budget item.

Proposal audits--- NSF does not require a proposal audit for high-risk, high-dollar CAs prior to award.
Proposal audits are important to evaluate the reasonableness of proposed budgets and are especially
useful when the award is not subject to full and open competition. Since proposed cost information is
generated by and is the responsibility of the awardee, it is essential to have the proposed costs and
associated supporting documentation verified by independent auditors to ensure the reasonableness of
awardee’s cost estimates. Without proposal audits, NSF is left making funding decisions without
adequate information to confirm the reasonableness of the estimates.

The value of and need for proposal audits can be seen in the results of the three proposal audits that
DCAA conducted on our behalf. In each case, although NSF had already approved the award, the
auditors found significant problems with the proposals:

   •   For COL (Ocean Observatories Initiative) the audit questioned $88 million of contingency
       costs that COL could not adequately support. Even after a reevaluation of the contingency
       costs was performed by the auditors, COL was unable to provide adequate supporting
       documentation for its proposed contingency costs. In short, after over 17 months of audit
       activity (over a year for the first audit and five months for the re-evaluation) the awardee could
       not provide adequate documentation to support $88 million in costs.
    •   For AURA (Advanced Technology Solar Telescope), significant deficiencies rendered the
        proposal unacceptable for audit and the auditors issued an inadequacy memo in March 2010
        stating that direct material estimates were not current, direct labor and indirect costs were
        insufficiently supported, and $62 million in contingency costs were unallowable and
        unsupported. Although the auditors continued working with the awardee for an additional six
        months, AURA still could not provide adequate documentation to support material, labor,
        contingencies, and indirect costs estimates, and the auditors issued a final inadequacy memo in
        October 2010. We continued to work with NSF to complete the audit. We were informed in
        June 2012 that, due to significant delays in construction, NSF requested AURA to re-baseline
        the project with a current cost estimate to complete. NSF agreed to provide this data upon
        receipt from AURA which is expected in October 2012. We are still waiting to complete this
        audit which began over two and a half years ago.

    •   For NEON (National Ecological Observatory Network), the auditors issued three inadequacy
        proposal memos over a four month period between June to September 2011. Despite working
        with NEON between January through September 2012 to proceed with the audit and clear some
        major inadequacies in the proposal, the auditors issued an adverse opinion on the proposal
        stating that the proposal did not form an acceptable basis for the negotiation of a fair and
        reasonable price. 4

        The auditors questioned approximately $154 million (35% of the $434 million proposal) for
        which NEON could not provide adequate supporting documentation --$81.7 million was for
        several items of proposed costs, including labor, materials, and equipment $72.6 million was
        for unallowable contingencies.

        This report is the most significant indicator that NSF’s proposal review process needs
        improvement. In addition to not being able to rely on the awardee’s cost estimates, the auditors
        informed us that they could not rely on the final design review panel’s report for technical
        verification of the number of proposed labor hours or material quantities and qualified their
        report accordingly. 5

The foregoing examples raise serious concerns about the adequacy of NSF’s proposal review process.
If the process worked as it should, then the output should withstand independent scrutiny. Awardees
should be able to readily provide the necessary supporting documentation for their proposed budget
estimates to third parties such as auditors, and that documentation should be sufficient to withstand
independent review. As the proposal audits reveal, that was not the case in the three proposals we
examined. The deficiencies in NSF’s process are much larger than the inclusion of unallowable
contingencies in proposals—it took months and multiple attempts for two recipients to obtain
sufficient documentation for DCAA to be able to conduct an audit. One of the audited proposals was
found to be an insufficient basis for negotiating a fair and reasonable price, and one proposal still
cannot be audited. NSF needs a much more robust process to ensure that it obtains better cost
information before funding its major CAs.
  NSF OIG Audit Report No. 12-1-008, Audit of National Ecological Observatory Network, Inc.’s
Proposed NEON Construction Budget, dated September 28, 2012.
  In each of these instances, the awardees stated that they prepared their proposals in accordance with
NSF’s policies and procedures contained in NSF’s Large Facilities Manual.
Accounting system audits—NSF does not require audits of the accounting systems of awardees that
will be managing large CAs, prior to making awards. Audits of prospective awardees’ accounting
systems and estimating practices prior to awarding large CAs determine if the awardee is able to
initiate, authorize, record, process, estimate and/or report costs in a manner that is consistent with
applicable Government laws and regulations. The adequacy of accounting systems and cost estimates
directly impacts the awardee’s ability to adequately monitor and manage federal funds.

NSF does have existing mechanisms it uses at the post-award stage to assess a recipient’s ability to
handle federal funds. However, at the pre-award stage, it should obtain accounting system audits
before entering into high dollar, high-risk awards. A recent audit of the accounting system for AURA,
which manages approximately $880 million in NSF awards, illustrates the value of such audits. That
audit found that AURA’s accounting system was not adequate to account for government funds and
that AURA was not eligible for advanced payments because its accounting system did not meet OMB
and NSF financial management standards in eight significant areas. In light of the deficiencies found
at AURA, and the fact that AURA has been managing large awards for many years without the
deficiencies being identified, accounting system audits should be performed on any other existing large
CA awardees that have not previously undergone such a review.

NSF’s processes also do not require written CA officer determinations documenting that awardees
have adequate accounting systems that meet OMB standards. Without accounting system audits and
CA officer determinations prior to awarding CAs, NSF cannot affirm that government funds are being
properly accounted for in accordance with federal standards, which increases the risk for unallowable

Use of Standard Form 424C-- NSF does not require the use of OMB’s SF 424C Form (Budget
Information - Construction Programs), for submitting proposals, which identifies allowable and
unallowable costs as well as amounts for contingencies. The Form 424C includes a column to identify
unallowable proposed costs, as well as a line-item cost classification for contingency costs. Since NSF
does not require use of Form 424C or an equivalent form or process, awardees are not required to
segregate unallowable from allowable costs in their proposed budgets. This puts NSF at risk of
unknowingly funding unallowable costs, especially when an awardee does not have a CA officer
determination that the awardee has an adequate accounting system. It also precludes stakeholders
from assessing the project’s level of risk and makes it difficult to track expenditures of certain funds in
the awardees’ accounting system.

Currently NSF allows its awardees to commingle contingency costs in other categories of direct cost
(such as equipment or other direct costs) on their proposal cover sheets. Awardees also do not identify
the amount of contingency costs that they estimate will be used for indirect costs. Without requiring
this information in proposals, NSF lacks assurance that awardees will properly account for and track
contingency expenditures or segregate unallowable costs in claims to the government.

Post-Award Process

We also have serious concerns about the adequacy of NSF’s post-award process for large high-dollar
high-risk CAs. NSF receives certain financial reports on its large facility CAs, but these reports do
not contain the level of detail needed to perform adequate cost surveillance. NSF only receives
sufficient cost details from a few awardees that also have large contracts and are therefore required to
provide annual incurred cost submissions. Large CA awardees that do not also have contracts are not
required to provide NSF with annual incurred cost submissions.

Incurred cost submissions, or their equivalent, are important for proper cost monitoring because they
provide visibility over awardees’ claimed costs since they include certified schedules of direct costs by
award (identified by cost element), and applied indirect expenses. Absent incurred cost submissions or
their equivalent, NSF cannot adequately monitor awardees’ expenditure of government funds during
the post-award stage, compounding our concern that unallowable costs could be charged to awards and
go undetected.

In addition, because NSF does not have incurred cost submissions, the OIG must work with NSF and
awardees to obtain submissions before an audit starts, thus excessively prolonging our audit process.
For example, for some time we have been attempting to obtain incurred cost submissions for
Associated Universities Inc. (AUI) and AURA, awardees that each manage $780 and $880 million of
NSF funds, respectively. It took us ten months (end of June through end of April 2012) to receive
AUI’s incurred cost submissions for three years. This was despite NSF’s cooperation in requesting its
awardee to provide the submissions.

We have been waiting seven months (since March 2012) for AURA’s incurred cost submissions for
three years, but still have not received them. This is again despite NSF’s cooperation in requesting
these submissions. NSF lacks first-hand knowledge of this problem because it does not routinely
conduct incurred cost audits of nonprofit awardees that have CAs and grants, but do not have federal
contracts because NSF’s post-award monitoring policies and procedures do not provide for conducting
incurred cost audits of large awardees without contracts. Nor has NSF set aside funding for this
important and necessary activity.

These waiting periods are inordinately long, and demonstrate the problems with NSF’s post-award
monitoring processes. Because NSF is the cognizant agency over 20 of the 38 cooperative agreements
over $50 million, it is prudent to require annual incurred cost submissions (that at least provide
detailed cost information by award and by cost element) and review these submissions upon receipt.

Audits of incurred cost submissions are also critical for proper monitoring, and would reveal instances
of noncompliance with federal regulations as well as costs claimed that are unallowable, unallowable
or unreasonable. The audits will provide vital information and also prevent recurrence of any infractions in
future periods of the awards. NSF does not routinely require such audits for high-dollar, high-risk CAs.

Our office has procured some incurred cost audits of large CAs. As a result of those audits, we found
that some awardees have employed accounting practices on large CAs that were not consistent with
their estimating practices. For example, some awardees applied their indirect cost rate to contingency
expenditures, but the application of an indirect cost rate to these expenditures was not provided for in
their approved budget estimates. Other awardees of large CAs did not separately track contingency
expenditures in their accounting systems. These awardees cannot demonstrate how their contingency
funds were actually spent in comparison to how they were budgeted and approved. This is, at least in
part, because NSF did not ensure prior to award that its awardees had financial management systems
that meet OMB requirements--which include providing for comparison of outlays (expenditures) with
budget amounts for each award.
The findings of our incurred cost audits demonstrate the value that can be added by conducting such
audits. NSF has indicated that it intends to initiate incurred cost audits and final audits on an as
necessary basis and consider risk analysis to determine the level of post award audit review necessary.
It has also indicated that it must work to identify funding for such audits. We strongly encourage the
agency to take the action necessary to develop a robust capacity for undertaking incurred cost audits,
as such audits are essential tools in ensuring appropriate oversight of federal funds.


Without improving end-to-end processes over CA monitoring from the proposal stage to award close
out, NSF cannot ensure that it receives reasonable value for taxpayer dollars and that those dollars are
not misused. NSF needs to institute a strengthened control environment together with additional pre-
and post-award cost surveillance measures to properly administer high-risk, high-dollar CAs in a
manner that protects federal funds.

We are not suggesting that NSF conduct proposal and accounting system audits, and require the use of
Form 424C for all 685 of its CAs. But, NSF should use a risk-based approach that at a minimum,
includes these elements for its high-risk, high-dollar (those over $50 million) cooperative agreements
that warrant the additional oversight necessary for proper accountability over federal funds.


We recommend that NSF management, using a risk-based approach, develop end-to-end cost
surveillance policies and procedures for its CAs to ensure adequate stewardship over federal funds. At
a minimum, NSF should implement such increased monitoring for its largest CAs valued at more than
$50 million.

At the pre-award stage, cost surveillance measures should include:

1.    Obtaining updated cost estimates, audits of awardees’ proposed budgets and cost accounting
     systems/estimating practices, and CA determinations of accounting system adequacy, prior to
     award. Accounting system audits should be performed for existing large CA awardees that have
     not undergone such a review.

2. Requiring use of Form 424C, or an equivalent form or process for construction proposals, to
   segregate allowable and unallowable direct and indirect costs and provide greater visibility.

At the post-award stage, cost surveillance measures should include:

3. Requiring annual incurred cost submissions and incurred cost audits to determine the
   reasonableness, allocability, and allowability of costs.

4. Requiring awardees that have been granted contingency funds to properly account for the funds
   consistent with their estimates and separately track budgeted versus actual contingency costs.
We provided a draft copy of this memo to NSF on September 17, 2012.

This memo is related to previously cited OIG reports (OIG Report Nos. 10-1-012, 11-1-001, 11-1-010,
11-1-021, 12-3-001, 12-1-008), and brings to NSF’s attention issues identified during that work that
warrants corrective action. In accordance with OMB Circular A-50, NSF and OIG should agree on a
corrective action plan for resolution of all findings. Please provide us with your proposed corrective
plan by April 1, 2013.

If you have any questions about this alert memo, please contact Jannifer Jenkins at (703) 292-4996 or
David Willems at (703) 292-4979.


Mary Santonastasso, DIAS
Jeffrey Lupis, DACS
Clifford Gabriel, OD
Michael Van Woert, NSB