oversight

Project Reporting of NSF Awards

Published by the National Science Foundation, Office of Inspector General on 2013-03-04.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                   National Science Foundation • Office of Inspector General
                   4201 Wilson Boulevard, Suite I-1135, Arlington, Virginia 22230


MEMORANDUM


DATE:                  March 4, 2013

TO:                    Dr. Cora B. Marrett
                       Deputy Director
                       National Science Foundation

FROM:                  Dr. Brett M. Baker
                       Assistant Inspector General for Audit

SUBJECT:               Audit of Project Reporting of NSF Awards, Report No. 13-2-006

Attached please find the final report on the subject audit. The report contains two findings
on the need for NSF to ensure that all project reports are submitted timely and to improve
controls to prevent assigning a second NSF identifier number for principal investigators.

In accordance with Office of Management and Budget Circular A-50, Audit Followup,
please provide a written corrective action plan within 60 days to address the report
recommendations. This corrective action plan should detail specific actions and milestone
dates.

We appreciate the courtesies and assistance provided by so many NSF staff during the
review. If you have any questions, please contact Marie Maguire, Director of
Performance Audits, at (703) 292-5009.

Attachment

cc:            Allison Lerner                   Daniel Hofherr
               G. P. Peterson                   Maureen Miller
               Cliff Gabriel                    Susan Carnohan
               Martha Rubenstein                Marie Maguire
               Mary Santonastasso               Wendell Reid
               Jean Feldman                     Jeffrey Stitz
               Jeffrey Vieceli                  Brittany DiChello
Audit of Project Reporting of NSF Awards




       National Science Foundation
       Office of Inspector General




              March 4, 2013
          OIG Report No. 13-2-006
Results in Brief

We conducted this performance audit to determine if the National Science Foundation
(NSF) has implemented effective controls over grantee project reporting since our prior
audit conducted in 2004 (Report No. 05-2-006, dated December 13, 2004). We
reviewed all annual and/or final project reports, approximately 55,500, which were due
or overdue between October 1, 2010 and March 31, 2012.

Unless the awarding agency has granted an exception, the Office of Management and
Budget requires that recipients of Federal funds submit reports at least annually and at
the end of a project. These project reports, prepared by the individual principal
investigator (PI) conducting the research, explain the progress of actual
accomplishments with the goals and objectives established for the period, and provide
accountability for taxpayer dollars used to fund the work.

We determined that NSF has made significant improvements in ensuring that grantees
submit project reports on time. We found that the percentage of final reports submitted
late declined from 53 percent in our prior audit to 20 percent for our current audit, while
the percentage of final reports not submitted declined from 8 percent to 5 percent.
Similarly, for annual reports, the percentage not submitted also declined from 42
percent to 2 percent. 1 While the percentages of late and not submitted reports are
much lower than in 2004, they represent almost 12,000 final and annual reports
submitted late and over 1,500 final and annual reports not submitted.

One control that NSF implemented to increase compliance with the reporting
requirement is to not provide new or supplemental funding to any PI whose project
report is past due. This control is not fully working as intended because there is no
automated control to prevent assigning a second NSF identification number to PIs who
have an NSF identification number and then transfer to another institution. While we
found only two instances of a PI or co-PI with a past due report who received new
funding after transferring to a new institution and receiving a NSF identification number
during the period we examined, there is the potential that it could occur more frequently.
NSF identified 129 PIs with more than one PI identification number in calendar year
2010, 167 in 2011, and 144 in 2012 through September.

We recommend that the NSF take appropriate action to ensure that all project reports
are submitted on time to NSF. Additionally, we recommend that the NSF take
appropriate action to improve procedures to prevent assigning a second identification
number for PIs. NSF management concurs with both of our recommendations.




1
 The 2004 audit could not determine the percent of annual reports submitted late because data was not available;
however, our current audit found that 22 percent of annual reports were submitted late.




                                                            1
Introduction

The National Science Foundation (NSF) funds approximately 22% of all federally-
supported basic research across all fields of science and engineering conducted at the
nation’s colleges, universities, academic consortia, nonprofit institutions, and small
business. In fiscal year 2011, NSF funded approximately $6.1 billion for 21,000 awards,
including new grants and cooperative agreements as well as continuing increments and
supplements on awards made in prior years.

Unless the awarding agency has granted an exception, the Office of Management and
Budget (OMB) Circular A-110 requires that recipients of Federal funds submit
performance reports at least annually and at the end of a project. The individual
principal investigator (PI) conducting the research prepares the project reports, which
explain the progress of actual accomplishments with the goals and objectives
established for the period and provide accountability for taxpayer dollars used to fund
the work. NSF uses these reports to monitor the progress and accomplishments of the
funded projects.

For most awards, NSF requires that, the PI must submit an annual project report at
least 90 days prior to the end of the project's current budget period. In addition, NSF
requires that the PI submit the final report within 90 days after the award expiration or
termination date. 2 NSF requires that the reports be submitted through the NSF’s
FastLane system, an interactive real-time system used to conduct NSF business over
the Internet. 3 If the PI is overdue on the annual report for a multi-year award, NSF
should not award the next year's funding or any new award. Additionally, NSF should
not award any new funding to the PI and any co-PIs until NSF receives and accepts the
final project reports from any previous awards.

In 2004, we conducted an audit of project reporting for NSF awards (Report No. 05-2-
006 dated December 13, 2004). The audit determined that 53 percent of the final
project reports were submitted late and 42 percent of annual project reports were not
submitted and that NSF lacked adequate controls to ensure PIs submitted timely
reports. The audit also determined that NSF's existing controls to prevent new awards
to PIs with overdue final project reports were ineffective. We recommended that NSF
continue with its plans to develop automated tracking and reminder systems for annual
and final project reports and improve the reporting mechanism in the FastLane system.
We also recommended that NSF update its policies and procedures on project reporting
and clarify roles and responsibilities of NSF staff for monitoring timeliness and enforcing
non-receipt of reports. Finally, we recommended NSF revise the award letter to include
the due dates for both annual and final project reports for that award. NSF completed

2
  We did not include the Project Outcomes Report in our audit. This new report is intended for the general public and
is required for new awards made or existing awards that receive funding increments or supplements on or after
January 4, 2010.
3
  NSF has begun transitioning the project reporting service to its Research.gov website, which modernizes
FastLane’s grant management services.




                                                             2
corrective actions on all OIG recommendations in 2007 and has fully implemented its
automated tracking and reminder notification system for project reporting.

Results of Audit

NSF has made significant improvements in ensuring that grantees submit project
reports and that the reports are timely. The following chart compares our current audit
results to our 2004 audit:


                   TIMELINESS OF FINAL AND ANNUAL PROJECT REPORTS

                                             2004 audit results            Current results

FINAL PROJECT REPORTS

Percent of final reports                     53%                           20% 4
submitted late
Average number of months late                5 months                      1.8 months 5
Percent of final reports not                 8%                            5%
submitted
Average number of months final               26 months                     5.1 months, with oldest 533 days
reports not submitted were
overdue 6



ANNUAL PROJECT REPORTS

Percent of annual reports                    Not available                 22%4
submitted late

Average number of months late                Not available                 1.4 months
Percent of annual reports not                42%                           2%
submitted
Average number of months not                 Not available                 5.5 months, with oldest 548 days
submitted annual reports were
overdue

Our audit determined that improvements to NSF’s tracking and automated reminder
system for annual and final project reports, which were fully implemented after our 2004
audit, are generally working as intended. However, some PIs have not submitted

4
  We are excluding reports submitted within 2 days of the due date.
5
  Outer range is not comparable with 2004 audit, which used a 5-year period as the scope.
6
  Average is not directly comparable with 2004 audit, which used a 5-year period




                                                            3
overdue annual and final project reports or have submitted reports late, and NSF needs
to further improve its efforts to obtain reports on a timely basis. In addition, there is a
risk that a PI with a past due report who then transfers to a new institution may receive
new funding, thus circumventing NSF’s established policy, because NSF needs to
improve its controls to prevent assigning a second NSF identification number to a PI
who transfers to another institution.


More Effort Needed to Ensure All Project Reports are
Submitted Timely

Grantees did not submit 5 percent of final reports due and approximately 2 percent of
annual reports due during the period between October 1, 2010 and March 31, 2012.
While the number of reports not received is relatively small, 70 percent of the final
reports not submitted and approximately 65 percent of annual reports not submitted
were past due at least 90 days. In addition, of the submitted reports, 20 percent of final
reports and 22 percent of annual reports were submitted at least 3 days late, with 18
percent of final reports and 14 percent of annual reports submitted at least 90 days late.
Furthermore, we identified three PIs who each did not submit three annual reports and
one PI with 11 reports submitted late. The following charts display the range of days
and number of reports not submitted or late 7. Appendix C provides information by NSF
Directorate.



                                    Range of Final and Annual Reports Not
                                                  Submitted
Number Not Submitted




                   400
                                                                                        331
                                              188                       293
                       200                               85
                                     0                             56         79
                                0        30
                                                    15        49                   93              51
                         0                                                                    90
                             3-29                                                                       Annual…
                                     30-59
                                               60-89                                                    Final…
                                                          90-119
                                                                   120-209
                                                                              210-364
                                                Range of Days Not Submitted                   >365
                                                     Final Reports    Annual Reports




7
      Because March 31, 2012 was on a Saturday, these amounts exclude reports received by Monday, April 2, 2012.




                                                                         4
                                     Range of Late Annual and Final Reports
                                           4920
Number Not Submitted



                       6000

                       4000
                                                      1797
                                    1559
                       2000                                        809
                                                  729                          472          574
                                                             382                                      127
                              0                                          228          255                     11
                                                                                                  108              Annual…
                                  3 - 29                                                                     15
                                            30 - 59
                                                        60 - 89                                                    Final…
                                                                     90 - 119
                                                                            120 - 209
                                                                                      210 - 365
                                                         Range of Days Not Submitted                        >365
                                                              Final Reports          Annual Reports


NSF award recipients are required to submit periodic progress reports, including a final
progress report within 90 days after grant expiration. Project reports provide valuable
insight into how well the NSF-funded project is operating and its progress on achieving
the research and other goals and objectives as stated in the funded proposal and
approved agreement. For example, project reports typically list senior personnel, post
doctorates and students involved in research; research, education, training and
outreach efforts to the research community; and articles published and conference
presentations as a result of the ongoing research.

NSF policies require the PI to submit the annual project reports at least 90 days prior to
the end of the current budget period and the PI must submit a final project report within
90 days of the grant’s termination or expiration. NSF’s automated tracking and reminder
system sends both the PI and institution a reminder notice for every annual and final
project report when it becomes due and every 30 days until the report is received or
becomes overdue. Once the annual or final project reports are overdue, the automated
tracking and reminder system sends a biweekly overdue notice to both the PI and the
institution until the report is submitted.

Based on our interviews, many program officers explained that while automated alerts
to the PIs were effective for receiving most reports, it should not be the sole approach.
For instance, some program officers found, although not required, personal phone calls
or e-mails to PIs with reports overdue for prolonged periods were more effective than
the automated alerts. NSF program officers, who often have a portfolio of over 100
awards, only receive a copy of the first due and overdue notices, and do not receive
subsequent system alerts or status information for reports that continue to be overdue.
After these first notices, program officers can only determine that reports are delinquent
if they actively search NSF systems to determine the status. Some of the program
officers we interviewed were not aware that the required reports for some of their
awards were overdue several months or were submitted several months late because
they were not monitoring the status of reports.


                                                                                       5
In addition, program officers we interviewed stated that receiving a periodic reminder
that includes a list of awards they manage with overdue reports would be beneficial for
monitoring awards, such as a list for awards with reports overdue more than 90 days.
While some program officers had administrative staff who helped them closely track the
required reports for their awards, especially near the end of the fiscal year, others did
not. Furthermore, according to some NSF program officers we interviewed, the NSF
systems did not easily identify awards with project reports that were overdue for a
prolonged period.

As a result of late or unsubmitted reports, NSF cannot fully assess the extent to which
grantees have met their program goals and objectives for these awards with overdue
reports. Without timely annual project reports, NSF program officers may not be able to
address potential problems that could impair the satisfactory performance of a funded
project. In addition, if final project reports are not received timely, NSF management
may not be fully informed about the results of the research it funded.


Recommendation

   1. We recommend that the NSF Deputy Director take appropriate action to ensure
      that all performance reports are submitted on time to NSF. Such actions could
      include:

             A) Periodically providing to program officers, or instructing them to obtain,
             a listing of the awards they manage which have annual or final project
             reports that are at least 90 days late. Program officers should then
             contact the PIs responsible for these outstanding reports; and

             B) Providing additional training and guidance to program officers and
             administrative staff on how to use NSF systems to identify overdue project
             reports and monitor their status.




More Control Needed in Assigning NSF PI Identification
Numbers

During our tests of compliance with NSF’s policy to not award new funding to PIs or co-
PIs with past due reports, we found one PI and one co-PI who should not have received
a new award. Both of these PIs transferred to other institutions and the receiving
institutions requested and received an NSF PI identification number. The PIs, under the
new PI numbers, then received new awards, even though they each had a past due



                                              6
report for an existing award obtained while at the prior institution and with a different PI
number.

In order to protect the confidentiality of PI information, NSF restricts access to most
FastLane features to authorized users. NSF’s PI Identification number is a unique
numerical identifier and is used along with last name and password, to verify the identity
of an authorized user accessing the FastLane system. In addition, the PI identification
number serves as the identifier for other internal agency system processes, such as the
check to determine if the PI is associated with any past due reports before releasing
funds for a new award, supplement, or increment.

There is currently no automated control to prevent assigning a second NSF
identification number to PIs with an NSF identification number who then transfer to
another institution. Previously, NSF matched PI application data with the social security
number (SSN), but this practice was discontinued several years ago when SSNs were
removed from FastLane and NSF did not implement an alternative control.

NSF staff informed us that the system will check PI first name, last name, and email
address combination for the institution. If any record is found with the same
combination, it will not allow the creation a new record. In addition, the system will
check for an existing PI identification number in the database if a user provides an
identification number during user account creation. In such a case, NSF will advise the
user that the PI is already in the database and hence information will not be updated.
However, NSF stated that if the PI identification number is not associated with the same
institution, the NSF system will allow a second identification number to be assigned to a
PI. Per NSF staff, NSF usually is informed of the two PI identification numbers by the PI
or NSF program officer, who request that all PI awards be consolidated under one
number.

The PI and co-PI with more than one NSF identification number that we identified were
able to receive additional funding of approximately $1.36 million despite both PIs each
having one past due report for an existing award. Once NSF was informed that the PIs
had two identification numbers, NSF eliminated one of the numbers, but this action
occurred after the new funding was made available to the PIs. While we found only two
instances of this occurring, there is the potential that it could occur more frequently.
NSF identified 129 PIs with more than one PI identification number in calendar year
2010, 167 in 2011, and 144 in 2012 through September. 8




8
 Other than the PI and co-PI that we identified, we did not perform any additional tests to determine if any other PIs
with a second identification number obtained new funding while having past due reports.




                                                              7
Recommendation

We recommend that the NSF Deputy Director:

   2. Take appropriate action to improve procedures to prevent assigning a second
      identification code for PIs that already have an NSF identification code. Such
      actions could include:

             A) Adding a FastLane warning prompt for any first and last name matches
             before assigning any new PI identification code; and

             B) Updating the instructions on FastLane to include a prominent notice
             and explanation for the institution and PI that a new PI identification
             number should not be established if the PI has previously applied or
             received funding from NSF.


Summary of Agency Response and OIG Comments

NSF management concurs with our first recommendation. NSF will develop a report for
program officers that will list awards with annual and/or final project reports overdue at
least 90 days. NSF will also review and update existing guidance and explore training
opportunities. NSF agrees with the intent of our second recommendation, but will
explore alternatives to addressing potential duplicate PI identification codes and agreed
to review the instructions provided in FastLane for registering PIs. NSF is migrating its
project reporting function from the FastLane system to Research.gov, and therefore
plans to implement corrective actions in the new system. We have included NSF's
response to this report in its entirety as Appendix A.

We consider management’s comments and planned actions to be responsive to our
recommendations.


OIG Contact and Staff Acknowledgements

Marie Maguire – Director of Performance Audits
(703) 292-5009 or mmaguire@nsf.gov

In addition to Ms. Maguire, Wendell Reid, Brittany DiChello, and Jeffrey Stitz made key
contributions to this report.




                                               8
Appendix A: Agency Response




                              9
Thank you very much for providing us the opportunity to review and comment on the Office of
Inspector General's (OIG) draft Audit Report, "Audit of Project Reporting of NSF Awards." In
general, we are pleased with the results of the audit. The audit indicates that the steps taken
since the last audit of project reporting in 2004 are working to ensure that project reports are
submitted in accordance with NSF policies. In terms of the recommendations set forth in the
report, the following is the National Science Foundation's (NSF) response.

Recommendation 1:

OIG recommends that the NSF Deputy Director take appropriate action to ensure that all
performance reports are submitted on time to NSF. Such actions could include:

A)     Periodically providing to program orlicers, or instructing them to obtain, a listing of the
awards they manage which have annual or final project reports that are at least 90 days late.
Program officers should then contact the Pis responsible for these outstanding reports; and

B)    Providing additional training and guidance to program officers and administrative staff
on how to use NS F systems to identify overdue project reports and monitor their status.

NSF R esponse:

       Recommendation l.A) We agree with the recommendation. By October 1, 2013, the
       Systems Office, Division ofinstitution and Award Support (DIAS) will ensure
       implementation of an "over-aged" report for use by NSF Program Officers that will list
       the awards with annual and/or final project reports that are at least 90-days overdue has
       occurred. Between now and the October implementation date, we will determine the
       overall design, distribution mechanism, audience, and frequency.

       Recommendation I.B) We agree with the recommendation. To ensure that Program
       Officers are knowledgeable of the tools available for them to track project reports and
       their submission, the Policy Office, DIAS, will review and make any appropriate updates
       to the Proposal and Award Manual (PAM) scheduled for release on October I, 2013. The
       Policy Office also will explore what in-reach and outreach activities may be used to
       facilitate this educational process.

Recommendation 2:

We recommend that the NSF Deputy Director:

Take appropriate action to improve procedures to prevent assigning a second identification code
for Pis that already have an NSF identification code. Such actions could include:

A)    Adding a FastLane warning prompt for any first and last name matches before assigning
any new PI identification code; and




                                                    10
B)      Updating the instructions on FastLane to include a prominent notice and explanation tor
the institution and PI that a new PI identification number should not be established if the PI has
previously applied or received funding from NSF.

NSF Response:

       Recommendation 2.A) NSF Management agrees with the intent of the recommendation
       and will explore alternatives to addressing potential duplicate PI identification codes.
       Currently, NSF is migrating Progress Reporting functions from the legacy FastLane
       system to Research.gov. Rather than investing in adding this additional capability in
       FastLane, NSF will explore how to address this recommendation in the Research.gov
       platform.

       Recommendation 2.B) NSF Management agrees to review the instructions provided in
       FastLane for registering Pb. Consistent with our response to 2.A above, depending on
       the complexity and resources needed to modify the current instmctions in fastLane , we
       may decide to make necessary changes in Research.gov.




                                                  2




                                                      11
Appendix B: Objective, Scope, and Methodology

The objective of this performance audit was to determine if NSF has implemented
effective controls over grantee project reporting. Our scope included all awards for
which an annual and/or final project report was due or overdue between October 1,
2010 and March 31, 2012. Our audit did not include a review of the new reporting
requirement, the Project Outcomes Report, which became effective for new awards
made or existing awards that receive funding increments or supplements on or after
January 4, 2010.

To identify all of the awards in our scope period and to determine whether annual and
final project reports were received timely, we used data analysis software to obtain NSF
final and annual project report data. We downloaded data from NSF’s systems,
including the Report Server database and Project Reporting system, on April 2, 2012,
for all standard grants, continuing grants, and cooperative agreements with final and
annual reports due or overdue within our scope period. Using data analysis software,
we determined which final and annual reports were either late or not received and
analyzed the data by directorate and division. We also identified any awards with
reporting requirements that NSF waived to determine if NSF properly documented its
waiver decision. In addition, we evaluated whether NSF implemented and followed its
policy that PIs and co-PIs with outstanding annual reports or final reports could not
receive new funding for subsequent awards, including supplements to an existing
award. To perform this test, we used data analysis software to match PIs and co-PIs
with overdue reports to any new awards or supplements they received while their report
was apparently overdue. We then reviewed grant documents and correspondence in
NSF’s Ejacket records system and discussed our results with NSF personnel to
determine if there were any reasons why the new award or supplement was made.

Furthermore, we interviewed a judgmental sample of 8 NSF program officers to discuss
their responsibilities and practices for obtaining and reviewing project reports. While the
sample mostly included program officers that had project reports that were overdue, we
also interviewed three program officers who had no overdue reports in order to identify
practices that they may have used to ensure that project reports for their portfolio of
awards were submitted on time. In addition, we interviewed two program support staff
that provided assistance to program officers in monitoring report submissions.

Through interviews with NSF staff and review of documentation, we also obtained an
understanding of the controls over the performance reporting process. We did not
identify any abuse during this audit.

Our tests to determine whether annual and final project reports were submitted on time
were tests of NSF’s compliance with the Federal regulation requiring project reports.
We did not identify any other laws and regulations pertinent to our audit.

During the course of this audit, the auditors relied on information and data received from
NSF in electronic format that had been entered into a computer system or that resulted


                                               12
from computer processing. We tested the reliability of NSF’s computer-processed data
by corroborating the results with other NSF documentation independent of the Report
Server database and Project Reporting system, such as copies of grant award letters
and correspondence for selected awards in Ejacket records, and by recomputing the
due and overdue dates for all awards in our scope period. For a random sample of
awards with final and annual report due, we also compared the dates established in the
automated tracking and reminder notification system (Project Reporting system) with
the award terms in the grant documents and correspondence. For these samples, we
also verified that the system issued all required notices to the PI and institution until the
required report was submitted and approved. Based on our assessment, we concluded
the computer-processed data was sufficiently reliable to use in meeting the audit’s
objective.

We conducted this performance audit between April 2012 and January 2013 in
accordance with generally accepted government auditing standards. Those standards
require that we plan and perform the audit to obtain sufficient, appropriate evidence to
provide a reasonable basis for our findings and conclusions based on our audit
objectives. We believe that the evidence obtained provides a reasonable basis for our
findings and conclusions based on our audit objective.

We held an exit conference with NSF management on January 23, 2013.




                                                13
Appendix C: Final and Annual Report Statistics for Period
October 1, 2010 through March 31, 2012

                                          Final Reports              Annual Reports                  Total

     Total Reports Due                         16,296                      39,210                   55,506

     Total Reports Received                    12,272                      29,701                   41,973
     on Time
     Total Reports Received
     Late7                                      3,276                       8,710                   11,986
     Percentage of Reports
     Received Late                              20%                          22%                       ---
     Total Reports Not
     Submitted 9                                 748                          799                    1,547
     Percentage of Reports
     Not Submitted                               5%                           2%                       ---


                 Final Reports - Percent of Late and Not Submitted Reports
                                       by NSF Directorate
                 Directorate               % of Late Reports     % of Reports Not
                                                                    Submitted
     Biological Sciences                         21.0%                 4.5%
     Computer and Information
     Science and Engineering                     26.7%                 4.8%
     Engineering                                 21.5%                 4.9%
     Education and Human
     Resources                                   16.6%                 4.5%
     Geosciences                                 20.6%                 4.9%
     Mathematical and Physical
     Sciences                                    17.5%                 4.0%
                                       10
     Office of Polar Programs (OPP)              24.0%                 3.5%
     Office of the Director, (excluding
     OPP)                                        17.8%                 2.9%
     Social Behavioral and Economic
     Sciences                                    15.6%                 5.5%

9
    Because March 31, 2012 was on a Saturday, these amounts exclude reports received by Monday, April 2, 2012.
10
  This Office manages basic research for the Arctic and Antarctic region. It is shown separately because of its
budget size.




                                                            14
    Annual Reports - Percent of Late and Not Submitted Reports by NSF
                                   Directorate
                                                        % of Reports Not
           Directorate                % of Late Reports    Submitted
Biological Sciences                         20.2%             1.8%
Computer and Information
Science and Engineering                     26.5%             2.9%
Engineering                                 24.0%             2.4%
Education and Human Resources               20.0%             1.5%
Geosciences                                 24.0%             1.9%
Mathematical and Physical
Sciences                                    17.8%             1.7%
Office of Polar Programs                    24.2%             1.9%
Office of the Director, (excluding
OPP)                                        31.8%             1.8%
Social Behavioral and Economic
Sciences                                    23.7%             2.2%




                                     15