National Science Foundation • Office of Inspector General 4201 Wilson Boulevard, Suite I-1135, Arlington, Virginia 22230 MEMORANDUM DATE: March 4, 2013 TO: Dr. Cora B. Marrett Deputy Director National Science Foundation FROM: Dr. Brett M. Baker Assistant Inspector General for Audit SUBJECT: Audit of Project Reporting of NSF Awards, Report No. 13-2-006 Attached please find the final report on the subject audit. The report contains two findings on the need for NSF to ensure that all project reports are submitted timely and to improve controls to prevent assigning a second NSF identifier number for principal investigators. In accordance with Office of Management and Budget Circular A-50, Audit Followup, please provide a written corrective action plan within 60 days to address the report recommendations. This corrective action plan should detail specific actions and milestone dates. We appreciate the courtesies and assistance provided by so many NSF staff during the review. If you have any questions, please contact Marie Maguire, Director of Performance Audits, at (703) 292-5009. Attachment cc: Allison Lerner Daniel Hofherr G. P. Peterson Maureen Miller Cliff Gabriel Susan Carnohan Martha Rubenstein Marie Maguire Mary Santonastasso Wendell Reid Jean Feldman Jeffrey Stitz Jeffrey Vieceli Brittany DiChello Audit of Project Reporting of NSF Awards National Science Foundation Office of Inspector General March 4, 2013 OIG Report No. 13-2-006 Results in Brief We conducted this performance audit to determine if the National Science Foundation (NSF) has implemented effective controls over grantee project reporting since our prior audit conducted in 2004 (Report No. 05-2-006, dated December 13, 2004). We reviewed all annual and/or final project reports, approximately 55,500, which were due or overdue between October 1, 2010 and March 31, 2012. Unless the awarding agency has granted an exception, the Office of Management and Budget requires that recipients of Federal funds submit reports at least annually and at the end of a project. These project reports, prepared by the individual principal investigator (PI) conducting the research, explain the progress of actual accomplishments with the goals and objectives established for the period, and provide accountability for taxpayer dollars used to fund the work. We determined that NSF has made significant improvements in ensuring that grantees submit project reports on time. We found that the percentage of final reports submitted late declined from 53 percent in our prior audit to 20 percent for our current audit, while the percentage of final reports not submitted declined from 8 percent to 5 percent. Similarly, for annual reports, the percentage not submitted also declined from 42 percent to 2 percent. 1 While the percentages of late and not submitted reports are much lower than in 2004, they represent almost 12,000 final and annual reports submitted late and over 1,500 final and annual reports not submitted. One control that NSF implemented to increase compliance with the reporting requirement is to not provide new or supplemental funding to any PI whose project report is past due. This control is not fully working as intended because there is no automated control to prevent assigning a second NSF identification number to PIs who have an NSF identification number and then transfer to another institution. While we found only two instances of a PI or co-PI with a past due report who received new funding after transferring to a new institution and receiving a NSF identification number during the period we examined, there is the potential that it could occur more frequently. NSF identified 129 PIs with more than one PI identification number in calendar year 2010, 167 in 2011, and 144 in 2012 through September. We recommend that the NSF take appropriate action to ensure that all project reports are submitted on time to NSF. Additionally, we recommend that the NSF take appropriate action to improve procedures to prevent assigning a second identification number for PIs. NSF management concurs with both of our recommendations. 1 The 2004 audit could not determine the percent of annual reports submitted late because data was not available; however, our current audit found that 22 percent of annual reports were submitted late. 1 Introduction The National Science Foundation (NSF) funds approximately 22% of all federally- supported basic research across all fields of science and engineering conducted at the nation’s colleges, universities, academic consortia, nonprofit institutions, and small business. In fiscal year 2011, NSF funded approximately $6.1 billion for 21,000 awards, including new grants and cooperative agreements as well as continuing increments and supplements on awards made in prior years. Unless the awarding agency has granted an exception, the Office of Management and Budget (OMB) Circular A-110 requires that recipients of Federal funds submit performance reports at least annually and at the end of a project. The individual principal investigator (PI) conducting the research prepares the project reports, which explain the progress of actual accomplishments with the goals and objectives established for the period and provide accountability for taxpayer dollars used to fund the work. NSF uses these reports to monitor the progress and accomplishments of the funded projects. For most awards, NSF requires that, the PI must submit an annual project report at least 90 days prior to the end of the project's current budget period. In addition, NSF requires that the PI submit the final report within 90 days after the award expiration or termination date. 2 NSF requires that the reports be submitted through the NSF’s FastLane system, an interactive real-time system used to conduct NSF business over the Internet. 3 If the PI is overdue on the annual report for a multi-year award, NSF should not award the next year's funding or any new award. Additionally, NSF should not award any new funding to the PI and any co-PIs until NSF receives and accepts the final project reports from any previous awards. In 2004, we conducted an audit of project reporting for NSF awards (Report No. 05-2- 006 dated December 13, 2004). The audit determined that 53 percent of the final project reports were submitted late and 42 percent of annual project reports were not submitted and that NSF lacked adequate controls to ensure PIs submitted timely reports. The audit also determined that NSF's existing controls to prevent new awards to PIs with overdue final project reports were ineffective. We recommended that NSF continue with its plans to develop automated tracking and reminder systems for annual and final project reports and improve the reporting mechanism in the FastLane system. We also recommended that NSF update its policies and procedures on project reporting and clarify roles and responsibilities of NSF staff for monitoring timeliness and enforcing non-receipt of reports. Finally, we recommended NSF revise the award letter to include the due dates for both annual and final project reports for that award. NSF completed 2 We did not include the Project Outcomes Report in our audit. This new report is intended for the general public and is required for new awards made or existing awards that receive funding increments or supplements on or after January 4, 2010. 3 NSF has begun transitioning the project reporting service to its Research.gov website, which modernizes FastLane’s grant management services. 2 corrective actions on all OIG recommendations in 2007 and has fully implemented its automated tracking and reminder notification system for project reporting. Results of Audit NSF has made significant improvements in ensuring that grantees submit project reports and that the reports are timely. The following chart compares our current audit results to our 2004 audit: TIMELINESS OF FINAL AND ANNUAL PROJECT REPORTS 2004 audit results Current results FINAL PROJECT REPORTS Percent of final reports 53% 20% 4 submitted late Average number of months late 5 months 1.8 months 5 Percent of final reports not 8% 5% submitted Average number of months final 26 months 5.1 months, with oldest 533 days reports not submitted were overdue 6 ANNUAL PROJECT REPORTS Percent of annual reports Not available 22%4 submitted late Average number of months late Not available 1.4 months Percent of annual reports not 42% 2% submitted Average number of months not Not available 5.5 months, with oldest 548 days submitted annual reports were overdue Our audit determined that improvements to NSF’s tracking and automated reminder system for annual and final project reports, which were fully implemented after our 2004 audit, are generally working as intended. However, some PIs have not submitted 4 We are excluding reports submitted within 2 days of the due date. 5 Outer range is not comparable with 2004 audit, which used a 5-year period as the scope. 6 Average is not directly comparable with 2004 audit, which used a 5-year period 3 overdue annual and final project reports or have submitted reports late, and NSF needs to further improve its efforts to obtain reports on a timely basis. In addition, there is a risk that a PI with a past due report who then transfers to a new institution may receive new funding, thus circumventing NSF’s established policy, because NSF needs to improve its controls to prevent assigning a second NSF identification number to a PI who transfers to another institution. More Effort Needed to Ensure All Project Reports are Submitted Timely Grantees did not submit 5 percent of final reports due and approximately 2 percent of annual reports due during the period between October 1, 2010 and March 31, 2012. While the number of reports not received is relatively small, 70 percent of the final reports not submitted and approximately 65 percent of annual reports not submitted were past due at least 90 days. In addition, of the submitted reports, 20 percent of final reports and 22 percent of annual reports were submitted at least 3 days late, with 18 percent of final reports and 14 percent of annual reports submitted at least 90 days late. Furthermore, we identified three PIs who each did not submit three annual reports and one PI with 11 reports submitted late. The following charts display the range of days and number of reports not submitted or late 7. Appendix C provides information by NSF Directorate. Range of Final and Annual Reports Not Submitted Number Not Submitted 400 331 188 293 200 85 0 56 79 0 30 15 49 93 51 0 90 3-29 Annual… 30-59 60-89 Final… 90-119 120-209 210-364 Range of Days Not Submitted >365 Final Reports Annual Reports 7 Because March 31, 2012 was on a Saturday, these amounts exclude reports received by Monday, April 2, 2012. 4 Range of Late Annual and Final Reports 4920 Number Not Submitted 6000 4000 1797 1559 2000 809 729 472 574 382 127 0 228 255 11 108 Annual… 3 - 29 15 30 - 59 60 - 89 Final… 90 - 119 120 - 209 210 - 365 Range of Days Not Submitted >365 Final Reports Annual Reports NSF award recipients are required to submit periodic progress reports, including a final progress report within 90 days after grant expiration. Project reports provide valuable insight into how well the NSF-funded project is operating and its progress on achieving the research and other goals and objectives as stated in the funded proposal and approved agreement. For example, project reports typically list senior personnel, post doctorates and students involved in research; research, education, training and outreach efforts to the research community; and articles published and conference presentations as a result of the ongoing research. NSF policies require the PI to submit the annual project reports at least 90 days prior to the end of the current budget period and the PI must submit a final project report within 90 days of the grant’s termination or expiration. NSF’s automated tracking and reminder system sends both the PI and institution a reminder notice for every annual and final project report when it becomes due and every 30 days until the report is received or becomes overdue. Once the annual or final project reports are overdue, the automated tracking and reminder system sends a biweekly overdue notice to both the PI and the institution until the report is submitted. Based on our interviews, many program officers explained that while automated alerts to the PIs were effective for receiving most reports, it should not be the sole approach. For instance, some program officers found, although not required, personal phone calls or e-mails to PIs with reports overdue for prolonged periods were more effective than the automated alerts. NSF program officers, who often have a portfolio of over 100 awards, only receive a copy of the first due and overdue notices, and do not receive subsequent system alerts or status information for reports that continue to be overdue. After these first notices, program officers can only determine that reports are delinquent if they actively search NSF systems to determine the status. Some of the program officers we interviewed were not aware that the required reports for some of their awards were overdue several months or were submitted several months late because they were not monitoring the status of reports. 5 In addition, program officers we interviewed stated that receiving a periodic reminder that includes a list of awards they manage with overdue reports would be beneficial for monitoring awards, such as a list for awards with reports overdue more than 90 days. While some program officers had administrative staff who helped them closely track the required reports for their awards, especially near the end of the fiscal year, others did not. Furthermore, according to some NSF program officers we interviewed, the NSF systems did not easily identify awards with project reports that were overdue for a prolonged period. As a result of late or unsubmitted reports, NSF cannot fully assess the extent to which grantees have met their program goals and objectives for these awards with overdue reports. Without timely annual project reports, NSF program officers may not be able to address potential problems that could impair the satisfactory performance of a funded project. In addition, if final project reports are not received timely, NSF management may not be fully informed about the results of the research it funded. Recommendation 1. We recommend that the NSF Deputy Director take appropriate action to ensure that all performance reports are submitted on time to NSF. Such actions could include: A) Periodically providing to program officers, or instructing them to obtain, a listing of the awards they manage which have annual or final project reports that are at least 90 days late. Program officers should then contact the PIs responsible for these outstanding reports; and B) Providing additional training and guidance to program officers and administrative staff on how to use NSF systems to identify overdue project reports and monitor their status. More Control Needed in Assigning NSF PI Identification Numbers During our tests of compliance with NSF’s policy to not award new funding to PIs or co- PIs with past due reports, we found one PI and one co-PI who should not have received a new award. Both of these PIs transferred to other institutions and the receiving institutions requested and received an NSF PI identification number. The PIs, under the new PI numbers, then received new awards, even though they each had a past due 6 report for an existing award obtained while at the prior institution and with a different PI number. In order to protect the confidentiality of PI information, NSF restricts access to most FastLane features to authorized users. NSF’s PI Identification number is a unique numerical identifier and is used along with last name and password, to verify the identity of an authorized user accessing the FastLane system. In addition, the PI identification number serves as the identifier for other internal agency system processes, such as the check to determine if the PI is associated with any past due reports before releasing funds for a new award, supplement, or increment. There is currently no automated control to prevent assigning a second NSF identification number to PIs with an NSF identification number who then transfer to another institution. Previously, NSF matched PI application data with the social security number (SSN), but this practice was discontinued several years ago when SSNs were removed from FastLane and NSF did not implement an alternative control. NSF staff informed us that the system will check PI first name, last name, and email address combination for the institution. If any record is found with the same combination, it will not allow the creation a new record. In addition, the system will check for an existing PI identification number in the database if a user provides an identification number during user account creation. In such a case, NSF will advise the user that the PI is already in the database and hence information will not be updated. However, NSF stated that if the PI identification number is not associated with the same institution, the NSF system will allow a second identification number to be assigned to a PI. Per NSF staff, NSF usually is informed of the two PI identification numbers by the PI or NSF program officer, who request that all PI awards be consolidated under one number. The PI and co-PI with more than one NSF identification number that we identified were able to receive additional funding of approximately $1.36 million despite both PIs each having one past due report for an existing award. Once NSF was informed that the PIs had two identification numbers, NSF eliminated one of the numbers, but this action occurred after the new funding was made available to the PIs. While we found only two instances of this occurring, there is the potential that it could occur more frequently. NSF identified 129 PIs with more than one PI identification number in calendar year 2010, 167 in 2011, and 144 in 2012 through September. 8 8 Other than the PI and co-PI that we identified, we did not perform any additional tests to determine if any other PIs with a second identification number obtained new funding while having past due reports. 7 Recommendation We recommend that the NSF Deputy Director: 2. Take appropriate action to improve procedures to prevent assigning a second identification code for PIs that already have an NSF identification code. Such actions could include: A) Adding a FastLane warning prompt for any first and last name matches before assigning any new PI identification code; and B) Updating the instructions on FastLane to include a prominent notice and explanation for the institution and PI that a new PI identification number should not be established if the PI has previously applied or received funding from NSF. Summary of Agency Response and OIG Comments NSF management concurs with our first recommendation. NSF will develop a report for program officers that will list awards with annual and/or final project reports overdue at least 90 days. NSF will also review and update existing guidance and explore training opportunities. NSF agrees with the intent of our second recommendation, but will explore alternatives to addressing potential duplicate PI identification codes and agreed to review the instructions provided in FastLane for registering PIs. NSF is migrating its project reporting function from the FastLane system to Research.gov, and therefore plans to implement corrective actions in the new system. We have included NSF's response to this report in its entirety as Appendix A. We consider management’s comments and planned actions to be responsive to our recommendations. OIG Contact and Staff Acknowledgements Marie Maguire – Director of Performance Audits (703) 292-5009 or firstname.lastname@example.org In addition to Ms. Maguire, Wendell Reid, Brittany DiChello, and Jeffrey Stitz made key contributions to this report. 8 Appendix A: Agency Response 9 Thank you very much for providing us the opportunity to review and comment on the Office of Inspector General's (OIG) draft Audit Report, "Audit of Project Reporting of NSF Awards." In general, we are pleased with the results of the audit. The audit indicates that the steps taken since the last audit of project reporting in 2004 are working to ensure that project reports are submitted in accordance with NSF policies. In terms of the recommendations set forth in the report, the following is the National Science Foundation's (NSF) response. Recommendation 1: OIG recommends that the NSF Deputy Director take appropriate action to ensure that all performance reports are submitted on time to NSF. Such actions could include: A) Periodically providing to program orlicers, or instructing them to obtain, a listing of the awards they manage which have annual or final project reports that are at least 90 days late. Program officers should then contact the Pis responsible for these outstanding reports; and B) Providing additional training and guidance to program officers and administrative staff on how to use NS F systems to identify overdue project reports and monitor their status. NSF R esponse: Recommendation l.A) We agree with the recommendation. By October 1, 2013, the Systems Office, Division ofinstitution and Award Support (DIAS) will ensure implementation of an "over-aged" report for use by NSF Program Officers that will list the awards with annual and/or final project reports that are at least 90-days overdue has occurred. Between now and the October implementation date, we will determine the overall design, distribution mechanism, audience, and frequency. Recommendation I.B) We agree with the recommendation. To ensure that Program Officers are knowledgeable of the tools available for them to track project reports and their submission, the Policy Office, DIAS, will review and make any appropriate updates to the Proposal and Award Manual (PAM) scheduled for release on October I, 2013. The Policy Office also will explore what in-reach and outreach activities may be used to facilitate this educational process. Recommendation 2: We recommend that the NSF Deputy Director: Take appropriate action to improve procedures to prevent assigning a second identification code for Pis that already have an NSF identification code. Such actions could include: A) Adding a FastLane warning prompt for any first and last name matches before assigning any new PI identification code; and 10 B) Updating the instructions on FastLane to include a prominent notice and explanation tor the institution and PI that a new PI identification number should not be established if the PI has previously applied or received funding from NSF. NSF Response: Recommendation 2.A) NSF Management agrees with the intent of the recommendation and will explore alternatives to addressing potential duplicate PI identification codes. Currently, NSF is migrating Progress Reporting functions from the legacy FastLane system to Research.gov. Rather than investing in adding this additional capability in FastLane, NSF will explore how to address this recommendation in the Research.gov platform. Recommendation 2.B) NSF Management agrees to review the instructions provided in FastLane for registering Pb. Consistent with our response to 2.A above, depending on the complexity and resources needed to modify the current instmctions in fastLane , we may decide to make necessary changes in Research.gov. 2 11 Appendix B: Objective, Scope, and Methodology The objective of this performance audit was to determine if NSF has implemented effective controls over grantee project reporting. Our scope included all awards for which an annual and/or final project report was due or overdue between October 1, 2010 and March 31, 2012. Our audit did not include a review of the new reporting requirement, the Project Outcomes Report, which became effective for new awards made or existing awards that receive funding increments or supplements on or after January 4, 2010. To identify all of the awards in our scope period and to determine whether annual and final project reports were received timely, we used data analysis software to obtain NSF final and annual project report data. We downloaded data from NSF’s systems, including the Report Server database and Project Reporting system, on April 2, 2012, for all standard grants, continuing grants, and cooperative agreements with final and annual reports due or overdue within our scope period. Using data analysis software, we determined which final and annual reports were either late or not received and analyzed the data by directorate and division. We also identified any awards with reporting requirements that NSF waived to determine if NSF properly documented its waiver decision. In addition, we evaluated whether NSF implemented and followed its policy that PIs and co-PIs with outstanding annual reports or final reports could not receive new funding for subsequent awards, including supplements to an existing award. To perform this test, we used data analysis software to match PIs and co-PIs with overdue reports to any new awards or supplements they received while their report was apparently overdue. We then reviewed grant documents and correspondence in NSF’s Ejacket records system and discussed our results with NSF personnel to determine if there were any reasons why the new award or supplement was made. Furthermore, we interviewed a judgmental sample of 8 NSF program officers to discuss their responsibilities and practices for obtaining and reviewing project reports. While the sample mostly included program officers that had project reports that were overdue, we also interviewed three program officers who had no overdue reports in order to identify practices that they may have used to ensure that project reports for their portfolio of awards were submitted on time. In addition, we interviewed two program support staff that provided assistance to program officers in monitoring report submissions. Through interviews with NSF staff and review of documentation, we also obtained an understanding of the controls over the performance reporting process. We did not identify any abuse during this audit. Our tests to determine whether annual and final project reports were submitted on time were tests of NSF’s compliance with the Federal regulation requiring project reports. We did not identify any other laws and regulations pertinent to our audit. During the course of this audit, the auditors relied on information and data received from NSF in electronic format that had been entered into a computer system or that resulted 12 from computer processing. We tested the reliability of NSF’s computer-processed data by corroborating the results with other NSF documentation independent of the Report Server database and Project Reporting system, such as copies of grant award letters and correspondence for selected awards in Ejacket records, and by recomputing the due and overdue dates for all awards in our scope period. For a random sample of awards with final and annual report due, we also compared the dates established in the automated tracking and reminder notification system (Project Reporting system) with the award terms in the grant documents and correspondence. For these samples, we also verified that the system issued all required notices to the PI and institution until the required report was submitted and approved. Based on our assessment, we concluded the computer-processed data was sufficiently reliable to use in meeting the audit’s objective. We conducted this performance audit between April 2012 and January 2013 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objective. We held an exit conference with NSF management on January 23, 2013. 13 Appendix C: Final and Annual Report Statistics for Period October 1, 2010 through March 31, 2012 Final Reports Annual Reports Total Total Reports Due 16,296 39,210 55,506 Total Reports Received 12,272 29,701 41,973 on Time Total Reports Received Late7 3,276 8,710 11,986 Percentage of Reports Received Late 20% 22% --- Total Reports Not Submitted 9 748 799 1,547 Percentage of Reports Not Submitted 5% 2% --- Final Reports - Percent of Late and Not Submitted Reports by NSF Directorate Directorate % of Late Reports % of Reports Not Submitted Biological Sciences 21.0% 4.5% Computer and Information Science and Engineering 26.7% 4.8% Engineering 21.5% 4.9% Education and Human Resources 16.6% 4.5% Geosciences 20.6% 4.9% Mathematical and Physical Sciences 17.5% 4.0% 10 Office of Polar Programs (OPP) 24.0% 3.5% Office of the Director, (excluding OPP) 17.8% 2.9% Social Behavioral and Economic Sciences 15.6% 5.5% 9 Because March 31, 2012 was on a Saturday, these amounts exclude reports received by Monday, April 2, 2012. 10 This Office manages basic research for the Arctic and Antarctic region. It is shown separately because of its budget size. 14 Annual Reports - Percent of Late and Not Submitted Reports by NSF Directorate % of Reports Not Directorate % of Late Reports Submitted Biological Sciences 20.2% 1.8% Computer and Information Science and Engineering 26.5% 2.9% Engineering 24.0% 2.4% Education and Human Resources 20.0% 1.5% Geosciences 24.0% 1.9% Mathematical and Physical Sciences 17.8% 1.7% Office of Polar Programs 24.2% 1.9% Office of the Director, (excluding OPP) 31.8% 1.8% Social Behavioral and Economic Sciences 23.7% 2.2% 15
Project Reporting of NSF Awards
Published by the National Science Foundation, Office of Inspector General on 2013-03-04.
Below is a raw (and likely hideous) rendition of the original report. (PDF)