UNITED STATES DEPARTMENT OF EDUCATION OFFICE OF INSPECTOR GENERAL The Inspector General September 13, 2010 The Honorable Earl E. Devaney, Chairman Recovery Accountability and Transparency Board 1717 Pennsylvania Avenue N.W., Suite 700 Washington, D.C. 20006 Dear Chairman Devaney: I am pleased to provide the enclosed final report entitled, Recovery Act Data Quality: Recipient Efforts to Report Reliable and Transparent Information. The report summarizes the results of a recent review of selected American Recovery and Reinvestment Act (Recovery Act) recipients’ processes for compiling and reporting selected data under the Act. Specifically, the objective of the review was to determine whether the recipients’ processes provided reasonable assurance of compliance with reporting requirements contained in Section 1512 of the Recovery Act. The review focused on the Section 1512 reporting period ended December 31, 2009, and covered five data elements—number of jobs, total amount of Recovery Act funds received or invoiced, total amount of Recovery Act funds spent, project status, and final report. The Inspectors General of the Department of Education, Department of Health and Human Services, Department of Homeland Security, Department of Labor, and National Science Foundation participated in the review. I want to thank the staff of the participating Offices of Inspector General for their hard work and cooperation throughout this review. I also want to thank the staff of the Recovery Accountability and Transparency Board for the assistance they provided to the audit team. If you, other members of the Board, or your staff have any questions about this report please contact me at (202) 245-6900, or Keith West, Assistant Inspector General for Audit, at (202) 245-7041. Sincerely, Kathleen S. Tighe Inspector General The Department of Education's mission is to promote student achievement and preparation for global competitiveness by fostering educational excellence and ensuring equal access. Recovery Act Data Quality: Recipient Efforts to Report Reliable and Transparent Information FINAL REPORT September 13, 2010 Prepared by: U.S. Department of Education Office of Inspector General (This page intentionally left blank) TABLE OF CONTENTS Page EXECUTIVE SUMMARY ................................................................ 1 PURPOSE OF THE REVIEW .......................................................... 6 BACKGROUND.............................................................................. 8 AUDIT RESULTS .......................................................................... 11 SECTION 1: Recipient Processes for Compiling and Reporting Recovery Act Data .................. 11 SECTION 2: Federal Guidance Applicable to Section 1512 Reporting ................................. 22 SECTION 3: Optimizing the Transparency of Reported Information .................................. 30 OBJECTIVE, SCOPE, AND METHODOLOGY................................ 36 APPENDIX 1 ................................................................................ 38 APPENDIX 2................................................................................ 40 APPENDIX 3................................................................................ 43 Final Report Page 1 of 44 EXECUTIVE SUMMARY The American Recovery and Reinvestment Act of 2009 1 (Recovery Act) was enacted in February 2009 to help the nation recover from the most severe economic downturn experienced since the Great Depression. The Recovery Act emphasizes unprecedented levels of accountability and transparency over the $787 billion in public funds committed by the Congress. Under the Recovery Act, Federal agencies primarily award funds through grants and contracts to grant recipients and Federal contractors (hereinafter referred to as recipients) who, in turn, can make separate awards (subawards) to subrecipients and subcontractors to assist them in implementing approved Recovery Act programs and projects. Recipient Reporting. To achieve the accountability and transparency provisions of the Recovery Act, recipients are required to submit quarterly reports containing detailed information on the projects and activities funded. Section 1512 of the Recovery Act established quarterly reporting as the primary means of providing transparency to the public, i.e., to ensure that the public is informed about the way funds are used and the outcomes achieved as a result of Recovery Act spending. At the end of a continuous correction period, during which time reported data are reviewed and corrected as needed, reports are made available on Recovery.gov. This public Web site was established to provide the public with Recovery Act information that is transparent and easy to access. Federal Reporting Guidance. Reporting instructions are contained in Office of Management and Budget (OMB) guidance and the Federal Acquisition Regulation (FAR). OMB is the primary Federal agency responsible for providing Section 1512 reporting guidance used by Federal agencies, grant recipients, and grant subrecipients. Similar reporting guidance was established for Federal contractors as an interim rule to the FAR. The Federal guidance clearly establishes that recipients have primary responsibility for the quality of data that are submitted. However, Federal agencies play an important oversight role in ensuring data quality and are required to review data reported by recipients and notify them of errors requiring correction. What We Did. This report summarizes the results of our recent review of selected recipients’ efforts in compiling and reporting selected data under Section 1512 of the Recovery Act and applicable Federal guidance. We undertook the review at the request of the Recovery Accountability and Transparency Board (Board). The Inspectors General of the Department of Education (lead), Department of Health and Human Services, Department of Homeland Security, Department of Labor, and National Science Foundation participated in the review. We performed our work in accordance with generally accepted government auditing standards. Our review focused on 1 Pub. Law No. 111-5, 123 Stat. 115 (Feb. 17, 2009). Final Report Page 2 of 44 Section 1512 reporting for the quarter ended December 31, 2009, and covered five data elements—number of jobs, total amount of funds received or invoiced, total amount of funds spent, project status, and final report. We judgmentally selected 20 grant recipients and 9 Federal contractors (29 recipients), along with 14 grant subrecipients, for our review based on factors such as the amount of Recovery Act funds awarded, an analysis of Section 1512 data they reported, and prior audit experience. These recipients’ Recovery Act funding ranged from hundreds of thousands of dollars to several billion dollars. Some recipients we reviewed did not subaward Recovery Act funds to any subrecipients whereas one recipient subawarded funds to more than a thousand subrecipients. Limitations of the Review. The recipients included in our review were not selected using statistical sampling methods. Therefore, none of the results presented in this report can be generalized across the universe of more than 64,000 recipients that submitted a report for the period covered by our review, including the extent that reporting errors occurred or whether over- reporting or under-reporting of jobs or funds spent was more common. In addition, recipient reporting of the number of jobs relied on estimates in many cases. Therefore, information on the extent of the errors presented in this report reflects only estimates of the impact of reporting errors for those specific recipients. Effectiveness of Recipient Reporting. The 29 recipients generally reported consistent and reliable information for 4 of the 5 data elements covered by our review. However, reporting the number of jobs created or retained was problematic, with only 7 of the 29 recipients reporting this data element consistent with applicable Federal guidance. For this review, a recipient reported consistent with Federal guidance if it followed the applicable guidance and we did not identify other data quality issues, such as incorrect or omitted data. A few recipients also experienced data quality issues related to the funds received/invoiced, funds spent, and project status data elements. The data quality issues identified through our review stemmed from the following factors: • Inability to implement Federal guidance as specified, • Use of alternative processes to those specified in Federal guidance, • Misinterpretation of Federal guidance, • Inconsistent implementation of Federal guidance, • Inadequate time available to effectively implement amended Federal guidance, • Non-existent or ineffective recipient internal controls and/or oversight reviews to detect and/or prevent posting and calculation errors, and • Not correcting known errors during the correction period. We were only able to reasonably estimate an impact on the number of jobs reported for 18 of the 22 recipients experiencing data quality issues, with Final Report Page 3 of 44 12 over-reporting, 5 under-reporting, and 1 neither over-reporting or under- reporting the number of jobs even though it did not follow Federal guidance. Over-reporting ranged from 1 job to 3,200 jobs. In contrast, under-reporting ranged from 1 job to about 1,950 jobs. One recipient did not follow Federal guidance to estimate the number of jobs but still reported jobs information that appeared reasonable. Of the 22 recipients experiencing data quality issues, 6 recipients reported estimated jobs following applicable Federal guidance but still experienced other data quality issues. Our review did not identify any instances of recipients or subrecipients intentionally misreporting Recovery Act information. Of the recipients that we concluded did not follow OMB guidance or the FAR, four were contractors that included subcontractor jobs contrary to the FAR guidance in effect for the reporting period ended December 31, 2009. If these contracts had been awarded after the issuance of FAR Case 2010-008 on July 2, 2010, which requires contractors to report subcontractor jobs, we would not have identified this as a data quality issue. Also, three of the recipients that we concluded did not follow OMB or FAR guidance used alternative methodologies to calculate jobs that we concluded would produce reasonable estimates if implemented properly. Data Quality Processes, Controls, and Oversight. The development of effective processes, internal controls, and oversight functions were key elements for ensuring data quality for the five data elements we reviewed. First, it was important for recipients to have a clear understanding of the Section 1512 reporting requirements, develop policies and procedures for complying with the requirements, and implement effective data review processes in order to prevent, or at least detect, errors or omissions that could have an adverse impact on data quality. In addition, oversight by awarding Federal agencies, state agencies, and other entities was a critical element in ensuring data quality. Secondary reviews of Recovery Act data performed by these entities, prior to submission and during the continuous correction period, helped to overcome deficiencies in recipient processes. Even the recipients themselves, responsible for ensuring the quality of subrecipient data that constituted a substantial portion of the information reported by many recipients, played an oversight role. We found that several recipients and some state oversight agencies strengthened internal controls and improved data quality review processes for reporting in future quarters. Additionally, recipients shared several important lessons learned that could help other recipients improve their reporting processes and reporting effectiveness. Enhancement and Clarification of Guidance. Insights from our review of 29 recipients, coupled with our independent assessment of existing Federal guidance, identified several areas in which Federal guidance could be enhanced or clarified to improve data quality for selected data elements. • The Federal guidance on estimating the number of jobs created or retained needs to be enhanced to provide more flexibility to Final Report Page 4 of 44 accommodate the use of alternative approaches to estimating job numbers. Even though some recipients did not follow the process specified in Federal guidance, our work showed that they developed alternative processes that would produce reasonable estimates for the number of jobs if implemented properly. • OMB guidance on whether to report jobs estimates for lower-tier subrecipients and small vendors needs clarification to avoid confusion and inconsistent reporting. • OMB guidance regarding correction of errors in prior-quarter jobs estimates may need clarification. • OMB guidance related to the amount of Recovery Act funds spent may need clarification in several areas, including guidance on third party in-kind contributions, reporting on the accrual basis, and what to include in the Total Federal Share of Expenditures. • OMB guidance relating to the Final Report data element needs additional clarification because it could lead to confusion and/or require recipients to continue reporting even after all grant funds are expended and the project is complete. Transparency in Reporting. Our review also identified several areas where the current reporting process may not result in optimal transparency for users of Recovery.gov. First, recipient reporting of funds spent in cases where funds are advanced to subrecipients may result in more Recovery Act funds appearing to be invested in the economy than actually have been. Second, despite the adoption of an interim rule requiring contractors to report the estimated number of jobs resulting from subawards to subcontractors beginning in July 2010, transparency on estimated jobs will continue to be less than optimal for awards issued prior to the interim rule because the rule is not retroactive. Lastly, the reporting of subrecipient jobs by the recipient may not accurately portray actual employment impacts by individual Congressional districts on Recovery.gov when recipients and subrecipients are located in different districts. Recommendations. We provide recommendations in several areas that would further enhance the quality of data being reported to the public and improve transparency over reported information. Specifically, we recommend that the Board work with OMB, the Federal Acquisition Regulatory Council, and Federal agencies, as warranted, to do the following: • provide more comprehensive technical assistance to recipients and subrecipients on effective processes and controls for jobs data reporting; • advise recipients and subrecipients to incorporate more analytical procedures into their data quality systems; Final Report Page 5 of 44 • issue reporting guidance more timely in relation to the established reporting deadlines; and • implement changes to FederalReporting.gov that would make it easier for recipients to report more effectively and efficiently. We also recommend that the Board consider conducting a comprehensive review of recipient reported information on the number of jobs using statistical sampling methods in order to assess the reliability of reported jobs data for all reporting entities. In addition, we recommend that the Board work with OMB and the Federal Acquisition Regulatory Council, as appropriate, to identify alternative methods for calculating the number of jobs created or retained; clarify guidance on subrecipient reporting of jobs data at the vendor and lower-tier subrecipient levels; and explore opportunities that would increase transparency of reported information, and thus more accurately reflect the amount of Recovery Act funds invested in the economy. Adopting these recommendations will help all parties that are involved in collecting, compiling, reviewing, and reporting Section 1512 data better understand their roles and the appropriate steps they should follow to ensure data quality. Adopting these recommendations will also help ensure that recipients have clear and unambiguous guidance on reporting related to several crucial data elements, which should lead to more reliable and transparent reports on Recovery.gov. Final Report Page 6 of 44 PURPOSE OF THE REVIEW Using its authority under Section 1521 of the Recovery Act, the Board has been working with various Inspectors General to undertake a multi-phased approach to assess ways to improve data quality. The Phase I reviews provided a snapshot of agencies’ data review processes before the start of the first reporting period in October 2009. The Phase II reviews, conducted after the first reporting period, assessed the data-review processes at seven Federal agencies. Phase III, which we are in now, focuses on the controls of Federal agencies and recipients to ensure data reliability and transparency. As part of Phase III, the Inspector General for the Department of Agriculture recently issued a report summarizing the work of six Inspectors General to determine whether their respective agencies’ internal controls were sufficient to ensure that recipient data were accurate, complete, timely, and free of significant errors or material omissions. 2 Our report is the second Phase III review. The Inspectors General of the Department of Education, Department of Health and Human Services, Department of Homeland Security, Department of Labor, and National Science Foundation participated in the review. This report summarizes the results of our review of 29 Recovery Act recipients’ efforts to report selected data under the Recovery Act. 3 In this report, the term “recipient” refers to entities receiving funds directly from an awarding Federal agency through a grant or contract. The purpose of the review was to determine whether the recipients’ processes and controls provided reasonable assurance of compliance with Section 1512 reporting requirements and applicable Federal guidance. Our intent was to provide insight into reporting at the recipient and subrecipient levels, highlight challenges and lessons learned, and make recommendations to enhance the reporting process. Our review focused on the Section 1512 reporting period ended December 31, 2009, and covered five data elements—number of jobs, total amount of funds received or invoiced, total amount of funds spent, project status, and final report. More detailed information on the 29 recipients and the Recovery Act programs we reviewed is presented in Appendix 1. The results presented in this report should not be generalized or applied across the universe of recipients reporting to FederalReporting.gov, including the extent that reporting errors occurred or whether over-reporting or under- reporting of jobs or funds spent was more common. The 29 recipients we did 2Prior data quality reports issued under the multi-phased approach are: Summary of Inspectors General Reports on Federal Agencies’ Data-Quality Review Processes, November 2009; Recovery Act Data Quality: Errors in Recipients' Reports Obscure Transparency, February 2010; and American Recovery and Reinvestment Act–Review of the Effectiveness of Department/Agency Data Quality Review Processes, June 2010. 3 Federal guidance clearly establishes that recipients have primary responsibility for the quality of data that are submitted under the Recovery Act. Final Report Page 7 of 44 review comprise a very small number of the more than 64,000 recipients 4 that reported Section 1512 data for the reporting period ended December 31, 2009. In addition, although the 29 recipients constitute a cross-section of grantees and contractors that received Recovery Act funding, ranging from hundreds of thousands of dollars to several billion dollars, the recipients were not selected using statistical methods. Further, some of the recipients included in our review had large numbers of subrecipients, while our results only pertain to information obtained from a small number of subrecipients. 4The universe of more than 64,000 recipients included more than 53,000 grant recipients and more than 11,000 Federal contractors. Information about more than 95,000 subrecipients was also reported. Final Report Page 8 of 44 BACKGROUND Congress enacted the Recovery Act in February 2009 in response to a national economic crisis considered the most serious downturn since the Great Depression. The Recovery Act places a heavy emphasis on accountability and transparency. In order to provide transparency over how funds are used, funding recipients must submit quarterly reports containing detailed information on the projects and activities funded by the Recovery Act. Requiring that recipients regularly report on how they are using Recovery Act funds, and the outcomes that result, also contributes to increased accountability over the funds. The Recovery Act provided $787 billion in Federal funds to stimulate the U.S. economy. Federal agencies awarded the funds to states, localities, non-profit and for-profit entities, contractors, and individuals to promote economic recovery, preserve and create jobs, minimize or avoid reductions in essential state and local government services, and invest in long-term economic growth. The Quarterly Reporting Timeline Section 1512 of the Recovery Act requires recipients of Recovery Act funds to report on various data elements, such as the type, date, and amount of award; project description and status; the number of jobs created or retained; and the amount of Recovery Act funds received and spent. No later than 10 days after the end of each calendar quarter, recipients must submit Recovery Act data to FederalReporting.gov, the nationwide data collection system deployed by the Board, in order to fulfill their Section 1512 reporting obligations. A continuous correction period takes place between this initial data submission and the end of the next quarter. During this period, Federal agencies are required to review the data reported by recipients and notify them of data errors that need to be corrected. Recipients should also review their submitted data and make necessary corrections. At the end of the continuous correction period, recipient data are made available to the public on the Recovery.gov Web site. The FederalReporting.gov Web site works in conjunction with the Recovery.gov Web site to provide a comprehensive solution for recipient reporting and Recovery data transparency. Recovery.gov was specifically designed to provide the public with Recovery Act data that are transparent and easily retrievable. Summary of Reporting Requirements Reporting instructions are contained in OMB guidance and the FAR as follows: • OMB memorandum M-09-21, Implementing Guidance for the Reports on Use of Funds Pursuant to the American Recovery and Reinvestment Act of 2009, June 22, 2009 (OMB guidance M-09-21). Supplement 1 to this guidance listed the programs subject to Section 1512 recipient Final Report Page 9 of 44 reporting. Supplement 2 provided a Recipient Reporting Data Model, which is a set of specific instructions for completing the various data elements contained in FederalReporting.gov. OMB guidance M-09-21 does not apply to Federal government contracts. • OMB Memorandum M-10-08, Updated Guidance on the American Recovery and Reinvestment Act – Data Quality, Non-Reporting Recipients, and Reporting of Job Estimates, December 18, 2009 (OMB guidance M-10-08). This guidance clarified that, beginning with the second reporting period ended December 31, 2009, the number of jobs should no longer be reported cumulatively but should be reported on a quarterly basis. The guidance also presented a new methodology for calculating the number of jobs created or retained and a standard methodology for Federal agencies to review the quality of recipients’ reported data. • OMB Memorandum M-10-14, Updated Guidance on the American Recovery and Reinvestment Act, March 22, 2010 (OMB guidance M-10-14). This guidance again clarified that the only data element requiring that data be reported on a quarterly basis (i.e., not cumulative) is the number of jobs. The amount reported for other data elements, such as Recovery Act funds received or invoiced, should be reported on a cumulative basis. The updated guidance also instructed recipients on when they should classify the Section 1512 report as “final.” • OMB also issued and has updated Frequently Asked Questions (FAQ) about the Recovery Act in general and about Federal contractors’ reporting requirements. Recovery Act reporting requirements for Federal contractors are contained in FAR case 2009-009, issued March 31, 2009. This interim rule implemented the public reporting requirements for Federal contractors under Section 1512 of Division A of the Recovery Act. Subpart 4.15 of the FAR, American Recovery and Reinvestment Act – Reporting Requirements, and FAR clause 52.204-11, contain the specific reporting requirements that Federal contractors must follow when reporting information to FederalReporting.gov. The principal difference in the reporting requirements between Federal grant recipients and contractors is described below: • Grant recipients spend Recovery Act funds to carry out a Federal project or program by employing in-house staff and resources, purchasing goods and services from vendors, 5 and awarding contracts to other entities. They can also subaward Recovery Act funds to other entities to support or carry out the program. Grant recipients must report data on 5A vendor is a dealer, distributor, merchant, or other seller providing goods or services that are ancillary to the operation of a Federal program. Final Report Page 10 of 44 payments made to both subrecipients and vendors, and may delegate certain reporting requirements to subrecipients, such as vendor information. • Federal contractors spend Recovery Act funds themselves, purchase goods and services from vendors, and subcontract with other entities. Like grant recipients, Federal contractors must report on payments made to subcontractors. However, Federal contractors were not required to report Recovery Act funds spent 6, vendor information, or the number of jobs created or retained by subcontractors. 7 Data Elements Covered by the Review From almost 100 data elements that are contained in the Recipient Reporting Data Model, we selected 5 specific Section 1512 data elements that were either deemed critical to ensuring transparency or that were considered more at risk of being reported inconsistently or inaccurately. Our review focused on the following data elements: • Number of Jobs. Recipients were to estimate the number of jobs created or retained by Recovery Act funds for the quarter. Grant recipients were to also include the number of jobs created or retained by subrecipients and vendors. • Total Federal Amount of Recovery Act Funds Received or Invoiced (Funds Received/Invoiced). Grant recipients were to report the total cumulative amount of Recovery Act funds received from the Federal agency. Federal contractors reported the total cumulative amount of funds invoiced to the Federal agency for payment. • Total Federal Amount of Recovery Act Expenditures (Funds Spent). Grant recipients were to report the total cumulative amount of Recovery Act funds that were spent on projects or activities. • Project Status. Recipients were to report the completion status of the project, activity, or federally awarded contract funded by the Recovery Act by selecting from among four options—not started, less than 50 percent completed, 50 percent or more completed, or fully completed. • Final Report. Recipients indicated whether this was their final report—that is, no further Section 1512 reports would be submitted for the grant or contract award. 6 Contractors were instead required to report the amount of Recovery Act funds invoiced. 7 On July 2, 2010, FAR Case 2010-008 revised the clause at FAR 52.204-11 to require subcontractors with Recovery Act awards of $25,000 or more to report information on jobs created or retained to the Federal contractor. The revised clause applied to all new solicitations and awards issued on or after July 2, 2010. The revised clause was not applicable to the Federal contracts we reviewed. Final Report Page 11 of 44 AUDIT RESULTS SECTION 1: Recipient Processes for Compiling and Reporting Recovery Act Data For four of the five data elements covered by our review, we concluded that most recipients’ processes for compiling and reporting information under the Recovery Act generally provided reasonable assurance of compliance with reporting requirements contained in Section 1512, applicable OMB guidance, or the FAR. However, we identified data quality issues related to the estimated number of jobs reported by most of the 29 recipients we reviewed. 8 We concluded that only 7 of the 29 recipients reported the number of jobs consistent with applicable OMB guidance or the FAR. We also identified data quality issues for one recipient’s reported funds received/invoiced, three recipients’ reported amount of funds spent, and one recipient’s reported project status. The results of our review of the 29 recipients’ reporting efforts are summarized in Table 1. Table 1: Summary of Recovery Act Reporting Results for the Reporting Period Ended December 31, 2009 (a) Grant Recipients Federal Contractors Grant and Contract (20) (9) Recipients (29) Number Total Number Number Reporting Data Element Reporting Reporting Consistent with Consistent with Consistent with Guidance Guidance Guidance Yes No Yes No Yes No Jobs 6 14 1 8 7 22 Funds 19 1 8 (b) 0 27 1 Received/Invoiced Not Not Funds Spent 17 3 applicable applicable 17 3 (c) (c) Project Status 19 1 9 0 28 1 Final Report 20 0 9 0 29 0 Indicator (a) We determined that a recipient reported consistent with Federal guidance if it (1) followed the applicable guidance and (2) we did not identify other data quality issues for the data elements reviewed. (b) Of the nine Federal contractors reviewed, we excluded one from the Funds Received/Invoiced data element because the contractor had not submitted invoices for Recovery Act funds as of December 31, 2009. The contractor did not report an amount for this data element. (c) Federal contractors were not required to report funds spent. 8The 29 recipients we reviewed included one recipient that was both a grant recipient and a Federal contractor. For reporting purposes, we treated this recipient as two separate recipients. Final Report Page 12 of 44 We believe that the complexities involved in the reporting process itself contributed to the reporting problems. To be highly effective, the process requires seamless integration between recipients, awarding Federal agencies, and FederalReporting.gov, and when applicable, subrecipients and state oversight agencies. We found that all of these reporting levels faced challenges that impacted data quality, particularly as it relates to estimated jobs created or retained. Our review did not identify any instances of recipients or subrecipients intentionally misreporting Recovery Act information. The remainder of Section 1 of this report describes the results of our review for the 29 recipients and 14 subrecipients. Specifically, we describe (1) data quality issues we identified related to reporting the number of jobs and other data elements, (2) our observations on the effectiveness of recipient processes and controls to ensure the quality of data reported, including data provided by subrecipients, (3) results of analytical procedures performed by recipients and the Offices of Inspector General (OIGs) participating in this review, (4) specific challenges that recipients faced during the reporting process, and (5) recipients’ reporting experiences that could inform future reporting. Reporting Errors Related to the Number of Jobs We identified data quality issues related to the number of jobs reported by 22 of the 29 recipients reviewed, as presented in Table 1 above. 9 Specifically, 14 of 20 grant recipients and 8 of 9 contractors experienced data quality issues for the number of jobs reported. We concluded that these reporting errors occurred because recipients did not follow OMB guidance or the FAR for the reporting period ended December 31, 2009, or because recipients made errors when calculating or posting the number of jobs. Of the recipients that we concluded did not follow OMB guidance or the FAR, 4 were contractors that included subcontractor jobs contrary to the FAR guidance in effect for the reporting period ended December 31, 2009. If these contracts had been awarded after the issuance of FAR Case 2010-008 on July 2, 2010, which requires contractors to report subcontractor jobs, we would not have identified this as a data quality issue. Also, three of the recipients that we concluded did not follow OMB or FAR guidance used alternative methodologies to calculate jobs that we concluded would produce reasonable estimates if implemented properly. Several factors contributed to the reporting errors we identified related to recipient jobs estimates. These included the inability to effectively implement 9 OMB guidance M-10-08 states that “Section 1512 reports contain only estimates of jobs created and jobs retained.” Because the number of jobs reported by recipients reflects only estimates, the information provided in this report related to the number of jobs including any discussion of specific instances in which the number of jobs was over-reported or under- reported are likewise only estimates. Final Report Page 13 of 44 jobs reporting guidance in effect for the reporting period ended December 31, 2009 (second quarterly report); problems associated with the timing of updated jobs guidance in relation to the reporting deadline for the second quarterly report; and inadequate recipient processes and controls for collecting, compiling, and reviewing jobs data. In addition, recipient use of alternative processes to those specified in Federal guidance, misinterpretation of applicable Federal guidance, inconsistent implementation of Federal guidance, and an inability to correct known errors during the correction period also resulted in data quality issues for the number of jobs. We were only able to reasonably estimate an impact on the number of jobs reported for 18 of the 22 recipients experiencing data quality issues. 10 Based on our estimates, we determined that 12 recipients over-reported, 5 recipients under-reported, and 1 recipient used an alternative estimation process that did not result in over- or under-reporting. • For the 12 instances of over-reporting, the estimated number of jobs over-reported ranged from 1 job to 3,200 jobs. Because the Recovery Act awards varied greatly in size from one recipient to the next, describing the over-reporting on a percentage basis provides a different perspective. Over-reporting ranged from about 10 percent to more than 400 percent, 11 with most of the 12 recipients reporting at least 25 percent more jobs than they should have. The 3,200 jobs that were over-reported occurred because a subrecipient incorrectly entered the amount of funds spent ($3,200) in the number of jobs data field when reporting Recovery Act data to the recipient. This error resulted in the recipient’s report on the number of jobs being over-reported by almost 10 percent on Recovery.gov. In another instance, an over-reporting error of more than 400 percent occurred because a Federal contractor reported about 200 jobs, which, contrary to the FAR, included more than 160 subcontractor jobs. Other causes of over-reporting included undetected posting or calculation errors and recipients incorrectly reporting jobs for vacant positions or jobs supported by non-Recovery Act funds. • For the 5 instances of under-reporting, the estimated number of jobs under-reported ranged from 1 job to almost 1,950 jobs. Expressed on a percentage basis, under-reporting ranged from less than 5 percent to more than 100 percent, with most of the 5 recipients reporting at least 25 percent fewer jobs than they should have. The almost 1,950 jobs that were under-reported occurred because a recipient did not adjust its formula for calculating the number of jobs to reflect quarterly reporting. The under-reporting error of more than 100 percent 10We could not fully determine the impact of data quality issues on the number of jobs reported by four recipients because sufficient information was not available. 11The over-reporting error of more than 400 percent was the second highest error among the 12 recipients. Another error resulted in over-reporting of more than 700 percent but the number of jobs reported and in error were less than one full-time equivalent (FTE). Final Report Page 14 of 44 occurred because almost 30 of the recipient’s more than 50 jobs did not upload correctly into the statewide data collection system for submission to FederalReporting.gov. In addition to calculation and posting errors, under-reporting resulted when recipients did not properly account for and report all jobs created or retained. • For the one instance of neither over- nor under-reporting, the recipient developed an alternative method for estimating the number of jobs that did not follow Federal guidance. The alternative method appeared to produce a reasonable estimate that did not over- or under-report the number of jobs for this recipient. OMB Guidance and FAR Implementation Issues Many of the recipients and subrecipients we reviewed did not effectively implement OMB guidance or the FAR when reporting the number of jobs. Three recipients used alternative methods to calculate the number of jobs because the manner in which Recovery Act funds were used precluded the recipients from adhering to the methodology specified in Federal guidance. OMB’s issuance of guidance 2 weeks before the end of the quarter, which changed the basis for reporting jobs numbers from cumulative to quarterly, was problematic for one recipient resulting in it reporting jobs cumulatively. Seven recipients were unclear about what jobs should and should not be included in the reported number of jobs. The implementation issues we identified are presented in more detail in Appendix 2. Recipient-Caused Processing and Other Errors Of the 22 recipients with data quality issues, we found that 6 recipients 12 generally followed OMB guidance but still experienced one or more data quality issues resulting in errors in their reported number of jobs. Three recipients under-reported jobs because of processing errors and/or not fully understanding applicable guidance. Accounting adjustments related to payroll costs resulted in inaccurate reporting for two recipients. Two recipients failed to correct identified changes in the number of jobs during the continuous correction period. The recipient-caused errors are also summarized in more detail in Appendix 2. Reporting Errors Related to Funds Received/Invoiced, Funds Spent, or Project Status We also identified data quality issues for the Recovery Act funds received/invoiced data reported by one recipient, funds spent data reported 12 One recipient experienced more than one data quality issue. Final Report Page 15 of 44 by three recipients, and the project status reported by one recipient, as presented in Table 1 above. Of the 20 grant recipients required to report funds spent, 3 recipients either did not implement OMB guidance correctly or made reporting errors resulting in the reporting of inaccurate amounts. Although all 29 recipients were required to report project status, we identified only 1 recipient that did not accurately report the project status for 2 of its 80 Recovery Act grants. The reporting errors related to funds received/invoiced, funds spent, and project status are discussed in more detail in Appendix 2. Observations on Recipients’ Data Quality Controls and Processes Our review provided insights on the impact that controls and processes, or the lack thereof, had on data quality. The development of effective processes, internal controls, and oversight functions were key elements for ensuring data quality for the five data elements we reviewed. First, it was important for recipients to have a clear understanding of the Section 1512 reporting requirements, develop policies and procedures for complying with the requirements, and implement effective data review processes in order to prevent, or at least detect, errors or omissions that could have an adverse impact on data quality. In addition, oversight by awarding Federal agencies, state agencies, and other entities was also a critical element in ensuring data quality. Secondary reviews of Recovery Act data performed by these entities, prior to submission and during the continuous correction period, helped to overcome deficiencies in recipient processes. Even the recipients themselves, responsible for ensuring the quality of subrecipient data that constituted a substantial portion of the information reported by many recipients, played an oversight role. We found that several recipients and some state oversight agencies strengthened internal controls and improved data quality review processes for reporting in future quarters. Additionally, recipients shared several important lessons learned that could help other recipients improve their reporting processes and reporting effectiveness. Recipient Policies and Procedures Over half of the 29 recipients we reviewed had developed written policies and procedures for Section 1512 reporting. In most cases, recipients had policies and procedures that focused on processes to calculate and report the estimated number of jobs. Some of these recipients had policies and procedures that covered all five data elements covered by our review. We noted that not all recipients we reviewed had updated their policies and procedures to reflect the new method for calculating the estimated number of jobs specified in OMB guidance M-10-08. The remaining recipients were able to rely on OMB guidance, such as the Recipient Reporting Data Model (Supplement 2 to OMB guidance M-09-21) or the FAR, to assist them in complying with specific reporting requirements. Final Report Page 16 of 44 Recipient Controls and Processes to Ensure Data Quality Recipients implemented various data quality controls to help ensure that the information being reported to FederalReporting.gov was accurate and reliable. These controls helped prevent or detect data quality issues before data were submitted or during the continuous correction period. In contrast, data quality issues were not detected when some recipients either lacked sufficient controls or bypassed existing controls. For example, some recipients did not detect data quality issues resulting from posting and math errors that led to inaccurate reporting of the number of jobs. One recipient appeared to bypass its established review processes in order to meet the FederalReporting.gov data submission deadline. Below we discuss our insights on controls that recipients implemented for Section 1512 reporting. We also provide more detailed information and specific examples of the types of controls that recipients had implemented in Appendix 3. Risk-Based Monitoring and Management Reviews. Several recipients used a risk-based approach to monitor data collected from subrecipients or implemented supplemental data quality reviews performed by managers or other independent reviewers to help ensure the quality of reported data. For example, one recipient tasked its internal auditor with reviewing its jobs calculations and other data before forwarding it to the state reporting agency for submission to FederalReporting.gov. Automated Data Quality Reviews. Five recipients implemented automated data quality checks. An example of an automated data check would be an edit built into a recipient’s data system that compared the cumulative amount of Recovery Act funds spent to the amount of the Recovery Act award. Some recipients considered these automated checks critical to ensuring that reliable data were reported. However, most recipients used manual review processes to ensure data quality. These manual reviews varied in complexity. Simpler reviews were designed to identify blank data fields and other easily detected errors. More complex review procedures at one recipient involved a team of six staff tasked with ensuring compliance with Section 1512 reporting requirements. Seven recipients assigned personnel to manually verify the accuracy of data included in Section 1512 reports. Recipient Controls over Subrecipient and Vendor Data. OMB guidance requires recipients to report on payments made to subrecipients and vendors. 13 About one-third of recipients we reviewed had implemented data quality controls and processes to ensure that the data collected from subrecipients and vendors were reasonable. These processes and controls can be critical to data quality because subrecipient and vendor data can comprise 13Grant recipients must report subrecipient and vendor information, while Federal contractors were not required to report vendor information or subcontractor jobs, as explained in the Background section of this report. Final Report Page 17 of 44 a substantial portion of the activity covered in recipients’ reports. Furthermore, subrecipient and vendor data have a greater risk of data quality issues because additional procedures are required to compile data at the recipient level. However, the risk can be mitigated if recipients implement effective controls over the collection and compilation of subrecipient and vendor data. Analytical Procedures Performed on Recipients’ Reported Data. Analytical procedures, such as logic and reasonableness checks, can help identify potential data anomalies for further review. OMB guidance M-10-08 addressed data anomalies that Federal agencies might encounter when reviewing recipient data. The guidance provided examples of inconsistencies that could be identified by comparing data elements that are logically related. For example, a data reasonableness check could be one or more queries programmed into a recipient’s information system to identify potential errors such as missing, duplicated, or mismatched data. Four recipients we reviewed performed one or more of the analytical procedures contained in OMB guidance and several performed other analytical procedures. For example, a few recipients compared current quarter data to the prior quarter’s data to check for reasonableness. As part of our review, we performed the analytical procedures contained in the OMB guidance that applied to the data elements we reviewed. For some recipients, we supplemented the OMB examples with two additional logic checks to confirm that (1) the reported amount of funds received/invoiced did not exceed the reported amount of the award and (2) grant recipients’ reported amount of Recovery Act funds spent did not exceed the reported amount of funds received/invoiced. We also assessed the reasonableness of some recipients’ jobs data by comparing the estimated number of jobs reported to the amount of Recovery Act funds spent across multiple recipients. We discuss several data quality problems we identified by applying analytical procedures in Appendix 3. Recipient Reporting Challenges and Future Reporting Many recipients described challenges they faced in reporting under Section 1512. The most common challenges that recipients identified related to the (1) timing of OMB guidance in relation to the deadline for submitting information for the second quarterly report, (2) availability of personnel to perform the work necessary to report accurate and reliable data, and (3) use of required data templates on FederalReporting.gov. In some cases, recipients developed strategies to overcome or at least mitigate the challenges they faced. Final Report Page 18 of 44 Timing of Guidance Update in Relation to Reporting Deadline The issuance of OMB guidance M-10-08 only 2 weeks before the reporting quarter ended and 1 month before the extended reporting deadline created challenges for many recipients in our review. More than half of the recipients told us that the late issuance of the updated guidance for estimating the number of jobs reduced the timeframe for interpreting and applying the guidance, and/or that collecting, processing, and reviewing large amounts of data within the short timeframe it afforded was a challenge. For three recipients, the inability to effectively respond to the compressed timeline, shifting deadlines, and the changes in the jobs estimation methodology itself resulted in inconsistency in the methods used to calculate the number of jobs reported across recipients and/or subrecipients. As a result, the number of jobs reported by some recipients was inaccurate. Resources to Monitor and Report Six recipients told us that they experienced resource constraints that had an adverse impact on their ability to perform administrative functions associated with their Recovery Act grant(s), including those related to Section 1512 reporting. Several recipients told us that existing staff were required to assume various data quality and reporting responsibilities in addition to the responsibilities and workload they had before the Recovery Act was enacted. Other recipients told us that they were not able to implement effective data quality systems to ensure that accurate and reliable data were reported because of budget and/or personnel resource constraints. Insufficient resources to effectively carry out needed oversight activities related to Section 1512 reporting responsibilities increases the risk that errors will not be detected and/or corrected by recipients. Reporting Templates and the Help Desk Several recipients reported that they experienced challenges when working with the FederalReporting.gov reporting templates and/or the FederalReporting.gov Help Desk. Two recipients expressed concern with the often time-consuming process of identifying grants and/or templates associated with error messages. When recipients experienced error messages, some were not specific as to which grant or template (grant recipient, subrecipient, or vendor) was in error or which specific line numbers were in question. Recipient personnel were required to expend significant time trying to determine the reason(s) for the error messages. In addition, the templates did not allow proper copy and paste functions. For example, when subrecipients’ Data Universal Numbering System (DUNS) information was copied to the template, the leading zeros were dropped and had to be keyed in manually. Four recipients indicated that they either received inconsistent guidance from Help Desk personnel or were unable to satisfactorily resolve agency comments despite contacting the Help Desk for assistance. Final Report Page 19 of 44 Lessons Learned and Suggestions for Future Reporting To improve Section 1512 reporting in subsequent periods, recipients provided insights about lessons learned from the reporting period ended December 31, 2009, and offered suggestions on ways to improve the processes for submitting data to Federal Reporting.gov. The experiences of these recipients could provide valuable information to others required to report under the Recovery Act. Several recipients described lessons they had learned from the reporting period ended December 31, 2009, that would help them more easily meet future reporting deadlines. One recipient experienced time pressures in reviewing and submitting Section 1512 data, because the data were not received until just a few days before the reporting deadline. This recipient determined that in the future it would be better to compile and submit the data by the specified reporting deadline and then review and revise the data as needed during the continuous correction period. Other recipients learned that using monthly interim reports and delegating data collection to project staff could ease the challenge of meeting reporting deadlines. Another recipient developed a computer program to electronically extract information needed for Section 1512 reporting from its accounting system to eliminate manual processes and expedite the data collection and reporting process. Some recipients offered suggestions for mitigating or eliminating technical issues that were encountered when submitting data to FederalReporting.gov, such as: • The copy-forward function 14 should only pre-populate data fields that are not subject to change from one reporting period to the next. • Entities required to submit information to FederalReporting.gov should be directly notified when the reporting templates are revised between reporting periods. • Online instructions associated with specific data elements in FederalReporting.gov should be consistent with applicable reporting guidance and be promptly updated to reflect changes in guidance. For example, the online instructions for reporting vendor payments incorrectly stated that the amount for the current period should be reported instead of the cumulative amount as required in applicable Federal guidance. 14The copy forward function is used to link a current report to a prior report when changing the Award Number, DUNS number, or Order number. If a prior report is not linked to the current report for the same award, the progress of the award will not be tracked. In addition, the two reports may appear as duplicates when the data are displayed on Recovery.gov. Final Report Page 20 of 44 • Online instructions should be provided for each data element, such as specifying whether the data should be cumulative or for the current reporting quarter only. • Entities submitting information to FederalReporting.gov should be able to print a draft version of the information to facilitate review and correction. RECOMMENDATIONS We recommend that the Board work with the Office of Management and Budget, the Federal Acquisition Regulatory Council, and Federal agencies, as warranted, to: 1.1 Provide technical assistance to recipients and subrecipients, especially pertaining to effective processes and controls for ensuring the reasonableness of the estimated number of jobs created or retained. Technical assistance could include webinars or additional FAQs to help recipients and subrecipients enhance their reporting effectiveness. Topics could include developing effective policies and procedures for reporting, assigning responsibility within entities for assuring the quality of data and reports, implementing automated techniques for assessing the quality of data, and implementing risk-based techniques to identify significant errors. 1.2 Advise recipients and subrecipients to include steps related to performing analytical procedures comparable to those contained in OMB guidance M-10-08, and other procedures as appropriate, in their data quality control systems in order to promote the detection of reporting errors and data anomalies prior to submission of information to FederalReporting.gov and during the continuous correction period. 1.3 Schedule the timing of future releases of guidance and FAR changes affecting key data elements to allow recipients and subrecipients a reasonable period of time to interpret the guidance and implement the policy, procedural, and process changes necessary to ensure that consistent and reliable data are reported. 1.4 Develop and publish a comprehensive list of lessons learned and best practices for recipients and subrecipients to consider implementing, as appropriate, which could include those identified in this report. 1.5 Implement changes to FederalReporting.gov, as warranted, to facilitate effective and efficient reporting to include possible adoption of the suggestions identified by recipients in this report and also solicit additional ideas for improvements via FederalReporting.gov. Final Report Page 21 of 44 1.6 Consider conducting a comprehensive review of recipient reported information on the number of jobs using statistical sampling methods in order to assess the reliability of reported jobs data for all reporting entities. Final Report Page 22 of 44 SECTION 2: Federal Guidance Applicable to Section 1512 Reporting Based on the work performed at 29 recipients and our review of Section 1512 reporting guidance, we found that some of the Federal guidance pertaining to the data elements we reviewed should be clarified to improve the reliability and consistency of reported data. Some of the recipients included in our review that were responsible for compiling and reporting Recovery Act data misinterpreted Federal guidance and reported inaccurate and/or inconsistent information. Our work also identified areas where reporting guidance could be enhanced to improve data quality and to provide more reporting flexibility to recipients. Guidance on Estimating Jobs Based on our work, we concluded that Federal guidance instructing recipients and subrecipients on how to calculate the number of jobs should be enhanced and clarified. Specifically, we found that the guidance does not recognize that recipients may need the flexibility to use alternative methods in determining the estimated number of jobs. Federal guidance regarding the reporting of small vendor and lower-tier subrecipients 15 should also be clarified. Guidance on the correction of prior quarter job estimates may also need to be clarified. Estimating the Number of Jobs Using Alternative Approaches Federal guidance instructing recipients and subrecipients on how to calculate the number of jobs created or retained does not recognize that recipients may need flexibility in determining this number. The guidance does not indicate that recipients can use alternative methods that could also produce reasonable jobs estimates. Our work showed that not all recipients spent funds intended to create or retain jobs in the same manner. Moreover, not all recipients tracked the number of hours of employees’ work that was funded by the Recovery Act. In addition, we identified other components of Federal guidance related to calculating the number of jobs that appear to warrant clarification. According to OMB guidance M-10-08, recipients of Recovery Act funds subject to Section 1512 are required to submit estimates of the number of jobs created or retained for each project or activity. In addition, “[r]ecipients must include an estimate of jobs created and retained on projects and activities managed by their funding recipients (i.e., sub-recipients)….” The guidance specifies a formula that recipients and subrecipients are to use in calculating the number of jobs. This formula is based on the number of actual hours worked in jobs paid with Recovery Act funds in relation to the number of 15A lower-tier subrecipient is a subrecipient that does not receive a subaward directly from a recipient. Final Report Page 23 of 44 hours representing a full-time work schedule for the kind of job being estimated. Although OMB guidance states that recipients and subrecipients should report the number of jobs based on FTEs, our review found that not all recipients could determine FTEs using the process described in OMB guidance. One grant recipient included in our review was unable to comply with the method for calculating the number of jobs specified in OMB guidance M-10-08 simply because the guidance did not recognize the way Recovery Act funds were used to minimize or avoid employee layoffs. In essence, the recipient’s approach to retaining jobs prevented them from following OMB’s process. This recipient received Recovery Act grant funds and subawarded the funds to subrecipients but was constrained by a state legislative mandate that prescribed how the funds could be used to pay for personnel costs. Although it was clearly the state legislature’s intent to use the Recovery Act funds to retain jobs, the legislature chose to restrict subrecipients’ use of Recovery Act funds to paying Social Security and other retirement program costs of employees. The rationale for using the funds this way was to replace a specific state budget line item that had been funded in the past using state funds with Recovery Act funds. The intent was to avoid budget reductions that would result in layoffs. As a result, subrecipients could not track the actual hours worked on jobs paid with Recovery Act funds as specified by OMB guidance. In this case, the recipient developed an alternative process for estimating the number of jobs. For each subrecipient, the recipient divided the amount of Recovery Act funds spent on the employee benefits by the average salary amount (including benefits) for that position type. The recipient added the results across all subrecipients to determine the total number of jobs created or retained as a result of Recovery Act funding. This alternative procedure appeared to be reasonable because the funds were used to support personnel costs, even though they were not attributed to specific employees. The requirements for reporting the number of jobs related to Federal contractors also do not recognize that alternative approaches can derive reasonable jobs estimates. One contractor included in our review was unable to follow the applicable guidance on calculating the number of jobs because it did not track the data needed to follow the specified process. FAR clause 52.204-11(d)(7)(ii) requires contractors to provide an estimate of the number of jobs created or retained by the prime contractor. Comparable to OMB guidance M-10-08, the FAR clause specifies that the number of jobs shall be expressed as “full-time equivalents” and be calculated as the number of hours worked divided by the total number of hours in a full-time schedule. Under its alternative procedures, the contractor allocated FTEs to projects funded by the Recovery Act based on the awarding Federal agency’s allocation of project funding between Recovery Act and non-Recovery Act funds. The contractor used an allocation methodology because its Recovery Act contract is a firm, fixed-price contract, and it does not report hours worked on Recovery Act Final Report Page 24 of 44 activities. This alternative approach also appears to provide a reasonable estimate of the number of jobs created or retained as a result of Recovery Act funds. Reporting of Lower-Tier Subrecipient and Small Vendor Jobs Clarification is needed on whether subrecipients should report jobs created or retained by lower-tier subrecipients and by vendors awarded less than $25,000. We found that the estimated number of jobs reported related to these entities was not consistent. The inconsistent treatment of these jobs estimates by recipients may result in part because OMB guidance appears to be confusing and does not definitively instruct recipients as to whether these jobs should be reported. Recipient uncertainty has resulted in inconsistent reporting across recipients on Recovery.gov. Guidance for estimating the number of jobs was first provided in OMB guidance M-09-21 issued in June 2009. In December 2009, OMB guidance M-10-08 was issued with updated instructions for reporting the number of jobs. However, the updated guidance replaced only Section 5 from the original guidance related to the number of jobs and did not restate some key definitions and other information that would help ensure that recipients and subrecipients report properly. For example, the detailed definition of “sub-recipient” contained in OMB guidance M-09-21, which is still applicable, is not included in OMB guidance M-10-08. When describing who is required to report under the Recovery Act, paragraph 2.2 of OMB guidance M-09-21 defines “sub-recipient” as “non-Federal entities that are awarded Recovery funding through a legal instrument from the prime recipient to support the performance of any portion of the substantive project or program for which the prime recipient received the Recovery funding.” Specifying that subrecipients receive their awards from the prime recipient would indicate that the definition of subrecipient for Recovery Act reporting purposes would only include first-tier subrecipients (i.e., those that received their award directly from the prime recipient) and would exclude lower-tier subrecipients (i.e., those that received their awards from a subrecipient). Paragraph 5.2 of OMB guidance M-09-21 requires recipients to include jobs estimates by their funding recipients and paragraph 5.2 of OMB guidance M-10-08 attempted to clarify this by emphasizing their funding recipients and adding “(i.e., sub-recipients).” This is also repeated in paragraph 5.7. This seems to support the interpretation that prime recipients are required to include in their reports only jobs created or retained by first-tier subrecipients. However, other statements in OMB guidance seem to conflict with the statements above. For example, paragraph 5.2 of OMB guidance M-10-08 requires recipients to “estimate the total number of jobs (emphasis added) that were funded in the quarter by the Recovery Act.” Paragraph 5.7 of the guidance also states “[t]o the maximum extent practicable, information should Final Report Page 25 of 44 be collected from all sub-recipients and vendors in order to generate the most comprehensive and complete job impact numbers available.” Based on the above cited definition of subrecipient, the reference to “all subrecipients” in paragraph 5.7 could be interpreted to mean all first-tier subrecipients. Alternatively, it could be interpreted to include both first-tier and lower-tier subrecipients. Lastly, paragraph 5.7 of OMB guidance M-10-08 that states “[t]he clarification that recipients must report jobs estimates for all sub-awarded funds (emphasis added) was an update from guidance prior to June 2009” seems to further compound recipient and subrecipient confusion. In contrast, we noted that the jobs reporting guidance in FAR Case 2010-008 published on July 2, 2010, provided more definitive instructions for contractors. This guidance, which amended Recovery Act reporting requirements in FAR clause 52.204-11(d)(7)(ii), requires contractors to provide “[a]n estimate of the number of jobs created and jobs retained by the prime Contractor and all first-tier subcontracts valued at $25,000 or more, in the United States and outlying areas.” Our work at recipients and subrecipients showed that recipients were not consistent on whether they excluded or included estimates of lower-tier subrecipient jobs funded by the Recovery Act in their second quarterly report. One subrecipient did not include lower-tier subrecipient jobs data to the recipient when it submitted the number of jobs resulting from its subaward. As a result, the lower-tier subrecipient jobs were not reported on Recovery.gov. However, other subrecipients did collect estimated jobs data from lower-tier subrecipients and submitted the data, along with their own jobs estimates, to the recipient. These jobs were reported on Recovery.gov. We also found that some recipients were unclear about the need to collect and report the number of jobs for vendors that were receiving awards of less than $25,000. Several recipients had not developed procedures to collect and report the number of jobs resulting from vendors receiving less than $25,000 in Recovery Act funds. These recipients believed such reporting was not necessary and would be negligible because of the small dollar amounts awarded. However, we found that consulting contracts, for example, can often be under the $25,000 threshold and still result in jobs being created. OMB needs to provide a definitive statement on the extent that recipients should report the estimated number of jobs resulting from funds subawarded by subrecipients to lower-tier subrecipients. It may also be necessary to reinforce to recipients and subrecipients that the number of jobs created or retained by their vendors should be reported, regardless of the amount of Recovery Act funding they have received. We also noted that neither the Recovery Act nor current Federal guidance requires that vendor contracts funded by the Recovery Act include a notice of the Section 1512 requirement to report the number of jobs. Earlier this year, the U.S. Government Accountability Office testified that subrecipients Final Report Page 26 of 44 thought that it would be useful to vendors to include Section 1512 reporting requirements in contracts. Based on the results of our work, including Section 1512 reporting requirements related to estimating the number of jobs would help ensure consistency and transparency across vendor contracts. Correction of Prior-Quarter Jobs Estimates OMB guidance regarding correction of prior-quarter errors related to the number of jobs may also need clarification. Section 5.10 of OMB guidance M-10-08 requires “recipients to maintain within their administrative records comprehensive information on any and all necessary corrections to prior quarter data.” Recipients will need to submit this information to the Federal government at a time and through a process to be specified in the future, after the Board determines the best approach for making this information available on Recovery.gov. Our review found that several grant recipients were unsure whether necessary payroll adjustments related to prior quarters should be made during the current quarter. These types of adjustments would affect the number of jobs reported for the current quarter if made. One awarding Federal agency indicated that since adjustments are made in the recipient’s records in the next or a future reporting quarter, the adjustment in the number of jobs reported should be reflected in the quarterly report associated with when the adjustments are made. Guidance on Reporting Recovery Act Funds Spent Our review of Federal guidance related to the amount of Recovery Act funds spent that recipients are to report identified several areas that we concluded may be inappropriate and/or subject to different interpretations. Recipients following the current expenditure guidance contained in the Recipient Reporting Data Model may over-report or under-report actual funds spent, depending on how they interpret the various components of the guidance. OMB guidance M-09-21 defines the expenditure data element as “the amount of Recovery funds received that were used to pay for projects or activities, including payments made to sub-recipients and vendors.” Supplement 2 (Recipient Reporting Data Model) to OMB guidance M-09-21 provides a more detailed definition of the data element, including specifying that it is the “Federal share of expenditures” and that it applies to grants and loans only. The full definition of the “Total Federal Amount of ARRA [American Recovery and Reinvestment Act of 2009] Expenditure” contained in Supplement 2 is as follows: This is for grants and loans only. Amount of recovery funds received that were expended to projects or activities (‘‘Federal Share of Expenditures’’). The cumulative total for the amount of Federal fund expenditures. For reports prepared on a cash basis, expenditures are the sum of cash disbursements for direct charges for property and Final Report Page 27 of 44 services; the amount of indirect expense charged; the value of third- party in-kind contributions applied; and the amount of cash advance payments and payments made to subcontractors and Subawardees. For reports prepared on an accrual basis, expenditures are the sum of cash disbursements for direct charges for property and services; the amount of indirect expense incurred; the value of in-kind contributions applied; and the net increase or decrease in the amounts owed by the recipient for (1) goods and other property received; (2) services performed by employees, contractors, subcontractors, Subawardees, and other payees; and (3) programs for which no current services or performance are required. Do not include program income expended. Although we did not identify issues related to this definition of funds spent at the recipients we reviewed, our review of OMB guidance identified the following issues for OMB consideration: • Third-Party In-Kind Contributions. It does not appear appropriate to include third-party in-kind contributions in the funds spent data element. Third-party in-kind contributions, whether in the form of services provided, equipment, supplies or other contributions, would not typically represent the expenditure of Federal funds received through a Recovery Act grant award. We noted that OMB Circular A-110, 16 states “[t]hird party in-kind contributions means the value of non-cash contributions provided by non-Federal third parties” (emphasis added). Reporting these contributions would appear to over-report the amount of actual Federal funds expended. • Reporting on the Accrual Basis – Net Increase or Decrease in Amounts Owed. Recovery Act funds spent may not be reported accurately if recipients report only the “net increase or decrease in the amounts owed by the recipient” when reporting on an accrual basis. Because the expenditure data element is to be reported on a cumulative basis, it appears that recipients should report the total amount owed (i.e., total obligations) at the end of a reporting period rather than reporting only the net change in the amount owed from one reporting period to another. Not reporting total obligations would appear to under-report the amount of Recovery Act funds spent. • Reporting on the Accrual Basis – No Services or Performance Required. Item 3 of OMB’s definition for accrual based funds spent--“programs for which no current services or performance are required”--could be confusing to recipients. Without additional explanation of what would typically be included in this component, it may be difficult for recipients to even know what to report. In addition, the description of this component of the funds spent data element appears to refer to potential future costs for which Uniform Administrative Requirements for Grants and Agreements With Institutions of Higher 16 Education, Hospitals, and Other Non-Profit Organizations. Final Report Page 28 of 44 there is no current commitment or obligation. If so, it may be inappropriate to include such costs until an obligation is made. • What to Include in Reported Funds Spent. We also identified several data elements associated with funds spent such as the “Total Federal ARRA Infrastructure Expenditure,” “Total Amount of Payments to Vendors less than $25,000/Award,” and/or “Total Subaward Funds Disbursed,” in which OMB may need to clarify whether the amounts reported should also be included in the “Total Federal Share of Expenditures” data element. These and other expenditure-related data elements are currently reported separately in the Recipient Reporting Data Model. However, there are no instructions to recipients indicating whether the “Total Federal Share of Expenditures” is a grand total that should include these other expenditure types or that each of these amounts are mutually exclusive and, thus, should be reported individually. In the absence of such guidance, it is likely that recipients’ interpretations may differ, leading to inaccurate and inconsistent reporting. Guidance on the Final Report Data Element As of the initial January 2010 reporting deadline, OMB had not provided any specific guidance addressing when recipients should select the “Yes” option for final report. The Recipient Reporting Data Model simply instructed recipients to “[c]heck ‘Y’ only if this is the final report and there will be no further quarterly reports.” On March 22, 2010, OMB issued more specific guidance (M-10-14) related to the Final Report data element for recipients of grants, loans, and other Federal assistance as follows: Recipients that have complied with their reporting requirements will no longer be required to submit Section 1512 reports under the following circumstances: • The award period has ended; and • All Recovery funds are received (through draw-down, reimbursement or invoice) and the project status is complete per agency requirements and/or performance measures; or • The award has been terminated or cancelled. A recipient will indicate a “Y” in the final report data field in FederalReporting.gov if its report is considered final and there will be no future reports submitted. Indication of a final Section 1512 report does not replace any other closeout procedures required by the recipient or Federal agency. Although none of the recipients we reviewed had selected “Y” for this data element for the second quarterly report, the OMB guidance could lead to confusion and/or require a recipient to continue reporting even after all grant funds were expended and the project was in fact completed. We identified Final Report Page 29 of 44 several issues that appear to warrant OMB attention related to its guidance for the Final Report data element: • A recipient may interpret the first bullet in the above instructions to mean that it should not select “final report” even if all Recovery Act funds were received and the project status was complete but the award period had not ended. As a result, the recipient would continue to report unnecessarily for one or more additional quarters. • For improved transparency, it might be advisable to make the primary determinant for selecting “final report” be that all funds were expended (grants/loans) or invoiced (contracts) by recipients and/or subrecipients as opposed to being received or drawn down. As discussed in Section 3, the fact that all Recovery Act funds have been received by a recipient does not necessarily mean that all funds were spent or invested in the economy. It may also not mean that no additional economic activity (e.g., jobs created or retained) will result from these funds in the future. • In some cases, it may be premature to instruct recipients to select “final report” before all closeout procedures have been performed on a grant or contract. For example, material issues may be identified during contract closeout procedures that would require additional actions by the contractor. If this occurs, it could be necessary to report additional information to FederalReporting.gov. RECOMMENDATIONS We recommend that the Board work with the Office of Management and Budget and Federal Acquisition Regulatory Council, as warranted, to implement the following changes to OMB guidance and/or the Federal Acquisition Regulation related to Section 1512 reporting by: 2.1 Identifying alternative methods for calculating the number of jobs created or retained in applicable Federal guidance, clearly articulating guidelines addressing the circumstances under which these alternative methods should be used, and identifying the factors recipients should consider to ensure that such alternative methods produce reasonable estimates of the number of jobs. 2.2 Clarifying guidance related to (1) subrecipient reporting on the number of jobs resulting from Recovery Act funds paid to vendors and subawarded to lower-tier subrecipients, (2) the types of costs that should be included or excluded when calculating and reporting the amount of Recovery Act funds spent, and (3) when recipients should designate a Section 1512 report as “final.” Final Report Page 30 of 44 SECTION 3: Optimizing the Transparency of Reported Information One of the hallmarks of the Recovery Act is an unprecedented level of accountability and transparency. This includes an expectation that the uses of all funds provided under the Recovery Act are transparent to the public. Another expectation is that the public benefits derived from the use of Recovery Act funds are reported accurately, consistently, clearly, and in a timely manner. We identified transparency issues related to four of the five data elements—amount of funds spent, project status, final report, and number of jobs. In these cases, the Section 1512 data reported by recipients may not present an accurate or complete picture of what is transpiring “on the ground,” even though the recipients reported the data as specified in Federal guidance. Our work also identified other situations where current guidance, including the Recipient Reporting Data Model, may not provide the desired level of transparency and may result in misleading information being posted to Recovery.gov. Reporting Disbursements to Subrecipients as Recovery Act Funds Spent Our work at grant recipients found that reporting the amount of Recovery Act funds disbursed to subrecipients as funds spent may reduce transparency and result in misleading information being posted to Recovery.gov. OMB guidance specifies that recipients report their disbursements to subrecipients as Recovery Act funds spent. However, the reported amounts may not reflect the actual Recovery Act funds spent by the subrecipients and, thus, distort the true amount of Recovery Act funds invested in the economy. Moreover, reporting funds spent in this way may impact other interrelated data elements, such as project status and final report, and confuse the public. In some instances, the recipient’s reported funds spent represented cash advances to subrecipients, which may not have needed the funds to cover Recovery Act program costs at the time the recipient disbursed the funds. For example, one recipient appropriately reported about $4 billion of Recovery Act funds spent representing the amount of funds disbursed to its subrecipients. In this case, the recipient drew down and disbursed all the Recovery Act program funds to its subrecipients. The recipient also required all subrecipients to track and report their actual funds spent related to the Recovery Act subawards. When we compared the reported amount of disbursements to subrecipients and the actual Recovery Act funds spent reported to the recipient by the subrecipients, we found that the amount of funds spent that was posted to Recovery.gov exceeded subrecipients’ actual funds spent by more than $1.3 billion, or almost 50 percent. Rather than being invested in the economy, the $1.3 billion in unspent Recovery Act funds was still being held in subrecipients’ bank accounts when the quarter ended on December 31, 2009. Final Report Page 31 of 44 Reporting disbursements to subrecipients as Recovery Act expenditures could also result in the reporting of project status and final report in a manner that does not convey the actual status of the recipient’s grant. When reporting project status, the Recipient Reporting Data Model states that the evaluation of project status “should be based on performance progress reports and other relevant non-financial information.” Recognizing that determining project status for formula grants 17 would be a rough estimate, the awarding Federal agency’s guidance advised recipients to measure project status by using the percentage of the grant award that the recipient drew down. The example described in the previous paragraph illustrates the interrelationship of the amount of reported funds spent, project status, and final report. If the recipient drew down and disbursed 100 percent of its grant award to subrecipients, according to the awarding Federal agency’s guidance it would report project status as “fully completed” and indicate the current report as “final” (i.e., no further Section 1512 reports would be submitted). However, because the subrecipients had not expended all of the Recovery Act funds received from the recipient, reporting project status and final report in this way would be misleading and obscure transparency. Contractor Reporting of Subcontractor Jobs For the quarter ended December 31, 2009, the FAR did not require Federal contractors to report the number of jobs created or retained by subcontractors as a result of Recovery Act funds. Only the contractor jobs were to be reported. In contrast, grant recipients were to report the number of jobs for both the recipient and its subrecipients. In one case, a contractor subawarded a $25 million contract in its entirety to a subcontractor. The contractor appropriately did not report the more than 220 jobs created or retained by the subcontractor on FederalReporting.gov. Our work also identified four contractors that, contrary to the guidance in effect for the reporting period ended December 31, 2009, reported subcontractor jobs because they believed that the additional reporting more accurately portrayed the number of jobs impacted by Recovery Act funds. Recognizing the inconsistency in the reporting required of grant recipients and Federal contractors, FAR Case 2010-008 revised contractors’ requirements to include the reporting of jobs created or retained by subcontractors with Recovery Act awards of $25,000 or more for all new solicitations and contracts issued on or after July 2, 2010. Although this change will improve data consistency and transparency for future reporting periods, there is no requirement or expectation for Federal contractors to revise prior Section 1512 reports. In addition, for contract solicitations and contracts issued before July 2, 2010, contractors will continue to report only 17Formula grant programs are noncompetitive awards based on a predetermined formula. These programs are sometimes referred to as state-administered programs. Formula grants are allocated to states or their subdivisions in accordance with distribution formulas prescribed by law or administrative regulation for activities of a continuing nature that are not confined to a specific project. Final Report Page 32 of 44 the jobs created or retained by them. Hence, some degree of inconsistency and reduced transparency will remain in the information posted to Recovery.gov indefinitely. However, this situation should be mitigated to some extent by those contractors that have already reported and continue to report subcontractors’ jobs in the future. Cash versus Accrual Basis of Reporting Funds Spent Differences in accounting methods recipients used to report the amount of Recovery Act funds spent may also result in reduced transparency. Recipients did not have a mechanism to indicate what accounting method they used when reporting funds spent to FederalReporting.gov, even though the Recipient Reporting Data Model allowed recipients to report the data on either a cash or accrual basis. The method used to report the amount of funds spent results in differences in the timing of when the funds spent are reported and potentially significant differences in the amount of funds spent reported for a particular quarter. As a result, users of the information available on Recovery.gov cannot effectively compare expenditure amounts reported by recipients that used different accounting methods. For example, one recipient used a cash basis of accounting when it reported Recovery Act funds spent for the period ended December 31, 2009. The reported funds spent included payments made by the recipient and the amount of funds drawn down and paid to two subrecipients included in our review. Both subrecipients requested funds on a reimbursement basis. Under reimbursement, each subrecipient would submit a reimbursement request by the 20th day of the month for amounts they had spent during the prior month. Because it reported on a cash basis, the recipient reported only the amounts it drew down and disbursed to the subrecipients during the quarter. As a result, the subrecipients’ actual funds spent for the last month of the second quarter (December 2009) were not reflected in the recipient’s Section 1512 report for the period ended December 31, 2009. The subrecipients’ December 2009 funds spent were not included because the recipient did not reimburse the subrecipients until January 2010. If this recipient reported under the accrual basis of accounting, it would have reported an additional $6.7 million in funds spent in its second quarterly report. Without information on the accounting method used, the amount of Recovery Act funds spent posted to Recovery.gov could be misleading to the public. Recipient Reporting of Subrecipient Jobs by Congressional District Recipient reporting of the number of jobs based on the recipient’s location may raise transparency issues when the data are viewed by Congressional district on Recovery.gov. Under current OMB guidance and the FAR, recipients are to report the number of jobs created or retained by Final Report Page 33 of 44 subrecipients. The location of the recipient determines the Congressional district that is credited for the number of jobs reported by a recipient, regardless of whether some or all of the jobs were created or retained by subrecipients located in other regions of the country. Therefore, when a user of information posted to Recovery.gov looks at the number of jobs for his or her Congressional district, the number reported may not reflect the actual number of jobs created or retained in that district. The adverse impact on transparency is magnified when the recipient subawards its entire grant to subrecipients outside its Congressional district. Examples from our work include: • One recipient located in Washington, D.C., subawarded 60 percent of a $106 million grant to four subrecipients located in the states of Oregon, California, Washington, and Massachusetts. The 55 jobs that the recipient reported were all “counted” in the recipient’s Congressional district (the District of Columbia), even though more than 40 of the jobs (about 75 percent) were actually created or retained in the 4 states. • A Federal contractor, also located in Washington, D.C., subawarded its entire $25 million contract to a subcontractor located in Texas. Appropriately, the contractor did not report the number of jobs created or retained by the subcontractor for the Number of Jobs data element in FederalReporting.gov. However, to enhance transparency the contractor chose to report the subcontractor’s more than 220 jobs in a different data element (Description of Jobs Created/Retained) in which recipients were to provide a narrative description of the employment impact of work funded by the Recovery Act. Again, the jobs reported for the Number of Jobs data element were not recognized in the Congressional district where the jobs were actually created or retained. • One recipient subawarded its entire Recovery Act grant to subrecipients located throughout the state. The recipient reported more than 36,000 jobs created or retained resulting from this grant for the reporting period ended December 31, 2009. However, more than 4,500 of the total estimated jobs (12.5 percent) were attributed to 1 subrecipient alone that was located in another region of the state. Although the situations described in these examples reduced transparency, there may be significant barriers to implementing a feasible solution that outweigh the benefits of enhanced transparency. Requiring all recipients to report the number of jobs based on where the actual employment impact occurred (i.e., each subrecipient’s Congressional district) could add significant complexity to recipient reporting processes. Many Recovery Act recipients we reviewed indicated they lacked sufficient resources to effectively meet their current reporting obligations. In addition, there is currently no mechanism for recipients to provide detailed information on the number of jobs by Final Report Page 34 of 44 Congressional district, except to describe the employment impact in narrative form in the Description of Jobs Created/Retained data element. However, a narrative description does not allow for easy aggregation of the number of jobs by Congressional district. For subrecipients with subawards of more than $25,000, the FederalReporting.gov reporting template already includes a data element for each subrecipient’s Congressional district. An additional data element for the number of subrecipient jobs would need to be added. In addition, recipients would need to ensure that the number of subrecipient jobs were not double- counted when reporting the number of jobs for the entire award, including subrecipients with subawards of less than $25,000. Furthermore, requiring all subrecipients to separately report the number of jobs created or retained with Recovery Act funds would dramatically increase the number of reporting entities, which could result in more data errors and reduced consistency in the reported data. Lastly, we noted that OMB took steps in recent guidance to reduce the reporting burden on Recovery Act recipients and simplify the process for estimating the number of jobs. RECOMMENDATIONS We recommend that the Board work with the Office of Management and Budget and the Federal Acquisition Regulatory Council, as warranted, to explore opportunities to increase the transparency of information available on Recovery.gov by: 3.1 Evaluating the feasibility of requiring recipients to report subrecipients’ actual funds spent instead of reporting the amount of funds a recipient disbursed to subrecipients to ensure that the reported amount of funds spent more accurately reflects the amount of Recovery Act funds invested in the economy. 3.2 Encouraging Federal contractors to include subcontractor jobs in the reported number of jobs created or retained for those solicitations and awards made before Federal Acquisition Regulation Case 2010-008 was issued on July 2, 2010, unless contract terms prohibit such reporting, to improve consistency in the number of jobs reported across Federal contracts. 3.3 Evaluating the feasibility of having recipients disclose whether the reported amount of Recovery Act funds spent were accounted for on a cash or accrual basis to ensure that reports available on Recovery.gov disclose when expenditure data may not be comparable because of the use of different accounting methods. Final Report Page 35 of 44 3.4 Evaluating the feasibility of having recipients report the number of jobs in the Congressional district where the actual jobs were created or retained when recipients subaward Recovery Act funds to subrecipients located outside the recipients’ Congressional district. If it is determined that reporting the jobs in the Congressional district where they are actually located is not feasible, then to promote transparency, we recommend that the Recovery.gov Web site acknowledge this limitation. Final Report Page 36 of 44 OBJECTIVE, SCOPE, AND METHODOLOGY The objective of our review was to determine whether selected Recovery Act recipients’ processes for compiling and reporting selected data provided reasonable assurance of compliance with reporting requirements contained in Section 1512 of the Act. The review focused on Section 1512 reporting for the period ended December 31, 2009, and actions taken to enhance data quality for subsequent reporting periods. The review focused on five Section 1512 data elements—number of jobs, total Federal amount of Recovery Act funds received or invoiced, total Federal amount of Recovery Act funds spent, project status, and final report—as described in the Background section of this report. Reviews were carried out by teams representing five OIGs from the Department of Education, Department of Health and Human Services, Department of Homeland Security, Department of Labor, and the National Science Foundation. In total, the 5 OIGs reviewed the reporting processes in place at 20 grant recipients, 14 grant subrecipients, and 9 Federal contractors. Some recipients we reviewed did not subaward Recovery Act funds to any subrecipients whereas one recipient subawarded funds to more than a thousand subrecipients. Our work covered four grant programs and six Federal contract programs under the Recovery Act. Information on the programs, funding, and other information is presented in Appendix 1. To meet the objective, each OIG selected one or more Recovery Act programs for review and judgmentally selected recipients (grant recipients and/or Federal contractors) and, for two OIGs, selected grant subrecipients. Factors considered in selecting recipients and subrecipients included the amount awarded under the Recovery Act program, prior audit experience with the recipient, analyses of Section 1512 data reported to FederalReporting.gov, number of jobs reported, input from the awarding Federal agency, and geographical location. Using a common review guide, the OIGs performed audit work at each selected recipient and subrecipient, including the following steps: • Reviewed criteria applicable to Section 1512 reporting, including OMB guidance and the FAR; • Interviewed recipient and (if applicable) subrecipient officials; • Reviewed available documentation of recipient and subrecipient processes; • Conducted limited non-statistical sampling to understand and evaluate reporting processes and related controls; • Performed limited analytical procedures, such as logic and reasonableness checks of data quality; and • Discussed fieldwork results with the Federal agency, recipients, and/or subrecipients (if applicable). Final Report Page 37 of 44 The fieldwork for this project was conducted between December 2009 and August 2010. The OIGs submitted the results of their reviews to the Education OIG, which analyzed the data for common themes and consolidated the results into this report. Each participating OIG planned to issue one or more separate products to its Federal agency and/or selected recipients to provide the results of their individual reviews. Each OIG performed its work related to this performance audit in accordance with generally accepted government auditing standards. Those standards require that we plan the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions. We believe that the evidence obtained provides a reasonable basis for the findings and conclusions contained in the report, based on the audit objective. Final Report Page 38 of 44 APPENDIX 1 Table 2 identifies the awarding Federal agency, Recovery Act programs, amount of funds received/invoiced, and types of entities we reviewed. Table 2: Recovery Act Programs, Funding, and Entity Types Reviewed Total Entity Type Reviewed Federal Amount of Awarding Recovery Act Funds Federal Grant Grant Federal Program(s) Received/ Agency Recipient Subrecipient Contractor (a) Invoiced for (20) (14) (9) Entities Reviewed (millions) State Fiscal Stabilization ED $5,576.1 4 10 0 Fund – Education Grants Health Resources and Services Administration: “ARRA-Health HHS $1.0 4 0 0 Center Integrated Services Development Initiative” (b) Aviation Security, Border Security, DHS Bridge and $51.5 0 0 8 Facility Improvements (c) Workforce DOL $218.8 2 4 0 Investment Act Recovery Act NSF $24.3 10 0 1 (d) Research Support (a) The Federal agencies are the Department of Education (ED), Department of Health and Human Services (HHS), Department of Homeland Security, (DHS), Department of Labor (DOL), and National Science Foundation (NSF). (b) ARRA is an acronym for the Recovery Act. (c) For brevity, the programs listed represent the activities covered by the projects and are not the actual titles for the 5 contract programs reviewed. (d) One recipient was both a grant recipient and a Federal contractor. Final Report Page 39 of 44 The following is a brief description of each agency’s Recovery Act program(s) and the authorized funding amount: The State Fiscal Stabilization Fund – Education Grants ED awarded governors approximately $48.6 billion by formula in exchange for a commitment to advance essential education reforms to benefit students from early learning through post-secondary education. Health Resources and Services Administration (HRSA): “ARRA- Health Center Integrated Services Development Initiative” HRSA received $2.5 billion in fiscal year 2009, including $2 billion to expand the Health Center Program to serve more patients, stimulate new jobs, and meet the significant increase in demand for primary health care services among the uninsured and underserved populations. Aviation Security, Border Security, Bridge and Facility Improvements DHS components received $2.75 billion in funding for the following activities: • Procurement and installation of baggage and passenger explosive detection systems at selected airports; • Construction/renovation of land ports of entry, purchase of non- intrusive inspection systems, development and deployment of the Secure Border Initiative Program, and the upgrade of tactical communications; • Grants for Emergency Food and Shelter National Board Program, Public Transportation and Railroad Security Assistance, Port Security and Assistance to Firefighters for the construction/renovation of non-Federal fire stations; • Alteration to bridges, improvements to shore facilities, and repairs to vessels; • Continued development of the DHS consolidated Headquarters; and • Upgrade of its tactical communication system. Workforce Investment Act The Employment and Training Administration awarded $2.9 billion using the Workforce Investment Act formula to the 57 states and territories to provide worker training for Adults, Dislocated Workers, and Youths. Recovery Act Research Support NSF received $3.0 billion to (1) further promote the progress of science; (2) advance the national health, prosperity, and welfare; and (3) secure the national defense. Final Report Page 40 of 44 APPENDIX 2 Examples of Recipient Reporting Errors In Section 1 of this report, we summarized data quality issues related to recipients’ reporting of the number of jobs, funds received/invoiced, funds spent or project status. More detailed information about reporting errors is presented in this appendix. Reporting Errors Related to the Number of Jobs OMB Guidance and FAR Implementation Issues Many of the recipients and subrecipients were unable to effectively implement OMB guidance or the FAR when reporting the number of jobs created or retained. Examples of identified reporting issues include: • Alternative Calculation Methods Were Used. Three recipients used a variety of alternative methods to calculate the number of jobs because they did not track the work hours paid with Recovery Act funds as specified in OMB guidance or the FAR. We concluded that the recipients’ alternative methods appeared to provide reasonable estimates of the number of jobs. Two grant recipients tracked Recovery Act funds spent or amounts awarded to subrecipients as an alternative because they either could not identify specific employees who were paid with Recovery Act funds or chose not to. One Federal contractor’s alternative method entailed allocating jobs based on the awarding Federal agency’s allocation of project funds between Recovery Act and non-Recovery Act funds. • Number of Jobs Were Reported Cumulatively. One grant recipient reported the number of jobs cumulatively because it was not able to fully implement the new methodology for calculating the number of jobs specified in OMB guidance M-10-08. The number of jobs reported in its initial submission to FederalReporting.gov was cumulative for the award in accordance with OMB guidance M-09-21. This was the guidance in effect when data were collected from subrecipients before the issuance of OMB guidance M-10-08. The recipient had mixed success in correcting the number of jobs by the end of the continuous correction period resulting in over-reporting of the number of jobs. • Jobs Not Paid with Recovery Act Funds Were Incorrectly Reported. Three recipients incorrectly reported the number of jobs because the jobs were either paid with non-Recovery Act funds or represented unfilled positions that were expected to be paid with Recovery Act funds when filled. For example, one recipient reported Final Report Page 41 of 44 almost 40 jobs that were retained when a mental health facility slated for closure was purchased by the recipient with non-Recovery Act funds. Without Recovery Act funds, the recipient told us it would not have had the organizational capacity to administer this new facility and, therefore, believed that the Recovery Act funds were indirectly related to the retention of these jobs. Another recipient reported about 10 jobs that it expected to fill using Recovery Act funds at a later date along with the actual number of jobs created during the quarter using Recovery Act funds. • Subcontractor Jobs Were Inappropriately Included. Four Federal contractors included subcontractors’ jobs in their reported number of jobs even though the FAR specified that only the prime contractors’ jobs should be reported. The subcontractors made up a substantial percentage of the total jobs reported by at least one contractor. In this case, about 160 of the more than 200 jobs (80 percent) reported by the contractor were subcontractors’ jobs. These contractors believed that subcontractors were an integral part of their supply chain, manufacturing operations, or contract deliverables, and not including subcontractor jobs would underestimate the impact of the Recovery Act on employment. As discussed in Section 1, if these contracts had been awarded after the issuance of FAR Case 2010-008 on July 2, 2010, we would not have identified this as a data quality issue. Other Recipient-Caused Errors Some recipients experienced one or more data quality issues resulting in posting, calculation, and other errors in their reported number of jobs. Examples of identified reporting errors include: • Processing Errors. Four recipients under-reported jobs because of posting errors, calculation errors, and/or not fully understanding applicable guidance. For example, one recipient under-reported more than 10 jobs, or about 24 percent fewer jobs than it should have, because of a math error in its worksheet for calculating the number of jobs. Another recipient used an alternative method to calculate the number of jobs but did not adjust its formula for the quarterly reporting specified in OMB guidance M-10-08. The formula error resulted in the recipient reporting about 1,950 fewer jobs (75 percent) than the number of jobs that should have been reported. • Accounting Adjustments. Two recipients made payroll adjustments that resulted in inaccurate reporting for the number of jobs. Final Report Page 42 of 44 • Uncorrected Number of Jobs. Two recipients failed to correct identified changes in the number of jobs during the continuous correction period. For example, one subrecipient reduced the number of jobs originally reported to the recipient from 40 to 30 jobs. By not making the correction in FederalReporting.gov during the correction period, the recipient’s reported number of jobs associated with the subrecipient was over-reported by 33 percent. Reporting Errors Related to Funds Received/Invoiced, Funds Spent or Project Status We identified data quality issues for one recipient’s Recovery Act funds received/invoiced data, three recipients’ funds spent data, and one recipient’s project status. Examples of identified reporting errors include: • Expenditure and/or Funds Received/Invoiced Errors. Of the 20 grant recipients required to report funds spent, 3 recipients either did not implement OMB guidance correctly or made reporting errors resulting in the reporting of inaccurate amounts of funds spent. One recipient did not limit reported funds spent to only those costs paid with Recovery Act funds as specified in OMB guidance. The recipient reported funds spent of $245 million in total, which inappropriately included $25 million paid with state funds. Another recipient incorrectly reported the amount of funds spent in the same amount as funds received/invoiced when using the cash basis to report Recovery Act funds spent resulting in the under-reporting of funds spent by more than $18,000. The third recipient mistakenly entered Section 1512 data for a $230,000 grant twice in FederalReporting.gov, which resulted in over-reporting of funds spent and funds received/invoiced by 100 percent. • Project Status Error. One recipient did not accurately report the project status for 2 of its 80 Recovery Act grants. For example, the project status for one grant was improperly reported as “more than 50% complete” instead of “less than 50% complete” because the recipient misunderstood that the determination of project status was to be based on the overall status of the project and not quarterly progress. Final Report Page 43 of 44 APPENDIX 3 Examples of Recipient Data Quality Controls and Processes In Section 1 of this report, we summarized recipient processes and controls for ensuring the quality of data reported to FederalReporting.gov. More detailed information about these controls and processes is presented in this appendix. Risk-Based Monitoring and Management Reviews: • Risk-Based Monitoring. One recipient developed a five-member monitoring project focused on the oversight of large subrecipients’ use of Recovery Act funds and reporting of the number of jobs. The recipient also implemented an information system to track and ensure the accountability of the amounts subrecipients spent. • Management Reviews. Four recipients implemented supplemental management reviews of Section 1512 data at multiple levels of the review process. For example, one recipient required internal certifications following each level of review attesting to the accuracy, completeness, and timeliness of the reported data. • Independent Reviews. One recipient tasked its internal auditor with reviewing its jobs calculations and other data before forwarding it to the state reporting agency for submission to FederalReporting.gov. In addition, the state reporting agency revised its policies and procedures to include a mandatory review of the recipient’s reported number of jobs. Also, several recipients have assigned, hired, or were in the process of hiring staff to independently conduct or oversee data quality reviews. Automated Processes to Enhance Data Quality: • One recipient developed a computer program to electronically extract information needed for Section 1512 reporting, such as the amount of funds received/invoiced, from its accounting system to minimize human error associated with manual collection and processing. Effective Practices for Ensuring the Quality of Subrecipient Data: • One recipient forwarded OMB and Federal agency reporting guidance and developed supplemental subrecipient guidance. The recipient also had procedures to review subrecipient data, including: (1) performing reasonableness checks focused on the 10 largest subrecipients and Final Report Page 44 of 44 noted data outliers; (2) spot-checking individual subrecipient records; (3) validating incoming data from subrecipients; and (4) using recipient-wide overviews to ensure data quality. The recipient’s use of manual and automated processes provided enhanced control over the accuracy of subrecipient-reported data. • One recipient’s accountant performed a comprehensive review of all subrecipient expenses allocated to any Recovery Act project. An annual reconciliation of the records ensured that the total amount allocated to the projects matched the totals in the accounting system. The recipient also verified salary expenses using information extracted from an established system and reviewed financial and grant drawdown information and the reported number of jobs and project status. • Some recipients issued or were developing guidance on reporting requirements for their subrecipients. One of the recipients planned to issue guidance to address data elements that subrecipients had difficulty with or were confused about for the second quarterly report, such as the recognition of vendor jobs. Data Quality Issues We Identified By Applying Analytical Procedures: • One recipient reported the number of jobs out of proportion with the amount it reported as funds spent. When comparing jobs and funds spent data with another recipient’s data, we found that the other recipient reported twice the number of jobs even though it reported less than one-third of the funds spent than the recipient we reviewed had reported. Having identified this inconsistency, we researched the matter further and determined that the recipient had significantly under-reported the number of jobs. • One recipient did not identify obvious reporting errors related to the number of jobs and funds spent for its subrecipients. Our review of recipient reports easily identified an error for one subrecipient simply by comparing the estimated number of jobs to the amount of funds spent that was reported by the subrecipient. This analytical procedure highlighted an outlier when the subrecipient reported 3,200 jobs and no funds spent for the quarter.
Recovery Act Data Quality: Recipient Efforts to Report Reliable and Transparent Information
Published by the Department of Education, Office of Inspector General on 2010-09-13.
Below is a raw (and likely hideous) rendition of the original report. (PDF)