oversight

Implementation of the Digital Accountability and Transparency Act of 2014

Published by the National Science Foundation, Office of Inspector General on 2017-11-17.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

Implementation of the
Digital Accountability and Transparency Act
of 2014
REPORT PREPARED BY KEARNEY & COMPANY




November 17, 2017
OIG 18-2-001
AT A GLANCE
Implementation of the Digital Accountability and Transparency Act of 2014
Report No. OIG 18-2-001
November 17, 2017

AUDIT OBJECTIVE
The National Science Foundation Office of Inspector General engaged Kearney & Company (Kearney)
to conduct a performance audit of the National Science Foundation’s second quarter fiscal year 2017
spending data submitted under the Digital Accountability and Transparency Act of 2014 (DATA Act).
The objectives of the audit were to assess the accuracy, completeness, timeliness, and quality of NSF’s
financial and payment data reported to the public through USASpending.gov and to assess NSF’s
implementation and use of the Government-wide financial data standards. Kearney is responsible for
the attached auditor’s report and the conclusions expressed in this report. NSF OIG does not express
any opinion on the conclusions presented in Kearney’s audit report.

AUDIT RESULTS
Kearney found that the spending data NSF submitted did not meet the quality requirements. Kearney
identified inaccuracies in one or more data elements for 62.2 percent of transactions tested;
specifically, 43.3 percent of transactions tested had errors due to NSF’s reporting and 18.9 percent had
errors due to Government-wide reporting issues. Kearney also identified that 98.8 percent of
transactions omitted one or more of the required data elements; specifically, 41.7 percent were due to
NSF’s reporting and 57.1 percent were due to Government-wide reporting issues. Kearney also
identified that 0.8 percent of transactions were untimely. Finally, Kearney noted that NSF had taken
many steps to implement and use the Government-wide data standards; however, improvements are
needed. As a result, if uncorrected, these errors increase the risk that inaccurate data will be uploaded
to USASpending.gov, decreasing the reliability and usefulness of the data.

RECOMMENDATIONS
Kearney’s report contains four recommendations aimed at improving the accuracy, completeness,
timeliness, and quality of NSF’s data reported through USASpending.gov.

AUDITEE RESPONSE
NSF disagreed with the percentage of the NSF-attributed errors and with calculating the error rate at
the transaction level, rather than at the data element level. NSF generally agreed with all of the
recommendations but stated that it has sufficient reconciliation and quality control procedures in its
data submission processes. NSF’s response is included in its entirety in Appendix E.

FOR FURTHER INFORMATION, CONTACT US AT (703) 292-7100 OR OIG@NSF.GOV.
NSF 	 NATIONAL SCIENCE FOUNDATION
         OFFICE OF INSPECTOR GENERAL


 MEMORANDUM

 TO: 	           Dr. France A. Cordova 

                 Director 

                 National Science Foundation 


                Ms. Teresa Grancorvitz
                Acting Chief Financial Officer and Office Head
                Office of Budget, Finance, and Award Management

 FROM: 	        Mark Bell 

                Assistant Inspector General 

                Office of Audits 


 DATE: 	        November 17, 2017

 SUBJECT: 	 Audit Report No. 18-2-001, Implementation of the Digital Accountability and 

            Transparency Act of2014 


 This memo transmits Kearney & Company's (Kearney) audit report on the National Science
 Foundation's (NSF) implementation of the Digital Accountability and Transparency Act of2014
 (DATA Act). The objectives ofthe audit were to assess the accuracy, completeness, timeliness, and
 quality of NSF' s financial and payment data for the second quarter of fiscal year 2017 reported to the
 public through USASpending.gov and to assess NSF's implementation and use of the Government-wide
 financial data standards.

 Kearney's rep01t contains four recommendations aimed at improving the accuracy, completeness,
 timeliness, and quality ofNSF's data reported through USASpending.gov. NSF generally agreed with
 the recommendations. Kearney has included NSF's response to its draft report as Appendix E.

 OIG Oversight of Audit

 To fulfill our monitoring responsibilities, we:

    •	  reviewed Kearney's approach and planning of the audit;
    •	  evaluated the qualifications and independence of Kearney and its staff;
    •	  monitored the progress of the audit at key points;
    •	  coordinated periodic meetings with Kearney and NSF management, as necessary, to discuss
        audit progress, findings, and recommendations;
    • 	 reviewed Kearney's audit rep01t to ensure compliance with generally accepted government
        auditing standards; and
    • 	 coordinated issuance of the audit report.
Kearney is responsible for the attached auditor’s report and the conclusions expressed in this
report. We do not express any opinion on the conclusions presented in Kearney’s audit report.

In accordance with Office of Management and Budget Circular A-50, Audit Followup, please
provide a written corrective action plan to address the report recommendations. This corrective
action plan should detail specific actions and associated milestone dates. Please provide the
action plan within 60 calendar days of the date of this report.

We thank your staff for the assistance that was extended to Kearney and the OIG staff during this
audit. If you have any questions regarding this report, please contact me at (703) 292-7100.

Attachment

cc:
John Anderson                  Joanne Tornow
John Veysey                    Dorothy Aronson
Ann Bushmiller                 Avinash Tembulkar
Christina Sarris               Allison Lerner
Joan Ferrini-Mundy             Ken Chason
Fae Korsmo                     Susan Carnohan
Charisse Carney-Nunes          Marie Maguire
Michael Wetklow                Jannifer Jenkins
       National Science Foundation

       Office of Inspector General


Implementation of the Digital Accountability

 and Transparency Act of 2014 (DATA Act)

           Performance Audit


          Performance Audit Report


                        November 17, 2017




                                 David Zavada, Senior Partner
                                  1701 Duke Street, Suite 500
                                     Alexandria, VA 22314
                              703-931-5600, 703-931-3655 (fax)
                                   dzavada@kearneyco.com
        Kearney & Company’s TIN is 54-1603527, DUNS is 18-657-6310, Cage Code is 1SJ14
                                                         1701 Duke Street, Suite 500, Alexandria, VA 22314
                                                         PH: 703.931.5600, FX: 703.931.3655, www.kearneyco.com



Dr. France A. Córdova
Director
National Science Foundation
Office of the Director
2415 Eisenhower Avenue
Alexandria, VA 22314


Ms. Teresa Grancorvitz
Acting Chief Financial Officer and Office Head, Office of Budget, Finance, and Award
Management and Senior Accountable Official (SAO)
National Science Foundation
2415 Eisenhower Avenue
Alexandria, VA 22314


RE: 	 Audit of the National Science Foundation’s Implementation of the Digital Accountability
      and Transparency Act of 2014


Dear Dr. Córdova and Ms. Grancorvitz:

Kearney & Company, P.C. (Kearney) has performed an audit of the National Science
Foundation’s (NSF) implementation of the Digital Accountability and Transparency Act of 2014
(DATA Act). This performance audit, performed under Contract No. GS-00F-031DA, was
designed to meet the objective identified in the OBJECTIVE section of this report and further
defined in Appendix A: Purpose, Scope, and Methodology.

Kearney conducted this performance audit in accordance with Government Auditing Standards,
2011 Revision, issued by the Comptroller General of the United States. Those standards require
that Kearney plan and perform the audit to obtain sufficient, appropriate evidence to provide a
reasonable basis for the findings and conclusions based on the audit objectives. We believe that
the evidence obtained provides a reasonable basis for the findings and conclusions based on the
audit objectives.

Kearney appreciates the cooperation provided by NSF personnel during the audit.




Kearney & Company, P.C.
Alexandria, VA
November 17, 2017
                                                                                                             National Science Foundation
                                                                       Implementation of the Digital Accountability and Transparency Act
                                                                                                      FY 2017 Performance Audit Report




                                                   TABLE OF CONTENTS

                                                                                                                                    Page #

EXECUTIVE SUMMARY .......................................................................................................... 1

OBJECTIVE ................................................................................................................................. 2

BACKGROUND ........................................................................................................................... 2

   Guidance Related to Federal Agency Accountability and Transparency............................ 3

   DATA Act Submission and Certification................................................................................ 4

   The NSF’s Process for Generating the Data Act Submission ............................................... 7

AUDIT RESULTS ........................................................................................................................ 9

   Finding 1: Instances of Inaccurate Data Elements .............................................................. 10

   Finding 2: Instances of Incomplete Data Elements.............................................................. 13

   Finding 3: Instances of Untimely Transactions.................................................................... 16

   Finding 4: Issues with DATA Quality ................................................................................... 17

APPENDIX A: PURPOSE, SCOPE, AND METHODOLOGY............................................. 19

   Testing Limitations for Data Reported from Files E and F ................................................ 21

   Government-Wide Broker Data Reporting Issues............................................................... 21

   Prior Reports ........................................................................................................................... 23

   Work Related to Internal Controls........................................................................................ 23

   Use of Computer-Processed Data .......................................................................................... 24

   Detailed Sampling Methodology............................................................................................ 24

APPENDIX B: REQUIRED DATA ELEMENTS FOR FEDERAL AGENCY 

REPORTING .............................................................................................................................. 26

APPENDIX C: DATA ACT INFORMATION FLOW DIAGRAM...................................... 31

APPENDIX D: SUMMARY OF THE NATIONAL SCIENCE FOUNDATION’S DATA 

ACT RESULTS........................................................................................................................... 32

APPENDIX E: MANAGEMENT’S RESPONSE .................................................................... 38

APPENDIX F: KEARNEY’S RESPONSE .............................................................................. 43

APPENDIX G: ABBREVIATIONS AND ACRONYMS........................................................ 44

                                                                                        National Science Foundation
                                                  Implementation of the Digital Accountability and Transparency Act
                                                                                 FY 2017 Performance Audit Report




EXECUTIVE SUMMARY

The National Science Foundation’s (referred to as “NSF” in this document) Office of Inspector
General (OIG) engaged Kearney & Company, P.C. (referred to as “Kearney,” “we,” and “our” in
this document) to conduct a performance audit over NSF’s second quarter (Q2) fiscal year (FY)
2017 spending data submitted under the Digital Accountability and Transparency Act of 2014 1
(DATA Act). The DATA Act requires Federal agencies to report financial and spending
information to the public through USASpending.gov in accordance with Government-wide
financial data standards developed and issued by the Office of Management and Budget (OMB)
and the Department of the Treasury (Treasury). The Q2 FY 2017 spending data was the first data
that Federal agencies were required to submit in accordance with these financial standards. The
objectives of our performance audit were to review a statistically valid sample of NSF’s Q2 FY
2017 spending data; assess the accuracy, completeness, timeliness, and quality of the data
sampled; and assess NSF’s implementation and use of the Government-wide data standards.

Kearney noted that the spending data NSF submitted did not meet the quality requirements as
outlined by OMB. Several data elements were inaccurate, incomplete, or untimely because NSF
did not have sufficient quality control procedures and there were reporting issues at the
Government-wide reporting level. Kearney identified total inaccuracies in one or more data
elements for 62.2 percent of the selected samples tested; specifically 43.3 percent of transactions
had errors due to NSF’s attributed errors and 18.9 percent had errors due to Government-wide
reporting issues. We also identified that 98.8 percent of the transactions tested omitted one or
more of the required data elements; specifically 41.7 percent were due to NSF’s attributed errors
and 57.1 percent were due to Government-wide reporting issues. Finally, Kearney identified 0.8
percent of transactions that were not reported within 30 days after the end of the quarter.

As required by OMB guidance, we calculated our sampling error at the transaction level, rather
than the data element level. Therefore, if one or more of the data elements within a transaction
was inaccurate, then the entire transaction was deemed an error. Inaccurate, incomplete, and
untimely data may not reflect complete information for the end user. As a result, if the data
remains uncorrected, there is a risk that inaccurate and incomplete data will be uploaded to
USASpending.gov, decreasing the reliability and usefulness of the data. Finally, Kearney noted
that NSF had taken many steps to implement and use data standards required by Federal
guidance; however, improvements are needed.

As a result of our findings, we made four recommendations to improve NSF’s implementation of
the DATA Act. We provided these findings and recommendations, as well as a draft version of
this report, to management for comment. Management’s response is included in its entirety in
Appendix E.




1
    Public Law (P.L.) No. 113-101


                                                 1
                                                                                                National Science Foundation
                                                          Implementation of the Digital Accountability and Transparency Act
                                                                                         FY 2017 Performance Audit Report




OBJECTIVE

The objectives of this audit were to review a statistically valid sample of NSF’s Q2 FY 2017
spending data; assess the accuracy, completeness, timeliness, and quality of the data sampled;
and assess NSF’s implementation and use of the Government-wide data standards.

BACKGROUND

In 2016, the Federal Government spent more than $3 trillion in payments to vendors, contractors,
and grantees in the form of contracts, grants, loans, and other financial awards. 2 To increase the
transparency of and accountability for that spending, Congress passed the Federal Funding
Accountability and Transparency Act (FFATA) in 2006. 3 The act, as amended by the
Government Funding Transparency Act of 2008, 4 requires OMB to ensure the existence and
operation of a free, publicly accessible website containing data on Federal awards (e.g.,
contracts, loans, and grants). In order to comply with FFATA requirements, OMB launched the
USASpending.gov website.

In May 2014, the DATA Act was signed into law. The DATA Act amends and augments
FFATA to increase accountability, transparency, accessibility, quality, and standardization in
Federal spending data. The DATA Act requires agency financial and payment information to be
reported to the public through USASpending.gov in accordance with Government-wide financial
data standards developed and issued by OMB and Treasury, beginning with FY 2017 second
quarter data.

The DATA Act also requires that the Inspector General (IG) of each Federal agency periodically
review a statistically valid sample of the spending data submitted by its agency; assess the
completeness, timeliness, quality, and accuracy of the data sampled; and assess the agency’s
implementation and use of Government-wide financial data standards. The IGs are required to
submit to Congress and make publicly available a report of the results of these assessments. In
February 2017, the Council of the Inspectors General on Integrity and Efficiency (CIGIE),
Federal Audit Executive Council, DATA Act Working Group, in consultation with the
Government Accountability Office (GAO), developed the Inspector General Guide to
Compliance Under the DATA Act that provided OIGs with a framework for the work required by
the DATA Act. 5




2
  Department of the Treasury (Treasury), https://beta.usaspending.gov/#/ (accessed on September 26, 2017). This

amount includes total spending awarded to individuals, private contractors, and local Governments, and it excludes

the cost of running the Government and direct services (i.e., non-award spending or money that was not given out

through contracts, grants, direct payments, loans, or insurance).

3
  P.L. No. 109-282, § 1 to 4 (September 26, 2006)

4
  P.L. No. 110-252 (June 30, 2008)

5
  This report is the required report described in the DATA Act. For details regarding the scope and methodology,

including use of the Inspectors General Guide to Compliance Under the DATA Act (Treasury OIG, OIG-CA-17­
012, February 2017), see Appendix A of this report.



                                                         2
                                                                                        National Science Foundation
                                                  Implementation of the Digital Accountability and Transparency Act
                                                                                 FY 2017 Performance Audit Report




Guidance Related to Federal Agency Accountability and Transparency

OMB has published several sources of implementation guidance relating to FFATA and the
DATA Act to facilitate consistency and compliance across Federal agencies. In addition,
Treasury published technical guidance to assist agencies in understanding the various files and
data elements of the DATA Act submissions and the functionality of the Treasury’s Broker
(Broker). Notable sources of guidance available to agencies include:

   •	   OMB-M-10-06, Open Government Directive, provides guidance for Executive
        departments and agencies to implement the principles of transparency and open
        Government. This includes publishing Government information online and taking steps
        toward improving the quality of Government information published. OMB also published
        the Open Government Directive – Federal Spending Transparency and the Open
        Government Directive – Framework for the Quality of Federal Spending Information to
        provide guidance to Federal agencies in implementing the requirements in OMB-M-10­
        06.

   •	   OMB-M-15-12, Increasing Transparency of Federal Spending by Making Federal
        Spending Data Accessible, Searchable, and Reliable, provides guidance to Federal
        agencies on the existing data reporting requirements pursuant to the FFATA, as well as
        new requirements that agencies must employ pursuant to the DATA Act. This guidance
        requires agencies to establish a linkage between their financial, grants, and procurement
        management systems, which is a key component to tracking spending more effectively.
        OMB M-15-12 specifies that agency implementation plans should: 1) identify a Senior
        Accountable Official (SAO); 2) estimate resource requirements; 3) propose an
        implementation timeline; and 4) identify foreseeable challenges and resolutions.

   •	   OMB Management Procedures Memorandum (MPM) No. 2016-03, Additional Guidance
        for DATA Act Implementation: Implementing Data-Centric Approach for Reporting
        Federal Spending, offers additional guidance to Federal agencies on reporting Federal
        appropriations account summary-level and Federal award-level data to
        USASpending.gov in accordance with FFATA, as amended by the DATA Act. This
        memo also discusses the requirement for Federal agencies to associate data in agency
        financial systems with a unique award identification number (Award ID) to facilitate the
        linkage of these two levels of data.

   •	   OMB-M-17-04, Additional Guidance for DATA Act Implementation: Further
        Requirements for Reporting and Assuring Data Reliability, provides additional guidance
        to Federal agencies on reporting to USASpending.gov. This guidance offers specific
        technical assistance on certain matters such as awards involving intragovernmental
        transfers and quarterly SAO assurances.

   •	   Treasury issued the DATA Act Information Model Schema (DAIMS), Version.1.1, to be
        the authoritative source for the terms, definitions, formats, and structures of the data



                                                 3
                                                                                                 National Science Foundation
                                                           Implementation of the Digital Accountability and Transparency Act
                                                                                          FY 2017 Performance Audit Report




           elements. DAIMS provides requirements for Federal agencies on reporting to the DATA
           Act Broker.

      •	   Federal Spending Transparency Data Standards, in accordance with the DATA Act,
           OMB, and Treasury, established the set of 57 Government-wide data standards 6 for
           Federal funds made available to or expended by Federal agencies. Agencies are required
           to report financial data in accordance with these standards beginning in Q2 of FY 2017.

DATA Act Submission and Certification

To facilitate compliance with the DATA Act, which required data to be published on
USASpending.gov by May 8, 2017, Treasury mandated Federal agencies to submit FY 2017 Q2
data to the DATA Act Broker by April 30, 2017. Treasury developed an information technology
(IT) system, the DATA Act Broker, to facilitate the submission of data for the DATA Act.
Agencies are required to use the Broker 7 to upload three files containing data from their internal
systems and records. In addition, agencies use the Broker to extract award and sub-award
information from existing Government-wide reporting systems to generate four additional files.
The SAO is then required to certify and submit all seven of the files in the Broker. Exhibit 1 and
Exhibit 2 provide details on the seven files.

                                       Exhibit 1: Agency-Created Files
     DATA Act
     Submission                                               File Description
        File
                       File A provides information about how budgetary resources are made
                       available and the status of budgetary resources at the end of the reporting
    File A –
                       period. File A includes six of the 57 required data elements, including the
    Appropriations
                       amount appropriated and obligated 8 during the FY. The information in File
    Account Detail
                       A is reported for each Treasury Account Symbol (TAS). 9 File A data is
                       reported at the summary level, rather than the individual transaction level.




6
  The 57 standard data elements, including their definitions, can be found in Appendix B of this report. They are also

available at https://fedspendingtransparency.github.io/data-elements/ (as of September 14, 2017).

7
  OMB MPM 2016-03, Additional Guidance for DATA Act Implementation: Implementing Data-Centric Approach 

for Reporting Federal Spending, requires agencies to submit data required by the DATA Act directly to Treasury.

Treasury issued the DAIMS, v.1.1, which directed agencies to complete the submission through the Broker.

8
  Obligations are definite commitments that create a legal liability of the Government for payment.

9
  A TAS represents individual appropriation, receipt, and other funds made available to Federal agencies. The TAS

is used to segregate funds to ensure that funds are spent in accordance with law.



                                                          4
                                                                                                  National Science Foundation
                                                            Implementation of the Digital Accountability and Transparency Act
                                                                                           FY 2017 Performance Audit Report




   DATA Act
   Submission                                                  File Description
      File
                       File B includes the same six data elements as File A; however, the
 File B – Object
                       information in File B is presented by program activity 10 and object class,
 Class and
                       which represent an additional two required data elements. 11 Similar to File
 Program
                       A, File B data is reported at the summary level, rather than the individual
 Activity Detail
                       transaction level.
                       File C includes transaction-level information for all awards, procurement,
                       and financial assistance (e.g., grants and cooperative agreements) processed
                       during the quarter. This includes modifications to existing awards. Payroll
 File C – Award        actions, classified transactions, and interagency awards are excluded from
 Financial Data        agency submissions. File C includes four of the 57 required data elements:
                       the TAS, Award Identification Number, Obligation, and Object Class. All
                       records in File C should be included in either File D1 or D2, which are
                       described below.
Source: Generated by Kearney based on OMB and Treasury guidance.

                               Exhibit 2: DATA Act Broker-Generated Files
   DATA Act
   Submission                                              File Description
      File
                       File D1 includes transaction-level information for all procurement awards
                       processed during Q2 of FY 2017. File D1 comprises 41 of the 57 required
                       data elements, including Award ID, Award Description, Place of
 File D1–              Performance, and Period of Performance. Records can be traced from File
 Award and             D1 to File C using the Award ID.
 Awardee
 Attributes            When agencies generate File D1 in the DATA Act Broker, the Broker pulls
 (Procurement)         the information from the Government-wide Treasury Federal Procurement
                       Data System – Next Generation (FPDS-NG). Agencies are required to
                       report all contracts with an estimated value over $3,500 and modifications
                       to those contracts into FPDS-NG.
 File D2–              File D2 includes transaction-level information for all financial awards
 Award and             processed during Q2 of FY 2017. File D2 comprises 38 of the 57 required
 Awardee               data elements, including Award ID, Awardee/Recipient Legal Entity Name,
 Attributes            Place of Performance, and Period of Performance. Records can be traced
 (Financial            from File D2 to File C using the Award ID.
 Assistance)

10
   A specific activity or project as listed in the program and financing schedules of the annual budget of the United

States Government.

11
   Object classifications identify the kinds of services, materials, and other resources for which U.S. Government

payments are made. They cover all types of obligations, payments, current operating expenses, and capital outlays.

The basic object classes are prescribed by OMB Circular A-11, Preparation, Submission and Execution of the

Budget.



                                                           5
                                                                                               National Science Foundation
                                                         Implementation of the Digital Accountability and Transparency Act
                                                                                        FY 2017 Performance Audit Report




     DATA Act
     Submission                                         File Description
        File
                      When agencies generate File D2 in the Broker, the Broker pulls the
                      information from the Government-wide Treasury Award Submission Portal
                      (ASP) for all awards reported during Q2. Agencies are required to report all
                      financial assistance awards of $25,000 or more to the ASP monthly.
                      File E includes information on organizations which received procurement
                      or financial assistance awards during Q2 of FY 2017. File E includes five of
                      the 57 required data elements. Three of these data elements are used to
                      identify the awardee and are included for all organizations with awards in
                      Q2. The remaining two required data elements are only reported for
                      organizations which receive over 80% or $25,000,000 of their annual gross
 File E –
                      revenues in Federal funding. 12 These elements are the names of the five
 Additional
                      most highly compensated officers and the total compensation for these
 Awardee
                      individuals.
 Attributes
                      When agencies generate File E in the Broker, the Broker pulls the
                      information from the General Services Administration’s (GSA) System for
                      Award Management (SAM). All organizations which currently or want to
                      conduct business with the Federal Government must have an active
                      registration in SAM.
                      File F includes information on certain organizations which received
                      procurement or financial assistance sub-awards during Q2 of FY 2017.

                      When agencies generate File F in the Broker, the Broker pulls information
                      from the FFATA Sub-award Reporting System (FSRS). If a prime
                      contractor issues a sub-award for more than $30,000 or if a prime grantee
 File F –
                      issues a sub-award for more than $25,000, the prime contractor/grantee
 FFATA Sub-
                      must report the sub-award in FSRS. In addition to details about the sub-
 award
                      award, the prime contractor/grantee is also required to report information on
 Attributes
                      the executive compensation of the organization to which the sub-award was
                      issued.

                      Because File F did not contain any required data elements as part of the
                      DATA Act file submission, we did not perform any testing procedures over
                      those data elements reported.
Source: Generated by Kearney based on OMB and Treasury guidance.

The responsibility for ensuring the accuracy of all files lies with an agency’s DATA Act SAO..
Each agency is required to designate a SAO, who is a senior official in the agency with the


12
  The Office of Federal Procurement Policy Act, as amended, 41 United States Code (U.S.C.) 401 et.seq., and
Federal Acquisition Regulation (FAR), Subpart 52.204-10.


                                                        6
                                                                                            National Science Foundation
                                                      Implementation of the Digital Accountability and Transparency Act
                                                                                     FY 2017 Performance Audit Report




ability to coordinate across multiple communities and Federal Lines of Business. 13 Although
OMB guidance does not name a position within the agency that should be the SAO, it states that
the SAO should be accountable for the quality and objectivity of internal controls over spending
information. The SAO must provide reasonable assurance over the quality of the data submitted
and document his/her assurance by certifying the DATA Act submission in the Broker. OMB
guidance directs SAOs to verify that there are the required linkages among all of the files prior to
certification. For example, for those awardees included in File E, there should be transactions in
Files C and D1 or C and D2. OMB guidance further states that when certifying the DATA Act
submission, SAOs should “provide reasonable assurance that their internal controls support the
reliability and validity of the agency account-level and award-level data.” 14

The NSF’s Process for Generating the Data Act Submission

At NSF, the Deputy Head of the Office of Budget, Finance, and Award Management serves as
the SAO. 15 Accordingly, the Office of the Chief Financial Officer (OCFO) is responsible for the
implementation of the DATA Act at NSF. On April 28, 2017, NSF uploaded and certified the
required files in the DATA Act Broker. The data needed to create Files A, B, and C resides in
NSF’s core financial management system, iTRAK, and is generated by NSF. Using the DATA
Act Broker, NSF extracted and generated the Files D1, D2, E, and F for submission on April 28,
2017. As noted above, the source for these files are Government-wide reporting systems. Exhibit
3 outlines how each file is populated into each of these systems.

   Exhibit 3: Process for Reporting Information in Government-wide Systems which are the 

                              Sources for Files D1, D2, E, and F

   DATA Act       Government-
                                  Process for Reporting Data to the Government-wide Source
  Submission      wide Source
                                                                System
       File          System
                                  As noted above, File D1 is generated based on the information
                                  included in FPDS-NG. iTRAK is configured so that when a
                                  procurement is recorded in iTRAK, certain fields will be
 File D1–                         automatically transmitted to FPDS-NG, creating a new record
 Award and                        in FDPS-NG. However, this process does not automatically
 Awardee          FPDS-NG         populate all required fields in FPDS-NG. The Contracting
 Attributes                       Officer (CO) enters the remaining fields directly into FPDS­
 (Procurement)                    NG. Once the CO completes all of the required fields in FPDS­
                                  NG, he/she clicks the Verify button. The action must pass
                                  automatic edit checks in FPDS-NG to be recorded, which is
                                  noted by a “Final” status.




13
   OMB-10-06, Section 2 a., Open Government Directive
14
   OMB-M-17-04, Additional Guidance for DATA Act Implementation: Further requirements for Reporting and
Assuring Data Reliability
15
   The NSF’s SAO is currently the Acting Chief Financial Officer (CFO).


                                                     7
                                                                                           National Science Foundation
                                                     Implementation of the Digital Accountability and Transparency Act
                                                                                    FY 2017 Performance Audit Report




   DATA Act        Government-
                                      Process for Reporting Data to the Government-wide Source
  Submission       wide Source
                                                               System
       File          System
 File D2–
                                     As noted above, File D2 is generated based on the information
 Award and
                                     included in ASP. NSF’s Awards system is an internal system to
 Awardee
                   ASP               record financial awards, which interfaces with iTRAK.
 Attributes
                                     Subsequently, the financial award data reported in Awards is
 (Financial
                                     uploaded by NSF to ASP on a monthly basis.
 Assistance)
 File E –
 Additional
                   SAM               Populated by vendors/awardees.
 Awardee
 Attributes
 File F –          FFATA Sub-
 FFATA Sub-        award
                                     Populated by awardees.
 award             Reporting
 Attributes        System
Source: Generated by Kearney based on OMB and Treasury guidance.




                                                    8
                                                                                                 National Science Foundation
                                                           Implementation of the Digital Accountability and Transparency Act
                                                                                          FY 2017 Performance Audit Report




AUDIT RESULTS

Kearney noted that the data reported in Files A and B was accurate, complete, timely, and met
the quality requirements outlined by OMB. We noted that there are six elements that are tested
within Files A and B; therefore, they are not tested as a part of Files C, D1/D2, or E. Kearney
noted that File F did not contain any required data elements as part of the DATA Act file
submission. Thus, we did not perform any testing procedures over those data elements reported
in File F.

We reviewed a statistically valid sample of spending data (254 transactions) that NSF submitted
under the DATA Act to assess the data reported in Files C, D1, D2, and E, and Kearney noted
that data reported in Files C, DI, D2, and E was inaccurate, incomplete, and untimely.
Accordingly, these items did not meet quality requirements as outlined by OMB. 16 Specifically,
of the 6,603 transactions included in NSF’s File C submission, Kearney reviewed a sample of
254 transactions and determined that 62.2 percent of transactions were inaccurate, 98.8 percent
of transactions were incomplete, and 0.8 percent of transactions were untimely. Each transaction
reported (Files C, D1/D2, and E) may contain up to 51 required data elements (See Appendix B
for the data element mapping) that may be tested for completeness, accuracy, and timeliness. If
any of the data elements result in an error (even if it is just one), this causes the entire transaction
to be considered an error.

We identified three types of errors, two of which are Government-wide reporting errors (Broker
errors and ASP errors) at the Treasury level, which are both outside of NSF’s control. See
Appendix A for a description of broker data reporting issues. Certain other data elements were
derived by ASP (i.e., Funding Office Name and Awarding Office Name). As a result, we
calculated separate rates for NSF errors, the Broker errors, and ASP derived errors to distinguish
between the nature and extent of the variances identified at the transaction level. For accuracy,
43.3 percent of the transactions had errors due to NSF, while 18.5 percent of the transactions had
errors due to ASP, and 0.4 percent of the transactions had errors due to the Treasury Broker. For
completeness, 41.7 percent of the transactions tested were incomplete due to NSF’s attributed
errors, while 57.1 percent were incomplete due to ASP. See Exhibit 4 for summary results of
testing.

Finally, Kearney noted that NSF had taken many steps to implement and use data standards
required by Federal guidance; however, improvements are needed.




16
  OMB’s Open Government Directive – Federal Spending Transparency requires agencies to report on three key
metrics: timeliness, completeness, and accuracy. These are the metrics that will be used to determine the quality of
information.


                                                          9
                                                                                                 National Science Foundation
                                                           Implementation of the Digital Accountability and Transparency Act
                                                                                          FY 2017 Performance Audit Report




                                  Exhibit 4: Summary Results of Testing
           Results                          Accuracy   Completeness Timeliness                              Quality
Number of Transactions
                                                   96                   3                 252                  96
Without Errors
                 Number of Transactions With Errors in One of More Data Element 17
Number of Transactions with
Errors Due to Data Elements                       110                 106                   2                 110
Controlled by NSF
Number of Transactions with
                                                   1                    0                   0                   1
Errors Due to the Broker 18
Number of Transactions with
Errors Due to Data Elements                        47                 145                   0                  47
Derived by ASP
Total Errors (NSF, Broker, and
                                                  158                 251                   2                 158
ASP)
Total Number of Transactions
                                                  254                 254                  254                254
Tested
Percentage Error Due to Data
                                                43.3%               41.7%                0.8%               43.3%
Elements Controlled by NSF
Percentage Error Due to the                      0.4%                0.0%                0.0%                0.4%
Broker
Percentage Error Due to Data                    18.5%               57.1%                0.0%               18.5%
Elements Derived by ASP
Total Percentage Errors                         62.2%               98.8%                0.8%               62.2%
Source: Generated by Kearney based upon the results of testing.

Finding 1: Instances of Inaccurate Data Elements

Condition: To test the accuracy of the award-level transactions in Files C, D1, D2, and E,
Kearney selected a sample of 254 transactions (244 Financial Assistance Identifier Number
[FAIN] transactions and 10 Procurement Instrument Identifier Data [PIID] transactions) included
in NSF’s File C submission. For each sampled transaction, Kearney evaluated whether the 51
standard data elements included in Files C, D1, D2, and E were accurate (see Appendix B).
Specifically, Kearney obtained supporting documentation, such as a contract, modification, or
system screen prints, and agreed the data included in the Files C through E submission to the
supporting documents. We determined that 158 transactions (62.2 percent) contained accuracy
errors with one or more data elements. The detailed results are presented in Exhibit 5.




17
   Kearney identified three types of errors during testing. ASP and Broker errors (Government-wide Reporting
Issues) consists of transactions that had discrepancies for ASP or Broker data elements only (no NSF-controlled
errors). NSF-controlled errors consists of transactions that had one or more errors related to NSF populated data
elements.
18
   Refer to Appendix A for the full description of the Government-wide Treasury Broker-related errors.


                                                         10
                                                                                               National Science Foundation
                                                         Implementation of the Digital Accountability and Transparency Act
                                                                                        FY 2017 Performance Audit Report




For example, for 56 of the tested transactions (22 percent), Kearney noted that the legal entity
address in File D2 was inaccurate. Thirty-four of the tested transactions (13.4 percent) contained
inaccurate elements within the Primary Place of Performance Address, which is made up five
individual fields, including Primary Place of Performance City Name and Primary Place of
Performance ZIP + 4, which contained 25 and six inaccuracies, respectively. Further, for 28 of
the tested transactions (11 percent), the Award Type did not agree to the Awards system.

Exhibit 5 shows the data elements with accuracy errors. See Appendix D for the testing results
of all 57 data elements by file name.

                 Exhibit 5: Accuracy Results by Data Element (Errors Only)
                                                          Number of         Number of
                    Data Element                         Transactions      Transactions
                                                        without Errors     with Errors
Action Date                                                        253                  1
Award Type                                                         226                 28
Awarding Office Name^                                              103                151
Business Types                                                     253                  1
Catalog of Federal Domestic Assistance Title^                      253                  1
Current Total Value of Award*                                      253                  1
Federal Action Obligation                                          250                  4
Funding Office Code                                                252                  2
Funding Office Name^                                               103                151
Legal Entity Address                                               198                 56
Legal Entity Congressional District^                               246                  8
Potential Total Value of Award*                                    253                  1
Primary Place of Performance Address                               220                 34
Primary Place of Performance Congressional District^               247                  7
Primary Place of Performance County Name*                          103                151
Ultimate Parent Legal Entity Name**                                208                 46
Ultimate Parent Unique Identifier**                                218                 36
Source: Generated by Kearney based upon the results of testing.
*Denotes a Government-wide Treasury Broker data element.

**For FAINs, the data element is only reported on File E (these elements are derived by SAM and were not included

within our sampling error rate calculation).

^Denotes an ASP derived data element.


Cause: Insufficient quality control procedures led to inaccuracies reported between Files C,
D1/D2, and E. While NSF did make corrections based on erroneous transactions, there was not a
complete reconciliation performed which showed effective linkage between Files C to D1/D2
prior to submission. NSF elected to rely on the Broker outputs/results to identify data
discrepancies, which resulted in inaccuracies with many of the elements and transactions.




                                                       11
                                                                                       National Science Foundation
                                                 Implementation of the Digital Accountability and Transparency Act
                                                                                FY 2017 Performance Audit Report




Kearney also noted that certain data elements are derived within ASP and FPDS-NG from other
NSF controlled data elements. For example, within FPDS, the Legal Entity Congressional
District is populated based on the Legal Entity Zip Code. If NSF enters the incorrect zip code in
FPDS, then the Legal Entity Congressional District will be inaccurate.

Certain derived fields are unable to be populated by NSF, thus being outside of NSF’s control.
For example, NSF entered the Awarding Office Code, from which the Broker should derive the
Awarding Office Name within ASP. In this case, the Broker did not populate the required
information, resulting in an ASP error.

Finally, it is the awardee’s responsibility to report award and executive compensation
information in SAM (data within File E). However, it is NSF’s responsibility to ensure this
information is correct and accurate as a part of its file submissions.

Effect: Due to the inaccurate transactions reported as part of the DATA Act file submissions,
Government-wide data reporting issues, and ASP’s deriving certain data elements, the data may
not accurately reflect factual information for the end user. As a result, there is a risk that
inaccurate data will be uploaded to USAspending.gov, decreasing the reliability and usefulness
of the data.




                                               12
                                                                                                   National Science Foundation
                                                             Implementation of the Digital Accountability and Transparency Act
                                                                                            FY 2017 Performance Audit Report




Finding 2: Instances of Incomplete Data Elements

Condition: To test the completeness of the award-level transactions in Files C, D1, D2, and E,
Kearney selected a sample of 254 (244 FAINs and 10 PIIDs) transactions included in NSF’s File
C submission. We tested each transaction to determine whether the transaction included all
applicable data elements required by the DATA Act. We noted that 251 transactions (98.8
percent) had completeness errors with one or more of the 51 required data elements at the
transaction level; specifically 41.7 percent were due to NSF’s attributed errors and 57.1 percent
were due to Government-wide reporting issues. Of the 254 transactions, 95 (37 percent) were
incomplete because they were included in File C but not in Files D1 or D2. 19 The remaining 151
transactions were included in Files C and D1 or D2 but contained incomplete information in one
or more of the required data elements. For example, the ultimate parent legal entity name field in
File D1 was blank for seven tested procurement awards transactions. Exhibit 6 illustrates the
specific completeness issues encountered during testing.

              Exhibit 6: Completeness Results by Data Element (Errors Only) 20
                                                           Number of            Number of
                                        Number of      Transactions with Transactions with
                                       Transactions Incomplete Data             Errors for
            Data Element
                                         without        Elements Due to        Reasons other
                                         Errors        File D1/D2 being      than File D1/D2
                                                          Incomplete        being Incomplete
Action Date                                159                95                    0
Action Type                                159                95                    0
Amount of Award                            161                93                    0
Award Description                          159                95                    0
Award Modification/Amendment
                                           159                95                    0
Number
Award Type                                 159                95                    0
Awardee/Recipient Legal Entity Name        159                95                    0
Awardee/Recipient Unique Identifier        159                95                    0
Awarding Office Code                       159                95                    0
Awarding Office Name^                       8                 95                   151
Awarding Sub Tier Agency Code              159                95                    0
Awarding Sub Tier Agency Name              159                95                    0
Business Types                             161                93                    0
Catalog of Federal Domestic
                                           161                93                    0
Assistance Number
Catalog of Federal Domestic
                                           161                93                    0
Assistance Title^

19
   Kearney did not consider these as exceptions for accuracy, as it would result in a double-counting of errors. In
addition, since data was not provided, we could not determine the accuracy of the missing elements.
20
   If a transaction contained multiple errors, it would appear as a part of multiple data elements, therefore causing the
total amounts to be larger than the sample.


                                                           13
                                                                                               National Science Foundation
                                                         Implementation of the Digital Accountability and Transparency Act
                                                                                        FY 2017 Performance Audit Report




                                                                         Number of                    Number of
                                                 Number of           Transactions with             Transactions with
                                                Transactions         Incomplete Data                   Errors for
              Data Element
                                                  without             Elements Due to                Reasons other
                                                  Errors             File D1/D2 being               than File D1/D2
                                                                        Incomplete                 being Incomplete
Current Total Value of Award*                         252                    2                             0
Federal Action Obligation                             159                   95                             0
Funding Agency Code                                   159                   95                             0
Funding Agency Name                                   159                   95                             0
Funding Office Code                                   159                   95                             0
Funding Office Name^                                   8                    95                           151
Funding Sub Tier Agency Code                          159                   95                             0
Funding Sub Tier Agency Name                          159                   95                             0
Legal Entity Address                                  159                   95                             0
Legal Entity Congressional District^                  159                   95                             0
Legal Entity Country Code                             159                   95                             0
Legal Entity Country Name                             252                    2                             0
North American Industrial
                                                      252                        2                            0
Classification System Code
North American Industrial
                                                      252                        2                            0
Classification System Description
Ordering Period End Date                              252                        2                            0
Period of Performance Current End
                                                      159                       95                            0
Date
Period of Performance Potential End
                                                      252                        2                            0
Date
Period of Performance Start Date                      159                       95                            0
Potential Total Value of Award*                       252                        2                            0
Primary Place of Performance Address                  159                       95                            0
Primary Place of Performance
                                                      159                       95                            0
Congressional District^
Primary Place of Performance Country
                                                      159                       95                            0
Code
Primary Place of Performance County
                                                      10                         0                           244
Name*
Record Type                                           161                       93                            0
Ultimate Parent Legal Entity Name**                   211                        2                           41
Ultimate Parent Unique Identifier**                   222                        2                           30
Source: Generated by Kearney based upon the results of testing.
*Denotes a Government-wide Treasury Broker data element.

**For FAINs, the data element is only reported on File E (these elements are derived by SAM and were not included

within our sampling error rate calculation).

^Denotes an ASP derived data element.





                                                       14
                                                                                       National Science Foundation
                                                 Implementation of the Digital Accountability and Transparency Act
                                                                                FY 2017 Performance Audit Report




Cause: Insufficient quality control procedures led to incomplete information being reported
between Files C, D1/D2, and E. The data was ultimately reported incorrectly within FPDS/ASP
due to NSF not performing reconciliations between Files C to D1/D2 and E to identify
incomplete transactions, whether fully or partially omitted.

Incomplete transactions that were identified within File D1 (two transactions out of 10) were due
to timing issues (see the Instances of Untimely Transactions finding for further details).

Management indicated that 93 of the 244 FAIN transactions (38 percent), which were in Files C
and not in File D2, were accounting adjustments. These adjustments were made within the
financial system (iTRAK), however, they were not recorded within the Awards system; thus,
they did not appear in File D2.

Effect: Due to the incomplete transactions or data elements omitted as part of the DATA Act file
submissions, the data may not reflect factual and complete information for the end user. As a
result, NSF runs the risk that incomplete data will be uploaded to USAspending.gov, decreasing
the reliability and usefulness of the data.




                                               15
                                                                                       National Science Foundation
                                                 Implementation of the Digital Accountability and Transparency Act
                                                                                FY 2017 Performance Audit Report




Finding 3: Instances of Untimely Transactions

Condition: To test the timeliness of the award-level transactions in Files C, D1, D2, and E,
Kearney selected a sample of 254 (244 FAINs and 10 PIIDs) transactions included in NSF’s File
C submission. For each sampled transaction, we determined whether the information for the
transaction was reported within 30 days after quarter-end. Kearney noted while most of NSF’s
data was reported timely, two transactions (0.8 percent) were not recorded in the files within 30
days.

Cause: NSF used the Broker to generate Files D1, D2, E, and F for submission and certification
on April 30, 2017, which is 30 days after the end of Q2. At that time, the Broker created Files D1
and D2 based upon the information included in FPDS-NG and ASP, respectively. Records,
which were not in these systems, were not included in the DATA Act files. We noted two of the
10 transactions sampled were not included in File D1 because NSF did not report them timely to
FPDS-NG. Kearney reviewed the source documentation for the two untimely samples and
determined that they represented actual Q2 activity, which should have been reported in File D1.

Effect: Most of NSF’s data was recorded timely; however, when transactions are not reported
timely in NSF’s DATA Act file submissions, data may not properly reflect information for that
time. As a result, NSF runs the risk that incomplete data will be uploaded to USAspending.gov,
decreasing the reliability and usefulness of the data.




                                                16
                                                                                       National Science Foundation
                                                 Implementation of the Digital Accountability and Transparency Act
                                                                                FY 2017 Performance Audit Report




Finding 4: Issues with DATA Quality

Condition: We determined that if a transaction was inaccurate, the data reported would not
include the proper conditions to meet quality standards as established by OMB. Of the 254
sampled transactions, 158 (62.2 percent) contained errors with the accuracy, completeness,
and/or timeliness of one or more data elements reported in Files C, D1, D2, or E. These errors
occurred despite the fact that Kearney noted that NSF had taken many steps to successfully
implement and use the data standards. For example, NSF took steps to enhance its ability to
compile, analyze, and reconcile data from multiple sources. However, improvements are needed
to reduce data quality issues and fully implement the data standards for all transactions.

Cause: Quality issues primarily occurred because management had insufficient quality control
procedures to ensure that data was entered into ASP and FPDS-NG accurately, completely, and
timely.

NSF populates certain data elements with ASP and FPDS-NG, which subsequently triggers other
elements to be derived, such as Funding Office Name and Awarding Office Name. NSF cannot
correct discrepancies within these data elements (i.e., ASP and FPDS-NG derived fields) because
they are not controlled by NSF.

Effect: Due to the inaccurate transactions reported as part of the DATA Act file submissions,
Government-wide data reporting issues, and ASP’s deriving certain data elements, the data may
not accurately reflect quality information for the end user. As a result, NSF runs the risk that
inaccurate data will be uploaded to USAspending.gov, decreasing the reliability and usefulness
of the data.




                                               17
                                                                                      National Science Foundation
                                                Implementation of the Digital Accountability and Transparency Act
                                                                               FY 2017 Performance Audit Report




Audit Result Recommendations

Recommendation 1: Kearney recommends that NSF develop and implement procedures to
validate the accuracy of the data reported to the FPDS-NG and ASP, including procedures to
ensure the accuracy of data entered into NSF-owned systems, which interface with FPDS-NG
and ASP.

Recommendation 2: Kearney recommends that NSF perform an assessment over the linkage
between Files C to D1/D2 to ensure compliance with DATA Act requirements and standards.
NSF should implement controls to confirm the completeness of data entered into NSF-owned
systems (iTRAK and Awards), which is submitted to FPDS-NG and ASP on a periodic basis.

Recommendation 3: Kearney recommends that NSF develop and implement a process (i.e.,
perform reconciliations) to enter procurement transactions into the Government-wide FPDS-NG
within 30 days of the end of the quarter.

Recommendation 4: Kearney recommends that NSF expand and improve upon existing internal
quality control procedures to confirm the accuracy and integrity of the data prior to submission
to allow the SAO to appropriately certify all data included in files.




                                               18
                                                                                               National Science Foundation
                                                         Implementation of the Digital Accountability and Transparency Act
                                                                                        FY 2017 Performance Audit Report




APPENDIX A: PURPOSE, SCOPE, AND METHODOLOGY

Purpose, Scope, and Methodology

The Digital Accountability and Transparency Act of 2014 21 (DATA Act) requires each Federal
agency Inspector General (IG) to review a statistically valid sample of the spending data
submitted by its agency; assess the completeness, timeliness, quality, and accuracy of the data
sampled; and assess the agency’s implementation and use of Government-wide financial data
standards. The Office of Inspectors General (OIG) are required to submit to Congress and make
publicly available a report of the results of the assessment.

The objective of this audit was to comply with the aforementioned requirement. An external
audit firm, Kearney & Company, P.C. (referred to as “Kearney,” “we,” and “our” in this
document), acting on behalf of the National Science Foundation (NSF) OIG, performed the
audit.

Kearney conducted fieldwork for this performance audit from May through October 2017 in the
Washington, D.C. metropolitan area. The audit was conducted in accordance with the
Government Accountability Office’s (GAO) Government Auditing Standards, 2011 Revision.
These standards require that Kearney plan and perform the audit to obtain sufficient, appropriate
evidence to provide a reasonable basis for the findings and conclusions based on the audit
objective. We believe that the evidence obtained provides a reasonable basis for the findings and
conclusions based on the audit evidence.

In February 2017, the Council of the Inspectors General on Integrity and Efficiency’s (CIGIE)
Federal Audit Executive Council, in consultation with GAO, published the Inspectors General
Guide to Compliance Under the DATA Act, 22 which served to provide IGs with a baseline
framework for DATA Act compliance reviews.

According to the Inspectors General Guide to Compliance Under the DATA Act, to accomplish
the objectives of the DATA Act compliance review, IGs should:

       •	   Obtain an understanding of any regulatory criteria related to its agency’s responsibilities
            to report financial and award data under the DATA Act.
       •	   Assess its agency’s systems, processes, and internal controls in place over data

            management under the DATA Act.

       •	   Assess the general and application controls pertaining to the financial management
            systems (e.g., grants, loans, procurement) from which the data elements were derived and
            linked.
       •	   Assess its agency’s internal controls in place over the financial and award data reported
            to USASpending.gov per Office of Management and Budget (OMB) Circular A-123.


21
     Public Law (P.L.) No. 113-101

22
     Department of the Treasury (Treasury) OIG Report No. OIG-CA-17-012 (February 27, 2017)



                                                       19
                                                                                                 National Science Foundation
                                                           Implementation of the Digital Accountability and Transparency Act
                                                                                          FY 2017 Performance Audit Report




     •	   Review a statistically valid sample from fiscal year (FY) 2017 second quarter (Q2)
          financial and award data submitted by the agency for publication on
          www.USASpending.gov.
     •	   Assess the completeness, timeliness, quality, and accuracy of the financial and award
          data sampled:
          - Completeness is measured in two ways: 1) all transactions that should have been
              recorded are recorded in the proper reporting period and 2) as the percentage of
              transactions containing all applicable data elements required by the DATA Act.
          - Timeliness is measured as the percentage of transactions reported within 30 days of
              quarter-end.
          - Accuracy is measured as the percentage of transactions that are complete and agree
              with the systems of record or other authoritative sources.
          - Quality is defined as a combination of utility, objectivity, and integrity.
     •	   Assess its agency’s implementation and use of the 57 data definition standards

          established by OMB and Treasury.


In accordance with the Inspectors General Guide to Compliance Under the DATA Act, the scope
of this audit was NSF’s submission of FY 2017 Q2 data. The Inspectors General Guide to
Compliance Under the DATA Act stated: “the [OIG] engagement team, to the extent possible,
should adhere to the overall methodology, objectives, and review procedures outlined in this
guide. The engagement team should not hesitate to modify this guide based on specific systems
and controls in place at its agency, but must use professional judgment when designing
alternative review procedures.” Generally, Kearney conducted our audit based upon this
guidance. Professional judgement was used to customize certain recommended testing
procedures based on NSF’s environment, systems, and data.

To obtain background information, Kearney researched and reviewed Federal laws and
regulations, as well as prior GAO audit reports. We also reviewed the United States Code
(U.S.C.), OMB Circulars and Memoranda, guidance published by Treasury, and information
available on NSF’s intranet.

Kearney met with NSF officials to gain an understanding of the processes used to implement and
use the data standards. Specifically, we obtained an understanding of the processes used to create
and perform quality controls on the DATA Act submission. This included understanding the
systems used to process procurement and financial assistance awards. Kearney also obtained an
understanding of processes to record procurement and financial assistance awards in NSF’s
systems and other Federal systems.

The Inspectors General Guide to Compliance Under the DATA Act instructed audit teams to
assess the agencies’ use and implementation of 57 standard data elements. 23 Six of these data
elements are reported at the summary level in Files A or B, rather than the individual transaction
level. As reported in the AUDIT RESULTS section of this report, to test these data elements,

23
  The 57 standard data elements, including their definitions, are included in Appendix B of this report. They are
also available at https://fedspendingtransparency.github.io/data-elements/ (as of September 14, 2017).


                                                         20
                                                                                                National Science Foundation
                                                          Implementation of the Digital Accountability and Transparency Act
                                                                                         FY 2017 Performance Audit Report




Kearney tested procedures implemented by NSF to confirm the validity and accuracy of these six
account summary-level data elements. Specifically, we confirmed that the data was appropriately
linked between Files A and B and Standard Form (SF)-133, Report on Budget Execution and
Budgetary Resources. For the remaining 51 data elements, Kearney selected a sample of
individual transactions included in NSF’s File C submission. See additional details in the
Detailed Sampling Methodology section of this appendix.

Testing Limitations for Data Reported from Files E and F

As noted above in Exhibit 2, File E contains additional awardee attribute information extracted
from the System for Award Management (SAM) via the Treasury Broker (Broker), and File F
contains sub-award attribute information extracted from the Federal Funding Accountability and
Transparency Act of 2006 (FFATA) Sub-award Reporting System (FSRS) via the Broker. It is
the prime awardee’s responsibility to report sub-award and executive compensation information
in SAM and FSRS. Data reported from these two award reporting systems is generated in the
Broker for display on USASpending.gov. As outlined in OMB’s Management Procedures
Memorandum (MPM) 2016-03, the authoritative sources for the data reported in Files E and F
are SAM and FSRS, respectively, with no additional action required of Federal agencies.

Government-Wide Broker Data Reporting Issues

Throughout the course of the audit, Kearney became aware of the following Government-wide
broker data reporting issues, which we segregated from our results to focus on NSF’s
implementation of the DATA Act.

Current Total Value of Award and Potential Total Value of Award Errors for Procurement
Award Modifications

Data from the Current Total Value of Award and Potential Total Value of Award elements is
extracted from the Federal Procurement Data System – Next Generation (FPDS-NG) via the
legacy USASpending.gov and provided to the Broker. 24 25 Specifically, data for these elements is
extracted from the following FPDS-NG fields, respectively: 1) base and exercised options value
and 2) base and all options’ value. These two fields are categorized in FPDS-NG under two
columns for data entry labeled “Current” and “Total.” The “Current” column contains amounts
entered into the system by the agency. The “Total” column contains cumulative amounts
computed by FPDS-NG based on the modification amounts entered into the system by the
agency. Procurement award modifications, included in our sample, reported values for these
elements from FPDS-NG’s “Current” column, which displays the modification amount, rather
than the “Total” column, which displays the total award value. As a result, data for the Current
Total Value of Award and Potential Total Value of Award elements was inconsistent with

24
   OMB defines the current Total Value of Award data element as the total amount obligated to date on a contract,

including the base and exercised options. Potential Total Value of Award is defined by OMB as the total amount 

that could be obligated on a contract if the base and all options are exercised.

25
   The legacy USASpending.gov uses FPDS, v.1.4, to extract and map that data from FPDS-NG. This was a one­
time extraction for Q2 transactions.



                                                        21
                                                                                               National Science Foundation
                                                         Implementation of the Digital Accountability and Transparency Act
                                                                                        FY 2017 Performance Audit Report




agency records. A no-cost modification would cause the “Total” column to display an erroneous
zero balance. Procurement awards (base awards) that were not modified did not produce these
same errors. Treasury’s Government-wide DATA Act Program Management Office (PMO)
officials confirmed that they are aware that the Broker currently extracts data for these elements
from the “Current” column rather than the “Total” column. A Treasury official stated that the
issue will be resolved once DATA Act Information Model Schema (DAIMS), Version 1.1, is
implemented in the Broker and related historical data from USASpending.gov is transferred to
https://beta.usaspending.gov during Fall 2017. However, as NSF does not have responsibility for
how data is extracted by the Broker, we did not evaluate the reasonableness of Treasury’s
planned corrective action.

Indefinite Delivery Vehicle (IDV) Type Errors

For procurement awards included in our sample, data from the IDV Type element should be
extracted from FPDS-NG and provided to the Broker. The FPDS-NG atom feed 26 delivers the
IDV Type and Contract Award Type in the same field. The Broker did not break down the data
for IDV Type, which resulted in inconsistencies with agency records. Treasury’s DATA Act
PMO officials confirmed that they are aware of this issue and have taken steps to avoid this issue
in future reporting periods. However, as NSF does not have responsibility for how data is
extracted by the Broker, we did not evaluate the reasonableness of Treasury’s planned corrective
action.

Legal Entity City Code and Primary Place of Performance County Name Errors

The interface definition document (IDD), a DAIMS artifact, states that data from Legal Entity
City Code and Primary Place of Performance County Name, for financial assistance awards in
File D2, is extracted via Treasury’s ASP. During fieldwork, we noted that data for these two
fields was consistently blank. A Treasury official stated that data for Legal Entity City Code had
not been derived since January 2017 and there were no plans to reconsider how this element
would be handled.

The Treasury official further explained that data derived for Primary Place of Performance
County Name would not be implemented until September 2017. Because data for these elements
was not derived or implemented, these data fields were consistently blank and, therefore, not
reported for display on USASpending.gov. However, as NSF does not have responsibility for
how data is extracted by the Broker from Treasury’s ASP, we did not evaluate the
reasonableness of Treasury’s planned corrective action.




26
  FPDS-NG has data reporting web services that provide access in real-time to a central data repository. FPDS-NG
also provides real-time feeds of the same contractual data using atom feeds.


                                                       22
                                                                                               National Science Foundation
                                                         Implementation of the Digital Accountability and Transparency Act
                                                                                        FY 2017 Performance Audit Report




Prior Reports

In November 2016, NSF OIG reported 27 one finding regarding NSF’s readiness to implement the
DATA Act and report the required financial and spending data to Treasury by the April 30, 2017
reporting deadline. Specifically, the readiness review identified that NSF had not fully developed
and implemented a detailed plan (e.g., Project Management Plan, Risk Management Plan,
Human Resource Plan, Process for Documenting Key Decisions, and Process to Track Data
Inventory Gaps) to ensure project tasks and milestones are not missed/overlooked when
implementing DATA Act requirements.

Work Related to Internal Controls

Based upon the information obtained from NSF during preliminary audit procedures, Kearney
performed a risk assessment that identified audit risks related to the audit objectives. Agency
files submitted for the DATA Act are often interrelated and repeat information provided during
separate submissions to Treasury and OMB for other purposes. To ensure the accuracy,
completeness, timeliness, and quality of the data submitted for the DATA Act, agencies were
required to perform quality control procedures of the data prior to submission, including ensuring
that there were appropriate linkages between the DATA Act files and the files from existing
Government-wide reporting systems. 28 According to DATA Act guidance, agencies are required
to confirm that: 1) the information reported in File A matched the March 31, 2017 SF-133; 2)
File A matched the totals included in File B; 29 3) the transactions included in Files C were
included in Files D1 or D2, as applicable; and 4) the transactions included in Files D1 and D2, as
applicable, were included in File C. Kearney noted that NSF effectively performed these quality
control checks between Files A and B. As a result, NSF’s reconciliations between File A and the
SF-133 and between Files A and B produced no differences. Kearney re-performed these two
quality control procedures and also noted no difference. Additionally, through these
reconciliations, we validated the required data elements, which are presented in the files. 30

Although NSF performed the appropriate reconciliations between Files A and B, NSF did not
reconcile Files C to D1/D2 and E; therefore, Kearney performed a partial reconciliation between
Files C and D1/D2, as well as performed a reconciliation of data linkages between Files C and
D1/D2 to Files E and noted various differences. We determined that the linkage discrepancies
between transactions reported in Files C to D1/D2 were the result of mostly upward/downward
obligation activity reported on File C, but omitted from Files D1/D2. According to NSF
management, the agency opted to not report the File C transactions to Files D1/D2 as part of its
business practices and operations for procurement and financial award. NSF does not perform
the reconciliations to show the appropriate way to relate Files C to D1/D2.

27
   DATA Act Readiness Review – NSF Should Take Additional Steps to Ensure Compliance with the DATA Act, OIG
17-3-001, November 29, 2016
28
   OMB M-17-04, Section 3, Quarterly SAO Assurance of DATA Act Data
29
   The NSF also reconciled iTRAK to Governmentwide Treasury Account Symbol Adjusted Trial Balance System
(GTAS), as iTRAK was the basis of File B and GTAS was the basis of File A.
30
   Kearney tested the following six data elements through reconciliations: Appropriation Account, Budget Authority
Appropriated, Other Budgetary Resources, Outlays, Program Activity, and Unobligated Balance.


                                                        23
                                                                                                 National Science Foundation
                                                           Implementation of the Digital Accountability and Transparency Act
                                                                                          FY 2017 Performance Audit Report




In addition to performing this DATA Act audit, Kearney also performs NSF’s financial
statement and information technology (IT) audits. Accordingly, we relied on this work to test
internal controls, specifically related to the DATA Act. Kearney’s Financial Audit Team tested
controls over grants processing/monitoring, grant closeouts, interface with other awards systems
(e.g., eJacket), and the procurement/contracts management process (e.g., obligations). Kearney’s
T Audit Team tested access controls, separation of duties, and change management as they relate
to NSF’s iTRAK (i.e., financial accounting system) and Awards systems (i.e., processing all
award actions, including funding and non-funding). Kearney relied on this work to understand
the internal controls as they related to the DATA Act audit.

Kearney identified additional internal controls, including general and application controls in
source systems and controls, to ensure that data was accurate, complete, and timely; however, we
chose not to rely on or specifically test those controls to determine NSF’s implementation and
use of the data standard. Based on the professional judgment of the Audit Team, an approach for
testing additional internal controls would be inefficient for purposes of this audit. In addition,
Kearney identified data elements that rely solely on accurate human data entry, such as a
vendor’s place of performance, rather than source system internal controls. Accordingly, we
designed additional substantive procedures that would enable us to obtain sufficient and
appropriate evidence to conclude upon the audit objectives.

Use of Computer-Processed Data

As discussed in the BACKGROUND section of this report, the files included in NSF’s DATA
Act submission were generated from multiple systems, including NSF-owned systems and
systems used across the Federal Government. As the purpose of this audit was to audit the
amounts included in this submission by tracing information to source documentation, other than
the reconciliations, described in the AUDIT RESULTS section of this report, additional steps
were not considered necessary to assess the sufficiency of computer-processed data.

Detailed Sampling Methodology

In accordance with the Inspectors General Guide to Compliance Under the DATA Act, Kearney
selected a sample of certified spending data transactions for transaction-level testing from NSF’s
Q2 FY 2017 DATA Act File C submission. 31 In accordance with the Inspectors General Guide
to Compliance Under the DATA Act, Kearney selected a random sample of 254 32 transactions
included in File C using sampling software. We used a 20% error rate in determining the sample

31
   Section 430.01 of the Inspectors General Guide to Compliance Under the DATA Act states: “the engagement team
should randomly select a statistically valid sample of certified spending data from the reportable award-level
transactions included in the agency’s certified data submission for File C, or Files D1 and D2 if file C is
unavailable.” Since Files D1 and D2 are available, Kearney selected the sample from File C.
32
   Section 430.02 of the Inspectors General Guide to Compliance Under the DATA Act requires a sample size of 385
transactions using an error rate of 50%; however, it also provides a correction formula for agencies with populations
smaller than 385. Applying this correction formula – 385/[1+(385/N)] – to NSF’s 6,603 transaction File C
population, a sample size of 364 would have been selected. However, Kearney and the NSF OIG determined that a
20% error rate would be more appropriate for our audit, which results in an updated sample size of 254 transactions.


                                                         24
                                                                                                National Science Foundation
                                                          Implementation of the Digital Accountability and Transparency Act
                                                                                         FY 2017 Performance Audit Report




size, as the DATA Act engagement is a performance audit, allowing flexibility in the way the
planning and results are reported. Kearney also noted that NSF is a small agency and has
received clean opinions on its audit reports for the past 19 years; therefore, the expected risk was
lower. We also determined that NSF implemented a new financial reporting system, iTRAK, and
there are strong internal controls associated with it. Further, through all of the audits performed,
there were no material weaknesses identified within the financial systems; therefore, Kearney
and the NSF OIG determined that a 20% error rate would be reasonable based on the parameters
listed below:

    •	   Source of Population/Sample – File C (6,603).
    •	   Confidence Level – 95% (Should not change).
    •	   Sample Precision – +-5% (Use 10% in software as it relates to precision range +/- 5%).
    •	   Expected Error Rate – 20% (CIGIE Guide recommends an expected error rate of 50%,
         however, Exhibit 7 shows different sample size results based on revised error rates).
    •	   Sample Size – 385 or formula (per Section 430.02, Footnote 40 of the Inspectors General
         Guide to Compliance Under the DATA Act) if population is classified as “small” (See
         Exhibit 7 for results).

         Exhibit 7: Population/Sampling Parameters (Using File C as Population)
   Confidence                         Precision     Expected Error
                  Population Size                                        Sample Size
     Level                            (Range)            Rate
      95%             10,000*           10%               50%                385*
      95%              6,603            10%               50%                 379
      95%              6,603            10%               40%                 365
      95%              6,603            10%               30%                 323
     95%               6,603            10%              20%                  254
      95%              6,603            10%               10%                 156
*10,000 transactions would be considered a “large population.” Thus, using the statistical software, the sample size
results in the recommended/maximum sample size noted per the Inspectors General Guide to Compliance Under the
DATA Act Guide (385 samples).




                                                        25
                                                                                    National Science Foundation
                                              Implementation of the Digital Accountability and Transparency Act
                                                                             FY 2017 Performance Audit Report




APPENDIX B: REQUIRED DATA ELEMENTS FOR FEDERAL AGENCY
REPORTING

                                                                                            Submission
   Data Element                           Data Description
                                                                                                 File
Appropriations         The basic unit of an appropriation generally reflecting              Files A and
Account                each unnumbered paragraph in an appropriation act.                   B
                       A provision of law (not necessarily in an appropriations
Budget Authority                                                                            File A and
                       act) authorizing an account to incur obligations and to
Appropriated                                                                                B
                       make outlays for a given purpose.
                       Categories in a classification system that present
                                                                                            Files B and
Object Class           obligations by the items or services purchased by the
                                                                                            C
                       Federal Government.
                       A legally binding agreement that will result in outlays,             Files A, B,
Obligation
                       immediately or in the future.                                        and C
                       New borrowing authority, contract authority, and
                       spending authority from offsetting collections provided
Other Budgetary        by Congress in an appropriations act or other                        File A and
Resources              legislation, or unobligated balances of budgetary                    B
                       resources made available in previous legislation, to
                       incur obligations and to make outlays.
                       Payments made to liquidate an obligation (other than
                       the repayment of debt principal or other disbursements               Files A and
Outlay
                       that are “means of financing” transactions). Outlays                 Ba
                       generally are equal to cash disbursements.
                       A specific activity or project as listed in the program
Program Activity       and financing schedules of the annual budget of the                  Files Bb
                       United Agency’s Government.
Treasury Account
                       The account identification codes assigned by the
Symbol (TAS)
                       Department of the Treasury (Treasury) to individual                  File C c
(excluding sub-
                       appropriation, receipt, or other fund accounts.
account)
                       The cumulative amount of budget authority that
                                                                                            Files A and
Unobligated Balance    remains available for obligation under law in unexpired
                                                                                            B
                       accounts at a point in time.
                       The date the action being reported was issued/ signed
                                                                                            Files D1 and
Action Date            by the Government or a binding agreement was
                                                                                            D2
                       reached.
                       A technical communication document intended to give                  Files D1 and
Action Type
                       assistance to users of a particular system.                          D2
                                                                                            Files D1 and
Award Description      A brief description of the purpose of the award.
                                                                                            D2
Award Identification   The unique identifier of the specific award being                    Files C, D1
(ID) Number            reported (i.e., Federal Award Identification Number                  and D2



                                             26
                                                                                      National Science Foundation
                                                Implementation of the Digital Accountability and Transparency Act
                                                                               FY 2017 Performance Audit Report




                                                                                              Submission
   Data Element                            Data Description
                                                                                                 File
                        [FAIN]) for financial assistance and Procurement
                        Instrument Identifier (PIID) for procurement.
Award Modification/     The identifier of an action being reported that indicates             Files D1 and
Amendment Number        the specific subsequent change to the initial award.                  D2
                        Description (and corresponding code) that provides
                        information to distinguish type of contract, grant, or
Award Type                                                                                    File D1
                        loan and provides the user with more granularity into
                        the method of delivery of the outcomes.
                        A collection of indicators of different types of
Business Types          recipients based on socio-economic status and                         File D2
                        organization/business areas.
Catalog of Federal
                        The number assigned to a Federal area of work in the
Domestic Assistance                                                                           File D2
                        Catalog of Federal Domestic Assistance.
(CFDA) Number
Catalog of Federal      The title of the area of work under which the Federal
Domestic Assistance     award was funded in the Catalog of Federal Domestic     File D2
(CFDA) Title^           Assistance.
North American          The identifier that represents the NAICS Code assigned
Industrial              to the solicitation and resulting award identifying the
                                                                                File D1
Classification System   industry in which the contract requirements are
(NAICS) Code            normally performed.
North American
Industrial
                        The title associated with the NAICS Code.                             File D1
Classification System
(NAICS) Description
Ordering Period End     The date on which no additional orders referring to it
                                                                                              File D1
Date                    (the award) may be placed.
                        The identifier of the procurement award under which
Parent Award ID
                        the specific award is issued, such as a Federal Supply                File D1
Number
                        Schedule.
Period of
                        The current date on which awardee effort completes or                 Files D1 and
Performance Current
                        the award is otherwise ended.                                         D2
End Date
Period of
                        The date on which awardee effort is completed or the
Performance                                                                                   File D1
                        award is otherwise ended.
Potential End Date
Period of
                        The date on which awardee effort begins or the award                  Files D1 and
Performance Start
                        is otherwise effective.                                               D2
Date
Primary Place of        The address where the predominant performance of the                  Files D1 and
Performance Address     award will be accomplished. Components include:                       D2




                                              27
                                                                                     National Science Foundation
                                               Implementation of the Digital Accountability and Transparency Act
                                                                              FY 2017 Performance Audit Report




                                                                                             Submission
   Data Element                            Data Description
                                                                                                File
                        Address Lines 1 and 2, City, County, Agency Code,
                        and ZIP+4 or Postal Code.
Primary Place of        U.S. congressional district where the predominant
Performance             performance of the award will be accomplished;                       Files D1 and
Congressional           derived from the Primary Place of Performance                        D2
District^               Address.
Primary Place of
                        Country code where the predominant performance of                    Files D1 and
Performance Country
                        the award will be accomplished.                                      D2
Code
Primary Place of        Name of the county represented by the county code
                                                                                             Files D1 and
Performance County      where the predominant performance of the award will
                                                                                             D2
Name*                   be accomplished.
                        Code indicating whether an action is an individual
Record Type                                                                                  File D2
                        transaction or aggregated.
                        The cumulative amount obligated by the Federal
                                                                                             Files D1 and
Amount of Award         Government for an award, calculated by
                                                                                             D2
                        USASpending.gov or a successor site.
Current Total Value     For procurement, the total amount obligated to date on
                                                                                             File D1
of Award*               a contract, including the base and exercised options.
                        Amount of Federal Government’s obligation, de­
Federal Action                                                                               Files D1 and
                        obligation, or liability, in dollars, for an award
Obligation                                                                                   D2
                        transaction.
Non-Federal Funding     For financial assistance, the amount of the award
                                                                                             File D2
Amount                  funded by non-Federal source(s), in dollars.
                        For procurement, the total amount that could be
Potential Total Value
                        obligated on a contract if the base and all options are              File D1
of Award*
                        exercised.
Awardee/Recipient       The name of the awardee or recipient that relates to the             Files D1 and
Legal Entity Name       unique identifier.                                                   D2
                        The unique identification number for an awardee or
Awardee/Recipient       recipient; most commonly the nine-digit number                       Files D1,
Unique Identifier       assigned by Dun & Bradstreet, referred to as the                     D2, E and F
                        DUNS® number.
                        The first name, middle initial, and last name of an
Highly Compensated
                        individual identified as one of the five most highly                 File E
Officer Name
                        compensated “Executives.”
Highly Compensated      The cash and noncash dollar value earned by one of the
Officer Total           five most highly compensated “Executives” during the                 File E
Compensation            awardee’s preceding fiscal year (FY).
                        The awardee or recipient’s legal business address                    Files D1 and
Legal Entity Address
                        where the office represented by the Unique Entity                    D2




                                              28
                                                                                      National Science Foundation
                                                Implementation of the Digital Accountability and Transparency Act
                                                                               FY 2017 Performance Audit Report




                                                                                              Submission
   Data Element                            Data Description
                                                                                                 File
                       Identifier (as registered in the System for Award
                       Management [SAM]) is located.
Legal Entity           The congressional district in which the awardee or
                                                                                              Files D1 and
Congressional          recipient is located. This is not a required data element
                                                                                              D2
District^              for non-U.S. addresses.
                       Code for the country in which the awardee or recipient
                       is located, using the ISO 3166-1 Alpha-3 GENC
Legal Entity Country                                                                          Files D1 and
                       Profile, and not the codes listed for those territories and
Code                                                                                          D2
                       possessions of the United States already identified as
                       “states.”
Legal Entity Country                                                                          Files D1 and
                       The name corresponding to the Country Code.
Name                                                                                          D2
                       The name of the ultimate parent of the awardee or
Ultimate Parent                                                                               Files D1,
                       recipient. Currently, the name is from the global parent
Legal Entity Name**                                                                           D2 and E
                       DUNS® number.
Ultimate Parent        The unique identification number for the ultimate                      Files D1,
Unique Identifier**    parent of an awardee or recipient.                                     D2 and E
Awarding Agency        A department or establishment of the Government as                     Files D1 and
Code                   used in the Treasury Account Fund Symbol (TAFS).                       D2
Awarding Agency        The name associated with a department or                               Files D1 and
Name                   establishment of the Government as used in the TAFS.                   D2
Awarding Office        Identifier of the level n organization that awarded,                   Files D1 and
Code                   executed or is otherwise responsible for the transaction.              D2
                       Name of the level n organization that awarded,
Awarding Office                                                                               Files D1 and
                       executed, or is otherwise responsible for the
Name^                                                                                         D2
                       transaction.
                       Identifier of the level 2 organization that awarded,
Awarding Sub Tier                                                                             Files D1 and
                       executed, or is otherwise responsible for the
Agency Code                                                                                   D2
                       transaction.
                       Name of the level 2 organization that awarded,
Awarding Sub Tier                                                                             Files D1 and
                       executed, or is otherwise responsible for the
Agency Name                                                                                   D2
                       transaction.
                       The three-digit CGAC agency code of the department
Funding Agency         or establishment of the Government that provided the                   Files D1 and
Code                   preponderance of the funds for an award and/or                         D2
                       individual transactions related to an award.
                       Name of the department or establishment of the
Funding Agency         Government that provided the preponderance of the                      Files D1 and
Name                   funds for an award and/or individual transactions                      D2
                       related to an award.




                                              29
                                                                                                 National Science Foundation
                                                           Implementation of the Digital Accountability and Transparency Act
                                                                                          FY 2017 Performance Audit Report




                                                                                                         Submission
     Data Element                                    Data Description
                                                                                                            File
                              Identifier of the level n organization that provided the
                                                                                                         Files D1 and
 Funding Office Code          preponderance of the funds obligated by this
                                                                                                         D2
                              transaction.
                              Name of the level n organization that provided the
 Funding Office                                                                                          Files D1 and
                              preponderance of the funds obligated by this
 Name^                                                                                                   D2
                              transaction.
                              Identifier of the level 2 organization that provided the
 Funding Sub Tier                                                                                        Files D1 and
                              preponderance of the funds obligated by this
 Agency Code                                                                                             D2
                              transaction.
                              Name of the level 2 organization that provided the
 Funding Sub Tier                                                                                        Files D1 and
                              preponderance of the funds obligated by this
 Agency Name                                                                                             D2
                              transaction.
*Denotes a Government-wide Treasury Broker data element.

**For FAINs, the data element is only reported on File E (these elements are derived by SAM and were not included

within our sampling error rate calculation).

^Denotes an ASP derived data element.

Source: https://max.gov/maxportal/assets/public/offm/DataStandardsFinal.htm
a
  Per Digital Accountability and Transparency Act of 2014 (DATA Act) and Office of Management and Budget
(OMB) reporting guidelines, data element is required to be submitted via Files A and B and may also be optionally
submitted via File C. The National Science Foundation (referred to as “NSF” in this report) elected to not report this
optional data element in File C. Accordingly, Kearney & Company, P.C. (referred to as “Kearney,” “we,” and “our”
in this report) tested this data element within the File A and B submissions.
b
  Per DATA Act and OMB reporting guidelines, data element is required to be submitted via File B and may also be
optionally submitted via File C. NSF elected to not report this optional data element in File C. Accordingly, Kearney
tested this data element within the File B submission.
c
  The data elements TAS and Appropriations Account are the same. To avoid double-counting, Kearney aligned the
appropriation account field to Files A and B and the TAS to File C.




                                                         30
                                                                                       National Science Foundation
                                                 Implementation of the Digital Accountability and Transparency Act
                                                                                FY 2017 Performance Audit Report




APPENDIX C: DATA ACT INFORMATION FLOW DIAGRAM





                                           31
                                                                                              National Science Foundation
                                                        Implementation of the Digital Accountability and Transparency Act
                                                                                       FY 2017 Performance Audit Report




APPENDIX D: SUMMARY OF THE NATIONAL SCIENCE FOUNDATION’S DATA
ACT RESULTS

                                        FILE C

                   Procurement Instrument Identifier (PIID) Data Element

                                                        Total        Total        Total         Total
 Data Element                                           Count        Count        Count         Count         Total
                                                         Ys           Ns           NTs           NAs
 DE 24        Parent Award Identification Number            10             0              0             0         10
 DE 34        Award Identification Number – PIID            10             0              0             0         10
 DE 50        Object Class (+D/R Funding Source)            10             0              0             0         10
 DE 51        Appropriations Account                        10             0              0             0         10
 DE 53        Transaction Obligated Amount                  10             0              0             0         10
 DE 56        Program Activity Name (+Code)                 10             0              0             0         10

                Financial Assistance Identifier Number (FAIN) Data Element
                                                               Total        Total        Total
      Data Element                                             Count        Count        Count        Total
                                                                Ys           Ns           NAs
                   Award Identification Number –
      DE 34                                                         244            0             0          244
                   FAIN
      DE 50        Object Class (+ D/R Funding Source)              244            0             0          244
      DE 51        Appropriations Account                           244            0             0          244
      DE 53        Transaction Obligated Amount                     244            0             0          244
      DE 56        Program Activity Name (+Code)                    244            0             0          244

         Y	       Indicates the data element was present and accurately reported as supported by
                  source systems or source documents, as applicable.
         N	       Indicates the data element was not present or not accurately reported as supported
                  by source systems or source documents, as applicable.
         NT	      Indicates the data element was not tested because the sample unit should not have
                  been included in file C. These were considered errors for our statistical sample.
         NA	      Indicates the data element was not applicable to the sample unit. These were not
                  considered errors for our statistical sample.




                                                      32
                                                                                          National Science Foundation
                                                    Implementation of the Digital Accountability and Transparency Act
                                                                                   FY 2017 Performance Audit Report




                                          File D1

                    Procurement Instrument Identifier (PIID) Data Element

                                                Total      Total        Total        Total        Total
Data Element                                    Count      Count        Count        Count        Count        Total
                                                 Ys         Ns           NTs          NDs          NAs
         Awardee / Recipient Legal Entity
DE 1                                                10            0             0           0             0         10
         Name
         Awardee / Recipient Unique
DE 2                                                10            0             0           0             0         10
         Identifier
DE 3     Ultimate Parent Unique Identifier**        10            0             0           0             0         10
         Ultimate Parent Legal Entity
DE 4                                                10            0             0           0             0         10
         Name**
DE 5     Legal Entity Address                       10            0             0           0             0         10
DE 6     Legal Entity Congressional District^        9            1             0           0             0         10
DE 7     Legal Entity Country Code                  10            0             0           0             0         10
DE 8     Legal Entity Country Name                  10            0             0           0             0         10
DE 11    Amount of Award                            10            0             0           0             0         10
DE 14    Current Total Value Of Award*               9            1             0           0             0         10
DE 15    Potential Total Value Of Award*             9            1             0           0             0         10
DE 16    Award Type                                 10            0             0           0             0         10
         North American Industrial
DE 17                                               10            0             0           0             0         10
         Classification System (NAICS)
DE 18    NAICS Description                          10            0             0           0             0         10
DE 22    Award Description                          10            0             0           0             0         10
         Award Modification / Amendment
DE 23                                               10            0             0           0             0         10
         Number
         Parent Award Identification
DE 24                                               10            0             0           0             0         10
         Number
DE 25    Action Date                                10            0             0           0             0         10
DE 26    Period Of Performance Start Date           10            0             0           0             0         10
         Period Of Performance Current
DE 27                                               10            0             0           0             0         10
         End Date
         Period Of Performance Potential
DE 28                                               10            0             0           0             0         10
         End Date
DE 29    Ordering Period End Date                   10            0             0           0             0         10
         Primary Place of Performance
DE 30                                               10            0             0           0             0         10
         Address
         Primary Place of Performance
DE 31                                               10            0             0           0             0         10
         Congressional District^
         Primary Place Of Performance
DE 32                                               10            0             0           0             0         10
         Country Code
         Award Identification Number –
DE 34                                               10            0             0           0             0         10
         PIID
DE 36    Action Type                                10            0             0           0             0         10
DE 38    Funding Agency Name                        10            0             0           0             0         10
DE 39    Funding Agency Code                        10            0             0           0             0         10
DE 40    Funding Sub Tier Agency Name               10            0             0           0             0         10
DE 41    Funding Sub Tier Agency Code               10            0             0           0             0         10
DE 42    Funding Office Name^                       10            0             0           0             0         10
DE 43    Funding Office Code                        10            0             0           0             0         10
DE 44    Awarding Agency Name                       10            0             0           0             0         10
DE 45    Awarding Agency Code                       10            0             0           0             0         10



                                                  33
                                                                                                    National Science Foundation
                                                              Implementation of the Digital Accountability and Transparency Act
                                                                                             FY 2017 Performance Audit Report




                                                      Total       Total       Total      Total    Total
Data Element                                         Count        Count      Count       Count    Count       Total
                                                        Ys          Ns         NTs        NDs      NAs
DE 46      Awarding Sub Tier Agency Name                    10          0           0          0         0        10
DE 47      Awarding Sub Tier Agency Code                    10          0           0          0         0        10
DE 48      Awarding Office Name^                            10          0           0          0         0        10
DE 49      Awarding Office Code                             10          0           0          0         0        10
 *Denotes a Government-wide Treasury Broker data element.

 **For FAINs, the data element is only reported on File E (these elements are derived by SAM and were not included

 within our sampling error rate calculation).

 ^Denotes an ASP derived data element.




  Y       Indicates the data element was present and accurately traced to the FPDS and source documents,
          as applicable.
  N       Indicates the data element was not present or not accurately traced to the FPDS and source
          documents, as applicable.
  NT      Indicates the data element was not tested because the sample unit was erroneously included in file
          C. These were considered errors for our statistical sample.
  ND	     Indicates the data element was not determinable because although the award was correctly
          included in file C, it was not correctly included in file D1. These were considered errors for our
          statistical sample.
  NA	     Indicates the data element was not applicable to the sample unit. These were not considered errors
          for our statistical sample.

 Notes: DE 5 Legal Entity Address includes the sub-elements Legal Entity Address Lines 1-3, Legal Entity City
 Name, Legal Entity State Code, Legal Entity State Description, and Legal Entity ZIP +4. We did not include Legal
 Entity State Description in our error rate. During our test work, we noted that this field was not populated for all
 sample units in file D1. Per the Federal Procurement Database System – Next Generation (FPDS-NG) Data
 Dictionary, the Legal Entity State Description is derived from the System for Award Management (SAM) unless
 SAM Exception is selected. The DATA Act broker system did not populate this field in file D1 as was intended.

 DE 14 Current Total Value of Award and DE 15 Potential Total Value of Award, were extracted incorrectly from
 the FPDS-NG by the DATA Act broker system. [The Office of Management and Budget defines the current total
 value of award data element as the total amount obligated to date on a contract, including the base and exercised
 options. Potential total value of award is defined as the total amount that could be obligated on a contract, if the base
 and all options are exercised.] Specifically, data for these data elements are extracted from the following FPDS-NG
 fields, respectively: (1) base and exercised options value and (2) base and all options value. FPDS-NG has two
 columns of data entry for these fields labeled “Current” and “Total”. The current column contains modification
 amounts entered into the system by the user. The total column contains cumulative total award values computed by
 the system based on the modification amounts entered. All procurement modifications included in our sample
 reported values for these data elements from the corresponding field’s current column, or modification amount,
 rather than the total column, or total award value. The Department of the Treasury’s (Treasury) Government-wide
 DATA Act Program Management Office (PMO) officials confirmed that the DATA Act broker system incorrectly
 extracted values for these data elements from the current column rather than the total column. A Treasury official
 stated that the issue will be resolved once related historical data from USASpending.gov are transferred to
 beta.USASpending.gov during Fall 2017. However, as the agency does not have responsibility for how data is
 extracted by the DATA Act broker system, we did not evaluate the reasonableness of Treasury’s planned corrective
 action. We did evaluate DE 14 Current Total Value Of Award and DE 15 Potential Total Value of Award based on
 the current value and not the calculated total value. Therefore, if the correct current column value was included in
 DE 14 Current Total Value Of Award and DE 15 Potential Total Value of Award, then we did not consider those to
 be errors when calculating our error rate.




                                                            34
                                                                                                 National Science Foundation
                                                           Implementation of the Digital Accountability and Transparency Act
                                                                                          FY 2017 Performance Audit Report




DE 16 Award Type includes the sub-elements Type of Contract Pricing, IDV_Type, and Contract Award Type. We
did not include IDV_Type in our error rate, as errors noted were due to the DATA Act broker system extracting the
wrong field. Treasury’s DATA Act PMO officials confirmed that they are aware of this issue and have taken steps
to avoid this issue in future reporting periods. However, as the agency does not have responsibility for how data is
extracted by the broker system, we did not evaluate the reasonableness of Treasury’s planned corrective action.




                                                         35
                                                                                       National Science Foundation
                                                 Implementation of the Digital Accountability and Transparency Act
                                                                                FY 2017 Performance Audit Report




                                        File D2

             Financial Assistance Identifier Number (FAIN) Data Element

                                                  Total         Total      Total        Total
                Data Element                      Count         Count      Count        Count        Total
                                                   Ys            Ns         NDs          NAs
          Awardee Or Recipient Legal Entity
DE 1                                                  244             0           0             0        244
          Name
          Awardee Or Recipient Unique
DE 2                                                  244             0           0             0        244
          Identifier
DE 5      Legal Entity Address                        188           56            0             0        244
          Legal Entity Congressional
DE 6                                                  237             7           0             0        244
          District^
DE 7      Legal Entity Country Code                   244            0            0             0        244
DE 11     Federal Action Obligation                   240            4            0             0        244
DE 12     Non-Federal Funding Amount                  244            0            0             0        244
DE 13     Total Funding Amount                        244            0            0             0        244
DE 14     Face Value Loan Guarantee                   244            0            0             0        244
DE 16     Assistance Type                             216           28            0             0        244
          Catalog of Federal Domestic
DE 19                                                 244             0           0             0        244
          Assistance (CFDA) Number
DE 20     CFDA Title^                                 243             1           0             0        244
DE 22     Award Description                           244             0           0             0        244
          Award Modification / Amendment
DE 23                                                 244             0           0             0        244
          Number
DE 25     Action Date                                 243             1           0             0        244
DE 26     Period Of Performance Start Date            244             0           0             0        244
          Period Of Performance Current
DE 27                                                 244             0           0             0        244
          End Date
          Primary Place of Performance
DE 30                                                 210           34            0             0        244
          Address
          Primary Place Of Performance
DE 31                                                 237             7           0             0        244
          Congressional District^
          Primary Place Of Performance
DE 32                                                 244             0           0             0        244
          Country Code
          Award Identification Number –
DE 34                                                 244             0           0             0        244
          FAIN
DE 35     Record Type                                 244            0            0             0        244
DE 36     Action Type                                 244            0            0             0        244
DE 37     Business Types                              243            1            0             0        244
DE 38     Funding Agency Name                         244            0            0             0        244
DE 39     Funding Agency Code                         244            0            0             0        244
DE 40     Funding Sub Tier Agency Name                244            0            0             0        244
DE 41     Funding Sub Tier Agency Code                244            0            0             0        244
DE 42     Funding Office Name^                          93         151            0             0        244
DE 43     Funding Office Code                         242            2            0             0        244
DE 44     Awarding Agency Name                        244            0            0             0        244
DE 45     Awarding Agency Code                        244            0            0             0        244
DE 46     Awarding Sub Tier Agency Name               244            0            0             0        244
DE 47     Awarding Sub Tier Agency Code               244            0            0             0        244
DE 48     Awarding Office Name^                         93         151            0             0        244
DE 49     Awarding Office Code                        244            0            0             0        244
     *Denotes a Government-wide Treasury Broker data element.



                                                36
                                                                                                    National Science Foundation
                                                              Implementation of the Digital Accountability and Transparency Act
                                                                                             FY 2017 Performance Audit Report




          **For FAINs, the data element is only reported on File E (these elements are derived by SAM and were not
          included within our sampling error rate calculation).
          ^Denotes an ASP derived data element.

Y         Indicates the data element was present and accurately traced to ASP, SAM, and source
          documents, as applicable.
N         Indicates the data element was not present or not accurately traced to ASP, SAM, and source
          documents, as applicable.
ND        Indicates the data element was not determinable because although the award was correctly
          included in file C, it was not correctly included in file D2. These were considered errors for our
          statistical sample.
NA        Indicates the data element was not applicable to the sample unit. These were not considered errors
          for our statistical sample.

    Notes: DE 5 Legal Entity Address includes sub-elements Legal Entity Address Lines 1-3, Legal Entity City
    Name, Legal Entity City Code, Legal Entity Foreign City Name, Legal Entity State Code, Legal Entity State
    Name, Legal Entity Foreign Province Name, Legal Entity ZIP 5, Legal Entity ZIP Last 4, Legal Entity Foreign
    Postal Code, Legal Entity County Name, and Legal Entity County Code. We did not include Legal Entity City
    Code in our error rate. During our test work, we noted that this field was not populated for all sample units in file
    D2. Per the DATA Act Information Model Schema (DAIMS), the Legal Entity City Code is derived from Award
    Submission Portal (ASP). The DATA Act broker system did not populate this field in file D2 as was intended.
    However, as the agency does not have responsibility for how data is extracted by the DATA Act broker system
    from Treasury’s ASP, we did not evaluate the reasonableness of Treasury’s planned corrective action.

    DE 30 Primary Place of Performance Address includes sub-elements Primary Place of Performance City Name,
    Primary Place of Performance State Name, Primary Place of Performance County Name, Primary Place of
    Performance ZIP +4, and Primary Place of Performance Code. We did not include Primary Place of Performance
    County Name in our error rate. During our test work, we noted that this field was not populated for all sample
    units in file D2. Per the DAIMS, the Primary Place of Performance County Name is derived from ASP. The
    DATA Act broker system did not populate this field in file D2 as was intended. However, as the agency does not
    have responsibility for how data is extracted by the DATA Act broker from Treasury’s ASP, we did not evaluate
    the reasonableness of Treasury’s planned corrective action.




                                                            37
                                                                      National Science Foundation
                                Implementation of the Digital Accountability and Transparency Act
                                                               FY 2017 Performance Audit Report




APPENDIX E: MANAGEMENT’S RESPONSE





                               38
                                       National Science Foundation
 Implementation of the Digital Accountability and Transparency Act
                                FY 2017 Performance Audit Report




39
                                       National Science Foundation
 Implementation of the Digital Accountability and Transparency Act
                                FY 2017 Performance Audit Report




40
                                       National Science Foundation
 Implementation of the Digital Accountability and Transparency Act
                                FY 2017 Performance Audit Report




41
                                       National Science Foundation
 Implementation of the Digital Accountability and Transparency Act
                                FY 2017 Performance Audit Report




42
                                                                                              National Science Foundation
                                                        Implementation of the Digital Accountability and Transparency Act
                                                                                       FY 2017 Performance Audit Report




APPENDIX F: KEARNEY’S RESPONSE

Kearney & Company, P.C. (referred to as “Kearney,” “we,” and “our” in this document)
appreciates NSF’s management’s thorough response to the draft DATA Act report. NSF
generally concurred with our recommendations and we commend NSF for its commitment and
responsiveness during the DATA Act Audit, specifically during the testing and reporting phases.

The purpose and objective of the DATA Act is to make agency data “easier to understand how
the Federal government spends taxpayer dollars but will also serve as a tool for better oversight,
data-centric decision-making, and innovation both inside and outside of government.” 33
Ultimately, the goal of the DATA Act is to allow the general public to understand individual
agency’s spending data. The intent of our DATA Act audit was to determine whether NSF’s
reported data, (Fiscal Year 2017 Quarter 2) complied with Government-wide financial data
standards developed and issued by OMB and Treasury.

NSF disagrees with our methodology regarding how the sampling error rates were calculated.
NSF stated, “...error rates for accuracy, completeness and timeliness would more appropriately
be calculated by reviewing each data element rather than at the transaction level.” Kearney
generally concurs that reporting sampling error rates by data element would provide further
insight over NSF’s overall error rate; however, we were required to report our sampling errors
rates by transaction in accordance with OMB guidance. Further, we included additional tables
within our report that detailed our testing results by data element. Additionally, we also noted
whether errors we identified were attributable to NSF’s reporting or to Government-wide
reporting issues outside of NSF’s control. Although we reported our sampling error calculations
at the transaction level, we believe the additional tables and information presented in the report
provide further information and transparency regarding our testing results.

NSF does not agree with our conclusion that insufficient reconciliations are the cause of the
errors we identified. NSF stated it has “sufficient reconciliation and quality control procedures
embedded into our data submission processes to ensure that information is accurate, complete
and timely, meeting the transparency goals of the DATA Act.” While we understand and
acknowledge that NSF has reconciliation procedures in place that enabled it to identify errors as
part of the Broker Warning/Errors reports, we noted that NSF did not have complete
reconciliations, which provide “linkage” between the File C to File D1/D2 submissions.

As a result, we noted several discrepancies associated with data elements during our testing. An
individual from the general public would be unable to understand and conclude that data
reporting discrepancies arise from NSF’s specific and unique business practices, thus reducing
the reliability and usefulness of the data. The aim of the DATA Act is to “to increase
accountability, transparency, accessibility, quality, and standardization in Federal spending
data.” NSF’s reported spending data displays completeness issues within NSF’s financial and
awards systems.


33   https://www.usaspending.gov/Pages/data-act.aspx


                                                       43
                                                                            National Science Foundation
                                      Implementation of the Digital Accountability and Transparency Act
                                                                     FY 2017 Performance Audit Report




APPENDIX G: ABBREVIATIONS AND ACRONYMS


      Acronym                                Definition
  ASP           Award Submission Portal
  Broker        DATA Act Broker
  CFDA          Catalog of Federal Domestic Assistance
  CIGIE         Counsel of Inspectors General on Integrity and Efficiency
  DAIMS         DATA Act Information Model Schema, v.1.1
  DATA Act      Digital Accountability and Transparency Act of 2014
  DACS          Division of Acquisition and Cooperative Support
  DFM           Division of Financial Management
  DIAS          Division of Institution and Award Support
  DIS           Division of Information Systems
  FAIN          Federal Award Identification Number
  FFATA         Federal Funding Accountability and Transparency Act of 2006
  FISCAM        Federal Information System Controls Audit Manual
  FPDS-NG       Federal Procurement Data System – Next Generation
  FSRS          FFATA Sub-award Reporting System
  FY            Fiscal Year
  GAO           Government Accountability Office
                Governmentwide Treasury Account Symbol Adjusted Trial
  GTAS
                Balance System
  ID            Identification
  IDD           Interface Definition Document
  IDV           Indefinite Delivery Vehicle
  Kearney       Kearney & Company, P.C.
  NAICS         North American Industrial Classification System
  NANP          North American Numbering Plan
  OGC           Office of General Counsel
  OIG           Office of Inspector General
  OMB           Office of Management and Budget
  PIID          Procurement Instrument Identifier
  PoP           Period of Performance
  Q             Quarter
  SAM           System of Award Management
  SAO           Senior Accountable Official
  SBR           Statement of Budgetary Resources
  SF            Standard Form
  SOW           Statement of Work
  TAFS          Treasury Account Fund Symbol
  TAS           Treasury Account Symbol
  Treasury      Department of the Treasury




                                     44