oversight

U.S. Department of Education's Compliance With Improper Payment Reporting Requirements for Fiscal Year 2015

Published by the Department of Education, Office of Inspector General on 2016-05-10.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

U.S. Department of Education’s Compliance With Improper
  Payment Reporting Requirements for Fiscal Year 2015


                                 FINAL AUDIT REPORT




                                   ED-OIG/A03Q0001
                                       May 2016


Our mission is to promote the                         U.S Department of Education
efficiency, effectiveness, and                        Office of Inspector General
integrity of the Department’s                         Washington, D.C.
programs and operations.
                                         NOTICE

Statements that managerial practices need improvements, as well as
other conclusions and recommendations in this report, represent the
opinions of the Office of Inspector General. Determinations of
corrective action to be taken will be made by the appropriate
Department of Education officials.

In accordance with the Freedom of Information Act (5 U.S.C. § 552),
reports issued by the Office of Inspector General are available to
members of the press and general public to the extent information
contained therein is not subject to exemptions in the Act.




  Our mission is to promote the efficiency, effectiveness, and integrity of the Department’s programs and operations.
                                    UNITED STATES DEPARTMENT OF EDUCATION
                                                    OFFICE OF INSPECTOR GENERAL

                                                                                                                     AUDIT SERVICES



                                                                     May 10, 2016

Memorandum
TO:                  Tim Soltis
                     Delegated to Perform the Functions and Duties of the Chief Financial Officer
                     Office of Chief Financial Officer
                     Lead Action Official

                     James W. Runcie
                     Chief Operating Officer
                     Federal Student Aid


FROM:                Patrick J. Howard /s/
                     Assistant Inspector General for Audit

SUBJECT:             Final Audit Report
                     “U.S. Department of Education’s Compliance with Improper Payment Reporting
                     Requirements for Fiscal Year 2015”
                     Control Number ED-OIG/A03Q0001

Attached is the subject final audit report that covers the results of our review of the Department’s
compliance with improper payment reporting requirements from October 1, 2014, through
September 30, 2015. We conducted our review at Federal Student Aid’s offices in Washington, D.C.
An electronic copy has been provided to your audit liaison officers. We received your comments
concurring with the findings and mostly concurring with the recommendations in our draft report.

Corrective actions proposed (resolution phase) and implemented (closure phase) by your office will be
monitored and tracked through the Department’s Audit Accountability and Resolution Tracking System
(AARTS). The Department’s policy requires that you develop a final corrective action plan for our
review in the automated system within 30 calendar days of the issuance of this report. The corrective
action plan should set forth the specific action items, and targeted completion dates, necessary to
implement final corrective actions on the findings and recommendations contained in this final audit
report.

In accordance with the Inspector General Act of 1978, as amended, the Office of Inspector General
is required to report to Congress twice a year on the audits that remain unresolved after six months
from the date of issuance.

In accordance with the Freedom of Information Act (5 U.S.C. §552), reports issued by the Office of
Inspector General are available to members of the press and general public to the extent information
contained therein is not subject to exemptions in the Act.

 The Department of Education's mission is to promote student achievement and preparation for global competitiveness by fostering educational
                                                   excellence and ensuring equal access.
We appreciate the cooperation given us during this review. If you have any questions, please call
Bernard Tadley at (215) 656-6279.

Enclosure

cc: John Hurt, Chief Financial Officer, FSA
                                              TABLE OF CONTENTS

                                                                                                                                   Page

EXECUTIVE SUMMARY .......................................................................................................... 1

BACKGROUND ........................................................................................................................... 4

AUDIT RESULTS ........................................................................................................................ 6

   FINDING NO. 1 – The Department Did Not Comply with IPERA Because it Did Not
   Meet the Reduction Target for the Direct Loan Program .................................................... 8

   FINDING NO. 2 – The Department Needs to Improve the Quality of its Improper
   Payments Estimation Methodologies ...................................................................................... 9

   FINDING NO. 3 – The Department Needs to Improve the Accuracy and Reliability
   of its Improper Payments Estimates ..................................................................................... 14

OBJECTIVES, SCOPE, AND METHODOLOGY ................................................................. 19

ENCLOSURE 1: ACRONYMS, ABBREVIATIONS, AND SHORT FORMS USED IN
THIS REPORT ........................................................................................................................... 22

ENCLOSURE 2: AUDITEE COMMENTS ............................................................................. 23
Final Report
ED-OIG/A03Q0001                                                                                 Page 1 of 31


                                    EXECUTIVE SUMMARY


The audit objectives were to (1) determine whether the U.S. Department of Education
(Department) complied with the Improper Payments Elimination and Recovery Act of 2010
(IPERA); (2) evaluate the accuracy and completeness of the Department’s improper payments
reporting; (3) evaluate the Department’s performance in reducing and recapturing improper
payments; and (4) for the high-priority programs (Federal Pell Grant Program and the William
D. Ford Federal Direct Loan Program): (a) evaluate the quality of the Department’s improper
payment estimate and methodology, (b) evaluate the Department’s assessment of the level of risk
associated with the high-priority programs, and (c) review the oversight and financial controls
described by the Department to identify and prevent improper payments.

We found that the Department did not comply with IPERA because the fiscal year (FY) 2015
improper payment rate did not meet the reduction target for the William D. Ford Federal Direct
Loan Program (Direct Loan). The Department established a FY 2015 reduction target of
1.49 percent for the Direct Loan program; however, the improper payment rate for the Direct
Loan program was 2.63 percent after the Department recalculated this rate to correct for the
formula execution errors we identified. Therefore, the Department failed to meet one of
IPERA’s six compliance requirements.

The Department’s estimation methodologies rely significantly on program reviews. The
Department’s ability to address the root causes of improper payments is limited because
primarily relying on program reviews leads to root causes that vary from year to year. Even
though we recommended last year for the Department to analyze root causes of improper
payments and determine whether its internal controls can be implemented, intensified, or
expanded, the Department is limited in its ability to assess progress over time due to these year-
to-year changes.

We also found that the Department’s improper payment methodologies for the Federal Pell Grant
(Pell) and Direct Loan programs were flawed for the following reasons:
    • The estimation methodologies did not include all program reviews that could identify
        improper payments.
    • The estimation methodology for the Pell program excluded sources of improper
        payments (such as the Free Application for Federal Student Aid/Internal Revenue Service
        Data Statistical Study, and fraud). 1
    • The estimation methodologies resulted in volatile improper payment estimates that could
        be significantly influenced by a single program review.
    • The estimation methodologies did not include all improper payments from ineligible
        programs or locations identified in program reviews.



1
  In response to our prior audits, the Department has formed a work group to determine whether its estimation
methodologies can account for improper payments identified in program reviews and improper payments resulting
from inaccurate self-reported income on the FAFSA by Pell and Direct Loan recipients.
Final Report
ED-OIG/A03Q0001                                                                       Page 2 of 31

Using the Office of Management and Budget (OMB)-approved methodology that includes the
flaws listed above, and after the Department’s recalculations to correct the errors we identified,
the improper payment rate for the Pell program was 1.52 percent. This recalculated rate was
below the 2.15 percent reduction target. However, we could not conclude that the Department
actually met its reduction target for the Pell program due to the flaws listed above. Specifically,
the Free Application for Federal Student Aid (FAFSA)/Internal Revenue Service Data Statistical
Study reported an improper payment rate for the Pell program of 2.26 percent, which is above
the reduction target.

The Department’s recalculated estimated improper payments in the Direct Loan program
increased more than $1 billion in FY 2015 compared to FY 2014, and its recalculated estimated
improper payments in the Pell program declined almost $227 million. Again, however, due to
the flaws listed above we could not determine whether the Department reduced or increased
improper payments. We did not evaluate the Department’s performance in recapturing improper
payments because the Department determined, OMB concurred, and we agreed that it was not
cost-effective to conduct a payment recapture audit program.

For FY 2015, OMB classified the Pell and Direct Loan programs as high-priority programs,
which are subject to additional reporting requirements. We found that for these two high-priority
programs, the Department adequately assessed improper payment risks and described its
oversight to identify and prevent improper payments.

We found that the Department’s reported improper payment estimates for both the Pell and
Direct Loan programs were inaccurate and unreliable. Specifically, spreadsheet formulas used in
the calculations were incorrect, and the calculations deviated from the OMB-approved
methodologies in the following ways:
    • The Department proposed that schools with risk scores assigned to the schools by Federal
        Student Aid’s Program Compliance - School Eligibility Service Group would be assigned
        to one of two risk categories (lower risk schools and higher risk schools). However, in
        the spreadsheets the Department used to classify these schools for purposes of calculating
        improper payment estimates, the Department entered an incorrect formula that resulted in
        lower risk schools being assigned to the higher risk category and higher risk schools
        being assigned to the lower risk category.
    • The Department did not use the correct risk score when assigning schools to a risk
        category.
    • The Department did not distribute schools not assigned a risk score proportionally across
        the three categories of schools. The Department assigned all of these schools to one risk
        category.
    • The Department excluded completed, applicable program reviews from the improper
        payment estimates.

Correcting for these formula execution errors, 2 the Department’s recalculated improper payment
rate for the Direct Loan program was 2.63 percent, rather than the initially reported rate of
1.30 percent; the recalculated improper payment rate for the Pell program was 1.52 percent,
rather than the initially reported rate of 1.88 percent.


2
    The formula execution errors are the first three errors noted above.
Final Report
ED-OIG/A03Q0001                                                                     Page 3 of 31

To address the improper payment rates, we recommend that the Chief Financial Officer for the
Department, in conjunction with the Chief Operating Officer for Federal Student Aid—
   • Analyze all available sources that identified improper payments for root causes of such
       improper payments and evaluate FSA’s existing controls to determine whether additional
       controls can be implemented, intensified, or expanded to reduce or prevent improper
       payments.
   • As required by IPERA, if the Director of OMB determines that additional funding is
       needed to help the agency become compliant with IPERA, take the necessary steps to
       implement OMB’s recommendation.

In addition, we recommend that the Chief Financial Officer for the Department, in conjunction
with the Chief Operating Officer for Federal Student Aid, require the Chief Financial Officer for
Federal Student Aid to:
    • Revise the improper payment estimation methodologies to include all improper payments
        in the calculation of the improper payment estimates, such as improper payments
        resulting from recipients submitting inaccurate self-reported income on the Free
        Application for Federal Student Aid, all improper payments resulting from schools
        disbursing Pell and Direct Loan funds to students enrolled in ineligible programs or
        students attending ineligible locations, and other improper payments not identified in
        program reviews.
    • Revise the improper payment estimation methodologies to mitigate the potential for
        volatility that a single program review can have on the improper payment estimate.
    • Disclose in its annual reporting how the methodologies are sensitive to a single
        observation (such as student or school), either by providing examples or noting how
        results are weighted in arriving at the final improper payment estimates.
    • Revise the improper payment estimation methodologies to account for the program
        reviews that do not reach the program review report stage in time for inclusion in that
        fiscal year’s estimated improper payment rates.
    • Publish the FY 2015 recalculated improper payment rates, notify OMB and Congress of
        any changes, and explain the basis for the revisions in the FY 2016 Agency Financial
        Report.
    • Develop, implement, and monitor the effectiveness of internal controls for (1) the
        contractor’s calculation of the improper payment estimates and (2) the Department’s
        oversight and review of the work provided by the contractor.
    • Develop, implement and monitor the effectiveness of internal controls to ensure that all
        applicable program reviews issued prior to the documentation acceptance date are
        included in the improper payment estimates.

In response to the draft audit results, the Department concurred with all three findings and
concurred or partially concurred with all recommendations. We did not make any changes to the
report based on the Department’s comments. We provide the full text of the Department’s
comments in Enclosure 2.
Final Report
ED-OIG/A03Q0001                                                                               Page 4 of 31



                                          BACKGROUND


The Improper Payments Elimination and Recovery Act and Programs Susceptible to
Significant Improper Payments

The Improper Payments Elimination and Recovery Act of 2010 (Public Law 111-204), which
amended the Improper Payments Information Act of 2002 (IPIA) (Public Law 107-300), requires
Federal agencies to reduce improper payments and to report annually on their efforts. The Office
of Management and Budget (OMB) issued government-wide guidance on the implementation of
IPERA on October 20, 2014, which is contained in OMB Circular A-123, Appendix C. 3

IPERA requires each agency, in accordance with guidance prescribed by OMB, to periodically
review all programs and activities that the agency administers and identify all programs and
activities that may be susceptible to significant improper payments. Significant improper
payments are defined as gross annual improper payments (the total amount of overpayments plus
underpayments) in the program exceeding (1) both 1.5 percent of program outlays and
$10 million of all program or activity payments made during the fiscal year reported, or
(2) $100 million (regardless of the improper payment percentage of total program outlays). For
each program and activity identified as susceptible to significant improper payments, the agency
is required to produce a statistically valid estimate, or an estimate that is otherwise appropriate
using a methodology that OMB approved, of the improper payments made by each program and
activity and include those estimates in the accompanying materials to the agency’s annual
financial reports.

IPERA also requires each agency’s Inspector General to determine the agency’s compliance with
the statute for each fiscal year. As specified in the OMB guidance, compliance with IPERA
means that the agency has met all six of the following requirements:
    • published a Performance and Accountability Report or Agency Financial Report (AFR)
        for the most recent fiscal year and posted that report and any accompanying materials
        required by OMB on the agency’s Web site;
    • conducted a program-specific risk assessment for each program or activity that conforms
        with IPERA (if required);
    • published improper payment estimates for all programs and activities identified as
        susceptible to significant improper payments under its risk assessments (if required);
    • published programmatic corrective action plans in the Performance and Accountability
        Report or AFR (if required);
    • published, and met, annual reduction targets for each program assessed to be at risk and
        measured for improper payments; and



3
  Under Section 2(g)(2) of IPIA, as amended, an “improper payment” is any payment that should not have been
made or that was made in an incorrect amount. Under OMB Circular A-123, Appendix C, improper payments also
include any payment lacking sufficient documentation.
Final Report
ED-OIG/A03Q0001                                                                               Page 5 of 31

    •   reported a gross improper payment rate of less than 10 percent for each program and
        activity for which an improper payment estimate was obtained and published in the
        Performance and Accountability Report or AFR.

If an agency does not meet one or more of these requirements, then it is not compliant with
IPERA.

As part of the Inspector General’s review of the agency’s compliance with IPERA, the Inspector
General should also evaluate the accuracy and completeness of the agency’s reporting and
performance in reducing and recapturing improper payments.

The Improper Payments Elimination and Recovery Improvement Act and High-Priority
Programs

The Improper Payments Elimination and Recovery Improvement Act of 2012 (IPERIA) (Public
Law 112-248), requires the Director of OMB to identify a list of high-priority programs for
greater levels of oversight. 4 OMB has designated the Pell program and Direct Loan program as
high-priority programs. OMB issued government-wide guidance on the implementation of
IPERIA on October 20, 2014, which is contained in OMB Circular A-123, Appendix C. The
OMB-established threshold for high-priority program determinations for FY 2015 reporting, and
for subsequent years, is $750 million in estimated improper payments as reported in an agency’s
AFR or Performance and Accountability Report, regardless of the improper payment rate
estimate. IPERIA and OMB guidance require each agency with a high-priority program to
report to its Inspector General and make available to the public, (1) any action that the agency
has taken or plans to take to recover improper payments and (2) any action the agency intends to
take to prevent future improper payments. According to IPERIA and OMB guidance, the agency
Inspector General must review the assessment of the level of risk associated with any high-
priority program and the quality of the improper payment estimates and methodology; determine
the extent of oversight warranted; and provide recommendations, if any, for modifying the
agency’s methodology, promoting continued program access and participation, or maintaining
adequate internal controls.




4
 IPERIA codifies the requirements from Executive Order 13520, “Reducing Improper Payments,” issued
November 20, 2009. OMB Circular A-123, Appendix C implements these requirements.
Final Report
ED-OIG/A03Q0001                                                                     Page 6 of 31



                                     AUDIT RESULTS


We found that the Department did not comply with IPERA because the FY 2015 improper
payment rate did not meet the reduction target for the Direct Loan program. The Department
met the remaining five IPERA compliance requirements. Under IPERA, if the Department does
not meet one or more of the six compliance requirements, then it is not compliant with IPERA.
We found that the Department’s improper payment estimation methodologies and actual
calculations of the improper payment rates for the Pell and Direct Loan programs were flawed,
resulting in estimates that were incomplete and unreliable. As a result, the Department could not
accurately evaluate its performance in reducing improper payments for the Pell and Direct Loan
programs.

Based on our review of the Department’s consolidated risk assessment performed in FY 2014,
we found that the Department adequately assessed improper payment risks for the Pell and
Direct Loan programs. The Department’s risk assessment concluded that these two programs
had high improper payment risk assessment ratings. In addition, we found that the Department
adequately described its oversight and controls to identify and prevent improper payments
through the program review process, increasing the usage of the Internal Revenue Service (IRS)
Data Retrieval Tool, enhanced verification procedures, system edits and data matches, and
compliance audits.

The Department met five of the six compliance requirements of IPERA, as described in the
following.

   1. Published an Annual Financial Report
       The Department complied with the requirement to publish an AFR. Under
       Section 3(a)(3)(A) of IPERA, the Department is required to publish on its Web site its
       AFR and any accompanying materials required under OMB guidance. The Department
       published its AFR, “FY 2015 Improper Payment Estimation Methodologies,” and its
       accompanying materials on November 13, 2015.

   2. Conducted a Risk Assessment
       The Department complied with the requirement to conduct a risk assessment. Under
       Section 3(a)(3)(B) of IPERA, if required, an agency must conduct a program-specific risk
       assessment of all programs and activities to determine which ones are susceptible to
       significant improper payments. In FY 2014, the Department performed risk assessments
       of all Federal Student Aid (FSA) managed programs and the Department’s administrative
       payments (salary, locality pay, travel, purchase card, and transit benefits). In FY 2013,
       the Department performed risk assessments of contract payments and all non-FSA
       programs.
Final Report
ED-OIG/A03Q0001                                                                                      Page 7 of 31

    3. Published Improper Payment Estimates
        The Department complied with the requirement to publish improper payment estimates.
        Under Section 3(a)(3)(C) of IPERA, if required, an agency must publish improper
        payment estimates for programs it identified as being susceptible to significant improper
        payments. As required, the Department published improper payment estimates for
        programs it identified as susceptible to significant improper payments—the Pell and
        Direct Loan programs. The Department also reported an estimated improper payment rate
        for Title I, Part A of the Elementary and Secondary Education Act of 1965, as amended
        (Title I). 5

    4. Published Report on Actions to Reduce Improper Payments (Corrective Action
       Plans)
        The Department complied with the requirement to report on its actions to reduce
        improper payments in programs susceptible to significant improper payments: the Pell
        and Direct Loan programs. Under Section 3(a)(3)(D) of IPERA, the Department is
        required to report on its actions to reduce improper payments for programs it deemed
        susceptible to significant improper payments. The Department also reported on its efforts
        to recapture improper payments in these programs, including reporting that it worked
        with Title IV program participants to resolve and to recover potential improper payments
        identified in compliance audits, Office of Inspector General (OIG) audits, and
        Department-conducted program reviews. The Department submitted to OMB an analysis
        that explained why conducting payment recapture audits for grants, contracts, and the
        Title IV programs would not be cost-effective. On September 21, 2015, OMB approved
        the Department’s analysis.

    5. Reported Improper Payment Rate of Less Than 10 Percent
        The Department complied with the requirement to report improper payment rates of less
        than 10 percent for all applicable programs. Under Section 3(a)(3)(F) of IPERA, the
        Department is required to report estimated improper payment rates of less than 10 percent
        for each program identified as being susceptible to significant improper payments for
        which an improper payment estimate is published. The Department reported estimated
        improper payment rates of 1.88 percent in the Pell program and 1.30 percent in the Direct
        Loan program. For reasons discussed in detail below, the Department recalculated the
        improper payment rates for these two programs, and the recalculated rates were
        1.52 percent and 2.63 percent, respectively. These estimated improper payment rates
        were significantly below the 10 percent threshold.




5
 The Department was not required to report on the Title I program under IPERA because it was not identified as a
program susceptible to significant improper payments. As a result, we did not perform additional work related to the
accuracy and completeness of the Department’s estimation methodology for this program. In February 2016, the
Department submitted a request to OMB for relief from reporting improper payment estimates for the Title I
program.
Final Report
ED-OIG/A03Q0001                                                                          Page 8 of 31

FINDING NO. 1 – The Department Did Not Comply with IPERA Because it Did Not Meet
              the Reduction Target for the Direct Loan Program

The Department did not comply with IPERA because the improper payment rate (which it
recalculated after we identified errors in its original calculation) did not meet the reduction target
for the Direct Loan program. Using the OMB-approved methodology that includes the flaws
detailed in Finding 2 and after the Department’s recalculations to correct the formula execution
errors we identified in Finding 3, the improper payment rate for the Direct Loan program was
2.63 percent which exceeded the reduction target of 1.49 percent. The recalculated improper
payment rate for the Pell program was 1.52 percent. This recalculated rate was below the 2.15
percent reduction target. However, the Free Application for Federal Student Aid (FAFSA)/IRS
Data Statistical Study reported an improper payment rate for the Pell program of 2.26 percent,
which is above the reduction target. As a result, we could not conclude that the Department
actually met its reduction target for the Pell program.

Under Section 3(a)(3)(E) of IPERA, an agency is required to report, and meet, improper payment
reduction targets when a program was identified as susceptible to significant improper payments.
To meet a reduction target, the improper payment rate for a program in the current year must fall
within plus or minus 0.1 percentage points of the reduction target set in the previous year’s AFR.
In its FY 2014 AFR, the Department reported for the Direct Loan program a reduction target of
1.49 percent for FY 2015. Although the Department reported in its FY 2015 AFR an improper
payment rate of 1.30 percent, we identified errors in its original calculation. The Department
recalculated the improper payment rate for the Direct Loan program as 2.63 percent. We
reviewed the calculation and verified that the recalculated estimates addressed the formula
execution errors. The FY 2015 reduction target and the reported and corrected improper
payment rates for the Direct Loan program are shown in Table 1.

Table 1. FY 2015 Direct Loan Reduction Target and Reported and Corrected Improper
Payment Rates
                                       Improper Payment          Corrected
                                       Rate Calculated by        Improper
                                                                                   Reduction Target
  Program       Reduction Target      the Department and       Payment Rate
                                                                                         Met
                                          Reported in         Recalculated by
                                         FY 2015 AFR          the Department

Direct Loan
                      1.49%                  1.30%                  2.63%                  No
Program


The Department also reported in its FY 2015 AFR that the amounts of estimated improper
payments for the Direct Loan program decreased from $1.53 billion in FY 2014 to $1.28 billion
in FY 2015; however, the corrected estimated FY 2015 amounts increased to $2.60 billion.

In its FY 2015 AFR, the Department reported that the root causes of the improper payments
included the failure to verify financial data and administrative or process errors made by other
parties, such as schools. These errors include incorrect awards based on expected family
Final Report
ED-OIG/A03Q0001                                                                                       Page 9 of 31

contribution, 6 incorrect processing of student data, student account data changes not applied or
processed correctly, incorrectly calculated return record, and satisfactory academic progress not
achieved. Errors associated with Direct Loan consolidations include incorrect processing of a
loan verification certificate, processing duplicate loan verification certificates, and incorrect
documentation provided by a servicer. The root causes of the improper payments reported by the
Department have changed from year to year based on the causes of improper payments in
program reviews. This limits the Department’s ability to assess progress in addressing root
causes.

In our FY 2014 audit of the Department’s improper payments, we found that the Department did
not meet the reduction target for the Direct Loan program. In accordance with section 3(c)(1) of
IPERA, the Department submitted a remediation plan to Congress and OMB. Because the
Department has again failed to meet its reduction target for the Direct Loan program, the
Department is not in compliance with IPERA for a second fiscal year. Under Section 3(c)(2) of
IPERA and OMB Circular A-123, Appendix C, if an agency is not in compliance with IPERA
for two consecutive fiscal years for the same program or activity, the Director of OMB will
review the program and determine whether additional funding would help the agency come into
compliance.

Recommendations
We recommend that the Chief Financial Officer for the Department, in conjunction with the
Chief Operating Officer for FSA―

         1.1 Analyze all available sources that identified improper payments for root causes of
             such improper payments and evaluate FSA’s existing controls to determine whether
             additional controls can be implemented, intensified, or expanded to reduce or prevent
             improper payments.
         1.2 As required by IPERA, if the Director of OMB determines that additional funding is
             needed to help the agency become compliant with IPERA, take the necessary steps to
             implement OMB’s recommendation.

Department Comments
The Department concurred with the finding and recommendations.


FINDING NO. 2 – The Department Needs to Improve the Quality of its Improper
              Payments Estimation Methodologies

We found that the estimation methodologies the Department used to calculate the improper
payment rates for the Pell and Direct Loan programs were flawed, resulting in estimates that
were incomplete and unreliable. As a result, the Department could not accurately evaluate its
performance in reducing improper payments for the Pell and Direct Loan programs.

Under IPERA and OMB Circular A-123, Appendix C, agencies are required to prepare a
statistically valid estimate of improper payments or an estimate that is otherwise appropriate

6
 The expected family contribution errors include (1) the inaccurate calculation of the expected family contribution
amount and (2) incorrect award amount based on the student’s expected family contribution.
Final Report
ED-OIG/A03Q0001                                                                                   Page 10 of 31

using an alternative methodology that OMB approves. For FY 2015, rather than preparing a
statistically valid estimate, the Department prepared its improper payment estimates for the Pell
and Direct Loan programs using alternative methodologies. The Controller of OMB approved
these alternative methodologies on October 20, 2015.

The Department’s Estimation Methodologies Were Flawed
The Department’s estimation methodologies used a cutoff date, which it refers to as the
documentation acceptance date, for program reviews conducted during a fiscal year; therefore,
not all programs reviews that could identify improper payments are included it its estimates, and
some reviews will never be included. In addition, the Department’s estimation methodologies
primarily relied on the results of program reviews and excluded other sources of improper
payments. The methodologies also can lead to volatile estimates because they could be
significantly influenced by a single program review, allowing for a single school or even a
student to significantly impact the improper payment rates. We also found the methodologies
excluded some improper payments identified in program review reports.

The Department’s Methodologies Do Not Include All Program Reviews That Could Identify
Improper Payments
The Department based the improper payment estimates for the Pell program solely on program
reviews, and for the Direct Loan program, primarily on program reviews that FSA conducted.
The improper payment estimate for the Pell program was based on the results of 130 program
reviews of schools that FSA’s School Eligibility Service Group (SESG) conducted during
FY 2014; the improper payment estimate was based therefore on the testing of disbursements
made to 1,444 students for the 2012–2013 award year. 7 In FY 2014, 5,702 schools participated
in the Pell program and Pell grants were disbursed to 8,954,468 recipients.

The improper payment estimate for the Direct Loan program was based on three components.
The first component consisted of the results of 133 program reviews of schools that SESG
conducted during FY 2014; this component of the improper payment estimate was based
therefore on the testing of disbursements made to 1,635 students for the 2012–2013 award year.
In FY 2014, 6,274 schools participated in the Direct Loan program and loans were disbursed to
10,163,311 borrowers. The second component consisted of testing a sample of 120 Direct Loan
consolidations (from a universe of 3,362,246) to determine which were considered to be
improper payments. 8 The third component consisted of testing a sample of 120 Direct Loan
refund payments (from a universe of 387,242) to determine which were considered to be
improper payments. The samples for the second and third component were drawn from
payments made from July 2014 through June 2015. The Department then combined the
estimated improper payments for all three components to estimate an overall improper payment
rate for the Direct Loan program. 9

7
  This methodology differed from the methodology FSA used before FY 2014. In prior years, FSA based the
estimate on the FAFSA/IRS Data Statistical Study. That study used tax return and FAFSA data for a sample of
students and parents to calculate an improper payment rate based on recalculated Pell awards where income figures
reported on the tax return did not match those reported on the FAFSA.
8
  The Direct Loan consolidations include both overpayments and underpayments. The sampled payments were
tested to determine which were actual improper payments.
9
  The Direct Loan program reviews account for 87.9 percent of the total disbursements used to estimate the Direct
Loan program improper payment rate, while the Direct Loan consolidations and refunds account for a combined
12.1 percent.
Final Report
ED-OIG/A03Q0001                                                                                         Page 11 of 31


In its OMB-approved methodology, the Department reported that to estimate improper payments
for the Pell and Direct loan programs, it would perform 337 program reviews. The methodology
also states that only a portion of these program reviews will be available to estimate the improper
payment rates. Of the 337 program reviews, 52 were performed for reasons that would not
identify improper payments (for example, reviews of campus crime, closed schools, or third-
party servicers) and 285 were performed for reasons that could identify improper payments and
therefore could be applicable to the improper payment estimates. Of the 285 program reviews
that could identify improper payments, 240 reached the draft report stage before the
documentation acceptance date. 10 The other 45 did not reach the program review report stage
before the cutoff date and were excluded from the improper payment estimates. For 33 of those
45 program reviews, SESG made a preliminary determination that the level of findings identified
during the program reviews represented serious deficiencies, very serious deficiencies, or fraud
or abuse. 11 Under the Department’s estimation methodologies, program reviews that are not
considered by the Department in its improper payment estimates for one year are not considered
in any subsequent year’s improper payment estimates. Because those 45 program reviews were
conducted in FY 2014, they will be excluded from next year’s estimate, which will only consider
program reviews conducted during FY 2015 (October 1, 2015 through September 30, 2016).

The Department’s Methodology for the Pell Program Excludes Other Sources of Improper
Payments
In prior years, we reported that the estimation methodology for the Pell program based on
program reviews was flawed because it excluded other sources of improper payments, such as
the results of the FAFSA/IRS Data Statistical Study, which focuses on inaccurate self-reported
income on the FAFSA and fraud. 12

While the Department’s current methodology is approved by OMB, we again found that
improper payment estimates may be understated by using program reviews as the primary source
of information to estimate improper payments. For example, program reviews do not identify
improper payments associated with recipients who do not use the IRS Data Retrieval Tool, who
provide inaccurate self-reported income on the FAFSA, and who are not selected for income
verification. 13

Program reviews cannot identify inaccurate self-reported income on the FAFSA as a significant
cause of improper payments. As a result, estimates of improper payments based solely on

10
   FSA reported that program reviews were excluded from the improper payment estimates for a variety of reasons,
such as the supporting documentation was not available by the documentation acceptance date, and the award year
subject to the program review was not 2012-2013.
11
   According to SESG, serious deficiencies represent findings that are more serious in nature, occur more frequently,
and/or may lead to liabilities or fines; very serious deficiencies represent findings of serious failure to adhere to
regulatory requirements and that usually result in a significant liability; and fraud or abuse represent findings that the
school is suspected of using Federal funds for its own purposes and has purposely circumvented regulatory
requirements.
12
   “U.S. Department of Education’s Compliance with Executive Order 13520, ‘Reducing Improper Payments’ for
Fiscal Year 2012 and 2013” (A03N0004); “U.S. Department of Education’s Compliance with the Improper
Payments Elimination and Recovery Act of 2010 for Fiscal Year 2012;” and “U.S. Department of Education’s
Compliance with Improper Payment Reporting Requirements for Fiscal Year 2014.”
13
   The IRS Data Retrieval Tool enables financial aid applicants to transfer certain income tax information from an
IRS Web site directly to their online FAFSA.
Final Report
ED-OIG/A03Q0001                                                                                Page 12 of 31

program reviews do not adequately measure all significant improper payments in the Pell
program. Program reviews can identify multiple root causes of improper payments. In the
Department’s FY 2015 analysis of the root causes of Pell program improper payments, the two
leading causes for improper payments were identified as a student’s ineligibility for a Pell grant
(about 38 percent of improper payments) and an incorrectly calculated Title IV return record
(about 26 percent of improper payments). Neither of these causes is directly associated with Pell
grant recipients who submit inaccurate self-reported income on the FAFSA. Furthermore, the
Department did not identify inaccurate self-reported income on the FAFSA as a unique root
cause of improper payments in the FY 2015 AFR. However, in FYs 2011, 2012, and 2013, the
Department identified inaccurate self-reported income on the FAFSA as the most significant root
cause of Pell program improper payments. In those fiscal years, the Department based its
improper payment estimates for the Pell program solely on the FAFSA/IRS Data Statistical
Study. In fact, as shown in Table 2, for award year 2012–2013, the Department estimated a
higher improper payment rate based on the FAFSA/IRS Data Statistical Study than the rate it
reported based on program reviews, which cover multiple root causes of improper payments (see
Table 2).

Table 2. Pell Program Improper Payment Rates From Program Reviews and FAFSA/IRS
Data Statistical Study

         Source of Improper                               Improper            Improper Payment
                                    Award Year
          Payment Estimate                              Payment Rate              Amount
       Program Reviews               2012–2013           1.52 percent           $454.62 million

       FAFSA/IRS Data                2012–2013           2.26 percent           $675.95 million
       Statistical Study

      Note: The improper payment rate and amount from program reviews displayed in the table was
      recalculated by the Department to correct for the errors identified by the OIG. The improper
      payment amounts are based on the award-year outlays, multiplied by the improper payment rates.
      For 2012–2013, the Pell outlays were $29,909.28 million.

In response to our FY 2014 IPERA audit, the Department formed a work group to determine
whether its estimation methodologies can account for improper payments identified in program
reviews and improper payments resulting from inaccurate self-reported income on the FAFSA by
Pell and Direct Loan recipients.

The Department’s Estimation Methodologies Resulted in Volatile Improper Payment Estimates
The Department’s estimation methodologies are based on SESG’s FY 2014 Compliance
Initiative plan for program reviews of schools. The plan identifies the percentage of program
reviews that the Department will conduct in risk-based categories, which focus more on higher
risk schools. For example, the FY 2014 plan indicates that at least 50 percent of program
reviews would be conducted at higher risk schools, about 10 percent of the program reviews
would be conducted at lower risk schools, and about 40 percent of the program reviews would be
initiated based on other programmatic priorities.

Fewer program reviews are conducted at lower risk schools, but this category of schools
accounts for a majority of the Direct Loan program disbursements. In the FY 2015 improper
payment estimate for the Direct Loan program, the lower risk schools accounted for about
Final Report
ED-OIG/A03Q0001                                                                                Page 13 of 31

$80.70 billion (78 percent) of the total $103.25 billion of new Direct Loan disbursements.
However, the estimate for the category of lower risk schools was based on the results of only
7 program reviews that sampled a total of 101 students. Although the disbursement amounts to
these 101 students ($1,244,789) accounted for only about 7 percent of the total disbursement
amounts to the 1,635 total students ($17,148,305) sampled for all three categories, their results
were extrapolated to represent 78 percent of all Direct Loan disbursements. A similar imbalance
exists for data supporting the Pell program improper payment estimate. Consequently, including
fewer program reviews of lower risk schools can result in a significant impact on the improper
payment estimates.

The disproportionate impact of the few program reviews at lower risk schools included in the
estimates was compounded by the relatively small sample sizes of students tested for program
reviews at each school. For program reviews that involve student-level testing, generally,
2 award years are reviewed with about 15 students sampled from each award year. To estimate
an improper payment rate, the results of the student-level tests are first extrapolated to the entire
school’s disbursement amount, and that result is then extrapolated to the total disbursement
amount for the risk category. Therefore, student-level test results for a small number of students,
or even one student, can influence the improper payment estimates and introduce the volatility in
the estimates, particularly when few program reviews are extrapolated to the majority of
disbursed dollars for a program.
As an example of the volatility introduced by this methodology, we found that the Direct Loan
program’s improper payment rate was heavily influenced by the results of one program review,
and in particular 1 of the 22 students sampled at that school. That student was associated with an
improper payment of $4,703. To assess the single student’s impact on the Direct Loan
program’s improper payment rate, we recalculated the improper payment rate after removing that
student from the sample. As a result of removing this one student, the estimated improper
payment rate would decrease from 2.63 percent to 1.51 percent, and the estimated total improper
payment amount would decrease from $2.60 billion to $1.49 billion—a difference of
$1.1 billion.

Another potential source of volatility is that improper payment estimates are based on the results
of program reviews and are susceptible to changes in the composition of schools selected for a
program review. For example, from one year to the next, FSA may change the selection of
schools for program reviews by focusing on compliance areas that may not identify improper
payments or would not result in improper payments. 14

The Department’s Estimation Methodologies Excluded Improper Payments Identified in
Program Reviews
According to the Department’s estimation methodologies, only improper payments associated
with disbursements to sampled students would be included in the improper payment estimates.
The Department’s methodologies account only for disbursements to sampled students (that is,
students who are a part of the program review sample) even though all students enrolled in an
ineligible program or at an ineligible location are ineligible for Title IV funds and all
disbursements to such students are considered improper payments in a program review.

14
  For example, program reviews on compliance with data reporting requirements would not identify improper
payments.
Final Report
ED-OIG/A03Q0001                                                                     Page 14 of 31

We identified program reviews at two schools that were included in the Department’s improper
payment estimates that identified improper payments associated with findings of an ineligible
location or an ineligible program. However, the Department did not include the improper
payments associated with ineligible locations or ineligible programs in its calculations of the
improper payment estimates for the Pell and Direct Loan programs because none of the sampled
students reviewed were enrolled in the ineligible program or at an ineligible location. While the
findings of an ineligible location or an ineligible program from these two program reviews were
not included in the improper payment estimates, the program reviews reported Pell improper
payments of $177,050 at one school and $37,646 at the other school. As a result of the
Department’s improper payment estimation methodologies not including all improper payments
identified from program review findings associated with an ineligible location or an ineligible
program, the improper payment rates for the Pell and Direct Loan programs are potentially
understated.

Recommendations
We recommend that the Chief Financial Officer for the Department, in conjunction with the
Chief Operating Officer for FSA, require the Chief Financial Officer for FSA to―

       2.1 Revise the improper payment estimation methodologies to include all improper
           payments in the calculation of the improper payment estimates, such as improper
           payments resulting from recipients submitting inaccurate self-reported income on the
           FAFSA, all improper payments resulting from schools disbursing Pell and Direct
           Loan funds to students enrolled in ineligible programs or students attending ineligible
           locations, and other improper payments not identified in program reviews.
       2.2 Revise the improper payment estimation methodologies to mitigate the potential for
           volatility that a single program review can have on the improper payment estimate.
       2.3 Disclose in its annual reporting how the methodologies are sensitive to a single
           observation (such as student or school), either by providing examples or noting how
           results are weighted in arriving at the final improper payment estimates.
       2.4 Revise the improper payment estimation methodologies to account for the program
           reviews that do not reach the program review report stage in time for inclusion in that
           fiscal year’s estimated improper payment rates.

Department Comments
The Department concurred with the finding and recommendations.


FINDING NO. 3 – The Department Needs to Improve the Accuracy and Reliability of its
              Improper Payments Estimates

We found that the Department’s reported FY 2015 improper payment estimates for the Pell and
Direct Loan programs were inaccurate and unreliable because spreadsheet formulas used in its
calculations were incorrect and the calculations deviated from the OMB-approved
methodologies. As a result, the Department could not accurately evaluate its performance in
reducing improper payments for the Pell and Direct Loan programs.
Final Report
ED-OIG/A03Q0001                                                                                     Page 15 of 31

The Department’s Calculations of Improper Payment Estimates Deviated from the OMB-
Approved Methodologies
The Department’s improper payment estimates for the Pell and Direct Loan programs were not
accurate and deviated from the OMB-approved methodologies in four ways. First, the
Department used an incorrect spreadsheet formula to assign schools to a risk category. Second,
the Department did not use the correct risk score to assign schools to a risk category. Third, the
Department did not distribute schools across the three categories as proposed. Fourth, the
Department excluded completed, applicable program reviews from the improper payment
estimates. As a result of these errors, the improper payment estimates were incorrect. After we
brought the execution errors to the Department’s attention, the Department recalculated the
improper payment rates to address the first three errors; however, the Department did not
recalculate the rates to address the excluded program reviews (see Table 3).

Table 3. Program Review Categories and Improper Payment Rates and Amounts, Based on
the Incorrect Formula and the Department’s Corrected Formula (Amounts in Millions)
                                                   Direct Loan
                           Direct Loan                                                            Pell Program
 Program Review                                     Program               Pell Program
                            Program                                                               Department’s
    Category                                      Department’s          Incorrect Formula
                        Incorrect Formula                                                       Corrected Formula
                                                Corrected Formula
Lower Risk
Disbursement                 $5,136.45               $80,696.62              $2,896.84               $23,192.19
Amount

Higher Risk
Disbursement                $85,351.39               $16,426.56              $26,933.45              $7,479.18
Amount

Other Reasons
Disbursement                $12,760.57               $6,125.23               $2,373.34               $1,532.26
Amount

Improper
                              1.30%                    2.63%                   1.88%                   1.52%
Payment Rates

Improper
                             $1,284.03               $2,597.69                $562.29                 $454.62
Payment Amount

Note: The improper payment amount is calculated by multiplying the outlays times the rate. In the Department’s
FY 2015 AFR, it reports that Pell outlays are $29,909.28 and Direct Loan outlays are $98,771.65.

The Department Used an Erroneous Formula to Assign Schools to a Risk Category
The OMB-approved methodologies describe how the Department will estimate improper
payments for the Pell and Direct Loan programs. According to the methodologies, the
Department leverages program reviews of schools for its calculations of the improper payment
rates for these programs. SESG issues an annual Compliance Initiative plan that outlines its plan
for the review year. The FY 2014 Compliance Initiative plan states that schools selected for
program review are assigned to one of three separate categories: higher risk, lower risk, and
other reasons (other programmatic reasons not based on risk scores). 15 SESG generated a risk

15
  The other reasons category includes schools selected for program review based on comprehensive compliance
reviews, referrals, complaints, and self-reported violations. These schools were not selected based on their risk
scores.
Final Report
ED-OIG/A03Q0001                                                                                   Page 16 of 31

score for schools that participated in the Title IV programs in award year 2012-2013. 16
However, in the improper payment rate calculations, the Department’s contractor used an
incorrect formula that assigned schools to the opposite risk category: lower risk schools were
assigned to the higher risk category and higher risk schools were assigned to the lower risk
category. As a result of using an incorrect formula, the reported improper payment estimates
were inaccurate, and significantly different from the Department’s corrected improper payment
estimates. This error had the most significant impact on the improper payment estimates.

We concluded that a data-entry error was the cause of the incorrect formula that the Department
used to assign schools to a risk category. The Department’s contractor acknowledged the data-
entry error on its part, and the Department acknowledged its failure to identify the error. Schools
that are assigned to an incorrect risk category cause its improper payment rates to be applied to
the incorrect total disbursements for each risk category.

The Department Failed to Use the Correct Risk Score to Assign Schools to a Risk Category
The Department’s improper payment calculations deviated from the OMB-approved
methodologies for the Direct Loan and Pell programs by not using the modified risk score when
assigning schools to a risk category. The OMB-approved methodologies indicate that the
Department would use risk scores provided in the Compliance Initiative plan, which indicated
that schools would be categorized using a modified risk score. However, when the Department
assigned schools to a risk category for purposes of estimating improper payments, it used the
total risk score and not the modified risk score. The Department’s contractor stated that it never
communicated with the Department specifically regarding the use of the total risk score to assign
the schools to a risk category for the purpose of calculating improper payments.

The Department Failed to Distribute Schools Proportionally Across Categories
The Department’s improper payment calculations deviated from the OMB-approved
methodologies for accounting for schools not selected for program review and not assigned a risk
score by SESG. According to the OMB-approved methodologies, these schools should be
distributed proportionally across the three categories (lower risk, higher risk, and other reasons).
However, all the schools that were not selected for program review and not assigned a risk score
by SESG were assigned to the other reasons category. The Department’s contractor stated that it
decided to assign these schools to the other reasons category because it did not want to affect the
results for the two risk categories determined by risk score (lower risk and higher risk). After we
notified the Department of this deviation from the OMB-approved methodologies, the
Department reviewed the schools incorrectly assigned to the other reasons category and
identified for reassignment the 790 schools with Direct Loan disbursements and the 209 schools
with Pell disbursements that were not selected for program review and not assigned a risk score
and were not proportionally distributed across the three risk categories. 17




16
   The FY 2015 improper payment estimates were based on program reviews started during FY 2014, and which
reviewed Title IV disbursements for the 2012–2013 award-year.
17
   A total of 6,274 schools participated in the Direct Loan program and 5,702 schools participated in the Pell
program.
Final Report
ED-OIG/A03Q0001                                                                                 Page 17 of 31

The Department Excluded Completed, Applicable Program Reviews from its Improper Payment
Estimates
The Department’s Pell and Direct Loan improper payment estimates are incomplete because
some program reviews that should have been included were excluded from the calculations. In
our review of the program review reports, we identified 54 completed and applicable program
reviews that the Department excluded because some or all supporting documentation 18 was not
in the contractor’s possession before the documentation acceptance date of October 5, 2015.
These program reviews could have identified improper payments. We reviewed 16 of these
54 program reviews and found that for 13, sufficient supporting documentation was available.
Therefore, these program reviews should have been included in the improper payment estimates.
We reviewed the contractor’s actions, consisting primarily of email communications, to obtain
supporting documentation from SESG. The contractor was not able to obtain the specific
information it needed to include the program reviews in the improper payment estimates. We
found internal control weaknesses within the Department: FSA Finance did not ensure that the
contractor actively and thoroughly requested the needed information from SESG. As a result, at
least 13 applicable program reviews were excluded from the calculations of the FY 2015
improper payment estimates and may never be considered in any subsequent year’s improper
payment estimates.

Recommendations
We recommend that the Chief Financial Officer for the Department, in conjunction with the
Chief Operating Officer for FSA, require the Chief Financial Officer for FSA to―

     3.1 Publish the FY 2015 recalculated improper payment rates, notify OMB and Congress of
         any changes, and explain the basis for the revisions in the FY 2016 AFR.
     3.2 Develop, implement, and monitor the effectiveness of internal controls for (a) the
         contractor’s calculation of the improper payment estimates, and (b) the Department’s
         oversight and review of the work provided by the contractor.
     3.3 Develop, implement, and monitor the effectiveness of internal controls to ensure that all
         program reviews applicable and issued before the documentation acceptance date are
         included in the improper payment estimates.

Department Comments

The Department concurred with the finding and Recommendations 3.1 and 3.2. The Department
partially concurred with Recommendation 3.3, stating that it is not feasible to include all
program reviews applicable and issued before the documentation acceptance date in the improper
payment estimates. In addition, the Department stated that it will evaluate the effectiveness of
existing internal controls for timely obtaining program review documentation.

OIG Response

We disagree with the Department’s comments on Recommendation 3.3. For the FY 2015
improper payment estimates, the documentation acceptance date was October 5, 2015, which
allowed the Department more than 3 weeks to obtain the necessary supporting documentation for

18
  The program review support documentation included an appendix with the sampled students listed and a recipient
data spreadsheet that listed all disbursements at the student level.
Final Report
ED-OIG/A03Q0001                                                                    Page 18 of 31

all applicable and issued program reviews. The supporting documentation for a program review
primarily consists of a list of sampled students and the Title IV disbursements made to these
students (recipient data spreadsheet). It is a reasonable expectation that both documents should
be available within 3 weeks of a program review’s issuance date.
Final Report
ED-OIG/A03Q0001                                                                    Page 19 of 31



                 OBJECTIVES, SCOPE, AND METHODOLOGY


The audit objectives were to (1) determine whether the Department complied with IPERA;
(2) evaluate the accuracy and completeness of the Department’s improper payments reporting;
(3) evaluate the Department’s performance in reducing and recapturing improper payments; and
(4) for the high-priority programs (Pell Grant and Direct Loan): (a) evaluate the quality of the
Department’s improper payment estimate and methodology, (b) evaluate the Department’s
assessment of the level of risk associated with the high-priority programs, and (c) review the
oversight and financial controls described by the Department to identify and prevent improper
payments.

Our audit covered the Department’s improper payment measurement methodologies, reporting,
and performance in reducing and recapturing improper payments for the Pell and Direct Loan
programs from October 1, 2014, through September 30, 2015.

Our audit was for the limited purpose described and would not necessarily identify all
deficiencies in internal controls. To accomplish our objectives, we gained an understanding of
internal controls applicable to the Department’s compliance efforts with IPERA and
development of its improper payment rate estimates, as detailed below. We determined that
control activities were significant to our audit objective. We reviewed and tested control
activities pertaining to the Department’s calculations of improper payment estimates, support for
such calculations, and improper payment reporting. We identified deficiencies in the
Department’s control activities; we fully discuss these deficiencies in Finding 3 of this report.
Specifically, we performed the following:

   1. Reviewed background information about the Department and its programs susceptible to
      improper payments (Pell and Direct Loan programs).

   2. Reviewed the following laws, regulations, and guidance to gain an understanding of the
      improper payment reporting requirements that the Department was required to follow:
         a. Improper Payments Elimination and Recovery Improvement Act of 2012;
         b. Improper Payments Elimination and Recovery Act of 2010;
         c. Improper Payments Information Act of 2002;
         d. OMB Circular A-123, Appendix C, “Requirements for Effective Estimation and
            Remediation of Improper Payments,” October 20, 2014;
         e. Executive Order 13520, “Reducing Improper Payments,” November 20, 2009;
         f. OMB Circular A-136, Section II.5.8., “IPIA (as amended by IPERA and IPERIA)
            Reporting Details,” August 4, 2015; and
         g. Guidance developed by the U.S. Social Security Administration OIG for the
            Council of Inspectors General on Integrity and Efficiency for evaluating an
            agency’s compliance with the improper payment requirements.

   3. Reviewed the Department’s FY 2015 Agency Financial Report, and specifically, the
      “Improper Payments Reporting Details” section, the “FY 2015 Alternative Improper
      Payment Estimation Methodologies,” and the documentation supporting the
Final Report
ED-OIG/A03Q0001                                                                  Page 20 of 31

     Department’s improper payment measurement methodologies, reporting, and
     performance in reducing and recapturing improper payments.

  4. Reviewed the Department’s “FY 2014 Assessment of Improper Payments Risk
     Assessment Plan and Results” to determine compliance with improper payment reporting
     requirements.

  5. Interviewed officials from FSA’s Finance, Customer Experience, and Program
     Compliance – SESG groups, and FSA’s designated contractor (PricewaterhouseCoopers)
     for its OMB Circular A-123, “Management’s Responsibility for Internal Control
     Testing.”

  6. Reviewed prior OIG audit reports relevant to our audit objectives, including:
        a. “U.S. Department of Education’s Compliance With Improper Payment Reporting
           Requirements for Fiscal Year 2014” (A03P0003);
        b. “Federal Student Aid’s Oversight of Schools Participating in the Title IV
           Programs” (A03L0001);
        c. “U.S. Department of Education's Compliance with the Improper Payments
           Elimination and Recovery Act of 2010 for Fiscal Year 2013” (A19O0002);
        d. “U.S. Department of Education’s Compliance with Executive Order 13520,
           ‘Reducing Improper Payments’ for Fiscal Year 2012 and 2013” (A03N0004);
        e. “U.S. Department of Education’s Compliance with the Improper Payments
           Elimination and Recovery Act of 2010 for Fiscal Year 2012” (A03N0001);
        f. “U.S. Department of Education's Compliance with the Improper Payments
           Elimination and Recovery Act of 2010 for Fiscal Year 2011” (A03M0001); and
        g. “U.S. Department of Education’s Compliance with Executive Order 13520,
           ‘Reducing Improper Payments’ for Fiscal Year 2011” (A03M0004).

  7. We reviewed the Department’s recalculated improper payment estimates that it provided
     based on the errors we identified. While we did not audit supporting documentation for
     the recalculated estimates, we reviewed the work provided by the Department and
     verified that the recalculated estimates addressed each error.

  Sampling Methodology

  Of the 337 program reviews performed by the Department during FY 2014, the Department
  reported that 240 were performed for reasons that could identify improper payments and
  were completed by October 5, 2015. Of the 240 program reviews, we judgmentally selected
  for review 21 program reviews and the associated supporting documentation.

  We judgmentally selected 16 program reviews to determine whether the Department
  correctly excluded these program reviews from the improper payment estimates.
      • From a population of 90 program reviews that were excluded from the improper
          payment estimates for reasons the Department cited, including 1) the supporting
          documentation was not available in time for the FY 2015 estimates, and 2) the
          program reviews were not applicable to the FY 2015 estimates, we selected a
          judgmental sample of 6 program reviews associated with the 5 schools with the
          highest Title IV disbursements for the 2012-2013 award year.
Final Report
ED-OIG/A03Q0001                                                                    Page 21 of 31

       •   We also selected 4 program reviews that had a deficiency related to an ineligible
           location or program. These 4 program reviews were included in the 90 program
           reviews excluded from the improper payment estimates.
       •   We selected 6 program reviews from a population of 7 program reviews that the
           Department identified as excluded from the improper payment estimates because all
           the supporting documentation was not available in time for the FY 2015 estimates.
           The seventh program review report and supporting documentation was not available
           for us to review.

Of the 240 program reviews, we judgmentally selected all 5 program reviews that were included
in the improper payment estimates, that had a deficiency related to an ineligible location or
program and where the improper payment estimates did not include any improper payments
amounts associated with these deficiencies. We reviewed these to determine the improper
payment amounts from deficiencies related to an ineligible location or program that the
Department excluded from its improper payment estimates.

Because there is no assurance that the judgmental sample was representative of the entire
universe, the results should not be projected over the universe of program reviews.

Data Reliability

Our use of computer-processed data for the audit was limited to documentation provided by the
Department to support its improper payment rate estimates and progress in reducing and
recapturing improper payments. We used the data to evaluate the accuracy and completeness of
the Department’s estimation methodologies for the Pell and Direct Loan programs. We assessed
the reliability of this data by comparing reported data to data contained in the supporting
documentation and by interviewing Department officials and its contractor knowledgeable about
the data. Based on our analysis, we concluded that the computer-processed data were
sufficiently reliable for the purposes of our audit.

We conducted onsite visits at the Department’s offices located in Washington, D.C., on
December 14–15, 2015, and February 16–18, 2016. We briefed Department officials on the
results of our audit on April 4, 2016.

We conducted this audit in accordance with generally accepted government auditing standards.
Those standards require that we plan and perform the audit to obtain sufficient, appropriate
evidence to provide a reasonable basis for our findings and conclusions based on our audit
objectives. We believe that the evidence obtained provides a reasonable basis for our findings
and conclusions based on our audit objectives.
Final Report
ED-OIG/A03Q0001                                                              Page 22 of 31

Enclosure 1: Acronyms, Abbreviations, and Short Forms Used in This Report


AFR               Agency Financial Report
Department        U.S. Department of Education
Direct Loan       William D. Ford Federal Direct Loan
FAFSA             Free Application for Federal Student Aid
FSA               Federal Student Aid
FY                Fiscal Year
IPERA             Improper Payments Elimination and Recovery Act
IRS               Internal Revenue Service
OIG               Office of Inspector General
OMB               Office of Management and Budget
Pell              Federal Pell Grant
SESG              School Eligibility Service Group
Title I           Title I, Part A of the Elementary and Secondary Education Act of 1965, as
                  amended
Final Report
ED-OIG/A03Q0001                                   Page 23 of 31

                  Enclosure 2: Auditee Comments
Final Report
ED-OIG/A03Q0001   Page 24 of 31
Final Report
ED-OIG/A03Q0001   Page 25 of 31
Final Report
ED-OIG/A03Q0001   Page 26 of 31
Final Report
ED-OIG/A03Q0001   Page 27 of 31
Final Report
ED-OIG/A03Q0001   Page 28 of 31
Final Report
ED-OIG/A03Q0001   Page 29 of 31
Final Report
ED-OIG/A03Q0001   Page 30 of 31
Final Report
ED-OIG/A03Q0001   Page 31 of 31