oversight

U.S. Department of Education's Compliance with Improper Payment Reporting Requirements for Fiscal Year 2014

Published by the Department of Education, Office of Inspector General on 2015-05-15.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

U.S. Department of Education’s Compliance With Improper
  Payment Reporting Requirements for Fiscal Year 2014


                                 FINAL AUDIT REPORT




                                    ED-OIG/A03P0003
                                       May 2015


Our mission is to promote the                         U.S Department of Education
efficiency, effectiveness, and                        Office of Inspector General
integrity of the Department’s                         Washington, D.C.
programs and operations.
                       NOTICE

Statements that managerial practices need improvements, as well as
other conclusions and recommendations in this report, represent the
  opinions of the Office of Inspector General. Determinations of
    corrective action to be taken will be made by the appropriate
                 Department of Education officials.

In accordance with the Freedom of Information Act (5 U.S.C. § 552),
  reports issued by the Office of Inspector General are available to
  members of the press and general public to the extent information
      contained therein is not subject to exemptions in the Act.
                                    UNITED STATES DEPARTMENT OF EDUCATION
                                                    OFFICE OF INSPECTOR GENERAL

                                                                                                                     AUDIT SERVICES



                                                                     May 15, 2015

Memorandum
TO:                  Thomas P. Skelly
                     Delegated the Authority to Perform the Functions and Duties
                     of the Chief Financial Officer
                     Office of Chief Financial Officer
                     Lead Action Official

                     James W. Runcie
                     Chief Operating Officer
                     Federal Student Aid

FROM:                Patrick J. Howard /s/
                     Assistant Inspector General for Audit

SUBJECT:             Final Audit Report
                     “U.S. Department of Education’s Compliance with Improper Payment Reporting
                     Requirements for Fiscal Year 2014”
                     Control Number ED-OIG/A03P0003

Attached is the subject final audit report that covers the results of our review of the Department’s
compliance with improper payment reporting requirements from October 1, 2013, through
September 30, 2014. We conducted our review at Federal Student Aid’s offices in Washington,
D.C. An electronic copy has been provided to your audit liaison officers. We received your
comments partially concurring with the findings and recommendations.

Corrective actions proposed (resolution phase) and implemented (closure phase) by your office
will be monitored and tracked through the Department’s Audit Accountability and Resolution
Tracking System (AARTS). The Department’s policy requires that you develop a final
corrective action plan for our review in the automated system within 30 calendar days of the
issuance of this report. The corrective action plan should set forth the specific action items, and
targeted completion dates, necessary to implement final corrective actions on the findings and
recommendations contained in this final audit report.

In accordance with the Inspector General Act of 1978, as amended, the Office of Inspector
General is required to report to Congress twice a year on the audits that remain unresolved after
6 months from the date of issuance.

In accordance with the Freedom of Information Act (5 U.S.C. §552), reports issued by the Office
of Inspector General are available to members of the press and general public to the extent
information contained therein is not subject to exemptions in the Act.

 The Department of Education's mission is to promote student achievement and preparation for global competitiveness by fostering educational
                                                   excellence and ensuring equal access.
We appreciate the cooperation given us during this review. If you have any questions, please
call Bernard Tadley at (215) 656-6279.

Enclosure

cc:    John Hurt, Chief Financial Officer, FSA
                                              TABLE OF CONTENTS

                                                                                                                                 Page

EXECUTIVE SUMMARY ...........................................................................................................1

BACKGROUND ............................................................................................................................4

AUDIT RESULTS .........................................................................................................................7

             FINDING NO. 1 – The Department Failed to Comply with IPERA Because
                             it Did Not Meet the Reduction Target for the Direct
                             Loan Program ................................................................................8

             FINDING NO. 2 – The Department Needs to Improve the Accuracy,
                             Completeness, and Quality of its Improper Payment
                             Estimates and Estimation Methodologies ..................................10

             FINDING NO. 3 – The Department Needs to Improve the Completeness of
                             its Improper Payment Reporting ...............................................20

OBJECTIVES, SCOPE, AND METHODOLOGY ..................................................................23

Enclosure 1: Acronyms, Abbreviations, and Short Forms Used in This Report

Enclosure 2: Auditee Comments
Final Report
ED-OIG/A03P0003                                                                     Page 1 of 24


                               EXECUTIVE SUMMARY


The Improper Payments Information Act of 2002 (Public Law 107-300), as amended by the
Improper Payments Elimination and Recovery Act of 2010 (IPERA) (Public Law 111-204) and
the Improper Payments Elimination and Recovery Improvement Act of 2012 (Public
Law 112-248), requires Federal agencies to reduce improper payments and to report annually on
their efforts. IPERA requires the Inspector General to review the agency’s reporting in its
Agency Financial Report (AFR) and accompanying materials, and to determine whether the
agency has met six compliance requirements. On October 20, 2014, the Office of Management
and Budget (OMB) issued OMB Circular A-123, Appendix C, “Requirements for Effective
Estimation and Remediation of Improper Payments,” which provides agencies with guidance on
implementing these laws and on requirements for the Inspector General’s review of the agency’s
improper payment reporting.

The objectives of our audit were to (1) determine whether the U.S. Department of Education
(Department) complied with IPERA; (2) evaluate the accuracy and completeness of the
Department’s improper payments reporting; (3) evaluate the Department’s performance in
reducing and recapturing improper payments; and (4) for the Federal Pell Grant (Pell) high-
priority program, (a) evaluate the quality of the Department’s improper payment estimate and
methodology, (b) evaluate the Department’s assessment of the level of risk associated with the
program, and (c) review the oversight or financial controls described by the Department to
identify and prevent improper payments.

We found that the Department did not comply with IPERA because it reported an improper
payment rate that did not meet its reduction target for the William D. Ford Federal Direct Loan
(Direct Loan) program. The Department established a fiscal year (FY) 2014 reduction target of
1.03 percent for the Direct Loan program and reported that the estimated improper payment rate
was 1.50 percent for FY 2014. Because the reported estimated improper payment rate for
FY 2014 was higher than the reduction target for FY 2014, the Department failed to meet one of
IPERA’s six compliance requirements. The Department complied with the other five
requirements.

We found that the Department’s reported improper payment estimates and estimation
methodologies in its FY 2014 AFR and accompanying materials for both the Pell and Direct
Loan programs were inaccurate, incomplete and unreliable. Therefore, the Department could not
accurately evaluate its performance in reducing improper payments for the Pell and Direct Loan
programs. Furthermore, we could not conclude that the Department actually met its reduction
target for the Pell program. We did not identify issues with the Federal Family Education Loan
program’s improper payment estimate. We found that the Department recaptured greater
amounts of improper payments in FY 2014 than in FY 2013. For the Pell high-priority program,
the Department adequately assessed improper payment risks and described an adequate level of
oversight to identify and prevent improper payments.

We found that the Department’s improper payment estimates for the Pell and Direct Loan
programs were not accurate and deviated from the OMB-approved methodologies in effect at the
Final Report
ED-OIG/A03P0003                                                                       Page 2 of 24

time the Department’s AFR was issued on November 14, 2014. The Department received initial
approval from the Controller of OMB for its FY 2014 estimation methodologies on
September 17, 2014, and again on November 6, 2014. On November 14, 2014, the Department
issued its AFR and accompanying materials. On March 17, 2015, after the close of our
fieldwork and four months after reporting its improper payment estimates in the AFR, the
Department submitted to OMB a request to change the approved methodologies. The Controller
of OMB retroactively approved the methodology changes on April 3, 2015. The methodologies
deviated from the approved methodologies at the time the AFR was issued, in the following
ways.
    • Under the OMB-approved methodologies in effect at the time the AFR was issued, the
        improper payment estimates were to be calculated, in part, on a school-count-based ratio
        estimate. However, the Department actually used a dollar-based ratio estimate for the
        Pell and Direct Loan programs. We concluded that the Department changed its
        estimation methodologies after learning that the improper payment estimates using the
        methodologies in effect at the time the AFR was issued were higher than the previous
        year’s estimates. After the close of our fieldwork, the Department submitted to OMB a
        request to use a dollar-based ratio estimate; OMB approved this request on April 3, 2015.
    • In the OMB-approved methodologies for the Pell and Direct Loan programs, the
        Department proposed that all final program review reports available from the FY 2013
        program review schedule of 311 program reviews would be included in the calculation of
        the FY 2014 improper payment rates for the Pell and Direct Loan programs. However, at
        the time the improper payment rates were calculated, about 69 percent of these program
        reviews had not been started, had not reached at least the draft program review report
        stage, were not applicable to the Pell or Direct Loan programs, or would not identify
        improper payments. We raised a similar issue in our prior audit of the Department’s
        compliance with IPERA for FY 2013, where we found that the results of 69 percent of
        anticipated program reviews were not included in the calculation of the Direct Loan
        program’s improper payment rate for FY 2013.
    • The OMB-approved methodologies for the Pell and Direct programs listed three strata,
        based on the reason a program review is initiated. However, the Department used only
        two strata in its calculations.

Previously, we found that estimation methodologies based on program reviews were flawed
because they excluded other sources of improper payments. While the Department’s current
methodology is approved by OMB, our concern that improper payment estimates may be
understated by using program reviews as the sole source of information to estimate improper
payments continues to exist.

In addition, the Department’s supporting documentation for its reported improper payment
estimates for the Pell and Direct Loan programs contained data transcription errors, data integrity
errors, and significant formula errors and omissions. Lastly, the Department’s AFR and its
“FY 2014 Improper Payment Estimation Methodologies” were incomplete. The Department did
not provide statistical sampling details for the Pell program and did not report amounts of
improper payments associated with each category of root cause.
Final Report
ED-OIG/A03P0003                                                                      Page 3 of 24

To correct the findings identified, we recommend that the Chief Financial Officer (CFO) for the
Department, in conjunction with the Chief Operating Officer (COO) for Federal Student Aid
(FSA), require the CFO for FSA to:
   • Analyze the program review reports that identified improper payments for root causes
       and evaluate FSA’s existing corrective actions to determine whether additional corrective
       actions can be implemented, intensified, or expanded to reduce or prevent improper
       payments in any program that fails to meet its reduction target.
   • Prepare and submit the required remediation plans to Congress and OMB within 90 days
       of the issuance of this report.
   • Recalculate the FY 2014 improper payment estimates for the Direct Loan and Pell
       programs in accordance with OMB-approved methodologies and correcting for all the
       data, calculation, and estimation methodology errors.
   • Publish the FY 2014 recalculated improper payment rates, notify OMB and Congress of
       any changes, and explain the basis for the revisions in the FY 2015 AFR.
   • Ensure that the results of an adequate number and percentage of program reviews are
       used in the calculation of the FY 2015 improper payment estimates.
   • Ensure that all improper payments are included in the calculation of the improper
       payment estimates.
   • Maintain adequate documentation of the sampling and estimation approaches used.
   • In its annual reporting on improper payments, (1) provide sufficient details as to the
       samples used in calculating the estimated improper payment rates and (2) include the
       error amounts when reporting on the root causes of improper payments.
   • Revise the estimation methodologies to include improper payments that are not identified
       in program reviews, such as those improper payments associated with recipients who do
       not use the IRS Data Retrieval Tool, who provide inaccurate self-reported income on the
       FAFSA, and who are not selected for income verification.

In response to the draft audit results, in its written comments, the Department concurred with our
finding that it failed to comply with IPERA because it did not meet the reduction target for the
Direct Loan program and with our finding that it needs to improve the completeness of its
improper payment reporting. However, in its written comments, the Department partially
concurred with our finding that it needs to improve the accuracy, completeness, and quality of its
improper payment estimates and estimation methodologies. The Department concurred with all
of the initial recommendations. We added a recommendation to this final report, after receiving
the Department’s written comments, for it to revise the estimation methodologies to include
improper payments that are not identified in program reviews. In discussions with the
Department subsequent to its written comments, it did not concur with this recommendation. We
summarized the Department’s comments at the end of each finding. We included the full text of
the Department’s written response as Enclosure 2 to this report. In response to the Department’s
comments, we made minor changes to the report, including specifying the dates that OMB
approved changes to the previously approved estimation methodologies and acknowledging that
program reviews identify both overpayments and underpayments.
Final Report
ED-OIG/A03P0003                                                                      Page 4 of 24



                                      BACKGROUND


The Improper Payments Elimination and Recovery Act and Programs Susceptible to
Significant Improper Payments

The Improper Payments Elimination and Recovery Act of 2010 (Public Law 111-204) (IPERA)
requires each agency, in accordance with guidance prescribed by the Office of Management and
Budget (OMB), to periodically review all programs and activities that the agency administers
and identify all programs and activities that may be susceptible to significant improper payments.
Significant improper payments are defined as gross annual improper payments (the total amount
of overpayments plus underpayments) in the program exceeding (1) both 1.5 percent of program
outlays and $10 million of all program or activity payments made during the fiscal year reported
or (2) $100 million (regardless of the improper payment percentage of total program outlays).
For each program and activity identified as susceptible to significant improper payments, the
agency must produce a statistically valid estimate, or an estimate that is otherwise appropriate
using a methodology that OMB approved, of the improper payments made by each program and
activity and include those estimates in the accompanying materials to the agency’s annual
financial reports. The Department identified the Pell and Direct Loan programs as susceptible to
significant improper payments. The Department assessed the Federal Family Education Loan
(FFEL) program as low risk (not susceptible to significant improper payments) in fiscal year
(FY) 2014 and, consistent with OMB guidance, will seek to formally change the risk
categorization of the FFEL program and remove the requirement for future annual reporting of
improper payment estimates for the FFEL program.

IPERA also requires each agency’s Inspector General to determine the agency’s compliance with
the statute for each fiscal year. As specified in the OMB guidance, compliance with IPERA
means that the agency has met the following six requirements:
    • published a Performance and Accountability Report or Agency Financial Report (AFR)
        for the most recent fiscal year and posted that report and any accompanying materials
        OMB required on the agency Web site;
    • conducted a program-specific risk assessment for each program or activity that conforms
        with IPERA (if required);
    • published improper payment estimates for all programs and activities identified as
        susceptible to significant improper payments under its risk assessments (if required);
    • published programmatic corrective action plans in the Performance and Accountability
        Report or AFR (if required);
    • published and is meeting annual reduction targets for each program assessed to be at risk
        and measured for improper payments; and
    • reported a gross improper payment rate of less than 10 percent for each program and
        activity for which an improper payment estimate was obtained and published in the
        Performance and Accountability Report or AFR.

If an agency does not meet one or more of these requirements, then it is not compliant with
IPERA.
Final Report
ED-OIG/A03P0003                                                                               Page 5 of 24


The OMB guidance also provides that, as part of the Inspector General’s review of the agency’s
compliance with IPERA, the Inspector General should also evaluate the accuracy and
completeness of the agency’s reporting and performance in reducing and recapturing improper
payments.

The Improper Payments Elimination and Recovery Improvement Act and High-Priority
Programs

The Improper Payments Elimination and Recovery Improvement Act of 2012 (IPERIA) (Public
Law 112-248), requires OMB to identify a list of “high-priority” programs for greater levels of
oversight. 1 OMB has designated the Federal Pell Grant (Pell) program as a high-priority
program. OMB issued guidance on the implementation of IPERIA on October 20, 2014, which
is contained in OMB Circular A-123, Appendix C. The OMB-established threshold for high-
priority program determinations for FY 2014 reporting, and for subsequent years, is $750 million
in estimated improper payments as reported in an agency’s AFR or Performance and
Accountability Report, regardless of the improper payment rate estimate. IPERIA and OMB
guidance require each agency that has a high-priority program to report to their Inspector
General, and make available to the public, (1) any action that the agency has taken or plans to
take to recover improper payments and (2) any action the agency intends to take to prevent future
improper payments. The agency Inspector General must review the assessment of the level of
risk associated with any high-priority program and the quality of the improper payment estimates
and methodology; determine the extent of oversight warranted; and provide recommendations, if
any, for modifying the agency’s methodology, promoting continued program access and
participation, or maintaining adequate internal controls.

Change in the Improper Payment Estimation Methodology for the Pell Program

In prior fiscal years, the Department’s methodology for estimating improper payments in the Pell
program was based on the Free Application for Federal Student Aid (FAFSA)/Internal Revenue
Service (IRS) Data Statistical Study. The FAFSA/IRS Data Statistical Study used tax return and
FAFSA data for a sample of students and parents to calculate an improper payment rate based on
recalculated Pell awards where income figures reported on the tax return did not match those
reported on the FAFSA. The FAFSA/IRS Data Statistical Study showed that a percentage of
recipients received the wrong Pell awards based on an expected family contribution (EFC)
derived from incorrect income data on the FAFSA. The FAFSA/IRS Data Statistical Study
focused equally on Pell overpayments and underpayments. Previously, the Department found
that the inaccuracy of self-reported income on the FAFSA was the most significant root cause of
improper payments in the Pell program.

The FY 2014 improper payment estimates for the Pell and Direct Loan programs were based on
the results of program reviews of schools performed by Federal Student Aid (FSA). Program
reviews primarily identify overpayments that need to be returned to the Department rather than
underpayments. Furthermore, unlike the FAFSA/IRS Data Statistical Study, program reviews
cannot identify inaccurate self-reported income on the FAFSA as a significant cause of improper

1
 IPERIA codifies the requirements from Executive Order 13520, “Reducing Improper Payments,” issued
November 20, 2009. OMB Circular A-123, Appendix C implements these requirements.
Final Report
ED-OIG/A03P0003                                                                     Page 6 of 24

payments for applicants who do not use the IRS Data Retrieval Tool and are not selected for
income verification.

OMB’s Approval of the Department’s FY 2014 Estimation Methodologies

The Department received initial approval from the Controller of OMB for its FY 2014 estimation
methodologies on September 17, 2014 (September 2014 OMB-approved methodologies). On
November 6, 2014, the Department requested and received OMB approval to change the
estimation methodologies for the Pell and Direct Loan programs to include draft program review
reports, in addition to final program review reports, in its calculations. On November 14, 2014,
the Department issued its AFR and accompanying materials. On March 17, 2015, after the close
of our fieldwork and four months after reporting its improper payment estimates in the AFR, the
Department submitted to OMB a request to change the approved methodologies that included the
use of a dollar-based ratio estimate rather than a school-based ratio estimate. The Controller of
OMB retroactively approved the methodology changes on April 3, 2015 (April 2015 OMB-
approved methodologies).
Final Report
ED-OIG/A03P0003                                                                      Page 7 of 24



                                    AUDIT RESULTS


 We found that the Department did not comply with IPERA because it reported an improper
 payment rate for the Direct Loan program that did not meet the FY 2014 reduction target. The
 Department met the remaining five compliance requirements. Under IPERA, if the Department
 does not meet one or more of the six compliance requirements, then it is not compliant with
 IPERA. We found that the Department’s improper payment estimates, estimation
 methodologies, and reporting in its AFR for both the Pell and Direct Loan programs were
 inaccurate and incomplete. The Department’s improper payment estimates for the Pell and
 Direct Loan programs deviated from the September 2014 OMB-approved methodologies. We
 did not identify issues with the FFEL program improper payment estimate. We were unable to
 evaluate the Department’s performance in reducing improper payments because of the
 inaccurate and incomplete improper payment estimates and estimation methodologies.
 However, the Department recaptured greater amounts of improper payments in FY 2014. For
 the Pell high-priority program, the Department adequately assessed improper payment risks and
 described an adequate level of oversight to identify and prevent improper payments.

The Department met five of the six compliance requirements of IPERA, as described in the
following.

   1. Published an Agency Financial Report
      The Department complied with the requirement to publish an AFR. Under
      Section 3(a)(3)(A) of IPERA, an agency is required to publish on its Web site its AFR
      and any accompanying materials that OMB guidance requires. The Department
      published its AFR and accompanying materials, “Improper Payments Reporting Details,”
      on November 14, 2014.

   2. Conducted a Risk Assessment
      The Department complied with the requirement to conduct a risk assessment. Under
      Section 3(a)(3)(B) of IPERA, if required, an agency must conduct a program-specific risk
      assessment of all programs and activities to determine which ones are susceptible to
      significant improper payments. IPERA and OMB guidance require that such risk
      assessments be performed at least once every 3 years. In FY 2014, the Department
      performed risk assessments for all FSA managed programs and the Department’s
      administrative payments (salary, locality pay, travel, purchase card, and transit benefits).
      In FY 2013, the Department performed risk assessments for all non-FSA grant programs
      and contract payments.

   3. Published Improper Payment Estimates
      The Department complied with the requirement to publish improper payment estimates.
      Under Section 3(a)(3)(C) of IPERA, if required, an agency must publish improper
      payment estimates for programs it identified as being susceptible to significant improper
      payments. As required, the Department published improper payment estimates for
      programs identified as susceptible to significant improper payments—the Pell, Direct
      Loan, and FFEL programs. The Department also reported an estimated improper
Final Report
ED-OIG/A03P0003                                                                                     Page 8 of 24

        payment rate for Title I, Part A of the Elementary and Secondary Education Act of 1965,
        as amended (Title I). 2

    4. Published a Report on Actions to Reduce Improper Payments
       The Department complied with the requirement to report on its actions to reduce
       improper payments in the Pell, Direct Loan, and FFEL programs. Under
       Section 3(a)(3)(D) of IPERA, an agency is required to report its actions to reduce
       improper payments for programs it deemed susceptible to significant improper payments.
       The Department also reported on its efforts to recapture improper payments from all its
       programs, including that it worked with grantees and Title IV of the Higher Education
       Act of 1965, as amended (Title IV) 3 program participants to resolve and recover amounts
       it identified in its compliance audits of program participants and program reviews. We
       found that the Department succeeded in recapturing greater amounts of improper
       payments in FY 2014 than in FY 2013. On October 30, 2014, the Department submitted
       to OMB its FY 2014 Report on IPERA Payment Recapture Audits.

    5. Reported Improper Payment Rate of Less Than 10 Percent
       The Department complied with the requirement to report improper payment rates of less
       than 10 percent. Under Section 3(a)(3)(F) of IPERA, an agency is required to report
       estimated improper payment rates of less than 10 percent for each program identified as
       being susceptible to significant improper payments for which an improper payment
       estimate is published. OMB guidance further specifies that these rates should be gross
       improper payment rates comprising both overpayments and underpayments. The
       Department reported estimated improper payment rates of 2.16 percent for the
       Pell program, 1.50 percent for the Direct Loan program, and 0.00 percent for the
       FFEL program. These estimated improper payment rates were significantly below the
       10-percent threshold.

FINDING NO. 1 – The Department Failed to Comply with IPERA
                Because it Did Not Meet the Reduction Target
                for the Direct Loan Program
The Department did not comply with IPERA because it reported an improper payment rate for
the Direct Loan program that did not meet the FY 2014 reduction target. For the other two
Department programs susceptible to significant improper payments (Pell and FFEL), the
Department reported that it met the FY 2014 reduction target. However, we could not conclude
that it met the reduction target for the Pell program because of problems with the Department’s
methodology and numerous data errors, as detailed in Finding 2.

Under Section 3(a)(3)(E) of IPERA, an agency is required to report, and meet, improper payment
reduction targets when a program was identified as susceptible to significant improper payments.
2
  The Department was not required to report on the Title I program under IPERA, because it was not identified as a
program susceptible to significant improper payments. Because the Department did not identify the Title I program
as susceptible to significant improper payments, we did not perform additional work related to the accuracy and
completeness of the Department’s estimation methodology for this program.
3
  Title IV programs include the Pell, Direct Loan, FFEL, and other student financial assistance programs that are
administered by FSA.
Final Report
ED-OIG/A03P0003                                                                                   Page 9 of 24

To meet a reduction target, the improper payment rate for a program in the current year must fall
within plus or minus 0.1 percentage points of the reduction target set in the previous year’s AFR.
In its AFR, the Department reported FY 2014 reduction targets and improper payments as
detailed in Table 1.

Table 1. FY 2014 Reduction Targets and Reported Improper Payment Rates
                                             FY 2014 Reported
                      FY 2014 Reduction                           Reduction Target
       Program                               Improper Payment
                            Target                                Reported as Met
                                                   Rate
Direct Loan Program          1.03%                1.50%                 No
Pell Program                 2.26%                2.16%                 Yes
FFEL Program1                0.00%                0.00%                 Yes
1
    The Department reported for FY 2014 an improper payment rate of 0.001 percent for the FFEL program.

In its FY 2013 AFR, the Department reported for the Direct Loan program a reduction target of
1.03 percent for FY 2014. However, in its FY 2014 AFR, the Department reported an improper
payment rate of 1.50 percent. The Department also reported that the amounts of estimated
improper payments for the Direct Loan program increased from $1.056 billion in FY 2013 to
$1.532 billion in FY 2014. The Department reported that the root causes of the improper
payments included documentation and administrative errors and verification errors. These errors
include incorrect awards based on EFC, 4 incorrect processing of student data, student account
data changes not applied or processed correctly, student ineligibility for a Direct Loan, and
incorrectly determining the amount of Title IV funds that a student earned when a student
withdraws. Errors associated with Direct Loan consolidations include incorrect processing of a
loan verification certificate, the processing of duplicate loan verification certificates, a loan not
intended for consolidation being processed, and incorrect information submitted on the loan
verification certificate and processed.

Under Section 3(c)(1) of IPERA and OMB Circular A-123, if the Inspector General determines
that an agency is not in compliance with IPERA for one fiscal year, the agency must submit a
remediation plan to Congress and OMB within 90 days of the Inspector General’s determination
describing the actions that the agency will take to come into compliance. The remediation plan
must also include measureable milestones, designate a senior official accountable for bringing
the agency into compliance, and establish an accountability mechanism, such as a performance
agreement.

Recommendations

We recommend that the Chief Financial Officer (CFO) for the Department, in conjunction with
the Chief Operating Officer (COO) for FSA, require the CFO for FSA to―
       1.1 Analyze the program review reports that identified improper payments for root causes
           and evaluate FSA’s existing corrective actions to determine whether additional
           corrective actions can be implemented, intensified, or expanded to reduce or prevent
           improper payments in any program that fails to meet its reduction target.


4
 The EFC errors include (1) lack of supporting EFC documentation, (2) incomplete or inaccurate EFC information,
and (3) inaccurate calculation of the EFC amount.
Final Report
ED-OIG/A03P0003                                                                    Page 10 of 24

       1.2 Prepare and submit the required remediation plan to Congress and OMB within
           90 days of the issuance of this report.

Department Comments

The Department concurred with the finding and recommendations.

FINDING NO. 2 – The Department Needs to Improve the Accuracy,
                Completeness, and Quality of its Improper
                Payment Estimates and Estimation
                Methodologies
We found that the Department’s improper payment estimates and estimation methodologies in its
FY 2014 AFR for both the Pell and Direct Loan programs were inaccurate, incomplete, and
unreliable. Therefore, the Department could not accurately evaluate its performance in reducing
improper payments for the Pell and Direct Loan programs. Furthermore, we could not conclude
that the Department actually met its reduction target for the Pell program. We did not identify
issues with the FFEL program’s improper payment estimate.

Under IPERA and OMB Circular A-123, Appendix C, agencies are required to prepare a
statistically valid estimate of improper payments or an estimate that is otherwise appropriate
using an alternative methodology that OMB approves. For FY 2014, rather than preparing a
statistically valid estimate, the Department proposed preparing its improper payment estimates
for the Pell, Direct Loan, and FFEL programs using alternative methodologies. The Controller
of OMB approved the alternative methodologies on September 17, 2014. Additionally, under
OMB Circular A-123, Appendix C, agencies are responsible for maintaining documentation for
the alternative sampling and estimation approaches used.

Improper Payment Estimate Methodologies for Pell and Direct Loan Programs
The Department based the improper payment estimates for the Pell and Direct Loan programs, in
part, on program reviews that FSA conducted. The improper payment estimate for the Pell
program was based on the results of 98 program reviews of schools that FSA Program
Compliance—School Eligibility Service Group conducted. The group conducted these program
reviews during FY 2013 and based the testing on payments made for the 2011-2012 award year.
This methodology differed from the methodology FSA used in prior fiscal years, which was
based on the FAFSA/IRS Data Statistical Study. That study used tax return and FAFSA data for
a sample of students and parents to calculate an improper payment rate based on recalculated
Pell awards where income figures reported on the tax return did not match those reported on the
FAFSA.

The improper payment estimate for the Direct Loan program was based on three components.
The first component consisted of the results of 96 program reviews of schools that the School
Eligibility Service Group conducted. The second component consisted of testing a sample of
120 Direct Loan consolidations (from a universe of 553,663) to determine which were
Final Report
ED-OIG/A03P0003                                                                                   Page 11 of 24

considered to be improper payments. 5 The third component consisted of testing a sample of
120 Direct Loan refund payments (from a universe of 288,030) to determine which were
considered to be improper payments. The samples for the second and third component were
drawn from payments made from July 2013 through June 2014. The Department then combined
the estimated improper payments for all three components to estimate an overall improper
payment rate for the Direct Loan program.

We have raised concerns about the Department using the results of program reviews to estimate
improper payments in prior audit reports. 6 Previously, we found that estimation methodologies
based on program reviews were flawed because they excluded other sources of improper
payments, such as the results of the FAFSA/IRS Data Statistical Study, which focused on
inaccurate self-reported income on the FAFSA. The Office of Inspector General (OIG) had
recommended that the methodologies consider this source of improper payments. Also,
improper payment estimates based on the results of program reviews are susceptible to changes
in the composition of schools selected for a program review. 7 For example, from one year to the
next; FSA may change the selection of schools for program reviews focusing on compliance
areas that may identify improper payments or would not result in improper payments. 8 While
the Department’s current methodology is approved by OMB, our concern that improper payment
estimates may be understated by using program reviews as the sole source of information to
estimate improper payments continues to exist. For example, program reviews do not identify
improper payments associated with recipients who do not use the IRS Data Retrieval Tool, who
provide inaccurate self-reported income on the FAFSA, and who are not selected for income
verification. 9

In prior fiscal years, when the Department based its improper payment estimates solely on the
FAFSA/IRS Data Statistical Study, the Department identified the inaccuracy of self-reported
income on the FAFSA as the most significant root cause of Pell program improper payments. In
the FY 2014 AFR, the Department did not specify the most significant root cause of Pell
program improper payments. However, in the Department’s FY 2014 analysis of the causes of
Pell program improper payments, the two leading causes for improper payments were identified
as schools failing to complete verification of information reported on a student’s FAFSA (about
35 percent of improper payments) and Pell funds disbursed to students attending an ineligible
school (about 33 percent of improper payments). Neither of these causes are directly associated
with students who submit inaccurate self-reported income on the FAFSA. Furthermore, the
Department did not identify inaccurate self-reported income on the FAFSA as a unique root
cause of improper payments in the FY 2014 AFR.

5
  The Direct Loan consolidations include both overpayments and underpayments. The sampled payments were
tested to determine which were actual improper payments.
6
  “U.S. Department of Education’s Compliance with Executive Order 13520, ‘Reducing Improper Payments’ for
Fiscal Year 2012 and 2013” (A03N0004) and “U.S. Department of Education's Compliance with the Improper
Payments Elimination and Recovery Act of 2010 for Fiscal Year 2012” (A03N0001).
7
  FSA identifies schools for program reviews based on risk-based compliance initiatives and management mandates,
referrals or complaints, comprehensive compliance reviews, self-reported violations, and Compliance Assurance
Reviews of schools assessed as low-risk. These five areas can trigger the need for a program review at a school.
8
  For example, program reviews on compliance with performance data reporting requirements would not identify
improper payments.
9
  The IRS Data Retrieval Tool enables financial aid applicants to transfer certain income and tax information from
an IRS Web site directly to their online FAFSA.
Final Report
ED-OIG/A03P0003                                                                    Page 12 of 24


Improper Payment Estimate Methodologies for Pell and Direct Loan Programs Deviated
From Methodologies That OMB Approved
The Department’s improper payment estimates for the Pell and Direct Loan programs were not
accurate and deviated from the September 2014 OMB-approved methodologies in three ways.
One, the Department used a dollar-based ratio estimate instead of the approved school-count-
based ratio estimate. Two, it used fewer program reviews in its estimates than it proposed.
Three, the Department did not classify program reviews into the approved categories.

Use of Dollar-Based Ratio Estimate Rather Than School-Count-Based Ratio Estimate
Both the September 2014 OMB-approved methodologies (school-count-based ratio estimate) and
the executed methodologies (dollar-based ratio estimate) used two steps to estimate total
improper payment dollars. The second step is where the two methods deviated. For both
methods, the first step was to calculate a ratio estimated total for each school with a program
review using the school’s ratio of improper payments to total disbursements for the sampled
students, multiplied by the total disbursements at the school. Under the September 2014 OMB-
approved methodologies, the improper payment estimates were to be calculated, in part, on a
school-count-based ratio estimate. The school-count-based ratio estimate weighted the program
review schools’ estimated total improper payments by the count of schools with program reviews
to total schools. However, in the calculations, the Department actually used a dollar-based ratio
estimate for the Pell and Direct Loan programs. The dollar-based ratio estimate weighted the
program review schools’ estimated total improper payments by the dollars disbursed at the
schools with program reviews to total dollars disbursed for all schools. As a result of using a
different ratio estimator, the improper payment rates are significantly different (see Table 2).

We concluded that the Department changed its estimation methodologies from the
September 2014 OMB-approved methodologies after learning that the estimates using these
OMB-approved methodologies were higher than the previous year’s estimates. FSA’s
Supervisor, Systems Accountant stated that the Department’s contractor initially calculated
improper payment estimates using the September 2014 OMB-approved methodologies, but the
Department decided to change the methodologies after learning that the resulting initial improper
payment estimates were not reasonable and did not meet expectations given that similar data was
used to calculate the FY 2013 and FY 2014 rates. We interpreted this to mean the Department
expected that using similar data from one year to the next should yield similar results. The
Department consulted with its contractor and decided to change the methodologies by using the
dollar-based ratio estimate, instead of the school-count-based ratio estimate, to calculate the
improper payment estimates. Using the dollar-based methodologies, the Department’s reported
improper payment estimates for FY 2014 were closer to the improper payment estimates for
FY 2013. For example, the Pell program reported improper payment estimates were 2.16
percent for FY 2014 and 2.26 percent for FY 2013. The Direct Loan program reported improper
payment estimates were 1.50 percent for FY 2014 and 1.03 percent for FY 2013. On
March 17, 2015, after the close of our fieldwork, and four months after reporting its improper
payment estimates in the AFR, the Department submitted to OMB a request to change the
approved methodologies that included the use of a dollar-based ratio estimator. The Controller
of OMB approved the changes on April 3, 2015.
Final Report
ED-OIG/A03P0003                                                                                      Page 13 of 24

As shown in Table 2, improper payment rates calculated using the September 2014 OMB-
approved methodologies were significantly higher than those calculated using the Department’s
revised methodologies.

Table 2. Reported and Recalculated FY 2014 Improper Payment Rates (Dollars in
Millions)
                                    Improper                            Improper
                  Improper                      Improper Payment
                                     Payment                        Payment Dollars
                Payment Rate                         Rate Using
                                  Dollars Using                     Using September
                    Using                         September 2014
                                  Department’s                         2014 OMB-
 Program        Department’s                      OMB-Approved
                                     Revised                            Approved
                   Revised                         Methodology,
                                  Methodology,                        Methodology,
                Methodology,                       School-Count-
                              1   Dollar-Based               2        School-Count-
                Dollar-Based                           Based
                                                                          Based
    Pell         2.16 percent          $682         4.28 percent          $1,351
Direct Loan      1.50 percent         $1,532        2.64 percent          $2,697
1
  The Department’s revised methodology, dollar-based data come from the Department’s FY 2014 AFR.
2
  The September 2014 OMB-approved methodology, school-count-based data come from the OIG recalculation
using the approved weighting. These rates and dollar amounts do not account for other data errors reported in this
finding. Due to the complexity of the other data errors noted, we did not perform recalculations to account for all of
these errors.

Use of Fewer Program Reviews Than Proposed
The Department’s methodologies for calculating the FY 2014 improper payment rates deviated
in another way from the methodologies OMB approved. In the OMB-approved methodologies
for the Pell and Direct Loan programs, the Department proposed that it would include all final
program review results (such as findings with improper payments) available from the FY 2013
program review schedule of 311 program reviews in the calculation of the FY 2014 improper
payment rates for the Pell and Direct Loan programs. However, when the Department calculated
the improper payment rate, about 69 percent of the 311 program reviews of schools specified in
the proposal submitted to OMB had not been started, had not reached at least the draft program
review report stage, were not applicable to the Pell or Direct Loan programs, or would not
identify improper payments (for example, campus crime reviews). 10 As a result, the actual
number of final program review reports available to perform the calculation was significantly
smaller than expected for both the Pell and Direct Loan programs (79 each). The Department
was concerned about the small number of final program review reports, so it received approval
from OMB to increase the number by including draft program review reports in its calculations.
Thus, the Department used 98 Pell and 96 Direct Loan program review reports, which were
either draft or final program review reports at the time of the calculation, to estimate the
improper payment rates.



10
  In its methodology submitted to OMB, the Department anticipated using 311 program reviews for the Pell
program and 311 program reviews for the Direct Loan program. The Department only used 98 Pell program reviews
and 96 Direct Loan program reviews in its calculation of the improper payment estimates. Therefore, for the Pell
program, 213 program reviews (311 minus 98), or about 68 percent were not included in the improper payment rate
calculation. For the Direct Loan program, 215 program reviews (311 minus 96), or about 69 percent were not
included.
Final Report
ED-OIG/A03P0003                                                                    Page 14 of 24

The OIG raised this issue in its audit of the Department’s compliance with IPERA for FY 2013.
In that audit, we found that the Department did not include results of 69 percent of anticipated
program reviews in its calculation of the Direct Loan program’s improper payment rate for
FY 2013 because the reports had not yet been issued or the program reviews did not test for
improper payments. In response to the finding and our recommendation, the Department stated
that it had modified its FY 2014 estimation methodologies to expand the pool of available
program reviews for use in its estimate. However, the Department’s FY 2014 methodology still
excluded about 69 percent of program reviews when calculating the FY 2014 estimate.

Use of Fewer Strata Than Proposed
The Department varied its approach from the September 2014 OMB-approved methodology by
not classifying program reviews into the approved categories (strata). The September 2014
OMB-approved methodology listed three strata, based on the reason a program review is
initiated, as (1) compliance initiatives, (2) Compliance Assurance Reviews, and
(3) Comprehensive Compliance Review, referrals, complaints, and self-reported violations. The
Department used two strata (compliance initiatives and Compliance Assurance Reviews) rather
than three strata in its calculations. The Department’s contractor stated that it used only two
strata for the improper payment estimates because none of the program reviews used to calculate
the improper payment estimate fell into the third stratum. We found that 40 Pell program
reviews and 37 Direct Loan program reviews used to calculate the improper payment estimates
should have been grouped into the third stratum. By not including the third stratum, the
estimates were not calculated using the September 2014 OMB-approved methodology.

Improper Payment Estimate Methodology for FFEL Program
We did not identify issues with the FFEL program’s improper payment estimate. The improper
payment estimate for the FFEL program was based on the results of program reviews of
9 lender/servicers (from a universe of 24) and 14 guaranty agencies (from a universe of 34)
performed by FSA Program Compliance—Financial Institution Oversight Service. These
program reviews focused on special allowance payments and interest payments to lenders and
reinsurance claim payments to guaranty agencies under the FFEL program. For the FFEL
program, the Department used an entity-count-based ratio estimate rather than a dollar-based
ratio estimate. The entities were lenders/servicers and guaranty agencies. In the proposal it
submitted to OMB, the Department proposed using 22 program reviews of lenders/servicers and
guaranty agencies in its improper payment estimate, and it used 23 program reviews in its
calculations.

Improper Payment Estimate Methodologies for Pell and Direct Loan Were Inaccurate
The Department’s estimates had numerous errors and inaccuracies, as described in the following
sections. 11

Exclusion of Improper Payments Identified in a Program Review
The Department did not properly calculate its estimates because it incorrectly excluded improper
payments identified in one program review. The program review of one school found that the
school disbursed Pell and Direct Loan funds to students enrolled in an ineligible program. Funds
disbursed to students in an ineligible program should be classified as improper payments.
However, the Department did not include the Pell funds ($30,074) and unsubsidized Direct Loan

11
     The OIG provided the Department with detailed information about each error.
Final Report
ED-OIG/A03P0003                                                                                      Page 15 of 24

funds ($3,010) disbursed to these students in the improper payment calculations for the Pell and
Direct Loan programs. As a result, the improper payment rates for the Pell and Direct Loan
programs are incorrect and understated. However, the Department correctly included subsidized
Direct Loan funds ($8,462) disbursed to these students in the improper payment calculation for
the Direct Loan program.

Data Transcription Errors and Data Integrity Errors
The Department’s supporting documentation for its reported improper payment estimates
contained both data transcription errors and data integrity errors. The Department created
extrapolation spreadsheets from the program review results and used the spreadsheets to
calculate the improper payment estimates. However, the extrapolation spreadsheets contained
numerous errors, including the following:
        • incorrect student counts for one school included in the calculation of the Direct Loan
            program’s improper payment estimate ($46.3 million was reported as disbursed to
            78 students, for an average Direct Loan amount of more than $593,000, which was
            more than 28 times greater than the maximum loan limit for that award year); 12
        • incorrect disbursement amounts for students sampled at another school included in
            the calculation of the Direct Loan program’s improper payment estimate; 13
        • incorrect total disbursement amounts for three schools included in the calculation of
            the Pell program’s improper payment estimate; 14
        • incorrect stratum assignments for two schools in the two strata used in the estimate; 15
        • duplicate entries for three schools in the calculation of the Direct Loan program’s
            improper payment estimate; and
        • incorrect cell references for one school included in the calculation of the Pell
            program’s improper payment estimate. 16

These errors indicate that the data, based on the results of program reviews, used to estimate the
improper payment rates for the Direct Loan and Pell programs are not reliable. Therefore, the
reported improper payment estimates are not accurate.

In addition, three key calculations in the Pell and Direct Loan extrapolation spreadsheets
contained statistical formulas without any citations to statistical references or disclosed formulas.
Specifically, the calculations had neither a formula description nor a citation for the (1) estimator
for total improper payment dollars across all schools, (2) variance of estimated total improper
payments at each program review school, and (3) variance for estimated total improper payments


12
   For the 2011–2012 award year annual loan limits ranged from $5,500 to $20,500.
13
   At one school, a disbursement amount was incorrectly listed as $25,774, when it should have been $31,684.
14
   For two schools, source documents contained differing disbursement amounts for Pell. For the first school, the
documents stated both $12,366,866 and $22,228,770, and for the second school, the documents stated both $721,170
and $21,400,932. At the third school, improper payments for multiple award years ($86,715) were included in the
Pell estimate, when only one award year ($39,352) should have been included.
15
   One school was listed in the incorrect stratum for the calculation of the Pell program estimate, and another school
was listed in an incorrect stratum for both the Pell and Direct Loan program estimates.
16
   For one school, the extrapolation spreadsheet used to calculate the improper payment estimate had incorrect
formulas with errors that precluded the spreadsheet from calculating the school’s ratio of improper payments to
disbursements for sampled students, the school’s estimated total improper payments, and the variance of the
school’s estimated total improper payments.
Final Report
ED-OIG/A03P0003                                                                            Page 16 of 24

across all schools. In these spreadsheets, the Department cites a statistical textbook reference 17
but does not include a reference to a page or formula number for those three calculations. When
we inquired about the specific formula references from the cited text for those calculations, the
Department’s contractor indicated that some of the statistical formulas were from a different
text. 18 However, we found that the calculation the Department used was not the same as the
formula in the cited reference. The Department’s documentation should include appropriate
citations and disclose formula for calculations performed so that the approach is transparent and
can be validated.

Errors and Incomplete Information in the Department’s Formulas
The Department’s improper payment estimation methodologies for the Pell and Direct Loan
programs were both incorrect and incomplete. The Department’s estimation methodologies it
submitted to OMB for approval contained significant formula errors and omissions. Specifically,
we found the following.
   • The variance formula used in the Department’s calculations to estimate the improper
       payment rates for the Pell and Direct Loan programs understated the confidence limits
       and gave a false sense of precision. The formula omitted a square (for example, X2) in
       the second stage formula (for students selected), and completely omitted the first stage
       variance formula (for schools selected) along with a description of the first stage formula.
       For the Direct Loan program, the confidence limits for the reported estimate ranged
       between 2.02 percent and 2.30 percent. For the Pell program, the confidence limits for
       the reported estimate ranged between 1.32 percent and 1.67 percent.
   • The Direct Loan program’s estimation methodology initially submitted to OMB
       contained no probability-proportion to size formulas for the loan consolidation and
       refund activities; therefore, we were unable to assess that portion of the methodology.
   • The Direct Loan program’s estimation methodology submitted to OMB contained no
       description of the weighting for the three sources of improper payment information
       (program reviews, loan consolidation activity, and loan refund activity) used to calculate
       the improper payment estimate.
   • The estimator formula proposed in the Department’s calculations to estimate the
       improper payment rates for the Pell and Direct Loan programs erroneously contained a
       multiplication sign where the correct formula should have contained a division sign. The
       Department used the correct sign in its actual calculations.

As a result, the Department reported unreliable improper payment estimates and did not
accurately and completely measure improper payments for the Pell and Direct Loan programs.

Recommendations

We recommend that the CFO for the Department, in conjunction with the COO for FSA, require
the CFO for FSA to―
    2.1 Recalculate the FY 2014 improper payment estimates for the Direct Loan and Pell
        programs in accordance with OMB-approved methodologies and correcting for all the
        data, calculation, and estimation methodology errors.

17
 William G. Cochran. Sampling Techniques (New York: John Wiley & Sons Inc., 1977, third edition).
18
 Carl-Erik Sarndal, Bengt Swensson, and Jan Wretman. Model Assisted Survey Sampling (New York: Springer-
Verlag, 1992, corrected fourth printing 1997).
Final Report
ED-OIG/A03P0003                                                                       Page 17 of 24

       2.2 Publish the FY 2014 recalculated improper payment rates, notify OMB and Congress of
           any changes, and explain the basis for the revisions in the FY 2015 AFR.
       2.3 Prepare and submit the required remediation plans to Congress and OMB if the
           FY 2014 improper payment estimate for the Pell program recalculated under
           Recommendation 2.1 does not meet the improper payment reduction target.
       2.4 Use the results of an adequate number and percentage of program reviews in the
           calculation of the FY 2015 improper payment estimates.
       2.5 Include all improper payments in the calculation of the improper payment estimates.
       2.6 Maintain adequate documentation of the sampling and estimation approaches used,
           including appropriate citations to statistical formulas.
       2.7 Revise the estimation methodologies to include improper payments that are not identified
           in program reviews, such as those improper payments associated with recipients who do
           not use the IRS Data Retrieval Tool, who provide inaccurate self-reported income on the
           FAFSA, and who are not selected for income verification.

Department Comments

The Department partially concurred with the finding and concurred with the initial
recommendations. The Department did not concur with recommendation 2.7. As discussed
below, the Department disagreed with statements and conclusions contained in the report.

1. Change in Estimation Methodologies for Pell and Direct Loan Programs

Department Comments

The Department stated that the OIG’s conclusion that “…the Department changed its
estimation methodologies from the September 2014 OMB-approved methodologies after
learning that the estimates using these OMB-approved methodologies were higher than
the previous year’s estimates…” is an inaccurate representation of its actions.

The Department stated that it changed the methodology not because the estimate was higher but
because of inaccuracies in the estimate which made the estimate an unreliable method. The
Department stated that the initial improper payment rate calculation results (using school-count
ratio estimates) were not reasonable, did not meet expectations, and were potentially unreliable.
The Department stated that to correct for bias in its initial calculations, it used dollar-weighted
ratio estimates to provide a less biased weighting to extrapolate the improper payment amount.
The Department changed the methodologies and did not obtain OMB approval of these changes
until April 2015. The Department recalculated the improper payment rates based on the
April 2015 OMB-approved methodologies and made the other corrections noted in its response
to our draft report. The Department’s revised improper payment rates for FY 2014 are
1.46 percent for the Direct Loan Program and 2.21 percent for the Pell Program. 19

OIG Response

We interpreted the Department’s response to mean the Department expected that using similar
data from FY 2013 to FY 2014 should yield similar results. Initial calculations did not. We

19
     We did not perform any work related to the Department’s revised calculations.
Final Report
ED-OIG/A03P0003                                                                      Page 18 of 24

based our conclusion on the fact that the estimates derived from the September 2014 OMB-
approved methodologies were significantly higher than the estimates calculated and reported by
the Department for FY 2013 and FY 2014. We also based our conclusion on the fact that the
Department decided to change the methodologies after it knew of the initial improper payment
rate calculation results.

Furthermore, neither the Department nor its contractor provided documentation to support their
basis for the change in estimation methodologies for the Pell and Direct Loan programs. The
Department’s contractor stated that the FY 2014 methodologies were revised to use a dollar-
weighted ratio estimate to correct for bias in the initial estimates. The contractor stated that it
had performed a correlation analysis (to confirm a relationship between a school’s total
disbursements and the school’s estimated improper payment amount), which could demonstrate
the suitability of a dollar-weighted ratio estimate. However, the contractor stated that it had not
maintained documentation of the analysis. Additionally, the Department did not disclose this
change in methodology to OMB prior to the issuance of the AFR. On November 6, 2014, 8 days
before the issuance of the AFR, OMB asked if the Department had changed the overall approach
in the alternative methodology. The Department responded that, except for including draft
program review reports in its calculations, “[t]he overall approach, for the alternative
methodology has not changed . . .” However, the Department had revised the methodologies to
use a dollar-based weighted ratio estimate in its calculations. After the end of our fieldwork
(February 26, 2015), on March 17, 2015, the Department requested approval from OMB to
revise the methodology and the Controller of OMB approved the revised methodology on
April 3, 2015.

Although the Department provided recalculated FY 2014 improper payment estimates for the
Pell and Direct Loan programs in its response, FSA did not provide supporting documentation
and we did not perform any work related to these revised calculations.

2. Estimation Methodologies Based on Program Review Results

Department Comment

The Department disagreed with the following OIG conclusions from the draft audit results. “The
FY 2014 improper payment estimates for the Pell and Direct Loan programs were based on the
results of program reviews of schools performed by FSA. Program reviews primarily focus on
overpayments that need to be returned to the Department rather than underpayments.
Furthermore, program reviews are unlikely to identify inaccurate self-reported income on the
FAFSA as a significant cause of improper payments.”

The Department stated that program reviews test for payments made for an incorrect amount,
including both overpayments and underpayments. In addition, program reviews identify
improper payments associated with various root causes, and include those caused by inaccurate
self-reported income, which was the sole focus of the prior estimation methodology for the Pell
program. The Department stated that 14.25 percent of improper payments for the Pell program
and 8.61 percent of improper payments for the Direct Loan program were associated with the
root cause that includes inaccurate self-reported income.
Final Report
ED-OIG/A03P0003                                                                      Page 19 of 24

OIG Response

We changed the statement, on page 5 of the final report, to note that program reviews primarily
identify overpayments. In the AFR, the Department reported that, for FY 2014, the Pell
program’s 2.16 percent improper payment rate consisted of overpayments of 2.11 percent and
underpayments of 0.05 percent. The Direct Loan program’s 1.50 percent improper payment rate
consisted of overpayments of 1.46 percent and underpayments of 0.04 percent.

Program reviews cannot identify inaccurate self-reported income as a significant cause of
improper payments. The data cited in the Department’s comment supports this statement. The
Department reported an improper payment rate of 2.16 percent for the Pell program. Of this
percentage, the Department attributed 14.25 percent of improper payments, or an improper
payment rate of 0.31 percent (2.16 percent times 14.25 percent), to root causes that include
inaccurate self-reported income on the FAFSA. In contrast, in its comments, the Department
stated that the Pell program’s improper payment rate for FY 2014 would be 1.94 percent if
calculated under the prior methodology used to calculate the FY 2013 estimate. 20 The prior
methodology was based on the FAFSA/IRS Data Statistical Study, which focused solely on
inaccurate self-reported income on the FAFSA. Comparing these results (0.31 percent using the
current methodology versus 1.94 percent using the prior methodology) supports our statement
that program reviews are unlikely to identify inaccurate self-reported income as a significant
cause of improper payments.

The Department’s current methodology, based on the result of program reviews, is approved by
OMB. However, our concern that improper payment estimates may be understated by using
program reviews as the sole source of information to estimate improper payments continues to
exist. As a result, we included a recommendation for the Department to revise the estimation
methodologies to include improper payments that are not identified in program reviews, such as
those improper payments associated with recipients who do not use the IRS Data Retrieval Tool,
who provide inaccurate self-reported income on the FAFSA, and who are not selected for
income verification.

3. Program Reviews Grouped in the Third Stratum

Department Comments

The draft audit results stated that 40 Pell program reviews and 37 Direct Loan program reviews
used for the improper payment estimates should have been grouped into a third stratum. The
Department did not specifically disagree with the statement, but it explained the stratification of
these Pell and Direct Loan program reviews. The Department stated that the numbers cited were
from a report that was not available at the time the stratification was performed, as it stratified
the population based on the limited initial program review information available in FY 2014.
Going forward, the Department will confirm that the reason for each school’s selection for a
program review is documented and available so the population of program reviews can be
properly stratified.



20
     The Department did not provide support for this calculation.
Final Report
ED-OIG/A03P0003                                                                     Page 20 of 24

OIG Response

Under the OMB-approved methodology, program reviews were to be classified into one of three
strata based on the reason a school was selected for a program review. The Department should
have identified the correct strata (the reason a school was selected for a program review) before
it initiated the program review and before it calculated an improper payment estimate based on
the results of the program review report. The Department should ensure the correct strata are
identified in the future.

4. Recommendation to Revise the Estimation Methodologies

Department Comments

Based on discussions with the Department, it agreed that program reviews risk not identifying
improper payments to recipients who do not use the IRS Data Retrieval Tool, who provide
inaccurate self-reported income on the FAFSA, and who are not selected for income verification.
However, the Department did not concur with our recommendation 2.7 to revise the estimation
methodologies to include improper payments that are not identified in program reviews. The
Department raised concerns over the feasibility of including improper payments identified from
sources other than program reviews in its improper payment estimates and the possibility of costs
outweighing the benefits of including such payments in the estimates.

OIG Response

We acknowledge the Department’s concerns. However, to the extent possible, all known
improper payments not currently captured by program reviews should be included in the
improper payment estimates. Including such improper payments is needed to develop an
effective plan to reduce improper payments. OMB guidance states that “…agencies shall
identify the reasons their programs and activities are at risk of improper payments and put in
place a corrective action plan to reduce them.” An estimation methodology that does not
consider categories of known improper payments does not provide the Department with the
information necessary to formulate plans to address the root causes of such payments.

FINDING NO. 3 – The Department Needs to Improve the
                Completeness of its Improper Payment
                Reporting
The Department’s AFR and its “FY 2014 Improper Payment Estimation Methodologies” posted
on the Department’s Web site were incomplete. The Department did not provide statistical
sampling details for the Pell program and did not report amounts of improper payments
associated with each category of root cause. OMB Circular A-136, Section II.5.8., “Improper
Payments Information Act of 2002 (as amended by IPERA) Reporting Details” establishes the
reporting requirements for each agency that has programs susceptible to significant improper
payments.

In the Department’s published methodology document, the statistical sampling details for the
Pell program estimate lacked the level of detail (for example, the number of students and schools
Final Report
ED-OIG/A03P0003                                                                                 Page 21 of 24

in the sample and in the universe) it provided for both the Direct Loan and FFEL programs.
OMB Circular A-123, Appendix C, Part I, Section A.14 requires agencies to provide a summary
of the alternate methodology that should include statistical sampling details.

The Department’s AFR and the published methodology document do not contain all of the
required root cause information (reasons for the improper payment). The Department did not
report the amounts of improper payments associated with each category of root cause. The
Department reported only the percentage of improper payments associated with each root cause.
OMB Circular A-136, Section II.5.8 states that agencies must report root cause information,
including error rate and error amount, based on root cause categories.

The Department reported error rates in the two categories of root causes (documentation and
administrative errors and verification errors), as shown in Table 3.

Table 3. Department’s Reported Percentages of Improper Payments by Root Cause
   Root Cause                                       Direct Loan
                     Pell Grants    Direct Loans                     FFEL
    Category                                       Consolidations
Documentation
and Administrative       15%            19%            100%          100%
Errors
Verification Errors      85%            81%             0%            0%
Source: Department’s FY 2014 AFR, page 128.

The Department should have reported the error amounts associated with the error rates. Based
on documentation provided by the Department, we calculated the error amounts associated with
each root cause category. The Department should have reported information similar to that in
Table 4.

Table 4. OIG-Calculated Amounts of Improper Payments by Root Cause
   Root Cause                                     Direct Loan
                     Pell Grants    Direct Loans                                               FFEL
    Category                                     Consolidations1
Documentation
and Administrative     $55,274        $78,188       $290,950                                     $0
Errors
Verification Errors   $310,733        $327,180          $0                                       $0
1
    The Direct Loan Consolidations consist of $123,122 of overpayments and $167,829 of underpayments.

The Department did not provide users of the annual report on improper payments, such as
Congress, OMB, and agency managers, with sufficient details about the sampling the
Department used to derive the Pell program’s improper payment rate, and the amounts of
improper payments associated with each category of root cause.
Final Report
ED-OIG/A03P0003                                                                  Page 22 of 24

Recommendations

We recommend that the CFO for the Department, in conjunction with the COO for FSA, require
the CFO for FSA to―
    3.1 In its annual reporting on improper payments, (1) provide sufficient details as to the
        samples used in calculating the estimated improper payment rates and (2) include the
        error amounts when reporting on the root causes of improper payments.

Department Comments

The Department concurred with the finding and recommendation.
Final Report
ED-OIG/A03P0003                                                                    Page 23 of 24



                 OBJECTIVES, SCOPE, AND METHODOLOGY


The objectives of our audit were to (1) determine whether the Department complied with IPERA;
(2) evaluate the accuracy and completeness of the Department’s improper payments reporting;
(3) evaluate the Department’s performance in reducing and recapturing improper payments; and
(4) for the Pell high-priority program, (a) evaluate the quality of the Department’s improper
payment estimate and methodology, (b) evaluate the Department’s assessment of the level of risk
associated with the program, and (c) review the oversight or financial controls described by the
Department to identify and prevent improper payments.

Our audit covered the Department’s improper payment measurement methodologies, reporting
and performance in reducing and recapturing improper payments for the Pell, Direct Loan, and
FFEL programs from October 1, 2013, through September 30, 2014.

Our audit was for the limited purpose described and would not necessarily identify all
deficiencies in internal controls. To accomplish our objectives, we gained an understanding of
internal controls applicable to the Department’s compliance efforts with IPERA and
development of its improper payment rate estimates, as detailed below. We identified
deficiencies in the Department’s system of internal control that were significant to our audit
objectives; we fully discuss these deficiencies in the Audit Results section of this report. We
reviewed applicable laws, OMB circulars, and guidance. Specifically, we performed the
following.

   1. Reviewed background information about the Department and its programs susceptible to
      improper payments (Pell, Direct Loan, and FFEL programs).

   2. Reviewed the following laws, regulations, and guidance:
         a. Improper Payments Elimination and Recovery Improvement Act of 2012;
         b. Improper Payments Elimination and Recovery Act of 2010;
         c. Improper Payments Information Act of 2002;
         d. OMB Circular A-123, Appendix C, “Requirements for Effective Estimation and
            Remediation of Improper Payments,” October 20, 2014;
         e. Executive Order 13520, “Reducing Improper Payments,” November 20, 2009;
         f. OMB Circular A-136, Section II.5.8., “Improper Payments Information Act of
            2002 (as amended by IPERA) Reporting Details,” September 18, 2014; and
         g. Guidance developed by the U.S. Social Security Administration OIG for the
            Council of Inspectors General on Integrity and Efficiency for evaluating an
            agency’s compliance with the improper payment requirements.

   3. Reviewed the Department’s AFR for FY 2014, including the “Improper Payments
      Reporting Details,” the “FY 2014 Improper Payment Estimation Methodologies,” and the
      documentation supporting the Department’s improper payment measurement
      methodologies, reporting, and performance in reducing and recapturing improper
      payments in effect at the time the AFR was issued.
Final Report
ED-OIG/A03P0003                                                                   Page 24 of 24

   4. Interviewed officials from FSA’s Finance and Program Compliance offices and FSA’s
      designated contractor (Deloitte Consulting) for its OMB Circular A-123, “Management’s
      Responsibility for Internal Control Testing.”

   5. Reviewed prior OIG audit reports relevant to our audit objectives, including:
         a. U.S. Department of Education's Compliance with the Improper Payments
            Elimination and Recovery Act of 2010 for Fiscal Year 2013 (A19O0002);
         b. U.S. Department of Education’s Compliance with Executive Order 13520,
            “Reducing Improper Payments” for Fiscal Year 2012 and 2013 (A03N0004);
         c. U.S. Department of Education's Compliance with the Improper Payments
            Elimination and Recovery Act of 2010 for Fiscal Year 2012 (A03N0001);
         d. U.S. Department of Education's Compliance with the Improper Payments
            Elimination and Recovery Act of 2010 for Fiscal Year 2011 (A03M0001), and
         e. U.S. Department of Education’s Compliance with Executive Order 13520,
            “Reducing Improper Payments” for Fiscal Year 2011 (A03M0004).

The Department provided recalculated improper payment estimates in its response dated
April 21, 2015. We did not perform any work related to these revised calculations.

Use of computer-processed data for the audit was limited to documentation provided by the
Department to support its improper payment rate estimates and progress in reducing and
recapturing improper payments. We used the data to evaluate the accuracy and completeness of
the Department’s estimation methodologies for the Pell, Direct Loan, and FFEL programs and to
verify recovery amounts. We assessed the reliability of this data by comparing reported data to
data contained in the supporting documentation and by interviewing Department officials and its
contractor knowledgeable about the data. Based on our analysis, we concluded that the
computer-processed data were sufficiently reliable for the purposes of our audit.

We conducted onsite visits at the Department’s offices, located in Washington, D.C., on
December 16, 2014, and February 26, 2015. We briefed Department officials on the results of
our audit on March 25, 2015, and again on May 14, 2015.

We conducted this audit in accordance with generally accepted government auditing standards.
Those standards require that we plan and perform the audit to obtain sufficient, appropriate
evidence to provide a reasonable basis for our finding and conclusions based on our audit
objectives. We believe that the evidence obtained provides a reasonable basis for our finding
and conclusions based on our audit objectives.
                                Enclosure 1

   Acronyms, Abbreviations, and Short Forms Used in This Report

AFR              Agency Financial Report

CFO              Chief Financial Officer

COO              Chief Operating Officer

Department       U.S. Department of Education

Direct Loan      William D. Ford Federal Direct Loan

EFC              Expected Family Contribution

FAFSA            Free Application for Federal Student Aid

FFEL             Federal Family Education Loan

FSA              Federal Student Aid

FY               Fiscal Year

IPERA            Improper Payments Elimination and Recovery Act of 2010

IPERIA           Improper Payments Elimination and Recovery Improvement Act of
                 2012

IRS              Internal Revenue Service

OIG              Office of Inspector General

OMB              Office of Management and Budget

Pell             Federal Pell Grant

Title I          Title I, Part A of the Elementary and Secondary Education Act of
                 1965, as Amended

Title IV         Title IV of the Higher Education Act of 1965, as Amended
                                          Enclosure 2




DATE:           April 21, 2015

TO:            PnHO"Gnid
                Assisum Inspector Genenl for Audit
                Office of Inspector Genenl

                Ben=d T;i.dley
                R.ei;i.oml Jnspector- Gmenl for Audit
                Office of Inspector Genenl


FROM:           Thomas P. SbDy Isl
                Delepted to Perfotm the Fanctian md Daties of the Oiief Fmmc:W Officer
                Office of the Oiief Fuwx:i<ll Officer

                Jzmes W. Rmicie / sJ
                Oiief~ Officer
                Fedlnl Student Aid

                Jolm W. Hmt, m Isl
                Oiieffinmc:W Officer
                Fedenl Student Aid


SUBJECT:        Response to OIG' s Rmew of the Oepmmmt's Improper P~ R.eportinr;
                R.equiremems

W e ;apprecute the opportunity to respond to the dnft ;mdit results of the Office of Inspector ~·s
(OIG) ~-iew of the Depmment's Improper-~ R.eportinr; R.equinmmts..

The Oepmment is committed to Jft'\"ellling the occurrence of improper pzyments ;ind detec::tinr; and
rteoveri.ng them u.fwi they do occur. Accordllir;ly, 'l\'"e ilio ~:lte the OIG' s work ideririfyinr; issues
pemininr; to the w;iy we calcuhtr, document, md nport improper payme;ats. We !="I! rec<llcubt~ the
improper P3ymenl rzf2s ~on the sum of your fu>dinr;s excludmg the chmge &1m1 the orir;UW OMB-
;zpprcr.~ methodoloi:y reuiting in ;i 0.0.Wo dea&se in the FY 2014 Direct Lom nte <lnd ;i 0.05%
incrose in the FY 2014 PHI rm. In ;i.dditiac., we completed the rt<:<Y"mrnded modifiatiom to our
iliermtii:'I! improper p;iyment melhodolo~ doc:m:nmution ;ind r~~ OMB ~-1 for the c:lwig:'l!S
OD. April 3, 2015.


As not~ in oar re<:ent A(pley Finm.cUl R.epam (AFRs), W'I! !="I! impl~ m <lltunatii."I! est:il=tiou
methodoloi:y t}gt lenngies prognm ~ reports to dei.tiop estiimtes for och of our risk~


                             Fed eraI Student
                                 830FntStNE w~ DC~
                                    wrrrr Fe<>er.11Siuoen1Aicl eo 00"
                                            1-aoo+r:D-AJO
JJ1'0P=lS-  By le\'U3&;m, proi:r.rm rn-iew reporb md ~ cbu, Ibis ~loi:y ~~mte ;a~
robust, npmded set of~ p~ rub th.m prior ~oPes. In FY 2014, 0}.IB zppl'O\-ed
this ml!thodol.ogy for ;all tbrM progrzms: P~ Dir'Kt I.om, md FFEL As noted in tbe "Cbmp in tbe
Improper P;a)ment Estiinmm Mediodolog for the Pell Prop=i" section on p;agie 2 of your dnft reu1ts,
this ~-;al J'HU!mdin ;a chml,l! for Pell inFY 2014 from the oldFAFSAIJR.SDm Sutis1ial Studydm
w;as used to alcub:te the FY 2013 estmme.. We clwigird the ~logy beau:se M! recog:ni%2d md
;a~ed the inherent liminticns of the study for this purpose (e..~ tmt ocly one p;aymmt type md
one improper payment risk :ll'f! considered). Usin,the study to alcahte the FY 2Dl4 estizmte "-ould
~e resulted in m estimate of 1.94 ¥.., continuin' the ~-oi:e--)'Nr dowmnrd trmd ~on the study.

We cli52cree with the roaclusion th;at you coald not ~almtr the Depmmem' s perl'ormmce in~
improper p;aymmts, 1-:;ause of the issues noted under Fin.din, 2, "The Dep;artmect Needs to lmprcn"!! the
A=uncy, Compl8aiess, ;md QiWity of its Improper P;ayu:iiem Estmutes ;md Estll=tion MethodoloPes"-
In c:OITKtini; for the error.; jdentifiM, we found tmt they h;ad ;a minor im;gct CJD the 0\-enll estll=tes
under the D.f!W metboclolor;y, ;as modified. Simibr to last yur, wbeft the OIG " found th;at, when
~ ~ report2d b;ased CJD simibr ml!thodologies, estizmted improper p;aymmts in risk-
Susc:t'JlllDle prop-mis gmen)ly dec:rused"1, aapt for Dir'Kt Lom you should~ obsu\-ed ;a
c:mtinmrion of this doW!lllrnli trmd frcm FY 2013 to FY 2014. For Dir'Kt Lom, the nte inaused due
to~ in the~ metbodology ;md smiple cbt:a.




D!Jl:artmmt's Recponu to Findin; l:

The ~<Utme:m c:cmc:an with Fiiidinc 1.

wI! ~the recognition in the nporl aw the Dep;artmmt met fin of the m aiteN for cemplimce
md, for the :ixth, did not ml!l!t the nqaitemeut for reduction~ for ocly CJDI! of lhz'ff risk-=iaptiole
pr~. The redaction~ ;md realc:nhted estinmes zre ~in the t;able below. As no~
the estizmte for Du-ed Lom ~ in tbe FY 2014 AFR excaded the redudicia tupt of 1.03%
reported for FY 2014 in the FY 2013 AFR.




                                                                                           No
                                    2.26%                      2-21'}'.                    Yes
                                    0.00-1.                    0.00-1.                     Yes



 "U..S. Dtpmtmmr o!Edllation' s Cmq>limce 1lrilh w Imprope-Psymmis FJimimrillll &Dd RKCJ\"tr}' Act o!2010
1

forFi.sal Year 2013", ID-OIGIAl.900002, April lS, 2014.

                            Fed
                            ,h
                                   era I Student 1J
                             crre.;: .,(" _, \ r cnri· wr ., ,. .
                                     ~,                     rtu c1 t 1) •
                              830 F·n.t St NE . WWiington. DC 20202
                                  YIV'f'ft Fi!'deralS~d ~ fPI
                                          1·800-4-FEO-AIO
For FY 2013 ~ OMB prorisionilly a.PJll"O"-ed tbel>ep;i.rtment' spoposedDirec:t LomProi:nm
improper payment methodology md tbe redndicm arr;ets reported were tbt-lined pending the
~·s refinements to the melhodology (or fuW OMB ~-;i.l As reported on page 125 o(the
FY 2013 AFR, OMB app?O\'-ed the Direct I.om methodology only Cartmt fual ymrwhile the
Oep;utment contiimed to wotk with OMB on plliinr; fuW "'PJll"O"-al for ;i. mutmlly :lr;rtt'ble ~ for
improper payments for all FSA pror;nms. The mgets reported in the FY 2013 AFR were b:asecl on the
estirmtion results or prior~ methodology m.d, by defmition, did not consider the new metbodolor;y for
FY 2014 tmt W2S not i!ppro~-ed by OMB until Septanber 17, 2014.

        Dep:utment's RMpome to Rttommmcbtion 1.1

        The Dep;utment concurs with R.eeommmdnion 1.1.

        As ;a fimction or the Departmmt' s stmdarc:l improper~ reporting process for Direct I.om
        md all risk-susaopti'ble program ~r;ndless or wbethu or not they met their reductWo mget U.'1!
        will continue to ei..Jmte the root ~for improper p~'lJlellts idemiDecl. We will e'\chme
        corredn"'I! :actio.u s Cor uc:h root ause to determine whether they mry be enlwiad or iinew
        corredii."'e ;actions mry be implemented to further reduce or Jlre''mt imprope' p;i.)'!Dmts.



        The Dep;utment concurs with R.eeommenchtian 12. The~ will prei=e md submit the
        reqWnd remedi2tion phn to Conr;ress zl:ld O?.ffi within 90 d.i)'S of the~ oC the rrport.



Deparrmmt's Response to rmding !:

The Depmmem p~y concms with this finding. The Department agrees !Im it nttds to correct the
deficiencies found md ccrrect the OMB-~-ed methodology doc:ummtrtian.

Wrthin the melhodology ~-ed by OMB, the Dep;i.ztment did not intend to perl"oon ;a sntistial
extnpolation; rather, the Departmem used m utenmn"'e methodology to estimate the improper~
nlH (or its pror;rmis due to the ml!tbod used to oboln the s;amplinr; popabtion. Upon initW akubtion
the~ rulized dw the~ was not rusomble, not meetmr; exped2tiom, mdpo!entiilly
IJDl'efu~ As shou..n in the able below, we noted 9r;nific:int fhx:tmtion in the school cotlZlt weighted
FY 2014 nte compared to prior yur publis.hed rates. As descn'becl bter in this response to finding #2, we
chm,C2Ci our methodology to use the dollu weir;htecl estimmon. We Uiled to documem this~ in
the orir;in;al FY2014 methodology document md to obbin oim a~~ Cor this~ tmfil April
2015. Milinr; this chmr;e md ;i.pp}yinr; the olhs- c<X?Kti.ans recommended by the OIG in the dnft
report, we reakubted the rates as presented below. These realculmd r*5 '\-"3liecl slir;htly from the
rates published in the AFR. The upd:;nM Direc:t Lom improper p;i.ymeut rate is now l.46%; the Direct
Lom improper payment rate publi.shed i:n the FY 2014 AFR W2S l.5o/o, md the~ Pell extnpobtion
ralf! is now 22lo/a; the Pell improperp;i.yment rate published in the FY 2014 AFR W2S 2.16%.


                            Federal
                            ·"
                                      St udenbccue1--.ia
                                o' :HJ>
                               crr1.:.~              11:i-.
                                               CCPr1R ·w c·n r:
                              830 First St N E . W.ashongtcn. DC 20202
                                  V/\Wl.federal$tuden!AKI eG rpl
                                          1-8004-FED-AIO
                        Rttakubtrd FY 201-1 r...........u PannmtR.ates
                   2013      2014 hbli.Urd Rttablated by OIG            Recakulatrd by tltr
                hblishrd         Rates         ~ thr Sept. 2014         Drpt. usinl tile Apr.
 Provam           Rates                           OMB-ApproTrd         2-015 OMB-Approvrd
                (udFYU                               Mrdaodolou            Mrthodoloirr
                 TUYet)
   Pell           2.26%          2.16 %                  4.28 %                2..21 %
Directl.oan       1.03 %         1.50 %                  2..64%                1.46%

   On page 3 of its report, the OIG imbs the following sl2tements reprding the " Ch.mge in the Improper
   Payment Estimation Mmodology for the Pell Progrmi":

           7h11 FY 2014 improptlT pq)'1ftaJ1 tlStimattJ/or tht1 Ptill and Dirt1et Loan progrimu wart1 bastld on
           tht1 rasula ofprovam rtl\it1M1S ofschool.Ipafonntlti by FS.~ Provam r11iit1wsfocwprimaril;y
           on ~'Jl!tlntl that nt1tld to b. rtllJIT1ltld to th. DilpaTTmtmt rathlfT than undRpaJ'TIUmlS.
           Funht1m1ort1. J7r017am r11iit1M·s art1 unlibl;y to iJt111rify inaCC11Taltl stlif-uporrtlli incomt1 on tlitl
           FUSA as a significant cauu ofimproptlr ~'Jl!tl1!U. -

   We diS2uee with that statement. All geDenl assessment and focused progrmi in~ tmt lliclude
   studeDt level assessment test for~ paymmts, that is, llrf pa)"IDl!Df that W2ll m;ade in m inconKt
   mx>unt (mcluding ~ and~) under sOtutocy, colltnctual, administtatii.'t!, OI'
   other le~y applicable requirements. As noted :abcr.·e and in the report these pro;ram rmews assess
   risks bey'Olld inaa:unte self-reported ~ the sole fOCllS of the old methodology.. U1.""l!npng program
   re>iews resulted in m expmsion of the root auses idemified n reported aa pa;e 126 of the FY 2014
   AFR. The root ause most associated with inaccante self-reported income under the new methodol.o gy is
   Incorrect A°"'~ds based cm Expemd Family Contribatiaa (EFQ, resulting in 14.15% of the tobl Pell
   estllmte and 8.61 % of total DU:ect Lam estimate.

   On J>all! 8, the OIG imbs the foll~ sutement:

           1Yt1 concluJtld thal thil DtlpOJ"rment changtlli in tlltimalion mt1thodo~.s.fram thil OMB-
           approvtld mtJt.hotiologitJS '1fttrT learning that tht1 tlStimattlS using th. OM/Hzpprcn'tld
           methodologitJS wart1 highilr than thil pr11iious )"tlaT '.s tlStimattu. -



    The methodology was ~not due to the estimate bmig higher, but ntber due to the in.accuncies
    within the estimate which made the ~ m unreliable method. In other WOl'ds, 1'1Imliiig the chtl
    through the methodology re>-wed deficiencies in the methodology that needed to be addres.sed. To
    estimate tbe FY 2014 improper P")'lileD.t ntes fOI' its risk ~le prop=, md similar to tbe
    estimation performed in FY 201 2, FSA used the dolhr weighted ntio (ntio estimation) in tbe two sages
    of the samplin;, ~d of the dolhr weighted ntio (ntio estim2tian) used in the fint ~ of smipling

                                  Fed           era I Student
                                  .• ' c r ri·:. : .., , nc-
                                                       J;
                                                                                    .\icJ
                                                             r c pr, n·w c-n ": t t.ot1 r 1o"f
                                    830 First St N E., W astungton. DC 2CL.""02
                                           Ytvrll Federa!Stuciefl1Aid.eo       90"
                                                     1-800-+FED-AIO
and the school count ~ ntio (mun estixmtion) used in tbe ~mp of smlplinr; to estinute
the improperpzymtllt ntts in FY 2013. The munestim2tion mes tbe~p oftbe mioantbellig
~ (111 this case the improper p;l)"IDl!ll! 21D01l1U) ~~ •-iih uch smiple i1em; this type of
estixm~ c::m be more bWed if; 2S we obsen-ed for FY 2014, tbe <n'en~ popuhtion ncorded miount is
not simibr to the ~ szmple r~ miount.. To help cClU'Kt f« the bias imrodnc-ed by the
d.i.ffermce betWffZl these two ;;n:en~ ;md gn'l!Zl tbe c:omtni:nts of a Slmllu chm ~ s=iple size
in FY 2014, FSA~ tbe mio esfumtion to provide al~ bused •'l!ighting to~ tbe smiple
improper ~t amount to popuhtion improper pzymtllt miount.

Fimlly, on~~ 9, the OIG mlies the followinr; satement:

        ..Wt1found that 40PNiproVmn rmtnt.: antl 37 Dirt1ct LoanproVmn rtl\itl'K~ Wit/for tlttl
       impropo-pt0mmt tlStimarc should Ira\,. b.n r;roup«/ into a tlrirtl stran.rm ".

These nnmben were~ on a Pos1secoodzry Eduatioa ~ Symm (PEPS) report which
included infonmtioo not n.ibble ~ the time the stntifiation wu performed. The ~t stntifi.ed
the popubtion b~ on tbe l.imUN UiitW pror;nm mini infcnmtion n'3ibble in FY 2014. Going
fonnrd, FSA will c:oufum the pror;nm rei."iew ~is documaited ;md ;n.,.jhbJe befure tbe smiple is
~so tbe papulition c::m be properly stratified~ on the ~ch.

       Dmartmrpf'' Rnpop1c tg Ruomipcpdarigp : .J·



       In~ md responding to questions from tbe OIG dminr; their fieldwmk far the FY 2014
       IPERlA Comp.l imc.e Audit, it ame to our anmtion that the 01\£6-~°'-ed ~"I! improper
       p~ methodolor;y d.ocmm!!lt iDclnded in the fommh Stt1lOn a school count -~ ntio
       (me= estiimtion) instud of the do~ weghttd ntio (ntio estllmtlon) ~ ped'ormed..
       01\ffi ~notified of tbe clwige to the ~l!!m2tii.'1! improper ~ymait methodolor;y dl!C"Dl'OPT!t on
       J\hrch 17, 2015 and prorided ;;ipprg,,;U of tbe upc:bted documem to tbe Drp~ on April 3,
        2015.

       In addition, :al.I of tbe errors identified dminr; tbe ;;indit were cor:reded and new improper ntes
       c:akubted for Direct Lom and Pell, resu.ltinr; in a 0.04% d«:ruse in the FY 2014 Direct~
       nte md a 0.05% inc'l!ase in tbe FY 2014 Pell r*.
        Depiartmmt's R.ueonn to Rttommnubtion ?.?:



       For the chmges identified as a result ofR.ec:ommeiubtiom 2.1 md 2.5, the Oepartmmt will
       publish the n=lai.bted improper pzymtnt ntes, notify 01\ffi ;md Conr;ress of the ~ md
       explain tbe re\isicms within the FY 2015 AFR.




                            Fed eraI Student
                            .f r ( f t'I:: 4 ' Ht J ) ( {P'/o fl •w c n° f' " tc,tJCt'f :1 "4
                              83£1 FrstSt 'IE Washington.                  DC~
                                     .,,.,.., FeOeQIS:uoenlAid f'd p
                                                1·800.+FIDA!D
1be Depmment CQllCtln with this rea>mmenmtion.

BMed on actions taken inJ'Mll(lllSe to Recammend.ztioo 2. 1, the results of the recalculated Pell
improper P3)'lDellt nte (221 %) Dlfft the FY 2014 ~n T~t for Pell (2269/o). Therefore,
no additional action is necessary.

Dtp;artmrnf's Rf100P$C to Rugmmtpdariqp ?..f•

1be Department COllCIIJ"S with this recommendation.

1be Department will coordinate with o~m OD the FY 2015 methodology, iDdndinr; the total
m=ber of prop-mi rei.-iews to be used.

1be FY 2014 estimation methodology attempted to increase the~ size from the sample u-1
in FY 2013 through modifying the methodo.logy penormed in FY 2013 to include draft Pror;ran
Rn-iew Reports (PRR.s), Expedited Determiuatioa Lettus (EDLs) and Final Progmn Rn-iew
Det.emriJJ>tiocs (FPRDs) for the smie ;;nnrd year (2011-U)u-1inFY 2013. 1be inimt of this
modification was to allow for more reports to be ;;n;iibble forreriewdmingthe FY 2014
improper payments usessment. In FY 2014, 98 Pell and 96 Direct Loa reports were rnieu-ed
u cam.pared to the 63 Pell md 55DirectLoareports11!\-iewed in FY 2013, which demonstntes
tmt addrlional reports wen 11!\ieu'ed in FY 2014.
Goinr; forward, the ~will coatinue to ev.duate ~ to inaea.se the pool of avaihble
PRRs, EDLs, md/or FPRDs to sample for~ of ~oping improper paymeit estimates
for Pell md Direct Lewi.

Fuztb.e., we ~ill u:pcbte the methodology doc:umeDt to chrify the mimber of 11!\Uws we mticipate
to ~'I! ai.-aihble to s;imple in FY 2015 b~ in part on our FY 2014 mdprioryurresults. The
m=ber ofrei.-iews plamied to be performed in FY 2013 u dommeJJte<i in the FY 2014
methodology docnmrnt w.is 31 l. ~, the imem was not to rttrii.·e ill 311 reports dJJe to
the timing of when reports are issued. We will nse a footnote for Pell md Direct Loa in the
updated docnmrnt similar to footnote 5 of the FY 2014 lmproper P;;rymBtt Methodology
doc:ummt ~-eel by OMB for the FFEL progr.mi::

        ~Du. to tha timing qf t&sring and compl.lllion qfrtnii•to,.z . ir u notposnola to inclua• all
        Program R8vfirKtt in tit. annual tJScimation qfimpropa-pa;rmma. •·




                     Federal Student
                     .•• crr1.:.:
                              CC"· R·w c·n
                                           Aid
                                    ~ , !\(' ,J $,          ('!   CtUCI TI) ..
                       830   Fi1'5l St
                                     NE. Washington. DC 20202
                             """"' Federa!SIUdenlAid.ed gov
                                           1·8004-FED-AIO
       The Depilrtl:Deilt CODC:m'S with this ~mznendafiOll.

       All of the aT'Ol'S ideiitified ~the 2udit ~corrected ;md mw improper nte alcul*<i fir
       Dind Loan md Pell, ~i; in 2 0.04% deause in the FY 2014 Dind Lom nte md 2 0.05%
       inau.se in tbe FY 2014 P~ rm.

       Drwrhptpf's Rgmonpc to Rt<ommmdariap : . §;

       The Depilrtl:Deilt CODC:m'S \\.jib this ~mznendafion.

       As noted in Recommeruhtion 2.1, OJMB W3S llOti1ied on~ 17, 2015 of the clwige from the
       school-count weighted ntio to dollu weidited ntio md pnn-icled ;ippnn-al of the upcbtM
       methodolo~ on April 3, 2015. Furthmnon!, goini; forward tbe Depilrtl:Deilt \\.j]J incorponh! 2l1
       cit2tiom to satistia.l foi::mu.bs nsed in the methodoloi;y, u llpproved by O}.m.


Df'p:artmmt's Rspon.se to Fmcting 3:

The DepJl'tl:Del.lt CODCm'5 wilh Finding 3. \\s'hile the ~ published root =ise infonmtico md tbe
pel'cenDi;e of emn for uch ~~. the~ did not inclnde the mlOlllll of improper~-ments
for uch a~~. The Depntmem ilio did not include the S3ZDe samp~ deails !Or the P~ ~
prou=i u it did for tbe Dind Loan md FFEL proi;nms.

        D!p:artmmt's Rspon.se t o RMommmcbtioa 3.1



                             pnn*
        The Depilrtl:Deilt will sufficient ~ llS to tbe samples nsed in c:iilcuhtinr; estinuted
        improper p;iymmt ~ md will inchided the emr llmCUZlf when reporti:nr; co the root auses of
        improper~.

Once 2i:;Wi, we ;appncUte the opportumty to re\Uw md respond to the~ II you ~-e :my questions
or need additi.on.U inli:>nmtion reprdinr; this response, pluse contxt Bill Blot ;at (202) 377-3097 or fay
Hurt at (202) 377-3453.



cc: Robert J;mney




                             FederalJ;.Student
                                        cc•t r·w c
                             .• ~ c rr r:.~ ~ · ~\t
                                                        .id
                                                   cc1.c1 · ' •
                                                              4• f' •
                               83{)   F:rst St •.tE Wasnngtor DC~
                                      .,,_, Federal.Sll>OEn:A.d ea p
                                                 1-800-4-FED-AID