oversight

Federal Student Aid's Estimation of Improper Payments in the Federal Family Education Loan (FFEL) Program.

Published by the Department of Education, Office of Inspector General on 2008-09-25.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                                     UNITED STATES DEPARTMENT OF EDUCATION
                                                          OFFICE OF INSPECTOR GENERAL

                                                                                                                              Audit Services

                                                        September 25, 2008

                                                                                                    Control Number
                                                                                                    ED-OIG/A09H0015

Lawrence A. Warder
Acting Chief Operating Officer
Federal Student Aid
U.S. Department of Education
Union Center Plaza, Room 112G1
830 First Street, NE
Washington, D.C. 20002

Dear Mr. Warder:

This Final Audit Report, entitled Federal Student Aid’s Estimation of Improper Payments in the
Federal Family Education Loan Program, presents the results of our audit. The purpose of the
audit was to assess Federal Student Aid’s (FSA) methodology for estimating improper payments in
the Federal Family Education Loan (FFEL) Program. Our review covered the methodology used
for reporting in the U.S. Department of Education’s (Department’s) Performance and
Accountability Reports (PARs) for fiscal year (FY) 2006 and FY2007.




                                                       BACKGROUND 



The Improper Payments Information Act of 2002 (IPIA) (Public Law 107-300) requires agencies
to annually review and report on improper payments in the programs and activities that they
administer. The IPIA specifies the annual three-step process that the agencies are to use to meet
the requirement. First, the agencies must review the programs and activities and identify those that
may be susceptible to significant improper payments (risk assessment). Then, for each identified
risk-susceptible program and activity, the agencies must estimate the annual amount of improper
payments. Lastly, for programs and activities with improper payment estimates exceeding a
specified threshold, the agencies are required to report on actions they are taking to reduce
improper payments. The IPIA provides definitions for the terms “payment” and “improper
payment.”




  The Department of Education's mission is to promote student achievement and preparation for global competitiveness by fostering educational
                                                    excellence and ensuring equal access.
Final Report
ED-OIG/A09H0015                                                                              Page 2 of 26

The IPIA directs the Office of Management and Budget (OMB) to provide agencies with guidance
on implementing the requirements of the Act. On August 10, 2006, OMB issued Appendix C to
OMB Circular A-123, Requirements for Effective Measurement and Remediation of Improper
Payments (Appendix C). 1 The OMB guidance requires agencies to “institute a systemic method of
reviewing all programs [and activities] and identifying those which they believe to be susceptible
to significant erroneous payments.” The OMB guidance defines “significant erroneous payments”
as annual improper payments in the program “exceeding both 2.5 percent of program payments
and $10 million.” However, OMB may determine on a case-by-case basis that a program or
activity is subject to the annual PAR reporting requirement even if it does not exceed the
2.5 percent and $10 million thresholds in OMB’s definition of “significant erroneous payments.”

The FFEL Program was identified as a program subject to the annual PAR reporting requirement
(i.e., estimation of the annual amount of improper payments). For loan guaranty programs, such as
the FFEL Program, the OMB guidance provided the following clarification on the definition of an
improper payment.

        Under a loan guarantee program, an improper payment may include disbursements
        to intermediaries, third-parties for defaults, delinquencies, interest and other
        subsidies, or other payments that are based on incomplete, inaccurate, or fraudulent
        information. They may also include duplicate disbursements, disbursements in the
        incorrect amount, or any disbursements that are not in compliance with law,
        program regulations, or agency policy.

The OMB guidance on estimating the annual amount of improper payments requires agencies to
obtain a statistically valid estimate of the annual amount of improper payments that is based on a
random sample with a specified confidence interval. The guidance states an agency may seek
OMB approval to deviate from the required steps or use an alternative method. The OMB
guidance also addresses the annual PAR reporting of the estimate and corrective action plan for
reducing improper payments.

The Department’s Office of the Chief Financial Officer (OCFO) has primary responsibility for
issuance of the PAR. FSA is responsible for providing information to the OCFO on programs it
administers. FSA’s Financial Partners Services developed the methodologies for estimating the
amount of improper payments for the FFEL Program reported in the PARs for FY2006 and
FY2007.




1
 Appendix C consolidated requirements that were previously communicated to agencies in OMB memoranda issued
on January 16, 2003, May 8, 2003, and May 21, 2003, and clarified and updated the requirements.
Final Report
ED-OIG/A09H0015                                                                                      Page 3 of 26



                                             AUDIT RESULTS 



FSA used different methodologies for estimating the improper payment rates for the FFEL
Program that were reported for FY2006 and FY2007, and plans to use another methodology for
FY2008. FSA consulted with OMB staff during the design and execution of the methodologies
and generally followed the IPIA definition and OMB guidance for loan guarantee programs when
it identified “payments” for the FFEL Program as the payments made by the Department to lenders
and guaranty agencies. Nevertheless, there were several factors that affected the reliability of the
improper payment rates reported in the PARs. To improve the reliability of the improper payment
rates for the FFEL Program, we recommend that FSA: (1) ensure that factors related to the use of
previously conducted audits and reviews are taken into account or their effects mitigated;
(2) ensure that all methods of payment are taken into account when identifying the FFEL payments
to be included in estimating methodologies; (3) ensure that future methodologies use information
on improper payments for the fiscal year for the PAR or, if not available, for the prior fiscal year;
(4) when sampling is used, consider focusing sample selection on higher-risk entities and areas to
increase the likelihood of identifying improper payments; (5) finalize and revise the estimated
improper payment rate for FY2007 in future PARs; 2 and (6) disclose in the PARs when the rate is
based on an interim calculation or there are other limitations in the reported information on
improper payments.

We also found that the program outlays for the FFEL Program reported in the Department’s PARs
for FY2006 and FY2007 and used to calculate the estimated dollars of improper payments
reflected different payment universes. In addition, FSA used an estimated program outlay amount
for the current year when information was available on the actual outlays for the year. We
recommend that FSA develop and implement a revised policy for identifying and reporting
program outlays for the FFEL Program in the PARs that provides consistent and comparable
information on outlays and dollars of improper payments reported and utilizes the most currently
available information on outlays.

In its comments on the draft report, FSA did not explicitly express concurrence with our findings
and recommendations, but did state it would implement a methodology for estimating FFEL
improper payments that meets OMB requirements. FSA also stated that it would finalize and
report the estimated improper payment rate for FY2007, disclose in future PARs when an interim
calculation is used or other limitations in reported information, and update its policy and
procedures to ensure consistency in information reported in future PARs. The comments are
summarized at the end of each finding and the full text of the comments is included as an
Attachment to this report.




2
  The estimated improper payment rate of 0.032 percent reported in the PAR for FY2007 was based on an interim
calculation. In April 2008, FSA updated its calculation of the estimated improper payment rate in order to respond to
an information request from the OIG auditors. The recalculation resulted in an estimated improper payment rate of
0.218 percent.
Final Report
ED-OIG/A09H0015                                                                                   Page 4 of 26

FINDING NO. 1 – 	Several Factors Affected the Reliability of FSA’s Estimated
                Improper Payment Rates

Several factors affected the reliability of the estimated improper payment rates for the FFEL
Program reported in the PARs for FY2006 and FY2007. The reliability of the planned
methodology for reporting in the FY2008 PAR also could be impacted by several of the factors
identified in the earlier methodologies.

Estimated Improper Payment Rate
Reported in the PAR for FY2006

FSA provided an internal document that described the methodology used to develop the estimated
improper payment rate of 2.2 percent for FY2006. The methodology extrapolated the monetary
findings from prior reviews of lenders, lender servicers, and guaranty agencies to the amount of
payments made to the lenders and guaranty agencies. To identify the monetary findings, FSA’s
Financial Partners Services reviewed reports issued in FY2005 for the following types of reviews:
single audits, program audits, OIG audits, and FSA program reviews of guaranty agencies, lenders
and lender servicers. 3

FSA compiled the identified monetary findings separately for lenders/lender servicers and
guaranty agencies by type of review, and extrapolated the identified improper payments to all
lenders and guaranty agencies. 4 FSA then added the calculated improper payments for each type
of review (totaled $185,129,426 for all reviews) and divided the total by the total payments to
guaranty agencies and lenders ($8,625,964,796) in FY2005 to derive the estimated improper
payment rate of 2.2 percent for the FFEL Program.

Several factors affect the reliability of FSA’s estimated improper payment rate for FY2006. Sole
reliance on audits and FSA program reviews of guaranty agencies, lenders, and lender servicers
excluded improper payments identified by other sources. Also, amounts extracted from reports on
previously conducted audits and program reviews may not reflect actual improper payments for a
federal fiscal year. Other factors were that the amounts used from the reports may be
unrepresentative of all lenders and guaranty agencies, duplicate amounts included in another
report, and contain amounts not applicable to the PAR reporting period. Also, the identified
amount of improper payments was incomplete (i.e., exclusion of contested monetary findings and
use of final determinations of liabilities). We also found that FSA had omitted certain payments to
guaranty agencies from the total payments used to calculate the estimated improper payment rate.


3
 Single audits are audits conducted in accordance with OMB Circular A-133, Audits of States, Local Governments,
and Non-Profit Organizations. Program audits are audits conducted in accordance with the Department’s Audit
Guide, Compliance Audits (Attestation Engagements) for Lenders and Lender Servicers Participating in the Federal
Family Education Loan Program.
4
  For example, the FSA program reviews covered guaranty agencies who guaranteed $174,034,845,527 of the
$300,142,555,788 in total loan balances. FSA identified $533,440 of monetary findings for program reviews at the
reviewed guaranty agencies. Using the following formula, FSA calculated $919,978 as the amount of improper
payments identified from program reviews at guaranty agencies:
                          $300,142,555,788
                                               X $533,440 = $919,978
                          $174,034,845,527
Final Report
ED-OIG/A09H0015                                                                      Page 5 of 26

Sole Reliance on Audits and FSA Program Reviews of
Guaranty Agencies, Lenders, and Lender Servicers
Excluded Improper Payments Identified by Other Sources

The amount of improper payments used in FSA’s methodology may be incomplete because all
sources for identifying improper payments were not utilized in the methodology. FSA did not
include monetary findings from audits and reviews of institutions of higher education, guaranty
agency reviews of lenders, or OIG investigations. Also, all improper payments self-identified by
lenders or lender servicers were not included in the methodology.

   •	 Institutions of higher education reviews. Institutions of higher education that administer
      the FFEL Program are subject to audits conducted by independent auditors and the OIG, as
      well as program reviews conducted by FSA and guaranty agencies. These audits and
      program reviews may have findings related to students’ eligibility for FFEL disbursements,
      amount of the disbursement, or amounts to be returned to the lender. For example, students
      enrolled in ineligible programs or at ineligible institutions are not eligible for loans under
      the FFEL Program. Student eligibility is significant because payments made on behalf of
      loans disbursed to ineligible students, such as interest and special allowance subsidies paid
      to lenders and claims paid to guaranty agencies, are improper payments.

   •	 Guaranty agency reviews. Guaranty agencies are required by federal regulations to
      conduct biennial program reviews of certain lenders participating in the FFEL Program.
      Federal regulations define the criteria for selecting the lenders for review, but guaranty
      agencies may consider other factors, including complaints from schools, students or
      borrowers, and evidence of potential fraud or abuse in a lender’s FFEL Program
      participation. Guaranty agencies reported that common program review findings at lenders
      included errors on lender invoices in origination fee calculations, interest, and special
      allowance reporting. These types of errors could result in improper payments.

   •	 OIG investigations. Documentation provided by OIG investigations showed that improper
      payments of almost $4 million were identified in FY2006 for all Student Financial
      Assistance programs, including but not limited to the FFEL Program. In one example, an
      OIG investigation identified $650,000 that the Department paid a guarantor for
      fraudulently consolidated federal student loans.

   •	 Lender self-reports. FSA included $370,740 in self-reported lender liabilities that was
      recorded in the Department’s Postsecondary Education Participants System (PEPS), but did
      not include adjustments made by lenders on their invoices. Adjustments of amounts
      included in prior invoices may occur when updated borrower information was subsequently
      received that impacts the calculation of interest and special allowance on the loans.
      Adjustments may also occur when the lender or lender servicer has identified errors in their
      billings. The lender invoice does not identify the reason for an adjustment.

By not including these additional sources when compiling monetary findings, FSA understated the
improper payments used to develop its estimated improper payment rate.
Final Report
ED-OIG/A09H0015                                                                        Page 6 of 26

Audit and Program Review Reports May
Not Reflect the Actual Improper Payments
For a Federal Fiscal Year

The reports from previously conducted audits and program reviews may not reflect the amount of
actual improper payments. Audits and program reviews generally cover a small sample or focus
on selected program requirements. Thus, there is a risk that the reviews may not detect conditions
that may have resulted in improper payments. For example, FSA receives about 380 program
audit reports annually. However, FSA identified only 11 program audit reports issued during
FY2005 that contained monetary amounts for inclusion in the improper payment rate.

Even when the audits and reviews have detected reportable conditions, the reports may not contain
information on the actual amounts of improper payment applicable to the period reported in the
PAR. The reports may not include a quantification that reflects the improper payment amount,
may be limited to the reviewed transactions, or be for a period other than the federal fiscal year.

   •	 Reported monetary impact may not reflect the improper payment amount. This situation
      was demonstrated by FSA’s use of monetary amounts from program audits. Our review of
      4 of the 11 program audit reports confirmed that, for one audit report, an improper payment
      had occurred and that FSA had used an accurate amount in its development of the
      estimated improper payment rate. The other three audit reports included findings
      discussing conditions that may have resulted in improper payments. However, the amounts
      from the three reports that FSA used to develop its estimated improper payment rate were
      not a quantification of an improper payment made to a lender or guaranty agency. Instead,
      the amounts represented the principal balance of a loan or a loan disbursement.

   •	 Quantification limited to reviewed transactions. Even when an audit or program review
      report includes a questioned cost (i.e., an improper payment), the amount is often limited to
      the specific erroneous transactions identified in the reviewed sample rather than an amount
      derived by projecting sample results across all transactions. Although not projecting
      sample results may have been valid (e.g., the sample was non-statistical), using only
      sample results understates the improper payment amount.

   •	 Period covered by quantification may be other than a federal fiscal year. The 12-month
      federal fiscal year is used for improper payments reporting. In contrast, the period covered
      by audits and program reviews may be an entity’s fiscal year, multiple fiscal years, or other
      period of time. Thus, monetary findings in audit and program review reports may be linked
      to a period of time other than the federal fiscal year or a period that is longer or shorter than
      the federal fiscal year. Using the reported monetary findings could consequently result in
      either overstating or understating the improper payments.
Final Report
ED-OIG/A09H0015                                                                                     Page 7 of 26



FSA also recognized in its methodology documentation that the improper payments were
incomplete because monetary findings were not always quantified by FSA in the final program
review report. The OIG audit report titled Review of Financial Partners’ Monitoring and
Oversight of Guaranty Agencies, Lenders, and Servicers (ACN A04E0009), issued
September 2006, discussed this condition and recommended that FSA develop a consistent policy
for identifying, quantifying, and reporting liabilities identified in program reviews.

Findings in OIG Audits and FSA Program Reviews at
Individual Entities May Not Be Representative of
Improper Payments at Other Entities

Findings from OIG audits and FSA program reviews may not be representative of conditions
effecting payments to all lenders and guaranty agencies. Because these audits and program
reviews are generally selected on a judgmental basis that may consider risk, extrapolating their
results across loan balances for all lenders and guaranty agencies may overstate the amount of
improper payments. For example, FSA included an improper lender payment of $688,767 from an
OIG audit report 5 and extrapolated the amount across loan balances for all lenders. Thus, the one
OIG audit, which covered less than 0.4 percent of loan balances, accounted for $181 million 6 of
the $185 million (97.7 percent) improper payments used to calculate the estimated improper
payment rate for the FFEL Program.

Audits and Program Reviews
Could Identify the Same Improper Payments

Audits and program reviews conducted by different organizations that review the same entity
could identify the same improper payments. For example, program reviews conducted by FSA
and the single audits and program audits conducted by independent public accountants may cover
the same loan balances. FSA’s documentation for lenders showed that the FSA program reviews
provided oversight for 26 percent of loan balances and reported monetary findings of $621,300.
Additionally, FSA’s documentation showed that the single audits and program audits provided
95 percent coverage of the same loan balances and reported monetary findings of $864,400.

The likelihood of two reviewers identifying the same improper payment is minimized by the
limited nature of the audits and program reviews. Auditors review an entity’s internal controls and
may limit their testing to a small sample of loans. Program reviewers may also only review
documentation for a limited number of loans and may limit the scope of their reviews to certain
aspects of the FFEL Program. However, if the auditors and program reviewers identify the same
condition at a lender or guaranty agency, the projection of the improper payments to the total
payments made to the entity could result in duplication of improper payment amounts.



5
 Special Allowance Payments to New Mexico Educational Assistance Foundation for Loans Funded by Tax-Exempt
Obligations (ED-OIG/A05E0017), issued May 2005.
6
    FSA calculated the amount of improper payments identified from OIG audit reports using the following formula:
                           $281,140,952,852
                                                X $688,767 = $180,880,488
                           $1,070,544,497
Final Report
ED-OIG/A09H0015                                                                       Page 8 of 26

Applicability of Identified Improper
Payments to the Reporting Period

In its Appendix C guidance, OMB recommended that the annual estimate of improper payments
reported in the PAR coincide with the fiscal year being reported, but allowed, in limited cases, that
agencies report using the previous year’s data. Improper payments identified by audits and
program reviews may be for periods prior to both the fiscal year being reported and the previous
fiscal year. FSA used audit and program review reports issued during FY2005 to identify the
improper payments used to calculate the estimated improper payment rate reported in the PAR for
FY2006. For example, the OIG audit report (ED-OIG/A05E0017) identified improper payments
that occurred in FY2003 and FY2004. The identified improper payments become less applicable
to the PAR reporting period as the time lengthens between the occurrence of the improper
payments and the fiscal year used for reporting in the PAR.

Exclusion of Contested Findings and Use of
Final Determinations of Liabilities Understate
Improper Payments Identified in Audits and Reviews

The amount of improper payments used in FSA’s methodology is understated due to FSA’s
exclusion of contested finding amounts (e.g., lender or guaranty agency disagreed with the finding)
and use of the liability amounts from final determinations on reported findings.

   •	 Contested findings. Improper payments identified in contested findings were not included
      in the improper payment total used to develop the estimated improper payment rate. The
      OIG audit report (ED-OIG/A05E0017) with the $688,767 improper payments that was
      mentioned in a prior section also included a finding that the lender may have been
      improperly paid $18.4 million in special allowance payments for loans that were not
      eligible to be billed under the 9.5 percent floor. FSA only included the $688,767 of
      improper payments, which related to a finding that the lender agreed with in its comments
      to the report.

   •	 Final determinations of liability. FSA’s document describing the FY2006 methodology
      showed that the amount of improper payments from FSA program review findings was
      obtained from the PEPS. FSA disclosed in its document that the PEPS data entry
      procedures, in effect at the time, delayed entry of the dollar amount of a program review
      finding into the PEPS until the review was closed on the system and monetary adjustments
      had taken place. FSA had delayed entering the amounts because the amounts in the final
      determination of a finding or after appeals may be different from the amount identified in
      the initial program review report.

Even when a contested finding is upheld or the final adjustment takes place, the amount of the
improper payments can exceed the amount of the recorded liability. An OIG audit of a lender that
was recently resolved provides an example of the possible magnitude of the difference. The audit
Final Report
ED-OIG/A09H0015                                                                                     Page 9 of 26

report disclosed $278 million of improper payments. 7 While the finding was upheld, the
Department determined a final liability of zero. Even when the Department decides not to
establish or recover a liability, the improper payments should be included in determining the
estimated improper payment rate.

Certain Payments to Guaranty Agencies
Were Not Included

FSA omitted certain payments to guaranty agencies from the total payments used to calculate the
estimated improper payment rate. When defining payments made to guaranty agencies, FSA only
included transactions that passed through the Department’s payment system. For example, for
account maintenance fees, FSA included the $160.7 million in payments made through the
payment system, but did not include the additional $103 million that guaranty agencies received by
transferring funds from their Federal fund to their Operating fund. FSA also did not include the
payments for default aversion fees in total payments. Guaranty agencies calculate these fees and
receive payment by transferring funds from their Federal fund to their Operating fund. Guaranty
agencies reported transferring about $86 million for default aversion fees for FY2005.

Estimated Improper Payment Rate
Reported in the PAR for FY2007

FSA used a statistical sampling methodology to develop the estimated improper payment rate
of 0.032 percent for FY2007. FSA reviewed statistically selected samples for two types of
payments made during FY2005: payments to lenders and payments to guaranty agencies. For
each payment type, FSA used a two-stage sampling process. In the first stage, a sample of
payments was selected by Financial Partners Services. In the second stage, FSA selected a
sub-sample of loan-level transactions from the invoice supporting the payment selected in the first
stage. FSA’s Financial Partner Eligibility & Oversight, a component of FSA’s Program
Compliance unit, selected the second stage sub-samples and conducted the associated reviews at
the guaranty agencies, lenders, and lender servicers.

Pilot reviews began in late November 2006. Lessons learned from these reviews were
incorporated into a 59-page review guide, which was finalized in March 2007. The guide included
provisions for consistency of reviews, such as detailed electronic templates, and oversight over
reviews. The team leaders, who conducted the pilot reviews, conducted training for the rest of the
reviewers during February and March 2007.

    •	 Payments from ED to Lenders: Lenders submit quarterly invoices to the Department via
       the Lender Reporting System (LaRS). Lenders invoiced the Department for interest and
       special allowance payments, which could be offset by loan origination and lender loan fees
       that lenders owed the Department. FSA identified 11,264 payments, totaling $3.2 billion,
       made in FY2005 to the approximately 3,200 lenders that participate in the FFEL Program
       (lender payment universe).

7
  Special Allowance Payments to Nelnet for Loans Funded by Tax-Exempt Obligations (ED-OIG/A07F0017), issued
September 2006. Since the audit report was issued in FY2006, the report was not included in the reports reviewed to
identify improper payments for development of the estimated improper payment rate for FY2006. The FY2006
estimated rate was based on improper payments identified in reviews of audit and program review reports issued in
FY2005.
Final Report
ED-OIG/A09H0015                                                                    Page 10 of 26

     In the first stage, FSA randomly selected 47 primary sample units, where a primary sample
     unit was a payment for a specific fiscal quarter tied to a specific invoice from a lender or a
     lender servicer. Depending on the number of lender servicers used by a lender, that lender
     could be associated with multiple invoices and multiple payments for any one fiscal
     quarter. The 47 payments selected in the first stage random sample were made to
     47 different lenders.
     For the second stage sub-sampling, FSA obtained the lender or lender servicer data used to
     support the invoice that had been the basis for the Department’s payment. The data was
     requested from the lender/lender servicer at the loan level, which linked each component of
     the invoice with a unique loan (i.e., a specific loan for a specific borrower.) In the second
     stage, FSA statistically selected unique loans and reviewed all transactions related to that
     loan which affected the selected invoice. For example, in an invoice, one loan might be
     associated with interest, special allowance, loan origination, and lender loan fee
     transactions, whereas another loan might only be associated with one of those transactions.
     The number of second stage sub-samples selected depended on the number of unique loans
     supporting the invoice.
     For each sampled loan, the review guide required the FSA reviewer to obtain electronic and
     hardcopy documentation from the lender that the reviewer used to examine the accuracy of
     data and calculations supporting the lender’s invoice. The activities that FSA reviewers
     were instructed to complete and document for each sampled loan included:
        ƒ   Verify that totals on the lender’s electronic back-up files agree with totals reported
            by the lender and paid through the LaRS process.
        ƒ   Verify the existence of a valid signed promissory note. Compare the signature date
            with the date of the first disbursement to verify interest rates and special allowance
            categories that are based on the date.
        ƒ   Verify that origination and lender fees due to the Department for loans originated
            during the quarter were paid timely and accurately.
        ƒ   Verify eligibility for interest and special allowance benefits during the quarter by
            examining lender documentation pertaining to loan amount, disbursement date,
            current status, payment amount, deferment, and forbearance.
        ƒ   Verify application of payments: Recalculate interest due on balance and verify that
            payment was applied correctly to interest, principal and late fees.
        ƒ   After recalculating the ending balance, recalculate interest accrued for the quarter
            on subsidized loans and payable by the Department, and compare to amount
            invoiced by lender.
        ƒ   Using transaction history, recalculate the average daily balance for the quarter and
            compare to amounts reported by lender to support special allowance payments by
            Department.
     We concluded from our evaluation of the documentation for five of the FSA lender reviews
     that the reviewers had adhered to the review instructions specified in the review guide.
     However, as we noted later in the finding, the review process did not consider subsequent
     transactions, did not verify all components of the special allowance category code, and did
     not confirm that lenders had satisfied due diligence requirements.
Final Report
ED-OIG/A09H0015                                                                                     Page 11 of 26

         After a cost-benefit discussion with OMB, FSA decided to cease the reviews before
         completion of all 47 lender reviews. FSA provided documentation showing that 42 of the
         47 reviews were completed, of which 38 reviews were conducted on-site at lenders or
         lender servicers and 4 were conducted as desk reviews. 8 FSA documented that the
         invoices for the 42 completed reviews totaled about $12 million, and that the absolute value
         of the sub-samples reviewed for those invoices totaled about $29,243.

    •	 Payments from ED to Guaranty Agencies: Guaranty agencies submit monthly invoices to
       the Department using the Guaranty Agency Financial Report (GAFR). Guaranty agencies
       request payments for various types of claims, such as claims for default, bankruptcy, death,
       disability, and closed schools. The amount invoiced would be offset by amounts the
       guaranty agency owed the Department for collections and refunds. FSA identified
       488 payments, totaling $3.5 billion, made during FY2005 to the 35 guaranty agencies that
       participate in the FFEL Program (guaranty agency payment universe).
         In the first stage, FSA randomly selected 44 primary sample units, where a primary sample
         unit was a payment for a specific month to a specific guaranty agency. The 44 payments
         selected in the first stage random sample were made to 27 of the 35 guaranty agencies.
         As had been done for the lender payments, FSA obtained the loan-level data used to
         support the invoice from the guaranty agency and used the data to randomly select the
         second stage sub-sample. Then, FSA examined all transactions for the specific month that
         were related to the selected loan.
         Similarly, for each sampled loan, the review guide required the FSA reviewer to obtain
         electronic and hardcopy documentation from the guaranty agency that the reviewer used to
         examine the accuracy of data and calculations supporting the guaranty agency’s invoice.
         The activities that FSA reviewers were instructed to complete and document for each
         sampled loan included:
             ƒ    Verify that totals on the guaranty agency’s electronic back-up files agree with totals
                  reported by the guaranty agency and paid through the GAFR process.
             ƒ    Analyze supporting documentation to determine the type of transaction being
                  reviewed and determine if it was reported under the correct line item(s).
             ƒ    Verify the existence of a valid signed promissory note and compare it with loan and
                  transaction records.
             ƒ    For claims, verify the accuracy of the invoiced amount by examining the supporting
                  documentation pertaining to type of loan, disbursement/guarantee dates and
                  amounts, and reinsurance rate. Verify that the claim payment date and amount paid
                  to lender were correctly reported to the Department.
             ƒ    For collections, verify that payments received were reported timely and accurately
                  by examining the supporting documentation pertaining to type of loan, first
                  disbursement date, disbursed amount, reinsurance rate, receipt dates and amounts,
                  payment attribution to all the borrower’s loans, and amount attributed to the specific
                  loan selected for the sub-sample.



8
 FSA used a number of criteria to determine if desk reviews were appropriate, such as entity portfolio size,
complexity of known issues, and the lender’s ability to provide appropriate requested information.
Final Report
ED-OIG/A09H0015                                                                                 Page 12 of 26

           We concluded from our evaluation of the documentation for two of the FSA guaranty
           agency reviews that the reviewers had adhered to the review instructions specified in the
           review guide. However, as we noted later in the finding, the review process did not include
           a confirmation of required due diligence activities for default claims.
           As discussed above, FSA curtailed the planned reviews. FSA informed us that it
           completed 42 of the 44 guaranty agency reviews, and conducted all 42 reviews on-site at
           the guaranty agencies. 9 FSA documented that the invoices for the 42 completed reviews
           totaled about $271 million, and that the absolute value of the sub-samples reviewed for
           those invoices totaled about $846,835.

FSA reviewers identified a gross total (i.e., absolute value of over- and under-payments) of $42.37
for the lender reviews (improper payments identified in 2 of 42 reviews) and $163.26 for the
guaranty agency reviews (improper payments identified in 5 of 42 reviews.) For each invoice
reviewed, FSA divided the amount of any improper payments by the total amount of the loan-level
transactions reviewed for that invoice to calculate an error rate for the reviewed loan-level
transactions. FSA then multiplied the error rate for the reviewed loan-level transactions by the
total amount of the invoice to extrapolate an amount of improper payments for each invoice. FSA
calculated the amount of improper payments for the 84 lender and guaranty invoices at $74,835.
While conducting the guaranty agency reviews, FSA identified an additional $18,157 of improper
payments that were outside the specific loan-level transactions reviewed. These two amounts
totaled $92,992. However, due to a worksheet formula error when summarizing the amounts,
FSA calculated $89,998 as the total improper payments. To calculate the estimated improper
payment rate, FSA divided the $89,988 in total improper payments by the total payments for the
84 invoices reviewed ($283,175,828). The resulting estimated improper payment rate of
0.032 percent was reported in the FY2007 PAR. 10

Several factors affected the reliability of FSA’s improper payment rate for FY2007. The
methodology did not take into account determinations of student eligibility and other loan-making
activities. Also, an estimating methodology that relies on the results of a review of sampled
payment invoices is impacted by the method of sample selection, completeness of the payment
universe, use of FY2005 payment data, and design of the review process. Finally, the estimated
improper payment rate reported in the PAR was based on an interim calculation that did not
include all improper payments identified by the reviews.

Methodology Did Not Take Into Account Determinations
of Student Eligibility and Other Loan-Making Activities

Educational institutions often perform loan-making activities that impact whether payments to
lenders and guaranty agencies are proper. For example, educational institutions determine the
borrower’s eligibility for loans and loan amounts. They may also perform other loan-making
activities, such as explaining borrower’s rights and responsibilities under the loan. When certain
loan-making activities are not performed correctly, the federal guarantee on the loan may be
impacted. In addition, educational institutions and lenders rely on statements made by borrowers


9
    One of the two cancelled reviews was for a closed guaranty agency.
10
  The estimated improper payment rate would have been 0.033 percent if FSA had used the correct amount for the
total of estimated improper payments ($92,992) in its calculation.
Final Report
ED-OIG/A09H0015                                                                                        Page 13 of 26

on their loan applications. Inaccurate application information could result in an improper
determination of student eligibility for the loan and, thus, impact the federal guarantee on the loan.
Payments made to lenders for interest and special allowances and payments to guaranty agencies
for claims on loans that do not have a valid federal guarantee are improper payments. In addition,
activities performed by educational institutions that take place over the loan’s life-cycle may
impact federal interest payments to lenders.

While student eligibility and other loan-making activities could effect the payments to lenders and
guaranty agencies that were reviewed under FSA’s methodology, it would not have been feasible
for FSA to evaluate these activities as part of its review. Loan-making activities are completed by
numerous educational institutions located throughout the nation. Also, the student’s preparation of
the application and the loan-making and other activities performed by educational institutions may
have occurred several years earlier.

Even when audits and program reviews of educational institutions have identified loans that were
made to ineligible students, the lender and guaranty agency may continue to receive payments on
the ineligible loans. When FSA identifies ineligible loans through its program reviews at
educational institutions, FSA requires the educational institution to assume the liability for the
interest subsidy cost and risk of loss from default by the borrower for the ineligible amount of the
loan. 11 FSA uses the estimated actual loss formula to determine the institution’s liability
amount, 12 and establishes a receivable to recover from the school the estimated costs resulting
from a school’s disbursement of the ineligible loan. According to FSA’s procedures, the school is
not given the option of purchasing the ineligible loan from the lender.13 Since FSA establishes the
receivable directly with the school, the lender and guaranty agency may have no knowledge of the
loan’s ineligibility. Even though payments to the lender for interest and special allowance and the
guaranty agency for claims on the ineligible loans may be allowable payments to those entities,
they should be classified as improper payments for the purpose of estimating the improper
payment rate.

Sample Size and Use of
Simple Random Sampling

FSA used statistical software (EZ-Quant) to determine the number of payments to be selected for
review in the lender and guaranty agency universes and the related loan-level transactions
(i.e., sample sizes). FSA’s goal was to determine the sample sizes at a 95% confidence level with
a presumed 3 percent error rate and precision of +/-5 percent. However, FSA did not enter the
correct amounts into the EZ-Quant’s input screens for the “desired maximum precision amount”
when it determined the sample sizes. FSA entered an amount equal to 5 percent of the total dollar
amount of the payment universe instead of 5 percent of the presumed dollar amount of improper
payments (i.e., 3 percent of the dollar amount of the payment universe). For example, to determine


11
     Except for certain exceptions, such as cases involving fraud or cases where the student never attended school.
12
  The formula takes into account a school’s default rate and the amount of the ineligible disbursements to calculate an
amount that includes the estimated actual loss to the government for which the school is now liable.
13
  FSA does not provide the school with the option of purchasing the ineligible loan from the lender and establishing a
receivable from the borrower as this may deprive the borrower of certain benefits, such as consolidating their loans or
requesting income contingent repayment plans. In addition, the loan agreement gives the borrower benefits, such as
deferment and cancellation provisions, that may not be easily enforced against the school.
Final Report
ED-OIG/A09H0015                                                                              Page 14 of 26

the sample size from the guaranty agency universe, FSA used $175,360,350 (5 percent of the total
payments of $3,507,207,010) for the “desired maximum precision amount” instead of $5,260,811
(5 percent of the presumed improper payments of $105,216,210). 14 Due to the procedural error,
FSA used sample sizes that were significantly smaller than those recommended by the EZ-Quant
software. In the above example, EZ-Quant recommended a sample size of 484 from the universe
of 488 guaranty agency payments, basically a 100 percent review, to obtain FSA’s goal of a
sample with a 95% confidence level with a presumed 3 percent error rate and precision of
+/-5 percent. FSA had determined a sample size of 44. The impact of using smaller sample sizes
will not be known until a statistical projection of the actual sample results is performed. The
following table shows the amount of the payments made to lenders and guaranty agencies in
FY2005 that were selected for detailed review.

                Table 1. Amounts In Payment Universes and Related Samples
                                            Payments for              Payments for
                            Payment
       Payment Type                       Selected Invoices          Selected Loans
                            Universe
                                         (First Stage Sample)   (Second Stage Sub-Sample)
 Lender Payments                 $3.2 billion         $12 million                    $29,243

 Guaranty Agency Payments        $3.5 billion         $271 million                   $846,835


Use of stratified sampling, rather than simple random sampling, could improve the efficiency of
sampling. FSA used a simple random selection process to identify the invoices and loans in its
samples. In its Appendix C guidance, issued August 10, 2006, OMB suggested that “most
agencies will need to consult with a statistician to design an appropriate sample that may involve
. . . multiple stages of selection or stratification (rather than a simple random sample).” Although
FSA conferred with a statistician and used a two stage sample selection process, the samples in
each of the two stages were selected by a simple random process, which gave every item an equal
chance of selection. FSA’s discussions with the statistician did not include consideration of
alternatives to a simple random sample selection process. Stratification, where a universe is
divided into strata, is an alternative for sample selection that can be used to give special emphasis
to certain groups within the payment universes, such as invoices of high dollar values or those with
a great error potential.

Completeness of the Payment Universe

The lender and guaranty agency payment universes did not include all types of payments. The
lender payment universe was based entirely on information in FSA’s payment system which did
not include “accounts receivable” invoices. These are invoices where, after netting amounts due
from the Department against amounts due from lenders, the lenders owed the Department.
Because FSA did not include the “accounts receivable” invoices in the lender payment universe,
the amount payable by the Department on those invoices was excluded from the improper payment
review process.




14
  The presumed improper payment amount is calculated by multiplying the payment universe amount
($3,507,207,010) by the presumed error rate (3%).
Final Report
ED-OIG/A09H0015                                                                      Page 15 of 26

The guaranty agency payment universe, which was also based on information in FSA’s payment
system, did not include amounts that guaranty agencies received by transferring funds from their
Federal fund to their Operating fund. For example, the amounts transferred for the default
aversion fees mentioned in an earlier section of this report.

Use of FY2005 Payment Data

The lender and guaranty agency payment universes did not use the most current payment
information available in its methodology. FSA’s use of the FY2005 payment information
exceeded the one year allowance outlined in OMB’s Appendix C guidance, issued
August 10, 2006, and resulted in the application of an estimated improper payment rate for
FY2005 to program outlays for FY2007 and later. We also noted that FSA’s use of FY2005
payment information provided no coverage of excess interest fees during the improper payment
reviews. However, the exclusions of the excess interest fee likely had no impact on the estimated
improper payment rate since the fee became effective on April 1, 2006 and totaled about
$1.5 million in FY2006.

Review Process Did Not
Consider Subsequent Transactions

FSA’s review process did not consider subsequent transactions when reviewing loan-level
transactions for lender payments. A transaction would be considered accurate if it was accurate for
the quarter reviewed, even if it had been adjusted in a subsequent quarterly invoice. As mentioned
in a previous section of this report, subsequent adjusting transactions are required when lenders
receive updated borrower information or identify errors in their procedures. When examining the
transactions for a loan for the sampled quarterly invoice, FSA’s review process did not include
reviewing later invoices to determine whether any subsequent adjustments had been made, and if
so, to determine the reason for the adjustment. Thus, FSA’s process for reviewing loan-level
transactions may not have identified improper payments.

Review Process Did Not Verify All Components
of the Special Allowance Category Code

FSA’s review process for lender payments did not include verification of all components of the
special allowance category code, which is used to calculate the special allowance subsidy payable
by the Department. The selection of the appropriate special allowance category code is determined
by four loan characteristics: loan type, date, borrower status, and funding source (i.e., from either
taxable or tax-exempt sources). The review process included verification of the first three loan
characteristics, but did not include verification of the funding source. FSA management stated that
it was not practical to verify the funding source, because funding documentation is maintained at
the lender whereas most of the reviews were conducted at the lender servicer. Additionally, FSA
management stated that the funding sources were not confirmed because FSA reviewers had
knowledge of the type of funding source generally used by a lender and that separate on-going
reviews were examining the funding source used to support special allowance invoicing. Thus,
FSA’s review of sampled loans would not have identified improper payments that could have
occurred from lenders placing loans in an inappropriate special allowance category for the funding
source. The OIG audit report (ED-OIG/A05E0017), which was included in the methodology used
Final Report
ED-OIG/A09H0015                                                                                 Page 16 of 26

for the FY2006 PAR, identified a substantial amount of improper payments that resulted from a
lender’s use of the inappropriate special allowance category for the funding source.

Review Process Did Not Confirm That Lenders
Satisfied Due Diligence Requirements

FSA’s review process for lender payments and guaranty agency payments on default claims did
not include a confirmation that lenders performed required collection activities (referred to as due
diligence). The federal guarantee on the loan may be lost if a lender fails to complete the required
activities within certain timeframes. If the guarantee on a loan is lost, the lender also loses the
right to collect interest and special allowance until the lender completes the activities required to
reinstate the guarantee. Guaranty agencies may only submit default claims for loans that have a
valid federal guarantee. An FPE&O team leader informed us that FSA had made a decision not to
include due diligence in the review process.

Improper Payment Rate Reported in PAR
Was Based on Interim Calculation

The estimated improper payment rate of 0.032 percent reported in the PAR for FY2007 was based
on an interim calculation that did not include all improper payments identified by the reviews.
FSA provided documentation confirming that, as of September 30, 2007, reviews had been
completed of the 84 lender and guaranty agency invoices. However, FSA did not include the
results of some reviews in the calculation of the rate because, at the time the calculation was
prepared, FSA had not yet determined the actual amount of the improper payments for those
reviews. According to the FSA staff responsible for conducting the calculation, the related
program review report had not yet been provided to him or FSA was awaiting information from
the lender or guaranty agency to determine the improper payment amount.

In April 2008, FSA updated its calculation of the estimated improper payment rate in order to
respond to an information request from the OIG auditors. The revised calculation identified a
gross total of $203 improper payments for the lender reviews (improper payments identified in 3 of
42 reviews) and $1,140 improper payments for the guaranty agency reviews (improper payments
identified in 11 of 42 reviews.) After FSA extrapolated the identified errors to the related invoices,
the improper payment amount for the 84 invoices totaled $569,850. FSA increased the amount for
the additional improper payments identified outside the reviewed loan-level transactions to
$51,798. The two amounts totaled $621,648. Due to the same worksheet formula error that we
noted in the previous calculation, FSA calculated $618,644 as the total of estimated improper
payments. To calculate the updated improper payment rate for the 84 invoice reviews, FSA
divided the $618,644 in total improper payments by the total payments for the 84 invoices
reviewed ($283,175,828). The calculation resulted in an estimated improper payment rate of
0.218 percent. 15 The FSA staff stated that the calculation may be further revised after FSA
reviews responses and additional documentation provided by lenders and guaranty agencies and
makes final determinations of the improper payments for the individual reviews.




15
  The estimated improper payment rate would have been 0.220 percent if FSA had used the correct amount for the
total of estimated improper payments ($621,648) in its calculation.
Final Report
ED-OIG/A09H0015                                                                      Page 17 of 26

The Department did not disclose in the PAR that the reported rate was based on an interim
calculation that did not include all improper payments identified in its reviews. Both the reported
rate and the recalculated rate are well below the benchmark (2.5 percent) for significant erroneous
payments. Nevertheless, the PAR should contain complete and accurate information and disclose
limitations in reported information on improper payments. The PAR should also disclose the
statistical precision of the estimated improper payment rate.

Improper Payments Methodology
Planned for FY2008
FSA plans to use another methodology to estimate improper payments for the FFEL Program for
reporting in the FY2008 PAR. On February 27, 2008, FSA signed an interagency agreement with
the U.S. Department of Energy’s Oak Ridge National Laboratory (ORNL). The agreement’s
Scope of Work states that “ORNL will evaluate available data sources and develop a model that
utilizes that data to evaluate the relative risk of erroneous payments in the FFEL Program. The
model shall also evaluate the relative risk of [the Department] being billed inappropriately by
lenders or guaranty agencies. If enough relevant data can be extracted from identified data
sources, an estimated percentage of improper payments, by payment types and/or type of recipient
(GA or lender) will be calculated.” The project description stated that data on audits was available
in PEPS and from the Department’s Audit Accountability Resolution Tracking System. Other data
for lenders and guaranty agencies was contained in the Department’s National Student Loan Data
System and Financial Management System (FMS). If ORNL develops a model that uses the
results of previously conducted audits and reviews, the reliability of ORNL’s risk assessment, and
any estimated percentage of improper payments, may be similarly impacted by the factors
discussed in this report for FSA’s FY2006 methodology.

Recommendations

We recommend that the Acting Chief Operating Officer for FSA—

1.1 	   Ensure that the design and implementation of any future improper payment estimating
        methodologies that utilize results of previously conducted audits and reviews take into
        account or mitigate the effects of the following factors, when feasible: (1) improper
        payments identified in reviews other than single audits, program audits, OIG audits, and
        FSA program reviews of lenders and guaranty agencies (including improper payments self-
        reported by lenders); (2) lack of quantification of monetary impact of findings in the
        reports; (3) review periods that do not correlate to the federal fiscal year; (4) not being
        representative of all lenders and guaranty agencies; (5) duplicate identification of improper
        payments; (6) the time between the occurrence of the improper payments and the fiscal
        year used for reporting in the PAR; and (7) improper payments identified in contested
        monetary findings and findings without a final determination of liabilities.

1.2 	   Ensure that future methodologies appropriately include all payments, such as payments
        made by transferring funds between accounts (i.e., guaranty agencies’ transfer of funds
        from their Federal fund to their Operating fund) and payments netted by larger amounts
        due to the Department (i.e., accounts receivable invoices).
Final Report
ED-OIG/A09H0015                                                                        Page 18 of 26

1.3 	   Ensure that future methodologies use information on improper payments for the fiscal year
        used for reporting in the PAR or, if not available, for the prior fiscal year.

1.4 	   If sampling is used in future methodologies, consider focusing sample selection on
        higher-risk entities or types of transactions, such as subsequent adjustments and special
        allowance subsidies, to increase the likelihood of identifying improper payments.

1.5 	   Finalize its calculation of the estimated improper payment rate for FY2007 (including the
        measure of the statistical precision of the estimated rate) and use the finalized rate and its
        statistical precision when reporting the estimated improper payments for FY2007 in the
        PAR for FY2008. The FY2008 PAR should also include an explanation of the change
        from the rate reported in the FY2007 PAR.

1.6 	   In future PARs, disclose when the information presented on improper payments is based on
        interim calculations and the limitations of the reported information.

FSA Comments

FSA stated it will design and implement, in consultation with OMB, a methodology for estimating
FFEL improper payments that meets the requirements of Circular A-123, Appendix C, and that the
Department’s Office of the Chief Financial Officer and FSA have been meeting with OMB
regularly to reach agreement on a methodology for fiscal years 2009 and beyond. FSA’s
comments did not address the specific actions contained in Recommendations 1.1, 1.2, 1.3, and
1.4. In its comments on Recommendations 1.5 and 1.6, FSA stated it will calculate and use the
final rate from the reviews to report the FY2007 improper payments in the FY2008 PAR and will
note the change from the rate used in the FY2007 PAR. FSA stated it will disclose in future PARs
when information presented is based on interim calculations and any limitations of reported
information.

OIG Response

In April 2008, the OIG provided FSA with a preliminary draft of this finding for use in the design
and implementation of the methodology for FY2008. The execution of the methodology should be
substantially complete at this time. As FSA noted in its comments, decisions are currently being
made, in conjunction with OMB, on the design of the methodology for FY2009. FSA should
ensure that the actions in Recommendations 1.1, 1.2, 1.3, and 1.4 are taken, if applicable, when
evaluating and reporting limitations, if any, in the FY2008 methodology, and when designing,
implementing, and reporting on the FY2009 and future methodologies.
Final Report
ED-OIG/A09H0015                                                                                   Page 19 of 26

FINDING NO. 2 – F
                	 FEL Program Outlays Reported in PARs Represented
                Different Payment Universes and Did Not Reflect the
                Most Current Information

The outlays for the FFEL Program reported in the Department’s PARs for FY2006 and FY2007
and used to calculate the estimated dollars of improper payments reflected different payment
universes. Also, the PARs contained an estimated outlay amount for the current year when
information was available on the actual outlays for the year.

The annual PAR includes a table showing the improper payment reduction outlook. The table
contains improper payment information for each risk-susceptible program for five fiscal years:
prior year, current year, and the next three years. The dollars of improper payment shown in the
table are calculated by multiplying the amount of the program outlays by the estimated improper
payment rate. The following table shows the amounts that FSA reported in the PAR for FY2007.

                            Table 2. Improper Payment Reduction Outlook
                          Reported in the FY2007 PAR for the FFEL Program
                                              FY2006            FY2007                         FY2008 (a)
                                            (Prior Year)     (Current Year)                 (Current Year+1)
 Program outlays for fiscal year (b)           $11,718 million         $5,861 million          $4,307 million

 Estimated improper payment rate (c)              2.2 percent           0.032 percent          0.032 percent

 Estimated dollars of improper
                                                 $258 million             $2 million             $1 million
 payments for fiscal year

 (a) The amounts for FY2008 were also used in the PAR to report for FY2009 (Current Year+2) and
     FY2010 (Current Year+3).
 (b) The program outlays for FY2006 reflect the actual payments made to lenders and guaranty agencies in FMS
     (the Department’s financial management system). The program outlays for FY2007 and later years reflect the
     estimated outlays for FY2007 and FY2008 reported in the President’s Budget for FY2008.
 (c) The rate used for FY2006 was developed using monetary findings identified in audit and program review
     reports issued in FY2005. The rate used for FY2007 and future years was developed from reviews of a
     random sample of payments made to lenders and guaranty agencies in FY2005. As we disclosed in Finding
     No. 1, the 0.032 percent was derived from an interim calculation that did not include all improper payments
     identified in the reviews and, thus, understates the estimated improper payment rate. Finding No. 1 provides
     additional details on the two methodologies.


For the prior year’s program outlays, FSA used an amount obtained from FMS. The amount
reflected actual payments made by the Department to lenders and guaranty agencies during the
fiscal year. For the current and future years’ program outlays, FSA used estimated outlay amounts
reported in the President’s Budget. The amounts reflected the net present value of the anticipated
payments that will be made to lenders and guaranty agencies for guaranteed loans disbursed by
lenders during the respective fiscal year. Thus, the program outlays in the President’s Budget
represent amounts for payments that will be made to lenders and guaranty agencies over an
extended period of years in the future. The payment universe reflected in the President’s Budget is
substantially different from the payment universe reflected in FMS. For example, the President’s
Budget for FY2008 reported actual outlays of $27,132 million for FY2006. As shown in Table 2,
the amount of actual outlays from FMS was $11,718 million.
Final Report
ED-OIG/A09H0015                                                                       Page 20 of 26

FSA advised us that amounts obtained from FMS and the President’s Budget were used to comply
with OMB requirements. OMB Circular A-123, Appendix C instructs agencies to follow the
format required by OMB Circular A-136, Financial Reporting Requirements for reporting
improper payment information in the PAR. OMB Circular A-136, III.5.7, IV (a)(vi) provides the
following guidance on amounts to be used when reporting on improper payments—

        [A]gencies are expected to report on CY [current year] activity, and if not
        feasible, then PY [prior year] activity is acceptable. (Future year outlay estimates
        (CY+1, +2, and +3) should match the outlay estimates for those years as reported
        in the most recent President’s Budget).

While FSA complied with the instructions in the OMB Circular, its use of different payment
universes for program outlays resulted in dollars of improper payments presented in the PAR that
were not comparable among the fiscal years.

Also, as noted in Table 2, FSA used estimated outlays from the President’s Budget for FY2008 for
current year outlays reported in the FY2007 PAR. Since the PAR, which was issued in November
2007, contained the Department’s financial statements for FY2007, the actual outlay amounts for
the current year should have been available for improper payment reporting. The OMB Circular
expects agencies to report current year activity, when available.

Recommendation

2.1 	   We recommend that the Acting Chief Operating Officer for FSA develop and implement a
        revised policy for identifying and reporting program outlays for the FFEL Program in the
        PARs that provides consistent and comparable information on outlays and dollars of
        improper payments reported and utilizes the most currently available information on
        outlays.

FSA Comments

FSA stated that its operational policy and procedures are being updated to include the FFEL
payment universe definition, steps used to extract the payment universe for outlay reporting, and
queries to use for improper payment reporting to ensure consistency in future PARs.
Final Report
ED-OIG/A09H0015                                                                   Page 21 of 26



                  OBJECTIVE, SCOPE, AND METHODOLOGY 



The purpose of the audit was to assess FSA’s methodology for estimating improper payments in
the FFEL Program. Our review covered the methodology used for reporting in the PARs for
FY2006 and FY2007.

To gain an understanding of the methodologies and related internal controls, we interviewed FSA
management and staff in the Business Operations unit and Program Compliance’s Financial
Partner Eligibility and Oversight (FPE&O) unit. We also contacted FSA’s offices of the Chief
Financial Officer and Enterprise Performance Management Services, as well as the Department’s
OCFO, to understand their involvement in FSA’s IPIA efforts for the FFEL Program.

To gain a more complete understanding of the requirements of the IPIA, we interviewed the OMB
personnel who were involved with the development of Appendix C to OMB Circular A-123 and
with whom FSA coordinated its IPIA activities.

To assess the methodologies, we reviewed FSA’s written policies and procedures, written
communications between FSA and OMB, and supporting documentation for each methodology.

   •	 For the FY2006 methodology, our review was limited to review of FSA’s documentation
      of its methodology and four program audits, and interviews with the staff who had
      implemented the methodology.

   •	 For the FY2007 methodology, we reviewed FSA’s documentation of its methodology and
      assessed the quality of FPE&O’s reviews during our site visits to its western and southern
      regional offices. At the western region, we examined all four reviews that had been
      completed by FPE&O at the time of our visit (reviews of one guaranty agency and three
      lender invoices). At the southern region, we judgmentally selected three reviews for
      examination (reviews of one guaranty agency and two lender invoices) to include reviews
      conducted by both the Dallas and Atlanta field offices and a review that had been
      conducted as a desk review. We compared source documentation from lenders/lender
      servicers and guaranty agencies against the data documented in FPE&O reviewers’
      summary worksheets. We used FSA’s GAFR guide to verify the reinsurance rates and
      retention rates for the guaranty agency reviews, and FSA’s LaRS guide and information
      provided by the National Council of Higher Education Loan Programs to verify the special
      allowance category codes and interest rates for the lender/lender servicer reviews. We
      reviewed the logic and consistency of formulas in the reviewers’ electronic worksheets.
      We also interviewed the statistician, assigned to the Department’s Institute of Education
      Sciences, National Center for Education Statistics, to gain an understanding of his role in
      reviewing FSA’s methodology for selecting a statistically valid sample and for projecting
      the sample results.
Final Report
ED-OIG/A09H0015                                                                      Page 22 of 26

We conducted our audit fieldwork at FSA’s headquarters office in Washington, D.C. and at its
western (San Francisco) and southern (Dallas) regional offices. We judgmentally chose
San Francisco because of its proximity to our Sacramento audit office and Dallas to meet with the
FPE&O staff who had obtained the supporting data for the stage one samples and used it to select
the stage two sub-samples for the FY2007 entity reviews.

We held an exit briefing with FSA officials on April 10, 2008. Our audit was conducted in
accordance with generally accepted government auditing standards appropriate to the scope of the
review described above.




                             ADMINISTRATIVE MATTERS



Corrective actions proposed (resolution phase) and implemented (closure phase) by your office
will be monitored and tracked through the Department’s Audit Accountability and Resolution
Tracking System (AARTS). Department policy requires that you develop a final corrective action
plan (CAP) for our review in the automated system within 30 days of the issuance of this report.
The CAP should set forth the specific action items and targeted completion dates necessary to
implement final corrective actions on the findings and recommendations contained in this final
audit report.

In accordance with the Inspector General Act of 1978, as amended, the Office of Inspector General
is required to report to Congress twice a year on the audits that remain unresolved after six months
from the date of issuance.

In accordance with the Freedom of Information Act (5 U.S.C. §552), reports issued by the Office
of Inspector General are available to members of the press and general public to the extent
information contained therein is not subject to exemptions in the Act.

We appreciate the cooperation given us during this review. If you have any questions, please call
Gloria Pilotti at (916) 930-2399.

                                             Sincerely,

                                             /s/
                                             Keith West
                                             Assistant Inspector General for Audit

Attachment
Final Report
ED-OIG/A09H0015                                         Page 23 of 26


                                                       ATTACHMENT




                  FSA’s Comments on the Draft Report
                                                                AUG 19 2008

TO:           Keith West
              Assistant Inspector General for Audit Services
              Office of Inspector General

FROM:         Lawrence A. Warder
              Acting Chief Operating Officer

SUBJECT:      Draft Audit Report - "Federal Student Aid's Estimation of Improper
              Payments in the Federal Family Education Loan Program," Control
              Number ED-OIG/A09H0015

Thank you for providing us with an opportunity to respond to the Office of
Inspector General's (OIG) Draft Audit Report, "Federal Student Aid's Estimation
of Improper Payments in the Federal Family Education Loan (FFEL) Program,"
dated July 9,2008.

Federal Student Aid is committed to meeting the requirements of OMS Circular
A-123, Appendix C, for estimating improper payments in the FFEL program. As
you know, Federal Student Aid contracted with Oak Ridge National Laboratory to
provide an FFEL improper payment estimate for FY 2008. In addition, the
Department's Office of the Chief Financial Officer and Federal Student Aid are
meeting with OMS regularly to reach agreement by the end of this fiscal year on
a methodology for fiscal years 2009 and beyond that will satisfy OMS's
requirements.

Our response to each of the recommendations follows:

Finding 1, recommendation 1: Ensure that the design and implementation of
any future improper payment estimating methodologies that utilize results of
previously conducted audits and reviews take into account or mitigate the effects
of the following factors, when feasible: 1) improper payments identified in reviews
other than single audits, program audits, OIG audits, and FSA program reviews
of lenders and guaranty agencies (including improper payments self-reported by
lenders); 2) lack of quantification of monetary impact of findings in the reports; 3)
review periods that do not correlate to the federal fiscal year; 4) not being
representative of all lenders and guaranty agencies; 5) duplicate identification of
improper payments; 6) the time between the occurrence of the improper
                          830 First St. N.E., Washington, DC 20202
                             www.FecleraIStudentAid.ed.gov
                                     1-800-4-FED-AID

   FEDERAL STUDENT              AID:~H::-;~1::;i1!'START   HERE. GO FURTHER.
payments and the fiscal year used for reporting the PAR; and 7) improper
payments identified in contested monetary findings and findings without a final
determination of liabilities.

Response: Federal Student Aid will design and implement, in consultation with
OMB, a methodology for estimating FFEL improper payments that meets the
requirements of Circular A-123, Appendix C.

finding 1, recommendation 2: Ensure that future methodologies appropriately
include all payments, such as payments made by transferring funds between
accounts (i.e., guaranty agencies' transfer of funds from their Federal fund to
their Operating fund) and payments netted by larger amounts due to the
Department (i.e., accounts receivable invoices).

Response: Federal Student Aid will design and implement, in consultation with
OMB, a methodology for estimating FFEL improper payments that meets the
requirements of Circular A-123, Appendix C.

finding 1, recommendation 3: Ensure that future methodologies use
information on improper payments for the fiscal year used for reporting in the
PAR, or if not available, for the prior fiscal year.

Response: Federal Student Aid will design and implement, in consultation with
OMB, a methodology for estimating FFEL improper payments that meets the
requirements of Circular A-123, Appendix C.

finding 1, recommendation 4: If sampling is used in future methodologies,
consider focusing sample selection on higher-risk entities or types of
transactions, such as subsequent adjustments and special allowance subsidies,
to increase the likelihood of identifying improper payments.

Response: Federal Student Aid will design and implement, in consultation with
OMB, a methodology for estimating FFEL'improper payments that meets the
requirements of Circular A-123, Appendix C.

finding 1, recommendation 5: Finalize its calculation of the estimated improper
payment rate for FY 2007 (including the measure of the statistical preciSion of the
estimated rate) and use the finalized rate and its statistical preCision when
reporting the estimated improper payments for FY 2007 in the PAR for FY 2008.
The FY 2008 PAR should also include an explanation of the change from the rate
reported in the FY 2007 PAR.

Response: Federal Student Aid will use the final rate from the 2007 reviews to
calculate the 2007 improper payments in the 2008 PAR and will note the change
from the rate used in the 2007 PAR.




                                        2
Finding 1, recommendation 6: In future PARs, disclose when information
presented on improper payments is based on interim calculations and the
limitations of the reported information.

Response: Federal Student Aid will so disclose in future PARs.

Finding 2, recommendation 1: Develop and Implement a revised policy for
identifying and reporting program outlays for the FFEL Program in the PARs that
provides consistent and comparable information on outlays and dollars of
improper payments reported and utilizes the most currently available information
on outlays.

Response: Federal Student Aid is updating operational policy and procedures to
include the FFEL payment universe definition. steps used to extract the payment
universe for outlay reporting, and queries to use for improper payment reporting
to ensure consistency in future PARs.

Once again, we thank you for your recommendations and the opportunity to
review and respond to the report.

cc: Gloria Pilotti, Regional Inspector General for Audit
    Patrick J. Howard, Director. Student Financial Assistance Advisory and
     Assistance Team




                                       3