oversight

Federal Student Aid's Oversight of Schools Participating in the Title IV Programs

Published by the Department of Education, Office of Inspector General on 2015-09-29.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

 Federal Student Aid’s Oversight of Schools Participating in
                   the Title IV Programs


                                 FINAL AUDIT REPORT




                                    ED-OIG/ A03L0001
                                    September 29, 2015


Our mission is to promote the                            U.S Department of Education
efficiency, effectiveness, and                           Office of Inspector General
integrity of the Department's                            Philadelphia, Pennsylvania
programs and operations.
                        NOTICE

Statements that managerial practices need improvements, as well as
other conclusions and recommendations in this report, represent the
opinions of the Office of Inspector General. Determinations of
corrective action to be taken will be made by the appropriate
Department of Education officials.

In accordance with the Freedom of Information Act (5 U.S.C. § 552),
reports issued by the Office of Inspector General are available to
members of the press and general public to the extent information
contained therein is not subject to exemptions in the Act.
                                    UNITED STATES DEPARTMENT OF EDUCATION
                                                    OFFICE OF INSPECTOR GENERAL

                                                                                                                     AUDIT SERVICES



                                                                     September 29, 2015

Memorandum
TO:                  James W. Runcie
                     Chief Operating Officer
                     Federal Student Aid

FROM:                Patrick J. Howard /s/
                     Assistant Inspector General for Audit

SUBJECT:             Final Audit Report
                     Federal Student Aid’s Oversight of Schools Participating in the Title IV Programs
                     Control Number ED-OIG/A03L0001

Attached is the subject final audit report that covers the results of our review of Federal Student
Aid’s oversight of schools participating in the Title IV programs. We conducted our review at
Federal Student Aid’s offices in Washington, DC; Philadelphia, PA; Dallas, TX; and
San Francisco, CA. Our review covered July 1, 2010, through January 31, 2011. We have
provided an electronic copy to your audit liaison officer. We received your comments partially
concurring with the findings and recommendations in our draft report.

Corrective actions proposed (resolution phase) and implemented (closure phase) by your office(s) will
be monitored and tracked through the Department’s Audit Accountability and Resolution Tracking
System (AARTS). The Department’s policy requires that you develop a final corrective action plan
(CAP) for our review in the automated system within 30 calendar days of the issuance of this report.
The CAP should set forth the specific action items, and targeted completion dates, necessary to
implement final corrective actions on the findings and recommendations contained in this final audit.

In accordance with the Inspector General Act of 1978, as amended, the Office of Inspector General
is required to report to Congress twice a year on the audits that remain unresolved after 6 months
from the date of issuance.

In accordance with the Freedom of Information Act (5 U.S.C. §552), reports issued by the Office of
Inspector General are available to members of the press and general public to the extent information
contained therein is not subject to exemptions in the Act.

We appreciate the cooperation given us during this review. If you have any questions, please call
Bernard Tadley, Regional Inspector General for Audit, at (215) 656-6279.

Enclosure




 The Department of Education's mission is to promote student achievement and preparation for global competitiveness by fostering educational
                                                   excellence and ensuring equal access.
                                              TABLE OF CONTENTS

                                                                                                                                 Page


EXECUTIVE SUMMARY ...........................................................................................................1

BACKGROUND ............................................................................................................................3

AUDIT RESULTS .........................................................................................................................6

           FINDING NO. 1 – FSA Did Not Conduct Program Reviews in Accordance
                           With Its Program Review Procedures .............................................7

           FINDING NO. 2 – FSA Was Not Considering Annual Dropout Rate Data for
                           Program Review Selections ............................................................14

OBJECTIVES, SCOPE, AND METHODOLOGY ..................................................................16

Enclosure 1: Required Uses of FSA Program Review Documents..........................................21

Enclosure 2: Federal Student Aid’s Comments ........................................................................22
               Abbreviations, Acronyms, and Short Forms Used in this Report

Department                               U.S. Department of Education

Direct Loan Program                      William D. Ford Federal Direct Loan Program

Eligibility Services                     School Eligibility Service Group

FFEL                                     Federal Family Education Loan Program

Fiscal Worksheet                         Fiscal Review Worksheet

FSA                                      Federal Student Aid

FY                                       Fiscal Year

HEA                                      Higher Education Act of 1965, as amended

Improvement Specialists                  Institutional Improvement Specialists

Pell                                     Federal Pell Grant Program

PRQC                                     Program Review Quality Control

Review Specialists                       Institutional Review Specialists

Student Worksheet                        Student File Worksheet

Workplan                                 Program Review Workplan
Final Report
ED-OIG/A03L0001                                                                                      Page 1 of 26


                                      EXECUTIVE SUMMARY


The objective of our audit was to determine how Federal Student Aid (FSA) oversees schools’
administration of the Title IV programs of the Higher Education Act of 1965, as amended. To
address our objective, we focused our audit on FSA’s Program Compliance division’s
program review process to ensure schools properly administered the programs.1 Our original
audit period covered program reviews conducted from July 1, 2010, through
January 31, 2011. We performed follow-up work with FSA in March, April, and July 2014 to
determine whether FSA had changed its policies and procedures for performing program
reviews. We also expanded our audit period and reviewed FSA Performance Improvement
and Procedures Services Group’s revised quality control process and newly developed
“Program Review Quality Control Procedures” for October 2013 through June 2014.

During our review of 47 program reviews FSA conducted during our original audit period, we
found weaknesses in the Program Compliance division’s processes for performing program
reviews. We also found a weakness in selecting schools for program reviews. Specifically,
we found the following.

       FSA Did Not Conduct Program Reviews in Accordance With Its Program Review
        Procedures. Program review staff did not (1) maintain all required forms and documents
        in the program review files or always complete the forms, (2) always adequately
        document fiscal testing for timely disbursement of funds and excess cash, (3) always
        conduct distance education program reviews in accordance with FSA’s distance
        education program review procedures, and (4) determine schools’ compliance with the
        Direct Loan Program quality assurance system requirement.

        We also found limited evidence of supervisory review of the program review files to
        ensure program review procedures were adequately completed. Further, the time allotted
        to perform program reviews may not have been adequate.

       FSA Did Not Consider Annual Dropout Rate Data for Program Review Selection.
        We found that FSA’s Program Compliance division managers did not consider high
        annual dropout rates when prioritizing schools for program reviews as required by the
        Higher Education Act of 1965, as amended.

During our follow-up work in 2014, we confirmed that FSA did not change its policies and
procedures for conducting program reviews. However, we determined that in October 2013, the
Performance Improvement and Procedures Services Group updated its program review quality
control process. According to FSA’s “Program Review Quality Control Procedures,” the revised
process was designed to address some of the findings identified in this audit and to review the
mandatory requirements contained in FSA’s program review procedures. We concluded that if

1
  FSA also monitors schools’ administration of the Title IV programs through performing technical assistance,
receiving student complaints, financial and administrative analysis, and audit resolution processes. We gained an
understanding of how FSA uses these processes to identify high-risk schools for program reviews.
Final Report
ED-OIG/A03L0001                                                                               Page 2 of 26

FSA follows the “Program Review Quality Control Procedures,” staff should identify
deficiencies such as the lack of documentation and supervisory review we found. We reviewed
36 quality control review reports created under the new procedure as of June 2014, and the
reports indicated that staff found similar deficiencies. However, according to the reports, School
Participation Team managers were not required to take corrective action on the
recommendations.

Additionally, we noted that the U.S. Department of Education uses the results of program
reviews to calculate its annual estimates of improper payments for the Federal Pell Grant and
Direct Loan Programs, under the Improper Payments Elimination and Recovery Act of 2010.
However, because of the extent of the deficiencies we found with the program reviews, the
annual estimates may not be valid. On May 15, 2015, the OIG issued a report that included a
finding citing additional flaws using the results of program reviews to calculate improper
payment rates for the Federal Pell Grant and Direct Loan Programs.2

We recommend that the Chief Operating Officer for FSA require the Program Compliance
division’s Chief Compliance Officer to—

       Revise FSA’s “Program Review Procedures” to (1) ensure all work is documented and
        require supervisory review of the program review files, and (2) develop steps to review a
        school’s compliance with the Direct Loan Program quality assurance system.

       Require School Participation Team managers to take corrective action on the
        recommendations made as a result of the Performance Improvement and Procedures
        Services Group’s quality control reviews.

       Reassess whether the allotted time to complete a program review is adequate to allow for
        staff to document all work performed and for supervisors to complete reviews.

       Consult with the National Center for Education Statistics regarding the feasibility of
        collecting and calculating annual dropout rates for schools and use the rates as a factor to
        prioritize schools for program reviews.

We provided a draft of this report to FSA. In FSA’s comments on the draft report, FSA agreed
with issues identified in Finding No. 1, “FSA Did Not Conduct Program Reviews in Accordance
With Its Program Review Procedures.” FSA agreed with three of the five recommendations.
While FSA did not explicitly agree with the other two recommendations, FSA noted that it will
take corrective actions to address the recommendations. FSA disagreed with Finding No. 2,
“FSA Was Not Considering Annual Dropout Rates for Program Review Selections,” and its
related recommendation. However, FSA stated that it will take corrective action to address the
recommendation.

We considered FSA’s comments on the draft report and did not make changes to the report in
response. We summarized FSA’s comments at the end of each finding. FSA’s complete
comments are included as Enclosure 2 of this report.

2
 “U.S. Department of Education's Compliance with Improper Payment Reporting Requirements for Fiscal Year
2014,” ED-OIG/A03P0003.
Final Report
ED-OIG/A03L0001                                                                                        Page 3 of 26



                                               BACKGROUND


The U.S. Department of Education (Department) operates loan and grant programs authorized
under Title IV of the Higher Education Act of 1965, as amended (HEA). The Title IV programs
include the Federal Family Education Loan Program (FFEL Program), William D. Ford Federal
Direct Loan Program (Direct Loan Program), Federal Perkins Loan Program, Federal Pell Grant
(Pell) Program, Federal Supplemental Educational Opportunity Grant Program, and Federal
Work Study Program. According to the Department’s Federal Student Aid (FSA) 2014 annual
report, in fiscal year (FY) 2014 FSA processed more than 20 million applications and delivered
$133.8 billion to about 12.9 million students and their families through the Title IV programs.

The Department operates two major student loan programs: the FFEL Program, which made
loans available to students and families through private lenders, and the Direct Loan Program,
which lends funds directly to students and families through participating schools. In response to
disruptions in the credit markets and concern over access to FFEL Program loans, the Ensuring
Continued Access to Student Loans Act of 2008 (Public Law 110-227) was enacted on
May 7, 2008, and provided the Department with the authority to purchase FFEL Program loans
to support new FFEL Program loan originations. However, between May 2008 and March 2010,
some schools participating in the FFEL Program began transitioning to the Direct Loan Program
because of the stability and reliability it provided.

On March 30, 2010, the Health Care and Education Reconciliation Act (Public Law 111-152),
which included the SAFRA Act, was enacted, and required that all new student loans3 be made
through the Direct Loan Program beginning July 1, 2010. Schools participating in the FFEL
Program transitioned to the Direct Loan Program. According to data obtained from the
Department’s National Student Loan Data System,4 from July 1, 2008, through July 1, 2013, the
number of schools participating in the Direct Loan Program increased by 185 percent, from
2,215 to 6,309 schools. According to the Department’s Budget Summaries for Fiscal Years 2010
and 2015, the loan volume of new Direct Loans increased from $18.2 billion in FY 2008 to
$101.3 billion in FY 2013, a 456 percent increase.

Under Section 498A of the HEA, the Department is required to conduct program reviews of
schools participating in the Title IV programs, giving priority to schools that meet certain
criteria. Within the Department, FSA is responsible for administering the Title IV programs.
FSA is a performance-based organization that is staffed by more than 1,200 full-time employees
working in various component groups in Washington, D.C., and regional offices throughout the
country. Within FSA’s Program Compliance division, the School Eligibility Service Group
(Eligibility Services) is responsible for monitoring and oversight, including conducting program
reviews of schools participating in the Title IV programs.

FSA’s Eligibility Services is divided into eight School Participation Teams—seven for domestic
schools and one team that focuses on foreign schools. Team members are located in FSA’s

3
    Exclusive of Federal Perkins Loans, which are provided to students through participating schools.
4
    The National Student Loan Data System is the Department’s central database for student financial aid.
Final Report
ED-OIG/A03L0001                                                                                      Page 4 of 26

10 regional offices and at headquarters. Each of the School Participation Teams is responsible
for the oversight and monitoring of the schools within their regions. The School Participation
Teams and staffing levels, as of June 2014, are shown in Table 1.

                 Table 1: School Participation Team Offices and Compliance Staff

                           Office                                              Number of Staff
        New York and Boston                                                         38
        Philadelphia                                                                28
        Atlanta                                                                     27
        Chicago and Denver                                                          35
        Dallas                                                                      23
        Kansas City                                                                 27
        San Francisco and Seattle                                                   33
        Washington, D.C., and New York                                              21
        (including foreign schools)
                          TOTAL                                                        232

According to FSA’s Program Compliance division’s policies and procedures, School
Participation Teams monitor schools and servicers through program reviews, financial and
administrative analysis, audit resolution, student complaint, and technical assistance processes.
Each School Participation Team has a division director who manages the School Participation
Team. Within each School Participation Team, compliance managers supervise the School
Participation Team staff. Institutional review specialists (review specialists) perform program
reviews of schools and audit resolution. Financial analysts perform financial analysis to assess
schools’ financial responsibility. Institutional improvement specialists (improvement specialists)
provide technical assistance to schools through telephone contacts, written guidance, and
specialized training for targeted groups; and as needed, assist review specialists with performing
program reviews. From July 1, 2010, through January 31, 2011, FSA’s Program Compliance
division hired 58 new review specialists. This resulted in a 74 percent growth in program review
staff.5 FSA also hired or transitioned seven people to the improvement specialist position.

A program review evaluates a school’s administration of the Title IV programs.6 Review
specialists are required to follow FSA’s “Program Review Procedures” when conducting
program reviews. Review specialists performed three types of program reviews: general
assessment, focused, and compliance assurance. A general assessment review, the most common
type, generally evaluates a school’s compliance with the Title IV program requirements. A
focused review has a narrowed scope to focus on specific issues, such as a school’s compliance
with Federal campus security regulations, consumer information policies, or Title IV refunds. A
compliance assurance review, which is conducted at schools determined as low-risk by FSA’s
Eligibility Services, also has a limited scope and does not require fiscal testing to be performed.
Fiscal testing is the review of a school’s cash management processes and delivery of funds to
students.


5
  The staffing levels as of the end of our audit period, January 2011, totaled 201. Before July 1, 2010, FSA had
78 review specialists. From July 1, 2010, through January 31, 2011, the total number of review specialists was 136.
6
  FSA did not make any modifications to its program review process after the passage of SAFRA.
Final Report
ED-OIG/A03L0001                                                                         Page 5 of 26

During our original audit period, FSA conducted 47 program reviews; 37 were conducted at
schools that participated in the Federal Family Education Loan Program before the
implementation of Public Law 111-152, which required that all new student loans be made
through Direct Loan Program beginning July 1, 2010. Ten of these schools were already
participating in the Direct Loan Program.

Section 498A of the HEA requires the program review selection process to give priority to a
school’s high cohort default rate, the high dollar volume of the school’s default rate, significant
fluctuations in a school’s loan volume, deficiencies or financial aid problems of a school, high
annual dropout rates, and risk of noncompliance with administrative capability or financial
responsibility requirements. According to FSA’s “Program Review Procedures,” schools are
also selected for program review as a result of one or more of the following:

          compliance initiatives, which target specific areas of potential weakness and are
           intended to improve compliance with Title IV program laws, regulations, and
           procedures;
          referrals or complaints; and
          a comprehensive compliance review process (also referred to as the case management
           process), which includes continuously researching and analyzing information
           available about a school.

FSA’s procedures state that the School Participation Team performs a risk assessment, using the
data obtained from the comprehensive compliance reviews process, to determine whether it
should conduct a program review of a school.

According to FSA’s Deputy Chief Compliance Officer, to aid its oversight process, FSA’s
Eligibility Services managers analyzes data and assesses overall risk of schools based on risk
factors known or indicated by FSA’s data workgroup, information collected from the National
Student Loan Data System, and other departmental information systems. Eligibility Services
develops compliance initiatives to target high-risk schools for program reviews. School
Participation Teams issue a report on the results of the program review at the end of the program
review process. The program review report includes recommendations to correct any findings of
noncompliance identified during the program review. Liabilities may be assessed depending on
the type of noncompliance identified. FSA’s Eligibility Services classifies overall program
review findings into three main levels:

      moderate—between $1 and $500,000 in liabilities,
      serious—between $500,000 and $1 million in liabilities, and
      very serious—more than $1 million in liabilities.
Final Report
ED-OIG/A03L0001                                                                               Page 6 of 26



                                         AUDIT RESULTS


The objective of our audit was to determine how FSA oversees schools’ administration of the
Title IV programs. FSA’s Program Compliance division’s policies and procedures state that
FSA oversees schools’ administration of the Title IV programs through performing program
reviews, providing technical assistance to schools, addressing student complaints on schools,
performing financial and administrative analysis, and performing audit resolution activities. To
address our audit objective, we focused our audit on FSA’s Program Compliance division’s
program review process to ensure schools’ properly administer the Title IV programs. We did
not assess the Program Compliance division’s other oversight processes; however, we gained an
understanding of how the processes were used to identify high-risk schools selected for program
reviews.

We found significant weaknesses in the processes for performing program reviews. We also
found a weakness in selecting schools for program reviews. These weaknesses are identified
below.

       Program review specialists did not always conduct program reviews in accordance with
        FSA’s program review procedures. Specifically, required forms and documents were
        missing from the program review files, and staff did not always complete forms,
        adequately document fiscal testing for timely disbursement of funds and excess cash,
        determine whether schools had implemented Direct Loan quality assurance systems, and
        conduct distance education program reviews. We also found limited evidence of
        supervisory review of the program review files to ensure program review procedures are
        adequately completed. Further, the time allotted to perform program reviews may not
        have been adequate.

       Program Compliance division managers did not consider high annual dropout rates when
        prioritizing schools for program reviews as required by the HEA.

As a result of the significant internal weaknesses we found, FSA has limited assurance that
program reviews are appropriately identifying and reporting all instances of noncompliance.

Because the Department uses the results of program reviews to calculate its annual estimates of
improper payments for the Pell and the Direct Loan Programs, under the Improper Payments
Elimination and Recovery Act of 2010, the estimates may not be valid. On May 15, 2015, the
OIG issued a report that included a finding citing additional flaws using the results of program
reviews to calculate improper payment rates for the Federal Pell Grant and Direct Loan
Programs.7

We provided a draft of this report to FSA. In FSA’s comments on the draft report, FSA agreed
with issues identified in Finding No. 1, “FSA Did Not Conduct Program Reviews in Accordance

7
 “U.S. Department of Education's Compliance with Improper Payment Reporting Requirements for Fiscal Year
2014,” ED-OIG/A03P0003.
Final Report
ED-OIG/A03L0001                                                                                  Page 7 of 26

With Its Program Review Procedures.” FSA agreed with three of the five recommendations.
While FSA did not explicitly agree with the other two recommendations, FSA noted that it will
take corrective actions to address the recommendations. FSA disagreed with Finding No. 2,
“FSA Was Not Considering Annual Dropout Rates for Program Review Selections,” and its
related recommendation. However, FSA stated that it will take corrective action to address the
recommendation.

We considered FSA’s comments on the draft report and did not make changes to the report in
response. We summarized FSA’s comments at the end of each finding. FSA’s complete
comments are included as Enclosure 2 of this report.


FINDING NO. 1 – FSA Did Not Conduct Program Reviews in Accordance With Its
Program Review Procedures

We found that FSA did not conduct the program reviews in accordance with FSA’s “Program
Review Procedures,” September 30, 2008. Specifically, we found that staff did not always
(1) maintain required forms and documents in the program review files, (2) complete the forms;
(3) adequately document fiscal testing for timely disbursement of funds and excess cash;
(4) document required school interviews, sample the correct student population, test the
mandatory program review elements, or obtain all required school student attendance
information for distance education program reviews; or (5) determine a school’s compliance with
the Direct Loan quality assurance system. Additionally, we found limited evidence that
supervisors reviewed the program review files, and the number of days allotted to conduct
program reviews may not have been adequate.

Missing and Incomplete Program Review Documentation

Review specialists were required to complete eight standard forms to document the results of the
program review and note any instances of noncompliance or findings. These forms were to be
maintained in the program review file. None of the 47 program review files we reviewed
contained all required forms. Examples of the missing forms included the following:
“Return of Title IV Funds Policies and Procedures Worksheet,” “Preliminary Findings
Worksheet,” “Findings Matrix Worksheet,” and “Institutional Worksheet.” We found that all
47 of the program review files we reviewed had one or more required forms that were
incomplete.

Required Forms and Documents Were Missing
Of the 317 required forms that should have been in the 47 program review files, 125 (39 percent)
were missing. The 317 required forms exclude both the “Student File Worksheet”
(Student Worksheet) for all 47 program reviews and the “Findings Matrix Worksheet” for
12 program reviews that did not have student-level findings.8 Table 2 lists the eight required




8
  The 317 required forms consisted of 35 program reviews (47 minus 12) with 7 required forms and 12 program
reviews with 6 required forms.
Final Report
ED-OIG/A03L0001                                                                                      Page 8 of 26

forms and the number of program reviews that were missing those forms by School Participation
Team.9

For the 35 program review files that had student-level findings, none contained a
“Findings Matrix Worksheet,” as reflected in Table 2. FSA’s “Program Review Procedures,”
Section 18.1, requires review specialists to use this form to document which program reviews
had student-level findings in order to ensure all findings were included in the program review
report. If the review specialist does not use the “Findings Matrix Worksheet,” it can lead to
student-level findings not being reported in the program review report. We found unreported
student findings in 13 (37 percent) of the 35 program reviews that had student-level findings.




FSA’s “Program Review Procedures,” Section 10.3.3 required review specialists to include a
copy of the first page of the Institutional Student Information Record in the program review files
for each student sampled. The first page of the Institutional Student Information Record contains
student eligibility data, which the review specialist was required to review and verify. We found
that the first page of the form was not included for 45 students in 15 (34 percent) of the
44 program reviews which required the form. The first page of the Institutional Student
Information Record was not required for 3 of the 47 program reviews because students were not
sampled for those reviews.

Required Forms Were Incomplete
We found that 16 (34 percent) of the 47 program review files contained one incomplete required
form. For these 16 program reviews, 5 of the 7 required forms were incomplete. Table 3 lists
the five required forms that we found were incomplete and the number of program reviews that
had the incomplete forms.




9
 See Enclosure 1 for details on the required uses of the forms not discussed in the finding. Only one form for each
program review conducted (except for the “Student Worksheet”) was required to be in the file.
Final Report
ED-OIG/A03L0001                                                                                      Page 9 of 26




Review Specialists Did Not Always Complete Required Sections of Student Worksheets
We found that 90 percent of the Student Worksheets reviewed were incomplete.10 Each of the
44 program reviews11 contained about 30 Student Worksheets, one for each student sampled
(generally 15 students sampled per award year). We reviewed a total of 1,148 Student
Worksheets within the 44 program review files. Forty-one of the 44 program review files had
incomplete Student File Worksheets. The Student Worksheet had 27 sections that were required
to be completed (FSA’s “Program Review Procedures,” Section 11.1). These sections included
elements such as confirmation of attendance, review of refund calculations, and confirmation of
eligibility requirements. Table 4 provides a breakdown of how many sections of the student
worksheets were incomplete.




Two Required Forms Were Not Always Approved by Compliance Managers
Compliance managers had not approved a “Program Review Workplan” (Workplan) for
5 (11 percent) of the 47 program reviews. FSA’s “Program Review Procedures” (Section 6.1)
required compliance managers to review and approve Workplans before schools were notified of
a program review. Workplans, which were required for all program reviews, defined the

10
     We considered the Student Worksheet incomplete if any of the sections on the form were not completed.
11
     Three program reviews did not have Student Worksheets because program eligibility was the focus of reviews.
Final Report
ED-OIG/A03L0001                                                                    Page 10 of 26

purpose, scope, and program review methodology. The approval of Workplans provides
evidence of supervision in the planning phase of the review and documents that management
approved the scope, methodology, and type of review (general, focused, or compliance
assurance) performed.

Further, compliance managers had not approved a “Preliminary Findings Worksheet” for
11 (23 percent) of the 47 program reviews. FSA’s “Program Review Procedures” (Section 16.1)
required compliance managers to sign and date the “Preliminary Findings Worksheet” for all
program reviews. The worksheet was used to list the preliminary findings and other issues noted
during a program review and to brief School Participation Team management. The approval of
the “Preliminary Findings Worksheet” documents evidence of supervision in the reporting phase
of the program review.

Review Specialists Did Not Always Adequately Document Fiscal Testing for Timely
Disbursement of Funds and Excess Cash

Review specialists did not always adequately document fiscal testing for the timely disbursement
of funds to students and excess cash during program reviews. Our review of 23 general
assessment reviews and 1 focused review showed that 8 had inadequate documentation of testing
for the timely disbursement of funds to students and 7 of the 8 also had inadequate
documentation of testing for excess cash. Fiscal testing was required only for general
assessment reviews and focused reviews on fiscal management.

Review specialists were required to review a sample of the school’s drawdowns of Title IV funds
from the Department and trace them to the school’s bank accounts and students’ records to
determine whether excess cash existed and whether funds were disbursed to students timely
(FSA’s “Program Review Procedures,” Section 14.1). Excess cash is any amount of Title IV
program funds12 that is not disbursed to students’ accounts within 3 business days after the date
the school received the funds. Although FSA’s “Fiscal Review Worksheet” could be used to
document fiscal testing, its use was not required.

Inadequate documentation of the testing of excess cash was previously reported as a finding in
an OIG audit report, “Case Management and Oversight’s Monitoring of Postsecondary
Institutions,” September 30, 2004, ED-OIG/A04D0014. FSA agreed to clarify that fiscal review
should be documented whether or not there are findings. However, FSA’s “Program Review
Procedures” do not provide such clarification.

School’s Compliance With the Direct Loan Quality Assurance System Requirement Was
Not Reviewed

Although required by “FSA’s Program Review Procedures,” in Section 13.3.7.3.14, all of the
17 review specialists we interviewed stated they were unfamiliar with the requirement that
schools had to establish a Direct Loan quality assurance system and did not determine a schools’
compliance. Also, the five compliance managers we interviewed also stated they were
unfamiliar with this requirement.


12
     Excess cash excludes Federal Perkins Loan Program funds.
Final Report
ED-OIG/A03L0001                                                                     Page 11 of 26

Schools that participate in the Direct Loan Program are required to implement a quality
assurance system, as established by the Department and developed in consultation with schools
of higher education, to ensure that the schools are complying with program requirements and
meeting program objectives (Section 454(a)(4) of the HEA and 34 Code of Federal Regulations
685.300(b)(9)). On November 13, 2013, FSA’s Program Management Office issued an
electronic announcement reminder memorandum to schools that participate in the Direct Loan
Program.

Distance Education Program Reviews Were Not Properly Conducted

FSA School Participation Team members did not comply with the procedures in OIG/FSA’s
“Guide for Review Conducted for 2009–2010 OIG/FSA Risk Project,” April 1, 2010, and the
“Program Review Distance Education Final Procedures,” July 30, 2010. We found that the
teams did not always (1) document required student and faculty interviews during all reviews,
(2) sample the correct student population, (3) test the mandatory program review elements for all
students in the distance education sample, or (4) obtain all of the required information about the
school’s student attendance policies and practices.

FSA performed a total of 25 program reviews of schools’ distance education programs. Our
audit included 10 (40 percent) of the 25 program reviews.

Student and Faculty Interviews
For 6 of the 10 distance education program reviews, we found no documentation of student
and/or faculty interviews, although the team should have conducted interviews to test for regular
and substantive interaction between the students and the instructor.

Testing of Student Eligibility
Only 5 of the 10 distance education reviews performed used samples drawn from the population
of distance education students. The other five distance education reviews used samples drawn
from the population of all students receiving Title IV program funds.

We also found no documentation to show that all of the required program review elements were
tested during 4 of the 10 distance education reviews. Specifically, the required elements that
were not documented included return of Title IV funds, student account credit balances,
satisfactory academic progress, professional judgment, dependency override, cost of attendance,
and calculation and disbursement of Title IV funds.

Student Attendance
We did not see any evidence of attendance documentation to show the review of the school’s
process for determining student attendance, a student’s withdrawal date, or a student’s last date
of attendance in 8 of the 10 distance education program review files. Two program reviews files
contained attendance documents but did not contain documentation of the team’s review of the
school’s process for determining student attendance.

Additionally, 9 of the 10 distance education program reviews did not have documentation in the
files to show that the review teams asked faculty questions about how the school verifies student
attendance.
Final Report
ED-OIG/A03L0001                                                                     Page 12 of 26

Overall Internal Control Deficiencies With Supervisory Review

We found no evidence, other than approval of the Workplan and the Preliminary Findings
Worksheet, to show that compliance managers reviewed the program review files. Supervisory
review of the program review files should have found and addressed the missing and incomplete
program review documentation, and the inadequately documented testing we identified.
Compliance managers should have documented their review of the program review files to
ensure that all elements were tested and adequately supported by documentation.

Quality Control Procedures

FSA’s Program Compliance division developed quality control procedures in December 2010;
however, the procedures were not adequate. FSA’s Performance Improvement and Procedures
Service Group was responsible for the ongoing quality control process for the program review
function. The quality control review focused on five general risk areas that included data entry
into FSA’s case management information system for four aspects of the program review process
and the timeliness and prioritization of program reviews. Although the quality control
procedures stated that the process was used to evaluate the effectiveness of and compliance with
program review procedures, the quality control procedures did not include a requirement to
review the work papers supporting a program review.

During our follow-up work in April 2014, we found that FSA’s Performance Improvement and
Procedures Services Group implemented a new peer review quality control process in
October 2013. School Participation Teams performed peer reviews of other School Participation
Teams’ program reviews; these peer reviews covered 28 program reviews that were started in
FY 2012 and closed in FY 2013.

In February 2014, FSA’s Performance Improvement and Procedures Services Group
implemented a nationwide program review quality control (PRQC) process. According to FSA’s
“Program Review Quality Control Internal Procedures,” June 5, 2014, the PRQC process will be
continuous and performed at least annually for all School Participation Teams. It also states that
the procedures address the exception areas the OIG identified in this audit and the “must” areas
from FSA’s 2008, 2011, and 2012 “Program Review Procedures.” We reviewed the PRQC
procedures and the results of the 36 PRQC reviews as of June 2014. All 36 reviews noted issues
similar to those we identified, such as missing or incomplete program review documentation.
The PRQC review reports noted there was inconsistent use of checklists as evidence of
supervisory review of the program review work papers. As a result, the PRQC review reports
contained recommendations that compliance managers (or division directors, where applicable)
review the program review work papers before approving the program review reports and that
review specialists ensure such approval is retained in the program review files. Although FSA
provided the PRQC reviews report to the School Participation Teams and included
recommendations to address the issues noted in the review, the reports specified that School
Participation Team managers were not required to take corrective action on the recommendations
in the report.
Final Report
ED-OIG/A03L0001                                                                     Page 13 of 26

Time Allotted to Perform Reviews May Not Be Adequate

The time allotted for a program review may not be sufficient to perform and document all of the
required tasks specified in FSA’s “Program Review Procedures.” Five of the 17 review
specialists that we interviewed from the School Participation Teams specifically stated that they
felt overwhelmed with the amount of work they were required to perform onsite in the time
allotted for a program review (generally, 1 week onsite). Four of the six compliance managers
that we interviewed stated that the 50 days allotted to complete a program review (from the end
of the onsite fieldwork to the preparation of the program review report) may not be adequate
because some reviews are more time-consuming and complex than others based of the types of
findings identified. The limited time allotted to perform program reviews could be a
contributing factor for the lack of documentation and limited supervisory review.


RECOMMENDATIONS

We recommend that the Chief Operating Officer for FSA require the Chief Compliance Officer
to––

1.1    Revise the “Program Review Procedures” to require documentation of supervisory
       review of the program review file.

1.2    Ensure review specialists complete and document all aspects of the program review
       process.

1.3    Ensure review specialists are reviewing and documenting the review of a school’s Direct
       Loan quality assurance system.

1.4    Require School Participation Team management to take corrective action on the
       recommendations made in the PRQC review reports.

1.5    Reassess whether the current total time allotted to perform a program review is adequate
       to complete and document all required procedures.

FSA’s Comments

FSA acknowledged that some staff did not conduct program reviews in accordance with FSA’s
program review procedures that require completion of the required documentation. However,
FSA did not agree with the assertions in the report that “FSA has limited assurance that program
reviews are appropriately identifying and reporting all instances of noncompliance” and our
conclusions that this calls into question the validity of the estimates used to calculate the
Department’s improper payments. FSA stated that it is unreasonable that FSA or any
organization can ensure that all instances of noncompliance are identified. The findings and
amounts of liabilities identified during the program reviews FSA conducted during the period in
question demonstrate that significant instances of noncompliance were identified.
Final Report
ED-OIG/A03L0001                                                                        Page 14 of 26

FSA agreed with Recommendations 1.1, 1.2, and 1.4. While FSA did not specifically agree with
Recommendations 1.3 and 1.5, FSA noted that it will take corrective actions to address these
recommendations.

OIG Response

FSA acknowledged that it did not conduct program reviews in accordance with its program
review procedures and did not dispute any of the issues identified in Finding No. 1. We did not
state that FSA must identify all instances of noncompliance. FSA’s program review procedures
required specific documentation to support the work performed during a program review and that
when there is documentation to support a finding that it be reported. This was not the case
during our review.

We found issues with all 47 program reviews in our sample, including unreported student-level
findings in 37 percent of the 35 program reviews that had student-level findings. Because
program review files contained insufficient documentation to support a school’s compliance in
multiple required program review areas, and because staff did not report actual findings, we
stand by our conclusions in this report that (1) FSA has limited assurance that program reviews
are appropriately identifying and reporting all instances of noncompliance and (2) because the
Department uses the results of program reviews to calculate its annual estimates of improper
payments for the Pell and the Direct Loan Programs, the resulting estimates may not be valid.

We did not draw any conclusions on the significance of the amount of noncompliance identified
through program reviews. We concluded only that FSA does not know whether the instances of
noncompliance and improper payment amounts reported in program review reports should have
been greater than those identified and used for the improper payment calculations. The instances
of noncompliance and improper payment amounts reported in program review reports in no way
compensate for the lack of documentation to support a school’s compliance or that staff did not
report actual findings.


FINDING NO. 2 – FSA Was Not Considering Annual Dropout Rate Data For Program
                Review Selections

FSA’s Program Compliance division management was not in compliance with the Title IV HEA
program review requirements at Section 498A(a)(2)(E) of the HEA because FSA did not
consider high annual dropout rates to prioritize schools for program reviews. FSA’s Program
Compliance division management did consider the other statutory program review requirements
included in Section 498A(a)(2)(A) through (a)(2)(D) and (F) of the HEA. These requirements
include giving priority consideration for program review to schools with a high loan cohort
default rate (in excess of 25 percent or a default rate that places them in the highest 25 percent of
such schools); a high default rate in dollar volume (school is in the top 25 percent of schools
based on loan dollar volume); significant fluctuations in loan or grant award volume; and
reported deficiencies or financial aid problems by a State oversight agency.

Section 1.1 of FSA’s “Program Review Procedures” states that schools do not currently report
the dropout rate data element. However, the National Center for Education Statistics could
collect dropout rates as a part of the data collection in the Integrated Postsecondary Education
Final Report
ED-OIG/A03L0001                                                                      Page 15 of 26

Data System (as provided for in Section 487(a)(17) of the HEA and 34 Code of Federal
Regulations 668.14(b)).


RECOMMENDATION

We recommend that the Chief Operating Officer for FSA require the Chief Compliance Officer
to––

2.1    Consult with the National Center for Education Statistics regarding the feasibility of
       collecting and calculating annual dropout rates for schools and use the rates as a factor to
       prioritize schools for program reviews.

FSA’s Comments

FSA disagreed with this finding. FSA stated that it used dropout rate data in selecting schools
for program reviews. FSA stated that the statute does not provide a definition nor does it
describe how to calculate such a rate, including whether the rate should include all students or
only Title IV recipients. FSA stated that the dropout rate could be calculated in a number of
different ways. FSA chose to use the number of withdrawals compared to the total number of
those who graduated and withdrew in calculating a school’s dropout rate. This data was based
on enrollment status data reported in the National Student Loan Data System.

Regarding Recommendation 2.1, FSA agreed to consult with the National Center for Education
Statistics on collecting dropout rate data, but FSA believes that the recommendation should be
directed to the National Center for Education Statistics.

OIG Response

The statutory requirement for FSA to use dropout rate data in selecting schools for program
review has been in Section 498A of the HEA since 1992. During our audit, FSA informed us
that the data to calculate a dropout rate comparing the number of students who enrolled with the
number who withdrew or graduated was not available.

FSA did not use the alternative rate based on number of withdrawals versus graduates and
withdrawals until 2013. The rate was not a reasonable approximation of a dropout rate because
it did not compare withdrawals with enrollments.

Regarding Recommendation 2.1, the HEA charges the Department with selecting schools for
program review based in part on dropout rates. FSA selects schools for and performs program
reviews; as such, FSA is responsible for obtaining the data sufficient to calculate a reasonable
dropout rate. The recommendation is appropriately directed to FSA, even if it needs to obtain
the assistance of other Department offices for implementation.
Final Report
ED-OIG/A03L0001                                                                     Page 16 of 26



                 OBJECTIVES, SCOPE, AND METHODOLOGY


The objective of our audit was to determine how FSA oversees schools’ administration of the
Title IV programs. Our original audit objectives were to determine how FSA ensures that
schools participating in the Direct Loan Program were (1) properly administering the program
and (2) timely delivering program funds to eligible students. We determined that FSA used the
same program review procedures for the Direct Loan Program as it used for all other Title IV
programs. Because the procedures we reviewed applied to all Title IV programs, we modified
the report title and objective accordingly. We also determined that program reviews generally
cover the timely delivery of program funds to eligible students, which negated the need to
separately examine that issue under the second original objective. To accomplish our audit
objective, we focused our review only on FSA’s Program Compliance division’s program review
process to ensure schools’ properly administer the Title IV programs that was used for
conducting program reviews during our audit period. Our original audit period covered program
reviews conducted during July 1, 2010, through January 31, 2011. We expanded our audit
period and performed additional work for the period October 2013 through June 2014.

We held follow-up discussions with FSA’s Program Compliance Division and Performance
Improvement and Procedure Services Group managers in March and April 2014. We also held
follow-up discussions with the division directors and compliance managers from the three
School Participation Teams we reviewed in July 2014. Further, we reviewed additional
documents pertaining to FSA’s new quality control processes developed in 2014 and reviewed
the 36 quality control review reports as of June 2014. We performed the follow-up work during
March, April, and July 2014 to determine whether FSA’s Program Compliance division has
taken any corrective actions. We did not verify whether the quality control processes have been
fully implemented.

The Program Compliance division’s policies and procedures state that it oversaw schools’
administration of the Title IV programs through performing program reviews of schools,
providing technical assistance to schools, addressing student complaints received on schools,
performing analysis related to the financial responsibility and administrative capability of
schools, and conducting audit resolution activities. We did not evaluate the technical assistance,
audit resolution, financial and administrative analysis, or student complaint processes that FSA’s
Program Compliance division performed. We gained an understanding of how these processes
were used in the identification of high-risk schools for selection for program review.
We gained an understanding of the program review process, which included performing
preliminary risk assessments of schools using analysis related to the financial responsibility and
administrative capability of the schools, selecting schools for program review based on the risk
assessment, and conducting program reviews of schools.

We judgmentally selected three (38 percent) of FSA’s eight regional School Participation Teams
to review: Philadelphia, Pennsylvania; Dallas, Texas; and San Francisco, California. Our
selections were based on a combination of (1) the number of schools participating in the Direct
Loan Program in the School Participation Team’s regional area (school volume), (2) the number
of review specialists on the School Participation Team relative to school volume, (3) the amount
Final Report
ED-OIG/A03L0001                                                                    Page 17 of 26

of Direct Loan Program funding received by schools in each regional area, (4) the number of
program reviews each School Participation Team conducted during our audit period, and (5) the
percentage of schools new to the Direct Loan Program after July 1, 2010, in each regional area.
We also based our selections on geographic location. We selected one School Participation
Team from the northwest, one from the northeast, and one from the south central regions of the
United States. We selected the San Francisco School Participation Team because it had the
highest volume of schools in its regional area and a high rate of schools new to the Direct Loan
Program. We selected the Dallas School Participation Team because of the small number of
program reviews completed relative to its staff size. We selected the Philadelphia School
Participation Team because it had a high volume of schools and the number of program reviews
completed. Because our audit was limited to three School Participation Teams, the results may
not be representative of the entire universe of program reviews conducted during our audit
period.

For background related to our audit objective, we obtained and reviewed background
information on the Title IV programs and FSA’s processes and systems.

   1. We reviewed the HEA and regulations related to our objectives.

   2. We reviewed the U.S. Government Accountability Office’s “Standards for Internal
      Control in the Federal Government,” November 1999.

   3. We reviewed prior audits and other reports:

       a. OIG management information report, “Federal Student Aid’s Efforts to Ensure the
          Effective Processing of Student Loans under the Direct Loan Program,”
          September 16, 2010, ED-OIG/X19K0008;

       b. OIG management information report, “Review of Federal Student Aid’s Enterprise
          Risk Management Program,” May 5, 2009, ED-OIG/I13I0005;

       c. OIG “Case Management and Oversight’s Monitoring of Postsecondary Institutions,”
          September 30, 2004, ED-OIG/A04-D0014;

       d. FSA’s Direct Loan Compliance Enhancement Workgroup report, July 2009, and
          related emails;

       e. FSA’s Internal Review Group “Program Compliance Management Review Executive
          Report,” July 2012; and

       f. FSA’s Compliance Initiative reports for FYs 2008, 2012, and 2013.

   4. To gain an understanding of FSA’s processes and systems used pertaining to the
      objective of our review we:

       a. Interviewed officials from FSA’s Program Compliance division, including the acting
          chief compliance officer, the deputy chief compliance officer, and the South Central
          team director. We also interviewed officials from the Administrative Actions and
Final Report
ED-OIG/A03L0001                                                                   Page 18 of 26

        Appeals Service Group, Performance Improvement and Procedures Service Group,
        and the Business Operations–Internal Control Division;

     b. Interviewed officials from the three School Participation Teams including the division
        directors (3) (formerly titled the area case manager), and the compliance managers (6)
        (formerly titled team leader), review specialists (17), and improvement specialists (6).
        We judgmentally selected 17 of the total of 42 review specialists to interview based
        on varying number of years in their position so that we could interview staff with a
        range of experience. We interviewed all of the improvement specialists from the
        three School Participation Teams reviewed;

     c. Reviewed the Direct Loan Program training tools provided to the schools on FSA’s
        Information for Financial Professionals Web site;

     d. Reviewed FSA’s program review selection processes;

     e. Reviewed FSA’s 2011, 2012, and 2013 “School Monitoring Risks and Mitigation
        Strategies” risk matrices;

     f. Reviewed FSA’s Direct Loan Program quality assurance requirements and guidance;

     g. Reviewed FSA’s “Program Review Procedures,” September 30, 2008;
        “Comprehensive Compliance Review Procedures,” May 3, 2007; “Guide for Review
        Conducted for 2009–2010 OIG/FSA Risk Project,” April 1, 2010; “FSA/OIG
        Distance Education Program Review Procedures,” April 10, 2010; and the “Program
        Review Distance Education Final Procedures,” July 30, 2010;

     h. Reviewed FSA’s procedures for performing reviews of Internal Quality Control;

     i. Reviewed all 47 program reviews (23 general assessment, 17 focused, and
        7 compliance assurance) conducted during our audit period by the three School
        Participation Teams. This represented 34 percent of the 137 total reviews conducted
        by all 8 of the School Participation Teams. School Participation Teams conducted the
        program reviews from July 2010 through February 2011. School Participation Teams
        conducted (started and completed) 38 reviews in 2010 and 8 in 2011; and 1 was
        begun in 2010 and completed in 2011. Thirty-seven of the schools reviewed were
        FFEL schools prior to the effective date of SAFRA and 10 schools were already
        participating in the Direct Loan Program; and

    j. Reviewed the program review files to determine whether (1) the review specialists
       followed FSA’s “Program Review Procedures,” and (2) the program review reports
       included all findings and exceptions noted during the program review. Table 5 shows
       the number of program reviews each School Participation Team conducted.
Final Report
ED-OIG/A03L0001                                                                             Page 19 of 26

                            Table 5: Reviews Conducted and Analyzed
                                     By School Participation Teams

                                      Program Reviews         Distance Education
              School Participation
                                       Conducted and          Reviews Conducted
                     Team
                                           Analyzed              and Analyzed
                   Philadelphia                17                       4
                      Dallas                   13                       0
                  San Francisco                17                       6
                     TOTAL                     47                      10
            Note: The 10 distance education reviews are included in the total of 47 program
            reviews.

          The program reviews covered award years 2008–2009 through 2010–2011, except for
          the 2 reviews noted in Table 6. Table 6 shows the number of reviews that covered each
          respective award year.

                                 Table 6: Award Years Reviewed

                              Award Year                  Number of Program
                        (July 1 through June 30)              Reviews

                        2008–2009                                     2
                        2007–2008 and 2008–2009                       1
                        2008–2009 and 2009–2010                      14
                        2009–2010                                    11
                        2009–2010 and 2010–2011                      16
                        2010–2011                                    1
                        Other13                                       2
                                TOTAL                                47

            Of the 47 reviews, School Participation Teams issued 14 program review reports in
            2010, 29 in 2011, and 1 in 2012. FSA did not issue a program review report for
            three reviews as of January 2015.

            School Participation Teams classified the finding levels for the 47 program reviews as
            follows:

                   34 reviews had moderate finding levels;
                   7 reviews had serious finding levels (no report was issued for 2 of these
                    reviews);
                   2 reviews had very serious finding levels;


13
  One program review was a follow-up review and covered award years 1998–1999 through 2005–2006; the other
program review covered three award years: 2008–2009, 2009–2010, and 2010–2011.
Final Report
ED-OIG/A03L0001                                                                     Page 20 of 26

                   3 reviews had no findings and a no finding program review report was
                    issued; and
                   1 review did not have a finding level identified, as of January 2015.

Internal Controls

As a part of our audit, we assessed the system of internal controls, policies, and procedures
applicable to program reviews for schools participating in the Title IV programs. We gained an
understanding of FSA’s internal controls used for assessing risk for the selection of schools for
program review and for performing program reviews, as well as FSA’s quality control processes.
We also reviewed FSA’s Office of Management and Budget Circular A-123, “Management’s
Responsibility for Internal Control,” Appendix A, Internal Controls Over Financial Reporting
assessments for FY 2010 through FY 2013. Because of inherent limitations, an evaluation made
for the limited purposes described above would not necessarily disclose all material weaknesses
in the internal controls. Our overall assessment disclosed significant internal control weaknesses
in FSA’s program review process. These weaknesses are fully discussed in the Audit Results
section of this report.

Data Reliability

Use of computer-processed data was limited to program review and school data obtained from
FSA’s Postsecondary Education Participants System. We used the information in this system to
determine the number of program reviews conducted during our audit period and to obtain
school and program review data, such as program review dates and award years, program review
results, and finding deficiency levels. We assessed the reliability of the data through
interviewing FSA officials knowledgeable about the data, and reviewing School Participation
Team program review files. We determined that the data were sufficiently reliable for the
objective of this audit.

We visited FSA’s offices at the following sites on the following dates: FSA offices in
Washington, DC, from February 14, 2011, through February 18, 2011; the Philadelphia School
Participation Team in Philadelphia, PA, from September 20, 2011, through February 27, 2012;
the Dallas School Participation Team in Dallas, TX, from June 4, 2012, through June 8, 2012;
and the San Francisco School Participation Team in San Francisco, CA, from August 20, 2012,
through August 28, 2012. We held an exit conference with FSA officials to discuss the results of
our audit on June 5, 2013. In addition, we performed follow-up work with FSA in March, April,
and July 2014.

We conducted this performance audit in accordance with generally accepted government
auditing standards. Those standards require that we plan and perform the audit to obtain
sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions
based on our audit objectives. We believe that the evidence obtained provides a reasonable basis
for our findings and conclusions based on our audit objectives.
Final Report
ED-OIG/A03L0001                                                                    Page 21 of 26

  Enclosure 1: Required Uses of FSA Program Review Documents
Institutional Worksheet. Reviewers were required to use the Institutional Worksheet to
document the review of a school’s information to ensure that the school met the Title IV program
requirements and had adequate policies and procedures in place. Areas requiring review
included school eligibility, a school’s admission and refund policies and procedures, and student
consumer information.

Return of Title IV Funds Policies and Procedures Worksheet. Reviewers were required to
use the Return of Title IV Funds Policies and Procedures Worksheet to document the review of a
school’s refund policy. The reviewer was required to determine whether the school’s refund
policies and procedures contained the required elements, such as how Title IV funds are treated
when a student withdraws from the school.

Program Review Report/Expedited Determination Letter Checklist. The compliance
manager and the division director were required to use the Program Review Report/Expedited
Determination Letter Checklist to document the review of the program review report or the
expedited determination letter (used for reviews that have only minor deficiencies). The
checklist also served as the compliance manager’s and the division director’s approval of the
program review report or the expedited determination letter.

Postsecondary Education Participants System Data Entry Form. Reviewers were required
to use the Postsecondary Education Participants System Data Entry Form to document pertinent
program review data in the system, such as the date of the program review, the scope of the
program review, and the level of finding deficiencies (for example, moderate or significant)
identified during the program review.
Final Repott
ED-OIG/A03LOOO I                                                                                        Page 22of26

             Enclosure 2: Federal Student Aid 's Comments 





                                                 AUG 2 0 2015
           MEMORANDUM

           DATE:

           TO: 	          Bernard Tadky
                          Regional Inspector General for Audit
                          Office of Inspector G~ncral          .
                                                 ,,.
                                           ,, . /,   ;7:7,~ . t'
                                                 ...,/~, , /	    ­     --
           FROM : 	       James W. Runcic ~
                          ChiefOperating Officer

          SUBJECT: 	 Response to Drafl Audi t Report:
                     federal Student 1\id"s Oversight of Schools
                     Participating in the Title IV Programs
                     Control No. ED-OIG/A03LOOOI

          Thank you for the oppo rtu nity to comment on the Office of Inspector General"s (O lG) draft audit
          report. Federal Student Aid"s Oversight o/Schools Participating in the Title IV Programs. dated
          July I 0. 20 l 5. Federal Student Aid (FSA) is com mined to a strong oversight policy of all Title
          IV panicipams. As yo u not.: in your drali report. we have taken a number of steps to improve
          our procedures and processes. and have continued to do so since your work concluded last year.

          It is important to note that the results of the audit did not identify significant weaknesses in the
          program review procedures themselves. but rather the files lacked documentation to confim1 that
          staff had followed the procedures. However. FSA agrees that staff failure to comply with the
          procedures· documentation requirements is a significant concern. When this finding was
          brought to our attention two years ago. FSA immediately took steps 10 tighten controls over the
          process. A senior managers· m.:cting was scheduled after the aud it exit conference to discuss the
          findings and identify corrective actions we could implement immediately. /\s a result of this
          meeting. FSA established a special quality review team to visit the regional offices to conduct
          o n-site reviews of program review work papers produced during the period whic h had been
          audited. As expected. the team noted the same missing documentation that your audit team
          found . After co mplet io n of the pilot process. in February 2014. the Prognim Review Quality
          Control Process (PRQC) was ro lled out as a pem1anent component of FSA ' s quality control
          processes to review School Participation Teams· work on at lea~! an annual basis to evaluate and
          ensure staff compliance with Program Compl iance's national standard procedures for program
          reviews. The PRQC process is monitored and modified as necessary lo ensure continuous
          process improvement and to identify staff training needs.



                                       Federal Student .\:­
                                       ..,, on ice   Cl ttl•   U.S . OCPARf M(llf of EDUCATIO N

                               830 First Street. NE. Washington. DC 20202
                                                       StudentA1d.gov
Final Report
ED-OIG/J\03LOOO I                                                                                                     Page 23of26




               FS1\ abo 1:r.:ah:d a Forms. Ch1:1:klis1s and l'm1:.:d1ir<'S ( l·Ci\ I') \\orkgruup to n:' icw and
               str.:ngth.:n the program rc\'iC\\' Jocum.:nlation process. FSA rnanag.:1111.:nt amJ st;1ff in th.:
               Wt•rkgroup ''.:re ask.:d to r.:"i.:w the current pmccdur.:s and i dcntil~ chang.:s to assist in
               consolidating thl.' v;irious worbh.:cts to crl.'al<' a m,irt• er"tki..:111 prui:ess that will tk111011~1rn11:
               dear l.'Vidl.'m.:c 01· supervisory rcvic\\'. Th<.: \\'llrkgmup's recom1111:nda1inns arc lo dc\\!lop a new
               checklist that is adaptable to the appropriate st:igc of the rc,·icw (e.g.. Program Rc\'iew Report
               (l'f{I{). Exp<.:ditccl Program Re,·iew De1crmina1ion l.c ll<: r (l~ J>I. ). or Final Program Review
               Dct.:rmin:nion ( Fl'RI))) lo ensure the process for th:i l stage of'rc\'il:W is appropriately fo llowed.
               Th..: \\'l•rkgroup also rcrnmmcnJ.:d mod ilica1io11s 111 the pr..:liminary lindings worksh.:cl to
               address the reso lution or pocenlial lindings that may or 111ay nnt become parl or the program
               rc,·icw report Jepending nn the n:solution of the item. Th..: Performanc<' and Procedures
               fmpro\'cmcnt S..:n•icc (iroup is in the process of impl.:m..:n1i11g lhc~c t11·0 rccom1m:nda1ions.

                In addition. each of the School l'anicipation Divisions h:1.; implemented additional imcrnal
               quality assurance prm:css.:s that staff :indfor complianc.: managers follow to c\ aluatc the
               completeness of r.:' icw documentation. These processes arc tailon:d lo support how each
               di"ision approaches workload assignments and includes steps lo r<·vic" the complclencss.
               organi1.ati o11 and managerial sign-off of re\'iew materials. For ex:unplc. some regions ha,·e
               cstahlishcd Quality Control (QC) proc.:sses that outline r.:' i.:w steps and rolcs 1rcsponsibilities
               for those steps among thei r team members: they have dc,..:il•pcd checklists to guide the review of
               program re\ ic" materials within their organi7.ation: or they ha\.: other re\ ie\\cr:< from the
               re,·ie" t.::un re\'io:\\ ing documcms for completeness: they have a supcr\'bor initial the date
               he sh.: rnndu.:1.:d th.: \\Ori.. paper re' icw on the Ff'R[) Re\'i.:\\ checklist: or the ):tmpling o f tiles
               for rcvic\\" condu1:t.:d by their team. Fach approach is intcnJcd t\l ensure tho: accurac~ and
               complcl<'lk'SS 11f rc,·ic'' Jocumentation. ensurim! rc\'ic\\ documents ar.: \\cl I orl!anizcd. and that
               managerial sign-o ff h;is 01:currcd.                  -                                 ­

               \\Ii.: ha\'..: responded 10 each linding ;ind rccommcndation in detail      !>ck"'·:

               FIND ING l\'O. I - FSA Did No t Conduct Proi.:rnm            f~"\' icws   in t\ccorda nc l'
               \Vith f l~ Proi,:ram Rc,·ic w Pro1:cdurcs

               FSr\ acknowlcdgl'S 1ha1 snme staff did not conduct program rc,·icws in accordanc.: with FS:\ ·s
               progr:11n r..-,·iew prn.:cdurcs that require completion or the rcq uired documcntmion.

               I lowC\'Cr. FSA docs not agree wi1h the assertions in th.: report that ··/·X·/ /im limited a.1s11rm1ce
               thl// pro}!_ra111 re1·ic!ll'.1· ure approprim(•/y ith•111if.)'i11g anti '"'J10rti11g 111/ im·tann•.\ cf
               111111co111pliw1ce '":which calls inw 1hc qm:siion the vn lidi ty or the estimates used in the
               calculation of the D.:p:irlmcnt"s improper paymcms. It i~ unrco~on:ihfc that FS/\ or an~
               organi:1ation can ensure that all instances (If noncompliance arc idc11111iecl. 111 fact. it is standard
               practico: 1i.1r an O\"ersight :igcncy to disclo;c in its report that although a r.:,·ic\\ i!> thorough. it
               Joo:~ mll cli!im 10 be all incfusi,·c. I IO\\C\t:r. th~· lin<li ng~ :md h.1hilitics idc11111icd during the
               review:; conducted during the period in qu<·stion clearly dc11111nstra1e th.nth.: signilicant instances
Fina l Repo11
ED-OIG/A03 LOOOI                                                                                              Page 24of26




           of noncompliance were idcmilicd. For liscal year 20 11. F!:);\ assessed $65 million i11 liabilities
           stemming from 315 progr:un rc\"icws. In contr:1st. for the sa me fiscal yc:;1r. $3-1 mi ll ion in
           liabili1ics were assessed from the 1.3 77 audits conducted by independent accountants. The
           signilicant differences in the amount or liabilities asscsseJ. n: l:uin: lo th e number of inst itutions
           revic\\'cd. clearly demo nstrates that FSA was not only selecti ng the highest ri sk institut ions 10
           revi<.:\\'. bu t was also identifying s ignificant instances or no11-1.:0111plia111.:e.

           Recnm111cndalio11 I. I: R e\' isc th e '' Proi.:rnm Rev iew P rocedures" lo r equire d ocumen tation
           o f s upe rvisory review of the p rog ra m review fik

           Rcspo nsl·: We agree :rnd the new l'RQC process requires ,·alidation that the supervisory review
           of spcei lie <locumcms has oceum:<l.

           Rccomml'11dati o11 1.2: E ns ure r c\'iew s pcc ia lis ls l"umplc11: and d ocument :111 as pects o f the
           proi.:ra111 revie w process.

           R es ponse: We agn:e :md based on recommendations from the FC.-\P workgroup. FS;\ has
           alre<i<ly revised many of the worksheets ;md documents usl.'d during a program re\ icw to ensure
           re\ ic'' specialists arc carefolly c.lonnrn:nting all as1x.~ts o f the process. Program rc\·iewcrs were
           pro,·idcd trnining on the completion of those documents. In addition. th.: PRQC process includes
           vali<latiun that thl· review specialists complete and documem the specific pl>ints in the program
           re,·icw process.

          Recomm e ndatio n 1.3 : Ensun: rc\'icw spccialbls arc rc\'il.'\\ inl! aud d oc um cn ling lh c rc\'icw
          o f a ~c hu o l's Oirccl Loan <1uality a ~s urancc sys rcm .

          Response: The l'RQC procedures requi re testing ora schoors Direct l.oan lJUality assurance
          sys ti:m. Verification  or
                                 this process is being rcvie\\Cd as p;1rt     or
                                                                          the l'RQC process as of
          Fcbruar) 20 14.

          Recommendation IA: Require Schoo l Parlicipatinn T ea m m anagemen t lo take corrective
          act ion o n the reco mm e ndatio11 s m ade in th e PRQ C rc\' icw re ports.

          R es ponse: We agree that S1:hool l'ani1.:ipation Ti:am management s hou ld be required to take
          corrective actions as warranted. The management of the School Pnrticipatillll Team has already
          been taking corrective !lctions as warranted. but they h:id not been properly documented. T he
          PRQC process will be updated to incorporntc vali<fation an<l documentation that co1Tcctivc
          actions have heen taken.

          Reco mmendation 1.5: Reassess whclhrr t he c urren t ro ta I rim e allollcd 10 prrfo rm a
          pro:,: rarn review is adcqualc to compll'l c and d ol: umcnl all n ·quircd prot•edurcs.
Final Report
ED-O IG/A03 LOOOI                                                                                                          Page 25of26




       f'   ~ .·   .:


                        Res ponse: FS;\ will n.:asses:; whL'ther the cu1Ten1 total time allolted 10 perform :i program
                        review is adequate to complete and document a ll required procedures. I lowcvcr. with 200 staff
                        members responsib le l(>r overview of more than 6.000 instiwtions. resource co nstr:iims wil l
                        come into play in these deci s ions. For i n~tance. ;ill o1ting morL' lime 10 .:;1ch progrnm review. due
                        lo these pcrsonnd t:unstrain ts. will result in fi:wer rev iews being performed.

                        F I N DI NG NO. 2 - FSA Was Not C onsiderin g Annu:tl Dro po ut R ate Data F o r
                        Prog ram Review S elections

                    \Ve di sagree wi th thi s linding. f.S/\ did use ;mnual dropout rate dat:i. as preseribcd in the stat ute.
                    in selecti ng schools for program reviews. Section 498/\ of the 1li gher Educ:ition i\ct of 1965, :is
                    arnc ndL'd. states that insti tutions with a hi gh annual d ropout rate shall be give n priority for
                    progr:.im reviews. The statu te docs no1 provide a definition no r docs it describe how to calculate
                    such a rate. including whether the rate s hou ld im:lude all students or on ly Ti tle IV recipients.
                    The statute only states that we "s hall g ive priorit y for program review to institu tions or higher
                    educnt ion that arc institulions wit h high a nnual dropout rates". r'SI\ detem1ined rhal calculating
                    1he dropou t rate for Title IV recipients meets the s ta tutory sta nd ard. i\ dropout rate wu ld be
                    calculated in a number o r di ffe re nt ways. for example. by eom pnring s tudents who stort to those
                    w ho complete. FSi\ chose to look ;it the number or dropouts vers us gr:itluutcs.

                    For 1his e;1ku la1ion, r'S;\ used enrollment status data reported 10 the National Swdc111 Loan Datu
                    System (NS! .DS) to cnlc ulatc the percentage o f s tudents who withdrew during an awanl year o ut
                    of a ll s1t1dcn ts who left school during the same awa rd year. For exa mple. i I 25 students
                    wi thdrew and 75 s tudents grad u;1ted. the dropout r:ite wou ld be 251 (75 r25) or 25%. FS.'\
                    perfonn s this e:i leu!a tion nnnu;il ly and considers li ve years o r' data to look for I rends.

                   FS.'\ also uses data from thL' lntcgr;itetl l'os rseeondary Educa1ion Data Sys11::m (IPEDS) 10
                   \':Jlidatc the resu lts or1hc dropou1 cakul;11io ns using NSLDS data. For most sc hools. lhc inverse
                   of t he complet io n ralc or a "did not complct~·" ra te. which is calcul::Hcd by considering w hether a
                   student earned a crcdc.:n ti:i l w ith in 150% time. c losdy tr;11:ks thc dropou1 rn tc.: calcula ted from the
                   NSl. DS described above. FSA comp:ircs th..:sc two ra tcs and if they d iffc:r s igni licantly further
                   analysis is co nducted.

                    Rccommend:llion 2.1: Co ns ult w ith th e Na tional Cent e r for Educat ion Stalis tics regarding
                    th e feas ibili ty of co ll ect in g anti c:1lcu latin g a nnual <l rnpout r:ll e~ fo r- sc hools and use the
                    rates as a factor to prioriti:t.c schoo ls fur program reviews.

                   Res ponse: As described abo,·c. FS/\ alrcady has a rrn:thodology for calculating the ;111nual
                   dropout rate which has been used in prioritizing sc hoo ls for program reviews per th e sta tu tory
                   rcquirc111en1. I lowevcr. FSA agrees to consult with National Center for Educati on Statistics
                   (NCl~S) o n th is issue. With regard 10 the feasibi lity ofNCES coll ec ting and calcula ting annuul
                   Jropout rates for schoo ls. wc.: bcl ie,·c that recon 1111cnda1ion should be direl: tcd tu NCES. nu t FSA.
                   lfNCES is able add a dropout r:11e to their I PEDS dat:i col lection. as the OIG suggests. FS/\ will
Final Report
ED-OIG/A03 LOOOI                                                                                         Page 26 of26




           consider using 1hose rates as a fac1or 10 prioritize schools for program reviews. per 1hc stallllorr
           rcquirc111cn1.

           Than!,. ) 011 again for lh..: opportunily 10 rcvi..:w and n.:spond lo 1hi$ report