oversight

Payback Provisions of the Rehabilitation Long-Term Training Program

Published by the Department of Education, Office of Inspector General on 2014-04-25.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                                       UNITED STATES DEPARTMENT OF EDUCATION
                                             OFFICE OF INSPECTOR GENERAL




                                                                                                                  Control Number
                                                                                                               ED-OIG/A19M0004
                                        April 25, 2014
Michael K. Yudin
Acting Assistant Secretary
Office of Special Education and Rehabilitative Services
U.S. Department of Education
400 Maryland Avenue, S.W.
Washington, DC 20202-4300

Dear Mr. Yudin:

This final audit report, titled Payback Provisions of the Rehabilitation Long-Term Training
Program, presents the results of our audit. The objectives of our audit were to determine
whether the Rehabilitation Long-Term Training (RLTT) program effectively (1) met program
objectives by training recipients who subsequently performed work related to the program, and
(2) obtained repayment of the assistance received for recipients that did not fulfill work
agreements. This audit was part of a review of payback provisions of selected grant programs
throughout the U.S. Department of Education (Department).



                                                      BACKGROUND


The Rehabilitation Services Administration (RSA) administers the RLTT program, one of a
number of grant programs authorized under the Rehabilitation Act of 1973 (Act), as amended.
The purpose of the RLTT program is to support projects that provide training and related
activities, including the provision of technical assistance, to increase the numbers of qualified
personnel trained in providing rehabilitation and other services provided by the Act for
individuals with disabilities. Congress has appropriated approximately $35-40 million for
training programs in each of the past few years, with the 5-year grants under the RLTT program
generally averaging $100,000 per grantee, per year.

The RLTT program regulations require that 75 percent of funds awarded to universities go
directly to students for tuition assistance and stipends. Students who receive financial assistance
from projects funded under this program, often referred to as “RSA scholars,” are required to pay
back such assistance either by maintaining acceptable employment in public or private nonprofit
rehabilitation agencies for a period of time after they complete their training, or by making a
cash repayment to the Federal government. Upon completing their respective program, scholars


 The Department of Education's mission is to promote student achievement and preparation for global competitiveness by fostering educational
                                                   excellence and ensuring equal access.
Final Audit Report
ED-OIG/A19M0004                                                                                       Page 2 of 22
who receive funding under the RLTT program must work 2 years for every 1 year of funding
received. Work payback must be completed within the sum of the number of years owed plus
2 additional years, often referred to as the grace period. Scholars are required to repay all or part
of any scholarship they receive if they do not complete their service obligation. The RLTT
program regulations do, however, provide for specific circumstances under which the Secretary
of Education (Secretary) may grant a deferral or exception to performance or repayment under a
scholarship agreement. 1

Grantees are required to fully inform students about their payback obligations and other
requirements before disbursing scholarship funds and students must sign a written payback
agreement before they receive funds. Grantees are also required to track current and former RSA
scholars and maintain accurate and complete information on them from the time they are
enrolled in the program until they successfully meet their payback requirements. To that end,
grantees must report annually on each scholar’s payback status via RSA’s web-based
Management Information System (MIS). 2 Program officers are responsible for reviewing and
approving these payback reports, which may entail following up with grantees if any issues are
noted. Information in these reports is used to ensure that grantees and scholars are in compliance
with the RLTT program requirements and to report on performance measures established under
the Government Performance and Results Act (GPRA). Scholars who are found not to be
complying with the RLTT program requirements are referred for repayment to the Department’s
Debt and Payment Management Group (DPMG).

RSA awarded a total of 11 RLTT grants in fiscal year (FY) 2006, the year of focus for our
review. We chose to focus on FY 2006 grants as they were more likely to have a larger number
of scholars that had completed the training and were in the payback portion of the program than
more current grants, thereby enabling us to obtain sufficient evidence to answer our audit
objectives. We noted that the number of scholars served under the five FY 2006 grants included
in our sample ranged from 13 to 60, with a total of 173 scholars having received funding under
these grants. Financial assistance provided over the course of scholars’ participation in these
projects ranged from $158 to $42,292, with a total of $1.6 million in financial assistance awarded
to the scholars served under these grants.




1
  Title 34 of the Code of Federal Regulations (C.F.R.) § 386.41 provides that the Secretary may grant a deferral or
exception to performance or repayment under the following circumstances: permanent disability, death, further
academic study, active duty in the military, service in the Peace Corps or as a Domestic Volunteer, temporary
disability, and inability to secure employment because of care for a disabled spouse.
2
  Among the data elements that grantees are required to report on are each scholar’s name, start date, exit date,
reason for exit, grace period end date, complete-by date, employment status, and financial assistance received.
Final Audit Report
ED-OIG/A19M0004                                                                         Page 3 of 22


                                      AUDIT RESULTS


We found that RSA appears to have met RLTT program objectives by training recipients who
subsequently performed work related to the program. Specifically, our audit results indicate that
the majority of exited RLTT scholars who received training under the grants in our sample are
working in acceptable employment. However, we are concerned about the data quality with
regard to grantee reporting. We also found that, although RSA has recently undertaken efforts
designed to strengthen its monitoring process, further improvements are needed in the process
for identifying and referring noncompliant scholars for financial repayment.

With regard to objective one, we found that available data show the majority of scholars who
received training under the five FY 2006 RLTT program grants in our sample obtained
acceptable employment. Specifically, we determined that 119 of the 158 (75 percent) scholars
who graduated or otherwise exited their respective program have either completed or are working
toward completing their service obligation. However, we identified data quality issues which
call into question the validity of these results. Specifically, we noted that payback reports that all
five grantees in our sample submitted to the Department did not always include all of the
requested data and, based on some reasonableness tests we performed, contained discrepancies.
Because source data for information contained in the payback reports is primarily located at
various grantees across the country, our ability to perform an assessment of the reliability of the
information in the reports was limited, and as such, we could not fully determine the reliability of
the data.

We also found that, although valid as one indicator of success, the GPRA performance measure
under which RSA reports scholar employment data in the RLTT program’s annual Congressional
budget justifications does not reflect fully the program’s effectiveness in training recipients who
work in fields providing rehabilitation and other services to individuals with disabilities.
Specifically, RSA’s calculation includes graduates only, even though non-graduates are also
permitted to work in acceptable employment to fulfill their payback requirement. Additionally,
the measure does not include scholars who have completed their service obligation in previous
years. Further, because RSA’s reporting on GPRA measures relies on the same data upon which
we based our analysis of a sample of FY 2006 grants, there are similar concerns regarding the
validity of these results. Grantees’ failure to complete payback reports or provide accurate and
consistent data, coupled with RSA’s failure to adequately and appropriately review these reports
and other relevant information, limits stakeholders’ ability to draw reliable conclusions on
program effectiveness.

With regard to objective two, we found that RSA does not appropriately identify and refer for
financial repayment scholars who are not fulfilling their service obligation. We identified 31 out
of 106 scholars (29 percent) who should have started working in acceptable employment who
were not on track to complete their service obligation within the number of years required, based
on information contained in grantees’ payback reports and additional information subsequently
provided by project officers from grant project directors. We noted that RSA staff generally did
not identify these scholars as not complying with the program requirements, and as a result, did
Final Audit Report
ED-OIG/A19M0004                                                                        Page 4 of 22
not follow up with grantees for the purpose of determining whether they should be referred to
DPMG. We also noted that program officers did not always timely refer to DPMG scholars who
should have completed their service obligation.

Overall, the weaknesses noted with the identification and referral of scholars for repayment
increase the susceptibility of the RLTT program to fraud, waste, and abuse. RSA’s failure to
appropriately identify scholars who are not on track to fulfill their service obligation increases
the risk that the Department will not timely recover funds owed to the Federal government
according to the agreements which these scholars signed upon starting their respective programs.
We noted that the 31 scholars who should have been identified as not on track to complete their
service obligation and therefore should have been placed in repayment status received
approximately $280,000 in Federal funds. Also, because RSA does not refer scholars for
repayment until their period of obligation has expired, the accrued interest owed to the Federal
government will be less than what it should have been had RSA, consistent with the regulations,
correctly identified the date the scholar entered into repayment status and timely initiated debt
collection activities.

Lastly, during our discussions with program officers, we learned that RSA may, on occasion,
grant scholars extensions to the period within which they must complete their service obligation.
RSA did not indicate that the extensions in question were related to the deferrals or exceptions
that the regulations authorize, but, rather, used in such a way as to not penalize working scholars
who appear to be able to fulfill their service obligation within a “reasonable amount of time”
from their initial completion date. We subsequently learned from the Office of General Counsel
(OGC) that RSA granted such an extension on at least one occasion. We noted that, in
implementing the current RLTT program regulations, the Department did not appear to view the
period of obligation as flexible, beyond allowing for the granting of deferrals or exceptions under
limited circumstances. As a result, it does not appear that the Department has the authority to
unilaterally extend scholars’ completion dates, nor the ability to work within the current
regulations to allow for such determinations.

In its response to the draft audit report, RSA did not state whether it agreed or disagreed with the
findings, but concurred with each of the 15 recommendations and noted that it will address them
in the months ahead. RSA stated that it has been and will continue to be its goal to ensure that
the information collected and reported for the RLTT program is accurate and that any monies
due to the Department are collected in a timely manner. RSA further stated that it has made and
continues to make significant strides in its collection of data and delivery of technical assistance
to grantees.

RSA’s comments are summarized at the end of each applicable finding. The full text of RSA’s
response is included as Attachment 2 to this report. No changes were made to the report as a
result of the response.
Final Audit Report
ED-OIG/A19M0004                                                                                                               Page 5 of 22
FINDING NO. 1 – Results Indicate that the Majority of Exited Scholars are
                Working, but Data Quality is a Concern

We found that the majority of scholars who received training under the five FY 2006 RLTT
program grants in our sample obtained acceptable employment, based on information reported to
the Department by grantees. Specifically, we determined that 119 of the 158 3 (75 percent)
scholars who graduated or otherwise exited 4 their respective program have either completed or
are working toward completing their service obligation. [See Table 1 for information on the
number and status of all scholars served under the grants in our sample.]

                       Table 1: Number and Status of All Scholars Served Under a
                                   Sample of FY 2006 RLTT Grants
        Total Scholars                                                                                   173
          Current Scholars                                                                                    15              9%
          Exited Scholars*                                                                                   158             91%
              Scholars Who Fulfilled Service Obligation or
                                                                                                                 119             75%
              Are Currently Working in Acceptable Employment**
              Scholars Who Are Not Working in Acceptable
                                                                                                                   22            14%
              Employment ***
              Scholars Who Paid Back Financial Assistance Received or
                                                                                                                     7                4%
              Have Been Referred to DPMG
              Scholars for Whom Employment Status Could Not Be Determined                                            6                4%
              Scholars Who Are in Process of Receiving a Deferral or Exception                                       4                3%
        * Of these 158 scholars, 143 (91 percent) graduated from their respective program and 15 (9 percent) withdrew or were
        expelled.
        ** Of these 119 scholars, 51 (43 percent) have already fulfilled their service obligation and 68 (57 percent) are currently
        working in acceptable employment. Also, this number includes some scholars who are not on track to fulfill their service
        obligation within the required timeframe—a situation addressed in Table 2.
        *** This number includes some scholars who are still in their 2-year grace period and, therefore, not yet required to have
        begun working in acceptable employment.


However, we identified data quality issues which call into question the accuracy of these results.
To perform our analysis, we relied primarily on grantee-reported data found in the RSA MIS
payback reports. These payback reports are the basis upon which RSA determines results on its
RLTT GPRA performance measures, for both internal and external reporting purposes. Because
source data for information contained in these reports is primarily located at various grantees
across the country, our ability to perform an assessment of the reliability of the information
contained in the reports was limited and, as such, we could not fully determine the reliability of
the data. We noted, however, that payback reports submitted by all five grantees in our sample
did not always include all requested data and, based on some reasonableness tests we performed,
contained discrepancies, including the following:

    •    the date that a scholar exited a program occurred before the date on which the same
         scholar allegedly started the program;
    •    the number of work years owed by a scholar did not appear to coincide with the scholar’s
         start and exit dates from the program;
3
  We identified a total of 173 scholars who received financial assistance under the 5 RLTT grants in our sample;
however, 15 of these scholars are still enrolled in their respective program.
4
  Of these 119 scholars, 115 (97 percent) graduated from their respective program and 4 (3 percent) withdrew (for
reasons known or unknown).
Final Audit Report
ED-OIG/A19M0004                                                                                      Page 6 of 22
    •   the total amount of service reported as having been completed by a scholar in one year
        decreased in the following year;
    •   the total amount of service reported as having been completed by a scholar in one year
        increased by more than one year in the following year;
    •   the total amount of service reported as having been completed by a scholar increased
        between years even though the scholar was twice identified as working in unqualified
        employment; and
    •   total scholarship support received by a scholar did not coincide with the scholarship
        amounts reported in previous years.

When we identified discrepancies such as those listed above, we followed up with program
officers to determine if there were explanations for the errors or if the information was
potentially incorrect. During the course of this followup, we became aware of several instances
in which a scholar’s start and/or exit dates and other data were, in fact, incorrect. Based on data
located in the payback reports alone, we initially found that 98 of 158 (62 percent) scholars who
graduated or otherwise exited their respective program had either completed or were working
toward completion of their service obligation. However, after following up with program
officers, this number increased to 119 of 158 (75 percent) scholars—a variance of 13 percentage
points, albeit positive, within the five FY 2006 grants in our sample alone.

We also reviewed other available information on program performance as part of our assessment
of the RLTT program’s effectiveness in meeting its objectives. Specifically, we noted that RSA
reports overall scholar employment data in the RLTT program’s annual Congressional budget
justifications, which require discussion of each Department program’s performance on measures
established under GPRA. Among these measures is the percent of RSA-supported graduates
fulfilling their payback requirement through acceptable employment in any given year, which in
FY 2011, the most recent year for which data is available, was 77 percent. 5 RSA’s target for
FY 2011 was 87 percent. However, because RSA’s reporting on GPRA measures relies on the
same data upon which we based our analysis of a sample of FY 2006 grants, there are similar
concerns regarding the accuracy of these results.

We further noted that this particular measure does not reflect fully the RLTT program’s
effectiveness in training recipients who subsequently perform work related to the program.
Specifically, RSA’s calculation includes graduates only, even though non-graduates (i.e.,
scholars who withdrew or were expelled from their respective program) are permitted to work in
acceptable employment to fulfill their payback requirement according to RSA’s nonregulatory
guidance. This measure also does not include scholars who have completed their service
obligation in previous years.




5
  The FY 2014 RLTT Budget Justification states that 83 percent of RSA-supported graduates were fulfilling their
payback requirement through acceptable employment in FY 2011, while data on the RSA website show a success
rate of 77 percent on the same measure. RSA management attributed this error to management oversight, with
83 percent instead representing the percent of Masters-level graduates fulfilling their payback requirement through
acceptable employment in FY 2011, another more specific measure by which RSA assesses program effectiveness.
Final Audit Report
ED-OIG/A19M0004                                                                       Page 7 of 22
Although the measure does reflect the performance of an important subset of scholars (i.e.,
graduates who are currently working toward fulfilling their service obligation) it is not
comprehensive because it does not reflect all scholars who are working or who have worked in
the area of rehabilitative services. To provide a more comprehensive view of the program’s
effectiveness, RSA could include additional measures that report on the percent of all RSA-
supported scholars fulfilling their payback requirement through acceptable employment in any
given year, and/or the percent of all RSA-supported scholars who have completed their service
obligation at any time.

Office of Management and Budget (OMB) Circular A-123, “Management’s Responsibility for
Internal Control,” states

       Management has a fundamental responsibility to develop and maintain effective internal
       control. The proper stewardship of Federal resources is an essential responsibility of
       agency managers and staff. Federal employees must ensure that Federal programs
       operate and Federal resources are used efficiently and effectively to achieve desired
       objectives. Programs must operate and resources must be used consistent with agency
       missions, in compliance with laws and regulations, and with minimal potential for waste,
       fraud, and mismanagement.

Section 13(b) of the Act, as amended (29 United States Code (U.S.C.) § 710(b)) requires that the
Commissioner of RSA collect information to determine whether the purposes of the Act are
being met and to assess the performance of programs carried out under the Act. Section
302(b)(5)(B) of the Act, as amended (29 U.S.C. § 772(b)(5)(B)) states that the Commissioner is
responsible for the enforcement of scholars’ service agreements.

Under 34 C.F.R. § 386.34(g), grantees are required to establish policies and procedures to
determine scholars’ compliance with the terms of their service agreement. Under 34 C.F.R.
§ 386.34(h), grantees are required to make reports to the Secretary that are necessary to carry out
the Secretary's functions under the program.

The Department’s “Handbook for the Discretionary Grant Process” (OS-01, dated
January 26, 2009) (Handbook) states that monitoring shall continue for as long as the
Department retains a residual financial interest in the project, whether or not it is providing
active grant support. Specifically, staff are to monitor each grantee to the extent appropriate so
as to achieve expected results under approved performance measures, while assuring compliance
with grant requirements.

Reasons for Data Quality Concerns

Our concerns with regard to the quality of data used to report on the effectiveness of the RLTT
program are attributable to (1) the overall design of the program, which places responsibility for
tracking individual scholars with grantees and relies in large part on self-reported data; and
(2) RSA’s monitoring policies, procedures, and processes, particularly with regard to
determining the reliability of such data. Specifically, we noted that although the RLTT program
is not unique in relying on grantee data for the purpose of determining progress toward meeting
Final Audit Report
ED-OIG/A19M0004                                                                         Page 8 of 22
GPRA performance measure targets, the program represents a distinct risk to the Federal
government in that scholars—many of whom received significant sums of money—must
continue to be tracked by grantees once they have exited their respective programs. We also
noted that RSA’s oversight efforts in this area do not provide assurance that steps are being taken
to validate grantee-reported data.

Scholar Tracking

We found that grantees did not appear to begin to systematically collect payback agreements and
addresses for scholars, nor provide RSA with certain other required information that is essential
to determining whether service obligations are being appropriately fulfilled, until the 2010
payback reports. We also found an independent evaluation of the RLTT program and related
third-party issue briefs that noted similar issues with program design and data quality. For
example, a 2003 independent evaluation of the RLTT program concluded that, “the data quality
problem encountered during data analysis is a reflection of the systemic problem of relying on
grantees to track RSA scholars over a period of time.” This report recommended that RSA,
“Delegate the tracking of the payback status of the exited RSA scholars to a third party that
specializes in tracking instead of relying on faculty member grantees who are not trained to
perform such duties.” An RSA-requested issue brief from 2011 found that there were a number
of significant gaps in the data grantees provided in their payback reports, including information
on work years owed and work years completed.

We noted that since 2010, RSA has used listserv emails, webinars, and project director meetings
to provide guidance to grantees on how to complete payback reports, offer technical assistance,
and answer frequently asked questions. Data quality, however, remains an issue.

Payback Monitoring Policies and Procedures

We noted that RSA lacks adequate policies and procedures governing program officers’ review
of payback reports. During our discussions, we learned that RSA has not established guidance
specific to this process nor communicated expectations. Program officers differed in their
understanding of what they should be looking for when they conduct their reviews and noted that
there is no defined timeline by which they must complete their initial review and initiate
followup with the grantee regarding any data quality or other issues. It is also unclear as to
whether program officers should review past payback reports for consistency, since we noted
that some do and others do not.

When we discussed this matter with RSA management and program officers, we learned that
they feel it is ultimately the grantee’s responsibility to ensure that data are reliable and correct—
an understanding that is reflected in the program regulations. However, even though the
regulations do place responsibility for tracking scholars on the grantees, they do not relieve RSA
of its monitoring responsibilities. We noted, for example, that several of the payback reports that
we reviewed were never formally approved by a program officer—and, in some cases, were
never even entered into MIS by the grantee. Specifically, of the 32 reports that we reviewed,
only 9 (28 percent) were completed by the grantee and approved by the assigned program
Final Audit Report
ED-OIG/A19M0004                                                                                      Page 9 of 22
officer. We found that 3 (9 percent) were never entered by the grantee 6 and 20 (63 percent) were
completed by the grantee but not approved by the assigned program officer. We further noted
that although MIS requires a “signature” from the individual submitting the payback report, it
does not include a statement pertaining to or certifying data quality.

Reviewing these payback reports would have allowed RSA to identify data quality and other
issues and provided some assurance that the information was accurate and complete for the
purpose of measuring program effectiveness. Additionally, requiring that RLTT grantees certify
that data submitted to the Department is accurate, reliable, and complete would provide an
additional level of internal control over the reliability of performance data.

During the course of our audit, we were provided with a suggested draft protocol dated
April 2013 for analyzing payback reports. This protocol was not in effect during the scope of
our audit, so we could not assess its effectiveness. Nevertheless, it does provide tips for review
that may be of value to program officers. Specifically, the suggested protocol advises staff to
ensure that information was input correctly (i.e., scholar name, start date, exit date, reason for
exit, grace period end date, complete-by date, employment status, financial assistance received,
etc.); thoroughly review all sections of the report prior to approval (and consider reviewing the
prior year’s report for consistency); and follow up with grantees if there are questions regarding
the accuracy or completeness of the data or if the grantee indicated that information on waiver or
deferral requests or repayment documentation would be forthcoming. However, RSA has not yet
incorporated this protocol into its formal monitoring policies and procedures, nor had the
opportunity to test its usefulness. We further noted that the MIS has only recently been updated
to include logic checks that should assist program officers in identifying some of the
discrepancies listed above. [See Finding No. 2 for additional details.]

We also noted that the Department does not obtain or otherwise verify evidence of scholars’
fulfillment of their service obligation. RSA management noted that, when funding allows,
program officers will conduct site visits that include reviewing scholar case files to ensure that
all required documentation is maintained. According to program officers, required
documentation, at a minimum, includes the signed payback agreement, exit certification, and
proof of citizenship. Other documentation that may be part of the scholar case file includes
financial disbursement information and any communications between the scholar and the
grantee. If a scholar has exited the program, program officers will look for evidence that the
grantee has obtained employment verifications. If a grantee refers a scholar to RSA for
repayment, deferral, or exemption, such documentation, along with the final decision letter from
RSA, also should be included in the scholar’s case file.

Although the process described above would provide better assurance that scholars are working,
we noted that the number of site visits conducted by RSA has been reduced dramatically in
recent years—a situation that RSA management and program officers attributed to budget
constraints. For example, we learned that site visits were conducted at seven universities in


6
 Data from payback reports carry over from year-to-year. Therefore, if a grantee submitted a payback report in
2008, but failed to submit one in 2009, a program officer would be able to log in to the MIS and view a 2009 report,
but not actually be seeing any new data.
Final Audit Report
ED-OIG/A19M0004                                                                       Page 10 of 22
FY 2011, at zero universities in FY 2012, and at three universities in FY 2013. We also noted
that only one of the five grantees in our sample received a site visit, in October 2008. This visit
occurred just 2 years after the grant was awarded and, although useful from a program
implementation and data quality perspective, did not allow for any substantial review of
documentation concerning work payback. This was due in part to the 2-year grace period
prescribed by the RLTT program regulations, whereby even if a scholar had completed the
program at the time of the site visit, the scholar would not yet have been required to be in
acceptable employment.

With regard to the concerns noted with the related performance measure, RSA itself has
recognized weaknesses in this area, noting in the FY 2014 RLTT Budget Justification that the
GPRA performance measures alone do not provide a comprehensive view of the RLTT program.
However, officials believe that the measures do provide some evidence as to the efficacy of the
program and its expenditures.

Grantees’ failure to complete payback reports or provide accurate and consistent data, coupled
with RSA’s failure to adequately and appropriately review these reports and other relevant
information, limits stakeholders’ ability to draw reliable conclusions on program effectiveness.
In addition, RSA is not currently reporting the most comprehensive data on program outcomes
and impacts because RLTT scholars who withdrew or were expelled from their respective
program or who have completed their payback obligation in previous years—either through
service or financial repayment—are not taken into account when determining results on GPRA
performance measures. Because these measures are based only on current fiscal year data, any
assessment of the program’s effectiveness is only relevant for that specific point in time.

Recommendations

We recommend that the Assistant Secretary for the Office of Special Education and
Rehabilitative Services (OSERS) require RSA to

1.1    Continue to emphasize to grantees, through training or by other means, the need to
       provide accurate and complete data for the purpose of determining total financial
       assistance received and work years owed by each scholar, and the regulatory requirement
       that grantees continue to track scholars once they have exited the program and provide
       information annually on work years completed or requests for waivers or deferrals.

1.2    Consider requiring RLTT grantees to provide certifications that data submitted to the
       Department is accurate, reliable, and complete in order to provide an additional level of
       internal control over the reliability of performance data.

1.3    Identify grantees who consistently fail to provide accurate and complete scholar data and
       take appropriate enforcement actions.

1.4    Finalize the protocol for program officers to reference when completing their reviews of
       payback reports and incorporate it into RLTT’s formal monitoring policies and
       procedures.
Final Audit Report
ED-OIG/A19M0004                                                                      Page 11 of 22
1.5    Provide training to program officers on the payback report review process.

1.6    Establish and implement timelines for the initial review of payback reports, followup
       with grantees, and formal approval in the MIS.

1.7    Ensure that key data quality/logic checks are built into the MIS.

1.8    Consider incorporating other means by which program officers can verify the adequacy
       of grantees’ tracking systems and scholars’ employment in the absence of adequate
       funding for site visits, to include periodic desk audits of grantees regardless of whether
       they have been selected for on-site monitoring.

1.9    Consider including and reporting on additional performance measures that would reflect
       a more comprehensive view of program effectiveness.

RSA Comments

RSA concurred with the recommendations and described actions that it will take to improve data
quality for the RLTT program. RSA placed particular emphasis on revisions being made to its
payback policy and procedures manual, which is currently undergoing internal review and
clearance. RSA stated that, in the updated manual, it will emphasize to grantees the need to
provide accurate and complete data, formally incorporate the elements of program officer
training on reviewing and addressing discrepancies and other errors in grantee payback reports,
and incorporate established timelines and procedures. The updated manual will be made
available online and a copy will be provided to all grantees. RSA noted that it will also explore
strategies that would enable it to conduct quality assurance checks in MIS to ensure that both
RSA staff and grantees are adhering to the timelines established. In addition, RSA will work
with the MIS technical lead to develop a way for grantees to submit a certification of data quality
and to develop additional quality/logic checks for the payback reports in MIS. RSA further
stated that it will continue to make these types of enhancements annually or as the need arises.

RSA’s described actions also include increased communication, both internally and externally.
Within the Department, RSA noted that it recently held discussions with Budget Service about
revising GPRA measures and established a small work group to review and refine the measures.
It is RSA’s hope to have new measures in place in the fall of 2014 for inclusion in the FY 2016
Congressional Budget Justification. Externally, RSA will develop communication from RSA
management to grantees that are not reporting accurate and complete data as a means to make
clear to grantees that accurate and complete reporting is a legal requirement of the award. Such
communications from RSA management will then become part of the official grant file. If a
grantee consistently fails to comply with the tracking system, documentation, reports, and
records requirements, RSA will, after warning, terminate the award.
Final Audit Report
ED-OIG/A19M0004                                                                                   Page 12 of 22
FINDING NO. 2 – Improvements are Needed in the Department’s Process for
                Identifying and Referring Scholars for Financial
                Repayment

We found that RSA does not appropriately identify and refer for financial repayment scholars
who are not fulfilling their service obligation. Upon completing their respective programs,
scholars who receive funding under the RLTT program must work 2 years for every 1 year of
funding received. Work payback must be completed within the sum of the number of years
owed plus 2 additional years, the latter of which is often referred to as the grace period. 7 We
determined that program officers should have identified 45 scholars within the five FY 2006
RLTT grants in our sample as not being on track to complete their service obligation within the
number of years required, based on information contained in grantees’ payback reports, and
should have subsequently followed up with grantees for the purpose of determining whether
these scholars should be referred to DPMG. However, we noted that RSA staff generally did not
identify these scholars as not complying with program requirements and, as a result, did not
conduct followup activities. We also noted that program officers’ unwritten practice of not
identifying scholars for repayment until after their complete-by date had passed did not always
result in the timely referral to DPMG of scholars who should have completed their service
obligation on or before September 30, 2012, the end date for the period covered by the most
recent payback report as of the start of our audit. 8

Identification of Scholars for Repayment

As noted above, based on data contained in the most recent RSA MIS payback report for each
grant in our sample, we identified 45 scholars as not on track to complete their service obligation
within the number of years required. This would mean that they should be in repayment status
per the program regulations. We arrived at this number by determining how many work years
each scholar had left to complete their service obligation, and comparing work years owed to the
amount of time remaining between September 30, 2012, 9 and each scholar’s complete-by date.

We followed up with program officers and subsequently determined, through other supporting
documentation they acquired from the grantees, that 14 of these scholars were actually in
compliance with program requirements and, therefore, should not be in repayment status.
Specifically, we learned that six scholars had actually completed their service obligation, six
scholars were on track to complete their service obligation, one scholar was eligible to receive a
deferral, and one scholar was eligible to receive a waiver. Therefore, in total, we identified 31
scholars as not being on track to complete their service obligation within the number of years
required. [See Table 2 for information on the number and status of exited scholars served under
the grants in our sample.]




7
  For example, if a scholar began a program in 2006, received 2 years of funding, and completed the program in
2008, they would have until 2014 to complete their service obligation.
8
  November 2012
9
  Grantees’ most recent payback reports were supposed to reflect information through the end of FY 2012.
Final Audit Report
ED-OIG/A19M0004                                                                                                                Page 13 of 22

                    Table 2: Exited Scholars’ Status in Fulfilling Work Requirements
          Exited Scholars                                                                                  158
             Exited Scholars Who Were Still Within 2-Year Grace Period as of
                                                                                                                 52             33%
             September 30, 2012
             Exited Scholars Who Should Have Started Working in
                                                                                                               106              67%
             Acceptable Employment by September 30, 2012*
                Scholars Who Fulfilled Service Obligation or
                                                                                                                      72            68%
                Are On Track to Fulfill Service Obligation**
                Scholars Who Are Not On Track to Fulfill Service Obligation***                                        31            29%
                Scholars Who Are in Process of Receiving a Deferral or Exception                                       2             2%
                Scholars Who Paid Back Financial Assistance Received                                                   1             1%
          * This number is composed of scholars whose 2-year grace period expired on or before September 30, 2012, as well as
          scholars who withdrew or were expelled from their respective program and were not permitted a 2-year grace period, but
          were eligible to fulfill their obligation by working in acceptable employment.
          ** Of these 72 scholars, 44 (61 percent) have already fulfilled their service obligation and 28 (39 percent) are currently
          working in acceptable employment and are on track to fulfill their service obligation within the required timeframe.
          *** Of these 31 scholars, 11 (35 percent) are currently working in acceptable employment and 20 (65 percent) are not
          currently working in acceptable employment. Also, 6 of these 31 scholars (19 percent) have been referred to DPMG. We
          noted that only one of these scholars was referred both before the start of our audit and before her complete-by date. Three
          other scholars were referred before the start of our audit, but long after their complete-by dates. The remaining two scholars
          were referred both after the start of our audit and after their complete-by dates. [See below for additional discussion
          regarding RSA’s scholar referral process, including reasons why scholars were not appropriately identified and referred for
          financial repayment.]


We asked each program officer to verify whether they contacted the grantees regarding these
scholars prior to our followup inquiries. Our review of documentation and additional
information provided by these program officers suggested that followup was initiated for 8 of the
45 (18 percent) scholars without prompting from the audit team. Overall, it was unclear if
followup was initiated prior to our inquiries for a majority of these scholars.

Referral of Scholars for Financial Repayment

During our audit, we learned that it is RSA’s unwritten practice to wait to refer scholars for
repayment until their complete-by date has passed. However, even under this unwritten practice,
we found that only 3 of the 10 (30 percent) scholars in our sample who should have completed
their service obligation by September 30, 2012, but who did not do so, were submitted to DPMG
for repayment prior to the start of our audit. 10 We found that it took, on average, more than
1.5 years after the complete-by date had passed for program officers to refer these three
individuals.

Following the start of our audit, two additional scholars were submitted to DPMG for repayment,
resulting in a total of 5 of 10 (50 percent) scholars who should have completed their service
obligation by September 30, 2012, and who have been referred to DPMG. We asked program
officers about the remaining five individuals to determine why they were not submitted for
repayment and whether they should have been. After obtaining additional information from the
grantee project directors, program officers stated that all five individuals will be submitted for
financial repayment. [See Table 3 for information on the number and status of exited scholars
served under the grants in our sample who should have started working.]


10
     These 10 scholars are a subset of the 31 scholars who should have been identified as being in repayment status.
Final Audit Report
ED-OIG/A19M0004                                                                                                        Page 14 of 22

            Table 3: Status of Exited Scholars Who Should Have Started Working
      Exited Scholars Who Should Have Started Working in Acceptable
                                                                                                    106
      Employment by September 30, 2012
         Exited Scholars Who Were Still Within Period of Obligation as of
                                                                                                          80            75%
         September 30, 2012
         Exited Scholars Whose Period of Obligation Had Expired as of
                                                                                                          26            25%
         September 30, 2012*
            Scholars Who Fulfilled Service Obligation                                                          15           58%
            Scholars Who Did Not Fulfill Service Obligation**                                                  10           38%
            Scholars Who Paid Back Financial Assistance Received                                                1            4%
      * This number includes only those scholars who should have completed working in acceptable employment by
      September 30, 2012.
      ** Of these 10 scholars, 5 (50 percent) have been referred to DPMG and issued repayment letters and 5 (50 percent) have not
      yet been referred to DPMG.


According to 34 C.F.R. § 386.43(e), a scholar enters repayment status on the first day of the first
calendar month after the earliest of the following dates, as applicable: (1) the date the scholar
informs the Secretary that he or she does not plan to fulfill the employment obligation under the
agreement, or (2) any date when the scholar's failure to begin or maintain employment makes it
impossible for that individual to complete the employment obligation within the number of years
required.

Under 34 C.F.R. § 386.34, scholars enter into an agreement to repay all or part of any
scholarship they receive if they do not complete their service obligation, and provided they do
not receive a deferral or exception to repayment under 34 C.F.R. §§ 386.41-42. The debt the
scholar owes is a debt to the United States subject to collection by the Secretary. According to
34 C.F.R. § 386.43, the debt equals the amount of the scholarship that has not been retired
through eligible employment, excluding deferral or repayment exceptions, plus interest on the
unpaid balance which accrues from the date that the scholar is determined to have entered
repayment status. The scholar’s debt may be cancelled once documentation to substantiate the
grounds for a repayment exception is provided to and found satisfactory by the Secretary.

The Handbook states that monitoring must address the Department’s fiduciary responsibility to
ensure grantees’ legal and fiscal compliance and to protect against fraud, waste, and abuse.

Reasons for Not Appropriately Identifying and Referring Scholars for Financial Repayment

RSA’s failure to appropriately identify and refer for financial repayment scholars who are not
fulfilling their service obligation results from (1) the lack of a uniform understanding by program
officers of the regulatory definition of repayment status; (2) inadequate guidance on what
program officers should look for when reviewing payback reports; and (3) insufficient internal
communication with regard to the development and use of the RSA MIS.

Definition of Repayment Status

We noted that program officers did not have a clear and consistent understanding of when a
scholar enters repayment status. Although program officers generally agreed that a scholar who
has dropped out or was expelled from their respective program is in repayment status, there was
Final Audit Report
ED-OIG/A19M0004                                                                      Page 15 of 22
confusion regarding the actual regulatory requirement and other scenarios under which a scholar
should be identified as noncompliant and referred for financial repayment. Specifically, we
learned that most program officers do not consider a scholar to be in repayment status until the
date by which the scholar must complete their service obligation has passed. This means that,
contrary to the regulations, scholars whose 2-year grace period has passed and who have not
found acceptable employment are not referred for financial repayment. For example, when we
asked program officers to comment on whether and when they initiated contact with project
directors regarding the scholars who we identified as not on track to complete their service
obligation, we found that program officers generally did not identify these scholars as in
repayment status in the first place. For one grant, we identified 16 scholars who should be in
repayment status. However, the program officer responsible for overseeing this grant stated that
only one of the scholars was in repayment status, as a scholar does not enter repayment status
until after the complete-by date has passed.

During our discussions, program officers noted that there is a considerable amount of time and
paperwork involved for the grantee, RSA staff, and DPMG staff when preparing and processing
repayment letters. Therefore, RSA tries to avoid preparing, processing, and then having to
cancel repayment letters as a result of receiving updated information that scholars have found
acceptable employment albeit after their grace period has expired. Program officers stated that
even if scholars are not on track to complete their employment obligation in the number of years
required, they may still complete acceptable employment up until the date by which they were
supposed to fulfill their service obligation, and will then be held responsible for the remaining
outstanding obligation. As a result, they will generally wait until the date has passed by which a
scholar should have completed his or her service obligation before submitting them for
repayment.

Payback Monitoring Policies and Procedures

As previously discussed in Finding No. 1, we found that RSA lacks adequate policies and
procedures governing program officers’ review of payback reports. While the suggested
protocol should assist program officers in their review of payback reports, we noted that it
continues to place the onus on grantees to determine whether a scholar should be referred for
financial repayment and does not provide clear guidance on steps that program officers might
take to verify grantee determinations or to draw their own conclusions.

MIS Capabilities

We noted that critical information necessary to augment program officers’ monitoring of scholar
payback was not communicated to the individual responsible for the coding of the MIS. The
individual responsible for providing technical and administrative support for the MIS is an RSA
program support staff employee rather than a dedicated RLTT employee, and is one of two
individuals responsible for providing support for the RSA website, which includes all RSA
programs. While we found that the technical lead for RLTT had an understanding of the
program, what the data were, and why the data were being collected, he did not know how this
information would be used to monitor scholar payback.
Final Audit Report
ED-OIG/A19M0004                                                                                      Page 16 of 22
According to the technical lead, the RSA MIS has the data elements necessary to identify
scholars who need to start payback. He added that it is his job to provide the tools needed to
augment staff’s efficiency when tracking scholars, as there are a lot of data but very few staff.
Although the technical lead stated that he would like to provide additional support to program
officers, he noted that this cannot be done without program officers first identifying and
communicating what additional enhancements or analyses would assist them in their monitoring
of scholar payback. The technical lead also noted that because there has been a lot of turnover in
staff over the years, there is no repository of knowledge. We found that RSA did not have a
handbook describing how the MIS was built to capture relevant data and to act as a reference for
program officers. Program officers noted that while they try to review MIS reports for
reasonableness, they might not catch every discrepancy.

As a result of informal requests from program officers, starting in 2011, the system was updated
to automatically flag certain issues or concerns. For example, we found that the MIS now alerts
program officers when a scholar’s grace period has expired without the scholar starting
acceptable employment and when a scholar’s obligation period has expired without the scholar
completing the service obligation. The system also includes logic checks, such as identifying
whether a scholar’s start date is later than their exit date, and automatic calculations of scholars’
grace period end dates and total financial assistance received. 11

Overall, the weaknesses noted with the identification and referral of scholars for repayment
increase the susceptibility of the RLTT program to fraud, waste, and abuse. RSA’s failure to
appropriately identify scholars who are not on track to fulfill their service obligation increases
the risk that the Department will not timely recover funds owed to the Federal government
according to the agreements which these scholars signed upon starting their respective programs.
We noted that the 31 scholars who should have been identified as not on track to complete their
service obligation, and therefore should have been placed in repayment status, received
approximately $280,000 in Federal funds.

Also, because RSA does not refer scholars for repayment until their period of obligation has
expired, the accrued interest owed to the Federal government will be less than what it should
have been had RSA, consistent with the regulations, correctly identified when a scholar entered
into repayment status and timely initiated debt collection activities. Specifically, we noted that
the referral process begins when RSA notifies both the scholar and DPMG, via a demand-for-
payment letter, that the individual is not complying with the RLTT payback requirement and
must repay immediately any financial assistance received. DPMG inputs information from this
letter into its accounting system, which is programmed to begin charging interest 45 days from
the date of the letter. As a result, the date on the demand-for-payment letter effectively serves as
the date on which the scholar was determined to have entered repayment status. If RSA were to
correctly identify when a scholar enters into repayment status, per the RLTT program
regulations, and timely submit scholars for repayment, then the appropriate amount of interest
would be charged.


11
   These logic checks have been implemented on an incremental basis since 2011. The list of MIS report
discrepancies that we note on page 5 in Finding No. 1 is cumulative, dating back to reports that were first submitted
in 2007, prior to the implementation of the logic checks.
Final Audit Report
ED-OIG/A19M0004                                                                      Page 17 of 22
Recommendations

We recommend that the Assistant Secretary for OSERS require RSA to

2.1    Review all grants for which payback reports are still being submitted to determine
       whether any scholars should be classified as being in repayment status; conduct followup
       with grantees as needed to ensure that information is current, accurate, and complete; and
       refer to DPMG for financial repayment any noncompliant scholars, along with all
       required documentation.

2.2    Remind grantees of the consequences of scholars’ failure to obtain acceptable
       employment and advise grantees that they provide such information to their scholars.

2.3    Provide training to program officers on identifying scholars in repayment status in
       accordance with the current RLTT program regulations and on referring noncompliant
       scholars for financial repayment.

2.4    Conduct a comprehensive assessment of the MIS to identify weaknesses and potential
       areas for improvement and communicate these findings to the RLTT technical lead.

2.5    Work with the appropriate individuals to develop an MIS handbook that describes the
       system’s features, defines payback report flags, and is updated when enhancements are
       made.

2.6    Consider changing the regulation defining when a scholar enters repayment status to be
       consistent with the Department’s unwritten practice or simplify the current process,
       consistent with the regulations, to allow for situations in which a scholar may complete
       some part of his or her service obligation within the required time period and, therefore,
       be liable for financial repayment of only a portion of any RLTT funds received.

[Implementation of recommendations made under Finding No. 1 (specifically, recommendation
1.4 on RLTT monitoring policies and procedures and recommendation 1.7 on MIS data
quality/logic checks) will also strengthen controls and oversight in this area.]

RSA Comments

RSA concurred with the recommendations and described actions that it will take to enforce the
service obligation requirement as outlined in the payback policy and procedures manual.
Specifically, if a scholar does not find qualifying employment after the 2-year grace period, he or
she will be asked to repay the funds provided. Grantees will also be held accountable for
providing scholars with information on the service obligation requirement and maintaining
adequate supporting documentation. RSA further stated that it is exploring strategies to develop
quality assurance checks to determine that the appropriate procedures are being followed
consistently by all staff with regard to the identification of scholars who should be in repayment
status, followup with grantees, and the referral of noncompliant scholars to DPMG. RSA will
also work with the appropriate individuals to develop an MIS Handbook that describes system
Final Audit Report
ED-OIG/A19M0004                                                                      Page 18 of 22
features, defines payback report flags, incorporates existing PowerPoint slides that were
provided as web-based training to RLTT grantees, and will be updated when enhancements are
made.

Finally, RSA concurred with the recommendation that it would be preferable to revise the
regulations to provide scholars more flexibility in satisfying the service obligation. RSA stated
that doing so would have the desirable effect of reducing the number of scholars who are not
successful in carrying out the service obligation within the time period allowed, thus reducing
the number of scholars who would then be required to repay all or part of the cost of the
scholarship. However, RSA did not indicate that it would be taking any specific actions to
address this issue.



                                     OTHER MATTER


RSA May Be Inappropriately Granting Extensions of Scholars’ Service Obligations

During our discussions with program officers, we learned that RSA may, on occasion, grant
scholars extensions to the period within which they must complete their service obligation. RSA
did not indicate that these extensions were related to the deferrals or exceptions that the
regulations authorize. 12 Specifically, while we did not identify any instances of this occurring
within the five FY 2006 RLTT grants in our sample, two program officers with whom we spoke
stated that if a scholar is in acceptable employment but is not on track to complete his or her
service obligation within the required number of years, RSA will consider extending the
scholar’s completion date. This is particularly true for scholars who appear to be able to fulfill
their service obligation within a “reasonable amount of time” from their initial completion date,
as RSA does not want to penalize working scholars. One of the program officers noted that such
decisions are made in consultation with RSA management and with OGC. RSA management
subsequently confirmed the program officer’s account of the process.

We followed up with OGC to determine whether (1) it is within RSA’s discretion to grant
scholars extensions to the period within which they must complete their service obligation, and
(2) the RLTT program attorney has been consulted regarding any such extensions. In its
response, OGC did not explicitly state whether or not RSA has the authority to grant extensions
to scholars’ completion dates, but noted that it is aware of only one instance in which RSA
appears to have taken such action—and stated that it believes RSA’s decision was reasonable, in
light of the circumstances, and served to advance the Federal interest with respect to the RLTT
program. OGC did note, however, that it would discuss with program officials whether they
wish to consider expanding the flexibility within the current regulations to more specifically
provide for similar exceptions in a more systematic manner. On the second question, OGC
stated that the RLTT program attorney had no recollection of having been consulted by RSA, but
noted that program offices are not expected to consult OGC on all decisions and that, even when


12
     See footnote 1 on page 2.
Final Audit Report
ED-OIG/A19M0004                                                                      Page 19 of 22
they do, OGC’s views and opinions are advisory. Program officials have the ultimate authority
to implement and administer their programs, in accordance with all applicable criteria.

Congress established the current period of obligation for RLTT scholars in the Rehabilitation Act
Amendments of 1992 (Public Law 102-569). Prior to this time, scholars were permitted 10 years
within which to complete their service obligation. We noted that, in implementing regulations
for the amendments, the Department did not appear to view the period of service obligation as
flexible, beyond allowing for the granting of deferrals or exceptions under limited circumstances.
One commenter on the proposed regulations expressed concern that scholars would have less
time to complete their service obligation. The Department responded that, “[t]he time limits are
set in the Rehabilitation Act Amendments of 1992,” and made no changes to the proposed
period of obligation—thereby implying that it did not view the statutorily-defined time period as
flexible.

Based on the above, it does not appear that the Department has the authority to unilaterally
extend scholars’ completion dates, nor the ability to work within the current regulations to allow
for such determinations. Any changes to the circumstances under which a deferral or exception
may be granted would require revisiting the program regulations. We noted, however, that
RSA’s October 2008 Frequently Asked Questions on the RLTT program reference a scholar’s
ability to appeal a finding of noncompliance with the payback requirement. We suggest that
RSA work within the existing rules—and also ensure that staff are aware of what is allowable
under these rules—until such time as the Department decides to revisit the RLTT program
regulations to allow for expanded flexibility with regard to deferrals or exceptions. Doing so
would provide some protection both for the Department and for the scholar and potentially
prevent abuses on the part of either party.

RSA Comments

RSA stated that it would like to clarify that granting extensions to scholar service obligation
deadlines is not standard practice and that any consideration that may be given to extending the
service obligation deadline is done in the best interest of the Federal government and only after
consultation with RSA management. However, RSA stated that, because it recognizes the
desirability of establishing a clear basis for extensions, it will consult with OGC about the best
avenue for establishing a procedure for granting extensions. If OGC determines that regulations
are required, likely as amendments to 34 C.F.R. 386.41, RSA will regulate; if OGC believes that
a policy document provides a sufficient basis, RSA will produce one and have it approved by
OGC.
Final Audit Report
ED-OIG/A19M0004                                                                      Page 20 of 22


                  OBJECTIVES, SCOPE, AND METHODOLOGY


The objectives of our audit were to determine whether the RLTT program effectively (1) met
program objectives by training recipients who subsequently performed work related to the
program, and (2) obtained repayment of the assistance received for recipients that did not fulfill
work agreements. This audit was part of a review of payback provisions of selected grant
programs throughout the Department.

To accomplish our objectives, we gained an understanding of internal control applicable to the
Department’s administration and oversight of discretionary grant programs, in general, and to
RSA’s process for monitoring grantees and scholars who receive funding under the RLTT
program. We reviewed applicable laws and regulations, OMB guidance, Department policies
and procedures, Federal Register notices, and the Government Accountability Office’s (GAO)
“Standards for Internal Control in the Federal Government.” In addition, to identify potential
vulnerabilities, we reviewed prior Office of Inspector General (OIG) and GAO audit reports with
relevance to our audit objectives.

We conducted discussions with RSA management and staff, including program officers and the
MIS technical lead, to obtain a more complete understanding of the RLTT program. These
discussions focused primarily on monitoring activities pertaining to the service obligation
component of the RLTT program, as well as information on GPRA performance measures and
relevant program studies. We conducted discussions with DPMG officials regarding the referral
and financial repayment processes and to determine whether RSA has utilized their services. We
also conducted discussions with OGC officials regarding RSA’s discretion to grant scholars
extensions to the period within which they must complete their service obligation.

The scope of our review was limited to the Department’s post-award activities for grants made
under the FY 2006 RLTT competition, with a particular focus on monitoring of scholars’
fulfillment of the applicable payback requirement. We chose to focus on FY 2006 grants as they
were more likely to have a larger number of scholars that had completed the training and were in
the payback portion of the program than more current grants, thereby enabling us to obtain
sufficient evidence to answer our audit objectives. We randomly selected a sample of 5 of the 11
(45 percent) RLTT grants awarded in FY 2006. Our sample included $2.3 million of the
$5.4 million (43 percent) awarded to FY 2006 RLTT grantees.

To achieve our objectives, we relied primarily on grantee-reported data found in MIS payback
reports. We reviewed payback reports for FYs 2006-2012 for each of the five FY 2006 RLTT
grants in our sample. We also reviewed official grant files, which are maintained in hardcopy
form, and emails and other documentation maintained in TRIMS, RSA’s electronic grant
database. We also reviewed information on scholars that RSA had recently received from
grantees, often in response to questions that arose during the course of our audit. We reviewed
documentation for each of the grants in our sample to determine: (1) the number of scholars
served; (2) financial assistance provided to each scholar; (3) the current program enrollment
status of each scholar; and (4) whether payback (either employment or financial) had begun for
Final Audit Report
ED-OIG/A19M0004                                                                      Page 21 of 22
those scholars who completed or otherwise exited their respective program. We also reviewed
referral and financial repayment documentation provided by DPMG and RSA.

We relied on computer-processed data from G5, the Department’s grants management system,
and from the MIS. We used G5 for the purpose of identifying the universe of RLTT grants
awarded in FY 2006 and related obligation amounts. Because G5 is the Department’s system of
record for such information and the data were used primarily for informational purposes and did
not materially affect our findings and resulting conclusions, we did not perform a data reliability
assessment.

We used the MIS for the purpose of identifying the universe of scholars served under the five
FY 2006 RLTT grants in our sample and also to determine the current status of each scholar.
We compared the total number of scholars served under each grant according to the MIS
payback reports with information contained in each grantee’s annual and final performance
reports. Based on this analysis, we concluded that the computer-processed data were sufficiently
reliable for identifying the universe of scholars served. However, because source data for
information contained in the payback reports is primarily located at various grantees across the
country, our ability to perform an assessment of the reliability of the information in the reports
was limited and as such, we could not fully determine the reliability of the data.

We compared MIS payback reports across years to evaluate the consistency of data reported by
grantees. We noted that payback reports submitted by all five grantees in our sample did not
always include all of the requested data and, based on some reasonableness tests we performed,
contained discrepancies. Several reports were missing information such as a scholar’s
employment status or total funding amount, contained illogical entries for a scholar such as a
program exit date that occurred prior to the reported start date, and/or incorrectly reported data
elements such as the number of work years owed or completed by a scholar. [See Finding No. 1
for additional information.] However, despite these limitations, we believe the evidence
obtained provides a reasonable basis for our findings and conclusions based on the audit
objectives. Specifically, the limitations noted did not impact our ability to assess RSA’s
processes related to the identification and referral of noncompliant scholars nor provide enough
basis for us to completely discount what available data indicate with regard to program results.

We conducted fieldwork at Department offices in Washington, D.C., from November 2012
through September 2013. We provided our audit results to Department officials during an exit
conference conducted on September 30, 2013.

We conducted this performance audit in accordance with generally accepted government
auditing standards. Those standards require that we plan and perform the audit to obtain
sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions
based on our audit objectives. We believe that the evidence obtained provides a reasonable basis
for our findings and conclusions based on our audit objectives.
Final Audit Report
ED-OIG/A19M0004                                                                     Page 22 of 22


                           ADMINISTRATIVE MATTERS


Corrective actions proposed (resolution phase) and implemented (closure phase) by your office
will be monitored and tracked through the Department’s Audit Accountability and Resolution
Tracking System. Department policy requires that you develop a final corrective action plan
(CAP) for our review in the automated system within 30 days of the issuance of this report. The
CAP should set forth the specific action items, and targeted completion dates, necessary to
implement final corrective actions on the findings and recommendations contained in this final
audit report.

In accordance with the Inspector General Act of 1978, as amended, the OIG is required to report
to Congress twice a year on the audits that remain unresolved after 6 months from the date of
issuance.

In accordance with the Freedom of Information Act (5 U.S.C. § 552), reports issued by the OIG
are available to members of the press and general public to the extent information contained
therein is not subject to exemptions in the Act.

We appreciate the cooperation given us during this review. If you have any questions, please
call Michele Weaver-Dugan at (202) 245-6941.


                                            Sincerely,



                                            Patrick J. Howard /s/
                                            Assistant Inspector General for Audit
                                                                       Attachment 1


               Abbreviations, Acronyms, and Short Forms
                          Used in this Report


Act          Rehabilitation Act of 1973, as amended

CAP          Corrective Action Plan

C.F.R.       Code of Federal Regulations

Department   U.S. Department of Education

DPMG         Debt and Payment Management Group

FY           Fiscal Year

GAO          Government Accountability Office

GPRA         Government Performance and Results Act

Handbook     Handbook for the Discretionary Grant Process

MIS          Management Information System

OGC          Office of the General Counsel

OIG          Office of Inspector General

OMB          Office of Management and Budget

OSERS        Office of Special Education and Rehabilitative Services

RLTT         Rehabilitation Long-Term Training

RSA          Rehabilitation Services Administration

U.S.C.       United States Code
                                                                                                            Attachment 2
                                   RSA Response to Draft Audit Report
                             UNITED STATES DEPARTMENT OF EDUCATION
                         O FFICE OF SPECIAL EDUCATION AND REHABILITATIVE SERVICES




                                                                                        APR 0 1 2014

Michele Weaver-Dugan, Director
Operations Internal Audit Team
U.S. Department of Education
Office of Inspector General
400 Maryland Avenue, S.W.
Washington, D.C. 20202-1500

Dear Ms. Weaver-Dugan:

1 am in receipt of the letter from the Assistant Inspector General for Audit, dated January 30, 2014,
which transmitted the draft audit report, titled, Payback Provisions ofthe Rehabilitation Long-Term
Training Program, control number ED-OIG-A I 9M0004. My staff and I appreciate the opportunity to
review the report. As is indicated below, the Rehabilitation Services Administration (RSA) will
address the 15 recommendations described in this report in the months ahead. It has been and will
continue to be our goal to ensure that the information collected and reported for the Rehabilitation
Long-Term Training (RLTT) program is accurate and that any monies due to the Department are
collected in a timely manner. To that end, we have made and continue to make significant strides in
our collection or data and delivery of technical assistance to grantees.

While we do not disagree with any of the recommendations in the report, we identify the s ignificant
actions RSA has taken and is currently taking to improve information collection and reports. In order
for you to understand our perspective, we will address the nine recommendations under finding I
and the six recommendations under Finding 2, in the order in which they were presented.

Finding I - Results Indicate that the Majority of Exited Scholars are Working but Data
            Quality is a Concern

We summarize this finding very briefly here. The OIG dratt audit report noted that, " We found that
the majority of scholars who received training under the five FY 2006 R LTT program grants in our
sample obtained acceptable employment, based on information reported to the Department by
grantees." The report also stated, "However, we iclenti fied data quality issues which call into
question the accuracy of these results."




                              400 MARYLAND AVE. S.W., WASH INGTON DC 20202-2500
                                                        www.ed.gov
  Tile Deparment of Education's mission is to promote student achievement and preparation for global competitiveness by
                               fos tering educational excellence and ensuring equal access.
Page 2 -Ms. Weaver-Dugan



OIG Recommendation 1.1:

Continue to emphasize to grantees, through training or by other means, the need to provide accurate
and complete data for the purpose of determining total financial assistance received and work years
owed by each scholar, and the regulatory requirement that grantees continue to track scholars once
they have exited the program and provide information annually on work years completed or requests
for waivers or deferrals.

RSA Response:

We concur with the recommendation to emphasize to grantees the need to provide accurate and
complete data. RSA has provided substantial training and technical assistance to grantees in four
ways. First, RSA has developed a payback policy and procedures manual for grantees that outlines
the payback regulations and service obligation requirements. This manual was developed in 2008
and a revision is undergoing internal review and clearance. In addition, in 20 11 RSA developed a tip
sheet for RSA scholars that outlines the following: roles and responsibilities of the scholar, including
how the scholar will be held accountable for completing the service obligation or for repaying the
debt in the event that qualifying employment is not obtained; processes, procedures, and required
documentation to confirm qualified employment; and procedures and documentation necessary to
confirm circumstances related to repayment, deferment, and waiver requests.

Second, RSA regularly conducts web-based training and technical assistance for grantees in the
RLTT program. For example, interactive webinars are conducted annually to orient grantees to any
changes, improvements, or enhancements to the RSA Management In formation System (M IS) site,
which can be found at rsa.ed.gov. All webinars are recorded and archived, and RSA strongly
recommends that any new project staff carefully review the archived training. In addition, each year
RSA asks grantees to submit questions and technical assistance needs related to the RSA MIS or the
RLTT program. Program officers review these questions, along with common errors identified in the
RSA MIS or in the annual performance reports, and then customize a webinar to emphasize issues
such as providing accurate and complete data and to address grantee technical assistance needs.

Third, RSA communicates regularly with grantees through its listserv. For example, RSA alerts
grantees to common errors found during program officer reviews of payback reports or annual
performance reports and provides technical assistance to correct those errors. Listserv messages also
provide grantees further guidance and clarification regarding processes, procedures, and
documentation required for scholar repayment and deferment and requests for waivers. RSA also
utilizes the listserv to enforce due dates and expectations for reporting accurate and complete data.

Finally, RSA actively participates in national conferences in order to interact with RSA grantees and
scholars and to provide in-person training and technical assistance to both RLTf grantees and
scholars. For example, RSA conducts a session for RLTT grantees that focuses on program
improvement, particularly in the areas of effectively managing scholar files and in reporting accurate
and complete data. In addition, RSA conducts a session designed for RSA scholars to ask questions
and share technical assistance needs and for RSA to obtain feedback about the program.
Page 3 - Ms. Weaver-Dugan



Please note that RSA transmitted electronic copies and/or web links for each of the training and
technical assistance materials listed below to the OIG on November 20, 20 12, and again on April 1,
2013. For your reference, we added any training and technical assistance conducted since
transmitting this material last April.

MANUALS, TIP SHEETS, and UPDATED FORMS

              •   RSA - Payback policy and procedures manual: This is the manual we have been
                  providing to grantees that describes payback policies and procedures; an updated
                  manual is undergoing internal review and clearance.
              •   Scholar Tip Sheet.
              •   Updated repayment, waiver, and deferral request forms.

       l . TRAINING and TECHNICAL ASSISTANCE WEBTNARS

              •   April 3, 2013: Annual Performance Report Training for RSA Long-Term
                  Training Grants.
              •   November 11 , 2013: Q&A on Payback.
              •   October 29, 2013: Payback Report Submission through the RSA
                  Management Information System.
              •   April 10, 2012: Answers to Questions about submitting the FY 2012 Annual
                  Performance Report (APR) Webinar.
              •   March 29, 2012: Annual Performance Report (APR) Submission through the GS
                  Website Webinar.
              •   March 22, 2012: 2012 Annual Performance Report (APR) Training for RSA
                  Training Programs.
              •   November 7, 2012: Q&A on Payback.
              •   October 16, 2012: Payback Webinar on recent changes that have been made for
                  reporting in the MIS.
              •   November 9, 201 1: Answers to Questions about Payback Policies, Procedures,
                  and Regulations.
              •   October 13, 2011: Payback Webinar regarding Payback Policies, Procedures, and
                  Regulations.
              •   October 11 , 2011: Payback Webinar regarding the basics of submission through
                  the MIS.
              •   October 28, 2010: Overview of the Payback Report.

       2. LISTSERV MESSAGES

              •   Summary of Payback Webinars Follow-up to Payback Webinar Series and
                  Payback Reporting for FY201 1.
              •   Listserv email 12.21.10 Long-Term Training Payback Report - REMINDER
                  Long-Term Training Grant FY 20 11 Payback Report Due Date.
              •   Listserv email 8.1 2.11 Important Announcements for Long-Term Training Grants.
Page 4 -Ms. Weaver-Dugan



            •   Listserv email 10.5.11 Important Payback Reporting for FY 2011.
            •   Listserv email 10.14.11 payback webinar series and payback reporting for FY 2011.
            •   Listserv email 10.18.11 Rescheduling Payback Webinar #3.
            •   Listserv email 11.4.11 Third Payback Webinar and helpful materials.
            •   Listserv email 11.16.11 ALERT Payback MIS Issue.
            •   Listserv email 11.28.11 REMINDER Long-Term Training Grant FY 2011
                Payback Report Due Date.
            •   Listserv email 3.9.12 Annual Perfonnance Report Webinar Series.
            •   Listserv email 9.28.12 Long Term Training Payback.
            •   Listserv email 10.5.12 Payback webinars.
            •   Listserv email 10.24.12 Reminder of Upcoming Payback webinar.
            •   Listserv email 11.19.12 Payback Information for Long-Term Training Grantees.
            •   Listserv email 3 .21.12 0 Ider Grant Follow-up.
            •   Listserv email 3.27.12 Follow-up on FY 2012 APR Webinars.
            •   Listserv email 3 .21.12 Webinar Reminder: FY 2012 Annual Performance Report.
            •   Listserv email 3.29.12 REMINDER FY 2012 APR Submission to GS Webinar today!
            •   Listserv email 3.30.12 Important: Submit questions for Webinar on April 10.
            •   Listserv email 4.11.12 Follow-Up to Annual Performance Report Webinar.
            •   Listserv email 4.16.13 IMPORTANT FY 2012 APR INSTRUCTION: 524B
                Project Status Chart.
            •   Listserv email 4.25.12 Annual Performance Reports: Tips and Reminders.
            •   Listserv email 5.11.12 FY 2012 APR Update & Use of Grant Funds.
            •   Listserv email 7.26.12 GRANT A WARD NOTICE & GUIDANCE.
            •   Listserv email 9.13.12 Long-Term Training Grants ending 9/30/2012.
            •   Listserv email 9.20.12 Long-Tem1 Training Follow-Up.
            •   Listserv email 9.28.12 Long Tenn Training Payback.
            •   Listserv email 10.5.12 Payback webinars.
            •   Listserv email 10.24.12 Reminder of Upcoming Payback webinar.
            •   Listserv email 11.19.12 Payback Information for Long-Term Training Grantees.
            •   Listserv email 12.19.12 Important: Payback Update.
            •   Listserv email 3.5.13 Long-Term Training Annual Performance Reports.
            •   Listserv email 3.12.13 Webinar for Annual Performance Reports for Long-Term
                Training Grants.
            •   Listserv email 4.10.13 Important Information for Long-Term Training Grants.
            •   Listserv email 4.11.13 Important Payback Issue.
            •   Listserv email 4.16.13 Reminder of April 30 Deadline for Annual
                Performance Reports.
            •   Listserv email 6.14.13 Important Items for Long-Term Training Grantees.
            •   Listserv email 6.27.13 Final Reports & Enclosure 7 guidance.
            •   Listserv email 9.27.13 Long Term Training Payback Reports.
            •   Listserv email 10.21.13 Message to Rehabilitation Long-Term Training Grants.
            •   Listserv email 1.29.14 Payback for Long-Term Training.
Page 5- Ms. Weaver-Dugan



        3. CONFERENCES

               •   Project Directors' Conference: RSA program officers conducted an in-person
                   training and technical assistance session for grantees at both the 2011 and 2012
                   conferences held in August of each year. The Project Directors' Conference did
                   not occur in 2013. A conference is currently scheduled for fall 2014, and RSA is
                   serving on the planning committee and is scheduled to conduct a training and
                   technical assistance session for RL Tr grantees focusing on payback.
               •   National Council on Rehabilitation Education (NCRE): RSA participates in this
                   arumal conference held in the fall of each year, and presents technical assistance
                   sessions for scholars and RLTT grantees.

OIG Recommendation 1.2:

Consider requiring RL TT grantees to provide certifications that data submitted to the Department is
accurate, reliable, and complete in order to provide an additional level of internal control over the
reliability of performance data.

RSA Response:

We concur with the recommendation to require RL TT grantees to provide certifications that data
submitted to the Department are accurate, reliable, and complete. RSA will continue its training and
technical assistance efforts, as detailed in RSA 's response to Recommendation 1.1, to improve the
accuracy and reliability of grantee data. In addition, RSA made significant changes to the 84. l 29B
Rehabilitation Long-Term Training Notice of Final Priority and Notice Inviting Applications, which
closed on February 4, 2014, in order to emphasize with new grantees the importance of accurate,
reliable, and complete data, as well as program outcomes and impacts. A link to the Notice of Final
Priority can be found here: http://www.gpo.gov/fdsys/pkg/FR-2013-11-05/pdf/2013-26500.pdf
(scroll down to the Final Priority section). Here are a few examples found in the Final Priority to
demonstrate RSA's efforts to continue to improve the RLTT program. Applicants must:

   •    Recruit highly capable prospective scholars who have the potential to successfully complete
        the academic program, all required practicum and internship experiences, and the required
        service obligation;
   •    Educate potential scholars about the terms and conditions of the service obligation under 34
        CFR 386.4, 386.34, and 386.40 through 386.43 so that they will be fully informed before
        accepting a scholarship;
   •    Maintain a system that ensures that scholars sign a payback agreement and an exit form when
        they exit the program, regardless of whether they drop out, are removed, or successfully
        complete the program;
    •   Maintain regular contact with scholars upon successful program completion (e.g., matching
        scholars with mentors in the field), to ensure that they have support during their search for
        qualifying employment as well as support during the initial months of their employment;
Page 6-Ms. Weaver-Dugan



   •   Maintain regular communication with scholars after program exit to ensure that scholar
       contact information is up-to-date and that documentation of employment is accurate and
       meets the regulatory requirements for qualifying employment; and
   •   Maintain accurate information on, while safeguarding the privacy of, current and former
       scholars from the time they are enrolled in the program until they successfully meet their
       service obligation.

A link to the NIA can be found here: http://www.upo.gov/fdsys/pkg/FR-2014-0l-28/pdf/2014-
01617.pdf. For the latter, please refer to Section VI Award Administration Information, 4.
Performance Measures. This section was strengthened to place a greater emphasis on reporting and
accountability. Here are a few examples of what grantees are required to maintain:

   •   Current contact information for all students receiving scholarships, including home address,
       email, and a phone number (home or cell);
   •   A point of contact for each scholar in the event that the grantee is unable to contact the
       student. This contact must be at least 21 years of age and may be a parent, relative, spouse,
       partner, sibling, or guardian;
   •   Cumulative financial support granted to scholars;
   •   Scholar debt in years;
   •   Program completion date and reason for exit for each scholar;
   •   Annual documentation from the scholar's employer(s) until the scholar completes the service
       obligation. This documentation must include the following elements in order to verify
       qualified employment: start date of employment to the present date, confirmation of full-time
       or part-time employment (if the scholar is working part-time the number of hours per week
       must be included in the documentation), type of employment, and a description of the roles
       and responsibilities performed on the job. This information is required for each employer if
       the scholar has worked in more than one setting in order to meet the service obligation.
   •   If the scholar is employed in a related agency, the agency must also provide documentation
       to validate that there is a relationship with the State YR agency. This may be a formal or
       informal contract, cooperative agreement, memorandum of understanding, or related
       document; and
   •   Annual documentation from the scholar's institution of higher education to verify dates of
       deferral , if applicable. The documentation may be prepared by the scholar's advisor or
       department chair and, if the scholar is seeking deferral on the basis that he or she is pursuing
       a full-time course of study, must include: confirmation of enrollment date, estimated
       graduation date, confirmation that the scholar is enrolled in a full-time course of study, and
       confirmation of the scholar's intent to fulfill the service obligation upon completion of the
       program.

Despite RSA's efforts to emphasize the importance of submitting data of high quality with both new
and current grantees, we do believe that having them submit a certification of data quality would be
helpful. We will work with the MIS technical lead to develop a way for grantees to submit a
certification of data quality. This certification will be submitted when grantees submit their annual
payback report through the MIS in the fall of 2014.
Page 7 - Ms. Weaver-Dugan



OIG Recommendation 1.3:

Identify grantees who consistently fail to provide accurate and complete scholar data and take
appropriate enforcement actions.

RSA Response:

We concur with the recommendation to identify grantees who consistently fail to provide accurate
and complete scholar data and to take appropriate enforcement actions. Each RSA program officer
currently makes notes in the official grant file regarding the accuracy and timeliness of a grantee's
payback reporting. Based on these notes, we will develop communication from RSA management to
grantees that are not reporting accurate and complete data as a means to make clear to grantees that
accurate and complete reporting is a legal requirement of the award. Such communications from
RSA management will then become part of the official grant file. If a grantee consistently fails to
comply with the tracking system, documentation, reports and records requirements of 34 CFR
386.34, RSA will, after warning, terminate the award. Additionally, we will consider the timeliness
and accuracy of any prior grantee's reporting performance as we develop the slate for new RLTT
grants. We have consistently infonned applicants that past performance will be considered when we
make new grant awards.

OIG Recommendation 1.4:

Finalize the protocol for RSA program officers to reference when completing their reviews of
payback reports and incorporate it into RLTT's formal monitoring policies and procedures.

RSA Response:

We concur with the recommendation to finalize the protocol for RSA program officers to reference
when completing their reviews of payback reports and to incorporate the protocol into formal
monitoring policies and procedures for the RLTT grants. In addition to implementing this
recommendation, it is important to note that RSA program officers discuss common payback report
matters on a regular basis and more in-depth during regular staff meetings convened prior to and
following the opening of the MIS for report submission. Since 20 I 0, it has been standard practice for
RSA program officers to take part in annual training around the payback review process. Training is
typically held in a series of sessions, the first of which occurs prior to the opening of the MIS for
grantees to enter their payback data. The purpose of this session is to orient and familiarize program
staff with enhancements to the RSA MIS site, often using mock payback reports for purposes of
instruction. Next, RSA program officers have the opportunity to participate in interactive training,
where the MIS is open and a step-by-step walkthrough of the payback report is conducted. RSA has
found this practice to be invaluable and will continue this effort in the future. Not only has the
training been critical to RSA program officers in improving their review of payback reports, but it
has proven to be helpful as they provide technical assistance to grantees who have payback report
questions or who encounter problems when the MIS is open for report submission. The provision of
training to all RSA staff who are responsible for RLTT grants also ensures that program officers are
reviewing payback reports as well as identifying potential errors in a consistent manner. The training
Page 8- Ms. Weaver-Dugan



provided identifies for RSA staff the key elements of the report that warrant particular attention, i.e.,
the amount of the scholarship, whether the sponsoring agency has a signed payback agreement on
fil e, whether the scholar's address is currently on file, the dates on which the scholar entered and
finished the program, and the date on which qualifying employment must begin and end. RSA
program officers are also trained to carefully review the number of years owed and completed as
well as the circumstance in which a scholar exits the program, which may require additional fo llow-
up, i.e., processing a deferral, waiver, or repayment request. If the RSA program officer discovers
any discrepancies or missing information, the training provides helpful information and successful
strategies to address such occurrences. RSA will formally incorporate the elements of the training
described above in its payback policy and procedures manual, which will be fin alized by the fall of
20 14 for use during the analysis of the FY 20 14 RL TT payback reports. All grantees will be
provided a copy and the manual will be made available online.

OIG Recommendation 1.5:

Provide training to RSA program officers on the payback report review process.

RSA Response:

We concur with the recommendation to provide training to RSA program officers on the payback
report review process. Please refer to our response to Recommendation 1.4 for a description of the
training that is currently being provided to RSA staff. In addition to the interactive training that is
annually provided, senior RSA program officers serve as a resource for newer staff throughout the
payback report review process. Finally, typically in April or May, a session is held in which RSA
program officers convene to discuss the review process in general, including lessons learned, grantee
experiences or feedback, recommendations for changes or enhancements to the MIS system, and
ways to improve the overall process in the months ahead.

OIG Recommendation 1.6:

Establish and implement timelines for the initial review of payback reports, follow-up with grantees,
and formal approval in the MIS.

RSA Response:

We concur with the recommendation to establish and implement timelines for the initial review of
payback reports, fo llow-up with grantees, and forma l approval in the MIS. We would like to note
that, since 20 l 0, it has been standard practice fo r RSA to establish guidelines for program officers to
start and complete the review of the payback reports. Such communication is conveyed to RSA
program officers during regularly scheduled staff meetings throughout the year, where staff
collectively identify and establish dates to perform or complete specific payback tasks. As a general
rule, the review of payback reports starts immediately after the submission deadline of November 30
and must be completed by the end of February. As RSA program officers review payback reports,
their standard practice is to review the report for complete and accurate data. They have been
instructed not to approve payback reports until all data have been corrected and all documentation to
Page 9-Ms. Weaver-Dugan



support any request for a waiver, deferment, or repayment has been received. While grantees are
responsible for supplying supporting documentation for repayment, deferment, or waiver
submissions, RSA program officers have responsibility for following-up to ensure timely receipt of
the documentation and subsequently approving reports in the MIS. Grantees are also made aware of
RSA' s timelines and practices through internal meetings, webinars, and presentations at national
conferences for the expressed purpose of ensuring that data submitted in payback reports are
accurate and complete. Although deadlines have been established, we are exploring strategies that
would enable us to conduct quality assurance checks in the MIS to ensure that both RSA staff and
grantees are adhering to the timelines established. All such timelines and procedures will be
incorporated in the payback policy and procedures manual described in response to recommendation
1.5.

OIG Recommendation 1.7:

Ensure that key data quality/logic checks are built into the MIS.

RSA Response:

We concur with the recommendation to ensure that key data quality/logic checks are built into the
MIS. RSA would like to note that, since 2010, it has been standard practice for RSA program staff to
provide the technical team with recommendations to ensure that key data quality/logic checks are
built into the system and are fully operational. As mentioned in our response to Recommendation
1.5, typically in April or May of each year, RSA program officers convene to discuss the review
process, make improvements, and provide recommendations for changes to the MIS system. These
recommendations are provided to the technical team, which provides technical assistance for the
MIS. Once the recommendations are reviewed for feasibility, the MIS technical team develops an
updated payback report for the upcoming reporting cycle. Any changes to the system are reviewed
by program staff during user acceptance testing, which typically occurs in August or September.
These enhancements in data quality/logic checks are provided to grantees generally in the first of a
series of three webinars designed to provide technical assistance and prior to the opening of the MIS
for payback report submission. We are fully committed to continuing to work with the technical lead
on the MIS to develop additional quality/logic checks for the payback report and will have additional
edit checks in the system by the fall of2015.

OIG Recommendation 1.8:

Consider incorporating other means by which RSA program officers can verify the adequacy of
grantees' tracking systems and scholars' employment in the absence of adequate funding for site
visits, to include periodic desk audits of all grantees regardless of whether they have been selected
for on-site monitoring.

RSA Response:

We concur with the recommendation to incorporate other means by which RSA program officers can
verify the adequacy of grantees' tracking systems and scholars' employment in the absence of
Page 10 - Ms. Weaver-Dugan



adequate funding for site visits, to include periodic desk audits. RSA notes that we have directed the
MIS technical lead in past years to make numerous enhancements in the MI S system to capture
greater reliability in data. To ensure the enhancements are made as requested, RSA has a dedicated
staff person who follows up and makes verification of the completed work. RSA will continue to
make enhancements annually or as the need arises and these enhancements include, but are not
limited to, the following:

   •   Improve overall analytics to allow the site to improve submission and reliability of payback
       reports.
   •   Allow secure submission of supporting documentation for payback reports (i.e. proof of
       qualifying employment, signed payback agreements, repayment documentation, etc.).
   •   Automatic generation of standard letters and responses based upon data submitted by the
       grantee.
   •   Improve analytics that automate alerts for both program staff and grantees based upon data
       submitted into the MIS.

OIG Recommendation 1.9:

Consider including and reporting on additional performance measures that would reflect a more
comprehensive view of program effectiveness.

RSA Response:

We concur with the recommendation to include reporting on additional performance measures that
would reflect a more comprehensive view of program effectiveness. We recently have had
conversations with Budget Service about revising our GPRA measures. We have established a small
work group to review and refine the measures and it is our hope to have the new measures in place in
the fall of2014 for inclusion in the FY 2016 Congressional Justification.

Finding 2 - Improvements arc Needed in the Department's Process for Identifying and
            Referring Scholars for Financial Repayment

We summarize this finding very briefly here. The OIG draft audit report noted, " We found that RSA
does not appropriately identify and refer for financial repayment scholars who are not fulfilling their
service obligation." The report also stated, "We also noted that program officers' unwritten practice
of not identifying scholars for repayment until after their complete-by date had passed did not
always result in the timely referral to DPMG of scholars who should have completed their service
obligation on or before September 30, 2012, the end date for the period covered by the most recent
payback report as of the start of our audit."

OIG Recommendation 2.1:

Review all grants for which payback reports are still being submitted to determine whether any
scholars should be classified as being in repayment status; conduct follow-up with grantees as
Page 11- Ms. Weaver-Dugan



needed to ensure that information is current, accurate, and complete; and refer to DPMG for
financial repayment any noncompliant scholars, along with all required documentation.

RSA Response:

We concur with the recommendation. We would like to note that it has been standard practice, since
2010, for RSA program officers to review all payback reports and determine whether any scholar(s)
should be classified as being in repayment status. RSA has conducted appropriate follow-up as well
as the proper referral of noncompliant scholars to the Debt and Payment Management Group
(DPMG) when the situation warrants. We further note that not only is such referral made following
the review of annual payback reports, but throughout the course of the year when a RSA program
officer is notified by the sponsoring agency that a scholar needs to be entered into repayment. In
such instances, the project director submits all supporting documentation to the RSA program officer
in order to initiate repayment. The documentation includes the signed payback agreement, the signed
Exit form, and any other documentation demonstrating that the scholar is not in qualifying
employment. Once received, the RSA program officer prepares a letter in a fina l attempt to confirm
the scholar situation. The scholar will be given 30 days from the date of the letter to respond. After
the debt is confirmed for repayment or 30 days elapse, whichever comes first, the scholar is assigned
a control number and then referred to the DPMG for collection. Appropriate notes are made in the
payback report to coincide with the action taken and all documentation is maintained in the payback
files. Status updates are made between RSA program staff and DPMG until the debt is repaid. Once
RSA receives notification from the DPMG that the debt is satisfied, direction is provided by RSA
program staff to the sponsoring agency to change the scholar' s status in the MIS. Appropriate fo llow
up by RSA program staff also occurs to ensure the MIS is current and accurate. Although these
procedures have been in place since 20 l 0, we are exploring strategies to develop quality assurance
checks to determine that such procedures are in fact being followed consistently by all staff and hope
to implement such strategies by the spring of 2015.

OIG Recommendation 2.2:

Remind grantees of the consequences of scholars' failu re to obtain acceptable employment and
advise grantees that they provide such information to their scholars.

RSA Response:

We concur with the recommendation to remind grantees of the consequences of scholars ' failure to
obtain acceptable employment and advise grantees that they provide such information to their
scholars. Through communication sent via RSA's listserv, participation at national conferences, and
information disseminated via other web-based training, RSA routinely reminds grantees of the
consequences of scholars' fail ure to obtain acceptable employment. Our effort to advise grantees that
they must provide such information to the scholars also occurs during Project Directors' meetings.
Similarly, in 2011 , RSA developed a tip sheet for RSA scholars, which was shared with grantees,
that outlines the following: roles and responsibilities of the scholar, including how the scholar will
be held accountable for completing the service obl igation or for repaying the debt in the event that
qualifying employment is not obtained; processes, procedures, and required documentation to
Page 12-Ms. Weaver-Dugan



confirm qualified employment; and procedures and documentation necessary to confirm
circumstances related to repayment, deferment, and waiver requests. Lastly, guidance and the
consequences are described in detail in the payback policy and procedures manual developed for
grantees that outlines the payback regulations and service obligation requirements. Specifically, if a
scholar does not find quali fying employment after the two-year grace period, he or she will be asked
to repay the funds provided. RSA will determine whether a grantee is in violation of the grant
agreement whenever a grantee does not have the documentation necessary to prove that the scholar
signed an agreement prior to accepting the scholarship, and the scholar fails to obtain qualifying
employment.

OIG Recommendation 2.3:

Provide training to RSA program officers on identi fying scholars in repayment status in accordance
with the current RL TT program regulations and on referring noncom pliant scholars for financial
repayment.

RSA Response:

We concur with the recommendation to provide training to RSA program officers on identify ing
scholars in repayment status in accordance with the current RLTT program regulations and on
referring noncompliant scholars for financial repayment. Please see our response provided to
Recommendations 1.4 and 1.5.

OIG Recommendation 2.4:

Conduct a comprehensive assessment of the MIS to identify weaknesses and potential areas for
improvement and communicate these findings to the RL TI technical lead.

RSA Response:

We concur with the recommendation to conduct a comprehensive assessment of the MIS to identi fy
weaknesses and potential areas for improvement and communicate these findi ngs to the technical
lead for the MIS. We would add that one RSA program officer serves as a dedicated liaison between
the Training Programs Unit staff and the MIS technical lead. This individual regularly assesses the
performance of the MIS and shares any identified weaknesses or potential areas for improvement
with the MIS technical team. Likewise, other RSA program officers are afforded the opportunity to
weigh in and identify weaknesses and potential areas for improvement or recommendations in
general following the immediate review of payback reports. As mentioned in our response to
Recommendations 1.5 and 1.7, typically in April or May, RSA program officers convene to discuss
the review process, make improvements, and provide recommendations for changes to the MIS
system. These recommendations are provided to the teclmical lead, which provides technical
assistance for the MIS. Once the recommendations are reviewed for feasibi lity, the MIS technical
team develops an updated payback report fo r the upcoming reporting cycle. Any changes to the
system are subsequently reviewed by program staff during user acceptance testing, which typically
Page 13 - Ms. Weaver-Dugan



occurs in August or September. RSA program staff are committed to working closely with the MIS
technical team to enhance the MIS and reduce any potential risk to users or the federal government.

OIG Recommendation 2.5:

Work with the appropriate individuals to develop an MIS Handbook that describes the system's
features, de fines payback report flags, and is updated when enhancements are made.

RSA Response:

We concur with the recommendation to work with the appropriate individuals to develop an MIS
Handbook that describes the system' s features, defines payback report flags, and is updated when
enhancements are made. As stated in our response to Recommendation 1.1 , RSA conducts annual
web-based training for RL TT grantees that describes RSA MIS features, defines payback report
flags, and highlights updates, improvements, and enhancements to the system in a user-friendly step-
by-step format. RSA will incorporate these existing PowerPo int Slides into an MIS handbook. The
handbook will also reflect the edits made in the MIS in response to the OJG audit. This handbook
will be completed by the fall of2015.

OIG Recommendation 2.6:

Consider changing the regulation defining when a scholar enters repayment status to be consistent
with the Department's unwritten practice or simpli fy the current process, consistent with the
regu lations, to allow for situations in which a scholar may complete some part of his/her service
obligation within the required time period and, therefore, be liable for financial repayment of only a
portion of any RLTT funds received.

RSA Response:

We concur with the recommendation that it would be preferable to revise the regulations to provide
scholars more flexibility in satisfying the service obligation through employment in state YR
agencies and related agencies. Increasing the flexibility for scholars would have the desirable effect
of reducing the number of scholars who are not successful in carrying out the service obligation
within the time period allowed, thus reducing the number of scholars who would then be required to
repay all or part of the cost of the scholarship. It also would allow RSA to more effectively meet the
Congressional intent to better provide YR services to eligible individuals with disabilities by training
YR professionals and providing incentives for them to move into eligible positions in state VR
agencies and related agencies.

Other Matter:

In addition to the OIG findings and recommendations, RSA would like to respond to the issue noted
on pages 17 and 18 of the audit report that RSA may be inappropri ately granting extensions of
scholars' service obligation deadlines.
Page 14-Ms. Weaver-Dugan



RSA would like to clarify that granting extensions to scholar service obligation deadlines is not
standard practice, and any consideration that may be given to extending the service obligation
deadline is done so in the best interest of the federal government, which is to maximize the number
of VR professionals trained with federa l funds to work in state VR agencies and related agencies.
RSA program officers do not make these decisions unilaterally. They are made after consultation
with RSA management.

RSA believes that there are some situations where an extension based upon equitable considerations
would uphold the intent of the RLTT program and, at the same time, provide a better outcome for
the scholar.

RSA also recognizes the desirability of establishing a clear basis for extensions and will consult with
OGC about the best avenue for establishing a procedure for granting extensions "in the best interest
of the government". If OGC determines that regulations are required, likely as amendments to 34
CFR 386.41, RSA will regulate; if OGC believes that a policy document provides a sufficient basis,
RSA will produce one and have it approved by OGC.

Again, thank you for the opportunity to respond to the draft audit report. Should you have questions
concerning our response or need additional information, please do not hesitate to contact Dr. Thomas
Finch, Director of the Training and Service Programs Division, at (202) 245-7343.


                                              Sincerely,




                                              Acting Assistant Secretary