oversight

Talent Search Program at the U.S. Department of Education.

Published by the Department of Education, Office of Inspector General on 2005-02-16.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                        UNITED STATES DEPARTMENT OF EDUCATION
                                            OFFICE OF THE INSPECTOR GENERAL




                                                            FEB 1 6 2005

Sally L. Stroup
Assistant Secretary
Office of Postsecondary Education
1990 K Street NW
Room 7115
Washington, DC 20006

Dear Ms. Stroup:

This final audit report, Control Number ED-OIG/ A07-E0009, presents the results of our
audit of the Talent Search program at the U.S. Department of Education (Department).
The objective of our audit was to detennine, through audits of a nationwide sample of
Talent Search projects, whether the practice of overstating the funded target population is
widespread. This report is a compilation of work conducted at the Federal TRIO Office
(TRIO Office), as well as issues identified from six Talent Search project audits listed in
Attachment 1.

We determined that the TRIO Office did not maintain sufficient internal control over
Talent Search participant numbers because it did not (i) properly maintain the records and
procedures needed to readily determine the correct number of participants planned or (ii)
provide the monitoring and policy guidance needed to insure accurate reporting of
participants served. We conclude that the practice of overstating the funded target
                                l
population could be widespread. As a result, the Department may be using overstated
Talent Search participant numbers for assessing grant performances and reporting to
Congress and the general public. We recommend changes to the grant award process and
improvements in record-keeping as well as enhanced monitoring and policy guidance to
improve the accuracy of Talent Search participant numbers.

We provided the Department with the draft of this report on November 2, 2004. In its
response dated January 3, 2005, the Department indicated that action has been taken to
address each of the recommendations (response attached in full in Attachment 2). Based
on the Department's response, minor edits were made to the finding. We also provided
more information on the sample selection methodology and results.




I Due to the small number of projects audited, we cannot combine the results of our audits to reliabJy
estimate the totaJ overstatement of Talent Search participants reported as planned or served. However, we
are 90 percent confident that participant numbers were overstated in some amount for reported participants
planned (in at least 20 percent of projects) and participants served (in at least 68 percent of the projects).

                                        400 MARYLAND AVE ..    s.w.. WASHINGTON. DC      20202-1510
                                                                ",w",.ed.go.· 


           Our mission is   /0   ensure equal access to education and to promote educatiorwi exceUence throughout the nation. 

                                  AUDIT RESULTS

      Talent Search Participants, Both Planned And Served, Were Overstated

Planned Talent Search participants posted on the TRIO website were overstated for half
of the projects audited and Talent Search participants served were over-reported for all
the projects audited. Differences in the Talent Search participant numbers, both planned
and served, were noted from various Department sources for the six audited projects.
These sources are the basis for the participant numbers the Department may use for
monitoring, generating reports, and providing information to Congress and the general
public. Participants planned were overstated due in part to the reduction of initial
numbers through negotiations between the Department and applicants. The differences in
the stated number of planned participants often occurred because the reduced numbers
were not reflected on the Department's internal database or the official TRIO website. In
addition, poor maintenance of official grant files further contributed to the inconsistent
numbers. Participants served were overstated because grantees often reported
participants for whom they could not document eligibility and/or at least one eligible
service.

     Differences In Talent Search Participant Numbers From Various Department Sources
       2001–2002 Award Year                       PLANNED                           SERVED
                                         Grant        TRIO      Actual
 Grantee                              Application Website Negotiated Reported Projected *
 Luther College                                 900       850         625           610       363
 University of New Hampshire                  1,350     1,200       1,150         1,207     1,089
 Case Western Reserve University                800       600         600           605       399
 Wahupa Educational Services                  2,665     2,300       2,300         2,584     2,381
 Communities in Schools of San                  700       600         600           604       481
 Antonio
 LULAC National Educational                 20,300 14,700          12,200        15,228    13,608
 Service Centers, Inc.
 *The number of participants served are point estimates based on our statistical samples.

Participants Planned Were Routinely Reduced and Grant Records Were Not Well
Maintained. According to TRIO officials, Talent Search applicants often proposed larger
projects than could be realistically funded. For the six projects audited, the Department
reduced the ultimate award amount, which resulted in a lower participant number. Once
projects were selected, TRIO officials developed an initial list of all funded Talent Search
projects for the award period. This funded list was used as the basis for the TRIO Office
internal management database and posted on the Talent Search website. However, prior
to the 2003-2004 award year, it was common practice for the Department to continue to
negotiate the participant number through a partnership agreement. The partnership
agreement often contained another revised participant number, but the database and
website were not always updated with the revised number.

ED-OIG                                  A07-E0009                                   Page 2
The handbook for discretionary grants2 requires program staff to create and maintain an
official grant file for each application selected for funding. Program specialists are also
responsible for updating participant numbers by reporting all changes of information to
the Program Management and Development Team Leader. However, discussions with
program staff indicated that file maintenance is a low priority and TRIO Office
management has not developed specific guidance for program specialists to assure timely
and accurate updates to the website.

For three of the six projects audited, the TRIO Office had difficulty determining the exact
number of participants these projects planned to serve because the grant file for one
project was missing, and proposed revisions to the 1998–1999 partnership agreements
were incomplete for all three. In addition, we found three grantees had further reductions
that had not been updated on the TRIO website.

               Differences in Talent Search Participants Planned 2001-2002 Award Year
                                                       Grant       TRIO         Actual
    Grantee                                         Application Website Negotiation Overstated
    Luther College1                                           900      850             625        225
    University of New Hampshire                             1,350    1,200           1,150          50
    Case Western Reserve University                           800      600             600           0
    Wahupa Educational Services2                            2,665    2,300           2,300           0
    Communities in Schools of San Antonio                     700      600             600           0
    LULAC National Educational Service                    20,300 14,700            12,200       2,400
    Centers, Inc.3
    Notes: 1. Partnership agreement in file was 850, however in 1999 the grantee requested to reduce
              the participant number to 625. The program specialist sent a letter dated September 4,
              2002, while we were on site doing audit work, approving the 625 participants
              retroactively to 1998.
           2. Missing File, grantee did not have a partnership agreement.
           3. Approved participant number change not in grant file, but obtained from auditee.

It may be difficult for the TRIO Office to determine the agreed upon numbers of Talent
Search participants funded for monitoring or reporting purposes, since neither the award
database nor the official grant files were complete for the six audited projects. In
addition, during the course of our audit work, we found no evidence that the Department
had conducted on-site monitoring of the six individual projects.

The Federal TRIO Director told us that the uncertainty about the number of participants
funded no longer exists, because partnership agreements were discontinued in the 2003-
2004 award year, and grantees will be held to the original grant application. Grantees
received a letter informing them of this action. The letter requested a statement declaring
the number of participants to be served in 2003-2004; however, it was unclear whether
that number needed to be approved by a program specialist. To test the effect of the
discontinuation of the partnership agreements, we randomly selected 89 of the 471 2003-

2
    Handbook for the Discretionary Grant Process, Handbook OCFO-04, § 5.12.

ED-OIG                                        A07-E0009                                  Page 3
2004 Talent Search projects. Although one grant file was missing, only 52 of the
remaining 88 files contained a statement on the number of participants the grantee would
serve.3 Again we noted differences from the grant application and differences in the
website list, with no documentation supporting the change. We noted some program
specialists stamped approved on the participant statement, but for statements missing
from the program file, or having no notation, the number of participants funded remains
questionable. Management’s change in policy implemented during the 2003-2004 grant
award period did not establish an effective internal control to provide certainty as to the
number of Talent Search participants funded.

Participants Served Were Overstated by Grantees. None of the six grantees audited could
provide sufficient reliable documentation to support the number of participants reported
to the Department as served for the 2001-2002 award period.

        Talent Search Participants Served From Audits of 2001-2002 Grant Projects
                                                                                 Overstated
Grantee                                             Reported Served * Number             %
Luther College                                               610       363        247     40 %
University of New Hampshire                               1,207      1,089        118     10 %
Case Western Reserve University                              605       399        206     34 %
Wahupa Educational Services                               2,584      2,381        203      8%
Communities in Schools of San Antonio                        604       481        120     20 %
LULAC National Educational Service Centers, Inc.         15,228 13,608          1,620     11 %
*The numbers served are point estimates based on audit statistical samples of reported
participants.

In part, the lack of quality data is due to the Department not providing clear, concise, and
consistent guidance necessary to ensure that grantees understand and comply with
Federal regulations. Consequently, grantees reported ineligible project services;
undocumented project services; undocumented citizenship; and served non-participants,
prior to determination of participant eligibility.

Additional Departmental guidance suggested by grantee officials included

       •	   sample forms,
       •	   a data library for directors,
       •	   examples of best practices,
       •	   clarification of allowable and unallowable project services, and
       •	   clarification of adequate documentation for eligibility, i.e., project services,
            citizenship, and first generation status.

The following are two examples of inconsistent guidance the Department provided to a
grantee and the OIG.

3
    Although files were incomplete, program specialists were able to locate statements for many grants.

ED-OIG                                           A
                                                 	 07-E0009                                        Page 4
   • 	 One grantee stated that it began serving students who were neither citizens nor
       permanent residents after attending a presentation where a TRIO Educational
       Specialist at a TRIO Conference said “intent” was not defined in the regulation
       and was “open to interpretation by the individual.” The grantee concluded that
       under its interpretation, “intent” was simply getting a family’s home address
       or showing that the student was enrolled in a public school. However, the
       regulation states that an individual is eligible to participate in a Talent Search
       project if the individual is in the United States for other than a temporary purpose
       and provides evidence from the Immigration and Naturalization Service (INS) of
       his or her intent to become a permanent resident (34 C.F.R. § 643.3(a)). The
       regulation clearly states that evidence must be provided from the INS to show the
       individual’s intent to become a permanent resident.

   • 	 In February 2003, during one of our Talent Search audits, we sought official
       guidance from the TRIO Office on accepting information on first generation
       status from a minor child. A TRIO official provided a written response stating
       that first generation status should be verified by the parent or other knowledgeable
       adult. In March 2004, while we were continuing our audit work, the Federal
       TRIO Director informed us that the Department had reversed its position and
       would now accept information provided by the minor with no further verification.

The Department’s Participant Numbers May Be Overstated in Its Reports to Congress
and the General Public. The TRIO Office uses Talent Search participant numbers to
monitor the program, generate reports, and provide information to Congress and the
general public. Based on the results of our work, these participant numbers may be
overstated. Unless the TRIO Office provides consistent and reliable guidance to
grantees, and makes monitoring a priority, overstated participant numbers may continue.
Participant numbers are reported both within and outside the Department:

   •	  To respond to Congressional requests.
   •	  To produce a TRIO profile report.
   •	  To assess a grantee’s progress in meeting its approved goals and objectives.
   •	  To determine if a grantee should receive prior experience points for a new grant
       or continuation of an existing grant.
   • 	 To provide national information on project participants and program outcomes,
       i.e., Council for Opportunity in Education (COE), TRIO website.
   • 	 To calculate the average cost per participant.
   • 	 To report information on the Program Assessment Rating Tool (PART).

Recommendations:
We recommend that the Assistant Secretary of the Office of Postsecondary Education

        1.1 Modify the application process by


ED-OIG                                 A
                                       	 07-E0009                                  Page 5
           • 	 requiring grantees to propose realistic participant numbers in the grant
               application, and
           • 	 discontinuing the practice of revising planned participant numbers after the
               Talent Search grant has been awarded.

       1.2 Ensure that program staff follow policies and procedures in
           • 	 maintaining official grant files in accordance with internal guidelines,
           • 	 prescribing specific procedures for updating grant information to senior
               program staff, and
           • 	 documenting justification for any deviation from the approved grant
               application, including a change in planned participant numbers.

       1.3 Make monitoring a higher priority by conducting on-site reviews focusing on
           grantees that
           • 	 are designated high-risk,
           • 	 have not submitted required reports, i.e., single audit, annual performance
               reports, or
           • 	 have not met program objectives, including serving the funded participant
               number.

       1.4 Establish a mechanism for publishing specific, accessible, and consistent policy
           guidance that enables grantees to effectively administer their project. Guidance
           may include examples of forms and best practices, and should address important
           questions such as:
           • 	 What is an acceptable project service?
           • 	 When do you count an individual as a participant? and
           • 	 What is sufficient reliable documentation to establish individual
               participant eligibility?

Auditee Response

The Department’s response noted that, upon Office of Postsecondary Education (OPE)
and the Office of Chief Financial Officer review, some of the OIG’s recommendations,
contained in the six audits this report was based on, were modified or not sustained and
four have been resolved.

In addition, OPE expressed an overall concern with the methodology used to demonstrate
a widespread programmatic practice because the audits did not constitute either a true
random sample or a sample with the statistical power to extrapolate to the entire
population of grantees. OPE stated that it was improper to combine a purposively
selected largest project with a random sample of other projects and that a proper
“nationwide sample” should be selected by grouping auditees into size categories for
sample selection.

OPE reported that, based on the six issued audits, the Federal TRIO Director had already
implemented policy and procedural guidance to ensure that grantees do not overstate

ED-OIG                                 A
                                       	 07-E0009                                 Page 6
participant populations, but acknowledged that there were some weaknesses in
implementing the changes. OPE is establishing the following additional policy and
procedural improvements in response to the specific recommendations.

Responses to Specific Recommendations

1.1 Talent Search applicants will be required to propose realistic participant numbers in
    their grant applications and be required to meet the proposed goals and objectives
    based on funding guidelines to be published. In Spring 2004, a letter to current
    grantees informed them that they will be held to participant numbers previously
    established.

1.2 Steps have already been taken that would ensure program staff maintain official
    grant files in accordance with internal guidelines including staff training and
    inclusion in revised EDPAS agreements. In addition program staff must obtain
    approval from the Team Leader to reduce a grantees participant numbers. If
    approval is granted, it is the program specialist’s responsibility to document the
    action in the grant file and ensure that the database is updated.

1.3 The Department pointed out that monitoring has and will continue to be a high
    priority for the TRIO Program Office and there are currently no grantees designated
    as “high risk.”

1.4 The Department responded that the TRIO program office has several mechanisms in
    place to disseminate information to the TRIO community, such as the Training
    Program for Federal TRIO Programs, a recently established TRIO newsletter, and the
    TRIO Website.

OIG Comments

While the Department either modified or did not sustain some of the recommendations
for the four resolved audit reports, it did concur that the Talent Search participant
numbers reported by the grantees to the Department were not fully supported, and
therefore overstated. Of the four audits resolved, the Department fully sustained the
findings and recommendations for two audits. For the third, the Department fully
sustained the finding that participants were over-reported, but did not sustain the
monetary refund because the entity provided additional post-audit documentation. The
Department partially sustained the number of over-reported participants on the fourth, yet
chose to assess a pro-rata monetary recovery even though it agreed that the reported
participants were overstated by approximately 11 percent, and that the project failed to
meet the minimum program size by that same percentage.

We disagree that the sample did not represent a “nationwide sample” and that the results
did not support our audit objective. Our sample, representative of all Talent Search
projects, was randomly selected from a stratified universe of all funded Talent Search
projects for the 2001-2002 award year, just as suggested in OPE’s response. The largest

ED-OIG                                 A07-E0009                                   Page 7
project was not selected purposively, but was selected because it was the only project in
its stratum. A chart showing the detail of each of the strata has been added to the Audit
Objective, Scope, and Methodology section of this report (page 10). To further explain
the audit results, we have added a footnote to page 1 of this report to explain, that while
the sample was too small to project the total number of participants overstated (a variable
sample), it was sufficient to conclude that the existence of projects with overstated
participant numbers (an attribute sample) was widespread.

Specific recommendations

    1.1 We agree with the action taken by the Department.

    1.2 We agree with the action taken by the Department.

    1.3 We commend the TRIO Program Office for the number of TRIO site visits
        conducted over the last two years, and for providing program staff the necessary
        training to conduct on-site monitoring and fiscal reviews. However, it is not
        clear how many site visits were conducted at Talent Search projects. Monitoring
        and on-site reviews of Talent Search projects is crucial to ensure that the
        information submitted by grantees is complete and reliable. As shown in this
        report, four of the six projects audited failed to meet their funded participant
        number, and three of the six projects failed to serve the required minimum of
        600 participants. Therefore, the Department should continue to make
        monitoring and on-site reviews a priority focusing on grantees that have not met
        their program objectives, which includes serving the funded participant number.

    1.4 The Department listed five priorities to be covered through its Training Program
        for Federal TRIO Programs. However, the Department does not have control
        over what particular topics or materials the grantees of the Training Program
        offer the TRIO community. Further, we noted the TRIO newsletters (Fall and
        Winter) and the TRIO Website contained general information about the TRIO
        programs. We did not find information in relation to the findings or
        recommendations from the OIG Talent Search audits, the Department’s program
        site reviews, or suggestions made by grantees, such as providing examples of
        best practices, a data library for directors, or sample forms. Providing specific
        information to grantees may help reduce or eliminate these types of findings in
        the future.

                                   OTHER MATTERS

Talent Search Participant Numbers Fall Below the Minimum 600 Participant
Requirement

According to 34 C.F.R. § 643.32(b), Talent Search grantees are required to serve a
minimum of 600 participants each budget period. Three of the six grantees failed to
serve 600 participants for the budget period reviewed. If the Department is not

ED-OIG                                 A07-E0009                                  Page 8
monitoring participant numbers and is not enforcing the regulatory requirement that
grantees meet the current minimum participant number, then the regulation is
meaningless and the Office of Postsecondary Education (OPE) should consider a
regulatory change to establish a more appropriate minimum number of participants
required for an eligible Talent Search project.

Talent Search Projects Provide Services to Uncounted and Often Ineligible Students

In two of our six audits, we identified instances where services were routinely provided
simultaneously with the collection of Intake forms (participant application) without
verifying information or obtaining necessary documentation for participant eligibility,
resulting in services being provided to ineligible individuals. OPE should consider a
regulatory change to require grantees to determine eligibility prior to providing services
to individuals. This would avoid diluting the services to eligible individuals.

                                     BACKGROUND

Title IV of the Higher Education Act of 1965, as amended (20 U.S.C. § 1070a-11 and
12), authorizes the Talent Search program. The Talent Search program is governed by
the regulations in 34 C.F.R. Part 643. All regulatory citations in the report are to the
regulations in effect as of July 1, 2001, for our work on the six individual audits, and July
1, 2003, for our work at the Federal TRIO office.

In 1965, Talent Search was created as part of the Higher Education Act. It is one of nine
TRIO outreach and support programs. The goal of Talent Search is to increase the
number of youths from disadvantaged backgrounds who complete high school and enroll
in postsecondary education. The Talent Search program identifies and assists individuals
from disadvantaged backgrounds who have the potential to succeed in higher education.
The program provides academic, career, and financial counseling to its participants and
encourages them to graduate from high school and continue on to the postsecondary
institution of their choice. Talent Search also serves high school dropouts by
encouraging them to re-enter the educational system and complete their education.

Talent Search projects may be sponsored by institutions of higher education, public and
private agencies or organizations, a combination of these, and in exceptional cases,
secondary schools. Students must be between the ages of 11 and 27 and have completed
the fifth grade. In any given project, at least two-thirds of the participants must be low-
income individuals and potential first-generation college students. A veteran, regardless
of age, may participate if otherwise eligible.

The Department awarded 360 Talent Search projects in the 2001-2002 award year with
appropriations totaling $109,960,406. In the 2003-2004 award year, the Department
reports 471 Talent Search projects were awarded with appropriations totaling
$144,810,906.




ED-OIG                                  A07-E0009                                   Page 9
              AUDIT OBJECTIVE, SCOPE, AND METHODOLOGY 


The objective of our audit was to determine, through audits of a nationwide sample of
Talent Search projects, whether the practice of overstating the funded target population is
widespread. Our review focused on the Department’s procedures to assure the accuracy
of participant numbers. We considered the results of each of the six 2001-2002
individual audits. (See Attachment 1.) In addition, we randomly selected 89 of 471
2003-2004 Talent Search grant files for review.

To achieve our overall audit objective we:

   • 	 reviewed applicable laws and regulations governing the Talent Search program;
   • 	 conducted interviews with program officials and staff in the Talent Search office
       located in Washington, DC;
   • reviewed a sample of the Department’s current Talent Search grant files;
   • obtained and analyzed documentation related to Talent Search projects; and
   • 	 audited six Talent Search projects.

We established four strata for the 360 Talent Search projects from the TRIO website
based on the number of participants reported ranging from 480 to 14,700. We selected
the largest project (in its own stratum) and randomly selected the remaining five projects
from the remaining three strata.

                                 Number of Projects           Number of
              Population          (Based on Talent         Projects Selected
                Count             Search Website)          from each Strata
                                                              for Sample
               14,700                      1                       1
            1,001 – 2,500                 64                       2
             700 – 1,000                 166                       1
                < 700                    129                       2

The primary objective for each of the individual audits was to determine, through a
random sample of participants from each project, if each grantee had administered its
Talent Search grant in accordance with the law and Talent Search regulations governing
the documentation of participant eligibility (34 C.F.R. § 643.32). A secondary objective
was to determine if each of the individual grantees met the two-thirds assurance
requirement for serving low-income and first-generation participants. The Office of
Inspector General (OIG) issued individual audit reports to each Talent Search grantee.
(See Attachment 1 for details.)

We relied on computerized records to determine the number of Talent Search projects
funded for the 2003-2004 award year. Based on our assessments, we concluded that the
data used was sufficiently reliable for the purpose of our audit.



ED-OIG                                 A
                                       	 07-E0009                                 Page 10
An entrance conference was held at the Federal TRIO Office located in Washington, DC
on November 19, 2003. Fieldwork was conducted during the weeks of November 17-21,
2003, and May 3-7, 2004. An exit conference was held on September 28, 2004, with
Office of Postsecondary Education officials, including Federal Talent Search officials.

This audit and each of the six individual audits were conducted in accordance with
generally accepted government auditing standards appropriate to the scope of each audit.


                     STATEMENT ON INTERNAL CONTROLS

As part of our review we assessed the system of internal controls, policies, procedures,
and practices applicable to the Department’s administration of the Talent Search program
relevant to the scope of our review. Our assessment included a determination of whether
the Federal TRIO Office provided a reasonable level of assurance that the number of
Talent Search program participants funded and served is being appropriately recorded
and reported.

For the purpose of this report, we assessed and classified the significant controls into the
following categories:

ƒ   Participant numbers funded during the 2001-2002 and 2003-2004 award periods; and
ƒ   Participant numbers reported as served during the 2001-2002 award period.

Because of inherent limitations, a study and evaluation made for the limited purpose
described above would not necessarily disclose all material weaknesses in internal
control. However, our assessment disclosed significant internal control weaknesses,
which adversely affected the Talent Search program office’s ability to record and report
accurate Talent Search participant numbers for the periods of our review. These
weaknesses and their effects are fully discussed in the AUDIT RESULTS section of this
report.

                            ADMINISTRATIVE MATTERS

Corrective actions proposed and implemented by your office will be monitored and
tracked through the Department’s Audit Accountability and Resolution Tracking
System (AARTS). Department policy requires that you develop a final corrective
action plan (CAP) for our review in the automated system within 30 days of the
issuance of this report. The CAP should set forth the specific action items, and
targeted completion dates, necessary to implement final corrective actions on the
findings and recommendations contained in this final audit report.

In accordance with the Inspector General Act of 1978, as amended, the Office of
Inspector General is required to report to Congress twice a year on the audits that remain
unresolved after six months from the date of issuance.


ED-OIG                                  A07-E0009                                   Page 11
In accordance with the Freedom of Information Act (5 U.S.c. § 552), reports issued by
the Office of Inspector General are available to members of the press and general public
to the extent infonnation contained therein is not subject to exemptions in the Act.

We appreciate the cooperation given us during this review. If you have any questions,
please call Janice Keeney, Assistant Regional Inspector General for Audit, at 816~268 ~
0500 or Richard J. Dowd, Regional Inspector General for Audit, at 312-886-6503.


                                    Sincerely,


                                   J(e~W
                                    Helen Lew
                                    Assistant Inspector General
                                    for Audit Services




Attachments




ED-OIG                                 A07-E0009                                  Page 12
                                                                                         Attachment 1

OIG REPORTS FOR TALENT SEARCH PROJECTS AUDITED WITH SUMMARY OF
                      AUDIT FINDING ISSUES
 Audit                   Auditee                OIG Final Findings
Control                                           Report
Number                                            Issued
A07-         Luther College                                                     March 28,         a, b, d, e
C0031        http://www.ed.gov/about/offices/list/oig/auditreports/a07c0031.pdf 2003
A07-         University of New Hampshire                                        January 15,       a, d, e
D0001        http://www.ed.gov/about/offices/list/oig/auditreports/a07d0001.pdf 2004
A07-         Case Western Reserve University                                    July 11, 2003     a, b, c, d,
D0002        http://www.ed.gov/about/offices/list/oig/auditreports/a07d0002.pdf                   e, f, i
A07-         Wahupa Educational Services                                        November          a, c, d, e, h
D0009        http://www.ed.gov/about/offices/list/oig/auditreports/a07d0009.pdf 25, 2003
A07-         Communities in Schools of San Antonio                              January 29,       a, b, c, d, e
D0015        http://www.ed.gov/about/offices/list/oig/auditreports/a07d0015.pdf 2004
A07-         LULAC National Educational Service Centers, Inc.                   June 18,          a, c, d, e, g
D0024        http://www.ed.gov/about/offices/list/oig/auditreports/a07d0024.pdf 2004


Legend – Findings

a: Reported participant numbers were overstated.

b: Did not meet the minimum (600) participant number.

c: Did not adequately document citizenship status, i.e., ineligible participant.

d: Did not adequately document project services.

e: Counted ineligible project services, i.e., mailings, newsletters, administrative activity.

f: Prior period participant – no services in 2001–2002.

g: Counted duplicate participants.

h: Did not meet two-thirds assurance requirement.

i: No written policies and procedures.




   ED-OIG                                      A07-E0009                                        Page 13
                      UNITED STATES DEPARTMENT OF EDUCATION

                                     OFFICE OF POSTSECONDARY EDUCATION                                 Attachment 2

                                                                                                   THE ASSISTANT SECRETARY




MEMORANDUM

DATE:                                                                                        By

TO:                Richard J. Dowd
                   Regional Inspector General for Audit

FROM:              Sally L. StrouIiJ)

SUBJECT:           Comments on Draft Audit Report of the Talent Search Program (Control
                   Number ED-OIG /A07-E0009)

We have reviewed the draft audit report of the Talent Search Program at the U.S.
Department of Education (Department). The primary objective of the audit was to
determine, through audits of a nationwide sample of Talent Search projects, whether the
practice of overstating the target population was widespread .

This draft audit report is based on six audits of 200 \-02 Talent Search grants, the first of
which was completed in April 0[2003 and the last in June 0[2004. The Office of
Postsecondary Education has previously responded to the findings in each of these audits.
It is important to note that, upon OPE and the Office of Chief Financial Officer review,
some of the Office of Inspector General's (OIG's) recommendations were modified or
not sustained. In fact, four of the audits have already been resolved . Further, based on
the information the OIG previously provided to OPE, some corrective actions have
already been taken on a program-wide scale and further program improvements are
planned.

While OPE's comments to the individual findings and recommendation of this draft audit
report are provided below, we wanted to bring to your attention our overall concern with
the methodology used for the stated objective of this audit. The OIG has attempted to
determine whether a programmatic practice is widespread using a group of audits that do
not constitute either a true random sample or a sample with the statistical power to
extrapolate to the entire population of grantees. The report states that "a nationwide
sample" was used. This wording does not provide any statistical information and will
likely be misleading to the general population, in that it implies there is validity in
applying the sample to the national program. Actually, given the methodology used, the
audit objective is not attainable. In general, it appears that the determinations in this
audit provide little value, ifany, beyond what has already been achieved through the
original six audit reports.

                                     1990 K STREET, N.W. WASHINGTON, D.C. 20006

      Our mission is to enslJ.re eqlUl! access to edlJ.cntion and to promote educntional excellence throughout die Nation.
Page 2 - Richard J. Dowd



If the OIG intends to publish the audit anyway, it should expand upon stated limitations
of the methodology and do this at the very beginning of the report. The draft audit report
now states that "Due to the small number of projects audited, we cannot combine the
results of our audits to reliably estimate the total overstatement of Talent Search
participants reported as planned or served." We believe that the reader should be advised
of the following:

    • 	 The six audits do not constitute a sampling that is reflective of the programs'
        grantees as a whole. The OIG audited the largest Talent Search project, along
        with 5 other randomly selected projects from the 360 projects funded in 2001-02
        (less than 2 percent of the Talent Search projects). It is not appropriate to
        combine the findings from the largest Talent Search project with the findings
        from the 5 randomly selected projects. The largest project was purposively
        selected, while the others are a random sample. A truly random sample cannot
        include any purposively selected projects.

   • 	 Talent Search projects vary considerably in size. A sample of 5 does not provide
       the statistical power to allow one to extrapolate to the entire universe of projects.
       A stronger sampling methodology would be to carefully examine the universe of
       projects and group them into appropriate size categories. For example, if a large
       proportion of projects serve 1,000-5,000 participants, that would be one size
       category from which to select a sample of projects to audit.

Given the weaknesses with this audit methodology, OPE strongly questions the validity
of including the section of this draft report entitled: "The Department's Numbers May
Be Overstated in Its Reports to Congress and the General Public." This section provides
a listing of uses for the TRIO information reported. Given that the OIG has not clearly
identified and documented that there is currently any program-wide abuse, the logic for
this section is questionable.

FINDING NO. ]: Talent Search Participants, Both Planned And Served, \Vere
Overstated

The Office of the Inspector General (OIG) states that planned Talent Search participants
numbers posted on the TRIO Web site were overstated for half(three) of the projects
audited (a total of six) and that the number of Talent Search participants actually served
were over-reported for all of the projects audited. The OIG further states that the TRJO
Office did not maintain sufficient internal control over Talent Search participant numbers
because it did not (1) properly maintain the records and procedures needed to readily
determine the correct number of participants to be served, or (2) provide grantees with
monitoring and policy guidance needed to insure accurate reporting of the number of
participants served. The OIG concluded that the practice of overstating could be
widespread. (For reasons stated above, this last statement is speCUlative rather than based
on sound statistical methodology.)
Page 3 - Richard J. Dowd


Prior to the drafting of this audit report, the Federal TRIO Director had already
implemented policy and procedural guidance to ensure that grantees do not overstate the
participant population, as follows:

   • 	 First, he discontinued partnership agreements in the 2003-2004 Award Year.
       Partnership agreements allowed the Department to continue to negotiate student
       participant numbers during the grant period.
   • 	 Second, the Federal TRIO Director has established an internal process that now
       holds grantees accountable based on the number of participants specified in the
       original grant application.
   • 	 Third, the grantee community has been informed of these changes.

The OIG did identify some weaknesses in implementing these changes. However, it is
not unusual for programmatic and procedural changes to encounter some problems in the
first stage of implementation. The TRIO Program Office is currently establishing
additional policy and procedural improvements to ensure that grantees provide accurate
participant numbers. We elaborate further on these actions in our responses to specific
recommendations.

As a result of its audits, the OIG recommended changes to the grant award process and
improvements in recordkeeping, as well as enhanced monitoring and policy guidance to
improve the accuracy of Talent Search participant numbers .

RECOMMENDATION

1.1 	The OIG recommended that Assistant Secretary for the Office of Postsecondary
     Education modify the application process by:

       • 	 Requiring grantees to propose realistic participant numbers in the grant
           application, and
       • 	 Discontinuing the practice of revising planned participant numbers after the
           Talent Search grant has been awarded.

RESPONSE

1.1 	The TRIO Program Office will require Talent Search applicants to propose realistic
     participant numbers in their Talent Search grant application. The Department has
     already begun planning for the upcoming competition for grants with a projected
     application due date ofiate summer or early fall. Applicants will be required to meet
     the goals and objectives they propose in their application based on funding guidelines
     to be published in the Federal Register notice inviting applications. Current grantees
     will be informed that they will receive no more than a fixed percentage over their
     prior year's award amount. New applicants will be informed of the maximum award
     amounts and the minimum number of participants they must serve.
Page 4 - Richard 1. Dowd


   In addition the TRIO Program Office has taken steps to ensure that current grantees
   meet their participant numbers. For example, in the spring of 2004, a letter was
   mailed to grantees with continuation grant awards infonning them that the project
   will be held to participant numbers previously established for the grant period. Also,
   the TRIO Program Office has implemented a policy that will deny Prior Experience
   points to grantees that have failed to meet their planned participant numbers.

RECOMMENOAnON

1.2 The OIG recommended that Assistant Secretary for the Office of Postsecondary
    Education ensure that program staff follow policies and procedures in:

       • 	 Maintaining official grant files in accordance with internal guidelines,
       • 	 Prescribing specific procedures for updating grant infonnation to senior staff,
           and
       • 	 Documenting justification for any deviation from the approved grant
           application, including a change in planned participant numbers.

RESPONSE

1.2 The TRIO Program Office has already taken the following steps to ensure that
    program staff will maintain grant files in accordance with internal guidelines:

       • 	 In the spring of 2004, the team responsible for administering the Talent
           Search Program was trained on the proper procedures for maintaining grant
           files. The training was based on the Grants Policy and Oversight (GPOS)
           Bulletin #2, "Organization of the Official File Folder." Each member of the
           staff was provided with a one·page handout titled, "Official Grant File
           Organization Guide."
       • 	 The EDPAS agreements for staff have been revised to include language
           regarding grant file maintenance. The Team Leader and the Director have
           advised staff regarding the importance of keeping files up-to·date. It is
           anticipated that the e-Monitoring system will also be a tool for assisting staff
           in the efficient maintenance of project files.
       • 	 The staff has been infonned that any request to reduce participant numbers
           must be approved by the Team Leader. The Team Leader will follow the
           Director 's guidance regarding grantee requests to revise participant numbers.
           If a grantee received approval to revise the participant numbers, the assigned
           program specialist must document the action in the grant file and ensure the
           database is updated.
Page 5 - Richard J. Dowd



RECOMMENDATION

1.3 	 The OIG recommended that Assistant Secretary for the Office of Postsecondary
      Education make monitoring a high priority by conducting on·site reviews focusing
      on grantees that:

       • 	 Are designated high risk,
       • 	 Have not submitted required reports, single audit, annual performance reports,
           or
       • 	 Have not met program objectives, including serving the funded participant
           number.

RESPONSE

1.3 	 Monitoring has been and will continue to be a high priority for the TRIO Program
      Office. In the last year, TRIO program staff and Program Oversight Staff (POS)
      conducted approximately 100 site visits. Many of the visits were to non.profit
      agencies that administer Talent Search projects. In the prior year, TRIO and POS
      staff conducted approximately 100 site visits of TRIO grantees. In addition, in
      November 2003, TRIO program staff participated in training for on·site monitoring
      and fiscal review techniques. OPE developed a reference manual for the fiscal
      review of grant projects and updated monitoring guides for several of the TRIO
      Programs including Talent Search.

    Currently, no Talent Search grantees are designated as "high risk" in accordance with
    the standards published in 34 CFR 74.14 of the Education Department General
    Administrative Regulations (EDGAR). However, the TRIO Program Office uses a
    variety of indicators to determine whether the Federal funds awarded to a particular
    grantee may be "at risk." The indicators include referrals from the Office of the
    Inspector General, complaints with merit, failure to submit required performance and
    other reports, "problem" audits, and poor performance. All relevant factors are
    considered in determining which grantees may require increased monitoring and/or a
    site visit.

RECOMMENDAT ION

1.4 The OIG recommended that Assistant Secretary for the Office of Postsecondary
    Education estab lish a mechanism for publish ing specific, accessible, and consistent
    policy guidance that enables grantees to effectively administer their projects.
    Guidance may include examples of forms and best practices, and answer questions
    such as:

       • 	 What is an acceptable project service?
       • 	 When do you count an individual as a participant?
Page 6 - Richard J. Dowd


       • 	 What is sufficient reliable documentation to establish individual participant
           eligibility?

RESPONSE

1.4 The TRIO Program Office already has several such mechanisms in place. One of the
    mechanisms for disseminating information to TRIO staff is through the Training
    Program for Federal TRIO Programs. This program provides training and policy
    guidance to staff and leadership personnel employed in or preparing for employment
    in projects funded under the Federal TRIO Programs. In fiscal year 2004, 13 new
    Training Program grants were awarded to train an estimated 3,688 staff in the
    following five areas:

       • 	 Priority 1: Training to improve: budget management, recordkeeping and
           reporting student and project perfornlance and the evaluation of project
           perfornlance.
       • 	 Priority 2: Training on: the legis lative and regulatory requirements for
           operat ion of the Federal TRIO programs; personnel management; and student
           aid.
       • 	 Priority 3: Training on: counseling, and retention and graduation strategies.
       • 	 Priority 4: Training to coordinate project activities with other available
           resources and activities, and training to design and operate a model TRIO
           project.
       • 	 Priority 5: Training in the use of educational technology.

    Additionally, in the fall of2004, the TRIO Program Office established a quarterly
    newsletter for the purpose of sharing information and providing program guidance.
    The Winter 2004 issue of the TRIO newsletter discusses, among other things, the
    Training Program for TRIO Programs, reiterates the five training priorities, and
    announces the new recipients of the Training Program grants along with contact
    information.

    Finally, the TRIO Web site provides a wealth of information on the TRIO programs
    and is the primary mechanism for providing information about each of the TRIO
    programs including: program and funding requirements; regulations; law;
    performance reports; and other information. For example, the Talent Search Home
    Page lists the services that can be provided under the Talent Search Program and
    provides links to the program statute, regulations, etc.

    It is also important to note that senior TRIO staff routinely participate in policy
    seminars, state. regional, and national meetings hosted by grantees for the purpose of
    providing program information and policy guidance to grantees.

Thank you for the opportunity to review and comment on this draft audit report. If you
have questions about any of our comments, feel free to contact Larry Oxendine, Director,
Federal TRIO Programs at (202) 502-7600 or Larry.Oxendine@ed.gov.