oversight

Audit of procedures at Federal Student Aid for monitoring the Ability-to-Benefit test publishers approved by the U.S. Department of Education (ED): American College Testing, The College Board, CTB/McGraw-Hill, and Wonderlic, Inc..

Published by the Department of Education, Office of Inspector General on 2002-08-22.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                            UNITED STATES DEPARTMENT OF EDUCATION

                                               OFFICE OF INSPECTOR GENERAL



                                                                                                        ED-OIGI A03-BOOO1
                                                        AUG 22 2002
Mr. James Manning
Acting Chief Operating Officer
Federal Student Aid
U.S. Department of Education
Union Center Plaza Building
830 151 Street, NE, Room 112Gl
Washington, DC 20202

Ms. Sally Stroup
Assistant Secretary for Postsecondary Education
u.s. Department of Education
Room 7115
1990 K Street, NW
Washington, DC 20006

Dear Mr. Manning and Ms. Stroup:

This Final Audit Report (Control Number ED-OIG/A03-BOOOl) presents the results of our audit
procedures at Federal Student Aid (FSA) for monitoring the Ability-to-Benefit (A TB) test
publlshers approved by the U.S. Department of Education (ED): American College Testing
(ACT), The College Board, CTB/McGraw-Hill, and Wonderlic, Inc. (Wonderlic).

A draft of this report was provided to FSA and the Office of Postsecondary Education (OPE). In
their joint response, FSA appears to concur with our recommendations for Finding Nos. 1 and 2.
However, OPE did not concur with our recommendation for Finding No.3. We summarized the
responses after each finding, and a copy of the complete response is provided as an attachment to
this report.

                                                 AUDIT RESULTS

The objective of our audit was to detclmine and evaluate FSA's monitoring methods for ED­
approved ATB test publishers for the period of July 1, 1997, through June 30,2000. During our
audit we found that­

• 	 FSA does not have an effective monitoring system in place to ensure that ED-approved ATB
    test publishers comply with applicable laws and regulations and with the terms of their
    agreements with the Secretary;

• 	 The agreement between ED and ACT for its Assessment test does not meet Federal
    requirements; and

                                   400 MARYLAND AVE., S.W. WASHINGTON, D.C. 20202-1510

           Our mission is to ensure equal access to education and to promote educational excellence throughout the Nation.
Audit of FSA' s Controls Over ED-Approved ATB Programs                       ED-OIG/A03-BOOOI


• 	 Federal regulations need to be improved to clearly establish accountability when a student
    receives funds under Title IV of the Higher Education Act of 1965, as amended (REA), on
    the basis of an improper ATB test administration.

Finding No.1           Oversight of ED-Approved ATB Test Publishers Needs Improvement

FSA does not have an effective monitoring system in place to ensure ED-approved ATB test
publishers comply with applicable laws and regulations and with the terms of their agreements
with the Secretary. ED-approved ATB test publishers failed to comply, and continue to fail to
comply, with the terms of their ATB agreements and with applicable laws and regulations, and
as a result, Title IV, REA funds may be disbursed to ineligible students.

Section 484( d) (1 ) of the REA states-

       In order for a student who does not have a certificate of graduation from a school
       providing secondary education, or the recognized equivalent of such certificate, to
       be eligible for any [Title IV program] assistance ... [t]he student shall take an
       independently administered examination and shall achieve a score, specified by
       the Secretary, demonstrating that such student can benefit from the education or
       training being offered. Such examination shall be approved by the Secretary on
       the basis of compliance with sllch standards for development, administration, and
       scoring as the Secretary may prescribe in regulations.

The initial list of eight approved ATB tests was published in the Federal Register, on October 25,
1996 (61 FR 55542). Efforts by FSA to develop management controls over test publishers'
administration of their ATB programs did not begin until February 2002, after the five-year
approval period for the initial list of ATB tests had expired. In February 2002, FSA established a
system for receiving and logging test publishers' three-year analyses of ATB test scores and for
sending reminder notices to test publishers, to notify publishers when the test approval is
scheduled to expire. Our audit did not include a review of this new system.

OMB Circular A-123 (A-123) requires Federal agencies and managers to develop and implement
management controls that reasonably ensure that laws and regulations are followed. In addition,
the United States General Accounting Office's "Standards for Internal Control in the Federal
Government" states, "Internal control should generally be designed to assure that ongoing
monitoring occurs in the course of normal operations."

FSA's procedures do not ensure that ATB test publishers comply with their agreements
with the Secretary.

Under 34 CFR. § 668.150(a), "If the Secretary approves [an ATB test], the test publisher mllst
enter into an agreement with the Secretary that contains the provisions set forth in paragraph (b)
of this section before an institution may use the test to determine a student's eligibihty for Title
IV, I-IEA program funds." We found that ED-approved ATB test publishers did not comply with
the criteria in 34 C.F.R. § 668.150(b), as described below:




                                                 2

Audit of FSA's Controls Over ED Approved ATB Programs                        EDOIG/A03-BOOOl


• 	 Under 34 C.F.R. § 66S.150(b)(3), a test publisher must-

       Decertify a test administrator for a period that coincides with the period for which
       the publisher's test is approved if the test publisher finds that the test
       administrator­
              (i) 	    Has repeatedly failed to give its test in accordance with the
                       publisher's instructions;
              (ii) 	 Has not kept the test secure;
              (iii) 	 Has compromised the integrity of the testing process; or
              (iv) 	 Has given the test in violation of the provisions contained in
                       § 66S.151 ....

       The process that ACT uses to certify an independent test administrator (IT A) for its
       Career Programs Assessment Test (CPAt) does not provide adequate assurance that the
       ITA is not affiliated in any way with all of the institutions for which he or she may
       perform CPAt ATB testing.

       ACT does not have any procedures in place to monitor the testing activity of ITAs for its
       CPAt. Our review of ACT's file documentation for a sample of 25 examinees from the
       universe of 629 examinees that retested on the same CPAt form at the same institution in
       consecutive months during the period July 1, 1997, through June 30, 2000, revealed that
       20 (SO percent) were retested improperly. Also, our analysis of the universe of 9,179
       examinees who were administered a retest of the CPAt examination during the period
       July 1, 1997, through June 30,2000, identified approximately 400 instances in which
       examinees were improperly tested twice on the same CPAt form, during the same month,
       at the same institution.

       Wonderlic did not begin to monitor the retesting activity of IT As for its Wonderlic Basic
       Skills Test (WBST) until January 2001, over four years after the test was approved for
       ATB purposes. We identified 1,270 applicants who passed an improper retest of lhe
       WBST for the period July 1, 1997, through November 12, 2000. Of the 1,270 applicants,
       724 received $3,362,839 in Title IV, HEA funds according to the National Student Loan
       Data System (NSLDS).

• 	 Under 34 c.F.R. § 66S.1S0(b )(5)(iii), a test publisher must, "provide the test administrator
   with software that will ... [p]rohibit any changes in test taker responses or test scores."

       The College Board's Windows and DOS versions of its Accuplacer test allow test
       administrators to modify test takers' database records including test scores.

• 	 Under 34 c.F.R. § 66S.150(b)(7), a test publisher must, "[k]eep for a period of three years
    each test answer sheet or electronic record forwarded for scoring and all other documents
    forwarded by the test administrator with regard to the test ...."

1 Our review revealed 410 cases where this set of circumstances occurred. We did not report the
exact number because the test date was not always accurate, as discussed later in this finding,
and because we did not confirm or sample the test dates to the original records.

                                                 3
Audit ofFSA's Controls Over ED-Approved ATB Programs                          ED-OIGI A03-BOOO 1


       The ACT Assessment User Handbook states that: "ACT keeps students' original
       registration folders and answer documents for one year." Also, according to information
       for the Assessment test on ACT's website (www.act.org), and confirmed by ACT's
       Assistant Vice President of Applied Research, examinees may direct ACT to drop any or
       all of their Assessment records from ACT's files at any time.

• 	 Under 34 C.F.R. § 668.1S0(b)(8), a test publisher must, "[t]hrce years after the date the
    Secretary approves the test and for each subsequent three-year period, analyze the test scores
    of students to determine whether the test scores produce any irregular pattern that raises an
    inference that the tests were not being properly administered, and provide the Secretary with
    a copy of this analysis ...."

       ACT and the College Board were required to submit analyses of student A TB test scores
       to ED for their approved ATB tests. ACT was required to submit analyses for its CPAt,
       Compass, and Asset tests, and the College Board was required to submit analyses for its
       Accuplacer tests and Descriptive Tests of Language and Mathematical Skills. However,
       ACT's and the College Board's analyses for these tests were submitted late. The
       analyses were due no later than October 1999, but ED did not receive the analyses until
       January 2001.

       ACT's analysis of CPAt A TB test scores did not contain the true CPAt population.

       The College Board's analyses of ATB test scores for both its Accuplacer test and its
       Descriptive Tests of Language and Mathematical Skills did not contain true test score
       populations.

       ACT docs not identify Assessment tests that are administered for ATB purposes.
       Therefore, ACT will not be able to prepare the required analysis of Assessment ATB test
       scores. On February 14,2002, ACT submitted its three-year analysis to FSA. Our
       review of the analysis revealed that its data is insuffjcient and that the submission does
       not meet the criteria in 34 C.F.R. § 668.1S0(b)(8).

:FSA's procedures do not ensure that ATB test publishers administer only approved
editions of A TB tests.

A test publisher that wishes to have its test approved by the Secretary for ATB purposes must
submit an application to the Secretary. Under 34 C.P.R. § 668. 144(c)(9)­

       A test publisher shall include with its application ... [i]f a test has been revised
       from the most recent edition approved by the Secretary, an analysis of the
       revisions, including the reasons for the revisions, the implications of the
       revisions for the comparability of scores on the current test to scores on the
       previous test, and data from validity studies of the test undertaken subsequent to
       the revisions ....




                                                 4

Audit of FSA' s Controls Over ED-Approved ATB Programs                        ED-OIG/A03-BOOOI


ACT's Assistant Vice President of Applied Research informed us that. for security purposes, a
new edition of ACT's Assessment test is developed each time it is administered. An application
for approval of each new edition of the test is not submitted to ED for approval.

FSA's procedures do not ensure that ATB test publishers maintain complete and accurate
A TB test records.

The test date field maintained in ACT's CP At database is a four-digit numelic field that includes
only the month and year data for test administrations. Our review of ACT's file documentation
for 95 of 73,455 CPAt ATB tests conducted during the period July 1,1997, through June 30,
2000, revealed that the test date recorded on 9 of the 95 CPAt ATB test answer sheets reviewed
(9.5 percent) did not agree with the test date in the CPAt database. In eight instances the wrong
test month was recorded in the CP At database, and in one case the wrong test year was recorded.
Sound business practice requires that data collection and recording procedures be reliable and
accurate.

Recommendations:

We recommend that the Chief Operating Officer (COO) for FSA­

1.1 	   Establish and implement control activities and monitoring and technical assistance
        strategies to ensure ED-approved ATB test publishers comply with applicable laws and
        regulations and with the terms of their agreements with the Secretary; and

1.2 	   Ensure that test publishers improve their processes for identifying and reporting retest
        errors, to ensure that institutions have accurate and timely information at the time that
        eligibility determinations are made.

FSA's Reply:

FSA appears to concur with Recommendation 1.1. FSA's response states-

        FSA has developed a tracking system to ensure that test publishers comply with
        the regulatory reporting requirements, and we will follow up on any test score
        irregularities that are identified by the test publishers. We recognize that the
        OIG's findings were made only after conducting on-site reviews of the acti vities
        of test publishers, and FSA is committed to conducting similar on-site reviews for
        those test publishers that have not already been reviewed by OIG. We will also
        provide technical assistance on the ATB regulatory requirements through regular
        contacts with these publishers, and we may conduct follow-up reviews, as needed.

FSA also appears to generally concur with Recommendation 1.2. FSA's response states that test
publishers do not necessarily receive submitted ATB test answer sheets in the chronological
order in which they were administered to students. As a result, in retest situations, a test
publisher can never guarantee that the official score report truly reflects complete and proper
compliance with its retesting rules and procedures. FSA's response explains­
Audit of FSA' s Controls Over ED-Approved ATB Programs                       ED-OIG!AOi-BOOO 1


       This outcome could only be prevented by requiring "real time" electronic
       submissions of the completed tests or requiring the school to delay disbursement
       of student aid funds for a period of time sufficient to eliminate the problem of test
       reslI Its being sent out hy the test puhlisher in reverse order. The "real time"
       submission requirement would be very costly, and could discourage test publisher
       participation in the ability-to-benefit testing process at a time when the number of
       participating test publishers is very limited.

OIG's Response:

Concerning FSA's reply to Recommendation 1.1, we concur with FSA's plans to conduct
additional on-site reviews; however, those reviews should be conducted on a periodic basis
rather than a one-time basis. FSA's planned actions must ensure it will conduct on-site reviews
at test publishers on an ongoing cyclical basis.

Concerning FSA's reply to Recommendation 1.2, we acknowledge that, without a "real tIme"
test submission process, answer sheets for a student who takes an ATB test more than once
within a short time period may not be scored in the sequential order in which the tests were
taken. Consequently, in a very small number of cases, a school may receive a passing test result
for a student who was not retested properly. If this occurs, FSA must ensure that the test
publisher has procedures to identify the retesting violation quickly and to notify the school
immediately that the passing test score is invalid.

Finding No.2 	         ED Entered into an Agreement with ACT that Does Not Meet
                       Regulatory Criteria

ACT's Assessment test is a national college admission examination designed to measure high
school students' general educational development and their ability to complete college level
work. It is administered on five national test dates each year and covers four content areas:
English, mathematics, reading, and science reasoning. ED approved the ACT Assessment test
for ATB testing purposes on October 27, 1998.

Under 34 c.F.R. § 66S.1S0(a), "If the Secretary approves a test under this subpart, the test
publisher must enter into an agreement with the Secretary that contains the proviSions set forth in
paragraph (b) of this section before an institution may use the test to determine a student's
eligibility for Title IV, HEA program funds." The agreement entered into by ED and ACT for
the use of the Assessment test for ATB purposes does not meet all of the criteria required by 34
C.F.R. § 668.1S0(b).

• 	 Under 34 c.F.R. § 668.1S0(b)(3), if an independent test administrator commits certain
   violations. he or she must be decertified by the test publisher "for a period that coincides with
   the period for which the publisher's test is approved ...." For the same violations, the
   agreement between ED and ACT states only that ACT "will not ship test materials to [the]
   test administrator for a specified period .... Shipment of tests will resume only after
   corrections to procedures have been thoroughly documented and approved."




                                                 6

Audit of FSA' s Controls Over ED-Approved ATB Programs                        ED-OIG/A03-BOOOl


• 	 Under 34 C.F.R. § 668.150(b)(6), the test publisher must "rplromptly send to the student and
    the institution the student indicated he or she is attending or scheduled to attend a notice
    stating the student's score for the test and whether or not the student passed the test ...."
    The agreement between ED and ACT lacks requirements for timeliness of reporting test
    scores and does not require the determination of a passing grade. The agreement states only
    that ACT "will generate score reports for answer sheets that it receives from test
    administrators."

• 	 Under 34 c.F.R. § 668.1S0(b)(7), the test publisher must "[k]eep for a period of three years
    each test answer sheet or electronic record forwarded for sCOIing and all other documents
    forwarded by the test administrator with regard to the test ...." The agreement between ED
    and ACT states only that ACT "will keep for a period of two years each test answer sheet or
    electronic record forwarded for scoring."

• 	 Under 34 C.F.R. § 668.1S0(b)(8), a test publisher must, "[t]hree years after the date the
    Secretary approves the test and for each subsequent three-year period, analyze the test scores
    of students to determine whether the test scores produce any irregular pattern that raises an
    inference that the tests were not being properly administered, and provide the Secretary with
    a copy of this analysis ...." The agreement between ED and ACT does not include this
    requirement.

Recommendation:

2.1 	   We recommend that the COO for FSA either withdraw ED's approval of the ACT
        Assessment test or revise the agreement between ED and ACT to ensure that all criteria
        for test publishers in Subpart J of 34 c.F.R. Part 668 are met, including but not limited to
        the requirements in 34 C.F.R. § 668.150.

FSA's Reply:

FSA appears to concur with our recommendation. FSA's response states that ACT has decided
to withdraw the use of the ACT Assessment scores for ATB purposes.

Finding No.3 	         Federal Regulations Regarding Accountability for Title IV, HEA
                       Program Funds Received by Students who Pass Improper A TB Test
                       Administrations Need Improvement

The OIG audit report, "Audit of All-State's Procedures for Administering ATB Tests" (Control
Number ED-OIG/A03-BOOI4), disclosed that the ITA at All-State Career School (All-State) did
not always comply with Wonderlic's procedures for administering retests of the WBST. The
audit found that during the period July 1, 1997, through November 12, 2000, 12 students who
received $57,994 in Title IV, REA program funds at All-State were improperly admitted to the
institution after passing a WBST that was not conducted in accordance with the publisher's
established procedures for retesting. The report recommended that the COO for FSA require
All-State to repay the $57,994 in Title IV, REA grants and loans made to the 12 students.




                                                 7

Audit ofFSA's Controls OverED-Approved ATB Programs                            ED-OIG/A03-BOOOI


All-State responded to the OIG audit and indicated that it did not concur with the finding and
recommendation. The OIG reviewed All-State's comments but detennined that its position
remained unchanged. The audit was then referred to the Case Management and Oversight
Division (CMO) of FSA for final resolution. In its final audit determination, CMO determined
that the recommended liability for the finding is not supportable.

CMO's final audit determination for the finding states-

        Section 484 (d) (1) of the Higher Education Act of 1965, as amended, (HEA)
        provides that students who do not have a high school diploma or its recognized
        equivalent, or who are not home schooled, may receive Title IV HEA program
        funds if they take an ability-to-benefit test approved for that purpose by the
        Secretary, and achieve a passing score on that test that was also approved by the
        Secretary....

        Section 668.154 describes the circumstances under which an institution is liable
        for disbursements made to ineligible students where the issue of ineligibility turns
        on whether the student was properly determined to be an "ability-to-benefit"
        student. That section provides that an institution is liable for such disbursements
        only if it used a test administrator who was not independent, compromised the
        testing process in any way, or was unable to document that a subject student
        received a passing score.

       The OIG audit report indicates that All-State used an approved test published by
       Wonderlic, that the test was administered by an independent test administrator
       certified by Wonderlic, and that Wonderlic notified All-State that the students in
       question had passed the test. The audit report further indicates that the test
       administrator made a mistake and did not follow Wonderlic's rules for giving
       retests. However, the audit report does not indicate that All-State knew about this
       error. Thus we have determined that under §668.154, All-State is not liable for
       any disbursements given retests [sic] by the certified independent test
       administrator.

       Moreover, it is longstanding FSA policy that an institution is not liable for the
       Title IV program funds it paid to an "ability-to-benefit" student who successfully
       completes its educational program within its satisfactory academic progress rules
       even if the student did not actually pass an independently administered and
       properly administered approved "ability-to-benefit" test. Thus, in any event, All­
       State would not have been liable for the Title IV program funds it paid to nine of
       the twelve students who were improperly given retests and completed its
       programs.

CMO's detennination fails to consider All-State's responsibility to maintain ATB test
administration records, to determine the eligibility of its students for Title IV funds, and to
document that students received passing scores on an approved test.




                                                  8

Audit of FSA's Controls Over ED-Approved ATB Programs                       ED-OIG/A03-BOOOI


Under 34 C.P.R. § 668.1S1(g), institutions must maintain records documenting­

       (1) The test taken by the student;
       (2) The date of the test; and
       (3) The student's scores as repOlied by the test publisher, assessment center, or State.

Although we agree our audit rcport did not indicate that All-State had actual knowledge of the
improper retests, our audit report did state that All-State had, or should have had adequate
information in its student files to determine that the students' tests had not been properly
administered. As a result, All-State was required to determine that those students were ineligible
to receive Title IV funds. Under its program participation agreement, All-State, not the test
publisher or the ITA, is responsible for identifying eligible students.

In addition, under 34 C.P.R. § 668.1S4(c), an institution is liable for Title IV, HEA program
funds disbursed to a student if the institution is "unable to document that the student received a
passing score on an approved test." The WBST was approved for use in retesting in accordance
with Wonderlic's instructions. Unless the proper retest form was used, or the required 60-day
time period had passed, a student did not receive a passing score on a test approved by the
Secretary. Since All-State is required by 34 C.F.R. § 668.151(g) to maintain records
documenting each students' test information, its records should have shown that the students it
sent for retesting did not take the approved version of the test that was applicable to their
circumstances.

The twelve students we identified in our report were admitted to All-State solely on the basis of
having passed an ATB test that was not administered in accordance with Wonderlic's established
procedures. They received $57,994 in Title IV, HEA funds even though they were not eligible
for those funds. Under the policy described in CMO's determination, there is no accountability
for the $57,994 in Title IV, HEA funds that were received by these ineligible students. Under
our reading of the HEA and Federal regulations, All-State is responsible for repayment of the
$57,994 in Title IV, HEA funds that were disbursed to ineligible students.

The United States General Accounting Office's "Standards for Internal Control in the Federal
Government" requires managers to "determine proper actions in response to findings and
recommendations from audits and reviews ...." In addition, OMB Circular A-50 requires that
agencies "[a]ssure that resolution actions are consistent with law, regulation, and Administration
policy ...."

As stated in Pinding No.1, we found that 1,270 applicants passed an improper retest of the
WBST for the period of July 1, 1997, through November 12,2000, and that of those 1,270
applicants, 724 received $3,362,839 in Title IV, HEA funds according to the NSLDS. We also
found that ACT does not have anyprocednres in place to monitor the testing activity of ITAs for
its CPAt. These results suggest that the set of circumstances for our finding at All-State are
widespread. Without developing and implementing regulations that clearly assign responsibility
under these circumstances, Title IV, HEA program funds may be disbursed to ineligible students
and proper audit resolution actions cannot be determined.




                                                9

Atdc:lit9f FSA' s Controls Over ED-Approved ATB Programs                       ED-OIG/A03-BOOOI


Recommendation:

3.1 	   We recommend that the Assistant Secretary for Postsecondary Education initiate
        appropriate action to ensure that the Federal regulations clearly define the responsihility
        for liabilities to ED for Title IV, HEA funds received by students on the basis of an
        improper ATB test administration.

OPE's Reply:

OPE does not concur with our recommendation. OPE states that prior Lo Lilt: passiug of the
statutory requirement that students who do not have a high school diploma or its equivalent pass
an independently administered test approved by ED, institutions administered and scored
approved ATB tests. Some institutions abused this process, and awarded Title IV, REA funds to
students who did not legitimately pass an approved ATB test.

OPE explains-

        When the regulations implementing the new statutory requirement were
        promulgated in 1995, one of the underpinnings of the new regulatory scheme was
        the removal of the institutions from the ATB testing process. Under the
        regulations, approved ATR tests can only he given by test administrators certified
        by the test publisher. The test publisher must score the tests and notify the
        institution and the student whether the student passes the test. In addition, the test
        administrators must be independent of the institution whose students are taking
        the test. Institutions are liable if they disburse Title IV, lIEA funds to ineligible
        ATB students if they used test administrators who are not independent of the
        institution when the test was given, if the institution compromised the testing
        process in any way, or if the institution does not have documentation from the test
        publisher that the student received a passing score on the test.

OPE further states that there has been a significant reduction in problems that were associated
with ATB students since the change in the statute and the implementing regulations. OPE
believes that the structure of the ATB testing process that removes institutions from involvement
with testing has led to that reduction. Therefore, it does not believe any changes to the
regulations are warranted.

OIG~s   Response:

Our recommendation remains unchanged. A prior reduction in the number of problems
associated with ATB students is not an acceptable justification for inaction in addressing
additional unresolved deficiencies.

Our finding identified a significant problem in the ATB testing process, for which up to
$3,362,839 may be at tisk. Under OPE's interpretation of the regulations, except in certain
limited cases, none of the parties involved in the current ATB process-not the test publisher,
the IT A, or the school-would be accountable for this amount. The only way to recover the
funds would bt: to demand payment from the bOITower.


                                                  10
Audit ofFSA's Controls OverED-Approved ATB Programs                           ED-OIGI A03-BOOO 1



Without accountability [or t.hese funds, and for any other Title IV, HEA program funds received
by students on the basis of improper ATB test administration, there is inadequate incentive for
test publishers, IT As, or schools to identify and correct any deficiencies in the existing process.
Consequently, the regulations need to clearly define responsibility for these funds.


                                       OTHER MATTERS

As part of its three-year analysis submitted to FSA in October 1999, Wonderlic performed. a
detailed review of the cumulative results of its ITA's administrations of the WBST ATB test.
Wonderlic's review found that 56 of its IT As' response patterns and distribution of scores
differed significantly from the patterns consistent with proper test admmistratlOn and handling of
test materials. As a result of this review Wonderlic tenninated 45 of the 56 IT As. Wonderlic has
decertified an additional ten ITAs, who were not identified as part of the three-year analYSis as
having irregular response patterns or test score distributions, for failure to comply with WBST
testing procedures. During our review at ACT, we found that 7 of the 55 ITAs that have been
tenninated from administering the WBST, are approved ITAs for the CPAt.

                                         BACKGROUND

The Higher Education Technical Amendments of 1991 amended the HEA, requiring
postsecondary students who do not have a high school diploma or its equivalent to pass an
independently administered examination that has been approved by ED before receiving Title
IV, HEA program funds. These examinations are intended to establish that students have the
abillty to benefit from postsecondary school training programs. This testing has become known
as "Ability-to-Benefit" (ATB) testing.

On December 1, 1995, ED published final regulations, effective July 1, 1996, specifying the
procedures and requirements for ATB testing which affect test publi shers, schools, and ITAs, as
Subpmt J of 34 C.F.R. Part 668. Compliance with these regulations is mandatory in detennining
the eligibility of applicants for Title IV, HEA program funds.

ED assesses tests submitted for ATB approval according to the requirements in Subpart J of 34
C.F.R. Part 668. ED approves a test for a period of no more than five years, although the
approval can be extended while a subsequent review is conducted to detennine re-approval. A
list of approved tests and passing scores is published in the Federal Register. The initial list of
approved publishers and tests was published in the Federal Register on October 25, 1996.

                       OBJECTIVE, SCOPE AND METHODOLOGY

The objective of our audit was to detennine and evaluate FSA's monitoring methods for ED­
approved ATB test publishers for the period of July 1,1997, through June 30, 2000. To
accomplish our objective­

•    We had discussions with FSA officials concerning monitoring of ATB test publishers .



                                                 11
Audit of FSA's Controls Over ED-Approved ATB Programs                      ED-OIGI A03-BOOO 1


•    We reviewed the agreements between ED and ATB test publishers regarding the
     administration of their ATB test programs.

•    We reviewecl the three-yenr nnnlyses ()f ATR test srnres sllhmittecl t() FSA hy ATR test
     publishers.

•    We reviewed A TB test publishers' user manuals for their ATB tests.

•    We conducted audits at two ATB test publishers to determine if they administered their
     ATB programs in accordance with their agreements with ED and with applicable laws and
     regulations: Wonderlic's WBST program (ED-OIG/A03 - B0022), and ACT's CPAt
     program (ED-OIG/A03 - B0024).

•    We reviewed the universe of compliance audit reports and program reviews covering fiscal
     years 1998 through 2000 that contained ATB audit findings.

•    We conducted audits at three schools that use the results ofWonderlic's WBST for
     determining student eligibility for Title IV, HEA program funds: Lincoln Technical
     Institute (ED-OIG/A03 - B0013), All-State Career School (ED-OIG/A03 - BOOI4), and
     Glendale Career College (ED-OIG/ A09-BOOI7).

•    We reviewed CMO's final audit determination concerning our audit of All-State Career
     School's ability-to-benefit testing process.

During our reviews at Wonderlic and ACT we tested the reliability of computerized WBST and
CPAt data by comparing selected data records with the completed tests answer sheets. We
concluded that the computerized information was sufficiently reliable for the purpose of our
audit at FSA. We did not rely on any computer data processed by FSA.

We conducteu   UUI fit:luwurk. at FSA's offices iu Washington, D.C., frum Octuber 18,2000,
through October 19,2000. In addition, we held discussions with FSA officials in March 2002.
Our exit conference was held on April 3, 2002. Our audit was performed in accordance with
generally accepted government auditing standards appropriate to the scope of the audit described
above.

                     STATEMENT ON MANAGEMENT CONTROLS

We have made a study and evaluation of FSA' s management control structure for monitoring
ED-approved ATB test publishers in effect during our audit period. Our study and evaluation
was conducted in accordance with generally accepted government auditing standards. For the
purpose of this report, we assessed and classified the significant management control structure
into the following category:

       •    Procedures for monitoring ED approved ATB test publishers

The management of FSA is responsible for establishing and maintaining a management control
structure. In fulfilling this responsibility, estimates and judgments by management are required


                                               12 

Auditof FSA's Controls Over ED-Approved ATB Programs                         ED-OIGI A03-BOQQ!


to assess the expected henefits and related costs of control procedures. The objectives of the
system are to provide management with reasonable, but not absolute, assurance that assets are
safeguarded against loss from unauthorized use or disposition and that the transactions are
executed in accordance with management's authorization and recorded properly, so as to pennit
effecti ve and efficient operations.

Because of inherent limitations in any management control structure, errors or irregularities may
occur and not be detected. Also, projection of any evaluation of the system to future periods is
subject to the risk that procedures may become inadequate because of changes in conditions, or
that the degree of compliance with the procedures may deteriorate.

Our assessment disclosed the following conditions in the management control structure of FSA
in effect during our audit period, which in our opinion, result in more than a relatIvely low risk
that errors, irregularities, and other inefficiencies may occur, resulting in inefficient andlor
ineffective performance:

   •   Inadequate Monitoring of ATB Test Publishers
   •   Inv::! lid Agreement Between ED and ACT

Material weaknesses are discussed in the AUDIT RESULTS section of this report. Nonmaterial
weaknesses, which in the auditors' judgment are reportable conditions, are included in the
OTHER MATTERS section.




                                                13 

Audit ofFSA's Controls Over ED-Approved ATB Programs                         EDOIGJA03-BOOOl



                                ADMINISTRATIVE MATTERS 


Statements that management practices need improvements, as wen as other conclusions and
recommendations in this report represent the opinions of the Office of Inspector General.
Determination of corrective action to be taken will be made by the appropriate Department of
Education officials.

Please provide us with your final response to each open recommendation within 60 days of the
date of this repon indicating what corrective actions you have taken or plan, and related
milestones.

In accordance with Office of Management and Budget Circular A-50, we will keep this audit
report on the OIG list of unresolved audits until all open issues have been resolved. Any reports
unresolved after 180 days from date of issuance will be shown as overdue in the OIG's
Semiannual Report to Congress.

Please provide the Supervisor, Post Audit Group, Office of the Chief Financial Officer and the
Office of Inspector General, with quarterly status reports on promised corrective actions until all
such actions have been completed or continued follow-up is unnecessary.

In accordance with the Freedom of Information Act (5 U.S.C. § 552), reports issued by the
Office of Inspector General are available, if requested, to members of the press and general
public to the extent information contained therein is not subject to exemptions in the Act.

We appreciate the cooperation given us in the review. Should you have any questions
concerning this report, please call Bernard Tadley, Regional Inspector General for Audit at 215­
656-6279.

                                              Si        ~7J-            ­


                                              Th   .    '" '   ~
                                                                   .­
                                              Assistant Inspector General for Audit

Attachment




                                                   14
                                                               Attachment - Auditee's Response
                 UNITED STA TES DEPARTMENT OF EDVCA TION

                                 JUL I 2 2002


Mr. Bernard Tadley, Regional Inspector General for Audit
U.S. Department of Education 

Office of Inspector General 

The Wanamaker Building 

100 Penn Square East. Suite 502 

Philadelphia, PA 19107 


DellI' Mr. Tndley:

We appreciate the opportunity to respond to your draft aUdit report (Control Number ED·
OIGI A03·BOOO I) regarding the oversight of abilitY-lo-benefil (ATB) lest publishers.

The findings in this nudit appear to be based upon three previous audit rcpons regarding
(he test administration procedures of specific ATB test publishers-ACT. Wonderlic, and
the College Board. In its previous audits, the Office of the Inspector General (010)
focused on recommendations [0 improve [he tcst administration practices of these
particular test publishers.

This audit focuses on the oversight responsibilities of Federal Studenl Aid (FSA) and the
Office of Postsecondary Educ:ltion. Our joint response is below.

Actions to Date on Previous Audit Report Findings

Before discussing (he OIG recommendations in this draft audi[ report, we wanted to
provide infonnation about OUT actions to resolve the previous findings The test
publishers have been very responsive to the recommendations made by the OIG in these
previous audits. FSA has worked with the identified publishers and they have already
taken prompt corrective action. Specifically.

   o 	 ACT has established a system to identify the institutions at which each cenified
       independent test administrator is 'lpproved for its Career Programs Assessment
       reSl (CPAt) lest adminjSlnujon. ACT wi]} process and score CPAt ATB answer
       sheets for an institution only if the answer sheets nre submitted by an independent
       test administrator (ITA) who is approved for resting at [hal institution.

   o 	 ACT will activa.te a computerized process for applying retest rules to each CPAt
       ATB answer sheet and has revised its retest procedures to facilitate this
       monitoring process. FSA has reviewed and approved ACT's new retest
       procedures.
  Page 1- Mr. Bernard Tadley. ED-OIGIA03-BOOOI


      o 	 Wonderlic began monitoring the retest activities of its IT As in January 2001.
          Wonderlic has also agreed to track the retest history of applicants who may have
          been tested ar a different institution. This action will help idemify students who
          have used the same test form within a short period of lime.

     o 	 The College Board no longer allows institutions to use the DOS or Windows
         versions of its Accuplacer ATB tcst This means that ITAs will no longer have
         the ability to revise (es( scores.

 Finding No.1: Oversight or ED-Approved ATB Test Publishers Needs
 Improvement

 Based upon the findings discussed above. the 010 draft audit report recommends
 changes related to ED's oversight responsibilities of ATRpublishers.

 Recommendation}.]: Establish and implement control activities and monitoring and
 technical assistance strategies to ensure ED-approved ATB test publishers comply with
 applicable laws and regulations and with the Icnns of their agreements with [he SecrelaTY.

Response: FSA has developed a lnicking system to ensure that test publishers comply
with the regulalOry reponing requirements, and we will follow up on any test score
irregularities that are identified by the test publishers. We recognize that the ~IG's
findings were made only after conducting on-sire reviews of rhe activities of test
publishers, and FSA is committed to conducting similar on-site reviews for those test
publishers that have not already been reviewed by 010. We wiIJ also provide technical
assistance on the ATB regulatory requirements through regular contacts with these
publishe,-s. and we may conduct follow-up reviews. as needed.

Historically, ATB test publishers have not been a SOurce of any deliberate efforts to abuse
ATB requirements and nOne of the DIG's findings involve intentional wrongrloing For
these reasons, £he Office of Management and Budget has mandated a review of the ATB
regulations to determine feasible ways to reduce the regulatory burden on test publishers.
The outcome of this review may affect FSA's monitoring procedures.

Recommendalioll 1.2: Ensure thal test pUblishers improve their processes for identifying
nnd reporting retest errors, to ensure that institutions have accurate and timely
infonnation at the time that eligibility determinations are made.

Response: The two test publishers that had findings related to retesting have revised their
procedures in ways that will significantly reduce the likelihood of retest errors. We also
found thai it may not be operationally feasible for some publishers to ensure that an
institution will always have accurate and timely information abom retesting errors for
every single student at the time that eligibility detenninations are made by an institution.
Thus, even with these improvements a very small risk remains.
 Page 3 - Mr. Bemard To.dley. EO-OIG/A03-BOOOl


 For tests that are not gi ven ll[ assessment centers, ITAs are required by regulation to
 submit completed tcsts.to {he (est pUblisher. The test publisher then scores the test and
 sends [he results to the school. This regulatory process is designed to ensure the test is
 administered independently of (he school and to prevent fraud and abuse.

 During this period, a student may apply to a different school, and take the same test form
 from a different test administrator. Depending on the vagaries of themail.thissecond
 test could possibly reach the test publisher, receive a score, and be mailed back to the
 school before the publisher receives the first tesl. If the student is eligible, the second
 school is permined (0 disburse student aid as soon as the test score is received from the
 publisher.

This outCOme could only be prevented by requiring "real time" electronic submissions of
the completed tests or requiring the school to delay disbursement of student aid funds for
a period of time sufficient to eliminate the problem of test resuhs being sent out by the
lest publisher jn reverse order. The "real time" submission requirement would be very
costly. and could discourage test publisher panicipation in the ability-to-benefit testing
process at a time when the number of participaling test publishers is very limited.

The risk of retesting errors has been reduced significantly. However. it does not appear
operationally feasible to completely eliminate the possibility of a few students taking the
same [est form within a shon period of time.


Finding No.2: ED Entered into an Agreement wfth ACT that Does Not Meet 

Regulatory Criteria 


 ACf's Assessment test is a national college admission examination designed to measure
high school students' general educCltional development and their ability to complete
college leveJ work. It is administered on five national t~s[da[es each year and covers four
content areas: English, mathematics, reading, and science reasoning. ED approved the
ACT Assessment test for ATB testing purposes on October 27. 1998. so that s[odenrs
who were home-schooled would not be required to take yet another test to establish their
eligibility for student aid. Unfortunately, ACT does not have a mechanism for
identifying students who are taking this test for the purpose of fulfilling the ability-to-­
benefit requirements. These ~tudenrs must be tracked so that ACT can analyze the
distribution of test scores for any testing irregularities, as required under current
regularions.

Recommendation 2. I: DIG recommends that FSA either withdraw ED's approval of (he
ACT Assessment test or revise the agreement between ED and ACT to ensure that aU
criteria for test publishers in Subpart J of 34 CFR Part 668 are met.
  Pnge 4 - Mr. Bernard Tadley, ED·OIG/ A03-BOOO J


  Response: ACT has decided to withdraw the use of ACT Assessment scores for ATB
 purposes. ACT made Ihis decision because they did not have a system in place to track
 examinees who elected [0 take [he ACT Assessment for ATB purposes. In addition, the
 010 had suggested that fSA review every new form of this lest before it was
 adminisrered. ACT does not, as a matter of practice, release secure forms of the ACT
 Assessment for review by others. New test forms must be developed several Urnes each
 year to ensure a fair testing process for thousands of college t'lpplicants. ACT believes
 the security risks are too gre:u.


 Finding No.3: Federal Regulations Re2arding Accountability for Title IV, H£A
 Program Funds Received by Students who Pass Improper ATB Test
 Administrations Need Improvement

 Recommendation 3.1: We recommend that Ihe Assistant Secretary for the Office of
 Postsecondary Education initiate appropriate action to ensure that the Federal regUlations
 clearly define rhe responsibility fOf Jiabilities to ED fOf Tille IV. HEA funds received by
 students on the basis of an improper ATB test administration.

Response: We do not believe that there is a need to revise the current regulations. Prior
(0 the enactment of Ihe statutory requirement that a student who does not have a high
school diploma or its recogniz.ed equivalent pass an independently administered test
approved by [he Secretary. inslitutions were involved in administering and scoring
approved ATB test, and certain institutions used that opportunity to lake advantage of the
process and award Title IV, MEA funds to student who did not legitimately pass an
approved ATB test.

 When the regulations implementing the new statutory requirement were promulgated in
  1995, one of the underpinnings of the new regulatory scheme was the removal of the
 institutions from the ATE testing process. Under the regUlations, approved ATB tests
 can only be give~by test administrators certified by the test publisher. The test publisher
 must score the teses and notify (he institution and the student whecher the student passes
 the test. In addition. the teSt adminislIators must be independent of the institution whose
students are taking the test. Institutions are liable if they disburse Title IV, HEA funds to
ineligible ATB students if they used test administrators who are nOl independent of the
institution when the test was given, if the institution compromised the testing process in
any way, or if the ins[itulion does not have documentation from the test publisher that the
student received a passing score on the test. This position on institutionalliabiJity was
clearly spelled out in bp[h the Notice of Proposed Rulemaking (AugUSt 16, 1994) and the
final regulations (December 1. 1995).

We haYel~q) ,l.Ipif1ificant reduction in th(! problems that were associated with ATB
sttlde9tt.fJnCf:       ... ~ in the statute and the implementing regulations and believe that
the4~\,I:J~.9:f '           l~ti!ll~ PfQC~5 that removes institutions from involvement with
   .   .       "\
Pa~e   5 - Mr. Bernard Tadley, ED-OIG/A03-BOOOl

testing has led to that reduction. Thcrefore. we do not believe that any changes to the
regulations are warranted.

We appreciate your effons to improve the ATB process, and we hope OUt actions have
been responsive to your recommendations~

Sincerely.



Oreg Woods
Chief Operating Officer
Federal Student Aid
                           REPORT DISTRIBUTION LIST 

                          CONTROL NO. ED-OIG/A03-BOOOl 


                                                                           No. of
Auditee/Action Official                                                    Copies

       James Manning                                                           1
       Acting Chief Operating Officer
       Federal Student Aid
       U.S. Department of Education 

       Union Center Plaza Building 

       830 15t Street, NE, Room 112Gl 

       Washington, DC 20202 


       Ms. Sally Stroup                                                        1
       Assistant Secretary for Postsecondary Education
       U.S. Department of Education 

       Room 7115 

       1990 K Street, NW 

       Washington, DC 20006 


Other ED Officials/Staff (electronic copy)

       Audit Liaison Officer, Federal Student Aid                              1
       Audit Liaison Officer, Office of Postsecondary Education                1
       Assistant General Counsel, Office of the General Counsel                1
       Correspondence Control, Office of the General Counsel                   1
       Chief of Staff, Office of the Secretary                                 1
       Under Secretary, Office of the Under Secretary                          1
       Deputy Secretary, Office of the Deputy Secretary                        1
       Director, Office of Public Affairs                                      1
       Deputy Director, Communications                                         1
       Acting Assistant Secretary, Office of Legislation and
              Congressional Affairs                                            1
       Chief Financial Officer, Office of the Chief Financial Officer
       Director, Financial Improvement and Post Audit Operations, Office
              of the Chief Financial Officer                                   1
       Post Audit Group Supervisor, Financial Tmprovement and Post
              Audit Operations, Office of the Chief Financial Officer          1
       Assistant Secretary, Office of Intergovernmental and Interagency
              Affairs                                                          1
       General Manager for Schools Channel, Federal Student Aid                1