oversight

Wonderlic's Ability-to-Benefit (ATB) program.

Published by the Department of Education, Office of Inspector General on 2002-02-04.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

Audit of Wonderlic’s Ability-to-Benefit Program                                     ED-OIG/A03-B0022


accordance with Wonderlic’s retesting procedures. The report also provides a description of
each retest violation and states that the tests are considered invalid for use in qualifying students
for Title IV Federal Financial Assistance.

The Wonderlic Basic Skills Test User’s Manual for Ability-To-Benefit Testing states —

        When an applicant has already taken both verbal and quantitative forms 1 & 2 of
        the WBST, but you believe that he or she has not been accurately assessed, you
        may retest the applicant again on either form in accordance with the following
        rules:

        1. The applicant must have already taken both forms of the WBST once.

        2. The applicant may be retested on the same test form once, and only once.

        3. The applicant must not have been told in advance that there would be an
           opportunity to take the same test form again.

        4. The applicant may be retested on the same form only if at least 60 days have
           passed since he or she was initially tested on that form.

The “Wonderlic Basic Skills Test ATB Retest Exception Report” is run for each institution,
considering only those WBST administrations conducted for the institution rather than the entire
testing history of each applicant. As a result, any retest violations that have occurred due to an
applicant testing at more than one institution are not reported.

Federal regulations state —

§     “The agreement between a test publisher and the Secretary provides that the test publisher
      shall . . . [c]ertify test administrators who have . . . [t]he necessary training, knowledge,
      and skill to test students in accordance with the test publisher’s testing requirements . . . .”
      34 C.F.R. § 668.150(b)(2)(i).

§     “An institution may use the results of an approved test to determine a student’s eligibility
      to receive Title IV, HEA programs funds if the test was independently administered and
      properly administered.” 34 C.F.R. § 668.151(a)(2).

§     “The Secretary considers that a test is properly administered if the test administrator . . .
      [a]dministers the test in accordance with instructions provided by the test publisher, and in
      a manner that ensures the integrity and security of the test . . . .” 34 C.F.R.
      § 668.151(d)(2).

Failure to timely report all retest administrations not conducted in accordance with Wonderlic’s
established procedures may result in invalid ATB determinations, improper admission of
students, and disbursements of Title IV, HEA program funds to ineligible students.




                                                   2
Audit of Wonderlic’s Ability-to-Benefit Program                                   ED-OIG/A03-B0022


Recommendations:

We recommend that the Chief Operating Officer (COO) for Student Financial Assistance (SFA)
require Wonderlic to —

1.1     Consider the entire WBST testing history of each applicant when preparing the
        “Wonderlic Basic Skills Test ATB Retest Exception Report.”

1.2     Improve its process for identifying and reporting retest errors, to ensure that institutions
        have accurate and timely information at the time that eligibility determinations are made.

Wonderlic’s Reply:

Wonderlic indicated that it concurred with our finding; however, it only agreed to implement our
recommendations upon ED’s re-approval of the WBST for ATB testing purposes. Wonderlic’s
response states that —

        Wonderlic will invest the necessary time and resources to make these changes to
        both the Individual Student and Quarterly WBST Reporting systems when the
        Department of Education completes its review of the WBST for ability-to-benefit
        purposes.

Wonderlic explained that it does not necessarily receive submitted WBST answer sheets in the
chronological order in which they were administered to students. As a result, Wonderlic
contends that in retest situations it can never guarantee that the official score report generated
truly reflects complete and proper compliance with its retesting rules and procedures.

Wonderlic further states that its ITA’s are responsible for testing applicants in compliance with
its rules for retesting. At the time of the original approval of the WBST the instructions to the
ITA in the Wonderlic Basic Skills Test User’s Manual For Ability-to-Benefit Testing stated —

        You are responsible for conducting retests in accordance with these rules.
        Therefore, you should maintain a record of all test forms administered to an
        applicant and the specific dates on which they were administered. Wonderlic will
        not review the appropriateness of individual retest administrations when answer
        sheets are submitted for official scoring . . . .

In its response, Wonderlic also asked questions about school procedures in certain circumstances
and proposed that schools only be held responsible for retest administrations at the same school
system or conducted by the same ITA in any other location.

OIG’s Response:

We have not made the implementation of our recommendations contingent upon the re-approval
of the WBST by ED. Since re-approval is based, in part, on approved testing procedures, ED




                                                  3
Audit of Wonderlic’s Ability-to-Benefit Program                                   ED-OIG/A03-B0022


should take Wonderlic’s implementation of our recommendations into account when considering
re-approval of the WBST.

Wonderlic’s response appears to confirm that it lacks the internal controls needed to ensure that
its approved procedures are followed and that institutions have accurate and timely information
when eligibility determinations are made. Because Wonderlic, not its ITA’s, maintains the entire
testing history of its applicants, only Wonderlic can determine for certain whether a retest was
administered in compliance with its retesting procedures. For these reasons, our
recommendations to SFA remain as originally proposed.

In addition, our conclusions and recommendations are based on the requirements and approved
procedures that were effective during the audit period. As a result, we are not responding to
Wonderlic’s request for clarification or its proposal to modify its procedures. Wonderlic’s
questions and proposal should be addressed to ED, either during the audit resolution process or
during consideration of re-approval of the WBST.

Finding No. 2:       Untimed Wonderlic Basic Skills Test Administration Requirements Were
                     Not Always Enforced

Our review revealed that Wonderlic did not always enforce its established procedures for
conducting untimed test administrations of the WBST. The untimed WBST administration
combines a standard 20 minute time period with a subsequent untimed period on the same test
form. Wonderlic provides scores on both a timed and untimed basis for each administration.
The untimed score contains a statistical score adjustment, while the timed score does not. As a
result, either, both, or neither of the applicant’s timed and untimed scores may reflect a passing
score.

We determined that there was inadequate documentation for 1,164 (27 percent) of the 4,262
untimed test administrations conducted by certified WBST independent test administrators (ITA)
and processed by Wonderlic for official scoring during the period July 1, 1997, through
November 12, 2000. Of the 1,164 untimed test administrations, 458 (39 percent) had a “PASS”
status. Of the 458 test administrations that were a “PASS”, 164 were cases where the applicant
passed with the untimed administration score, but would not have passed with the timed
administration score.

Federal regulations applicable to this finding are the same as those cited for Finding No. 1:
34 C.F.R. §§ 668.150(b)(2)(i), 668.151(a)(2), and 668.151(d)(2).

The Wonderlic Basic Skills Test User’s Manual for Ability-To-Benefit Testing states —

        The WBST can be administered on an untimed basis when a physical or mental
        condition prevents an applicant from fluidly responding to the test questions. For
        example, applicants with extremely poor vision, dyslexia, or excessive test
        anxiety may be candidates for an untimed administration. . . .




                                                  4
Audit of Wonderlic’s Ability-to-Benefit Program                                  ED-OIG/A03-B0022


        The WBST measures whether an applicant currently has the basic language and
        math skills necessary to succeed in the classroom. Therefore, administering the
        WBST on an untimed basis merely because an applicant has failed to demonstrate
        the necessary skills is inappropriate. If [the ITA determines] that an untimed
        administration is necessary to accurately measure the skills of a particular
        applicant, [the ITA] must document the reason for the untimed administration in
        the applicant’s permanent record folder and in the Problems/Limitations box on
        the reverse side of the ATB answer sheet.

If a reason for an untimed administration is not documented in the Problems/Limitations box, an
edit prevents the scoring program from processing the WBST answer sheet. Wonderlic officials
explained that, if the justification for an untimed administration is not documented in the
Problems/Limitations box on the reverse side of the ATB answer sheet, Wonderlic personnel
input “No Reason Given” or a variation of this terminology in the Problems/Limitations box so
that the answer sheet can be scored.

Processing ATB test answer sheets for invalid untimed administrations of the WBS T may result
in improper ATB determinations, inappropriate admission of students, and disbursements of
Title IV, HEA program funds to ineligible students.

Recommendations:

We recommend that the COO for SFA require Wonderlic to —

2.1     Strengthen its management controls to ensure that ITA’s follow the already established
        procedures for conducting an untimed WBST.

2.2     Process only WBST answer sheets for untimed test administrations that contain a valid
        reason for testing the applicant on an untimed basis.

Wonderlic’s Reply:

Wonderlic concurred with the basis for the finding and recommendations, but it believed that the
cause of the finding was immaterial. Wonderlic’s response stated that it has already modified its
procedures to ensure that it only scores WBST answer sheets for untimed test administrations
that contain a valid reason for testing the applicant on an untimed basis:

        After this issue was raised during Wonderlic’s on-site audit, Wonderlic
        immediately modified its operational procedures to include a personal contact to
        either the ITA or school to obtain this information in any instance when it has not
        been recorded on the Answer Sheet.

Wonderlic’s response explained that with respect to WBST ATB student norms and pass rates, it
is irrelevant whether the test is administered on a timed or untimed basis:




                                                  5
Audit of Wonderlic’s Ability-to-Benefit Program                                   ED-OIG/A03-B0022




        With respect to untimed WBST test administrations, it is important to note that:
               1) the lack of a recorded reason for untimed administration has no
                  bearing on the proper scoring of the WBST on an untimed basis, and,
               2) that the scoring procedure for the WBST on an untimed basis contains
                  a statistical score adjustment that allows timed and untimed WBST
                  results to be interpreted on a equivalent basis. . . .

OIG’s Response:

We reviewed Wonderlic’s comments but our conclusion remains unchanged. Federal
requirements only allow institutions to use an approved ATB test to determine a student’s
eligibility to receive Title IV, HEA programs funds, if that test was administered in accordance
with the test pub lisher’s instructions.

Our recommendations remain unchanged as well. Regarding Wonderlic’s modification of its
procedures, Wonderlic’s contact should be with the ITA alone, rather than the ITA or the school,
to ensure the independent administration of the test.

                                             BACKGROUND

The Higher Education Technical Amendments of 1991 amended the HEA, requiring
postsecondary students who do not have a high school diploma or its equivalent to pass an
independently administered examination that has been approved by ED before receiving Title
IV, HEA program funds. These examinations are intended to establish that students have the
ability to benefit from postsecondary school training programs. This testing has become known
as “Ability- to-Benefit (ATB) testing.”

On December 1, 1995, ED published final regulations, effective July 1, 1996, specifying the
procedures and requirements for ATB testing which affect test publishers, schools, and ITA’s, as
Subpart J of 34 C.F.R. Part 668. Compliance with these regulations is mandatory in determining
the eligibility of applicants for Title IV, HEA program funds.

ED assesses tests submitted for ATB approval according to the requirements in Subpart J of 34
C.F.R. Part 668. ED approves a test for a period of no more than five years, although the
approval can be extended while a subsequent review is conducted to determine re-approval. A
list of approved tests and passing scores are published in the Federal Register. The initial list of
approved tests was published in the Federal Register on October 25, 1996. The Wonderlic Basic
Skills Test was approved by ED and appeared on the list published on October 25, 1996.

                          OBJECTIVE, SCOPE AND METHODOLOGY

The objective of our audit was to determine whether Wonderlic properly administered its ATB
program in accordance with its agreement with ED, for the approved use of the WBST, and with
applicable laws and regulations. To accomplish our objective —




                                                  6
Audit of Wonderlic’s Ability-to-Benefit Program                                   ED-OIG/A03-B0022


§       We analyzed a computer file, provided by Wonderlic, that contained data about the
        universe of WBST ATB test administrations (159,310) conducted during the period July 1,
        1997, through November 12, 2000.

§       We analyzed computer files, provided by Wonderlic, covering the period December 1996
        through October 2000 that contained data about the universe of certified WBST ITA’s
        (2107) and the universe of decertified WBST ITA’s (54). We examined the file
        documentation for 25 randomly selected certified WBST ITA’s and 5 randomly selected
        decertified ITA’s.

§       We reviewed Wonderlic’s procedures for WBST ATB test administration, scoring and
        reporting. We interviewed Wonderlic personnel to obtain an understanding of these
        procedures.

§       We reviewed Wonderlic’s report “An Analysis of Wonderlic Basic Skills Test and
        Wonderlic Scholastic Level Exam Test Score Distributions from Ability-to-Benefit Test
        Administrations.”

During our review at Wonderlic, we also tested the reliability of computerized WBST ATB data
by comparing selected data records with the completed WBST answer sheets. We concluded
that the computerized information was sufficiently reliable for the purposes of our audit.

We conducted our on-site fieldwork at Wonderlic, Inc., in Libertyville, IL, from November 13,
2000, through November 17, 2000. Our exit conference was held on November 17, 2000.
Subsequent to the completion of our fieldwork, we performed additional analyses, from
November 2000 through January 2001. Our additional analyses did not identify any new
reportable conditions. We sent a draft of this audit report to Wonderlic on November 30, 2001,
and we received Wonderlic’s comments on the draft on December 28, 2001. Our audit was
performed in accordance with government auditing standards appropriate to the scope of the
audit described above.

                         STATEMENT ON MANAGEMENT CONTROLS

As part of our review, we assessed Wonderlic’s management control structure, as well as its
policies, procedures and practices applicable to the scope of the audit. We assessed the level of
control risk for determining the nature, extent and timing of our substant ive tests to accomplish
the audit objective.

For the purpose of this report, we assessed and classified the significant controls into the
following categories:

    §    ITA Training and Certification
    §    WBST Administration
    §    WBST Scoring
    §    WBST Reporting
    §    WBST Records Management




                                                  7
                         REPORT DISTRIBUTION LIST
                        CONTROL NO. ED-OIG/A03-B0022

                                                                             No. of
Auditee                                                                      Copies
      Mr. Justin E. Long                                                          1
      Director Educational Operations
      Wonderlic Incorporated
      1795 N. Butterfield Road
      Libertyville, IL 60048

ED Action Official
      Mr. Greg Woods                                                             1
      Chief Operating Officer
      Student Financial Assistance

Other ED Officials/Staff (electronic copy)

       Chief of Staff, Office of the Secretary                                   1
       Under Secretary, Office of the Under Secretary                            1
       Deputy Secretary, Office of the Deputy Secretary                          1
       Assistant Secretary, Office of Intergovernmental and Interagency          1
              Affairs
       Assistant Secretary, Office of Postsecondary Education                    1
       Assistant Secretary, Legislation and Congressional Affairs                1
       Chief Financial Officer, Office of the Chief Financial Officer            1
       Director, Financial Improvement and Post Audit Operations, Office         1
              of the Chief Financial Officer
       Supervisor, Post Audit Group, Office of the Chief Financial Officer       1
       Chief Financial Officer, Student Financial Assistance                     1
       Assistant General Counsel for Postsecondary Education, Office of          1
              the General Counsel
       Correspondence Control, Office of the General Counsel                     1
       Audit Liaison Officer, Student Financial Assistance                       1
       Audit Liaison Officer, Office of Postsecondary Education                  1
       General Manager for Schools Channel, Student Financial Assistance         1
       Area Case Director for Case Management and Oversight, Student             1
              Financial Assistance
       Public Affairs Officer, Office of Public Affairs                          1
       Press Secretary, Office of Public Affairs                                 1