oversight

Implementation of Supplemental Educational Services in California

Published by the Department of Education, Office of Inspector General on 2006-09-21.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                       UNITED STATES DEPARTMENT OF EDUCATION

                                        OFFICE OF INSPECTOR GENERAL




                                             September 21, 2006


FINAL MANAGEMENT INFORMATION REPORT
State and Local No. 06-02

To:       	     Henry Johnson
                Assistant Secretary
                Office of Elementary and Secondary Education

                Morgan Brown 

                Assistant Deputy Secretary 

                Office of Innovation and Improvement 



From: 	         Helen Lew /s/
                Assistant Inspector General for Audit Services

Subject: 	      Implementation of Supplemental Educational Services in California
                Control Number ED-OIG/X09G0007

The purpose of this Final Management Information Report is to provide the U.S. Department
of Education’s (Department) Office of Elementary and Secondary Education and Office of
Innovation and Improvement with information that may be beneficial in future oversight of the
Supplemental Educational Services (SES) provisions of Title I, Part A of the Elementary and
Secondary Education Act of 1965, as amended by the No Child Left Behind Act of 2001. Our
intent was to provide insight to the Department on selected issues identified during audits of five
SES providers serving local educational agencies in California and offer suggestions for
enhancing the Department’s Supplemental Educational Services Non-Regulatory Guidance
published in June 2005. While the suggestions in this report are directed at the Department, the
OIG recognizes that the state educational agencies (SEA) also play an important role in ensuring
that local educational agencies (LEA) comply with the SES provisions, through SEA monitoring
activities. SEAs should also take an active leadership role in guiding LEAs on proper
implementation of SES by providing timely and meaningful technical assistance. In its
comments to a draft of this report, the Department generally concurred with the information
presented in this Final Management Information Report. The Department’s response is attached
at the end of this report.


                              400 MARYLAND AVE., S.W. WASHINGTON, D.C. 20202-1510

              Our mission is to ensure equal access to education and to promote educational excellence
Final Management Information Report                                                          ED-OIG/X09G0007 

State and Local No. 06-02                                                                          Page 2 of 17 



                                             BACKGROUND



Title I, Part A, of the Elementary and Secondary Education Act (ESEA) of 1965, as amended by
the No Child Left Behind Act of 2001, requires local educational agencies (LEAs) to offer SES
to students from low-income families attending a Title I school in its second year of school
improvement, or that has been identified for corrective action or restructuring.1 SES consists of
tutoring, remediation, and other educational interventions designed to increase students’
academic achievement, and are in addition to instruction provided during the school day.

State-approved SES providers, selected by the individual student’s parents, provide SES to
eligible students under agreements that the LEAs are required to develop. These agreements,
which should be developed in consultation with the parents and the provider, are required to
include a statement of specific achievement goals, identify how the student’s progress will be
measured, and set a timetable for improving the student’s achievement. The state educational
agency (SEA) is responsible for evaluating potential providers, maintaining a current list of
approved providers, and monitoring all providers delivering services in the state.

The OIG has conducted five SES provider audits in California. We selected a variety of provider
types (for-profit, non-profit, LEA) and delivery modes (individual, small group, and online) for
these audits to identify and understand specific issues associated with each type of provider. All
providers audited were approved providers in California serving students attending schools in
California. Table 1 identifies the providers and LEAs we reviewed and provides the respective
OIG Audit Control Number (ACN) for your reference. The reports are available on the OIG’s
Webpage at http://www.ed.gov/about/offices/list/oig/areports.html under the heading Office of
Elementary and Secondary Education.

                  Table 1: Providers Audited and the LEA Associated with Each Audit
       ACN                          Provider                                 LEA
                                                            Oakland Unified School District
    A09F0009       ARC Associates (ARC)
                                                            (OUSD)
                                                            Stockton Unified School District
    A09F0012       Learning Excitement Inc. (LEI)
                                                            (SUSD)
                                                            Los Angeles Unified School District
    A09F0013       Professional Tutors of America (PTA)
                                                            (LAUSD)
    A09F0019       San Diego City Schools (SDCS)                       Same as provider
                                                                       Salinas Union High School District
    A09F0022       Progressive Learning (PL)
                                                                       (SUHSD)




1
  Under the No Child Left Behind Act of 2001, Title I schools that fail to make adequate yearly progress (AYP) for
two consecutive years are identified for school improvement. Title I schools are identified for corrective action if
they do not make AYP for four years, while Title I schools not making AYP for five years are identified for
restructuring. The “low-income family” determination is based on the same poverty data that an LEA uses to
allocate Title I, Part A funds to its schools under section 1113 of Title I. Those data are usually a student’s
eligibility for free or reduced price lunch under the National School Lunch Program.
Final Management Information Report                                                         ED-OIG/X09G0007 

State and Local No. 06-02                                                                         Page 3 of 17 


Table 2 presents attributes for each provider’s SES program. All providers delivered SES
outside of the regular school day—usually after school. Of those providers that delivered SES in
a group setting, the maximum number of students served by each tutor ranged from 4 to 1 at the
non-profit provider (ARC) to 15 to 1 at the online provider (PL). Except for PTA, the providers
had regularly scheduled days for delivering SES and a fixed duration for each tutoring session.
For example, ARC’s tutoring sessions were scheduled four days per week at each OUSD school
site and lasted for two hours each day. PL provided one-hour sessions twice a week at SUHSD
schools. In contrast, PTA offered more flexibility in the tutoring schedule and number of hours
of SES available per week. While the actual number of hours a student received SES per week
varied because the parents determined the tutoring schedule, PTA indicated that it expected to
deliver two to three hours of tutoring per week with sessions lasting one to two hours. PTA was
able to offer more flexibility than other providers as it was the only provider we reviewed that
delivered individual tutoring in students’ homes.

                                 Table 2: Selected Provider Attributes
                                                               Student-Teacher                    Delivery
    Provider          Provider Type       Delivery Mode
                                                                   Ratio (a)                      Location
 ARC                 Non-profit        Group                                4 to 1             School
 LEI                 For-profit        Individual or Group                  8 to 1             School
 PTA                 For-profit        Individual                           1 to 1             Home
 SDCS                District          Group                              10 to 1              School
 PL                  For-profit        Group (online)                     15 to 1              School
 (a) The student/teacher ratio presented in the table is the maximum ratio the provider would allow.




                           OBSERVATIONS AND SUGGESTIONS



This Management Information Report provides information relevant to SEA and LEA
implementation of the SES provisions of the ESEA and specific information on the SES
providers we reviewed. The report contains four sections as follows: (1) LEA Implementation of
SES Prioritization, (2) LEA Preparation of Student Learning Plans, (3) Provider Payments for
SES, and (4) SES Provider Effectiveness. Each section summarizes information obtained during
our audits, identifies OIG observations, and provides suggestions, when appropriate, for the
Department to consider for future enhancements to its SES guidance.


Section 1: LEA Implementation of SES Prioritization

This section discusses our findings related to LEA prioritization of SES and offers a suggestion
for enhancing the Department’s guidance.

The regulations at 34 C.F.R. § 200.45(b) and (c) provide that students from low-income families
attending Title I schools in the second year of improvement, corrective action, or restructuring
status are eligible for SES. The regulation at 34 C.F.R. § 200.45(d) states that “if the amount of
funds available for supplemental educational services is insufficient to provide services to each
student whose parents request these services, the LEA must give priority to the lowest-achieving
students.”
Final Management Information Report                                                        ED-OIG/X09G0007 

State and Local No. 06-02                                                                        Page 4 of 17 


Three of five LEAs prioritized SES during the period covered by our audits. However, we found
that none of the three LEAs implemented prioritization properly. In each case, the LEA did not
offer SES to all eligible students and the LEA had significant amounts of unexpended SES
funding remaining at the end of the school year, which could have been used to provide SES to
additional students.

    ƒ   OUSD inappropriately prioritized by first applying academic criteria to determine 

        eligibility for SES and then applying income criteria to make a final eligibility 

        determination. OUSD’s list of eligible students did not include students from

        low-income families who scored above the basic level.2


    ƒ   SUSD determined a student’s eligibility for SES based solely on a student’s performance
        on California’s standardized tests. The District did not identify SES-eligible, low-income
        students that performed at the proficient level or above on standardized tests and
        identified low-achieving students that were not low-income as eligible for SES.

    ƒ   SUHSD inappropriately limited access to SES by restricting eligibility to low-achieving
        students that were classified as low-income without first determining whether the demand
        for services, as measured by the number of requests for SES from families meeting the
        income criteria alone, would exceed available funding. SUHSD’s list of eligible students
        did not include students from low-income families who scored above the basic level.

Similar problems have been noted in other OIG reviews of SES implementation. Specifically,
two of six School Choice/SES audits performed recently noted instances where LEAs
misinterpreted the SES eligibility provisions.3

    ƒ   Washoe County School District in Nevada did not consider all eligible students when it
        requested approval from the Nevada Department of Education to reduce its Title I SES
        set-aside for school year 2004-2005. The District only included the “lowest-achieving”
        Free and Reduced Price Lunch (FRPL) eligible students when estimating the funds
        needed to provide SES at the two schools. Ultimately, the two schools made adequate
        yearly progress (AYP)4 and were not required to offer SES in school year 2004-2005, but
        had the schools been required to provide SES, the amount of funding set-aside would not
        have been sufficient to provide SES to all eligible students.




2
 California has five classifications for student achievement—advanced, proficient, basic, below basic, and far below
basic. Students scoring at basic or below on California standardized tests are identified as not meeting state
academic content standards.
3
  The OIG also performed audits of SEA and LEA implementation of ESEA’s School Choice and SES provisions in
the states of Illinois (ACN A07F0003), Michigan (ACN A05F0007), Indiana (ACN A05E0014), Delaware
(ACN A03F0002), New Jersey (ACN A02F0006) and Nevada (ACN A09F0002). These reports are also available
at the Department’s OIG Webpage, under the heading Office of Elementary and Secondary Education.
4
  LEAs must annually review the progress of each Title I school to determine whether the school is making AYP.
AYP is the measure of the extent to which students in a school, taken as a whole and in certain groups within the
school, demonstrate proficiency in at least reading/language arts and mathematics. It also measures the progress of
schools under other academic indicators, such as the graduation or school attendance rate. Each State has developed
its own definition of AYP, and these definitions have been approved by the U.S. Department of Education.
Final Management Information Report                                                        ED-OIG/X09G0007 

State and Local No. 06-02                                                                        Page 5 of 17 


      ƒ    Several LEAs in Michigan also denied access to eligible students. For example, one LEA
           denied SES to low-income students because other schools provided SES to low-achieving
           students who were not low-income or were enrolled at a school that was not required to
           offer SES. Another LEA denied SES to low-income students who were at or above a
           certain academic achievement level so it would have SES funds available for low-
           achieving students who applied for SES later in the school year. LEAs should not deny
           SES to eligible students in anticipation of future requests from lower-achieving students.

The Department first issued SES guidance in August 2003 and issued revised guidance in
June 2005. Question A-5 of the Department’s guidance reiterated the applicable ESEA provision
and regulatory language on prioritization and instructed LEAs to use objective criteria to identify
the lowest achieving students if prioritization was necessary.5 Question F-3 of the revised
Department guidance contains the following additional instructions:

           An LEA should not assume, before it contacts parents, that it will have limited
           resources for supplemental educational services. Rather, the LEA should notify
           all eligible families of their children’s eligibility. Only if more families request
           supplemental educational services than there are funds available to serve should
           the LEA set priorities or criteria to determine which eligible students can get
           services. The LEA should review the information available about the
           performance of eligible students and apply those priorities or criteria in a manner
           that is careful, fair, and objective.

The revised Department guidance in Question F-3 makes it clear that LEAs should determine the
actual demand for SES before determining whether prioritization is necessary.

OIG Suggestions for Enhancing the Department’s
Guidance on SES Prioritization

The audit work performed by OIG indicates that there has been confusion at the LEA level
surrounding the SES prioritization provisions. The Department should continue to monitor
implementation of the prioritization provisions of the ESEA and be prepared to enhance its
guidance further if LEAs continue to misinterpret the applicable requirements. If further
enhancements are warranted, we suggest that the Department’s guidance include a step-by-step
process that LEAs can follow to ensure consistent determinations on whether an LEA should
prioritize. Illustrative examples that show different scenarios in which LEAs either should or
should not prioritize SES might also help guide LEAs through this important process.


Section 2: LEA Preparation of Student Learning Plans

This section discusses our findings related to the preparation of Student Learning Plans (SLP)
and offers a suggestion for enhancing the Department’s guidance.

ESEA § 1116(e)(3)(A) requires that the LEA “develop, in consultation with parents (and the
provider chosen by the parents), a statement of specific achievement goals for the student, how
the student’s progress will be measured, and a timetable for improving achievement that, in the

5
    Question A-5 of the guidance was not modified when the revised guidance was issued in June 2005.
Final Management Information Report                                                       ED-OIG/X09G0007 

State and Local No. 06-02                                                                       Page 6 of 17 


case of a student with disabilities, is consistent with the student’s individualized education
program under section 614(d) of the Individuals with Disabilities Education Act.”
ESEA § 1116(e)(3)(B) requires a description of how the student's parents and teacher(s) will be
regularly informed of the student's progress. The required elements are reiterated in
34 C.F.R. § 200.46(b)(2) and are also described in the response to Question G-2 in the revised
Department guidance.

Our reviews of four LEAs’ contracts with SES providers and one LEA’s delivery of SES found
that four of the five LEAs either did not develop SLPs or omitted some of the elements required
by ESEA § 1116(e)(3)(A) and (B). Officials from two of the LEAs indicated that they lacked the
time and resources to develop a statement of goals, description of how progress will be
measured, and a timetable for improvement for each student.6

    ƒ   Our review of OUSD’s contracts with ARC for the 2003-2004 and 2004-2005 school
        years found that the contracts did not include most of the elements required by the
        applicable ESEA provisions and Federal regulations. As a result, SLPs were not
        developed for any students that received SES from ARC. OUSD officials acknowledged
        that they did not prepare SLPs because of resource constraints.

    ƒ   While the contracts between LEI and SUSD provided for the development of individual
        SLPs, they were not prepared for most students. SUSD developed a SLP template that
        contained areas to record the information required by the contract and ESEA § 1116(e)(3)
        and an area for LEI staff and parent signatures. For school year 2003-2004, the District
        SES Coordinator intended to use the template to prepare plans for every student receiving
        SES but did not have enough time to meet with all parents and providers to complete the
        SLPs. For school year 2004-2005, the SES Coordinator stated that the District decided
        not to develop individual student plans due to the increasing demand for SES and the
        limited availability of District staff to prepare the plans. Instead, SUSD specified in the
        contract that LEI and SUSD would be jointly responsible for development of SLPs.
        However, no SLPs were prepared for SUSD students receiving SES that year.

LAUSD was the only LEA whose SLPs met all the requirements of the ESEA. LAUSD required
providers serving the District to prepare the SLPs with the student’s parents, who also signed the
SLPs. The District’s standard contract for SES providers incorporated individual SLPs
containing a statement of specific achievement goals for each student, how the student’s progress
will be measured, and a timetable for improving achievement. The contract also incorporated an
individual service agreement that described how PTA would deliver the services. LAUSD
reviewed the individual SLPs and either accepted each as submitted or required PTA to provide
additional information. LAUSD did not pay PTA for SES unless a SLP was completed and
approved. LAUSD’s SES Program Specialist informed us that the District delegated the
preparation of SLPs to SES providers due to District resource constraints.




6
  The other two providers cited other reasons for why the SLP requirements were not met. SUHSD did not prepare
student agreements for students at the schools that were reviewed because the SES Coordinators were unaware that
the agreements were required. SDCS’ policy did not require schools to prepare SLPs for students with disabilities
receiving SES who already had individualized education programs, and some schools failed to prepare student
agreements for other students receiving SES.
Final Management Information Report                                            ED-OIG/X09G0007 

State and Local No. 06-02                                                            Page 7 of 17 


Beginning with school year 2005-2006, SDCS has also decided to have SES providers prepare
SLPs because of resource constraints. The District is implementing a web-based system to
manage SES provided by the District and other SES providers, including the preparation of SLPs
conforming to the Federal requirements. The system includes an individual SLP to be completed
for each student receiving SES that contains the elements identified in ESEA § 1116(e)(3)(A)
and (B). SDCS will review all SLPs prior to paying non-district providers or allowing Federal
funds to pay for District-provided SES.

OIG Suggestion for Enhancing the Department’s
Guidance Related to SLPs

The Department should consider enhancing its guidance to specify that LEAs have the option of
contracting with SES providers to prepare individual SLPs, in consultation with the LEA and the
student’s parents, while continuing to hold the LEA responsible for ensuring that the SLPs
include all of the elements contained in ESEA § 1116(e)(3)(A) and (B). Contracting for SLP
preparation would alleviate some constraints on LEAs’ time and resources and still ensure that
1) SES were tailored to the academic needs of each student, and in the case of a student with
disabilities, consistent with the students individualized education program, and 2) parents and the
student’s regular teachers are advised of the progress made in meeting the student’s academic
goals.


Section 3: Provider Payments for SES

This section discusses our observations on LEA payments to SES providers and provides
suggestions for enhancing the Department’s guidance.

There are no statements in ESEA or Federal regulations governing fiscal matters related to SES
providers such as how the provider bills for services, the amount a provider can charge for its
services, or whether a provider can be paid when a student does not attend tutoring sessions.
ESEA § 1116(e)(3)(D) only requires that the LEA/provider agreements “contain provisions with
respect to the making of payments to the provider by the local educational agency.” However,
the Department’s guidance allows SEAs to specify, at their discretion, certain program design
parameters including establishing acceptable ranges for providers’ billing rates and student-
teacher ratios. Question B-3 of the guidance states:

       [A]s part of its process to approve providers and ensure that supplemental
       educational services are of the highest quality, an SEA may establish certain
       program design criteria for providers to meet. An SEA could, for example, set a
       range for acceptable student/tutor ratios. If establishing criteria for student/tutor
       ratios, an SEA should define acceptable ranges (e.g. 1-10:1 ratio) as opposed to
       absolute values (e.g. 6:1) in order to not unduly restrict providers’ service delivery
       options.
Final Management Information Report                                         ED-OIG/X09G0007 

State and Local No. 06-02                                                         Page 8 of 17 


Question B-4 includes the following guidance:

       An SEA may, if it so chooses, define acceptable ranges for program design
       parameters that influence the hourly rates providers charge throughout the State,
       in order to prohibit grossly exorbitant charges or unrealistically low rates. An
       SEA should avoid arbitrarily setting uniform pricing or hourly rates and, if
       defining acceptable program design parameters for providers, should consider the
       following factors:
           ƒ The pupil/tutor ratio; 

           ƒ The variation in per-pupil allocations among LEAs in the State; 

           ƒ The number of instructional hours; 

           ƒ The qualifications (and therefore cost) of the tutoring staff; 

           ƒ The cost of instructional materials and equipment (books, computers, 

               manipulatives, etc.);
           ƒ The amount of rent charged by the LEA and other landlords (including
               variations throughout a State);
           ƒ The LEAs’ payment policies regarding attendance; and
           ƒ The variation in the cost of doing business among LEAs in the State.

       An SEA should avoid setting uniform rates within the State, because this could
       ultimately limit parents’ choices of providers or reduce services provided to
       students. Uniform hourly rates do not accommodate local variations in charges
       and payment schedules, and may result in rates that underpay providers in more
       expensive markets and overpay them in less expensive ones. In the case of
       underpayment, this may lead to providers being unable or unwilling to service the
       market, which will then limit parental choice.

The Department’s guidance also states that an LEA may impose reasonable administrative and
operational requirements through its agreements with SES providers. Question G-4 of the
guidance states:

       [A]n LEA may include, in its contracts with providers, administrative provisions
       dealing with issues such as the fees charged to providers for the use of school
       facilities, the frequency of payments to providers, and the issue of whether
       payments will be based in part on student attendance.

Observations on LEA Payment Policies

All five of the LEAs associated with the provider audits only paid providers for students that
actually attended tutoring sessions. However, an “actual attendance” payment policy increases
the business risk of SES providers because they hire and pay tutors based on anticipated demand
that may ultimately be significantly lower than expected because of low student attendance.

Low student attendance was a significant issue for PL in serving SUHSD in 2004-2005. Of the
88 students that enrolled with the provider, 20 students (23 percent) never attended a tutoring
session and 24 other students (27 percent) attended five or fewer sessions. Thus, half of the
students that PL expected to serve either never attended or had very low attendance. On average,
Final Management Information Report                                                         ED-OIG/X09G0007 

State and Local No. 06-02                                                                         Page 9 of 17 


SUHSD students enrolled with PL only attended 11 out of 37 (30 percent) of the sessions offered
in 2004-2005. Only 2 of 88 students attended all 37 sessions even though the provider offered an
incentive for regular attendance.7 Poor attendance partially explains the comparatively low
revenue generated by PL, as shown in Table 3.

The increased business risk of SES providers inherent in an “actual attendance” payment policy
could cause a reduction in the number of providers willing to serve a particular LEA because the
number of students enrolled with the provider, coupled with the provider’s expectations on
student attendance (and thus revenue), may not justify serving an LEA. Even though an LEA
could choose to pay a provider based in whole or in part on enrollment (as opposed to actual
attendance) under the Department’s guidance, none of the reviews we conducted involved a
LEA/provider arrangement in which enrollment, rather than attendance, was the basis for
payment.

Observations on Provider Revenue

Four of the five providers we reviewed charged for their services on a per-hour and per-student
basis. These providers billed LEAs for each tutoring session at a specified hourly rate for each
student attending a session. For example, if PL taught a one-hour session with 15 students in
attendance, it would have billed the LEA for $450 for that session. This amount is the result of
multiplying the provider’s hourly rate per student of $30 by 15 students in attendance.

Multiple factors affected provider revenue/LEA costs. The revenue generated by SES providers
we reviewed (and thus the cost incurred by the LEAs) ranged from about $28,000 to over
$1.1 million in school year 2004-2005. Factors such as the hourly rate, a provider’s individual
billing method, the number of sessions provided, and student attendance affected the providers’
ultimate 2004-2005 revenue. Table 3 provides the school year 2004-2005 revenue received by
each provider from the respective LEA.

Rates varied by type of entity. As shown in Table 3, the for-profit entities we reviewed charged
the highest rates for tutoring services, ranging from $30 per hour to $43 per hour, per student in
2004-2005. In contrast, the non-profit provider only charged $16.50 per hour, per student.
However, ARC also charged OUSD an additional 45 minutes of setup time per session, per
student, which effectively increased its direct rate to about $23 per hour, and billed separately for
what ARC characterized as start-up costs. (Unique aspects of ARC billings that increased
OUSD costs beyond the actual hourly rates charged are discussed in more detail on page 11.)
The LEA provider was the lowest cost SES provider by a wide margin. Rather than charging for
SES tutoring per hour and per student, SDCS only charged for those hours its teachers served as
SES tutors at the teachers’ normal hourly rate of pay. Thus, the charge to Federal funds was a
teacher’s hourly rate of pay for the number of hours of tutoring provided, regardless of how
many students attended the session.




7
 Students received points for attendance and completion of tutoring modules. Students attending at least 30 hours
of tutoring were rewarded with a retail store gift card not to exceed $25, depending on the number of points earned.
Final Management Information Report                                                           ED-OIG/X09G0007 

State and Local No. 06-02                                                                          Page 10 of 17 



              Table 3: Provider Revenue in School Year 2004-2005 from Reviewed LEAs
                                            2004-2005 Hourly Rate
         Provider             Type                                         Amount Paid
                                                 per Student
    ARC                     Non-profit                               $16.50                            $598,958
    LEI                     For-profit                                39.00                           $253,370
    PTA                     For-profit                               $43.00                          $1,150,132
    SDCS                    District                             Variable (a)                         $744,227
    PL                      For-profit                               $30.00                             $28,410
    (a) SDCS’s hourly rate was based on the teacher’s rate of pay and was not charged per student. The billing rate
    varied from the base teacher salary of $34.76 per hour to a maximum of $45.73 per hour at the schools included
    in our audit.


Provider rates affected hours provided. The rates charged by SES providers directly affected the
maximum number of hours of tutoring each student could have received in 2004-2005. As
shown in Table 4, the maximum tutoring hours available to students served by for-profit
providers were comparable. In contrast, students served by ARC could have received
significantly more hours of tutoring than those served by the for-profit providers, even after
adjusting ARC’s hourly rate to reflect the non-instructional time added to its billings.8 Because
SDCS did not charge per-hour and per-student for SES, in effect, the maximum number of hours
of tutoring an individual student could have received was only constrained by the number of
sessions and hours the provider offered during the school year.

                        Table 4: Maximum Tutoring Hours Available in 2004-2005
                                                             2004-2005 SES      Maximum Per
                         Hourly
     Provider                            Billing Method        Per Pupil       Student Hours of
                          Rate
                                                               Allocation          Tutoring
    ARC                        $22.69      Hourly per student (a)              $1,624.59                       71
    LEI                        $39.00      Hourly per student                  $1,379.96                       35
    PTA                        $43.00      Hourly per student                  $1,580.63                       36
    SDCS                      Variable     Hourly per tutor (b)                $1,462.74                       (c)
    PL                         $30.00      Hourly per student                  $1,237.04                       41
    (a) ARC’s maximum per student tutoring hours reflects the cost of the 45 minutes the provider billed to OUSD
    for each student served in each session. The rate shown above does not include amounts ARC charged for
    start-up costs that are itemized in Table 6.
    (b) Unlike the other providers we audited, the SDCS provider charged for SES based on the salary expense
    (including fringe benefits) for each individual District teacher serving as an SES tutor. Thus, this district
    provider billed the same amount for each tutoring session regardless if the tutor served one student or 10
    students in a particular session.
    (c) SDCS’ maximum per student tutoring hours cannot be calculated because the provider does not charge on a
    per-hour, per-student basis.



For-profit group providers’ revenue was significantly higher. The cost of one hour of tutoring,
as well as the amount a “typical” tutoring session would cost an LEA varied significantly
because of (1) differences in the rates charged, (2) the maximum student/teacher ratio, and
(3) how providers bill for services. Comparison of the data in Table 5 shows that the two

8
 The higher number of hours available to ARC students is also due in part to the higher per pupil allocation in
OUSD.
Final Management Information Report                                                       ED-OIG/X09G0007 

State and Local No. 06-02                                                                      Page 11 of 17 


for-profit providers (LEI and PL) that offered their services in a group setting were able to
produce a much greater amount of revenue per tutoring hour than the for-profit provider offering
individual tutoring or the non-profit and LEA providers that charged much lower rates for SES.
Likewise, these same two for-profit providers could generate much greater revenue per tutoring
session than the other providers.

               Table 5: Providers’ Maximum Hourly and Tutoring Session Revenue
                              Maximum                           Maximum        Maximum
                 Hourly                   Maximum Session
    Provider                 Students per                        Hourly         Session
                  Rate                     Duration (hours)
                               Teacher                          Revenue        Revenue
    ARC            $22.69                    4                     2.0            $90.76          $181.52
    LEI            $39.00                    8                     1.5           $312.00          $468.00
    PTA            $43.00                    1                     2.0            $43.00           $86.00
    SDCS          Variable                  10                     1.5            $45.73           $68.60
    PL             $30.00                   15                     1.0           $450.00          $450.00

Substantial differences in the revenue providers can generate per hour, based on whether they
provide SES on an individual or group basis, may call into question the appropriateness of group
providers’ rates when the student/teacher ratio exceeds a certain level. All providers incur
similar types of costs to provide SES including wages and benefits, instructional materials, tutor
training, and overhead items such as background checks.9 However, the direct labor costs
(specific payroll costs attributed to each tutor) for an individual tutoring session remain fairly
constant whether tutoring is provided to an individual or a group. As Table 5 shows, PL could
generate $450 per hour in a group session, which is more than 10 times the hourly revenue PTA
could generate for one tutor hour of expenses incurred. In other words, both providers incur a
similar cost to provide one hour of tutoring, yet the group provider’s revenue is greater in
relation to its costs.

Unique Billing Strategies at ARC

Four of the five providers we reviewed factored all costs into the hourly rate billed for SES. The
other provider, ARC, billed separately for what it characterized as start-up costs. Table 6
provides specific details on the amounts ARC billed OUSD for start-up costs in the first three
years it provided SES and shows the amount OUSD had paid at the time of our review.




9
 SES providers may also incur costs for tutoring facilities, such as rent paid to an LEA to use its classrooms,
although none of the four non-LEA providers we reviewed that used LEA facilities were required to pay for space.
Final Management Information Report                                                     ED-OIG/X09G0007 

State and Local No. 06-02                                                                    Page 12 of 17 


                     Table 6: ARC Start-up Costs Billed and OUSD Payments
                                  2002-2003             2003-2004           2004-2005
         Description
                              Billed      Paid      Billed      Paid   Billed       Paid
 Program Planning
                                $13,500      $13,500      $17,463      $17,463      $13,900
 (managers’ salaries)
 Curriculum & Related
                                 $9,099       $9,099      $40,320      $40,320      $38,017
 Materials
 Fingerprinting &
                                   $608         $608       $3,589       $3,589       $1,680      None (a)
 Background
 Training                         ---          ---         $8,265       $8,265      $11,798

 Indirect Costs                  $1,857       $1,857        ---          ---          ---

 Total                          $25,065      $25,065      $69,636      $69,636      $65,395
 (a) At the time of our review, OUSD had declined to pay any startup costs claimed by ARC for 2004-2005.


We also discovered that ARC was billing OUSD for what we describe as “non-instructional”
time, without the knowledge of OUSD personnel. ARC management indicated this time was for
tutor setup/cleanup before and after a tutoring session. The non-instructional time was billed
per-session and per-student, at the same hourly rate that ARC billed for direct tutoring hours. In
2003-2004, ARC billed OUSD for 30 minutes of non-instructional time for each student served
during a session. If ARC served students at its maximum student/teacher ratio of 4:1 during a
session lasting two hours, it billed OUSD for two hours of non-instructional time for every eight
hours of direct tutoring. This represented a 25 percent surcharge on top of the tutoring hours
provided and accounted for more than $118,000 (20 percent) of the $592,000 of tutoring revenue
(excluding startup costs) that ARC received from OUSD. In 2004-2005, ARC increased the
non-instructional billing time to 45 minutes per student, per session. This increased its
non-instructional hours billed per session from two to three hours when four students attended a
two-hour session. This resulted in the surcharge increasing to 38 percent of the direct hours
billed per session and increased ARC’s non-instructional revenue to 27 percent of the amount
received from OUSD.10

OIG Suggestions for Enhancing
Guidance on SES Provider Payments

Questions B-3 and B-4 of the Department’s guidance define the authority SEAs have in
establishing guidelines on specific program design parameters such as the rates providers charge
and student/teacher ratios. Question G-4 of the guidance informed LEAs that they may impose
reasonable administrative and operational requirements through their agreements with SES
providers. We believe the Department could provide more specific instruction to SEAs and
LEAs on issues related to provider payments by considering the following actions and suggested
enhancements to its guidance:



10
 We only reviewed two months of billings in 2004-2005 because our audit of ARC was initiated in April 2005 and
ARC’s 2004-2005 SES program had only been operating since the end of January 2005. During this period, ARC
was paid more than $51,000 for non-instructional time out of total payments from OUSD of about $187,000.
Final Management Information Report                                             ED-OIG/X09G0007 

State and Local No. 06-02                                                            Page 13 of 17 


ƒ   Assess the impact that provider payments based solely on “actual attendance” have had on
    the number of SES providers willing to serve an LEA. If the impact is significant, the
    Department should consider providing additional guidance regarding LEA discretion over
    payment methods that could reduce a provider’s business risk due to low student attendance
    and still provide for the effective use of Federal funds. Informing LEAs of flexibility that is
    available to retain and/or attract more providers would help to meet an important program
    goal—increasing the provider options available to parents. It is also possible that providers
    would be willing to operate with lower student/teacher ratios, which could be beneficial to
    students, if payments were not based entirely on actual student attendance.

ƒ   Clarify that SEAs are also authorized to establish separate hourly rate ranges for providers
    depending on whether they serve students individually or in a group. This clarification
    would help SEAs to prevent charges for SES that would be considered unreasonable.

ƒ   Identify “promising practices” related to contractual relationships between LEAs and SES
    providers. The Department’s guidance should encourage LEAs to develop provider
    agreements that fully disclose how providers bill for services, including identifying specific
    payment provisions such as the provider’s hourly rate, the length of each session, and any
    additional charges that would be included in billings.

ƒ   Clarify that a reasonable amount of non-instructional time could be afforded to SES
    providers and stress that LEAs should identify in their contracts with SES providers the
    amount of setup/cleanup time each provider will be compensated for, if not already built into
    its hourly rate for direct tutoring services. It is reasonable for an SES provider to be
    compensated for all hours spent serving SES eligible students, including a reasonable amount
    of time for setup/cleanup each time a session is held. Thus, charging an LEA for 30 to 45
    minutes of setup/cleanup time each day that a tutoring session is held seems reasonable.
    However, we believe it is inappropriate for a provider to bill for this non-instructional time in
    the same manner it bills for direct tutoring hours, i.e. 45 minutes of non-instructional time for
    each student, each session.


Section 4: SES Provider Effectiveness

This section discusses our observations on a number of issues related to provider effectiveness
determinations and offers suggestions for enhancing the Department’s guidance.

ESEA Section 1116(e)(4)(D) requires SEAs to “develop, implement, and publicly report on
standards and techniques for monitoring the quality and effectiveness of the services offered by
approved providers under this subsection, and for withdrawing approval from providers that fail,
for 2 consecutive years, to contribute to increasing the academic proficiency of students
served ….”
Final Management Information Report                                                      ED-OIG/X09G0007 

State and Local No. 06-02                                                                     Page 14 of 17 


In Question D-2, the Department’s guidance states:

        [E]ach SEA should develop a system for gathering information about the
        effectiveness of providers on an annual or periodic basis. Below are some
        examples of information that an SEA might want to collect; however, each SEA
        should tailor its system to its own needs:
             ƒ   Academic gains made by students who participated in and completed a
                 provider’s program;
             ƒ   The fidelity with which a provider’s program, as enacted, reflects its
                 program design, as proposed in its application to the SEA;
             ƒ   Student enrollment (including enrollment of students with disabilities
                 and English language learners) and daily attendance in a provider’s
                 program;
             ƒ   Parents’ and students’ satisfaction with a provider; and
             ƒ   How often a provider reports students’ progress to teachers and
                 parents.

The Department’s guidance also states in Question D-3:

        Student performance can be measured in a number of ways. For example,
        providers might use their own assessments, or could use standardized assessments
        given by the State or LEA. The best practice would be to specify, in the contract
        between the LEA and the provider, the assessment or assessments that will be
        used.

CDE Evaluations of Provider Effectiveness

California issued final state regulations in May 2005 that require SES providers to submit annual
end-of-fiscal-year reports to the California Department of Education (CDE). Section 13075.3 of
Title 5 of the California Code of Regulations lists the information to be provided, including
beginning and ending scores on national, state, district, or other assessments in English language
arts and/or mathematics for individual students. Section 13075.3 also requires that the
assessments used by providers “must be developed in accordance with the standards for validity
and reliability as set forth in the Standards for Educational and Psychological Testing (1999).”11
CDE will use the assessment data submitted by SES providers in the end-of-fiscal-year reports to
evaluate each provider’s effectiveness in increasing student achievement.

The first SES provider report is due on October 1, 2006 and will cover services provided
in school year 2005-2006.12 CDE advised us that the submitted information on student
achievement on the provider assessments will be compared with the students’ results on
the California Standards Tests (CST) for each student served by the provider. CDE also
advised us that it is in the process of developing a formal mechanism to obtain user
feedback on SES providers.

11
  The standards are jointly published by the American Educational Research Association, the American
Psychological Association, and the National Council for Measurement in Education and were last revised in 1999.
12
  CDE advised us that a contractor conducted year-end electronic consumer satisfaction surveys in school years
2002-2003 and 2003-2004 and that the survey information received from school districts was used by CDE when
providers re-applied to continue as approved providers.
Final Management Information Report                                            ED-OIG/X09G0007 

State and Local No. 06-02                                                           Page 15 of 17 


Provider Assessments Used and
Ability to Meet CDE Requirements

We determined that the five providers we audited were capable of providing the types of
information that CDE would require for provider monitoring. Table 7 shows the type of data
each provider had available during the audit period that would have been used to demonstrate
effectiveness if CDE had requested assessment data from the providers. Four providers were
using diagnostic assessments to evaluate student progress by comparing student test scores at the
beginning and end of the provider’s program (pre and post test scores). SDCS had access to
individual student scores on the CST and would have compared the scores from the prior school
year to the school year just completed to gauge student progress.

               Table 7: Provider Effectiveness Data Available During Audit Period
               Provider                               Effectiveness Data Available
 ARC                                     Scores from pre and post test diagnostics
 LEI                                     Scores from pre and post test diagnostics
 PTA                                     Scores from pre and post test diagnostics
 SDCS                                    CST scores
 PL                                      Scores from pre and post test diagnostics

CDE now requires prospective providers to identify the assessment data they plan to use to
demonstrate effectiveness as part of the SES provider application for state approval. However,
CDE does not maintain a list of assessments that meet the Standards for Educational and
Psychological Testing cited in its SES regulations for use in determining whether the assessment
data is acceptable.

One provider that was planning to use assessments contained in a new software program it
purchased to demonstrate effectiveness in school year 2005-2006 had to abandon this approach
because it learned through our audit that the assessments had not been evaluated against the
standards in California’s SES regulations. This situation occurred even though the software
vendor itself was an approved provider in California for school year 2005-2006. Neither the
provider nor the vendor was aware of this problem until we alerted them to it. A software
vendor representative indicated that its assessments would be evaluated against the standards in
California’s SES regulations. However, it is likely that the evaluation will not be completed
before the provider is required to submit effectiveness data to the State. As a result, the provider
indicated that it will only be able to serve LEAs in California that are willing to provide
individual students’ CST scores, until the evaluation of the assessments is completed.

Availability of Standardized Test Results

In cases where a provider intends to use standardized test scores to demonstrate effectiveness, it
may have difficulty meeting CDE’s reporting requirements if an LEA is not willing to provide
such data. The SES Coordinator at one LEA did not share individual student academic
achievement data with SES providers in school years 2003-2004 and 2004-2005 because the
Coordinator was not aware that student information could legally be shared with providers with
written authorization from parents. Title 5, Section 49075 of the California Education Code
allows a LEA to share student academic information with an SES provider, as long as the LEA
obtains the parent’s consent. Officials at another LEA were reluctant to provide any student
information to providers beyond the student’s name and identification number. SES providers
Final Management Information Report                                          ED-OIG/X09G0007 

State and Local No. 06-02                                                         Page 16 of 17 


that choose to demonstrate effectiveness using standardized test scores must have access to this
information.

OIG Suggestions to Enhance the Department’s
Guidance Related to SES Provider Effectiveness

We suggest that the Department take the following actions to enhance its guidance:

ƒ   Encourage SEAs to ensure that all providers are informed as to how they will be evaluated.

ƒ   In cases where SEAs will require providers to submit effectiveness data, encourage SEAs to
    require prospective providers to identify the specific assessments that would be used to
    demonstrate effectiveness in the provider applications. The guidance should also encourage
    SEAs to determine whether the assessment data providers plan to use will be accepted as
    evidence of provider effectiveness during the provider approval process, rather than waiting
    until the provider evaluation occurs and then possibly rejecting the effectiveness data.

ƒ   Encourage LEAs to provide individual students’ scores on standardized tests to providers, if
    needed to demonstrate effectiveness, provided that parents have authorized the release of
    such information.




                          PURPOSE AND METHODOLOGY 



The purpose of this project was to provide information to the Office of Elementary and
Secondary Education and Office of Innovation and Improvement on specific issues that were
identified during the individual SES audits and offer suggestions for enhancing the Department’s
Supplemental Educational Services Non-Regulatory Guidance. To achieve our purpose, we
gained an understanding of the ESEA sections, Federal regulations, Department guidance,
external publications, and California State Regulations covering SES. We reviewed the audit
reports and associated audit documentation from the five SES provider audits. We also reviewed
the final OIG audit reports on SEA and LEA implementation of School Choice and SES
provisions in the states of Illinois, Michigan, Indiana, New Jersey, Delaware, and Nevada. We
provided a copy of the Draft Management Information Report to Department officials and
requested comments. The Department’s response is attached below.

We briefed Department officials on the information that would be presented in this Management
Information Report on May 19, 2006.


Attachment
Final Management Information Report                       ED-OIG/X09G0007 

State and Local No. 06-02                                      Page 17 of 17 





     Attachment: Department Comments to Management Information Report
                            UNITED STATES DEPARTMENT OF EDUCATION

                                omCE OF ELEMENTARY AND SECONDARY EDUCATION



TO: 	             Gloria Pilotti
                  Regional Inspector General for Audit Services
                  Office of Inspector General                                                   JUL 25 2006
F)~OM: 	          Henry L. Johnson
                  Assistant Secretary
                  Office of Elementary and                o..;.'U-•. " ....... 





                  Christopher Doherty 

                  Acting Assistant Deputy Secretary 

                  Office of Innovation and Improvement 


CI~: 	            Richard Rasa
                  A&A Director
                  Office of Inspector General

s{ rBJECf: 	 Implementation of Supplemental Educational Services
                  (SES) in California
                  Control Number ED-OIG/X09GOOO7

'I1l.ank you for giving us the opportunity to review the draft Management
Iniormation Report (MIR), Implementation of Supplement~ Educational
Services in California, prepared by the Office of Inspector General (OIG)..The·
report provides a summary of the five SES provider audits conducted by the OIG
. five local educational agencies (LEAs) in California. The' auditors selected a
v 'ety of provider types (for-profit, non-profit, LEA) and delivery modes and
r liewed a wide range of issues, including LEA implementation of SES
p .oritization, LEA preparation of student learning plans, provider payments for
S S (including methods of calculating costs of services, rates charged for
    'ces, billing and general payments), and provider effectiveness. In each of
   se areas, the OIG presents suggestions for enhancing the Department's SES
n n-regulatory guidance.

   e Office of Elementary and Secondary Education and the Office of Innovation
     Improvement appreciate the OIG's recommendations and concur with many
     em. Although we do not anticipate revising the current SES non-regulatory
   idance prior to the reauthorization of the Elementary and Secondary
E( ucation Act (ESEA) scheduled to begin next year, there are several other



                                     400 MARYLAND AVE., S.W., WASHINGTON, D.C. 20202
                                                       www.ed.gov

         OUr missimt is   eo en.swe e q u a l _ eo ecfucation and eo ptDfIIDte eciucaliorud ~ tJuoughout !he nation.
qoria PHotti - Page 2

mechanisms that the Department will consider using to share additional SES
~tidance with States and LEAs, including:

       •   Sending "letters to the field" to clarify or amplify Department policy and
           guidance;
       •   Providing information at national conferences and meetings of education
           administrators and practitioners; and
       •   Providing ongoing technical assistance to State educational agencies
           (SEAs) and LEAs as they implement SES.

w ·~   will also use the information and recommendations the OIG has provided to
~:orm  discussions (both internally and externally) about the reauthorization of
thr ESEA and expand and improve the Department's mortitoring of SEAs in the
implementation of SES. Finally, we recommend that the OIG indicate in its
reo ort that, although these recommendations are directed toward the
D~partment, SEAs have an important role to play in mortitoring their districts for
c~npliance with the SES provisions of the No Child Left Behind Act of 2001 and
th" Department's Title I regulations and providing them with the necessary
tee hnical assistance. Although there is clearly a significant role for the
D~partment to play in these matters, the SEAs also must take responsibility and
leddership to ensure that their LEAs are fully compliant with the statute and
re,;uIations goverrtiug SES.

Ag·ain, we appreciate the information provided by the OIG and also for the
op portunity to provide comment on the draft MIR.