oversight

Learning Excitement Incorporated and Stockton Unified School District's Compliance With Supplemental Educational Services Provisions.

Published by the Department of Education, Office of Inspector General on 2005-11-12.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                                 UNITED STATES DEPARTMENT OF EDUCATION

                                        OFFICE OF INSPECTOR GENERAL 

                                                 501 I STREET, SUITE 9-200
                                              SACRAMENTO, CALIFORNIA 95814
                                         PHONE (916) 930-2388 • FAX (916) 930-2390



                                                      November 10, 2005
                                                                                                                 Control Number
                                                                                                               ED-OIG/A09F0012

Jack T. O’Connell
State Superintendent of Public Instruction
California Department of Education
1430 N Street
Sacramento, CA 95814

Dear Superintendent O’Connell:

This Final Audit Report, entitled Learning Excitement Incorporated and Stockton Unified
School District’s Compliance With Supplemental Educational Services Provisions, presents the
results of our audit. The purpose of the audit was to determine whether, for school years
2003-2004 and 2004-2005, (1) Stockton Unified School District (SUSD) contracts with
Learning Excitement Incorporated (LEI) for providing supplemental educational services (SES)
contained the elements specified in the Elementary and Secondary Education Act of 1965
(ESEA), as amended by the No Child Left Behind Act of 2001 and applicable Federal
regulations; (2) LEI performed the services for which it received payment under the contracts
and the services were provided in a manner consistent with the contract terms and Federal
requirements; and (3) LEI collects and maintains the data that will be used by the California
Department of Education (CDE) to evaluate the quality and effectiveness of the services offered
by the provider.



                                                     BACKGROUND 



Title I, Part A of the ESEA requires local educational agencies (LEAs) to offer SES to students
from low-income families when the students attend a Title I school that is in the second year of
school improvement, or identified for corrective action or restructuring.1 SES consists of
tutoring, remediation, and other educational interventions that are designed to increase the
academic achievement of students, and are in addition to instruction provided during the school
day. State-approved SES providers, selected by the individual student’s parent or guardian,
provide the services to eligible students under agreements with LEAs. SES providers must align

1
  Under the No Child Left Behind Act of 2001, Title I schools that fail to make adequate yearly progress (AYP) for
two consecutive years are identified for school improvement. Title I schools are identified for corrective action if
they do not make AYP for four years, while Title I schools not making AYP for five years are identified for
restructuring. The “low-income family” determination is usually based on the student’s eligibility for free or
reduced price lunch under the National School Lunch Program.

            Our mission is to promote the efficiency, effectiveness, and integrity of the Department’s programs and operations.
Final Report
ED-OIG/A09F0012                                                                                   Page 2 of 15

their instructional programs with state academic content standards and tailor their services to the
academic needs of individual students. CDE is the state educational agency responsible for
administering the ESEA, Title I, Part A program, approving SES providers, and monitoring the
quality and effectiveness of services offered by the approved providers.

LEI, also known as Reading Revolution, is a privately held for-profit corporation based in
Concord, California, which was initially approved as an SES provider in California for school
year 2002-2003. LEI provides tutoring in English language arts in one-on-one or small group
sessions with a student-to-teacher ratio of eight to one or less. The tutoring is provided after the
regular school day at the school site for 1½ to 3 hours, depending on the program structure,
timing, and budget. LEI tutoring sessions ran for 2 to 10 weeks. LEI served 61 SUSD students
during school year 2003-2004 and 160 SUSD students during school year 2004-2005.2

In school year 2003-2004, SUSD allocated $3.67 million of Title I funds for SES and reported
that 1,141 students received services from 4 SES providers. In school year 2004-2005,
SUSD allocated $3.71 million of Title I funds for SES and reported that about 3,000 students
received services from 4 SES providers.




                                           AUDIT RESULTS 



We found that LEI’s contracts with SUSD included all elements required by the applicable
ESEA provisions and Federal regulations. However, neither SUSD nor LEI completed
individual student plans, as required by the ESEA § 1116(e)(3), the 2003-2004 and 2004-2005
contracts, and Federal regulations. LEI complied with other contractual and Federal
requirements reviewed as part of our audit.

Based on our review of selected invoices, we concluded that LEI provided SES to SUSD
students for which LEI received payment under the contracts. We also found that LEI provided
SES to students after the regular school day and the content of instructional material used by LEI
was aligned with California’s student academic achievement standards. LEI’s Chief Executive
Officer (CEO) stated that LEI gave progress reports to schools and parents; however, we could
not confirm from the available documentation that progress reports had been, in fact,
disseminated to schools and parents.

The California State Board of Education adopted regulations in January 2005 that require
SES providers to submit annual end-of-fiscal-year reports to CDE. The first report is due on
October 1, 2006 and will cover services provided from July 1, 2005 to June 30, 2006. The state
regulations list the information to be provided, including beginning and ending scores on
national, state, district, or other assessments in English language arts and/or mathematics for the
individual students served. Failure to contribute to student academic proficiency for two

2
 The number of students for school year 2004-2005 does not include additional students who were enrolled in one
session that was on-going at the time of our review.
Final Report
ED-OIG/A09F0012                                                                      Page 3 of 15

consecutive years, as demonstrated by assessment scores, is one reason CDE may remove a
provider from the state list of approved providers. We concluded that LEI currently maintains
the type of data that CDE will use to determine the quality and effectiveness of services.
However, we found that effectiveness data contained in assessment reports provided to SUSD,
and included in LEI’s 2005 provider application to CDE, were incomplete and inaccurate.

CDE did not explicitly express concurrence with our findings in its comments to the draft report,
but it did describe the corrective actions taken or planned to address each of our
recommendations. CDE’s comments are summarized at the end of each finding and the full text
of the comments is included as an attachment to the report.


FINDING NO. 1 –SUSD Did Not Complete Individual Student Plans for
               All Students Who Received SES

While the SES contracts between LEI and SUSD provided for the development of individual
student plans, neither SUSD nor LEI prepared such plans. ESEA § 1116(e)(3)(A) requires an
LEA to enter into an agreement with the approved SES provider that shall require the LEA to
develop a statement of specific achievement goals for the student, a description of how the
student's progress will be measured, and a timetable for improving achievement in consultation
with parents and the provider chosen by the parents. The lack of individual student plans
tailored to the academic needs of each student may result in students not receiving the services
they need to improve their academic achievement.

   • 	 SUSD Did Not Prepare Individual Student Plans for All Students in School Year
       2003-2004. The SES contract between LEI and SUSD for school year 2003-2004
       required “[SUSD] to develop, in consultation with the parents and the provider chosen by
       parents, a statement of specific achievement goals for the student, how the student’s
       progress will be measured, and a timetable for improving achievement.” SUSD
       developed a Student Learning Plan (SLP) form and the District SES Coordinator intended
       to prepare plans for every student receiving SES by completing the form. The SLP form
       contained areas to record the information required by the contract and ESEA § 1116(e)(3)
       and an area for LEI staff and parents to sign the plan.

       SUSD could only provide us with SLP forms for 26 of the 61 students served by LEI in
       school year 2003-2004. We also found that only 3 of the 26 provided SLP forms were
       complete. The other 23 SLP forms lacked required information and 15 of the 23 SLP
       forms also lacked signatures. SUSD’s SES Coordinator stated she did not have enough
       time to meet with all parents and providers to complete the SLP forms.

   • 	 SUSD Did Not Develop Individual Student Plans in School Year 2004-2005. The SES
       Coordinator stated that the District decided not to develop individual student plans for
       school year 2004-2005 due to the increasing demand for SES and the limited availability
       of District staff to prepare the plans. Instead, SUSD specified in the 2004-2005 contract
       that LEI and SUSD were jointly responsible for development of the individual student
       plans. The contract stated, “. . . PROVIDER and District shall develop a Statement of
Final Report
ED-OIG/A09F0012                                                                        Page 4 of 15

        Goals in consultation with the student’s parents . . . The Statement of Goals shall include
        a statement of specific achievement goals for the student, how the student’s progress will
        be measured, and a timetable for improving achievement . . . .” According to the
        SES Coordinator, the intent of the change was to shift the District’s ESEA § 1116(e)(3)
        responsibility for completion of the plans to SES providers. Thus, the District relied on
        the providers to develop and maintain the information required by the contract and
        ESEA § 1116(e)(3).

        An LEA cannot shift the responsibility for developing individual student plans to
        SES providers. The ESEA § 1116(e)(3) clearly states that an LEA’s agreements with
        SES providers shall require the LEA to develop, in consultation with the parents and the
        provider, a statement of specific achievement goals for the student, a description of how
        the student's progress will be measured, and a timetable for improving achievement.

Since SUSD did not prepare individual student plans for all students, we reviewed LEI’s student
files to determine if LEI had conducted the activities in school years 2003-2004 and 2004-2005
to satisfy the ESEA requirement. Our review of a random sample of 25 students who received
services found that LEI had documented specific achievement goals for 2 students, assessed the
strengths and weaknesses for 14 students, and prepared progress reports for 17 students. LEI’s
CEO stated that the documentation was not needed in individual student files because the goals,
timetables for improvement, and methods of measuring progress were built into LEI’s SES
program. Essentially, if a student enrolls with LEI, the goal is for the student to complete the
program by the end of the session (timetable). To measure progress, LEI has written procedures
requiring students to take an assessment test at the beginning and end of the session.

Also, at the time of our review, LEI could only provide files for 203 of the 221 students served
during the 2003-2004 and 2004-2005 school years. LEI’s CEO cited the difficulty of
maintaining records and instructional materials in school closets and auditoriums as the reason
for missing documentation in its student files. The 2003-2004 and 2004-2005 contracts between
LEI and SUSD required SES providers to maintain all fiscal and student records for five years.
Also, the storage of student records in school closets and auditoriums could expose the records to
unauthorized access by others unless the records are properly secured at those locations. The
ESEA § 1116(e)(3)(E) prohibits an SES provider from disclosing to the public the identity of any
student eligible for, or receiving, SES without the written permission of the parents.

Recommendations

We recommend that the Assistant Secretary for Elementary and Secondary Education, in
conjunction with the Assistant Deputy Secretary for Innovation and Improvement, require CDE
to—

1.1 	   Ensure that SUSD explicitly states in its SES contracts how the District will develop for
        each student a statement of specific achievement goals, a description describing how
        progress will be measured, and a timetable for improving achievement in consultation
        with the provider and parents as required by ESEA § 1116(e)(3), and that SUSD ensures
        such plans are completed for each student.
Final Report
ED-OIG/A09F0012                                                                        Page 5 of 15


1.2 	   Ensure that LEI maintains student records in accordance with the terms of its contracts
        with LEAs and that student records are stored in secured areas.

CDE Comments

In its comments on the draft report, CDE stated it would provide technical assistance via
workshops and written communications on (1) the requirements for implementing and providing
SES (including elements to be included in the agreement and how the LEA will develop a
statement of specific achievement goals for each SES student) and (2) securing student records in
accordance with contract provisions. CDE also plans to review SUSD’s revised SES agreement
template. CDE stated that SUSD has designed an Individual Student Learning Plan form that
will be distributed to SES providers when parents submit an application for service and that SES
will not be provided to students until the plan is completed and distributed to all parties. CDE
also described the changes LEI made to its procedures to ensure that it has an individual student
plan for each student prior to providing SES. CDE stated that LEI implemented additional
internal control procedures for assuring the proper handling, storage, and retention of SES
documents and provided its program managers with training on the new procedures.


FINDING NO. 2 – 	LEI Reported Effectiveness Data To SUSD and CDE That Were
                Incomplete and Inaccurate

LEI submitted assessment reports to SUSD at the end of each SES session to communicate the
effectiveness of the services provided to students. LEI measured effectiveness using grade
growth on pre- and post-session assessment tests. We found that the assessment reports omitted
students for which LEI had test scores and contained test scores that did not match information in
the student files. We also found inaccurate effectiveness data, contained in SUSD’s Van Buren
Elementary School assessment report, were included in a letter of reference with LEI’s
2005 SES application to CDE.

ESEA § 1116(e)(4)(B) requires states to consider a potential SES provider’s demonstrated record
of effectiveness prior to granting approval to provide services and § 1116(e)(2)(A) requires
LEAs to provide an annual notice to parents that includes a description of approved providers’
demonstrated effectiveness. ESEA § 1116(e)(4)(D) requires states to monitor the effectiveness
of services offered by approved providers. To ensure that states, LEAs, and parents have reliable
information to make decisions, provider reports on effectiveness data need to be complete and
accurate.

For a random sample of 25 students who received SES, we compared the original test scores
recorded in the student files to the results presented in the assessment reports. We found that—

    • 	 Eleven students were not included on the assessment reports. Student files for 6 of the
        11 students contained the pre- and post-test scores necessary to include the students in the
        assessment reports. The other student files did not contain at least one of the scores.
Final Report
ED-OIG/A09F0012                                                                        Page 6 of 15

    • 	 Fourteen students were included on the assessment reports, but information reported for
        five students differed from information in the student files.

LEI’s Office Manager attributed the exclusion of students and inaccurate information to human
error. Correction of the errors and omissions identified by our review would increase the
provider’s reported effectiveness. However, given the six student omissions and five students
with data errors identified in our 25 student sample, we concluded that LEI’s assessment reports
were not reliable.

Recommendation

2.1 	   We recommend that the Assistant Secretary for Elementary and Secondary Education, in
        conjunction with the Assistant Deputy Secretary for Innovation and Improvement,
        require CDE to ensure that LEI implements internal control procedures that provide a
        reasonable level of assurance that complete and accurate effectiveness data are reported
        to LEAs and in SES provider applications and future annual reports submitted to CDE.

CDE Comments

In its comments on the draft report, CDE stated that it plans to offer SES provider training on the
State’s SES reporting requirements and reporting form. CDE also stated it would advise LEI
about the State’s SES regulation requirements for accurate and full reporting of SES
effectiveness and that non-compliance with these requirements could be cause for termination as
a state-approved SES provider. CDE stated that LEI has revised its procedure to include
comparing 100 percent of the data entered by field personnel in an Excel spreadsheet to the
supporting documents. CDE will require LEI to submit a detailed description of its internal
control procedures to ensure full and accurate reporting of effectiveness data.

CDE also provided LEI’s position on the reliability of its reports. LEI acknowledged that human
error led to a small amount of misreported or unreported data resulting in some inaccuracy and
incompleteness to its report to SUSD and CDE. However, LEI asserted that the reports were
reliable. LEI stated that the reliability of data and reports means that they are dependable and
that different clinical experiments or statistical trials would yield the same or compatible results.
LEI claimed that the measures received from students in LEI programs at SUSD yielded results
that were compatible with the statewide results. Additionally, LEI stated that the materiality of
the finding should have been assessed before concluding the reports were not reliable. LEI
stated that findings that have a material affect on reported information are findings that when
disclosed to the user of the information would influence their opinion of the information. LEI
stated that inclusion of the missing data and correction of misreported data changed LEI’s overall
performance at SUSD by only 2.3 percent. LEI stated that the change in overall performance
was not material and should not affect SUSD’s or CDE’s assessment of LEI’s program.
Final Report
ED-OIG/A09F0012                                                                         Page 7 of 15

OIG Response

We have not changed our conclusion regarding the reliability of LEI’s reports. We reviewed the
data for 25 of 221 students who received SES during school years 2003-2004 and 2004-2005.
The fact that 11 of the 25 students in our sample were either omitted or had inconsistent data is
sufficient to raise concerns about the data entered for the other students and the reliability of LEI
reports that include information based on the data. We noted in the report that correction of the
errors and omissions identified by our review would increase the provider’s reported
effectiveness. However, given the error rate for our sample, there is a significant risk that data
may have been omitted or incorrect for students not reviewed. Thus, the 2.3 percent change
calculated by LEI may not be an accurate measure of the impact on the reported performance.



                                     OTHER MATTERS 



During our review, we found that SUSD did not properly identify students who were eligible for
SES in school year 2003-2004 or 2004-2005. Title 34 C.F.R. § 200.45(b) and (c) states that
students from low-income families at Title I schools in the second year of school improvement
or identified for corrective action or restructuring are eligible for SES. SUSD considers a student
to be low-income if the student qualifies for free-or-reduced-price-lunch under the National
School Lunch Act. However, in school years 2003-2004 and 2004-2005, SUSD determined a
student’s eligibility for SES based solely on a student’s performance on California standardized
tests. Thus, SUSD would not have identified SES-eligible, low-income students who performed
at the proficient level or above on the standardized tests. Additionally, SUSD would have
identified low-achieving students who were not low-income as eligible for SES. In its comments
to the draft report, CDE described the corrective action taken by SUSD. For school year 2005­
2006, SUSD identified all low-income students and noted, as eligible for SES, all students who
attend schools that were in the second or more year of school improvement. For funding
priority, the eligible students will be ranked from greatest to lowest academic need.

We also found SUSD did not share its individual student academic achievement data with SES
providers in school years 2003-2004 and 2004-2005. Title 5, Section 49075 of the California
Education Code allows a district to share student academic information with an SES provider, as
long as the district obtains the parent’s consent. SUSD’s SES Coordinator was unaware that
student information could legally be shared with providers with written authorization from
parents. The sharing of student academic data could be beneficial when SUSD is developing, in
consultation with the provider, the specific achievement goals for the student. In its comments
on the draft report, CDE described the procedures that SUSD will use to obtain parental consent
and that SUSD will release student academic achievement data to SES providers upon receipt of
the parental consent.
Final Report
ED-OIG/A09F0012                                                                                 Page 8 of 15



                    OBJECTIVES, SCOPE, AND METHODOLOGY 



The objectives of the audit were to determine whether (1) SUSD contracts with LEI for
providing SES contained the elements specified in the ESEA § 1116(e)(3) and applicable Federal
regulations; (2) LEI performed the services for which it received payment under the contracts
and the services were provided in a manner consistent with the contract terms and Federal
requirements; and (3) LEI collects and maintains the data that will be used by CDE to evaluate
the quality and effectiveness of the services offered by the provider. Our audit covered school
years 2003-2004 and 2004-2005.3

To achieve our objectives, we gained an understanding of the ESEA provisions, Federal
regulations, U.S. Department of Education guidance, and California regulations covering SES.
We also interviewed officials and staff at LEI and SUSD. We obtained SUSD contracts with
LEI for school years 2003-2004 and 2004-2005 and compared the contracts’ terms to the
elements specified in ESEA § 1116(e)(3). Our review of SUSD internal controls was limited to
the procedures used to contract for SES, process provider invoices, and prioritize students for
SES.

To assess whether LEI performed the services for which it received payment, we gained an
understanding of LEI’s internal controls for collecting student attendance data used to prepare
invoices. LEI’s accounting records showed four invoices, totaling $52,790, were submitted for
school year 2003-2004 and nine invoices, totaling $145,255, were submitted for school year
2004-2005 (as of May 2005). We judgmentally selected the invoices with the largest dollar
amount charged from each of the four schools where LEI provided services to SUSD (the
invoices totaled $8,578, $20,016, $27,846, and $22,874). For each invoice, we reviewed student
attendance data, confirmed that students were qualified to receive SES, and recalculated the
invoice amount.

To determine if LEI’s services were provided in a manner consistent with the contract terms and
Federal requirements, we gained an understanding of LEI’s internal controls for documenting
student goals and timetables, informing parents and teachers of student progress, and procedures
for maintaining student records. We reviewed the provider application packages (2003, 2004,
and 2005) that LEI submitted to CDE and available documentation at LEI related to the
development of individual student goals, timetables for improvement, reporting of student
progress, aligning of services to State content standards, and assessing the quality and
effectiveness of services. We also reviewed information contained in LEI’s student files for
25 students randomly selected from students that received services from LEI in school years
2003-2004 and 2004-2005.




3
 LEI provided one additional SES session for SUSD at the end of school year 2004-2005, which we did not review
due to the timing of our audit fieldwork.
Final Report
ED-OIG/A09F0012                                                                                      Page 9 of 15


         SES Site and Session                    School Year        Students Served         Students Sampled
Van Buren Elementary School                       2004-2005               67                        8
Madison Elementary School                         2004-2005               93                       10
Grunsky Elementary School-Session 1               2003-2004               23                        2
Grunsky Elementary School- Session 2              2003-2004               22                        3
Fillmore Elementary School                        2003-2004               16                        2
                                                                         221                       25

Our review for consistency with contract terms and Federal requirements was limited to the
following areas:
    • 	 Developed a statement of specific achievement goals for each student, how the student’s
        progress will be measured, and a timetable for improving achievement.
    • 	 Regularly informed student’s parents and teacher(s) of the student’s progress.
    • 	 Supplied services that were in addition to the instruction provided during the school day.
    • 	 Used instructional materials that were aligned with State student academic achievement
        standards.4

While LEI maintained copies of student progress reports, we were unable to determine whether
LEI complied with the contract terms for informing parents and teachers of students’ progress
because LEI did not document communication with either.

To confirm that LEI maintained the data that can be used by CDE to evaluate the quality and
effectiveness of services provided by LEI, we reviewed assessment reports (Excel worksheets)
provided to SUSD for school years 2003-2004 and 2004-2005 and gained an understanding of
LEI’s procedures for the collection, maintenance, and reporting of student assessment data. To
assess the reliability of the effectiveness data contained in LEI assessment reports submitted to
SUSD, we reviewed the same random sample of 25 student files used to test for compliance with
contract terms and Federal requirements. For the 25 students, we compared the original test
scores recorded in the student files to the results presented on assessment reports given to SUSD.
We then compared the average effectiveness results highlighted in the SUSD assessment reports
to effectiveness results LEI reported in the 2005 SES provider application to CDE. While we
noted inaccuracies and omissions on reports prepared from the data, we concluded the data
contained in student files was sufficiently reliable to be used in meeting our audit objectives.

We performed our fieldwork at LEI’s corporate office in Concord, California and the
administrative offices of SUSD in Stockton, California. An exit conference was held with CDE
officials on August 10, 2005. An exit conference was held with officials from LEI and SUSD on
August 15, 2005. We performed our audit in accordance with generally accepted government
auditing standards appropriate to the scope of the review described above.



4
  ESEA § 1116(e)(5(B) states that a provider must ensure that instruction provided and content used by the provider
are consistent with the instruction provided and content used by the LEA and State, and are aligned with State
student academic achievement standards. Our review was limited to confirming that LEI’s instructional materials
were aligned with State standards.
Final Report
ED-OIG/A09F0012                                                                     Page 10 of 15



                            ADMINISTRATIVE MATTERS 



Statements that managerial practices need improvements, as well as other conclusions and
recommendations in this report, represent the opinions of the Office of Inspector General.
Determinations of corrective actions to be taken will be made by the appropriate Department
officials.

If you have any additional comments or information that you believe may have a bearing on the
resolution of this audit, you should send them directly to the following Education Department
officials who will consider them before taking final Departmental action on this audit.

                              Henry Johnson
                              Assistant Secretary
                              Office of Elementary and Secondary Education
                              U.S. Department of Education
                              FB6, Room 3W315
                              400 Maryland Avenue, SW
                              Washington, DC 20202

                              Nina Rees
                              Assistant Deputy Secretary
                              Office of Innovation and Improvement
                              U.S. Department of Education
                              FB6, Room 4W317
                              400 Maryland Avenue, SW
                              Washington, DC 20202

It is the policy of the U.S. Department of Education to expedite the resolution of audits by
initiating timely action on the findings and recommendations contained therein. Therefore,
receipt of your comments within 30 days would be greatly appreciated.

In accordance with the Freedom of Information Act (5 U.S.C. § 552), reports issued by the
Office of Inspector General are available to members of the press and general public to the extent
information contained therein is not subject to exemptions under the Act.


                                             Sincerely,

                                             /s/

                                             Gloria Pilotti
                                             Regional Inspector General for Audit
Final Report
ED-OIG/A09F0012                                        Page 11 of 15

                                                     ATTACHMENT




                  CDE Comments on the Draft Report
                                     October 24, 2005




Gloria Pilotti, Regional Inspector General for Audit
United States Department of Education
Office of Inspector General
501 I Street, Suite 9-200
Sacramento, CA 95814

Dear Ms. Pilotti:

This is the California Department of Education’s (CDE) response to the United States
Department of Education (ED) Office of Inspector General’s (OIG) draft audit report
entitled, “Learning Excitement Incorporated and Stockton Unified School District’s
Compliance With Supplemental Educational Services Provisions.” This response
incorporates information from the CDE, Stockton Unified School District (SUSD), and
Learning Excitement Incorporated (LEI). State Superintendent of Public Instruction Jack
O’Connell has asked me to respond on his behalf.

Finding No. 1 – SUSD Did Not Complete Individual Student Plans for All Students
Who Received Supplement Educational Services

Recommendation 1.1:

Require CDE to ensure that SUSD explicitly states in its supplement educational
services (SES) contracts how the District will develop for each student a statement of
specific achievement goals, a description describing how progress will be measured, and
a timetable for improving achievement in consultation with the provider and parents as
required by the Elementary and Secondary Education Act of 1965 (ESEA) § 1116(e)(3),
and that SUSD monitors to ensure that such plans are completed for each student.

       Corrective Action Plan:

       The CDE will provide regional technical assistance via workshops and written
       communications to LEAs on the requirements for implementing and providing
       SES (including elements to be included in the agreement, and how the LEA will
       develop a statement of specific achievement goals for each SES student).
       Furthermore, the CDE plans to review the revised SES agreement template
       before it is approved by the local school board.
Gloria Pilotti, Regional Inspector General for Audit
October 24, 2005
Page 2



       For each student, LEI developed specific achievement goals and timelines based
       upon initial assessment tests, and documented the individualized goals in the
       daily lesson plans and curriculum to meet each student’s needs. However, LEI
       agrees that explicit documentation of individual student plans is needed;
       therefore, LEI changed its SES policies and procedures for the academic year
       2005-06. The new procedures require all students receiving SES from LEI to have
       individualized student learning plans. In September 2005, LEI included this
       procedure in its Program Manager Guide, and discussed it in recent training with
       all LEI program managers.

       SUSD designed an Individual Student Learning Plan (ISLP) on multi-copy NCR
       paper that will be distributed to SES providers when parents submit an application
       for service. The SES services will not begin until the ISLP is completed and
       distributed to all parties.

Recommendation 1.2:

Require CDE to ensure that LEI maintains student records in accordance with the terms
of its contracts with local education agencies (LEA) and that student records are stored
in secured areas.


       Corrective Action Plan:

       The CDE will provide regional technical assistance via workshops and written
       communications to securing student records in accordance with contract
       provisions.

       In the fall 2005, LEI implemented additional internal control procedures for
       assuring the proper handling, storage, and retention of all SES documents. LEI
       included this in its Program Manager Guide in September 2005, and provided
       training to all LEI program managers.
Gloria Pilotti, Regional Inspector General for Audit
October 24, 2005
Page 3


Finding No. 2 – LEI Reported Effectiveness Data to SUSD and CDE That Was
Incomplete and Inaccurate

Recommendation 2.1:

Require CDE to ensure that LEI implements internal control procedures that provide a
reasonable level of assurance that complete and accurate effectiveness data is reported
to LEAs and in SES provider applications and future annual reports submitted to CDE.

       General Comments by LEI:

       LEI acknowledges that human error led to a small amount of misreported or
       unreported data resulting in some inaccuracy and incompleteness to its report to
       SUSD and CDE. However, LEI asserts its reports is reliable. Reliability of data and
       reports means that they are dependable, and that different clinical experiments or
       statistical trials would yield the same or compatible results. Statewide, over
       thousands of students, LEI’s grade level growth averages between 1 and 2.3 full
       grades for 28 hours of tutoring (depending upon the performance measure used).
       The measures received from students in LEI’s Stockton programs yielded results
       that were compatible with the statewide results.

       Additionally, LEI stated the materiality of the finding should have been assessed
       before concluding the reports were not reliable. Findings that have a material
       affect on reported information are findings that when disclosed to the user of the
       information would influence their opinion of the information. In this case, including
       all of the missing data, and correcting all of the misreported data, changes LEI’s
       overall performance in Stockton by only 2.3 percent. Such a minute change in
       overall performance is not material, and should not affect SUSD’s or CDE’s
       assessment of LEI’s program.

       Corrective Action Plan:

       The CDE plans to offer SES provider training sessions regarding the state SES
       reporting requirements and reporting form with 12 data elements about provider
       effectiveness. This report will be due yearly on October 1, beginning in 2006.
       Additionally, CDE will advise LEI about the state SES regulation requirements for
       accurate and full reporting of SES effectiveness. Non-compliance with these
       requirements could be cause for termination as a state-approved SES provider.
       Furthermore, CDE will require LEI to submit a detailed description of its internal
       control procedures to ensure full and accurate reporting of effectiveness data.
Gloria Pilotti, Regional Inspector General for Audit
October 24, 2005
Page 4


        Furthermore, to ensure accuracy, LEI revised its procedure to include comparing
        100 percent of the data entered by field personnel in an Excel spreadsheet to the
        supporting documents once they come into the corporate office.

Other Matters:

1. SUSD did not offer SES to all eligible students for school years 2003-04 and 2004-
   05.

   Corrective Action Plan:

   SUSD identified all low-income students using the free or reduced lunch under the
   National School Lunch Act, and included all students who attend a Provision 2,
   Program Improvement Year 2 or more school. All eligible students will be ranked
   from greatest to lowest academic need.

2. SUSD did not share its individual student academic achievement data with SES
   providers in school years 2003-04 and 2004-05.

   Corrective Action Plan:

   The SUSD’s ISLP that will be distributed to providers includes the statement,
   “Parent AGREES to release academic and testing information, regarding their
   student, to the parent selected agency/provider.”          (Parent’s Initials). Upon
   parental consent, the California Standards Test scores will be released to the
   provider. Additionally, SUSD ensures that attendance records are verified by
   observations during tutoring sessions, parental initials, and teacher’s and
   provider’s signature on each student’s attendance records.

If you have any questions regarding CDE’s response, please contact Kim Sakata, Audit
Response Coordinator, Audits and Investigations Division, at (916) 323-2560 or by
e-mail at .

Sincerely,


/s/
GAVIN PAYNE
Chief Deputy Superintendent of Public Instruction

GP:ks