oversight

California's Inclusion of Migrant and Limited English Proficient Students in the Statewide Assessment and Accountability System.

Published by the Department of Education, Office of Inspector General on 2005-10-25.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

               California’s Inclusion of Migrant and
             Limited English Proficient Students in the
          Statewide Assessment and Accountability System




                                 FINAL AUDIT REPORT




                                    ED-OIG/A09F0003
                                      October 2005


Our mission is to promote the                         U.S Department of Education
efficiency, effectiveness, and                        Office of Inspector General
integrity of the Department's                         Sacramento, California
programs and operations.
                         NOTICE

Statements that managerial practices need improvements, as well as
other conclusions and recommendations in this report, represent the
opinions of the Office of Inspector General. Determinations of
corrective action to be taken will be made by the appropriate
Department of Education officials.

In accordance with Freedom of Information Act (5 U.S.C. § 552),
reports issued by the Office of Inspector General are available to
members of the press and general public to the extent information
contained therein is not subject to exemptions in the Act.
                         UNITED STATES DEPARTMENT OF EDUCATION
                              OFFICE OF INSPECTOR GENERAL
                                               501 I STREET, SUITE 9-200
                                            SACRAMENTO, CALIFORNIA 95814

                                       PHONE (916) 930-2399 • FAX (916) 930-2390


                                                        October 25, 2005


Jack T. O’Connell
State Superintendent of Public Instruction
California Department of Education
1430 N Street
Sacramento, California 95814

Dear Superintendent O’Connell:

Enclosed is our final audit report, Control Number ED-OIG/A09F0003, entitled California’s
Inclusion of Migrant and Limited English Proficient Students in the Statewide Assessment and
Accountability System. This report incorporates the comments you provided in response to the
draft report. If you have any additional comments or information that you believe may have a
bearing on the resolution of this audit, you should send them directly to the following Education
Department official, who will consider them before taking final Departmental action on this
audit:

                                         Henry Johnson, Assistant Secretary
                                         Office of Elementary and Secondary Education
                                         U.S. Department of Education
                                         400 Maryland Avenue, SW
                                         Washington, DC 20202

It is the policy of the U. S. Department of Education to expedite the resolution of audits by
initiating timely action on the findings and recommendations contained therein. Therefore,
receipt of your comments within 30 days would be appreciated.

In accordance with the Freedom of Information Act (5 U.S.C. § 552), reports issued by the
Office of Inspector General are available to members of the press and general public to the extent
information contained therein is not subject to exemptions in the Act.

                                                               Sincerely,


                                                                  /s/
                                                               Gloria Pilotti
                                                               Regional Inspector General for Audit


Enclosure

            Our mission is to promote the efficiency, effectiveness, and integrity of the Department’s programs and operations.
Final Report
ED-OIG/A09F0003



                                              TABLE OF CONTENTS


                                                                                                                               Page

EXECUTIVE SUMMARY ...........................................................................................................1

BACKGROUND ............................................................................................................................3

AUDIT RESULTS .........................................................................................................................6

          FINDING NO. 1 – CDE Needs to Take Additional Steps to Ensure
                          LEP   Students   Are    Provided        Reasonable
                          Accommodations on Assessments..................................................6

          FINDING NO. 2 – CDE Needs to take Additional Steps to Ensure
                          Eligible Students Have Access to Public School
                          Choice in Accordance with Applicable Laws and
                          Regulations ....................................................................................10

          FINDING NO. 3 – CDE Could Enhance Procedures to Ensure All
                          Migrant and LEP Students Are Being Assessed ........................15

          FINDING NO. 4 – CDE Should Disaggregate CAHSEE Summary
                          Reports by Students’ Migrant Status..........................................18

          FINDING NO. 5 – LEAs Could Enhance the Use of Assessment
                          Results to Improve Migrant Students’ Academic
                          Achievement ..................................................................................19

OBJECTIVES, SCOPE, AND METHODOLOGY ..................................................................21

ENCLOSURE – CDE Comments on the Draft Report............................................................23
Final Report
ED-OIG/A09F0003                                                                      Page 1 of 30



                               EXECUTIVE SUMMARY


The purpose of the audit was to review California’s inclusion of migrant and limited English
proficient (LEP) students in the statewide assessment and accountability system. Our specific
audit objectives were to determine whether the California Department of Education (CDE) and
selected local educational agencies (LEAs) within the State had sufficient controls to ensure
(1) migrant and LEP students were being assessed; (2) assessment results were disaggregated by
students’ migrant and LEP status; (3) assessment reports enabled educators to understand and
use the reported information to address the specific academic needs of migrant and LEP students;
and (4) migrant and LEP students, who were enrolled in Title I schools identified for
improvement, gained the same access to public school choice and supplemental educational
services (SES) as other eligible students.

We found that CDE and the selected LEAs had various internal controls in place to ensure that
migrant and LEP students were included in the statewide assessment and accountability system.
Controls were in place to ensure migrant and LEP students were assessed, their status was
properly coded for assessment purposes, disaggregated assessment results were available,
educators were provided assessment data and guidance on using the results to address the
academic needs of individual migrant and LEP students, and migrant and LEP students gained
the same access to choice and SES as other eligible students. We concluded that the controls
were generally effective, but found that—

   •   While assessment procedures addressed the use of LEP accommodations, the three
       selected LEAs did not monitor test administrators to ensure appropriate use of LEP
       accommodations, test administration documents lacked clarity, and CDE did not report
       data on LEP accommodations.

   •   CDE did not provide assessment and accountability data to the three selected LEAs in a
       timely manner, and the LEAs did not provide all the required information when
       informing parents about public school choice.

   •   Even though CDE and the LEAs had various controls in place to ensure migrant and LEP
       students were assessed, CDE did not include subgroup enrollment data in assessment
       reports and did not use an independent source to check the reasonableness of test
       enrollment data for migrant students.

   •   While disaggregated assessment results by students’ migrant and LEP status were made
       available, CDE did not produce summary reports that disaggregated the high school exit
       exam results for the migrant subgroup as it did for the LEP subgroup.

   •   Although the schools we visited used a variety of analytical tools and reports to examine
       an array of assessment and student demographic data to improve student academic
       achievement, particularly for LEP students, the schools did not use the data effectively
       for migrants as a subgroup.
Final Report
ED-OIG/A09F0003                                                                       Page 2 of 30

We recommend that the Assistant Secretary for Elementary and Secondary Education instruct
CDE to take additional steps to ensure LEP students are provided reasonable accommodations on
assessments. We further recommend that the Assistant Secretary, in collaboration with the
Assistant Deputy Secretary for Innovation and Improvement, require CDE to take additional
steps to ensure eligible students have access to public school choice in accordance with
applicable laws and regulations. We also recommend that the Assistant Secretary for Elementary
and Secondary Education encourage CDE to consider enhancing procedures to ensure migrant
and LEP students are being assessed by reporting subgroup enrollment data and using an
independent source to check test enrollment data for migrant students, require CDE to
disaggregate the high school exit exam results for the migrant subgroup, and encourage CDE to
provide additional guidance to LEAs to enhance the use of assessment results to improve migrant
students’ academic achievement.

CDE did not explicitly express concurrence with our findings in its comments to the draft report,
but it did describe the corrective actions taken or planned to address our recommendations. The
corrective actions did not fully address two of our recommendations. CDE’s comments on the
draft report are summarized at the end of each finding and included in their entirety as an
ENCLOSURE.
Final Report
ED-OIG/A09F0003                                                                      Page 3 of 30



                                      BACKGROUND


Title I, Part A, of the Elementary and Secondary Education Act (ESEA), as amended by the
No Child Left Behind Act of 2001 (NCLB), provides funds to states to establish or improve
programs that help disadvantaged children and youth and to ensure that all students have the
opportunity to reach proficiency on challenging state academic standards and assessments. The
NCLB strengthens Title I requirements for state assessments, accountability systems, and
support for school improvement. While collectively referred to as the “accountability system,”
that system is comprised of statewide assessment and accountability systems.

       The statewide assessment system must include annual student academic assessments that
       are aligned with state academic standards and measure students’ proficiency in at least
       reading-language arts and mathematics. Additionally, the assessments must provide for
       the participation of all students.

       The statewide accountability system must ensure that all LEAs and public elementary and
       secondary schools make adequate yearly progress (AYP), based primarily on their
       performance on the academic assessments.

LEAs must identify for school improvement any Title I elementary or secondary school that fails
to make AYP for two consecutive years, and offer all students in those schools the option to
transfer to another public school that is not in improvement (public school choice). If a Title I
school fails to make AYP after a year in improvement status, the LEA must continue to offer
public school choice to all enrolled students, as well as make SES available to low-income
students. LEAs must continue to offer public school choice and SES until the school exits
improvement status by making AYP for two consecutive years.

For school year 2003-2004, California’s statewide assessment system was comprised of the
Standardized Testing and Reporting (STAR) program and the California High School Exit
Examination (CAHSEE). All students in grades 2-11 were to be assessed through the STAR
program, which had the following four components—

       California Standards Tests (STAR-CSTs) – Administered to all students in grades 2-11 to
       assess students’ achievement of the State's academic content standards.
       California Achievement Tests, 6th Edition Survey (STAR-CAT-6) – Administered to all
       students in grades 2-11 to compare each student’s score to the scores of a national sample
       of students tested in the same grade at the same time of the school year.
       Spanish Assessment of Basic Education, 2nd Edition (STAR-SABE/2) – Administered to
       Spanish-speaking English learners in grades 2-11, who have been in California public
       schools for less than 12 months, to test basic skills in Spanish reading, language, and
       mathematics.
       California Alternate Performance Assessment (STAR-CAPA) – Administered to students
       in grades 2-11 with significant cognitive disabilities who are unable to take the CSTs and
       CAT-6 tests even with accommodations or modifications.
Final Report
ED-OIG/A09F0003                                                                           Page 4 of 30

Like the STAR-CSTs, the CAHSEE was aligned with State academic content standards, but is
only administered to high schoolers beginning in the 10th grade. Passing the exam is a
graduation requirement for the class of 2006 and beyond.

To make AYP, the State requires schools and LEAs to annually meet or exceed criteria in four
areas—test participation rate, percent proficient on annual measurable objectives for the
statewide assessments used for AYP, graduation rate, and Academic Performance Index (an
additional indicator to measure academic performance and growth). To determine the
participation rate and percent proficient for AYP purposes, the State uses the following
assessments in English-language arts and mathematics—STAR-CSTs for grades 2 through 8 and
STAR-CAPA for grades 2 through 8 and 10, and CAHSEE for grade 10. CDE calculates AYP,
identifies Program Improvement (PI)1 schools and their improvement status, and makes the
results available to LEAs on the CDE website.

Statewide, about 6.3 million students were enrolled in 9,206 public elementary and secondary
schools within 1,040 LEAs in school year 2003-2004. About 0.2 million students (or 4 percent
of total enrollment) were migrant and 1.6 million students (or 25 percent) were LEP. Based on
2004 AYP, 1,601 of the 5,714 Title I schools (or 28 percent) were identified as newly or
continuing in PI.2

For school year 2003-2004, CDE allocated about $1.2 billion in Title I funds to 958 LEAs. The
allocation included about $46 million to the three LEAs we reviewed. As shown below, a
substantial proportion of student enrollment in the three LEAs was comprised of migrant and
LEP students.




1
    California refers to schools in improvement as Program Improvement (PI) schools.
2
 CDE Sources: October 2003 California Basic Educational Data System (total enrollment); California’s
Consolidated State Performance Report, Attachment B, dated April 15, 2005 (migrant enrollment);
March 2004 Language Census (LEP enrollment); California’s Consolidated State Performance Report:
Part I . . . For Reporting on School Year 2003-2004, submitted to the U.S. Department of Education on
January 31, 2005 (school and LEA counts); Program Improvement Database, updated February 2005
based on final 2004 AYP (Title I and PI schools).
    Final Report
    ED-OIG/A09F0003                                                                         Page 5 of 30

                Enrollment (Percent of Total) and School Data, School Year 2003-2004
                                         Number of Students                 Number of Schools
LEA                                    a            bc          ac
                                 Total      Migrant        LEP         Totald     In Improvementa
Fresno Unified School District    81,408        8,037         25,319
                                                                            102               51
(Fresno USD)                      (100%)        (10%)         (31%)

Lodi Unified School District      29,178        1,688         8,347
                                                                             47               9
(Lodi USD)                        (100%)        (6%)          (29%)

Pajaro Valley Unified School      19,522        8,239         8,908
                                                                             29               17
District (Pajaro Valley USD)      (100%)        (42%)         (46%)

a
  See Footnote 2 above for source.
b
  Source: March 2004 Migrant Student Information System
c
  A student may be both migrant and LEP and, thus, included in both the migrant and LEP counts.
d
  Source: California Basic Educational Data System, October 2003
Final Report
ED-OIG/A09F0003                                                                      Page 6 of 30



                                    AUDIT RESULTS


CDE and the selected LEAs had various controls in place to ensure that migrant and LEP
students were included in the statewide assessment and accountability system. To ensure
migrant and LEP students were assessed, controls included procedures that—

       Provided LEAs the option to use a pre-identification process for coding student
       demographic information on student answer documents;
       Required a student answer document for each student enrolled on the first day of testing;
       Checked for the quality of student demographic data upon receipt of assessment results at
       CDE from the contracted test publisher; and
       Required LEAs to ensure that student demographic data were accurate and complete.

To ensure that students’ migrant and LEP status was properly coded for assessment purposes, the
three LEAs had procedures in place for respective program personnel to assure that the status
information was correctly entered into the LEA student information system, and to use the
information system as the basis for the students’ assessment demographic data. We found that
CDE made available to LEAs and on the Internet the STAR-CST and CAHSEE assessment
results disaggregated by students’ migrant and LEP status, and provided guidance for LEAs and
schools on reviewing the data and developing strategies for addressing identified instructional
needs of individual students and the school as a whole. We also found that CDE’s and the
selected LEAs’ procedures for implementing public school choice and SES provided migrant and
LEP students with the same access to choice and SES as other eligible students.

While we found that the identified controls were generally effective, we concluded that:
(1) CDE needs to take additional steps to ensure LEP students are provided reasonable
accommodations on assessments; (2) CDE needs to take additional steps to ensure eligible
students have access to public school choice in accordance with applicable laws and regulations;
(3) CDE could enhance procedures to ensure all migrant and LEP students are being assessed;
(4) CDE should disaggregate CAHSEE summary reports by students’ migrant status; and
(5) LEAs could enhance the use of assessment results to improve migrant students’ academic
achievement.


FINDING NO. 1 – CDE Needs to Take Additional Steps to Ensure LEP Students
                Are Provided Reasonable Accommodations on Assessments

Section 1111(b)(3)(C)(ix) of the ESEA requires States to include LEP students in its academic
assessment system and to provide reasonable accommodations on assessments, including native
language assessments to the extent practicable. The U.S. Department of Education’s
(Department’s) implementing regulations at 34 C.F.R. § 200.6(b)(1) state—
Final Report
ED-OIG/A09F0003                                                                           Page 7 of 30

          [T]he State must assess limited English proficient students in a valid and reliable
          manner that includes—
             (A) Reasonable accommodations; and
             (B) To the extent practicable, assessments in the language and form most
          likely to yield accurate and reliable information on what those students know and
          can do to determine the students’ mastery of skills in subjects other than English
          until the students have achieved English language proficiency.

The Department’s Standards and Assessments Non-Regulatory Guidance, dated March 10, 2003,
clarifies that accommodations for LEP students may include native language assessments, extra
time, small group administration, flexible scheduling, simplified instructions, allowing the use of
dictionaries, and providing instructions in the native language.

CDE administers the STAR-CST and CAHSEE assessments in English only and has not offered
native-language assessments that are aligned with State academic standards. According to the
CDE Student Assessment Division Director, State policy has been to test all students in English
and the State Legislature only recently permitted CDE to begin developing a native language
assessment. CDE has been administering the STAR-SABE/2 to Spanish-speaking LEP students
who have been enrolled within the State for less than a year, but that assessment is not aligned
with State academic standards and is only used to test basic skills in Spanish reading, language,
and mathematics. Also, while the STAR-SABE/2 is a component of the STAR assessment
program, that test is not used for AYP purposes.

Consistent with the Department’s guidance, CDE assessment procedures allow LEP students
four types of accommodations—flexible setting, flexible schedule, translated directions, and
glossaries. Using these accommodations, LEP students may be tested in a separate room, be
given a break before going onto the next test section, have the test administrator translate any
directions that are read aloud to the students in their primary language, and use English-primary
language glossaries or word lists regularly used in the classroom for mathematics or science. If
an LEP student used any of the above accommodations, test administrators were instructed to
mark “EL used accommodations” in the Accommodations and Modifications section of the
student answer document after the test was completed.3 LEP students are allowed to use the
accommodations during testing only if the accommodation is regularly used in the classroom.
CDE provided STAR and CAHSEE administration guidance for LEAs and test administrators
that addressed the LEP accommodation procedures. LEA test coordinators received training
from CDE and, in turn, were to distribute the guidance and train school test-site coordinators
who then trained the school’s test administrators.

While assessment procedures addressed the use of LEP accommodations, we found that CDE
and the three selected LEAs did not have sufficient controls in place to ensure adherence to the
procedures.




3
    In California, EL is an English learner or LEP student.
Final Report
ED-OIG/A09F0003                                                                                Page 8 of 30

LEAs Did Not Monitor for Appropriate
Use of LEP Accommodations

Test administrators at the three LEAs received written instructions and training on appropriate
LEP accommodations and the procedures for marking the student answer document. However,
the LEAs we visited did not have procedures to determine whether test administrators had
provided LEP students with appropriate accommodations or marked the student answer
document, when applicable. There was no documentation of what accommodations, if any,
individual LEP students used on a daily basis. Additionally, LEA test coordinators and school
test coordinators did not review the completed student answer documents to assure that test
administrators had offered appropriate accommodations or marked the answer document, when
applicable.

Assessment Directions and Student
Answer Document Lacked Clarity

The written directions for test administration and the student answer document for the 2003-2004
assessments did not clearly distinguish accommodations applicable to LEP students versus
students eligible under the Individuals with Disabilities Education Act (students with disabilities)
or covered under § 504 of the Rehabilitation Act of 1973 (section 504 students).4 As a result,
test administrators may have misunderstood the directions, or overlooked and failed to mark the
student answer document when a student used an LEP accommodation for testing.

CDE’s Directions for Administration addressed procedures for marking the student answer
documents when an LEP student used a test accommodation. However, accommodations
specific to LEP students were addressed in the midst of several pages of directions devoted to the
other special versions, accommodations, modifications, and testing variations. Similarly, the
student answer document, onto which the test administrator was to indicate whether a student
used “special versions, accommodations, or modifications,” embedded the LEP accommodations
among the more numerous accommodations applicable only to students with disabilities, section
504 students, or both. Among the multiple accommodations and modifications listed on the
answer document, the “EL used accommodations” option was listed as the third of 8 types of
accommodation on the STAR answer document, and the third of 11 accommodations listed on
the CAHSEE answer document.

The Student Assessment Division Director stated that CDE adopted a new variation matrix for
the 2005-2006 assessments. The matrix identifies the types of accommodations and
modifications that may be used by LEP and other students for each State assessment, such as
STAR and CAHSEE. To reflect the new matrix, test administration instructions will be revised
and the student answer document will more comprehensively collect data on variations,
accommodations, and modifications for both LEP students and students with special needs. The
documents will more clearly distinguish the LEP accommodations from the ones for students
with disabilities and section 504 students.




4
    Section 504 students are persons with a qualified handicap under the Rehabilitation Act of 1973.
Final Report
ED-OIG/A09F0003                                                                      Page 9 of 30

CDE Did Not Report Data on LEP Accommodations

When reporting and making STAR and CAHSEE assessment data available to LEAs and
schools, CDE did not provide data on those students who used an LEP accommodation during
testing. CDE had the data available since it collected student demographic data, including the
use of LEP accommodations, from the student answer documents. Such data would provide
visibility regarding the use of LEP accommodations and a basis for LEAs to monitor for
appropriate use and marking of the student answer document.

Based on CDE assessment data, we found that only two percent of LEP students used
accommodations on the 2003-2004 STAR and CAHSEE assessments statewide. For the three
LEAs reviewed, the percentages ranged from less than 1 percent to about 5 percent of the LEP
students tested LEA-wide. According to the data, LEP students did not use accommodations for
the STAR assessment in six of the nine schools visited, and in two of the three high schools for
the CAHSEE.

When we first inquired about the use of LEP accommodations, representatives from five schools
stated that LEP students were provided testing accommodations, such as translated directions,
glossaries, and placement with their English language development teacher. When we later
brought the CDE data to their attention, the responding LEA and school administrators did not
know whether the data accurately reflected the number of students who should have or did use
LEP accommodations. The test coordinator for one LEA stated that LEP accommodations may
have been offered, but the answer documents may not have been marked when LEP
accommodations were used. Even though 70 percent of students tested in one school were LEP,
the new school principal confirmed that the school did not have any testing accommodations for
the 2003-2004 assessments and stated that queries of individual teachers would be needed to
determine what accommodations were provided on a daily basis and should have been used for
testing. Another school principal did not know whether more than 3 percent of the LEP students,
as indicated by the CDE data, should have used LEP testing accommodations.

The LEP population in California is substantial and LEP students comprise one of the subgroups
that must meet AYP in order for a school to meet AYP. The roughly 1.6 million LEP students
enrolled statewide in school year 2003-2004 comprised about 25 percent of all enrolled students.
While California’s LEP students speak over 57 different primary languages, Spanish was the
primary language for 85 percent. In the three selected LEAs, the proportion of Spanish-speaking
LEP students ranged from 57 percent to 99 percent. Additionally, the LEP subgroup did not
make AYP in all or nearly all the PI schools in the three LEAs.

At the time of our review, the State was in the initial process of developing a Spanish-language
assessment aligned to content standards for reading and writing. Implementation of the native
language assessment is expected in 2007. We encourage CDE to make every effort to complete
and administer native language assessments to the extent practicable, especially in the primary
languages that comprise a significant portion of the LEP population. Without native language
assessments or assurance that appropriate LEP accommodations are used, assessment results may
not accurately reflect LEP students’ mastery of skills in subjects other than English.
Final Report
ED-OIG/A09F0003                                                                           Page 10 of 30

Recommendations

We recommend that the Assistant Secretary for Elementary and Secondary Education instruct
CDE to—

1.1    Monitor LEAs’ implementation of State testing accommodation policies more vigorously
       to ensure test administrators provide appropriate testing accommodations for LEP
       students and properly mark the student answer document, when applicable.

1.2    Ensure that the test administration instructions and student answer documents for the
       2005-2006 and future assessments clearly distinguish the LEP accommodations from the
       ones for students with disabilities and section 504 students.

1.3    Consider including data on LEP students’ use of accommodations in the assessment data
       made available to LEAs to ensure LEP students are provided reasonable
       accommodations.

CDE Comments

In its comments, CDE reiterated information about the corrective action taken regarding testing
accommodations for limited English proficient students. For the 2005-2006 test administration,
CDE has included a separate section on the student answer document for test examiners to mark
testing variations used by LEP students. In the electronic data file provided to each district, CDE
will include all testing variations used by LEP students along with students’ test scores and other
demographic data.


FINDING NO. 2 – CDE Needs to take Additional Steps to Ensure Eligible Students
                Have Access to Public School Choice in Accordance with
                Applicable Laws and Regulations

States must ensure that academic assessment results are available to LEAs before the start of the
new school year so that LEAs can inform parents of public school choice options before the start
of the school year. Section 1116(a)(2) of the ESEA states that—

       [T]he State educational agency shall ensure that the results of State academic
       assessments administered in that school year are available to the local
       educational agency before the beginning of the next school year.

For schools identified for school improvement, the ESEA §1116(b)(1)(E) states—

       [T]he local educational agency shall, not later than the first day of the school
       year following such identification, provide all students enrolled in the school
       with the option to transfer to another public school served by the local
       educational agency . . .
Final Report
ED-OIG/A09F0003                                                                        Page 11 of 30

We found that CDE did not provide assessment and accountability data to the three selected
LEAs in a timely manner, and the LEAs did not provide all the required information when
informing parents about public school choice.5

CDE Assessment and Accountability
Data to LEAs Were Untimely

CDE did not provide STAR-CST assessment data, determine AYP, and identify Title I schools’
PI status before the beginning of the school year for the three LEAs reviewed. CDE made
2003-2004 assessment data available to all LEAs statewide on August 8, 2004; AYP results on
August 31, 2004; and the list of schools’ PI status on October 13, 2004. Additionally, CDE
provided guidance in mid-August 2004 on its methodology for calculating AYP and PI status,
which would have enabled LEAs to perform their own pre-calculations once the assessment data
became available.

We found that schools in the three LEAs began the 2004-2005 school year before the dates that
CDE made the AYP results and PI status list available to the LEAs. Although Pajaro Valley
USD was on a traditional calendar, all its schools began on August 18, 2004. Fresno USD and
Lodi USD had non-traditional calendars with year-round schools, and many or nearly all schools
were in session during the month of July or in early August. When the assessment results
became available, staff at two of the LEAs performed pre-calculations to determine AYP and PI
status and the LEAs acted on the results for the schools continuing in PI. However, LEA
officials stated that the LEAs waited for CDE’s official results before acting on schools newly
identified as PI. According to CDE officials, the LEAs waited because CDE requires LEAs that
begin to provide public school choice for newly identified PI schools to continue providing
choice-related transportation for the rest of the school year, even if CDE later determines that the
school was not identified as a PI school.

CDE officials from the three divisions that administer the State’s assessment and accountability
system stated that CDE procedures for administering the STAR tests, processing and scoring the
assessments, checking for data quality, calculating AYP, and determining PI status, did not
enable the State to provide the assessment, AYP, and PI results any earlier. Factors that
constrain CDE’s ability to adjust their assessment and reporting schedules include—

       State-Mandated Testing Schedule. State regulations specify that the STAR test must be
       administered to each student during a testing window of 21 instructional days that
       includes 10 instructional days before and after completion of 85 percent of the school’s or
       track’s (for year-round schools) instructional calendar. According to the CDE officials,
       students are tested between February and August of the school year depending on the
       instructional calendar. The test publisher needs a minimum volume of assessments and
       cannot start scoring the assessments until about May. A quarter of the test items are
       refreshed each year, which essentially results in a new assessment for the test publisher to
       score.


5
  As noted in the introductory paragraph of the AUDIT RESULTS section, we found that CDE and the
selected LEAs’ implementation of public school choice and SES provided migrant and LEP students with
the same access to choice and SES as other eligible students. We noted no problems with SES.
Final Report
ED-OIG/A09F0003                                                                              Page 12 of 30

       Extensive Data Review Process. Once testing has been completed, time is needed to
       review and report the data. According to CDE officials, diligent effort is made to assure
       that the assessment and demographic data are correct before CDE calculates AYP and PI
       status because of the consequences for PI schools. CDE’s Policy and Evaluation
       Division performs edit and reasonableness checks related to assessment demographic
       data. However, CDE has placed final responsibility on the LEA and school to ensure the
       accuracy of the data. LEAs have the opportunity to review the demographic data for the
       STAR, CAHSEE, and CAPA assessments and make corrections in correctable data fields
       through the test publisher.

       For the 2003-2004 assessment cycle, the timeline for posting and updating results for the
       STAR assessments, AYP, and PI status took about 6 months, as the table below
       illustrates.

                       Timeline for Reporting Results for 2003-2004 STAR Assessments,
                                  2004 AYP, and 2004-2005 PI Status Results
        August 8, 2004             First posting of 2003-2004 STAR assessment summary results

        August 31, 2004            First posting of 2004 Accountability Progress Reports, including AYP
        August 31 to
                                   Window for LEAs to submit AYP appeals to CDE
        September 15, 2004
                                   Second posting of STAR assessment results, including those LEAs
        September 15, 2004a
                                   whose data were not available for the first posting
        September 17 to            Window for LEAs to submit edit corrections of their demographic
        November 5, 2004           data to the test publisher
                                   First update to 2004 AYP Report, based on the second posting of
        September 30, 2004
                                   STAR results
        October 13, 2004           First posting of Title I PI Status Report, including appeal status

        January 15, 2005 a         Third posting of STAR assessment results
                                   Revised 2004 AYP Report, including data corrections and appeal
        January 27, 2005
                                   status
        February 14, 2005          Revised Title I PI Status Report
        a
            Approximate dates provided by CDE officials.



To improve on the above timeline, CDE officials stated that, when the process for a new test-
publisher contract begins in July 2005, CDE will ask prospective bidders for ways to make raw
assessment scores available sooner. Additionally, CDE will attempt to post the 2005 AYP
results by August 25, and release the initial PI results about two weeks later without the appeal
status. While the timeline would improve by about a week, these efforts still would not be
timely for those LEAs with schools that start in July or earlier in August. CDE should continue
to review its processes and procedures to better ensure that LEAs receive assessment, AYP, and
PI results in a timely manner.
Final Report
ED-OIG/A09F0003                                                                         Page 13 of 30

Even though CDE was technically out of compliance with the statutory requirement to ensure
that the assessment results were available to LEAs before the beginning of the school year, the
affected LEAs were still required to implement public school choice requirements but were not
bound to do so before the start of school. In the event that an LEA does not receive student
achievement results or the PI list from the State in time to offer choice before the beginning of
the school year, the Department’s draft Public School Choice Non-Regulatory Guidance, dated
February 6, 2004, states that the LEA should make choices available as quickly as possible
before the school year gets well underway. We found that the three LEAs reviewed adhered to
this guidance.

LEA Public School Choice Letters Did
Not Contain All Required Information

Section 1116(b)(6) of the ESEA requires LEAs to provide notice to parents when a school has
been identified for improvement and addresses the types of information that must be included in
the notice, including an explanation of the parents’ option to transfer their child to another public
school. The implementing regulations at 34 C.F.R. § 200.37(b)(4) further define what
information must be included in the explanation related to public school choice—

       (ii) The explanation of the parents’ option to transfer must include, at a minimum,
       information on the academic achievement of the school or schools to which the
       child may transfer.

The Department’s draft Public School Choice Non-Regulatory Guidance further clarifies that the
notification should—

       1. Inform parents that their child is eligible to attend another public school due to
          the identification of the current school as in need of improvement;
       2. Identify each public school, which may include charter schools, that the parent
          can select;
       3. Include information on the academic achievement of the schools that the
          parent may select.

In all three LEAs visited, students in PI schools could transfer to two or more other public
schools and the parental notification letters about public school choice addressed the school’s PI
status and student eligibility to transfer. However, the letters did not identify at least two public
school options that parents could select, achievement data for those school options, or both.
According to LEA officials, the LEAs were not aware of the information requirements, did not
want to tailor the letter by students’ residence area, or did not want to overload parents with too
much information in the letter. In the last case, the parental notification letter in one LEA
offered parent information meetings at which more detailed information was provided on the
school choice options. Based on meeting sign-in sheets, we found that few parents attended the
meetings and, according to the Title I program director, those parents who attended were more
interested in hearing about supplemental educational services. We also found that CDE guidance
to LEAs on parental notification letters did not address the content requirements, except to
provide an electronic link to the Public School Choice Non-Regulatory Guidance on the
Department’s website. While CDE provided a template for LEAs to use in notifying parents
Final Report
ED-OIG/A09F0003                                                                         Page 14 of 30

when the LEA was identified for improvement, it did not provide a template for when a school
within the LEA was identified as a new or continuing PI school.

Thus, parents in the three LEAs reviewed did not receive all the required information for making
an informed decision about public school choice. Based on LEA data, we determined that less
than one percent of eligible students transferred to another public school at the beginning of
school year 2004-2005. Officials at two LEAs attributed the low transfer rate to parents’ desire
to keep their students in their home schools. However, incomplete notice letters could also have
contributed to parental decisions. To address concerns about the amount of information in the
letters, CDE could advise LEAs to consider sending a brief letter with the required details
appearing as an attachment.

CDE’s compliance monitoring procedures, which were revised for the 2004-2005 school year,
included a step to review the LEA’s notification to parents about the transfer option. However,
the monitoring guide did not include the statutory reference that addresses parental notifications
and did not specifically instruct reviewers to confirm that the letters contained the required
information.

Recommendations

We recommend that the Assistant Secretary for Elementary and Secondary Education, in
collaboration with the Assistant Deputy Secretary for Innovation and Improvement, require CDE
to—

2.1    Continue to refine its assessment and accountability processes and procedures to ensure
       that LEAs will receive assessment, AYP, and PI results before the beginning of the
       school year, including July and August starts, so that parents of eligible students can be
       informed of public school choice options in a timely manner.

2.2    Provide written guidance to LEAs describing the information that must be included in
       parental notification letters about public school choice, including the identification of at
       least two school options, where applicable, and the schools’ academic achievement
       information.

2.3    Enhance its monitoring procedures by incorporating (1) the reference to the applicable
       statutory provision on parental notifications and (2) specific instructions for reviewers to
       confirm that LEA parental notification letters regarding public school choice contain the
       required information.

CDE Comments

Recommendation 2.1. CDE stated that substantial changes in the law and policies governing the
STAR program and CAHSEE would be needed to provide for the early delivery of test results to
LEAs for AYP and PI purposes. The August 31 date shown in the timeline in the table (page 12)
was the date CDE had provided to, and was approved by, the Department for the release of all
elements of the AYP determination. CDE reiterated that LEAs are responsible for identifying
schools for PI immediately after the release of the AYP data. CDE also stated that this year’s list
of PI schools was released to LEAs 30 days earlier than the previous year.
Final Report
ED-OIG/A09F0003                                                                        Page 15 of 30


Recommendation 2.2. CDE stated that detailed instructions and sample parent notification
letters addressing all components required in the ESEA § 1116(b)(6) have been posted to its
website. CDE stated that the sample letters provide contact information for parents interested in
school choice and inform them that interested parents will receive information about the schools
available to transfer and the schools’ academic achievement.

Recommendation 2.3. CDE plans to provide training to inform LEAs about the required
components for the public school choice parental notification letters, and has included a
compliance item in the 2005-06 Categorical Monitoring Program Monitoring Document that
requires reviewers to ensure that the LEA parent notification letter incorporates those
components.

OIG Response

Recommendation 2.2. In addition to the above corrective action, CDE needs to ensure that the
parent notification letters include information on the academic achievement of schools available
for transfer. The regulation at 34 C.F.R. § 200.37(b)(4)(ii) states that information on the
academic achievement of the school or schools to which the child may transfer must be included
in the notice. Providing contact information to obtain the information is not sufficient.


FINDING NO. 3 – CDE Could Enhance Procedures to Ensure All Migrant and
                LEP Students Are Being Assessed

The ESEA § 1111(b)(3)(ix) requires that all students be assessed under the statewide assessment
system. Nearly all California students, including migrant and LEP students, were assessed in
school year 2003-2004, and both CDE and the LEAs reviewed had various controls in place to
ensure that migrant and LEP students were assessed. However, we identified two enhancements
that could better assure that all student subgroups, particularly for migrant students, are being
assessed.

CDE Did Not Include Subgroup
Enrollment Data in Assessment Reports

CDE assessment procedures require that a student answer document be completed for each
student enrolled on the first day of testing, whether or not the student actually took the test. The
student answer document contains student demographic information, including migrant and LEP
status. However, the STAR-CST and CAHSEE assessment data and summary reports that CDE
made available to LEAs and on the Internet did not include subgroup data on the number of
migrant and LEP students enrolled on the first day of testing. As a result, neither CDE nor the
LEAs and schools reviewed could use the data to determine test participation rates for the
subgroups. Because the enrollment data are already collected for each assessment, CDE could
readily summarize and provide the data to enable users to compare the resulting number of
students that should have been tested against the number of students who were tested.

Based on CDE assessment data, we found that the proportion of migrant and LEP students who
were assessed in school year 2003-2004 mirrored the proportion of all students assessed (over 95
Final Report
ED-OIG/A09F0003                                                                                  Page 16 of 30

percent for STAR and about 90 percent for CAHSEE). Making available the subgroup
enrollment data would enable the State, LEAs, and schools to more readily determine the extent
that different subgroups of students, including migrants and LEPs, are being assessed to ensure
their inclusion in the statewide assessment system. Additionally, LEAs would have the test
participation rate data needed to pre-calculate AYP, which could facilitate identifying schools’
improvement status and meeting timing requirements for implementing public school choice.

CDE Did Not Check the Reasonableness of
Test Enrollment Data for Migrant Students

CDE used an independent source to check the reasonableness of test enrollment data for the LEP
subgroup, but did not perform a similar check for the migrant subgroup. CDE requires LEAs to
annually submit a Language Census to collect data about LEP students and services. Because
the Census date annually occurs on March 1 and roughly coincides with STAR and CAHSEE
testing dates, CDE can use the data to perform reasonableness checks of LEP students’ test
enrollment data for assessment purposes. Although CDE’s Migrant Education Program (MEP)
maintained a statewide database on migrant students, CDE Policy and Evaluation Division and
Standards and Assessment Division officials stated that they did not check the reasonableness of
test enrollment data for migrant students, except to compare the data to the prior year. Instead,
CDE relied solely on LEAs to ensure the accuracy of the assessment demographic data.

Based on our review at the three LEAs, we found that the MEP used STAR-CST data to
prioritize migrant education services and maintained databases that LEAs could use as an
independent source to check test enrollment data for migrant students. We found that the LEAs
had controls in place to ensure students’ migrant status was properly coded at the time
assessments were administered. The LEAs had procedures for identifying migrant students and
entering their status in the LEA student information system, which was the basis for coding
student demographic data on student answer documents for test purposes. The procedures in two
of the LEAs included restricting access to the migrant and LEP status fields in the student
information system to ensure that unauthorized personnel did not change the data.6 The three
LEAs also participated in a pre-identification process to reduce the number of student answer
documents that had to be manually coded at the time of testing.7 Additionally, the cognizant
MEP regional offices for the three LEAs maintained migrant student databases that provided an
independent source for checking the reasonableness of students’ migrant status in the LEA
student information system.8 In particular, we found that—


6
    The third LEA did not restrict access to migrant-related data fields in its student information system.
7
  The pre-identification process entails LEA submission of student demographic data electronically to the
test publisher to create and provide bar-coded labels containing the student information to be affixed to
the student answer document. Pre-identification helps to reduce coding errors in the assessment
demographic data. CDE Standards and Assessment Division officials stated that most LEAs do not
participate in pre-identification because of the associated costs, although the LEAs that do participate
cover most of the students statewide.
8
 CDE administers the MEP through 23 regional offices that each serves one or more LEAs. The regions
periodically report program data, including migrant enrollments, for inclusion in the statewide MEP
database.
Final Report
ED-OIG/A09F0003                                                                         Page 17 of 30

       Fresno USD and Pajaro Valley USD frequently reconciled their student information
       system with their respective MEP region’s migrant student database. The reconciliation
       helped the LEA to ensure students’ migrant status were accurate and complete when
       student answer documents were coded for assessment purposes.

       Lodi USD did not have procedures to reconcile its data on migrant student status with its
       MEP region, but has since begun to take steps to reconcile the data. Although we found
       discrepancies between the LEA and MEP data on the number of migrant students, the
       LEA and MEP regional staff worked together to resolve the discrepancies after we
       brought the matter to their attention.

CDE has not provided guidance to LEAs about potential independent sources, such as the MEP
region, for assessing the reliability of student demographic data. Because migrants are not a
subgroup for AYP purposes, little attention has been paid to their inclusion in assessments as a
subgroup. Using an independent source to check the reasonableness of migrant students’
participation on assessments at the State and LEA levels would provide additional assurance that
students’ migrant status were coded correctly and that migrant students were included in
assessments.

Recommendations

We recommend that the Assistant Secretary for Elementary and Secondary Education encourage
CDE to—

3.1    Consider enhancing the ability of the State, LEAs, and schools to ensure that all student
       subgroups, including migrants and LEPs, are being assessed by reporting or making
       available data on the number of students (by subgroup) enrolled on the first day of
       testing.

3.2    Consider enhancing the ability of the State, LEAs, and schools to ensure all migrant
       students are being assessed by (1) taking the steps necessary to ensure the reliability and
       use of the statewide MEP database to check the reasonableness of migrant enrollment
       data for assessment purposes, and (2) instructing LEAs to reconcile student information
       with MEP regions’ data at the time of testing.

CDE Comments

In its comments, CDE stated that Migrant Regional Offices will be required to provide the most
current list of migrant students to the districts for inclusion in the STAR header sheets [as part of
the pre-identification process], and again at the beginning of the STAR testing period. CDE will
also direct Migrant Education Regional Offices to upload the most current enrollment data on
migrant students to the State migrant student database on or about March 1 of each year.
Final Report
ED-OIG/A09F0003                                                                    Page 18 of 30

FINDING NO. 4 – CDE Should Disaggregate CAHSEE Summary Reports by
                Students’ Migrant Status

Section 1111(b)(3)(C)(xiii) of the ESEA requires the State to enable academic assessment results
to be disaggregated by, among other means, students’ migrant status and English proficiency
(i.e., LEP) status. We reviewed the assessment data and summary reports that CDE made
available to LEAs and on the Internet. The assessment data included student demographic
information, which would enable LEAs to disaggregate the assessment results by students’
migrant and LEP status. CDE produced summary reports disaggregating STAR-CST assessment
results for the migrant subgroup and the LEP subgroup, and the CAHSEE results for the LEP
subgroup. CDE did not produce summary reports that disaggregated CAHSEE results for the
migrant subgroup and did not provide LEAs with instructions to produce the report. None of the
three LEAs reviewed disaggregated CAHSEE results for migrant students as a subgroup.
Passing the CAHSEE is a graduation requirement for the high school class of 2006 and beyond.
Disaggregated CAHSEE summary reports would provide visibility regarding migrant students’
performance on this high-stakes assessment.

Recommendation

4.1    We recommend that the Assistant Secretary for Elementary and Secondary Education
       require CDE to make available, or instruct LEAs to produce, summary reports that
       disaggregate CAHSEE results for the migrant subgroup.

CDE Comments

In its comments, CDE stated that it will explore modifying the current CAHSEE Results website
to include summary reports that disaggregate CAHSEE results for the migrant subgroup.

OIG Response

In the draft report, the recommendation only required CDE to “consider” making available, or
instructing LEAs to produce, the CAHSEE results disaggregated for the migrant subgroup. We
modified the language to no longer provide CDE the option of implementing the
recommendation because the State must report CAHSEE results for the migrant student
subgroup to comply with annual State report card requirements specified in the ESEA §
1111(h)(1)(C).
Final Report
ED-OIG/A09F0003                                                                      Page 19 of 30

FINDING NO. 5 – LEAs Could Enhance the Use of Assessment Results to Improve
                Migrant Students’ Academic Achievement

The ESEA § § 1111(b)(3)(C)(xii) and (xv) require that the State’s assessment system produce
individual student interpretive, descriptive, and diagnostic reports, and enable itemized score
analyses to be produced and reported. The Department’s Standards and Assessments
Non-Regulatory Guidance, issued March 10, 2003, specifies the qualities the student reports
should possess and clarifies that the analyses are to help parents, teachers, principals, and
administrators interpret test results and address the specific academic needs of students. The
Guidance states—

       These analyses should provide information to improve student achievement,
       strengthen curriculum alignment, and identify problem areas and areas of strong
       performance for entire schools and demographic subgroups of students.

Additionally, the ESEA § 1111(b)(10) requires each State plan to describe how it will ensure that
LEAs, schools, and teachers use assessment results to improve individual students’ educational
achievement.

Consistent with the Federal statute and Departmental guidance, CDE made STAR-CST and
CAHSEE assessment data available to LEAs and provided guidance on a suggested process for
reviewing the data and developing strategies for addressing identified instructional needs of
individual students and the school as a whole. In the three LEAs reviewed, the MEP region used
the STAR-CST results to prioritize migrant education services for migrant students, and the LEP
program used various assessment results to track LEP students’ progress and monitor the
program. In addition to the programmatic use, we also found that the schools we visited used a
variety of analytical tools and reports to examine an array of assessment and student
demographic data to improve student academic achievement individually and for groups of
students, particularly LEP students. We found that the schools were not using the data
effectively for migrants as a subgroup.

Fresno USD Did Not Make Migrant
Status Data Readily Available

Fresno USD did not make data on students’ migrant status readily available to schools. The
LEA Assessment Information System, which included students’ demographic and assessment
information, did not provide a menu option for students’ migrant status. Unlike the ready access
to assessment data for LEP students, school personnel needed to take additional steps involving a
separate migrant student list and individual student identification numbers in order to generate
assessment results for migrant students individually or as a subgroup. As a result, the schools
visited could not use the available assessment and demographic data as effectively for the
migrant subgroup.

LEAs Lacked Focus on Migrants

The three LEAs each made a variety of student demographic and assessment data, including
migrant and LEP status, available to schools electronically and through printed reports. The
schools we visited used assessment results to address the academic and English proficiency
Final Report
ED-OIG/A09F0003                                                                      Page 20 of 30

needs of the LEP subgroup, but lacked a specific focus on migrant students. According to school
administrators in two LEAs, migrant students were generally also LEP and school personnel did
not separately track migrants’ academic performance. Ongoing efforts at schools in the three
LEAs addressed academic achievement for LEP students individually and as a group, at the
school-level and through the school’s LEP program. On some student lists, we found that
migrant status might be listed with other demographic and assessment data, but the schools did
not look at the migrants as a subgroup to track their progress compared to the LEP subgroup, or
identify other needs that may be unique to migrants.

Recommendation

5.1    We recommend that the Assistant Secretary for Elementary and Secondary Education
       encourage CDE to consider providing additional guidance to LEAs to (1) ensure that
       schools have access to students’ migrant status along with assessment results; and (2)
       take steps to look at migrants as a subgroup to track their progress on academic
       assessments, including comparison with the LEP subgroup where appropriate, and to
       identify needs that may be unique to migrants.

CDE Comments

In its comments, CDE stated that it will make available to school districts and schools “read
only” access to the Migrant Student Information Network that provides disaggregated
assessment reports for migrant students. In Spring 2006, CDE will provide training to the
Migrant Regional Offices on the effective use of assessment data in program planning and
increasing student achievement.
Final Report
ED-OIG/A09F0003                                                                      Page 21 of 30



                 OBJECTIVES, SCOPE, AND METHODOLOGY


Our audit objectives were to determine whether CDE and selected LEAs within the State had
sufficient controls to ensure (1) migrant and LEP students were being assessed; (2) assessment
results were disaggregated by students’ migrant and LEP status; (3) assessment reports enabled
educators to understand and use the reported information to address the specific academic needs
of migrant and LEP students; and (4) migrant and LEP students, who were enrolled in Title I
schools identified for improvement, gained the same access to public school choice and SES as
other eligible students. Our audit period was the 2003-2004 assessment cycle and the
implementation of public school choice and SES during school year 2004-2005.

To address these objectives, we interviewed CDE officials and staff responsible for
implementing the statewide assessment and accountability system and administering the Title I,
Title III (English language acquisition), and MEP programs. To determine whether internal
controls ensured the inclusion of migrant and LEP students, we assessed CDE’s policies,
procedures, and guidance for identifying migrant and LEP students; administering the
STAR-CST and CAHSEE assessments; reviewing, reporting, and using the assessment data;
reporting accountability results and Title I schools’ improvement status to LEAs; and
implementing public school choice and SES. We also reviewed CDE procedures for monitoring
and oversight of LEA and school implementation in those areas.

To evaluate LEA procedures and determine whether CDE controls were in place, we selected
three LEAs (Fresno USD, Lodi USD, and Pajaro Valley USD) that had (1) sizable migrant, LEP,
and Title I populations and (2) Title I schools across two or more school levels that were subject
to public school choice and SES requirements. We defined as “sizable” those LEAs that ranked
in the top 100 (among the unified school districts) across the Migrant Education, Title III, and
Title I programs, based on CDE data on the proportion of migrant enrollment to total enrollment,
proportion of LEP enrollment to total enrollment, and Title I allocation amount. To ensure that
the selected LEAs would include elementary, middle, and high schools, we limited our selection
pool to the 320 unified school districts from a total of 964 LEAs statewide. Of the six LEAs that
met our selection criteria, we judgmentally selected the three LEAs we visited based on the
number of Title I schools subject to choice and SES; LEA total enrollment to provide a mix of
large-, middle-, and small-sized LEAs; and diverse geographic locations.

At each LEA, we reviewed procedures and interviewed managers and staff responsible for
implementing the statewide assessment and accountability system, administering the Title I and
Title III programs, and approving parental requests for public school choice transfers. To
determine whether internal controls ensured the inclusion of migrant and LEP students, we
assessed LEA policies, procedures, and guidance for identifying migrant and LEP students;
administering the STAR-CST and CAHSEE assessments; reporting and using the assessment
data; reporting accountability results and Title I schools’ improvement status to schools; and
implementing public school choice and SES. Additionally, we reviewed LEA procedures for
monitoring and oversight of school implementation in those areas. For each LEA, we also met
with program managers and staff from the cognizant MEP regional office about program
procedures for identifying migrant students and using assessment results.
Final Report
ED-OIG/A09F0003                                                                     Page 22 of 30

We also visited a total of nine schools (an elementary, middle, and high school, in each LEA).
For each LEA, we used CDE data to select a Title I school subject to choice and SES (in Year 2
or higher in PI status) at each level with large proportions of migrant and LEP enrollments. Lodi
USD and Pajaro Valley USD did not have Title I schools at the high school level and, thus, none
of their high schools were subject to the choice and SES requirements. For these two LEAs, we
selected from among all the high schools, based on the proportion of migrant and LEP
enrollments. For each of the three LEAs, we randomly selected a school to visit when more than
one school met our selection criteria.

To determine whether CDE and LEA controls were in place at each school, we reviewed
documentation and interviewed school administrators, testing coordinators, the Title III
coordinator, and other staff on their procedures and practices for identifying LEP students,
administering STAR-CST and CAHSEE assessments, reviewing and using assessment data, and
implementing public school choice and SES where applicable.

We assessed the reliability of computer-generated data at CDE and the three LEAs, and
concluded that the data were sufficiently reliable to use in meeting our audit objectives.
Specifically, we assessed the databases that CDE used to report 2003-2004 disaggregated STAR
and CAHSEE summary results. Statewide and for the three LEAs, we verified: (1) migrant and
LEP demographic and participation results to the data posted on CDE’s website; (2) LEP
enrollments to the March 2004 Language Census; and (3) migrant enrollments to MEP data from
CDE’s Migrant Student Information System, the cognizant MEP region, and/or the applicable
LEA student information system. At each LEA, we assessed the reliability of the LEA student
information system, which was used to code student demographic information for assessments,
by verifying migrant enrollments to the MEP statewide or regional database, and LEP
enrollments to the March 2004 Language Census. We also reviewed LEA controls to ensure that
students’ migrant and LEP status information in the student information system were accurate
and complete. Additionally, we assessed the database that CDE used to report Title I schools’
improvement status by comparing CDE’s list of PI schools and their status to the LEA’s list at
the three selected LEAs.

We performed our fieldwork at CDE and LEA offices in Sacramento, Fresno, Lodi, and
Watsonville, California. We held an exit briefing with CDE officials on June 8, 2005. We
performed the audit in accordance with generally accepted government auditing standards
applicable to the scope of the review.
Final Report
ED-OIG/A09F0003                                      Page 23 of 30




                           ENCLOSURE

                  CDE Comments on the Draft Report
                                   September 28, 2005




Gloria Pilotti, Regional Inspector General for Audit
United States Department of Education
Office of Inspector General
501 I Street, Suite 9-200
Sacramento, CA 95814

Dear Ms. Pilotti:

This is the California Department of Education’s (CDE) response to the United States
Department of Education Office of Inspector General’s draft audit report entitled,
“California’s Inclusion of Migrant and Limited English Proficient Students in the
Statewide Assessment and Accountability System.” Superintendent Jack O’Connell
asked that I respond on his behalf.

Finding No. 1 – CDE Needs to Take Additional Steps to Ensure Limited English
Proficient Students Are Provided Reasonable Accommodations on Assessments

Recommendation 1.1:

Instruct local educational agencies (LEA) to establish and implement procedures to
ensure test administrators provide appropriate testing accommodations for limited
English proficient (LEP) students and properly mark the student answer document,
when applicable.

       CDE’s Corrective Action:

       The CDE-developed Testing Variations, Accommodations, and Modifications
       Matrix allows various variations for administering the tests within the
       Standardized Testing and Reporting (STAR) Program and the California High
       School Exit Examination (CAHSEE) to LEP students. This matrix ensures
       consistent testing variations across all state testing programs and provides
       necessary guidance to school and LEA staff members in understanding how the
       variations are to be provided to LEP students.

       To help ensure the appropriate variations are provided to LEP students,
       beginning with the spring 2006 test administrations, test examiners will be
       required to mark, on the answer documents, the specific testing variations the
Gloria Pilotti, Regional Inspector General for Audit
September 28, 2005
Page 2


       LEP students use. This information will then be included in the District Student
       Data File that each LEA receives, and it will be available in the annual technical
       report produced by the state's contractor.

Recommendation 1.2:

Ensure that the test administration instructions and student answer documents for the
2005-06 and future assessments clearly distinguish the LEP accommodations from the
ones for students with disabilities and section 504 students.

       CDE’s Corrective Action:

       The CDE improved the 2005-06 answer documents for both the STAR Program
       and CAHSEE. The answer documents now include a separate section for test
       examiners to mark testing variations used by LEP students. The testing
       variations used for the STAR are to be marked separately for each content area
       test—English-language arts, mathematics, science, and history-social science.

Recommendation 1.3:

Include data on LEP students’ use of accommodations in the assessment data made
available to LEAs to ensure LEP students are provided reasonable accommodations.

       CDE’s Corrective Action:

       Beginning with the 2005-06 test administration, all testing variations used by LEP
       students will be included in each district’s and the state’s electronic student data
       files that include all test scores and student demographic data. The state’s file
       includes no student names.
Gloria Pilotti, Regional Inspector General for Audit
September 28, 2005
Page 3


Finding No. 2 – CDE Needs to take Additional Steps to Ensure Eligible Students
Have Access to Public School Choice in Accordance with Applicable Laws and
Regulations

Recommendation 2.1:

Continue to refine CDE’s assessment and accountability processes and procedures to
ensure that LEAs will receive assessment, adequate yearly progress (AYP), and
program improvement (PI) results before the beginning of the school year, including
July and August starts, so that parents of eligible students can be informed of public
school choice options in a timely manner.

       CDE’s Corrective Action:

       The early delivery of test results to LEAs for AYP and PI purposes is a difficult
       outcome to achieve without substantial changes in the law and policies governing
       the administration of the STAR Program and CAHSEE. These changes would
       include eliminating year-round schools or changing regulations specifying the
       testing window for the STAR Program, drastically changing procedures for
       scaling tests (which could increase the error in the scale score metric), reducing
       the edit checks currently conducted on the student demographic data, and
       decreasing the number of STAR Program tests offered so that the complexity of
       compiling score results in the shorten time span is made less complicated.

       In CDE’s September 2004 approved State of California Consolidated State
       Application Accountability Workbook, critical element 1.4 was revised to include a
       specific date in August (August 31) that California will release all elements of the
       AYP determination. AYP calculations were made available to LEAs on the CDE’s
       website on August 25, 2005 and to the public on August 31, 2005. The data were
       made available to LEAs one week earlier this year in order for them to review the
       information for accuracy and make PI determinations for their schools.

       Furthermore, in numerous communications regarding the 2005 AYP reporting
       cycle, the CDE stated that it is the responsibility of the LEA to identify schools for
       PI immediately after the release of the 2005 AYP data. LEAs are not to delay the
       identification of schools for PI pending data corrections, appeals of AYP data or
       until the CDE releases the official list of schools in PI.

       The CDE released the list of schools and LEAs in PI to LEAs for review on
       September 13, 2005, more than 30 days earlier than this information was
       provided in 2004, and released it to the public on September 20, 2005.
Gloria Pilotti, Regional Inspector General for Audit
September 28, 2005
Page 4


Recommendation 2.2:

Provide written guidance to LEAs describing the information that must be included in
parental notification letters about public school choice, including the identification of at
least two school options, where applicable, and the schools’ academic achievement
information.

       CDE’s Corrective Action:

       Sample parent notification letters for use by school districts, county offices of
       education, and direct-funded school, including a cover letter with more detailed
       instructions on the use of the letters, are posted on CDE’s website. The sample
       letters address all the components required in No Child Left Behind (NCLB),
       Section 1116(b)(6). Additionally, the LEA and School Improvement
       Nonregulatory Guidance of January 7, 2004 recommends providing information
       about the schools available for transfer. Therefore, the sample letters provide
       contact information for parents interested in school choice and inform them that
       interested parents will receive information about the schools available for transfer
       and the schools' academic achievement.

Recommendation 2.3:

Enhance CDE’s monitoring procedures by incorporating (1) the reference to the
applicable statutory provision on parental notifications and (2) specific instructions for
reviewers to confirm that LEA parental notification letters regarding public school choice
contain the required information.

       CDE’s Corrective Action:

       Training institutes will be held in eight regions throughout the state to inform the
       LEAs about the required parent notification components and the need to ensure
       that parents are adequately informed about the schools available for transfer and
       the academic status of the schools.

       The Program Improvement component of the 2005-06 Categorical Program
       Monitoring Document includes compliance Item II-PI 1, which requires CDE
       reviewers to ensure that the LEA parent notification letter incorporates all the
       required components.
Gloria Pilotti, Regional Inspector General for Audit
September 28, 2005
Page 5


Finding No. 3 – CDE Could Enhance Procedures to Ensure All Migrant and LEP
Students Are Being Assessed

Recommendation 3.1:

Consider enhancing the ability of the State, LEAs, and schools to ensure that all student
subgroups, including migrants and LEPs, are being assessed by reporting or making
available data on the number of students (by subgroup) enrolled on the first day of
testing.

       CDE’s Corrective Action:

       CDE will require the Migrant Regional Offices to provide the districts with the
       most current list of migrant students when the district is completing the STAR
       header sheets, and again at the beginning of the STAR testing period. CDE will
       inform the districts that the list includes all migrant students that need to be
       assessed, and that migrant students assessed are on the list.

Recommendation 3.2:

Consider enhancing the ability of the State, LEAs, and schools to ensure all migrant
students are being assessed by (1) taking the steps necessary to ensure the reliability
and use of the statewide Migrant Education Program (MEP) database to check the
reasonableness of migrant enrollment data for assessment purposes, and (2) instructing
LEAs to reconcile student information with MEP regions’ data at the time of testing.

       CDE’s Corrective Action:

       Currently, the California Basic Education Data System (CBEDS) and Language
       Census data are used to perform reasonability checks on the participation rates
       for LEP students tested. In both the CBEDS and Language Census, there is an
       established census collection date, March 1 of each year. The migrant data has
       no such official census count since it is constantly updated. Therefore, there is no
       specific number from the migrant database that can be used to perform
       reasonability checks on the data. However, CDE will direct Migrant Education
       Regional Offices to upload the most current enrollment data on migrant students
       to the state migrant student database on or about March 1 of each year.
Gloria Pilotti, Regional Inspector General for Audit
September 28, 2005
Page 6


Finding No. 4 – CDE Should Disaggregate CAHSEE Summary Reports by
Students’ Migrant Status

Recommendation 4.1:

Consider making available, or instructing LEAs to produce, summary reports that
disaggregate CAHSEE results for the migrant subgroup.

       CDE’s Corrective Action:

       As required under the Elementary and Secondary Education Act (ESEA) Section
       1111(b)(3)(C)(xiii), CDE enables CAHSEE results to be disaggregated by, among
       other things, students’ migrant status and English proficiency status. In the
       electronic student data files provided to the LEAs following each of the CAHSEE
       administrations and twice after each school year, data is provided to enable
       assessment results to be disaggregated by migrant and English proficiency
       status. Additionally, as part of CDE’s annual data release of CAHSEE results,
       CDE posts on its website a research file, which contains all of the data for a
       school district, or member of the public, to disaggregate the data by migrant
       status. Although CDE has provided LEAs with the requisite data up to seven
       times annually, CDE will explore modifying the current CAHSEE Results Web
       Site to include summary reports that disaggregate CAHSEE results for the
       migrant subgroup.

Finding No. 5 – LEAs Could Enhance the Use of Assessment Results to Improve
Migrant Students’ Academic Achievement

Recommendation 5.1:

Consider providing additional guidance to LEAs to (1) ensure that schools have access
to students’ migrant status along with assessment results; and (2) take steps to look at
migrants as a subgroup to track their progress on academic assessments, including
comparison with the LEP subgroup where appropriate, and to identify needs that may
be unique to migrants.

       CDE’s Planned Corrective Action:

       CDE will make available to school districts and schools “read only” access to the
       Migrant Student Information Network that provides disaggregated assessment
       reports for migrant students; and in the spring of 2006, provide training to the
       Migrant Regional Offices regarding the effective use of assessment data in
       program planning and increasing student achievement.
Gloria Pilotti, Regional Inspector General for Audit
September 28, 2005
Page 7


If you have any questions regarding CDE’s response, please contact Kim Sakata, Audit
Response Coordinator, Audits and Investigations Division, at (916) 323-2560 or by
e-mail at ksakata@cde.ca.gov.

Sincerely,

/s/ Sue Stickel for

GAVIN PAYNE
Chief Deputy Superintendent of Public Instruction

GP:ks