oversight

Improving Title I Data Integrity for Schools Identified for Improvement.

Published by the Department of Education, Office of Inspector General on 2002-03-27.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

IMPROVING TITLE I DATA INTEGRITY FOR SCHOOLS

        IDENTIFIED FOR IMPROVEMENT 





                                   FINAL AUDIT REPORT

                              Control Number ED-OIG/A03-B0025

                                         March 2002





Our mission is to promote the efficiency,                 U.S. Department of Education
effectiveness, and integrity of the                       Office of Inspector General
Department’s programs and operations..                    Philadelphia, Pennsylvania
                              NOTICE


Statements that management practices need improvement, as well as
other conclusions and recommendations in this report, represent the
opinions of the Office of Inspector General. Determinations of
corrective action to be taken will be made by appropriate Department
of Education officials.

In accordance with the Freedom of Information Act (5 U.S.C. §552),
reports issued by the Office of Inspector General are available, if
requested, to members of the press and general public to the extent
information contained therein is not subject to exemptions in the Act.
                          UNITED STATES DEPARTMENT OF EDUCATION

                                            OFFICE OF INSPECTOR GENERAL




MEMORANDUM
                                                                                                     MA.R 27 2002
TO: 	             Susan B. Neuman




FROM: 	           Thomas A.. Carter-
                  Assistant Inspector General for Audit

SUBJECT: 	        FINAL AUDIT REPORT
                  Improving Title I Data Integrity for Schools Identi,fiedfor Improvement
                  Control No. ED-OIG/A03-B0025

Attached is our subject final report that covers the results of our review of management controls
over Title I, Part A performance data concerning schools identified for improvement at the Office
of Elementary and Secondary Education and the Planning and Evaluation Service, Office of the
Under Secretary for school years 1997-98, 1998-99 and 1999-2000. We reviewed your
comments concurring with the findings and recommendations in our draft audit report.

You have been designated as the primary action official for this report. The Under Secretary is
the collateral action official. Please coordinate with him regarding any action in connection with
recommendati ons.

Pleas\:: provide the Supervisor, Post Audit Group, Office of Chief Financial Officer and the Office
of Inspector General with quarterly status reports on promised corrective action until all such
actions have been completed or continued follow-up is unnecessary.

In accordance with the Freedom of Information Act (5 U.S.C.§552), reports issued by the Office
of Inspector General are available, if requested, to members of the press and general public to the
extent information contained therein is not subject to exemptions in the Act.

We appreciate the cooperation given us in the review. Should you have any questions concerning
this report, please call Bernard Tadley, Regional Inspector General for Audit at (215) 656-6279.

Attachment

cc: Joseph Johnson, Director, Compensatory Education Programs




                                 400 MARYLAND AVE .• S.W. WASHINGTON. D.C. 20202-1510

         Our mission is to ensure equal access to education and to promote educational exceUence throughout the Nation.
                          UNITED STATES DEPARTMENT OF EDUCATION

                                            OFFICE OF INSPECTOR GENERAL




MEMORANDUM
                                                                                          MAR 27 2002
TO:




FROM:
                  Assistant Inspector General for Audit

SUBJECT:          FINAL AUDIT REPORT
                  Improving Title I Data Integrity for Schools Identifiedfor Improvement
                  Control No. ED-OIG/A03-B0025

Attached is our subject final report that covers the results of our review of management controls
over Title I, Part A performance data concerning schools identified for improvement at the Office
of Elementary and Secondary Education (OESE) and the Planning and Evaluation Service, Office
of the Under Secretary for school years 1997-98, 1998-99 and 1999-2000. We reviewed your
comments concurring with the findings and recommendations in our draft audit report.

You have been designated as the collateral action official for this report. The Assistant Secretary
for OESE has heen assigned as the primary action .official. Please coordinate with her regarding
any action in connection with recommendations.

Please provide the Supervisor, Post Audit Group, Office of Chief Financial Officer and the Office
of Inspector General with quarterly status reports on promised corrective action until all such
actions have been completed or continued follow-up is unnecessary.

In accordance with the Freedom of Information Act (5 U.S.C.§552), reports issued by the Office
of Inspector General are available, if requested, to members of the press and general public to the
extent information contained therein is not subject to exemptions in the Act.

We appreciate the cooperation given us in the review. Should you have any questions concerning
this report, please call Bernard Tadley, Regional Inspector General for Audit at (215) 656-6279.

Attachment

cc: Alan L. Ginsburg, Director, Planning and Evaluation Service
    Hugh Walkup, Director, Planning and Policy Services




                                 400 MARYLAND AVE., S.W. WASHINGTON, D.C. 20202-1510

         Our mission is to ensure equal access 10 education and to promote educational excellence throughout the Nation.
                              TABLE OF CONTENTS


                                                                              Page



Executive Summary                                                             1



Background                                                                    2



Audit Results                                                                 3



Finding No. 1 – Management Controls Over the Timely Publication of School 

                Improvement Data Need to be Strengthened                      3



Finding No. 2 – Department Needs Stronger Management Controls to Ensure that

               Title I School Improvement Data are Reliable and Valid        6



Other Matters                                                                 9



Objective, Scope and Methodology                                              9



Statement on Management Controls                                              10



Attachment – Auditee Comments                                                 12

Improving Title I Data Integrity for Schools
Identified for Improvement                                    Control Number ED-OIG/A03-B0025



                                       Executive Summary


The purpose of this review was to assess whether the U. S. Department of Education (the
Department) has management controls to ensure that Title I, Part A performance data
concerning schools identified for improvement (i.e., schools that fail to meet State criteria
for adequate yearly progress) was reliable, valid, and timely. Local Educational
Agencies (LEA) were required to review the progress of each Title I school to determine
whether the school was making adequate yearly progress. Schools that did not make
adequate progress for two consecutive years were to be identified by LEAs for
improvement. LEAs reported this information to State Educational Agencies (SEA),
which in turn reported the data to the Department. The Office of Elementary and
Secondary Education (OESE) and the Planning and Evaluation Service (PES), Office of
the Under Secretary (OUS) have responsibility for management controls over school
improvement data reported by SEAs.

While the Department had a process in place to collect and publish the Title I States’
school improvement performance data, it needs to strengthen management controls over
this process. The Department needs to take steps to improve its management controls
over school improvement data to ensure that the data are reliable, valid, and timely.
Specifically, our audit disclosed that:

    •	 Management controls over the timely publication of school improvement data
       need to be strengthened; and
    •	 Department needs stronger management controls to ensure that Title I school
       improvement data are reliable and valid.

The No Child Left Behind Act of 2001, Public Law 107-110, enacted January 8, 2002,
places even more emphasis on the accountability for results and defines consequenc es for
schools that do not make adequate yearly progress. We recommend that the Assistant
Secretary for OESE and the Under Secretary take steps to improve management controls
over the timeliness, reliability and validity of the data.

OUS and OESE generally concurred with our findings and recommendations. A
complete copy of the ir joint response is included as an attachment to the report.




                                               1

Improving Title I Data Integrity for Schools
Identified for Improvement                                   Control Number ED-OIG/A03-B0025



                                               Background


The Title I, Part A program was enacted under the Elementary and Secondary Education
Act (ESEA), as amended by the Improving America’s Schools Act of 1994, Public Law
103-382. The 1994 reauthorization of the ESEA introduced standards-driven reform.
Specifically, Title I services were to be linked to the same State content and performance
standards that were expected of all children, and aligned assessments were to be used to
measure students’ progress toward meeting those standards. In addition, States were
required to put in place a system of accountability designed to identify and assist schools
that did not make adequate yearly progress toward meeting the standards. LEA were
required to annually review the progress of each Title I school to determine whether the
school was making adequate yearly progress. Schools that did not make adequate
progress for two consecutive years were to be identified for improvement.

In fiscal year 2000, the Department awarded approximately $8 billion in ESEA Title I
funds. Title I, Part A provides financial assistance through SEA’s to LEA’s and schools
to help meet the educational needs of children who were failing, or most at risk of failing,
to meet challenging State academic standards, and who resided in areas with high
concentrations of children from low- income families.

By the 1997-98 school year, each State was to develop challenging content (i.e., reading
and math) and student performance standards. According to OESE program officials, 51
States and jurisdictions (out of 52) had approved content standards by February 2002, and
28 States and jurisdictions had performance standards for all required grade spans by
February 2002. By the 2000-2001 school year, States were to have in place, assessments
that were aligned with these standards and procedures for holding schools accountable for
the results. A status report as of September 19, 2001, disclosed that only 16 States had
approved assessment systems, of the remaining 36 States and territories, 28 had received
a waiver, and eight had not complied with the law. SEAs were required to take corrective
actions to improve low performing schools and districts whose students failed to make
adequate yearly progress toward meeting the challenging State standards.

To meet its administrative responsibilities and report performance information to
Congress, the Department required States to submit annual performance data. The
reporting instrument, Consolidated State Performance Report, required all States to
provide data under eight formula grant programs. The Department required States to
submit the reports by December 1. The report reflected data for the previous school year
(for example, the December 1, 1999, report covered school year July 1, 1998- June 30,
1999).

OESE administers the Title I program of the Elementary and Secondary Education Act.
The administration of the program included the collection, review and monitoring of the
performance information provided by States. PES helps guide the Title I program
priorities by evaluating the effectiveness, efficiency, and equity of the programs. OESE
and PES have a shared responsibility for school improvement data reported by SEAs.


                                                   2

Improving Title I Data Integrity for Schools
Identified for Improvement                                      Control Number ED-OIG/A03-B0025

The Department hired a contractor to analyze and edit, as appropriate, the participation,
services, and achievement data from the Title I portion of the Consolidated State
Performance Report. The contractor consolidated the analysis of the States’ data into the
State ESEA Title I Participation Information Summary Report. This report provided the
scope of the Title I program services for two successive school years. The data was
aggregated by States and by several aspects of the Title I program including the number
of school districts, schools and students served, range of instruction and support services
provided, Title I staffing patterns, and schools meeting the adequate yearly progress
requirement.

The No Child Left Behind Act of 2001, enacted January 8, 2002, reauthorized the ESEA.
The Act strengthens Title I accountability by requiring States to implement statewide
accountability systems covering all public schools and students. These systems must be
based on challenging State standards. School districts that do not make adequate yearly
progress toward statewide proficiency goals will over time be subject to improvement,
corrective action, and restructuring measures aimed at getting them back on course to
meet State standards.


                                               Audit Results


The Department had a process in place to collect and publish the Title I States’ school
improvement data. However, the Department needs to strengthen management controls
over the process. The Department’s management controls did not ensure that school
improvement data provided by States was reliable, valid, and timely. Specifically, our
audit disclosed that (1) management controls over the timely publication of school
improvement data need to be strengthened, and (2) Department needs stronger
management controls to ensure that Title I school improvement data are reliable and
valid.


Finding No. 1 – Management Controls Over the Timely Publication of School
Improvement Data Need to be Strengthened

Management controls were not adequate to ensure the timely receipt, review, and
publication of school improvement data. As a result, the Department has not published
the State ESEA Title I Participation Information Summary Report in a timely manner.
This report contains performance data on schools identified for improvement. Based on
our review of the 2000 (1997-1998 data) and 2001 (1998-1999 data) reports, we
determined that the Department published those reports approximately 22 months after
the data was due from the States. Not publishing information timely can cause it to lose
its usefulness. The newly enacted No Child Left Behind Act of 2001 places an even
greater emphasis on the importance and timeliness of this data, because it defines
consequences for schools that do not make adequate yearly progress.

The general instructions in the Department’s Consolidated State Performance Report
required that all States, whether or not they receive funding on the basis of consolidated
State plans, must respond. Each State report was due December 1 of each year and
                                               3

Improving Title I Data Integrity for Schools
Identified for Improvement                                      Control Number ED-OIG/A03-B0025

reflected data for the previous school year (for example, the December 1, 1999 report
covered school year July 1, 1998- June 30, 1999). This information was needed by the
Department to meet its administrative responsibilities. 1

To ensure that performance data were of the highest quality, the Department adopted
Data Quality Standards 2 for Department of Education program managers. One of these
standards required that data be reported as soon as possible after collection. Specifically,
Standard Five: Timeliness requires data to be recent and reported in time to inform policy
action.

Management controls over the timely collection of Title I performance data need
improvement

OESE did not have adequate controls to ensure that program officials’ followed-up with
States that did not submit Title I performance data on time. Based on our review of the
Title I data collection process for three years (school years (SY) 1997-98, 1998-99, and
1999-00), OESE did not have a systematic process for follow-up with States that were
late in submitting data. The OESE program officials noted that they generally contacted
State officials by telephone and did not keep consistent records of these contacts. For the
most recent year (SY 1999-00), OESE program staff advised us that actions on late
submission of data from States were not initiated until three months after the due date.

OESE initiated the use of a log to record the receipt and status of State consolidated
reports for SY 1998-1999. A review of the log for the reports due on December 1,
1999, disclosed the following:

                                             School Year 1998-1999
                          On         5 to 30       31 to 120       121 to 270 days     Total
                          time       days late     days late       late              Reported
Number of                 4          17            16              15                523
Consolidated State
Performance
Reports




1
  The Department provides information to Congress, including that under the
Government Performance Results Act (GPRA) of 1993 and congressional mandates for
the national assessment of ESEA, Title I (section 1501), and evaluation of the federal
impact on reform (ESEA, Title XIV, section 14701) contained in the 1994
reauthorization of the ESEA.
2
  Subsequent to the publication of the 1999 Performance Reports and 2001 [Annual]
Plans in March 2000, ED revised the Data Quality Standards by increasing the number
of standards from six to eight. In March 2001, ED published the revised standards as
Draft Data Quality Standards in its 2000 Performance Report and 2002 Annual Plans.
For the purpose of this audit, we used the March 2000 standards that were in effect
during our audit period.
                                            4

Improving Title I Data Integrity for Schools
Identified for Improvement                                 Control Number ED-OIG/A03-B0025

The subsequent year’s log (SY 1999-2000) was not complete. Therefore, we could not
determine the extent of late submissions.

In addition to late submissions from States, the Department’s contractor informed us that
for SY 1998-1999, approximately 40 States were requested to revise their initial data
submission because of conditions identified during the edit check process. Also, one
State did not finalize its data submission until two weeks before the SY 1998-1999 report
was published in October 2001, because the largest LEA in the State was late in
submitting its data to the State.

Our review disclosed that OESE’s management controls did not ensure timely follow- up
of late submissions because OESE did not ha ve a systematic process. Specifically, (1)
there were no time frames established for obtaining late data submissions, (2) OESE
program officials did not enforce the submission deadline (i.e., December 1) with clear
and frequent reminders, and (3) no actio n was taken against States that repeatedly
submitted data late or States that were non-responsive.

Management controls over the timely review of Title I performance data need
improvement

OESE and PES need to strengthen controls to ensure timely review and publication of
school improvement data. Specifically, the review process on the various drafts of the
report were not performed concurrently and time frames were not established by the
Department to ensure the timely publication of the State ESEA Title I Participation
Information Summary Report.

The recently enacted No Child Left Behind Act of 2001 has raised the level of
importance of performance data on schools identified for improvement. The new law
emphasizes accountability for results and consequences for schools that do not meet goals
once identified for improvement. The new law requires annual report cards from LEAs
and SEAs on achievement indicators, including school improvement data. Further, the
law includes a requirement for the Department to report achievement results to Congress.
Since these submissions will affect future program decisions, and will be the basis of
information reported to Congress, the Department needs to strengthen its controls for
ensuring that the submissions and subsequent reporting of the data are timely.

Recommendation:

1.1    The Assistant Secretary for Elementary and Secondary Education and the Under
Secretary should strengthen management controls to ensure the timely receipt, review and
publication of performance data concerning schools identified for improvement.




3
  The State of Alaska, for unknown reasons, was left off the log, reducing the number of
entities to 52.
                                          5
Improving Title I Data Integrity for Schools
Identified for Improvement                                    Control Number ED-OIG/A03-B0025

Auditee’s Comments:

OUS and OESE generally concurred with the finding and recommendation. They
informed us that they have revised the report review process and currently use a process
that includes concurrent reviews by reviewing offices. In addition, OUS and OESE are
working on developing more efficient follow-up procedures. For the most recent draft
report (1999-2000 school year data), they stated that OESE called States earlier and as a
result, received data more quickly; and they expect to publish the 1999-2000 report this
spring. The Department plans to provide technical assistance to States on implementing
the new reporting requirements of No Child Left Behind Act. They believe that this
assistance will also expedite the collection and reporting of State data as well as improve
its quality.


Finding No. 2 – Department Needs Stronger Management Controls to Ensure that
Title I School Improvement Data are Reliable and Valid

The Department needs to strengthen its process for ensuring that Title I school
improvement data are reliable and valid. The Department relied on a contractor to
perform edit checks on the school improvement data prior to its publication. This edit
check process involved reviewing the data for anomalies, providing States with edit
check reports, making telephone contacts with the submitters of the data, and responding
to changes made by States. Our audit work concerning school improvement data in a
large State determined that the edit check process alone was not adequate to ensure that
the published school improvement data submitted by the State was reliable and valid.

The Department’s Data Quality Standards 4 require that performance data be valid and
accurate. Specifically:

       Standard One: Validity - requires that the data adequately represents performance
       and that performance indicators actually measure the goal or objective of interest.

       Standard Two: Accurate - data are based upon correct counts, agreed- upon
       definitions and the phenomena being measured occurred or existed at the time for
       which they were reported.




4
 In March 2000, ED published Data Quality Standards to assist its internal managers as
they collect, analyze, and report data about Federal programs, including Title I, Part A.
OIG used these standards to evaluate the California Department of Education's
management controls over data quality. The OIG has suggested that ED distribute the
standards to States to help ensure they provide reliable, valid, and timely performance
data to ED for such programs as Title I (OIG Information Memorandum - State and Local
No. 01-01, dated August 3, 2001, titled State-Reported Data Used in Measuring
Performance of Education Programs). While ED has not yet distributed the standards to
States, the Data Quality Standards are accessible on the Department's Web site.

                                               6

Improving Title I Data Integrity for Schools
Identified for Improvement                                   Control Number ED-OIG/A03-B0025

The contractor and OESE staff advised us that they were unable to check the accuracy of
the Title I performance data without requesting supporting documentation from States.
We were advised that the supporting documentation maintained by States was not
collected because it would have required Office of Management and Budget approval.
We also found that the OESE has not requested or reviewed school improvement data
during State monitoring visits to ensure that data was reliable and valid. The OESE
advised us that they will be piloting an achievement focused monitoring process, during
fiscal year 2002. The monitoring visits will be based on a performance matrix and school
improvement data will be reviewed during those visits.

Monitoring Data

Monitoring of school improvement data can identify weaknesses in a State’s data
controls. For example, an OIG audit of California Department of Education (CDE)
performance data disclosed that CDE reported inaccurate school improvement data to the
Department. The audit identified weaknesses in management controls in the reporting of
school improvement data. Specifically, CDE did not:

•    report schools that were previously identified for improvement;
•    document its data collection process or retain supporting documentation; and
•    review the reported data for reliability.

While these conditions were found during an audit, they could have been identified
during an OESE monitoring visit, if the visit included procedures to review data
maintained to support a State’s submission of school improvement data.

Validation Controls are Needed

In addition to weak controls over the validation of school improvement data, the
Department did not require States to validate the performance data submitted to the
Department, such as having the appropriate State official attest that the data reported for
the Consolidated State Performance Report was valid, and reliable. Without the
appropriate State official attesting to the validity of the data submitted, the Department
lacked additional assurance that the data adequately represented performance.

The Single Audit Act of 1984 established requirements for audits of States, local
governments and Indian tribal governments that administer Federal financial assistance
programs. The Compliance Supplement provides guidance to assist auditors in
determining compliance requirements relevant to the audit, audit objectives and
suggested audit procedures. The Department has not included the area of school
improvement data controls in the Compliance Supplement used for conducting audits at
SEAs and LEAs under the Single Audit Act.

On August 3, 2001, the Office of the Inspector General (OIG) issued an information
memorandum to the Under Secretary informing him of action that the Department could
take to increase its assurance that States provide reliable data for use in GPRA reports.
The OIG suggested that the Department:

                                               7

Improving Title I Data Integrity for Schools
Identified for Improvement                                  Control Number ED-OIG/A03-B0025

•	 Distribute the Department’s Draft Data Quality Standards to SEAs and encourage
   SEAs to provide the standards to LEAs located in their State.

•	 Include in the Compliance Supplement audit cove rage of data used for performance
   indicators because SEA and LEA reviews of the data are limited and may not ensure
   that the data are reliable.

Implementation of the suggestions in the memorandum will also provide the Department
managers with increased assurances that the SEA and their LEAs school improvement
data are reliable. The Department is currently reviewing the memorandum.

The recently passed No Child Left Behind Act of 2001 strengthens the accountability for
results and defines consequences for schools that do not make adequate yearly progress,
increases the importance of the data and the need for the Department to have assurances
that school improvement data are reliable and valid. Improving management controls
over the school improvement data and implementation of the suggestions in the OIG
memo to the Under Secretary will provide added assurances to Department managers.

Recommendations: The Assistant Secretary for Elementary and Secondary Education
and the Under Secretary should:

2.1	     Develop and implement written procedures to assess, during monitoring visits to
         SEAs and LEAs, whether school improvement data are reliable and valid.

2.2	     Distribute the Department’s Data Quality Standards to SEAs and encourage SEAs
         to provide the standards to LEAs for their use.

2.3	     Include audit procedures in the OMB Compliance Supplement to review controls
         over Title I, Part A school improvement data at LEAs and SEAs during annual
         Single Audits.

Auditee’s Comments:

OUS and OESE generally concurred with the finding and recommendations. OESE plans
to develop written procedures to assess the reliability and validity of school improvement
data during monitoring visits to States. Further, OESE hopes to improve the quality of
Federal data, as well as minimize burden on States by implementing a performance-based
data management initiative.




                                               8

Improving Title I Data Integrity for Schools
Identified for Improvement                                   Control Number ED-OIG/A03-B0025



                                           Other Matters


We reviewed the State ESEA Title I Participation Information for 1997-98 and 1998­
1999 Final Summary Reports. Our review found that, while an explanation was made
regarding the impact that changes in a State’s assessment system could have on the data,
the reports did not identify the States that changed their assessment systems. A change in
a State’s assessment system used to measure schools’ performance may impact the
number of schools identified as in need of improvement. As a result, year to year
comparisons of the data, without adequate disclosure of a change in the State’s
assessment system could lead a reader or decision maker to draw incorrect conclusions or
question the validity and reliability of the data. We suggest that PES identify in the
report the States that change assessment system from one period to the next.

Auditee’s Comments:

PES plans to note significant changes in these areas so that readers will have more
complete information as they review the school improvement data.


                            Objective, Scope and Methodology


The purpose of our audit was to determine whether the U. S. Department of Education
has management controls in place to ensure that Title I, Part A performance data
concerning schools identified for improvement (i.e. schools that fail to meet State criteria
for adequate yearly progress) was reliable, valid, and timely. 5

Our audit period covered Title I, Part A data concerning schools identified for
improvement for the 1998-1999 school year (July 1, 1998 - June 30, 1999). Because the
work indicated weaknesses in the management controls outside the audit period, we
extended our review to include school years 1997-1998 and 1999-2000.

To accomplish our audit objective, we conducted interviews and reviewed available
documentation in the OESE, PES, and the Department’s contractor. The contractor
performs an analysis of the State’s data and prepares the State ESEA Title I Participation
Information Summary Report. We also tested the accuracy and completeness of the
published data by comparing the State’s data with any changes and reconciling it to what
was published. We did not rely on computer-processed data in this audit. The

5
  This audit was conducted as part of a joint project of the U. S. Comptroller General’s
Domestic Working Group to determine whether data used for the purpose of the
Government Performance and Results Act of 1993 (GPRA) for the Title I program are
reliable, valid, and timely. The participants in the joint effort are the U. S. General
Accounting Office (GAO), U. S. Department of Education’s Office of Inspector General
(OIG), Texas State Auditor’s Office, Pennsylvania Auditor General’s Office, and the
Philadelphia City Controller’s Office.

                                                 9

Improving Title I Data Integrity for Schools
Identified for Improvement                                    Control Number ED-OIG/A03-B0025

Department’s data collection and review process was not automated. The Department’s
contractor captures and edits the data on a personal computer Access database program.

From the universe of 53 reporting entities (50 States, District of Columbia, Puerto Rico
and Bureau of Indian Affairs), we stratified the States into large, medium, and small
based on the reported number of schools identified for improvement in the Title I reports
issued October 2001, for school years (SY) 1997-1998 and 1998-1999. We judgmentally
selected 10 (three large, four medium, and three small) Title I reports for each year under
review.

We reviewed the Department’s GPRA report, FY 2000 Interim Program Performance
Reports and FY 2002 Program Plans, to determine whether the Department had
indicators related to Title I. We found that the Department established 11 performance
indicators for Title I grants to schools serving at-risk children. Two of the 11 indicators
used the annual Title I State performance report as a data source. The other Title I
indicators used various studies from the National Longitudinal Study of Schools, Public
School Survey on Education Reform, National Assessment of Education Progress and
Title I peer review records

We conducted our fieldwork at OESE and PES offices in Washington, DC, and the
contractor’s office in Rockville, Maryland, during the period September 23, 2001,
through December 7, 2001. We held an exit conference with OESE Title I and PES
officials on December 11, 2001. We conducted the audit in accordance with government
auditing standards appropriate to the scope of review described above.


                           Statement on Management Controls


We have made a study and evaluation of the management control structure of OESE, PES
and the Department’s contractor in effect during our fieldwork. Our study and evaluation
was conducted in accordance with Government Auditing Standards. For the purposes of
this report, we assessed and classified the significant management controls structure into
the following categories:

•   Process for receipt and follow-up of States’ Title I performance data;
•   Data review and edit process; and
•   Reporting of Title I performance data.

The management of OESE and PES are responsible for establishing and maintaining a
management control structure. In fulfilling this responsibility, estimates and judgment by
management are required to assess the expected benefits and related cost of control
procedures. The objectives of the management control structure are to provide
management with reasonable, but not absolute, assurance that assets are safeguarded
against loss from unauthorized use or disposition and that the transactions are executed in
accordance with management’s authorization and recorded properly, so as to permit
effective and efficient operations.


                                               10

Improving Title I Data Integrity for Schools
Identified for Improvement                                    Control Number ED-OIG/A03-B0025

Because of inherent limitations in any management control structure, errors or
irregularities may occur and not be detected. Also, projection of any evaluation of the
system to future periods is subject to the risk that procedures may become inadequate
because of changes in conditions, or that the degree of compliance with the procedures
may deteriorate.

Our assessment disclosed the following conditions in management control structure of
OESE, and PES in effect during our fieldwork, which, in our opinion, results in more
than a relatively low risk that errors or irregularities in amounts that would be material in
relation to reported information may occur and not be detected within a timely period:

    •	 Management controls over the timely publication of school improvement data
       need to be strengthened; and
    •	 Department’s controls to validate Title I performance data are not adequate.

These conditions and their effects are fully discussed in the Audit Results section of this
report.




                                               11

 Improving Title I Data Integrity for Schools
 Identified for Improvement                                                         Control Number ED-OIG/A03-B0025




                                             UNlTFAl STATES DEPARTMENT OF' EDUCATION




                    M£MORANOUM

                    TO; 	 ,-,~,,,, _ ,Bem~rd Tadl~ .       t

                          -, '., <,,'   Rcg10n81lf\speitO{Gtneral rOt Audit 


                    FROM: 	             Dr.~kok_
                    DA'l'E: 	           Man:h 18,2002
                    SUBJECT; 	 rmpnwing TUle I D4Wr ImtriJy for Smoo111tItntifod for ImpromMlfi 

                               Control No. £D.OIGlA03·B002S 


                    ThanJ: YOII for providing OUS and OES£ the opponunily to ~t on the draft Imprtwilfg 

                    TWt 1 DIlUl Jntlgrily for Schools ItknJfI"dfor Imp~ report. Our comments follow: 


                    • 	 The Baclc,mmd $tCtion of the repon: 1'IOIeS malOESE affu:ilIls Jepon that aU Swes bl\1~
                        met me    requirement to bavc Wllent and pcrforttWl>Ce standards is place by fIle 1997-98
                        iChool.)'Cal. This. is not accurate. .~ rcquin:d. by 1991-1998, 51 s.atcs and
                        jllrisdictioos (out of 52) had approved oonlClll SUIS1dar4s by Pebruary 2002, and 28 Stlt~ and
                       jurisdictions had pedonnance Stlndard'$ for aU required gade spans, by f'~ 2001.

                   .. 	 In 1he $eoood to last paragns.pb of the Backgl'ow section. we suggest you nOft (hilt the data
                        reponed in die Title I Participation repoxt. are ~ for t.at:b Stale and not brok.-eI1 out
                        fOt"each school and district, Also, the JasLsetltellCC indit;1I{eS Chat dw report includes "schooi
                        pro~ lOW&Id meednt content $t!Il\dards." This is oot quire correct. SUttes establish
                        oonlCnt standards but an: not required to report on ~hool progress toward meeting those
                        standardS. Radler. the SW4\$ tepM on schools meeting adequate yearly ~
                        requiremolll$,

                   • 	 Finding'1 note$ mat ED did flO/! publish me Swe £SEA Tille 1Participation lntomution
                       Sumnwy Repon in ! timely manner for the 1997·98 and 1998-99 ~ing yunlmd goos
                       on 10 discuss a key reason for report: delays, On !his issue we wooJdJib: to make (he
                       follow;ng poinlS. The vast majority of States we~ lalC in turning in their consolidated State
                       perf~ reports. mU:ing it impDSSib16 r.o analyze IIId ttmllllatizc the dauI eaniu. In
                       addition, as l'\Qfbd in the report, we asked 4() States \0 reVl$e their initial data submissions
                       because of prob1c:ms idenlified during the edit-check process. We ag,tte that Ol!SE has n(l(
                       had a systomatlc process in plact for following up wilh laic Stala. OESE will implement
                       addition.aJ ptOCCdures to ~re: such follow-up ""ilh (u!Ure submissions. In addition. OESE
                       plans 10 usc its Title rmomton and SUte contacts to motJror progmss on lCport s.ubtnlssicms
                       and work wilh Swes on data<l~lilyiC$OO& as put oftbe.monitoringproc.ess. In $0 doing,
                       not only $bould !he fepotU be SIIbmirted eulier. but lhere would be 1m follow·up needed
                       OOC-e the datl are $Ilbtnitted.


                                                 ,
                           O u r _ .. " ' _........... _
                                                                     --........
                                                     _~"'V2:,.4-W.• W~b.c._

                                                           .. _ _ .. _ _ 
        ~_~.,.._
                                                                                             





Attachment

                                                                12
Improving Title I Data Integrity for Schools
Identified for Improvement                                                               Control Number ED-OIG/A03-B0025




                       P£S is ~ ,Jos</y ooi ... OE.'lE .. "" ...:op;IIJI_' dfjcjo" follo"'""I' ~ fo<
                       Stolt< II" , lUI ... "'" ~ to 40<0~. I'« II>< _                   <e<:etII draf, tq>M (IWNQOO
                       " haol ,..., do<o~ OESE WIed ",... r&l $ ........1 .... ill II><edit4ot<k JI<OC"" and ... ,",.II..
                       ,..., r=.i"«l con""""do(.O __ q.oi<I<Iy. W~ O.<pea to poIIli'" .... 1'I99-2000 .. p""!his
                       .priIIJI. "oil _       of .... bst'~ roo'" ~. Tho ~.oI .. pl... "'I"".;dc
                       ttehni<.ol oui$I>nU '" Stales "" I~ It........ ttpOI'tio, f<lQUilM>tnt> of doe No
                       OIildlLfl M;,Id At<- We bel ..... 110, ~ willa;J><4j"' .... ooilcctiot1 ood ttpOI'ti"1
                       d Sac. dot> .. "",11 .. i"'f'JO"" i<s    "Jali',.
                  •    I'iaai"l ' I ,I .. >wu "'"' OESE and PF..s oe«I!O wmJ:lMn C<IanOI, '" ""''''' dOlO!)'
                       .. ..;.... on<! f"'bb<";oo of ""__ ~ <1> ... M idi", I~ """"<00I>C<IllUII .evi<wJ of
                       "'" fq>M by .... re'-;''';lItoff>eel . 11oi, ,"'-"" tnlaht be .,;«tiI<>CUd. W. 010 US<.
                       ,....... p«><eU _       i",,_ ,.....,_ ~.x ... by,,~ oIfl«>. The"';n pt<>bkat i,
                       11>< I~""" ond .... I... ,..pcooses 10 prot.I«n> id<odf>«l io .... «Ii, u...);;s.

                  • 1'Indi",1~ IIOWdlatW - ' " .. ~il>~f",......,,,,_Ti"'I_
                                                               .,11<1
                                                                  Tho r<p«t """""7 poinII. 001 <hat W', =r<nI
                    i""""van<a< d:so.o ... rcli./.ble and
                    _       of "'" <eho<>I imp-o_ .....              1._"""     ,t.o:h .. " ..... w ....
                    . - . chao I """" i<MkpIII PfOCI''' Ill!" """.-.. !eli1h!;,y"" _&1;01;<,. ID Iw "'"
                                                                                                             01...
                                                                                                              fo<...omaIia •

                    <OiI<c<ed """"","n"';'" !ton> s..... "" _ !IaI.l. .. st>dt I <Ollto:lioo ~ ""loln: 00-"",
                    d r.t.u_ &DC! &.It« __ -Ill... d;«:UIS<d m<bo o<porl. H,,,·,<va."                                "'Q:eSItII'"
                    .... o<porl. CFSI! does pl .. II:> build in """" OIIalti ... '" T otlo 1 d&t.o oubmi.1'd I.......
                    ptrlO<lt\U>« repon .. pall olltf;!llu monitori.. " .... '" Sta:<.. OES~ pl-'" ""......,.
                    "",nco ~ to ...... 1/1< "liIbili'y ond .-01"';'1 of _                     ~ d.ou &rie.l
                    ..... torint ""il> .
                       ............. " 'y "" I>op<-" i"'f'!"W"!he: qtWity d    Ft6enI ........ ... 011 ....1Iin'Wu - . on
                       sw.. i. by ;1IlI'I<"""'"I' p<rlorm~ 4oto ~ iortio<ivt. W• ...;l1 ..oot
                       ";111 S,.... to boild and J'UIIlo"' ...... . . - incla.:!inl ~ <Iou do.-. ..'I(!
                       <lob ........ w. e>1*' '" poIIIil.b "'" d:sIoqoality •.-41 .... n in Ih< CPRA _ obo to
                       eoog.l< 00 MoIc11 29,
                  •    Tho f<9<CI.oI'" ............ 0<1...... Pl'B id<ntij~ (ut III< ...........,- Kpo<I) "'" S .......... 1>ov<
                       ~~'J1I'<DIi'KCUI,.ur<,"'_~ooiD                                            ___d
                       a.. ~ i.    ~I MI!IIb<n. GiV«\ ...... " O«OUIlI>bilil)' """
                                               _
                       >$~ requ;_.lo !he:.'Io 0Ii1d Lri< 1ldIit>4 A<t. """, S ...... o.\l! t>: cI\oIIJIi1',!heir
                       ' ''''' mer'    'Y""'" .. -n .. \h<i,    def_   of &d<quaIo )Qo1y """""'- ~·.,.;U ""'"
                       <illlil\a;tll <ltaosu ;" <be .. ...,.. '" "'" ~, "ill 110-.. """" <"""",,"            _oioo ..
                                       tho..-
                                   -
                       ~ ....x....                 ;uOPOO,CJD<1I1 <Iou.

                  «0

                            ....
                           Susan~

                           '_'w
                           ... loot Gi",bw'1

                           noKio Walbop




                                                               13

Improving Title I Data Integrity for Schools
Identified for Improvement                                Control Number ED-OIG/A03-B0025

                            REPORT DISTRIBUTION LIST

                          CONTROL NUMBER ED-OIG/A03-B0025




Auditee

Susan B. Neuman
Assistant Secretary
Office of Elementary and Secondary Education

Eugene W. Hickok
Under Secretary
Office of the Under Secretary


Other ED Officials (electronic copy)

Audit Liaison Officer, Office of Elementary and Secondary Education
Assistant General Counsel, Office of General Counsel
Assistant Secretary, Office of Legislation and Congressional Affa irs
Assistant Secretary, Office Intergovernmental and Interagency Affairs
Director, Financial Improvement and Post Audit Operations, OCFO
Director, Compensatory Education Program, Office of Elementary and Secondary
Education
Director, Budget Service, Office of the Under Secretary Post Audit Group Supervisor
Deputy Secretary, Office of the Deputy Secretary
Chief of Staff, Office of the Secretary
Press Secretary, Office of Public Affairs
Director, Office of Public Affairs
Supervisor, Post Audit Group, Office of the Chief Financial Officer
Headquarters and Regional Audit Managers
General Operations Team




                                               14