oversight

Audit of Michigan Department of Education Management Controls over IDEA, Part B - Special Education Performance Data.

Published by the Department of Education, Office of Inspector General on 2001-09-21.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

         Michigan Department of Education
             Management Controls Over
  IDEA, Part B - Special Education Performance Data

                                   FINAL AUDIT REPORT




                              Control Number ED-OIG/A05-A0031
                                        September 2001



Our mission is to promote the efficiency,                  U.S. Department of Education
effectiveness, and integrity of the                        Office of Inspector General
Department’s programs and operations.                      Chicago, Illinois
                                             NOTICE

Statements that management practices need improvement as well as other conclusions and
conclusions and recommendations in this report, represent the opinions of the Office of Inspector
General. Determination of corrective action to be taken will be made by appropriate Department
of Education officials.

In accordance with the Freedom of Information Act (5 U.S.C. §552), reports issued by the Office
of Inspector General are available, if requested, to members of the press and general public to the
extent information contained therein is not subject to exemptions in the Act.
                                       Table of Contents


Executive Summary......................................................................................1

Audit Results..................................................................................................4
Finding No. 1 – MDE’s Management Controls Over its Electronic Data
                Collection and Reporting Process for IDEA, Part B
                Performance Data Are Inadequate and Result in Inaccurate
                Reports ...................................................................................6

Finding No. 2 – MDE Lacks Controls Over Data Input and Retention of
                Documents for Manually Submitted Discipline Data and
                Supplemental Performance Data ........................................ 14

Finding No. 3 – MDE’s Ability to Provide Reliable Data is Affected by
                Inconsistent Local Educational Agencies and Intermediate
                School District Data Systems............................................... 17

Other Matters ............................................................................................. 20

Background................................................................................................. 22

Purpose, Scope, and Methodology.......................................................... 24

Statement on Management Controls ..................................................... 28

Attachment A – IDEA, Part B Program Objectives, Performance
               Indicators and Performance Data ................................... 29

Attachment B – MDE’s Comments to the Report............................... 30
                                 Executive Summary


The Michigan Department of Education (MDE) should take additional steps to improve
management controls over the collection and reporting of the Individuals with Disabilities
Education Act (IDEA), Part B performance data provided to the U.S. Department of Education
(Department). The integrity of IDEA, Part B state-reported data is of particular importance
because the Department relies on it to provide to Congress an objective and accurate measure of
the success of its special education programs, as required under the Government Performance
and Results Act (GPRA) of 1993. The Office of Special Education Programs (OSEP) within the
Department’s Office of Special Education and Rehabilitative Services (OSERS) administers
programs funded under IDEA, Part B. OSEP uses performance data reported by state
educational agencies in preparing the Department’s report to Congress on the outcomes of the
IDEA, Part B programs.

For reporting outcomes under the Department’s 2001 Annual Plan, OSEP uses state educational
agencies’ performance data for the following performance indicators:

   •     Earlier identification and intervention (intervention)
   •     Inclusive settings/Regular education settings (placement)
   •     Graduation (exiting)
   •     Suspensions or expulsions (discipline)
   •     Qualified personnel (personnel)
MDE is required by IDEA, Part B to submit this performance data to the Department.
Attachment A to this report shows the relationship between the IDEA, Part B program
objectives, performance indicators, and performance data.

Performance Indicator 4.7.c of the Department’s 2001 Strategic Plan states that all departmental
program managers will assert that the data used for their program’s performance measurement is
reliable, valid and timely, or will have plans for improvements. Annually, Assistant Secretaries
must provide the Office of the Under Secretary with a signed formal attestation covering their
data.

Our review of procedures and available documentation at MDE, two intermediate school districts
(ISD), and two local educational agencies (LEA) identified weaknesses in MDE’s management
controls covering performance data for intervention, placement, exiting, personnel, and
discipline for the 1998-99 school year. MDE lacked adequate management controls to ensure
that the data provided to the Department is reliable, valid, and timely.

In order to ensure that MDE provides reliable, valid, and timely data that Department managers
can attest to, MDE needs to:




ED-OIG                           Control Number ED-OIG/A05-A0031                        Page 1
n        strengthen management controls over its IDEA, Part B electronic data collection and
         reporting process,
n        implement data input controls and document retention controls for manually received
         data, and
n        require ISDs and LEAs to use consistent data systems.

MDE’s statewide databases for the intervention, placement, exiting, personnel, and discipline
included numerous errors and irregularities, such as duplicate children, reported counts and data
not reconcilable to supporting databases, inconsistent count data between reports, incomplete
data items, incomplete databases, and data items with undefined codes.

MDE needs stronger management controls over its IDEA, Part B data. Stronger management
controls are also needed at the LEA and ISD levels. Specifically, we found that MDE lacked
edit checks of performance data; an independent review function for its IDEA, Part B data;
written policies and procedures for its IDEA, Part B data collection and reporting process; a
contingency plan for this process; and effective use of its audits of the December 1 count. MDE
also needs to implement controls to ensure that all discipline data is obtained, source documents
for supplemental data are retained, and data entry of discipline data and supplemental data is
verified for accuracy. MDE’s ability to provide reliable, valid, and timely data is also affected
by the LEAs and ISDs use of inconsistent and various data systems. Without a strong
management control environment, MDE’s management cannot ensure that its IDEA, Part B data
is reliable, valid, and timely.

Michigan has 57 ISDs and 4 state agencies that report performance data to MDE for over 700
LEAs. Our audit was limited to reviews of procedures and documents at MDE, 2 ISDs, and 2
LEAs. Since the procedures used by other ISDs and LEAs may vary from those covered by our
review, our audit would not necessarily disclose all material weaknesses in the management
controls related to the reporting of performance data. We concluded that the identified
management control weaknesses disclosed in the report may be systemic in nature rather than
limited to the particular ISD or LEA.

MDE has already taken steps to address some of the identified weaknesses. We recommend that
the Assistant Secretary for Special Education and Rehabilitative Services request MDE to take
additional action to address the remaining identified weaknesses in the management controls
over reported performance data. The Audit Results section of the report describes the corrective
actions taken by MDE and our specific recommendations for each of the findings.

The Other Matters section of the report discloses that the annual individualized education
program (IEP) and triennial assessment dates for children were not within the required time
frames. Federal regulations require that a child’s IEP should be reviewed at least annually and
that an assessment should be conducted at least once every three years.




ED-OIG                         Control Number ED-OIG/A05-A0031                           Page 2
MDE concurred with our findings. In its response to the draft report, MDE described features of
a new information system and related procedures that address the recommended corrective
actions. The full text of MDE’s comments is included as Attachment B.




ED-OIG                        Control Number ED-OIG/A05-A0031                           Page 3
                                             Audit Results

MDE’s management controls did not ensure that the data submitted by MDE to the Department
met all of the standards in the Department’s Data Quality Standards.1 The Data Quality
Standards contain six standards for evaluating the quality of reported data. The six standards
are:

    •    Validity – Data adequately represent performance.
    •    Accurate Description – Definitions and counts are correct.
    •    Editing – Data is clean.
    •    Calculation – The math is right.
    •    Timeliness – Data is recent.
    •    Reporting – Full disclosure is made.

For each of the Data Quality Standards, the Department provided examples of conditions that
meet or fail to meet the standard. The Department also provided Data Quality Checklists for use
by primary data providers and secondary data managers. For school year 1998-99, MDE
management controls over the collection and reporting of performance data for intervention,
placement, exiting, personnel, and discipline did not meet all elements contained in the Data
Quality Standards for accurate description, editing, and reporting. In addition, MDE’s
management controls also did not meet several of the control activities of the Standards for
Internal Control in the Federal Government.

MDE used two data collection processes to collect the IDEA, Part B performance data that it
reported to OSEP for school year 1998-99. For intervention, placement, exiting, and personnel
data MDE used LEA created electronic data files of computerized counts for active children,
exited children, and personnel that ISDs combined and submitted to MDE. MDE then compiled
statewide databases for active children, exited children, and personnel that it used to report
performance data to the Department. MDE also received hardcopy documents for supplemental
data revisions to the electronically received data. For discipline data, MDE used manually
prepared hardcopy documents submitted by LEAs to create a statewide database.




1
 The Department issued these standards, as part of the 1999 Performance Report and 2001 Annual Plan, to assist
Department managers as they collect, analyze, and report data about Federal education programs. Program
managers can use the standards as a tool when monitoring grantees and evaluating the quality of the reported data
and preparing submissions for the GPRA annual report. The standards are the Department’s attempt to provide
criteria against which to evaluate grantees’ data quality.

ED-OIG                               Control Number ED-OIG/A05-A0031                                       Page 4
Intervention, placement, exiting, and personnel. The active child database contained student-
level information on each child receiving special education services. The active child database
contained information as of the December 1st child count and was used to report intervention and
placement data. The exited database contained information on children who have exited special
education within the last twelve months and was used to report exiting data. The personnel
database contained information on special education personnel currently employed and those
who had left within the last twelve months prior to December 1st and was used to report
personnel data.

MDE instructed the LEAs and ISDs to check the count data for errors using the Registry
Management System (RMS) Validation Report or the Error Check Program supplied by MDE.
MDE also instructed the LEAs and ISDs to provide signed certifications along with their data
submissions indicating that the data had been checked to ensure accuracy.

Discipline. For the 1998-1999 and 1999-2000 school years, MDE had LEAs submit discipline
data manually via individual hardcopy suspension forms that were completed and submitted for
each special education child with a suspension or discipline incident. The LEAs completed and
submitted the suspension forms to the ISDs. The ISDs then submitted the suspension forms to
MDE. MDE entered the discipline data from the individual suspension forms into a computer
database to create a discipline database that it used to report discipline data to the Department.

Our review disclosed that MDE needs stronger management controls over its IDEA, Part B data.
Stronger management controls are also needed at the LEA and ISD levels. Specifically, MDE
lacked edit checks of performance data; an independent review function for its IDEA, Part B
data; written policies and procedures for its IDEA, Part B data collections and reporting process;
a contingency plan for this process; and effective use of its audits of the December 1 count.
MDE also needs to implement controls to ensure that all discipline data is obtained, source
documents for supplemental data are retained, and data entry of discipline data and supplemental
data are verified for accuracy. MDE’s ability to provide reliable data is also affected by the
LEAs and ISDs use of inconsistent and various data systems. Without a strong management
control environment, MDE’s management cannot ensure that its IDEA, Part B data is accurate
and reliable.

In order to ensure that data is reliable, valid and timely, MDE needs to (1) strengthen
management controls over its IDEA, Part B electronic data collection and reporting process; (2)
implement data input controls and document retention controls for manually received data; and
(3) require ISDs and LEAs to use consistent data systems.

MDE concurred with our findings. In its response to the draft report, MDE described features of
a new information system and related procedures that address the recommended corrective
actions. The full text of MDE’s comments is included as Attachment B.

ED-OIG                         Control Number ED-OIG/A05-A0031                            Page 5
Finding No. 1 – MDE’s Management Controls Over its Electronic Data
              Collection and Reporting Process for IDEA, Part B
              Performance Data Are Inadequate and Result in Inaccurate
              Reports

MDE’s management controls over its electronic data collection process are inadequate to ensure
that performance data reported to the Department is accurate. The MDE IDEA, Part B
performance data reported for the 1998/1999 school year was not accurate or reconcilable to
MDE’s statewide database. One individual at MDE is responsible for collecting the data from
the ISDs. This individual compiles, categorizes, and reports the data with limited supervision
and no review of his work. Specifically, MDE lacked edit checks to validate the data as it was
received and after it was compiled, an independent review function related to the data it reports,
written procedures, and a contingency plan. Several of these same management controls were
also lacking at the LEA and ISD levels. In addition, MDE needs to implement procedures to
ensure that data received correlates to the Federal reporting categories and that the results of its
audits are used to correct systemic problems. Overall, MDE needs to strengthen the controls
over its entire electronic IDEA, Part B data collection and reporting activities to ensure the
reporting of accurate and valid performance data.

Need for Edit Checks Of Performance Data

MDE instructed the LEAs and ISDs to run the RMS Validation Report or Error Check Program
before submitting their data to MDE and provide a certification indicating that the data has been
checked to ensure accuracy. However, MDE did not check to ensure that the LEAs and ISDs
performed the edit checks or that it received the completed certifications.

MDE’s data collection and reporting official confirmed that MDE does not perform any edit
checks of the ISD submitted performance data before or after the data is compiled into the
statewide databases. The MDE official indicated that he looks for missing data. However, he
said it is difficult because he does not have the specific software or technical programming skills
needed to perform sophisticated edit checks of the entire statewide database for duplicates or
other more specific edits. Our analysis of MDE’s statewide database for the December 1, 1998
count, the source for the intervention and placement data reports, found that the count included at
least 891 apparent duplicate children where the first name, last name, and date of birth all
matched. The actual number of duplicates is probably higher because our analysis only counted
exact matches. Some of these duplicates were within the same ISD and some were among
different ISDs.



ED-OIG                          Control Number ED-OIG/A05-A0031                             Page 6
MDE did not obtain certifications from 11 ISDs and 9 ISDs for the December 1, 1998 and
December 1, 1999 counts, respectively. In addition, the certifications for 4 ISDs and 7 ISDs for
the December 1, 1998 and December 1, 1999 counts, respectively, were incomplete. The ISD
certifications are significant because MDE does not perform edit checks of the ISD data.

The two ISDs we reviewed ran validity edit checks of the individual LEAs data before merging
the data into a combined database. The Genesee ISD ran additional edits of its combined
database to eliminate duplicate child counts within the ISD. The Oakland ISD did not. Because
the Oakland ISD does not check for duplicates within its combined database, duplicates
involving a child that moves to another LEA within the ISD will not be detected and eliminated.
Of the 891 apparent duplicates, 241 pertained to the Oakland ISD and 26 to the Genesee ISD.
For Oakland, 157 were the same child counted by multiple LEAs within the ISD. The remaining
84 were the same child counted by an Oakland LEA and by another LEA. For Genesee, 1 child
was counted by 2 LEAs within the ISD and 25 were the same child counted by a Genesee LEA
and by another LEA. Both ISDs ran one of the MDE validation programs against the combined
database before submitting the data to MDE.

We also found that the two reviewed ISDs viewed their role differently. The Genesee ISD
exercised more control over its LEAs and ran additional edit checks. The Oakland ISD gave its
LEAs a lot of autonomy. An official from the Oakland ISD commented that there was no clear
description of the ISD’s role with regard to the special education data reporting. The Oakland
ISD viewed its role mostly as a pass through for the data reported. Reviews and audits by MDE
have also identified concerns related to data verification by LEAs and ISDs. In response to a
recent Michigan Office of the Auditor General performance review, MDE officials commented
that some ISDs merely accumulated the December 1 count data from their constituent LEAs and
submitted the data to MDE, with little or no data verification to ensure the accuracy of counts
prior to submission. MDE’s audits of the December 1 count have found that ISDs that perform
little or no verification or have ineffective verification procedures usually have more errors in
their reported count.

The Department s publication, Standards for Evaluating the Quality of Program Performance
Data, lists six Data Quality Standards. For each of the Data Quality Standards, the publication
provides examples of conditions that meet or fail to meet the standard. The publication also
provides a Data Quality Checklist for use by primary data providers and secondary data
managers. Standard Two – Accurate Description states that definitions and counts are correct.
An example of failing this standard is disabled students that are double counted or undercounted.
Standard Three – Editing states that data is clean. The Data Quality Checklist for this standard
provides that an electronic edit checking program has been used to clean the data.




ED-OIG                         Control Number ED-OIG/A05-A0031                           Page 7
Without adequate edit checks and complete certifications, MDE lacks assurance that the
statewide database is accurate prior to preparing and submitting the performance reports to the
Department.


Need for Independent Review of Data Submitted to the Department

MDE relied on one individual who has sole responsibility for collecting, compiling, and
reporting its IDEA, Part B performance data. This individual stated that, other than himself, no
one reviewed the MDE IDEA, Part B data reported to the Department. As a result, there was no
independent review of the data submitted to the Department to ensure its accuracy and validity.
Our attempt to reconcile the reported counts to the supporting statewide databases found errors in
the reported intervention, placement, exiting, and discipline data and instances where the
reported placement and personnel data were not reconcilable to the statewide databases.

The errors and unreconcilable data are indications that MDE’s data does not meet two of the
Data Quality Standards. Standard Two – Accurate Description states that definitions and counts
are correct and Standard Three – Editing provides that a different person, who is familiar with
the data, systematically reviews the data. In addition, the Standards for Internal Control in the
Federal Government2 provides that management perform reviews at each functional or activity
level.

Errors in Reported Data. MDE reported children of one LEA with blank ethnic codes in the
database as white. In another example of a data error, intervention counts reported for four
disability categories did not agree with the supporting database. The reported placement count
data, (Table #3, Section B), for four disability categories was also not consistent with the
reported intervention count data (Table #1, Section C).

MDE also collected exit data using 14 exit reason codes that are different than the 8 Federal
codes used for reporting exit data to the Department. MDE then reclassified its 14 codes into the
8 Federal codes. MDE’s reclassification resulted in the counts for two of its codes
inappropriately reported in the counts for two of the Federal codes.

Unreconcilable Data. We could not reconcile the MDE reported personnel counts to the
personnel database. It appears that MDE categorized the personnel count data prior to reporting
it to the Department. The supporting personnel database included personnel with employment
status codes indicating that their employment with special education had terminated. We were
unable to determine whether these personnel were included in the personnel count that MDE
reported to the Department. The database also included 3,586 records with a “00" value for the

2
 The General Accounting Office issued the Standards for Internal Control in the Federal Government in November
1999, hereinafter referred to as the “Federal Internal Control Standards.”

ED-OIG                             Control Number ED-OIG/A05-A0031                                    Page 8
Employment Status code and 343 records with “0" for Level of Educational Assignment code.
MDE’s technical manual for special education student and personnel data counts does not list
“00” or “0” as valid codes for these two categories. We also noted that there are 213 records that
are coded as active teachers, but have names of either “Vacant” or “Vacancy.”

MDE did not collect placement data that correlated to the placement categories required for
Federal reporting. Instead, MDE used a very complicated formula that used 8 different variables
and 53 decision points to calculate the educational placement category for each of the students
reported. As a result, we could not reconcile the reported placement data to MDE’s supporting
placement database files.

These reporting errors and unreconcilable differences between the statewide database and the
data reported to the Department indicate the need for independent review of the data MDE
submits to the Department. Without an independent review function at MDE and at the LEA
data input level, MDE management and the Department users do not have reasonable assurance
of the reliability of its IDEA, Part B data reported by MDE.

Need for Written Data Collection and Reporting Policies and Procedures

MDE did not have a written description of the process it used to collect and compile the
statewide databases that were the sources for the data it reported to the Department for
intervention, personnel, placement, exiting, and discipline. The two ISDs and LEAs we
reviewed did not have complete written descriptions of the processes they used to collect and
compile the data that they submit to the ISDs and MDE.

Standard Six – Reporting of the Department’s Data Quality Standards ensures that full
disclosure is made. One example of meeting the standard is that data collection processes are
documented. The Federal Internal Control Standards provides that all transactions and other
significant events be clearly documented and readily available for examination. The standards
require that this documentation should appear in management directives, administrative policies,
or operating manuals. Furthermore, the standards state that transactions should be promptly
recorded to maintain their relevance and value to management in controlling operations and
making decisions.

MDE management will not have an understanding of the controls needed and reasonable
assurance of data reliability, until it documents the policies and procedures for the entire data
collection and reporting process. In addition, without complete written descriptions of the
LEAs’ and ISDs’ procedures used to collect and compile data, there is little assurance that the
LEAs and ISDs are providing MDE with reliable data.



ED-OIG                         Control Number ED-OIG/A05-A0031                            Page 9
Need for Contingency Planning

MDE did not have a written contingency plan for the IDEA, Part B data collection and reporting
function. MDE officials agreed that a contingency plan is needed because the entire function is
currently handled by one individual with no independent review of this person’s work.

The Federal Internal Control Standards provide that management establish a positive control
environment in which, among other things, it plans and ensures continuity of needed skills and
abilities, and ensures that data center and client-server operation controls include contingency
and disaster planning. This would include data backup and recovery procedures.

The absence of a written contingency plan, combined with the absence of written operating
policies and procedures for the data collection and reporting function increases the risk that MDE
will not be able to report reliable, valid, and timely data.

Need for More Effective Use of the Audits of the December 1 Count

MDE has not used the results of its audits to correct identified systemic problems. Our review
and the Michigan Office of the Auditor General’s performance review of MDE’s audit process
found that MDE does not use a risk-based approach when selecting ISDs for on-site audits of the
December 1 count. MDE conducts an on-site audit of the December 1 count at each ISD every
three years regardless of past on-site audit results or the effectiveness of the ISDs’ December 1
count process. The Michigan Office of the Auditor General recommended that MDE should
focus its limited audit efforts on those ISDs that have a greater risk of reporting inaccurate
December 1 count data.

MDE conducted audits of the December 1, 1998 count for 17 ISDs and 1 state agency. MDE
generated an exception report only for those ISDs it scheduled for audits for that year. The
exception report identified children included in the December 1 count with IEPs that were older
than 12 months as of the December 1 count. MDE used the exception report to select a portion
of the children to review. The audits reduced the reported December 1, 1998 counts reported for
13 of the 17 ISDs and the 1 state agency. MDE reduced the December 1, 1998 count for 3 ISDs
by 246 children, based on its review of 2,179 children. The 3 ISDs were Genesee ISD, 23
children; Oakland ISD, 71 children; and Wayne County ISD, 152 children. MDE found these
children ineligible for the count because 107 had exited, 1 had an IEP that indicated the child
was ineligible, 83 had IEPs that were older than 12 months of the December 1 count, 3 52 had no
file, and 3 had no IEP in their file. MDE does not project the results to the entire December 1


3
  MDE was not required by the IDEA, Part B to exclude children with disabilities who were receiving services from
its December 1, 1998 count even if their IEPs were not current.

ED-OIG                              Control Number ED-OIG/A05-A0031                                      Page 10
count or expand the review to previous December 1 counts when, the current audit identifies a
significant number of exceptions.

Our analysis found that the statewide database included 6,179 children whose IEPs were older
than 12 months at the December 1, 1998 count date. MDE’s audits of the December 1 count,
found that a significant number of the children with out-of-date IEPs had exited special
education but were not excluded from the intervention count.

MDE could generate this exception report for all ISDs and require that the ISDs determine the
propriety of children identified on the report within a specified time frame. MDE could also use
the results as part of a risk-based approach in selecting ISDs for audits.

We found that the reports issued to the ISDs on the results of MDE’s audits of the December 1
count did not include recommendations to:
                       Eliminate or remedy the procedural or management weaknesses.
                       Require ISDs to conduct reviews of the entire child count data.
                       Impose administrative consequences or penalty.
A recent Michigan Office of the Auditor General performance review of MDE confirmed these
weaknesses in MDE’s audit process.

We also noted that the number of children reviewed in the December 1, 1998 count audits for the
Oakland, Genesee, and Wayne County ISDs was less than the 5 percent sample specified in
MDE’s December 1 count audit procedures.

The scope of MDE’s audits are limited to children included in the December 1 count and the
verification of selected data items for accuracy and to determine if the child is eligible to be
included in the count. MDE’s audits do not cover personnel, placement, exiting, and discipline
data. Because of the limited scope, other inaccurate data such as those identified in our
verification review will not be detected.

Verification of MDE’s Databases to the LEAs’ Records

We compared selected data for samples from MDE’s intervention and placement, exited
children, and personnel databases to supporting records at two LEAs. We identified high levels
of inconsistency between the supporting records and each of the databases.               These
inconsistencies indicate that MDE needs to strengthen both its management controls and those at
the LEAs/ISDs over the IDEA, Part B electronic data collection and reporting process.

For the December 1, 1998 child count intervention and placement database, the MDE database
information for 19 of the 61 children reviewed was inconsistent with the LEAs’ records. The
inconsistencies for these 19 children totaled 27 and included the following: (a) 10 incorrect
ED-OIG                         Control Number ED-OIG/A05-A0031                           Page 11
special education F.T.E. percentages, (b) 8 incorrect program service codes, (c) 3 incorrect ethnic
group codes, (d) 3 incorrect IEP dates, (e) 1 incorrect support service code, and (f) 2 with no
services provided.

For the exited children database, children exiting between December 1, 1997 to December 1,
1998, the MDE database information for 19 of the 30 exited children reviewed did not agree with
the LEAs’ records or was not supported by the LEAs’ records. The inconsistencies for these 19
children totaled 29 and included the following: (a) 6 incorrect exit dates, (b) 6 incorrect special
education full time equivalent percentages (c) 3 incorrect ethnic group codes, (d) 3 incorrect
program service codes, (e) 2 incorrect support service codes, (f) 2 incorrect exit reasons, (g) 3
incorrect IEP dates, (h) 1 incorrect primary educational setting, (i) 1 incorrect primary handicap
condition; (j) 1 incorrect birth date, and (k) 1 missing file.

For the personnel database, special education personnel employed on December 1, 1998, the
MDE database information for 7 of the 21 personnel reviewed was either incomplete or did not
agree with the LEAs’ records. The inconsistencies for these 7 personnel totaled 9 and included
the following: (a) 5 missing birth dates and age, (b) 2 incorrect ethnic group codes, (c) 1
incorrect birth date, and (d) 1 incorrect approval status.

These inconsistencies are a result of the weaknesses in MDE’s management controls discussed in
the previous sections and weaknesses in the LEAs’ management controls. At the two LEAs
reviewed, there were no reviews of data input to ensure the accuracy of the LEAs’ databases.

Recommendations

We recommend that the Assistant Secretary for Special Education and Rehabilitative Services:

1.1      Request MDE to develop and perform edit checks of the performance data received from
         ISDs and the statewide database to eliminate reported duplicate children, ineligible
         children, and invalid data.

1.2      Request MDE to develop written detailed policies and procedures for the entire data
         collection and reporting function that includes independent reviews, backup personnel,
         and a contingency plan.

1.3      Request MDE to implement procedures to ensure that ISDs and LEAs perform error
         checks of their data prior to submission and ensure that Data Submission Certifications
         are obtained from ISDs certifying that the required error checks have been performed.

1.4      Request MDE to use the Federal placement categories instead of calculating placement
         categories.

ED-OIG                          Control Number ED-OIG/A05-A0031                            Page 12
1.5      Request MDE to use the 8 Federal exit reason codes instead of the 14 MDE exit reason
         codes used to report exit statistics.

1.6      Request MDE to implement procedures for a review by a second person to confirm that
         data fields related to the OSEP reporting form are properly recorded in the MDE database
         and the Federal guidelines for reporting placement and exited students are followed.

1.7      Request MDE to use audits to identify and correct systemic problems; mandate recounts
         when error rates exceed specified limits; and implement risk based targeting of reviews.




ED-OIG                          Control Number ED-OIG/A05-A0031                           Page 13
Finding No. 2 – MDE Lacks Controls Over Data Input and Retention of
              Documents for Manually Submitted Discipline Data and
              Supplemental Performance Data

The discipline data MDE compiled and reported to the Department for the 1998/1999 and
1999/2000 school years was incomplete and inaccurate. This occurred because MDE lacks
controls to ensure that all discipline data received via hard copy documents was collected and
entered accurately. MDE also lacked controls over the input of supplemental revisions to the
performance data received via hardcopy documents to ensure that data was entered accurately
and the source documents were retained. As a result, MDE lacks support for the supplemental
data revisions. MDE needs to implement controls to ensure that all discipline data is obtained,
source documents for supplemental data are retained, and data entry of discipline data and
supplemental data is verified for accuracy.

Need for Discipline Data Collection and Data Input Controls

The discipline data that MDE reported to the Department was gathered manually via individual
forms (SE-4568 Suspension forms) that were completed and submitted for each special
education student with a suspension or discipline incident. The LEAs completed and submitted
the suspension forms to the ISDs. The ISDs then submitted the suspension forms to MDE.
MDE entered the discipline data from the individual suspension forms into a computer database
to create a discipline database that supported the discipline data that it reported to the
Department.

Incomplete Data. MDE did not collect suspension forms from all the LEAs for the 1998/1999
school year. As a result, the discipline data MDE reported was incomplete. MDE did not
disclose this data limitation to the Department. Our review of the discipline data for the 2 ISDs
reviewed found that 7 of the 21 LEAs within the Genesee ISD and 20 of the 28 LEAs within the
Oakland ISD did not submit discipline data. Flint, the largest LEA within the Genesee ISD, was
1 of the 7 LEAs that did not submit any discipline data for the 1998/1999 school year. Standard
Two – Accurate Description of the Department’s Data Quality Standards specifies that all
instances are counted, and no instances are omitted. The number of LEAs reporting discipline
data for the Genesee ISD increased to 18 of 21 LEAs for the 1999/2000 school year. The
number of LEAs reporting discipline data for the Oakland ISD increased to 16 of 28 LEAs for
the 1999/2000 school year.

Inaccurate Data. MDE’s discipline database included inaccurate data. Our review identified
two reasons for the inaccurate discipline data: (a) the LEAs did not complete the suspension
forms accurately, and (b) MDE did not enter the data from the suspension forms into the

ED-OIG                         Control Number ED-OIG/A05-A0031                           Page 14
database accurately. Our review found that MDE did not have procedures for independent
verification of the discipline data entered to ensure accuracy. Our review identified instances
where the information in MDE’s discipline database was incorrect. These instances indicate
MDE’s need for data input controls to ensure the accuracy of inputted discipline data. We
compared selected information on the MDE’s discipline database for 10 Pontiac LEA students
for the 1998/1999 school year with the records at the Pontiac LEA. We found that the database
information for all 10 children reviewed was inconsistent with the LEA’s records and the
information recorded on the suspension forms. The data recorded on the suspension forms was
also inconsistent with the LEA records. These inconsistencies included (a) 1 wrong primary
handicap code, (b) 10 wrong ethnic codes, and (c) 1 different spelling of last name. Pontiac also
did not disclose the total number of suspension acts for any of the 10 students on the individual
suspension forms.

When MDE received the suspension forms, it incorrectly recorded the discipline data. These
instances included: (a) 10 children suspended for drugs, (b) 4 children suspended due to a
weapon, and (c) 1 child suspended for consecutive days of 10 or more.

MDE also did not record data included on the suspension forms. These instances included: (a) 1
child suspended due to a weapon, (b) 5 children suspended due to a hearing officer’s
determination, (c) 4 children suspended for consecutive days of 10 or more, and (d) 5 children
suspended for cumulative days that totaled 10 or more days.

For the Flint LEA, we compared the information recorded on the suspension forms for 10
children for the 1999/2000 school year to the records at the LEA because the Flint LEA did not
report any discipline data for the 1998/1999 school. The ethnic group recorded on the
suspension form and the total number of suspension acts on the suspension form were incorrect
for two children.

The above instances of inaccurate entry of discipline data into the discipline database indicate
that MDE’s management controls do not meet Standard Three – Editing of the Department’s
Data Quality Standards that specifies the data is correct, internally consistent, and without
mistakes.

Without adequate data input controls to ensure the accuracy of discipline data entered into the
statewide database, data errors and incomplete data will go undetected, and as such, MDE
management and the Department do not have reasonable assurance that the data is accurate.

Need for Retention of Supplemental Data Supporting Documents

MDE electronically received the IDEA, Part B intervention, personnel, placement, and exiting
data from the ISDs and the LEAs. Subsequent to the original electronic submissions, ISDs and

ED-OIG                         Control Number ED-OIG/A05-A0031                           Page 15
LEAs provided MDE with revisions to their performance data via hard copy documents. MDE
revised the ISDs and LEAs performance data based on these hard copy submissions. Our
review found that MDE did not retain the hard copy documents for the revisions that four ISDs
and six ISDs submitted for the December 1, 1998 and December 1, 1999 counts, respectively.
Federal regulations require that programmatic documentation be retained for three years. Title
34 Code of Federal Regulations (CFR) §80.42(b) states, “...records must be retained for three
years from the starting date specified in paragraph (c) of this section.” Paragraph (c) states “the
retention period for the records of each funding period starts on the day the grantee or subgrantee
submits to the awarding agency its single or last expenditure report for that period.” Without
supporting documents for the supplemental data, MDE staff, auditors, or other reviewers cannot
evaluate the accuracy of the revisions.

Recommendations

We recommend that the Assistant Secretary for Special Education and Rehabilitative Services:

2.1      Request MDE to establish and implement data entry edits to systematically review the
         IDEA, Part B performance data entered from the supporting source documents to ensure
         the data is accurate.

2.2      Request MDE to establish and implement controls to ensure that the ISDs and LEAs
         submit accurate discipline data.

2.3      Require MDE to establish and implement a record retention policy for all IDEA, Part B
         performance data documentation.




ED-OIG                          Control Number ED-OIG/A05-A0031                            Page 16
Finding No. 3 – MDE’s Ability to Provide Reliable Data is Affected by
              Inconsistent Local Educational Agencies and Intermediate
              School District Data Systems

More than 700 LEAs reported their special education performance data to 57 ISDs. The ISDs
merged the LEAs data and electronically submitted the LEAs combined data to MDE. MDE
then combined the ISDs data into statewide databases. The databases were used to report
performance data to the Department. The LEAs and ISDs used a variety of computer data
systems to compile the computerized student and personnel counts that MDE reports to the
Department. The use of various data systems has affected the consistency and reliability of the
reported student and personnel count data. A single consistent data system for recording and
reporting special education data would help to ensure consistent and accurate data.

Several of the larger LEAs and ISDs did not use the computer data system MDE provided to
record and report their special education data. The LEAs used various data systems to compile
the computerized special education student and personnel counts that MDE reported to the
Department. In 1984, MDE had a contractor develop a computer system called Registry
Management System for the LEAs and ISDs to record, manage, and report their special
education data. MDE made the RMS available to all LEAs and ISDs. There were three versions
of the RMS in use, a DOS version, a Windows version called WinRMS, and a RMS-2000
version. A majority of the LEAs and ISDs used one of the three versions of the RMS data
system. However, 4 of the 57 ISDs and at least 31 of the LEAs used other data systems.

We found that LEAs within the same ISD used different versions of the RMS. The Genesee ISD
has 1 LEA that used the DOS version, 4 LEAs that used the Windows version (WinRMS), and
16 LEAs that used the RMS-2000 version. None of the Oakland ISD’s 28 LEAs used the RMS
but instead used 10 different data systems. Oakland ISD officials commented that it was difficult
to provide technical support to its LEAs because the LEAs used many different systems to record
and report their special education data. Officials at these two ISDs also commented that MDE
has provided very limited guidance and technical support.

The LEAs and ISDs that used a computer data system other than the RMS, to record, collect, and
report special education data increased the risk of reporting inconsistent and inaccurate data. It
also made it difficult for MDE to provide adequate technical assistance and oversight. Accurate
and consistent data are two elements covered in the Department’s Data Quality Standards for
evaluating the quality of reported data. Standard Two – Accurate Description ensures that data
definitions and counts are correct and Standard Three – Editing ensures that data is correct,
internally consistent, and without mistakes.


ED-OIG                         Control Number ED-OIG/A05-A0031                            Page 17
Annually, MDE issued instructions to LEAs and ISDs to complete the December 1 count for
special education active students, exited students, and personnel. MDE’s instructions specified
that the count data should be computerized and that the data must be submitted to the MDE via
the ISD on computer disk or electronically over the Internet. The instructions reference the
Technical Manual for the Special Education Student and Personnel Data Counts that outlines the
requirements for data format and data transmission. The technical manual included a record
layout with field specifications, acceptable codes and definitions for each field that is required
data for each student and personnel included in the count. MDE’s instructions specified that
LEAs generating the computerized count were required to use the MDE supplied RMS
Validation Report or the Error Check Program to check the data for errors.

We found the following effects as a result of LEAs and ISDs using various data systems:
      (a) The data MDE received from 10 ISDs had problems with the alignment of characters
          within the records for the 1998/1999 and the 1999/2000 school years.
      (b) LEAs experienced problems with implementation and use of the RMS-2000 version
          of RMS and attributed the problems to the development of the RMS-2000 version, on
          a MAC platform rather than on a Windows PC based platform that the LEAs use.
      (c) LEA and ISD officials indicated that technical assistance provided is difficult to
          follow and not adequate because the training and technical support provided is
          directed to those LEAs and ISDs that use the RMS-2000 version and not the other
          RMS versions.
      (d) The Pontiac LEA’s data system used different ethnic code definitions than the ethnic
          code definitions MDE specified in the record layouts for the student and personnel
          count data files that were submitted to MDE. This LEA’s use of different ethnic code
          definitions resulted in incorrect data.

MDE was aware that the technical support provided by the developer of the RMS was not
adequate and that varying degrees of computer expertise and funding affected the uniformity of
the LEAs special education data management. MDE has realized the need and started taking
action to implement a special education data system to ensure that performance data is compiled
consistently and accurately. MDE was in the process of soliciting proposals for the development
of a new data management system to replace the RMS. The LEAs and ISDs would use the new
system to interface with the Michigan Education Information System’s Single Record Student
Database (SRSD) system. The SRSD was being pilot tested and was anticipated to be
operational in the 2001-2002 school year. The objective of the SRSD was to establish the
essential student elements that must be maintained and reported by LEAs on each public student
in Michigan. The primary focus of the SRSD was accurate accounting of student information
that will be rationally linked to teacher, fiscal, and performance data. The directory information
collected through the SRSD will be used for a variety of purposes including Federal and state
reporting.


ED-OIG                         Control Number ED-OIG/A05-A0031                            Page 18
Recommendations
We recommend that the Assistant Secretary for Special Education and Rehabilitative Services:

3.1      Request MDE to direct the LEAs and ISDs to implement a consistent data system to
         record, manage, and report special education data or establish alternative controls to
         ensure the LEAs and ISDs provide consistent data.

3.2      Request MDE to ensure that adequate training and technical assistance is provided to all
         users.




ED-OIG                          Control Number ED-OIG/A05-A0031                           Page 19
                                    Other Matters


Individualized Education Program and
Triennial Assessment Dates

IEP and triennial assessment dates for children included in MDE’s database for the December 1,
1998 child count were not within the required time frames. Federal regulations require that a
child’s IEP is reviewed at least annually and that a re-evaluation (triennial assessment) is
conducted at least once every three years (34 CFR §300.343(c)(1) and §300.536). For children
included in the MDE database for the December 1, 1998 child count, LEA records should
include an IEP that was prepared or reviewed between December 1, 1997 and December 1, 1998
and a triennial assessment that was conducted between December 1, 1995 and December 1,
1998.

Our analysis of the MDE database for the December 1, 1998 child count found that 6,179 of the
208,403 children, ages 3-21, in the database had IEP dates that were not within the required time
frame. Our review of LEA records for 61 children included in the December 1, 1998 count
included determining whether there was an IEP within 12 months prior to December 1, 1998 and
whether there was a triennial assessment conducted within the three years prior to December 1,
1998. The IEPs and/or triennial assessments for 14 children were not conducted within the
required time frames or were missing. Both the IEP and the triennial assessment for four
children were out-of-date, for five children the IEP was out-of-date, for four children the
triennial assessment was out-of-date, and for one child the triennial assessment was missing from
the LEAs’ records.

For one of the four children with both an out-of-date IEP and triennial assessment, the LEA’s
records indicated that the child had left the LEA in March 1998. The LEA and MDE should not
have included this child in the reported child count since the child was not receiving special
education services on December 1, 1998. In conducting audits of the December 1 count, MDE
requires the children included in the December 1 count to have an IEP that is within 12 months
of the December 1 count date. The majority of the children that MDE’s audits find ineligible for
the December 1 count were children with out-of-date IEPs that had exited special education. As
a result, some of the 6,179 children with out-of-date IEPs included in the December 1 count had
exited special education and were not receiving special education services on the count date.
The IEP dates for three children in MDE’s database also did not agree with the date of the IEP in
the child’s file. These incorrect IEP dates may indicate that the number of children with out-of-



ED-OIG                         Control Number ED-OIG/A05-A0031                           Page 20
date IEPs included in MDE’s database for the December 1, 1998 count could be higher than the
6,179 children we identified.

As mentioned previously in Finding No. 1, MDE generated an exception report for those ISDs
scheduled for audits that identified children included in the December 1 count whose IEP date is
older than 12 months as of the December 1 count. MDE could generate this exception report for
all ISDs and require that the ISDs determine within a specified time frame whether the children,
have a more current IEP and are currently receiving services.




ED-OIG                         Control Number ED-OIG/A05-A0031                           Page 21
                                      Background

The Government Performance and Results Act (GPRA), was enacted on August 3, 1993.
GPRA’s purpose includes:

         •   To help Federal managers improve service delivery, by requiring that they plan for
             meeting program objectives and by providing them with information about program
             results and service quality;

         •   To improve congressional decision-making, by providing more objective
             information on achieving statutory objectives and on the relative effectiveness and
             efficiency of Federal programs and spending; and

         •   To improve internal mana gement of the Federal Government.

GPRA requires that Federal agencies prepare a five-year strategic plan for their program
activities. Starting with fiscal year 1999, Federal agencies must prepare annual performance
plans and report on program performance. The performance plans must establish performance
indicators to be used in measuring or assessing the relevant outputs and outcomes of each
program activity.

The Department published its Strategic Plan 1998-2002 in September 1997. The Department’s
1999 Performance Report and 2001 Annual Plan were submitted to Congress in March 2000.
The 2001 Annual Plan contained nine performance indicators for the IDEA, Part B Special
Education Program. The Department relies on state-reported data for measuring performance for
six of the nine listed indicators. The six performance indicators included: inclusive settings
(placement), earlier identification and intervention (intervention), regular education settings
(placement), suspensions or expulsions (discipline), graduation (exiting), and qualified personnel
(personnel). Attachment A to this report shows the relationship between the IDEA, Part B
program objectives, performance indicators, and performance data.

Performance Indicator 4.7.c of the Department’s 1999 Performance Report and 2001 Annual
Plan states that all departmental program managers will assert that the data used for their
program’s performance measurement is reliable, valid, and timely, or will have plans for
improvement. Annually, the Assistant Secretaries must provide the Office of the Under
Secretary with a signed formal attestation covering their data. The Department developed Data
Quality Standards to assist departmental managers as they collect, analyze, and report data about


ED-OIG                         Control Number ED-OIG/A05-A0031                            Page 22
Federal programs. For the IDEA, Part B special education programs, the data used for
measuring performance included data reported by the individual states.

MDE is responsible for administering the IDEA, Part B Special Education Program in the State
of Michigan. The State has 57 ISDs and over 700 LEAs. Each ISD is responsible for providing
oversight to a regional group of LEAs. The ISDs are responsible for collecting special education
data from the LEAs and submitting the data to MDE.

MDE received $143 million of IDEA, Part B funds for the 1999-2000 award year. MDE
reported that 208,403 children, ages 3-21, were receiving special education services in the State
on December 1, 1998.




ED-OIG                         Control Number ED-OIG/A05-A0031                           Page 23
                      Purpose, Scope, and Methodology


The objectives of the audit were to: (1) identify the process used by MDE to accumulate and
report IDEA, Part B performance data to OSEP, (2) determine whether MDE management
controls ensured that the performance data was reliable, and (3) identify barriers or obstacles that
may impact MDE’s ability to provide quality performance data. Our audit covered the State-
reported 1998-99 school year data for the performance indicators: placement (Indicators 1.1 and
3.1), intervention (Indicator 2.1), discipline (Indicator 3.3), exiting (Indicator 4.1), and personnel
(Indicator 5.1).

To accomplish these objectives, we interviewed State officials and staff responsible for
collecting, processing, and reporting the performance data to OSEP. We evaluated MDE’s
procedures to ensure that data reported by ISDs and LEAs was accurately recorded in MDE’s
databases and that the data reported to OSEP was supported by the data contained in MDE’s
databases. We also interviewed State officials responsible for monitoring ISDs. For the two
ISDs and two LEAs selected for our review, we interviewed ISD and LEA officials and reviewed
their procedures for recording, processing, and reporting special education data.

MDE reported for the December 1, 1998 count a total of 208,403 children age 3-21, receiving
special education services. For the 1998-99 school year, the State of Michigan had over 700
LEAs that reported performance data to 57 ISDs. The 57 ISDs and 4 state agencies reported the
performance data to MDE. MDE reported a total count of 101,270 children for 5 of the 57 ISDs,
that represents 49 percent of the total. The five ISDs include three of the four ISDs that use their
own system rather than the MDE sponsored RMS for recording and reporting special education
data. We judgmentally selected two ISDs, from this group of five ISDs, and reviewed their
procedures and the procedures used by one LEA in each ISD.

To ensure that we evaluated the procedures at the ISDs using different data systems, we selected
the Genesee ISD which is 1 of the 53 ISDs that use the RMS, and the Oakland ISD which is 1 of
the 4 ISDs that uses its own system. We selected the Flint LEA because it had the largest
reported child count of the 21 LEAs that comprise the Genesee ISD and the Pontiac LEA which
also had the largest child count of the 28 LEAs that comprise the Oakland ISD. We also selected
the 2 LEAs because MDE’s audits of the December 1, 1998 count identified these LEAs with the
largest number of ineligible students included in the count.




ED-OIG                           Control Number ED-OIG/A05-A0031                              Page 24
The following tables show a breakdown of the reported 208,403 children for the December 1,
1998 count by reported child count and the 5 ISDs with the largest reported count.

    Reported Child Count           Number of ISDs          Combined Total            Percent of Total
    Over 10,000                    5                       101,270                    49
    Between 2,000 and 10,000       19                       68,839                    33
    Between 400 and 2,000          33                       38,294 ( a)               18
    Total                          57                      208,403                   100
  (a) Total includes 627 children reported by the four state agencies .

                                  ISD                         Reported Child Count
                                Genesee                               10,068
                                 Kent                                 13,995
                                MaComb                                15,688
                                Oakland                               21,183
                                 Wayne                                40,336
                                 Total                               101,270

The Flint LEA reported child count of 2,894 represents 29 percent of the Genesee ISD’s total
and the Pontiac LEA reported child count of 2,108 represented 10 percent of the Oakland ISD’s
total.

In addition, we reviewed MDE’s single audit report that covered the two fiscal years ended
September 30, 1997 and draft findings from the Michigan Office of the Auditor General’s
performance review of the special education program for the two year period ended June 30,
1999. We also reviewed MDE’s most recent monitoring reports for the Genesee and Oakland
ISDs and the two most recent single audit reports for each selected ISD and LEA.

Intervention, Placement, Exiting, and Personnel. To achieve our audit objectives for the
intervention, placement, exiting, and personnel performance indicators, we relied on computer-
processed data extracted from MDE’s three statewide databases for the December 1, 1998 count,
exiting, and personnel. Our assessment of the reliability of these databases was limited to (1)
gaining an understanding of the procedures used by MDE, the two ISDs and the two LEAs to
collect, process, review, compile, and report the data, (2) reconciling the data reported to OSEP
to MDE’s database files, (3) comparing the LEAs’ data to the ISDs’ and MDE’s electronic
databases that are used to compile and report performance data, and (4) confirming that data
provided by the two LEAs and included on the MDE’s databases was supported by information
contained in LEA records.

To test the accuracy of MDE’s December 1, 1998 count, exit, and personnel databases we
randomly selected children and personnel for the two ISDs from the databases and compared the
children and personnel information to the children and personnel records for two LEAs. The



ED-OIG                              Control Number ED-OIG/A05-A0031                                     Page 25
following table shows the total children and personnel reported for each group and the number of
children and personnel LEA files reviewed:

    LEA            Children Receiving                         Exiting                       Personnel
                    Special Education                       Indicator 4.1                  Indicator 5.1
                Child Count & Placement
                Indicators 1.1, 2.1 and 3.1
                   Total         Children           Total           Children           Total         Personnel
                  Children          Files          Children           Files          Personnel         Files
                 Reported        Reviewed          Reported         Reviewed         Reported        Reviewed
Flint              2,894             31              622               15               416             11
Pontiac             2,108              30              133              15               261                10


We concluded that MDE should take additional steps to improve management controls over the
collection and reporting of performance data reported to OSEP for intervention, placement,
exiting, and personnel. The Audit Results section of the report provides details on our findings.

Discipline. To achieve our audit objectives for the discipline performance indicator, we (1)
gained an understanding of the procedures MDE, the two ISDs, and the two LEAs use to collect,
process, review, compile, and report the data, (2) reconciled the data reported to OSEP to MDE’s
database file, (3) compared the LEAs’ data to the MDE’s database, (4) tested MDE’s entry of the
LEAs’ discipline data into the MDE’s statewide discipline database, and (5) confirmed that data
provided by the two LEAs was supported by information contained in the LEAs’ files. The
following table shows the total number of children reported for discipline by each LEA and the
number of files reviewed:

                                  Discipline
                                 Indicator 3.3

                     Total Children           Children Files
        LEA
                       Reported                Reviewed
 Flint                       0                     10 (a)
 Pontiac                     47                     10
(a) Flint did not report any discipline children for the 1998/1999 school year. Our review of discipline records
included a judgmental sample of 10 children from the 292 children reported for the 1999/2000 school year.

From our assessment and tests, we concluded that MDE needs stronger management and system
controls over the collection of IDEA, Part B data that it reports to OSEP for discipline. Stronger
controls are also needed at the ISD and LEA levels. The Audit Results section of the report
provides details on our findings.

We performed our fieldwork at MDE in Lansing, Michigan, and at the special education offices
of the Genesee ISD and the Flint LEA in Flint and the Oakland ISD in Waterford and the Pontiac
LEA in Pontiac. We also performed work at our offices in Chicago, Illinois and St. Paul,
ED-OIG                               Control Number ED-OIG/A05-A0031                                        Page 26
Minnesota. Fieldwork was conducted from August 14, 2000 to February 16, 2001. Our audit
was performed in accordance with generally accepted government auditing standards appropriate
to the scope of the review described above.




ED-OIG                        Control Number ED-OIG/A05-A0031                         Page 27
                    Statement on Management Controls


As part of our review, we assessed the system of management controls, policies, procedures, and
practices applicable to MDE’s process for collecting and reporting performance data for the
IDEA, Part B program as required by GPRA. Our assessment was performed to determine
whether the processes used by MDE and the reviewed ISDs and LEAs provided a reasonable
level of assurance that MDE reported reliable performance data to OSEP.

For the purpose of this report, we assessed and classified MDE’s significant controls related to
collection and reporting of performance data into the following categories:

   •     Guidance and technical assistance,
   •     Collection of data from ISDs and LEAs,
   •     Data compilation and report preparation, and
   •     Monitoring ISD and LEA data collection and reporting processes.

Because of inherent limitations, a study and evaluation made for the limited purpose described
above would not necessarily disclose all material weaknesses in the management controls.
However, our assessment disclosed management control weaknesses that adversely affected
MDE’s ability to report accurate performance data for IDEA, Part B. These weaknesses relate to
all four control categories. The weaknesses and the related recommendations are discussed in
the Audit Results section of this report.




ED-OIG                          Control Number ED-OIG/A05-A0031                          Page 28
                                                                                                                                                   Attachment A

                   IDEA, Part B Program Objectives, Performance Indicators and Performance Data
                                                2001 Annual Plan

                                                                                                                PERFORMANCE DATA COLLECTED
    PROGRAM OBJECTIVE                                   PERFORMANCE INDICATOR                                         FROM OSEP FORMS
All preschool children with disabilities       1.1 Inclusive settings. The percentage of preschool              State educational agencies report the number of
receive services that prepare them to enter        children with disabilities who are receiving special         students ages 3-5 by age and educational placement.
school ready to learn.                             education and related services in inclusive settings will
                                                   increase.
All children who would typically be            2.1 Earlier identification and intervention.             The     State educational agencies report number of disabled
identified as being eligible for special           percentage of children served under IDEA ages 6 or 7,        children receiving special education by:
education at age 8 or older and who are            compared to ages 6-21, will increase.                            § disability and age and
experiencing early reading or behavioral                                                                            § disability and ethnicity
difficulties receive appropriate services
earlier to avoid falling behind their peers.
All children with disabilities have access     3.1 Regular education settings (school age).               The   State educational agencies report the number of
to    the     general     curriculum     and        percentage of children with disabilities ages 6-21 who      students ages 6-21, by age category, disability and
assessments,          with       appropriate        are reported by states as being served in the regular       placement.
accommodations, supports, and services,             education classroom at least 80 percent of the day will
consistent with high standards.                     increase.
                                               3.3 Suspensions or expulsions. The percentage of children        State educational agencies report the number of
                                                    with disabilities who are subject to long-term              students suspended or expelled, unilateral removal or
                                                    suspension or expulsion, unilateral change in placement     removal based on a hearing by:
                                                    or change in placement if their current placement is            § disability and basis of removal and
                                                    likely to result in injury to someone, will decrease.           § ethnicity and basis of removal
Secondary       school     students   with     4.1 Graduation.          The percentage of children with         State educational agencies report the number of
disabilities receive the support they need          disabilities exiting school with a regular diploma will     students ages 14-21 that exited special education by:
to complete high school prepared for                increase and the percentage who drop out will decrease.          § age, disability and basis of exit,
postsecondary education or employment.                                                                               § age and basis of exit and
                                                                                                                     § ethnicity and basis of exit
States are addressing their needs for          5.1 Qualified personnel. The number of states and                State educational agencies report the number and
professional development consistent with           outlying areas where at least 90 percent of special          type of teachers and other personnel to provide
their comprehensive system of personnel            education teachers are fully certified will increase.        special education and related services for children
development.                                                                                                    ages 3-21. State educational agencies must report the
                                                                                                                number of staff:
                                                                                                                      § fully certified and
                                                                                                                      § not fully certified




ED-OIG                                     Control Number ED-OIG/A05-A0031                                             Page 29.
ED-OIG   Control Number ED-OIG/A05-A0031   Page 30.
                                                  .
                       REPORT DISTRIBUTION LIST
                                     ED-OIG/A05-A0031

                                                                   No. of Copies
Auditee                                                                    1

Mr. Thomas D. Watkins, Jr., Superintendent
Michigan Department of Education
P. O. Box 30008
Lansing, Michigan 48909

Action Official                                                            2

Robert Pasternack, Assistant Secretary
Office of Special Education and Rehabilitative Services
U.S. Department of Education
330 C Street, SW Room 3124
Washington, DC 20202

Other ED Offices

Director, Office of Special Education Programs                             1
Chief of Staff, Office of the Secretary                                    1
Office of the Under Secretary                                              1
Deputy Secretary, Office of the Deputy Secretary                           1
Office of the Chief Financial Officer
 Financial Improvement and Post Audit Operations                           1
Office of Public Affairs                                                   1
Assistant Secretary, Office of Intergovernmental and Interagency Affairs   1

Office of Inspector General

Inspector General                                                          1
Deputy Inspector General                                                   1
Deputy Assistant Inspector General for Audit                               1
Assistant Inspector General for Audit                                      1
Assistant Inspector General for Analysis and Inspection                    1
Assistant Inspector General for Investigations                             1
Director, State and Local Program Advisory and Assistance                  1
Counsel to the Inspector General                                           1
Regional Inspectors General for Audit                                      1 (each)