oversight

District of Columbia Public Schools: Audit Confirms Reasonableness of Enrollment Count, but Report's Presentation Is Unclear

Published by the Government Accountability Office on 1999-03-31.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

United States
General Accounting Office
Washington, D.C. 20548

Health, Education, and
Human Services Division


B-282033

March 31, 1999

The Honorable Ernest J. Istook
Chairman, Subcommittee on the District of Columbia
Committee on Appropriations
House of Representatives

Subject:    District of Columbia Public Schools: Audit Contis    Reasonableness
            of Enrollment Count, but Reuort’s Presentation Is Unclear

Dear Mr. Chairman:

The District of Columbia Public Schools (DCPS) is one of the largest public
school districts in the country-ranking in the top 40 of the nation’s nearly 15,000
school districts in terms of enrollment. A portion of its funding is federal, with
about $95 million of its $645 rnilhon budget coming from federal funds in fiscal
year 1999. Over the past decade, DCPS’enrollment count has been
controversial, with critics charging that the count has been overstated. We
reviewed DCPS’enrollment policies and procedures in 1995 and 1997 and found
a lack of documentation for enrollment information as well as other problems.’

Concerns about the accuracy of DCPS’enrollment count led the Congress to
include a requirement in the District of Columbia School Reform Act of 1995 for
an independent audit of the count. The most recent audit, completed in May
1998, used a sampling methodology to conduct its verification, and you asked us
to review the firm’s approach. Specifically, we asked (1) Was the audit
methodology reasonable? and (2) Was the audit report’s presentation clear and
complete?

To answer your request, we reviewed the audit report and supporting
documentation and discussed the audit approach and results with officials
responsible for the audit. We also compared the auditors’ sampling approach to



‘District of Columbia: Weaknesses in Personnel Records and Public Schools’
Management Information and Controls (GAO/T-AND-95-170, June 14, 1995) and
District of Columbia Public Schools: Student Enrollment Count Remains
Vulnerable to Errors (GAO/HEHS-97-161, Aug. 21, 1997).
                GAO/HEHS-9946R      District of Columbia   Schools Enrollment Audit
                                                        -



B-282033

generally accepted sampling methods.2 Our statisticians reviewed the report and
supporting documentation, and one of our statisticians discussed the analytic
techniques with the auditors’ statistician. We also reviewed previously
completed work related to the District of Columbia schools’enrollment
procedures. We conducted our review between September 1998 and February
1999 in accordance with generally accepted government auditing standards.

In summary, we found the audit methodology to be reasonable given the
limitations of DCPS’data. Both the sampling approach and the analysis of the
data involved analytical techniques that were carried out by using generally
accepted statistical methods. In presenting the results, the audit report
determined that DCPS’enrollment count had been fairly stated. It did not,
however, fully present supporting detail explaining its conclusion. For example,
the report did not clearly explain that the sample data actualIy yielded a wider
range of estimates than it presented. Better presentation would have made it
easier to fully understand the basis for the report’s conclusion.

BACKGROUND

In the past, DCPS’enrollment count was not a basis for determining its funding.
However, the school district’s funding for f%scaIyear 1997 and subsequent years
was directly linked to school enrollment under the District of Columbia School
Reform Act of 1995.3 The appropriations law that provided fiscal year 1999
federal funding for DCPS underscores continuing congressional concern about
DCPS’enrollment information.4 It prohibits the use of federal funds to pay the
salaries of any DCPS school teacher, principal, administrator, official, or
employee who knowingly provides false enrollment or attendance information.
It also prohibits the use of federal funds to subsidize the education of
nonresidents of the District of Columbia at DCPS elementary or secondary
schools unless they pay tuition to the District of Columbia.

Our 1997 review of DCPS’enrollment count process found that even though
DCPS had changed parts of the process to address past criticisms, it remained


2Using Statistical Samuling (GAO/PEMD-10.1.6, May 1992) and Richard L.
Scheaffer, William MenderhaIl, and Lyman Ott, Elementarv Survev Samuling, 4th
ed. (Boston, Mass.: TWS-KENT Publishing Co., 1990).
3Although federal appropriations for DCPS are not directly tied to enrollment,
annual payments to DCPS from the District of Columbia’s general fund are
related to student enrollment.

4Public Law 105-277(1998).

 2             GAOLHEHS-99-66R      District of Columl$a Schools Enrollment   Audit
B-282033

flawed in several areas, including problems with the procedures used to correct
errors in the school district’s automated database and problems with the
procedures used to verify student residency. We also determined that DCPS and
the District of Columbia Financial Responsibility and Management Assistance
Authority were not in compliance with certain requirements of the School
Reform Act of 1995. We recommended that DCPS comply with the reporting
requirements of the act and that the Authority comply with the auditing
requirements of the act. Since then, DCPS has implemented various new
procedures and requirements intended to improve the accuracy of its enrollment
count, and the Authority has contracted for an audit of the DCPS enrollment
count.

In March 1998, the Authority awarded a contract to the firm of Thompson, Cobb,
Bazilio, and Associates to conduct an audit of the official enrollment count that
DCPS had established for October 30, 1997. The firm was charged with
providing an opinion as to the accuracy of certain information in DCPS’official
enrollment count. After familiarizing themselves with how the count had been
conducted, the auditors found that the records on which the count was based
contained errors. For example, in some cases school officials had not made
corrections to the records although school district officials had identified
duplicate student names in the system and had requested that the records be
updated. In other cases, proper transfer procedures had not been followed,
allowing students to be counted by the system more than once.

Because of the problems the auditors encountered with data quality, they were
unable to verify the student count by certain categories (such as by grade) but
were able to report a count total. To assess the accuracy of the enrollment
count, the audit staff developed a methodology that was based on a sample of
DCPS’records. They concluded that despite data and process deficiencies,
DCPS had managed to conduct an enrollment count that was fairly stated.5

THE AUDITORS’ METHODOLOGY WAS REASONABLE
GIVEN DATA LIMITATIONS

The auditors’ use of a sampling approach for verifying the accuracy of DCPS’
enrollment count was reasonable, given the limitations they identified in DCPS’
automated database. The size of the auditors’ data sample and the sampling
methods they used were also reasonable for producing data that had statistical
reliability. To develop their sample, the auditors first collected membership lists


5Thompson, Cobb, Bazilio, and Associates, Audit of the Official Membershiuf
the District of Columbia Public Schools as of October 30, 1997 (Washington,
D.C.: May 1998).

3              GAOLHEHS-99-66R      District of Columbia Schools Enrollment   Audit
B-282033

f&m all the school homerooms that had been used to record the October 30,
1997, student enrollment count6 Using these lists as a base, the auditors
selected a random sample of 237 homerooms from the total 3,557 homerooms.

Next, the auditors collected enrollment information for all the students in their
sample homerooms. To collect this information, the auditors visited the sample
homerooms and verified their lists against actual students. In the final count,
the auditors included only students for whom they had collected adequate
evidence of enrollment.7 If no acceptable alternative evidence was provided for
the students in DCPS’count, they were considered errors and were not included.
The auditors then compared their homeroom totals to the official DCPS count
and noted the differences to prepare for their data analysis.

The auditors used a ‘difference estimation” approach to conduct their data
analysis and develop their estimates of the school district’s enrollment. This
standard analytical approach allowed the auditors to estimate the count for the
homeroom sample by comparing the differences between DCPS’October 30,
1997, enrollment count and the corrected count that they had determined from
their on-site visits. Using the results from the 237 sampled homerooms, they
estimated the differences for all homerooms to obtain a corrected total
homeroom estimate.8

The auditors used a different approach to calculate the number of DCPS tuition
grant students. These students are paid for by funds available to DCPS but are


‘The auditors noted, however, that there were no standard policies throughout
DCPS regarding what constituted a homeroom. As a result, schools used
different groupings to report their counts, such as a “homeroom” class (in some
schools), first period class, or group advisory. In all cases, these groupings were
referred to as homerooms.
7They followed up on students noted as absent, transferred in or out, or
withdrawn to determine their enrollment status and reviewed enrollment dates
for students present for their site visits but not listed on the homeroom lists.
They also reviewed files to verify student residency information.
‘The auditors reported that DCPS’count included 187 students who were not
residents of the district and had withdrawn from school since the official count
on October 30, 1997. The auditors subtracted these students from their final
estimates. A more direct comparison with DCPS’count would have left these
students in, just as they had been included in the district’s count. However, the
effect on the auditors’ final conclusion was minimal, as the DCPS count would
still be withm the confidence interval for the estimates adjusted on this basis.

4              GAO/HEHS-99-66R      District of Columbia Schools Enrollment   Audit
B-282033

not enrolled in DCPS schools and thus were not included in the homeroom
count. To count these students, the auditors surveyed each of the 73 schools
identified as having tuition grant students. Using the confirmation information
received through their survey, the auditors calculated this population separately
fkom the homeroom count.

Table 1: DCPS’and Auditors Renorted Totals



(I  Source


    DCPS’reported count
                                   Homeroom
                                   counta
                                             76,035
                                                          Tuition grant
                                                          counlf
                                                                   1,076
                                                                           Total count

                                                                                  77,111
                                             76,268                1,302          77,570

                                             76,076                1,195          77,271
                                                      I
aNote that the homeroom count was obtained by a “difference estimation” approach while the
tuition grant count was obtained by collecting survey information.

Like DCPS’total figure, the auditors’ total figures included the homeroom count
and the tuition grant students. As table 1 shows, DCPS reported a total student
count of 77,111,while the auditors reported-as a range-two count estimates of
77,271 and 77,570. Although slightly lower, DCPS’number was within 1 percent
of the auditors’ estimates.

The auditors reported two numbers for their homeroom count, each based on a
different assumption about the data they were using. For audit estimate 1, the
auditors included in their base calculation a group of students who they thought
DCPS might have excluded in error from its enrollment count. For audit
estimate 2, the auditors excluded this same group of students, just as had been
done in DCPS’calculations. This second estimate, calculated from the same
base number that DCPS used, made it easier for the auditors to identify
differences between DCPS’count and their count because the base number was
held constant.

The auditors also reported two numbers for tuition grant students. The low
number (1,195) was based on verifying the presence of these students through
the auditors’ school surveys and adjusting the school district lists accordingly.
The higher number (1,302) reflects the survey total plus the potential addition of
107 students who either were unconfirmed (because their survey information
was not returned) or may have had their tuition paid for by another agency.


5               GAO/HEHS-99-66R         District   of Columbia Schools Enrollment Audit
B-282033
T%IE REPORT MET THE MAIN AUDIT
OBJECTIVE BUT DID NOT FULLY
PRESENT SUPPORTING DETAIL

The audit report answered the audit’s main objective of determining whether
DCPS’enrollment count was fairly stated; however, the analysis information the
auditors provided to support their conclusion was not as complete and clear as
it could have been. For example, the auditors presented their final results in a
manner that appears as though the homeroom count estimates are the endpoints
of a single range, stating, “we estimate the total DCPS students enrolled and
attending at October 30, 1997, to be in the range of 76,07676,268.”With this
presentation, readers could be led to believe that the projected number of DCPS
students fell between these two point estimates. Instead, the number could have
fallen outside the point estimates. Our follow-up with a member of the audit
firm showed that these two estimates were based on the two different
assumptions discussed earlier and that each one has its own confidence interval
establishing a likely range for the true student enrollment.

Figure 1 shows the difference between what readers might infer from the
report’s presentation and what the statistical projections actually were. Under
estimate 1, the homeroom count for DCPS is identified as 76,268with a precision
of plus or minus 614 students. Under estimate 2, the homeroom count is
identified as 76,076 with a precision of plus or minus 131.’ Figure 1 shows that
DCPS’count of 76,035 is within the range of either estimate and that the school
district’s and auditors’ counts are within about 1 percent of each other.
Confidence intervals or other measures identifying the precision of the estimates
were also missing &om the written report. Without this type of information, a
reader is left unaware of the true range of the estimates.




‘The range under both assumptions reflects a 95percent cotidence interval. A
confidence interval is a range of values that is expected (with a reasonable
degree of confidence) to include the true value. For these two estimates, it may
be interpreted that there is 95percent confidence that the true value is between
75,654 and 76,882 for estimate 1 and between 75,945 and 76,207for estimate 2.

 6             GAO/HEHS-99-66R      District of Columbia Schools Enrollment Audit
             B-282033

             Fi&re 1: ReDort Presentation Compared With Auditors’ Actual Results




What Readers Might Infer About the Estimates, Given the Report’s Presentation




                                                                76,5Cil--

Estimating
Approach I                /.         )       76,268 (High Estimate)
Estimating           :   ::, .y +   ‘. ‘1,                                                             District of Columbia
Approach     2                        *          76,076 (Low Estimate)_- 76,035        --              Public Schools Official
                                                                                                       Enrollment Count




                                             -
                                                                75,500      --


What the Estimates Actually Showed




                                                 76,882 (High Estimate)



                                     :

                                             76,268 (Point Estimate)

                                                                                                        District of Columbia
                                                                                 76,035 4-              Public Schools Official
                                                                                                        Enrollment Count




                 In addition, if the auditors had more clearly linked the information in the report
                 body with the information in the report appendix, it would have been easier to
                 understand the report results. For example, the auditors did not provide a direct
                 link between their analysis in the appendix and their subtraction of the 187

                 7                  GAOIHEHS-99-66R                              District of Columbia Schools Enrollment          Audit
B-282033

students who were not district residents and had withdrawn from school since
DCPS’October 30, 1997, count. Although the calculation related to these 187
students was noted in the report body, it is not apparent in the appendix. As a
result, in our opinion, readers are left with the difficulty of interpreting results
and making calculations to link the results in the body to the results in the
appendix. It was not until we posed questions to one of the auditors that we
were able to link the appendix numbers with the final results identified in the
report body. lo

AGENCY COMMENTS

We provided a draft of this report to the Authority and DCPS for comment.
Both the Authority and DCPS concurred with our finding that the audit
methodology was reasonable and that the audit firm’s report presentation could
have been more clear. The Authority commented that itwill request the current
auditor of the DCPS enrollment count to include its analysis in the body of its
audit report. We also discussed the contents of the draft with representatives of
the audit firm and they concurred with our factual representation of the audit
and generally supported the need for improvements in future audit reports.



We are sending copies of this correspondence to appropriate congressional
committees, the District of Columbia Financial Responsibility and Management
Assistance Authority, and the District of Columbia Public Schools. We will make
it available to others who request a copy. If you or your staff have any questions
or wish to discuss this material further, please call me or Harriet Ganson,
Assistant Director, at (202) 512-7014. Major contributors included Dawn Hoff,
Wayne Dow, and Stan Stenersen.

Sincerely yoxs,


Marnie S. Shaul
Associate Director, Education,
  Workforce, and Income Security Issues

(104954)



“We also identified other smaller presentation errors, where data had been
incorrectly calculated or stated. However, none of these errors affected the final
conclusion of the audit report.


 8                GAO/HEHS-99-66R     District   of Columbia   Schools Enrollment   Audit
Ordering    Information

The first copy of each GAO report and testimony is free.
Additional  copies are $2 each. Orders should be sent to the
foliowing address, accompanied by a check or money order
made out to the Superintendent    of Documents, when
necessary. VISA and Mastercard     credit cards are accepted, also.
Orders for 100 or more copies to be mailed to a single address
are discounted 25 percent.

Orders by mail:

U.S. General Accounting   Office
P.O. Box 37050
Washington, DC 20013

or visit:

Room 1100
700 4th St. NW (corner of 4th and G Sts. NW)
U.S. General Accounting Office
Washington, DC

Orders may also be placed by calling (202) 512-6000
or by using fax number (202) 512-6061, or TDD (202) 512-2537.

Each day, GAO issues a list of newly available reports and
testimony.   To receive facsimile copies of the daily list or any
list from the past 30 days, please ca.B (202) 512-6000 using a
touchtone phone. A recorded menu will provide information         on
how to obtain these lists.

For information on how to access GAO reports on the INTERNET,
send an e-mail message with “info” in the body to:

info@www.gao.gov

or visit GAO’s World Wide Web Home Page at:

http://www.gao.gov
United States
General Accounting Office
Washington, D.C. 20548-0001
                                   I   Permit   No. GlOO   I
Official Business
Penalty for Private    Use $300

Address   Correction   Reauested