oversight

Audit of the External Audit Followup Process -- OSERS.

Published by the Department of Education, Office of Inspector General on 2004-06-02.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                         UNITED STATES DEPARTMENT OF EDUCATION

                                         OFFICE OF INSPECTOR GENERAL




                                                 June 2, 2004



                                                                                        CONTROL NUMBER
                                                                                         ED-OIG/A19-E0006

Troy Justesen
Acting Deputy Assistant Secretary
Office of Special Education and Rehabilitative Services
U.S. Department of Education
Mary E. Switzer Building, Room 3006
330 C Street, SW
Washington, DC 20202

Dear Mr. Justesen:

This Final Audit Report, (Control Number ED-OIG/A19-E0006), presents the results of our
audit of the audit followup process for external audits in the Office of Special Education and
Rehabilitative Services (OSERS). This audit was part of a review of the audit followup process
for Office of Inspector General (OIG) external audits being performed in several principal
offices. A summary report will be provided to the Chief Financial Officer, the Department of
Education (Department) audit followup official, upon completion of the audits in individual
principal offices.


                                           BACKGROUND

Office of Management and Budget Circular A-50, entitled “Audit Followup,” provides the
requirements for establishing systems to assure prompt and proper resolution and
implementation of audit recommendations. The Circular states,

       Audit followup is an integral part of good management, and is a shared
       responsibility of agency management officials and auditors. Corrective action
       taken by management on resolved findings and recommendations is essential to
       improving the effectiveness and efficiency of Government operations. Each
       agency shall establish systems to assure the prompt and proper resolution and



                          400 MARYLAND AVE., S.W. WASHINGTON, D.C. 20202-1510

           Our mission is to ensure equal access to education and to promote educational excellence
Mr. Justesen	                                                                      Page 2 of 7



        implementation of audit recommendations. These systems shall provide for a
        complete record of action taken on both monetary and non- monetary findings and
        recommendations.

The Department has established a Post Audit User Guide to provide policy and procedures for
the audit resolution and followup process. This guide provides that, “Each Assistant Secretary
(or equivalent office head) with cooperative audit resolution or related responsibilities must
ensure that the overall cooperative audit resolution process operates efficiently and consistently.”
As an Action Official (AO), the Assistant Secretary’s responsibilities include,

    •	 Determining the action to be taken and the financial adjustments to be made in resolving
       findings in audit reports concerning respective program areas of responsibility,
    •	 Monitoring auditee actions in order to ensure implementation of recommendations
       sustained in program determinations, and
    •	 Maintaining formal, documented systems of cooperative audit resolution and followup.


                                      AUDIT RESULTS

We found that improvements were needed in OSERS’ audit followup process. Our audit
revealed that documentation was not always maintained to support that corrective actions were
completed. OSERS staff stated that they did not conduct audit followup once the Program
Determination Letter (PDL) was issued and that they assumed that corrective actions were taken.
OSERS staff further stated that they were unsure of the amount of documentation needed and
record retention requirements. As a result, OSERS does not have assurance that corrective
actions are implemented.

OSERS responded to our draft report, concurring with the results and supporting the
recommendation provided. OSERS also described specific corrective actions it has taken and
intends to take to address the issues noted. The full text of the OSERS response is included as
Attachment 2 to this audit report.


Finding 1	 OSERS’ Corrective Action Followup and Documentation Needs
           Improvement

We found that OSERS’ audit followup process was not always effective. OSERS did not always
maintain documentation that showed corrective actions were completed. We found OSERS files
did not contain documentation to support that corrective actions had been taken for 3 of the 15
(20 percent) recommendations included in our review.

Specifically, we found supporting documentation was not maintained to support completion of
corrective actions for the following recommendations:



                                        ED-OIG/A19-E0006
Mr. Justesen                                                                      Page 3 of 7



Audit Control Number (ACN) A09-A0001: “Arizona Department of Education Management
Controls over IDEA [Individuals with Disabilities Education Act], Part B-Special Education
Performance Data,” issued September 2000.

        Recommendation 1.1: Require ADE [Arizona Department of Education] to
        develop written policies and procedures for collecting, reviewing and reporting
        performance data for exiting, discipline and personnel. The procedures should
        include reviews of LEA [Local Educational Agency] reports and periodic reviews
        of LEA procedures and supporting documentation.

In response to the recommendation, OSERS requested that the ADE advise the Department of its
progress in implementing recommendation 1.1 and submit a timetable for completion of revisions
to a Desk Manual. OSERS also requested ADE provide a copy of the Desk Manual when
revisions were complete.

ADE provided a draft copy of the Desk Manual, and indicated in its August 30, 2001, response
that enhancements would be made to the manual by June 30, 2002. We found that OSERS staff
did not followup to ensure that ADE completed and implemented the revised Desk Manual.
OSERS did not receive a copy of the Desk Manual when revisions were complete.

        Recommendation 2.1: Implement procedures for verifying the accuracy of data
        entry for exiting, discipline and personnel data.

In response to the recommendation, OSERS requested that ADE submit a written description of
the procedures that were under development and copies of applicable written communications
with LEAs. ADE indicated a new web-based application to collect data was created and tha t this
application would include several edit checks.

We found OSERS files did not contain documentation to support that this new application had
been implemented. ADE’s response also indicated that new procedures were being implemented
to ensure greater accuracy and completeness of ADE’s data, but OSERS files did not contain
copies of any such procedures. There was no documentation in the files indicating that OSERS
had accepted the software revisions in lieu of the written descriptions of the procedures and
copies of any applicable written communications with the LEAs, as originally requested.


ACN A09-A0016: “California Department of Education Management Controls over IDEA, Part
B-Special Education Performance Data,” issued March 2001.

        Recommendation 1.1: Require CDE [California Department of Education] to
        issue guidance to school districts on the proper category to use when the reason
        for the exit is unknown so that CDE can properly include such exits in the
        “dropped out” category on the OSEP [Office of Special Education Programs]
        reporting form.


                                        ED-OIG/A19-E0006
Mr. Justesen                                                                                Page 4 of 7



OSERS requested that CDE submit an anticipated schedule of meetings along with all available
draft or final agendas and memoranda issued to the Special Education Local Plan Areas
(SELPAs) and school districts regarding the proper category for use when the reason for the exit
is unknown. Although CDE indicated that training was provided on the proper use of exit
categories, we did not find any documentation in OSERS files that indicated CDE issued
guidance or memoranda to the SELPAs and school districts in this area.


Audit Followup Requirements

The Department’s Post Audit User Guide (Draft Version as of January 2, 2001, in effect during
our audit scope) 1 , Section III, Chapter 5, Part B, states:

        Primary responsibility for following up on nonmonetary determinations rests with
        AOs, who must have systems in place to ensure that recommended corrective
        actions are implemented by auditees. The OCFO [Office of the Chief Financial
        Officer] has responsib ility for verifying that AOs have systems in place to
        followup on corrective actions and ensuring overall effectiveness of ED’s
        [Department of Education’s] audit resolution followup system.

Part B further states, “Accurate records must be kept of all audit followup activities including all
correspondence, documentation and analysis of documentation.”

OSERS staff stated that they did not conduct audit followup once the PDL was issued and that
they assumed that corrective actions were taken. OSERS staff further stated that they were
unsure of the amount of documentation needed and record retention requirements. OSERS staff
stated that, as a result of our audit, they were in the process of developing a system to track
completion of corrective actions.

Without appropriate documentation and followup of corrective actions, OSERS did not have
assurance that the deficiencies identified were corrected.

Subsequent to the resolution of the audits we reviewed, the Department established additional
guidelines that expand upon the documentation requirements for audit resolution files. The
Department’s “Guidelines for Establishing File Folders and Maintaining Documentation For
External Audits,” were effective as of September 1, 2002, and state that audit resolution files
should contain “...All documentation pertaining to audit follow-up activities, e.g., documentation
from the auditee substantiating the corrective action taken….” These guidelines are provided in
Attachment 1 to this report.




1
 The Post Audit User Guide was revised and updated as of March 31, 2003. The statements quoted are also in the
current version of the guide.

                                             ED-OIG/A19-E0006
Mr. Justesen	                                                                    Page 5 of 7



Recommendation

We recommend that the Acting Deputy Assistant Secretary for OSERS:

        1.	 Ensure that for future audits, adequate documentation is maintained to support the
            completion of all corrective actions, in accordance with the Department’s external
            audit documentatio n and file requirements.


                   OBJECTIVE, SCOPE, AND METHODOLOGY
The objective of our audit was to evaluate the effectiveness of the Department’s process to ensure
that external auditees implement corrective action. To accomplish our objective, we reviewed
applicable laws and regulations, and Department policies and procedures. We conducted
interviews with OSERS program staff responsible for resolving and following up on corrective
actions for the audits selected. We also reviewed documentation provided by OSERS staff to
support the corrective actions taken for the recommendations included in our review.

The scope of our audit included OIG audits of OSERS programs at external entities issued during
the period October 1, 1997, through September 30, 2002. The audits in the scope were reported
by the Department’s audit resolution system as having been “closed” on or prior to September 30,
2002. A total of 7 OSERS audits, representing 38 recommendations, met these criteria.

To select OSERS audits to review, we evaluated the status of the recommendations and the
corrective actions required by the Department. We judgmentally selected two audits to review
based on the fact that a series of audits was conducted on the same program. The 2 audits
reviewed represented 15 recommendations and were as follows:

    •	 ACN A09-A0001, “Arizona Department of Education Management Controls Over IDEA,
       Part B-Special Education Performance Data,” issued September 2000, and

    •	 ACN A09-A0016, “California Department of Education Management Controls over
       IDEA, Part B-Special Education Performance Data,” issued March 2001.

We relied on computer-processed data initially obtained from the OIG’s Audit Tracking System
to identify OIG audits issued during the scope period. We reconciled this data to the
Department’s Common Audit Resolution System (CARS), and to audits reported in the
Semiannual Reports to Congress to ensure that we had captured all audits issued during the
period. We also reviewed copies of the audit reports to determine that the audits met the scope
period under review. We confirmed data in the audit reports to data in the Department’s Audit
Accountability and Resolution Tracking System, which replaced CARS in July 2003. Based on
these tests and assessments, we determined that the computer-processed data was reliable for
meeting our audit objective.



                                        ED-OIG/A19-E0006
Mr. Justesen                                                                     Page 6 of 7



We conducted fieldwork at Department offices in Washington, DC, during the period November
2003 through March 2004. We held an exit conference with OSERS staff on March 9, 2004.
Our audit was performed in accordance with government auditing standards appropriate to the
scope of the review described above.


                 STATEMENT ON MANAGEMENT CONTROLS

As part of our review, we assessed the system of management controls, policies, procedures, and
practices applicable to the audit followup process for OIG external audits of OSERS programs.
Our assessment was performed to review the level of control risk and determine the nature,
extent, and timing of our substantive tests to accomplish the audit objective.

Because of inherent limitations, a study and evaluation made for the limited purpose described
above would not necessarily disclose all material weaknesses in the management controls.
However, our assessment disclosed management control weaknesses that adversely affected
OSERS’ ability to ensure corrective actions were taken by external entities in response to audits
of OSERS programs. These weaknesses and their effects are fully discussed in the AUDIT
RESULTS section of this report.



                            ADMINISTRATIVE MATTERS
Corrective actions proposed (resolution phase) and implemented (closure phase) by your office
will be monitored and tracked through the Department’s Audit Accountability and Resolution
Tracking System. Department policy requires that you develop a final corrective action plan
(CAP) for our review in the automated system within 30 days of the issuance of this report. The
CAP should set forth the specific action items, and targeted completion dates, necessary to
implement final corrective actions on the finding and recommendation contained in this final
audit report.

In accordance with the Inspector General Act of 1978, as amended, the Office of Inspector
General is required to report to Congress twice a year on the audits that remain unresolved after
six months from the date of issuance.

Statements that managerial practices need improvements, as well as other conclusions and
recommendations in this report represent the opinions of the Office of the Inspector General.
Determinations of corrective action to be taken will be made by the appropriate Department of
Education officials.

In accordance with the Freedom of Information Act (5 U.S.C. § 522), reports issued by the Office
of Inspector General are available to members of the press and general public to the extent
information contained therein is not subject to exemptions in the Act.



                                        ED-OIG/A19-E0006
Mr. Justesen                                                                   Page 7 of 7



We appreciate the cooperation provided to us during this review. Should you have any questions
concerning this report, please call Michele Weaver-Dugan at (202) 863-9526. Please refer to the
control number in all correspondence related to the report.


                                       Sincerely,



                                       Helen Lew
                                       Assistant Inspector General for Audit Services


Attachments




                                      ED-OIG/A19-E0006
                                                                                        Attachment 1


       Guidelines for Establishing File Folders & Maintaining Documentation 

                                 For External Audits 

                            (Effective September 1, 2002)


The following procedures are set forth as guidelines for establishing file folders and
maintaining accurate and complete documentation on all actions taken to resolve findings of
external audits of ED programs.

   1.	 An official audit resolution file folder should be established for each audit report.

   2.	 Each file folder should contain, at a minimum, the following documents:

           •	 The Federal Audit Clearinghouse’s audit cover sheet titled “Audit Description
              Data”
           •	 Copy of the CARS generated “Summary of Findings Requiring Resolution”
           •	 Copy of the audit report or pages of the audit report that provide relevant
              information to the resolution of the audit findings, including the findings, the
              auditee’s corrective action plan or response to the findings, the section on the
              status of prior year findings, and the ED portion of the Schedule of Expenditures
              of Federal Awards
           •	 A listing of the triage decisions for each audit finding
           •	 Documentation of all correspondence and communication with the auditee, the
              auditor, and other appropriate individuals, including corrective action plans and
              necessary work papers
           •	 Copy of the PDL
           •	 Copy of the Audit Clearance Document (ACD)
           •	 All documentation pertaining to audit follow-up activities, e.g., documentation
              from the auditee substantiating the corrective action taken, results of any
              monitoring visits, relevant information from the next year’s audit that reports
              whether appropriate corrective action was taken on a prior year finding.
           •	 Documented evaluations or conclusions of the [Principal Office (PO)] that
              support the adequacy of the corrective actions taken by the auditee, if not
              included in the PDL and/or occurring after the PDL is issued

   3. Each official file folder should also contain, as appropriate, the following documents:

           •	 Documented evidence of technical assistance provided
           •	 [Office of General Counsel (OGC)] and ED-OIG comments
           •	 ED-OIG concurrence/non-concurrence of PDLs for all audits issued by ED-OIG
              or in which the audit has questioned costs of $500,000 or more
           •	 In the event an Administrative Stay has been requested and approved, all
              documents pertaining to the request for an Administrative Stay, e.g., the request
              and approval memoranda
           •	 In the event an auditee requests a grantback, all documentation pertaining to the
              grantback
                                                                                        Attachment 2




MEMORANDUM

TO:	            Michele Weaver-Dugan, Director
                Operations Internal Audit Team
                Office of Inspector General

FROM: 	         Troy R. Justesen, Ed.D.
                Acting Deputy Assistant Secretary
                Office of Special Education
                and Rehabilitative Services


SUBJECT:	       Comments Regarding ED-OIG/A19-E0006 (Draft Audit Report - OSERS’
                Follow-up for External Audits)


The Office of Special Education and Rehabilitative Services (OSERS) would like the following
comments to be considered prior to the Office of Inspector General (OIG) issuance of the above-
referenced draft audit report in final. The audit report concerns OSERS follow-up activities for
corrective actions specified in external audits issued by the OIG. This audit is part of a review of
the audit follow-up process for
OIG external audits being performed in several principal offices. Specifically, the audit report
concludes that documentation was not always maintained in the file to support that the corrective
actions undertaken by the auditee were completed.

OSERS concurs with OIG’s conclusions reached in this audit. In addition to highlighting the
steps that will be instituted in response to the specific findings in the Audit Report, we would like
to point out the actions that OSERS initiated immediately after the issuance of Arizona,
California, and Michigan Performance Data audits intended to increase the quality of collecting,
reviewing and reporting performance data for students exiting school, discipline, and personnel.
At the Annual Data Manager’s Meeting held March 30 through April 2, 2002, OSERS staff,
during certain sessions, highlighted particular issues that OIG identified as problematic in these
audits as part of the effort to respond to these audits.
Page 2 – Ms. Michele Weaver-Dugan




Immediate steps that OSERS intends to implement in response to the recommendations in the
Draft Audit Report include:

    •	 Assign staff at the Office of Assistant Secretary level to coordinate and review the
       resolution of both external and internal OIG Audits to ensure that proper follow-up
       activities are satisfactorily completed prior to closing the audit.
    •	 Develop and OSERS-wide process and appropriate internal procedures for obtaining and
       maintaining documentation that supports implementation of corrective actions.

If you have any further questions, please contact Ms. Amy Egan of my staff at (202) 205-4746.