oversight

The U.S. Department of Education's Oversight of the Carl D. Perkins Career and Technical Education Improvement Act of 2006 Program

Published by the Department of Education, Office of Inspector General on 2016-03-28.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                            UNITED STATES DEPARTMENT OF EDUCATION
                                         OFFICE OF INSPECTOR GENERAL

                                                                                                      AUDIT SERVICES




                                                    March 28, 2016

                                                                                         Control Number
                                                                                         ED-OIG/A05P0002


Dr. Johan Uvin
Acting Assistant Secretary
Office of Career, Technical, and Adult Education
U.S. Department of Education
400 Maryland Avenue, SW
Washington, DC 20202

Dear Dr. Uvin:

This final audit report, “The U.S. Department of Education’s Oversight of the Carl D. Perkins
Career and Technical Education Improvement Act of 2006 Program,” presents the results of our
audit. The purpose of this report is to provide the results of our assessment of the adequacy of
selected aspects of the oversight of the Carl D. Perkins Career and Technical Education
Improvement Act of 2006 (Perkins IV) program provided by the U.S. Department of Education
(Department), Office of Career, Technical, and Adult Education (OCTAE). The objectives of
our audit were to determine whether OCTAE developed and implemented control activities that
provided reasonable assurance that

         1. States submitted reliable Perkins IV program performance data to the Department and

         2. States and subrecipients took corrective action when the Department or others
            identified unreliable Perkins IV program performance data or inadequate Perkins IV
            program performance results. 1

To answer our objectives, we evaluated the adequacy of the design of OCTAE’s control
activities as of January 26, 2015. We then judgmentally selected 5 States from the universe of
55 States to evaluate the operating effectiveness of OCTAE’s control activities. 2 Because we did
1
  We use “States” to refer to the 57 grantees (50 States, the District of Columbia, Puerto Rico, Guam, Palau,
American Samoa, Northern Mariana Islands, and the Virgin Islands) that received a Perkins IV program allocation
for fiscal year 2014.
2
  Although American Samoa and the Northern Mariana Islands received Perkins IV program funds, OCTAE did not
assign either outlying area a risk ranking in fiscal year 2014; therefore, we did not consider either outlying area as
part of the universe when selecting our sample of five States (see “Sampling Methodology” in the “Objectives,
Scope, and Methodology” section of this report).

    The Department of Education's mission is to promote student achievement and preparation for global competitiveness by
                                 fostering educational excellence and ensuring equal access.
Final Report
ED-OIG/A05P0002                                                                          Page 2 of 18

not select the States as part of a statistical sample, our results might not be representative of the
entire universe and, therefore, cannot be projected to the universe. For the first objective, we
reviewed Perkins IV program performance data included in the five States’ most recent
consolidated annual reports. We also evaluated OCTAE’s processes for ensuring that States
submitted reliable performance data to the Department. For the second objective, we reviewed
the corrective actions that OCTAE required the five States to implement (1) to resolve the
findings reported in OCTAE’s most recent monitoring reports, (2) to resolve the Perkins IV
program findings reported in the five States’ most recent single audits, and (3) for not meeting
performance goals for at least 3 consecutive years. We also evaluated OCTAE’s processes for
ensuring that States took steps to address deficiencies noted in monitoring and audit reports and
to improve performance.

For objective 1, we concluded that, as of January 26, 2015, OCTAE had developed and
implemented control activities that provided reasonable assurance that States submitted reliable
Perkins IV program performance data to the Department. We did not identify any significant
areas for improvement but suggest that OCTAE consider requiring auditors conducting single
audits to evaluate the adequacy of a State’s system of internal control, including information
systems controls, over data reporting. For objective 2, we concluded that, as of January 26,
2015, OCTAE had developed and implemented control activities that provided reasonable
assurance that States and subrecipients took corrective action when the Department or others
identified unreliable Perkins IV program performance data or inadequate Perkins IV program
performance results. However, OCTAE could strengthen its control activities by ensuring that it
adheres to Department policies and procedures for obtaining and retaining monitoring and
oversight documentation.

In its comments on the draft of this report, OCTAE agreed with the results for objective 1 and
agreed to consider the suggestion. OCTAE generally agreed with the results for objective 2 and
the finding. However, OCTAE clarified that the documentation retention failure occurred as a
result of the Department’s transfer of computer files to a cloud-based server, not as a result of
employee recordkeeping practices. OCTAE did not state whether it agreed with the
recommendation but told us that it has recovered the missing files and placed them on its
centralized drive. The full text of OCTAE’s comments on the draft of this report is included as
Attachment 2.

We updated the finding to include OCTAE’s clarification and revised the recommendation.




                                        BACKGROUND


The Perkins IV program is the principal source of Federal funding for the improvement of
secondary and postsecondary career and technical education programs. The program provides
about $1 billion annually to 50 States, American Samoa, the District of Columbia, Guam, the
Northern Mariana Islands, Palau, Puerto Rico, and the Virgin Islands. Perkins IV requires the
Final Report
ED-OIG/A05P0002                                                                        Page 3 of 18

Department to distribute funds to States through a formula based on each State’s populations in
certain age groups and per capita income. Perkins IV requires each State to distribute at least
85 percent of its program funds by formula to local educational agencies, vocational and
technical schools, community colleges, and other public or private nonprofit institutions that
offer career and technical education programs. Perkins IV does not allow States to spend more
than 15 percent of their program funds on administrative and State leadership activities.

Perkins IV requires each State to annually report aggregated data on 13 core indicators of
the performance of all students, disaggregated data on the performance of students in special
population categories, and disaggregated data on the performance of students by gender, race,
and ethnicity. The seven required core indicators of performance at the secondary level were
(1) academic attainment in reading, language arts, and mathematics; (2) technical skill
attainment; (3) secondary school completion; (4) student graduation rates; (5) secondary
placement; (6) nontraditional participation; and (7) nontraditional completion. The six required
core indicators of performance at the postsecondary level were (1) technical skill attainment;
(2) credential, certificate, or degree completion; (3) student retention or transfer; (4) student
placement; (5) nontraditional participation; and (6) nontraditional completion.

OCTAE was responsible for overseeing States’ administration of the Perkins IV and
Workforce Investment Act grant programs; providing assistance to States to improve program
quality, implementation, and accountability; and establishing national initiatives to help States
implement career and technical education programs. OCTAE required States receiving
Perkins IV program funds to annually submit a consolidated annual report by December 31.
The report had to include descriptions of performance and financial status, as well as the
secondary and postsecondary core indicators of performance. States used the Department’s
Perkins Information Management System to submit their consolidated annual reports. OCTAE
also required each State’s Perkins IV director and financial officer to attest to the completeness
and accuracy of the data reported in the consolidated annual report.

To fulfill its oversight responsibilities, OCTAE conducted onsite and virtual monitoring of States
receiving Perkins IV program funds. OCTAE selected States to monitor using a risk assessment
process that it developed in consultation with the Department’s Risk Management Service.
As part of the risk assessment process, OCTAE considered factors such as the last time that
it conducted a monitoring visit and each State’s current fiscal status, prior single audit findings,
grant award conditions, and Perkins IV program performance data. To prepare for an onsite
monitoring visit, OCTAE’s monitoring team members reviewed the State’s consolidated annual
report. They also held a meeting to discuss any findings identified the last time OCTAE
monitored the State, any single audit findings pertaining to the Perkins IV program, and any
concerns OCTAE had with the State’s program performance data. OCTAE recorded the results
of its monitoring visits and noted any deficiencies in States’ compliance with Perkins IV
requirements. OCTAE then worked with States to correct the deficiencies.
Final Report
ED-OIG/A05P0002                                                                      Page 4 of 18


                                     AUDIT RESULTS


The objectives of our audit were to determine whether OCTAE developed and implemented
control activities that provided reasonable assurance that

       1. States submitted reliable Perkins IV program performance data to the Department and

       2. States and subrecipients took corrective action when the Department or others
          identified unreliable Perkins IV program performance data or inadequate Perkins IV
          program performance results.

We evaluated OCTAE’s control activities as of January 26, 2015.

In its comments on the draft of this report, OCTAE agreed with the results for objective 1 and
agreed to consider our suggestion that it consider requiring auditors conducting single audits
to evaluate the adequacy of a State’s system of internal control, including information systems
controls, over data reporting. OCTAE generally agreed with the results for objective 2 and the
finding. However, OCTAE clarified that the documentation retention failure occurred as a result
of the Department’s transfer of computer files to a cloud-based server, not as a result of
employee recordkeeping practices. OCTAE did not state whether it agreed with the
recommendation but told us that it has recovered the missing files and placed them on its
centralized drive. The full text of OCTAE’s comments on the draft of this report is included as
Attachment 2. We did not include the documents that OCTAE provided with its comments on
the draft of this report. However, we will provide the documents on request.

We updated the finding to include OCTAE’s clarification and revised the recommendation.

Results for Objective 1: Perkins IV Program Performance Data Reporting
As of January 26, 2015, OCTAE had developed and implemented control activities that provided
reasonable assurance that States submitted reliable Perkins IV program performance data to the
Department. These control activities included (1) using computerized data input controls and
validation techniques, (2) providing training to States, (3) analyzing the information in
consolidated annual reports for possible inaccuracies, (4) conducting onsite or virtual monitoring
of States, and (5) discussing any findings identified the last time OCTAE monitored the State
and any single audit findings pertaining to Perkins IV. We did not identify any control activities
that needed improvement.

To ensure that States reported complete and accurate Perkins IV program performance data in
their consolidated annual reports, OCTAE used computerized data input controls and validation
techniques. States submitted their consolidated annual reports through the Department’s Perkins
Information Management System. The system had pulldown menus, automatic computations,
and edit checks. Although States submitted postsecondary performance data directly through the
Perkins Information Management System, the secondary performance data shown in the States’
Final Report
ED-OIG/A05P0002                                                                             Page 5 of 18

consolidated annual reports came from the Department’s EDFacts Submission System. States
entered secondary Perkins IV performance data into the EDFacts Submission System. The
EDFacts Submission System ran edit checks to determine whether the data were (1) entered in
the correct file format, (2) valid, and (3) reasonable. OCTAE could not upload the performance
data into the consolidated annual report unless the EDFacts Submission System showed that the
data were error-free. OCTAE required each State’s Perkins IV director to provide the following
attestation:

       I certify to the best of my knowledge and belief that this report, consisting of narrative
       performance information, financial status reports (FSRs)*, and performance data, is
       accurate and complete. I understand that the U.S. Department of Education will use only
       the performance data that it receives by the December 31 submission deadline each year
       to determine whether my State has met at least 90 percent of its agreed upon State
       adjusted performance levels for each of the core indicators of performance under section
       113 of Title I of [Perkins IV] or whether the State must submit a program improvement
       plan as required in section 123(a)(1) of Perkins IV. I further understand that the use of
       the Personal Identification Number (PIN) supplied to me by the Department to certify
       and submit the CAR is the same as certifying and signing the document with a hand-
       written signature.

       *Note: The FSRs contained in this report must be separately certified and signed by the
       State official authorized by State law to perform these functions on behalf of the State.
       This official may use a separate PIN supplied to the State by the Department to certify
       and submit the FSRs.

To help States understand the process for submitting their consolidated annual reports, OCTAE
published instructions for preparing the consolidated annual report. The instructions covered
topics such as submission deadlines, requests for extensions, amended performance data
submissions, and forms required to be included in each State’s consolidated annual report
submission. OCTAE also annually hosted two training sessions on preparing the consolidated
annual report. Further, it provided training on data collection, use, and quality. The training
included information on how to use data to enhance student outcomes, processes to transform
programs for underrepresented students, and strategies to analyze individual and composite
subrecipient data.

To identify possible inaccuracies in the data that States reported in their consolidated annual
reports, OCTAE first reviewed the information in the States’ consolidated annual reports. If
necessary, OCTAE asked States to clarify or correct data that appeared inaccurate. OCTAE also
used analytical procedures, such as comparing a State’s data from one reporting year to the next,
to identify trends and significant changes. If OCTAE found anything questionable, it contacted
the State for clarification or correction. For example, OCTAE found questionable elements in
the 2012–2013 consolidated annual reports from each of the five States covered by our audit.
OCTAE asked one State to describe how it served special populations during the program year.
OCTAE asked another State to explain in greater detail how it used Perkins IV funds to develop
or enhance data systems to collect and analyze data on secondary and postsecondary academic
employment outcomes.
Final Report
ED-OIG/A05P0002                                                                       Page 6 of 18

To evaluate the adequacy of the processes that States used to report performance data, OCTAE
conducted onsite monitoring visits to States or virtually monitored the States. Whether
monitoring was conducted onsite or virtually, OCTAE followed the steps in a monitoring visit
form that it created. OCTAE designed the monitoring visit form to help reviewers gain an
understanding of a State’s processes for ensuring that the data subrecipients submitted to the
State were reliable. For example, the monitoring visit form included questions about how States
ensured that their subrecipients submitted reliable data and had examples of the types of
evidence that States could provide to demonstrate that their subrecipients were providing reliable
data. As part of the monitoring, OCTAE also sampled subrecipients and compared the
subrecipients’ data to the corresponding data in the State’s and Department’s reports. OCTAE
employees used the monitoring visit form to document the results of their monitoring.

Finally, OCTAE considered the last time it monitored the State and whether the State had any
single audit findings pertaining to the Perkins IV program. For the single audit, OCTAE ensured
that the compliance supplement directed auditors to test a sample of cells on the State’s
consolidated annual report to ensure that the State had data that supported the numbers in the
report. For the five States included as part of our audit, we did not identify any findings related
to unreliable Perkins IV program performance data in either OCTAE’s monitoring reports or in
single audits reports. However, OCTAE might attain enhanced assurances that States’
self-certified data are complete and accurate if it required auditors conducting single audits
to evaluate not only a sample of cells on the consolidated annual report but also the adequacy of
a State’s system of internal control, including information systems controls, over data reporting.

Auditee Comments

OCTAE agreed with the results for objective 1 and agreed to consider our suggestion that it
consider requiring auditors conducting single audits to evaluate the adequacy of a State’s system
of internal control, including information systems controls, over data reporting.

Results for Objective 2: Implementation of Corrective Actions
As of January 26, 2015, OCTAE had developed and implemented control activities that provided
reasonable assurance that States and subrecipients took corrective actions when the Department
or others identified unreliable Perkins IV program performance data or inadequate Perkins IV
program performance results. OCTAE’s control activities included (1) imposing more stringent
reporting requirements on States that did not meet performance goals, (2) verifying that States
implemented corrective actions for findings identified in single audit reports, and (3) verifying
that States and subrecipients implemented corrective actions for findings identified in OCTAE’s
monitoring reports. However, we concluded that OCTAE could strengthen its control activities
by ensuring that it adheres to Department policies and procedures for obtaining and retaining
monitoring and oversight documentation.

To provide reasonable assurance that States took corrective actions when they did not meet
Perkins IV program performance goals, OCTAE imposed more stringent reporting requirements.
For example, if a State did not meet performance goals for more than 3 consecutive years,
OCTAE required the State to submit quarterly progress reports. The quarterly progress reports
included descriptions of the actions that the State planned to take to improve performance, the
Final Report
ED-OIG/A05P0002                                                                       Page 7 of 18

timelines for completing the actions, the people responsible for implementing the corrective
actions, and any completed actions. In 2014, OCTAE required four of the five States covered by
our audit to submit quarterly progress reports because they did not meet their Perkins IV
program performance goals for more than 3 consecutive years.

To provide reasonable assurance that States took corrective actions to adequately resolve
Perkins IV program findings identified in single audits, OCTAE employed audit resolution
specialists. OCTAE’s audit resolution specialists contacted the States and tracked their
corrective actions planned or taken, targeted and actual completion dates, and any other
information that could affect the resolution of the single audit findings through the Department’s
Audit Accountability Resolution Tracking System. Once OCTAE confirmed that a State had
taken steps sufficient to address the issues of a particular single audit finding, the Assistant
Secretary for OCTAE issued a program determination letter to inform the State that the finding
was resolved and closed.

To provide reasonable assurance that States and subrecipients took corrective action for
Perkins IV program findings identified in OCTAE’s monitoring reports, OCTAE monitored the
implementation of States’ and subrecipients’ corrective actions for monitoring visit findings and
worked with States to correct the deficiencies. OCTAE tracked the resolution of its monitoring
visit findings by using a chart that included the finding, the OCTAE employee responsible for
following up on the finding, the due date for the corrective action, and whether the finding was
resolved. OCTAE provided more training and assistance to States that struggled to implement
their corrective actions.

Although it developed and implemented control activities that provided reasonable assurance
that States and subrecipients took corrective action when the Department or others identified
unreliable Perkins IV program performance data or inadequate Perkins IV program performance
results, OCTAE did not always retain documentation that was sufficient for us to verify that it
fully implemented the control activities.

FINDING NO. 1 – OCTAE Did Not Retain All Required Documentation
OCTAE did not retain all required documentation pertaining to its monitoring of States’
implementation of corrective actions for Perkins IV program findings that OCTAE identified in
its monitoring reports issued on or before January 26, 2015. We reviewed the most recent
monitoring reports that OCTAE issued, monitoring documentation that OCTAE retained, and
States’ corrective action plans for 5 (9 percent of the 55 States that received Perkins IV program
funds) judgmentally selected States (see “Sampling Methodology” in the “Objectives, Scope,
and Methodology” section of this report). We found that OCTAE sufficiently documented that
three of the five States implemented all required corrective actions. For all three States, OCTAE
issued letters to the States explaining that it accepted the States’ corrective actions and
considered the findings resolved. OCTAE did not provide us with sufficient documentation to
determine whether the other two States implemented all required corrective actions.

For one of the two States, OCTAE did not provide us with a closeout letter stating that the State
had implemented the required corrective actions and that OCTAE considered the findings
resolved. In its monitoring report, OCTAE reported that the State’s secondary local application
Final Report
ED-OIG/A05P0002                                                                                         Page 8 of 18

did not (1) identify the levels of performance for each of the core indicators of performance and
(2) describe the process that the State would use to evaluate and continuously improve the
performance of eligible subrecipients. To address the findings, OCTAE required the State to
revise its local applications to include subrecipients’ levels of performance and to ensure that
subrecipients provided descriptions of how they would evaluate and continuously improve their
performance. OCTAE did not provide us with any evidence showing that the State implemented
both required corrective actions. However, when we visited this State and asked it for such
evidence, the State was able to provide us with its response and supporting documentation
sufficient to show that it implemented both corrective actions that OCTAE required.

For the other State, OCTAE provided us with its closeout letter and the State’s response
describing the steps that it had taken to address the deficiencies noted in OCTAE’s monitoring
report. In its monitoring report, OCTAE reported that the State did not require subrecipients to
(1) submit an annual local report identifying gaps or disparities in performance between any
category of students and all students and (2) provide an improvement plan with special
consideration to performance gaps. To address the findings, OCTAE required the State to revise
its annual local report and improvement plan templates to comply with the applicable sections of
Perkins IV. However, OCTAE did not provide us with any evidence showing that it ensured the
State took sufficient corrective actions to address the deficiencies noted in OCTAE’s monitoring
report. We did not ask this State to provide us with the supporting documentation showing the
steps it had taken to correct the deficiencies because we contacted only the two States that we
visited. 3 Instead of us contacting this State, we are bringing this instance of insufficient
documentation to OCTAE’s attention so it can obtain evidence that the State took the required
corrective actions and then retain that evidence or require the State to correct the deficiencies.

Office of Management and Budget Circular A-123, “Management’s Responsibility for Internal
Control,” December 21, 2004, states that management must establish and maintain internal
control to achieve the objectives of effective and efficient operations, reliable financial reporting,
and compliance with applicable laws and regulations. Management should continuously monitor
and improve the effectiveness of internal control associated with their programs and ensure it has
the appropriate level of documentation needed to support this assessment.

According to the Department’s Administrative Communications Systems Directive OS: 1-108,
“Guide for Managing State Administered Programs” (VIII)(D)(2), February 3, 2010, a formula
grant should be closed out only if (1) the single audits for the grant period have been completed,
all audit findings (including from Federal audits) have been resolved, corrective action plans are
in place, and amounts due back have been paid or a payment plan has been agreed to; and (2) no
issues that were identified in the program office’s monitoring reports are outstanding.

According to the Department’s Grant Bulletin 14-06, “Policy, Guidance, and Responsibilities for
Principal Office Monitoring Frameworks for Formula Grant Programs,” (VI)(H)(1),
April 28, 2014, program offices should maintain documentation on a grantee’s progress in


3
  We visited only the two States that received the largest Perkins IV allocations of the five selected States and had
different risk rankings (see “Sampling Methodology” in the “Objectives, Scope, and Methodology” section of this
report).
Final Report
ED-OIG/A05P0002                                                                                     Page 9 of 18

meeting performance standards and compliance with grant requirements. 4 At a minimum, the
program office should maintain (1) monitoring reports, (2) corrective action plans,
(3) communications with the grantee related to monitoring activities, (4) evidence of the actions
taken by a grantee to resolve and close findings in monitoring reports, (5) technical assistance
provided to help the grantee correct the findings of noncompliance identified in the monitoring
report, and (6) documents supporting key internal controls in the monitoring process.

The disposition of documentation on a grantee’s progress in meeting standards and compliance
with grant requirements should adhere to ED Records Schedule No.: 254, December 19, 2011,
“Grant Administration and Management Files.” According to ED Records Schedule No.: 254
and Grant Bulletin 14-06, Enclosure 1, formula grant documentation should be
“destroyed/deleted five years after closure of grant, completion of monitoring, audit resolution,
or final judicial determination, or until the grant program has been reauthorized, whichever is
later.” In 2014, the National Archives and Records Administration and the Department revised
their records schedules to increase the retention period for formula grant files to at least 10 years
after final action is taken on the file.

According to OCTAE, the documentation retention failure occurred as a result of the
Department’s transfer of computer files to a cloud-based server, not as a result of employee
recordkeeping practices. After commenting on the draft of this report, OCTAE informed us that
the monitoring documentation that was not available at the time of our audit work had been
recovered. OCTAE provided us with one State’s revised local application, which showed that
the State implemented both required corrective actions, but not a closeout letter stating that the
State had implemented the required corrective actions and that OCTAE considered the findings
resolved. Additionally, OCTAE provided the other State’s revised improvement plan but not the
State’s revised annual local report. OCTAE provided us with computer screen shots showing the
location of the recovered documentation on its centralized drive.

The goals of Perkins IV include an increased focus on the academic achievement of career and
technical education students, strengthening the connections between secondary and
postsecondary education, and improving State and local accountability. If OCTAE does not
maintain all required monitoring and oversight documentation, it cannot be sure that States
promptly address deficiencies noted in OCTAE’s monitoring reports. OCTAE also increases the
risk that States will not timely implement appropriate corrective actions or implement corrective
actions at all, which puts States at risk of not accomplishing the goals of Perkins IV.

Recommendation
We recommend that the acting Assistant Secretary for OCTAE—

1.1        Ensure that it adheres to Department policies and procedures for obtaining and retaining
           monitoring and oversight documentation.




4
    The Department issued Grant Bulletin 14-06 after OCTAE completed its monitoring reports on the two States.
Final Report
ED-OIG/A05P0002                                                                      Page 10 of 18

Auditee Comments

OCTAE generally agreed with the results for objective 2 and the finding. However, OCTAE
clarified that the documentation retention failure occurred as a result of the Department’s transfer
of computer files to a cloud-based server, not as a result of employee recordkeeping practices.
OCTAE stated that it has since recovered the missing files and placed them on its centralized
drive. OCTAE did not state whether it agreed with the recommendation.

OIG Response

We revised the finding and the recommendation. In the draft of this report, we stated that,
according to OCTAE, when the Department migrated OCTAE’s email and file systems to new
systems, employees lost some of the required monitoring documentation that they stored on their
government-furnished computers and had backed up on external drives. Based on OCTAE’s
comments on the draft of this report and additional documentation that OCTAE provided us, we
are revising the final report. We now state that required monitoring documentation was
misplaced during the Department’s transfer of computer files to a cloud-based server, not as a
result of employee recordkeeping practices. We also added that, after recovering the missing
monitoring documentation, OCTAE provided us with one State’s revised local application,
which showed that the State implemented both required corrective actions, but not a closeout
letter stating that the State had implemented the required corrective actions and that OCTAE
considered the findings resolved. Additionally, OCTAE provided the other State’s revised
improvement plan but not the State’s revised annual local report. OCTAE provided us with
computer screen shots of the location of the recovered documentation on its centralized drive.
Finally, we changed the recommendation to state that OCTAE, rather than the employees, should
adhere to Department policies and procedures for obtaining and retaining monitoring and
oversight documentation.




                  OBJECTIVES, SCOPE, AND METHODOLOGY


The objectives of our audit were to determine whether the OCTAE developed and implemented
control activities that provided reasonable assurance that

       1. States submitted reliable Perkins IV program performance data to the Department and

       2. States and subrecipients took corrective action when the Department or others
          identified unreliable Perkins IV program performance data or inadequate Perkins IV
          program performance results.

We evaluated OCTAE’s control activities as of January 26, 2015.
Final Report
ED-OIG/A05P0002                                                                                   Page 11 of 18

For the first objective, we reviewed Perkins IV program performance data included in the
2012–2013 consolidated annual reports submitted by five judgmentally selected States and the
Perkins IV program performance data included in the 2013–2014 consolidated annual report for
one of the five States (see “Sampling Methodology”). 5 We also evaluated OCTAE’s process for
ensuring that States submitted reliable Perkins IV program performance data to the Department.
For the second objective, we reviewed the corrective actions that OCTAE required the
five judgmentally selected States to implement (1) to resolve the findings reported in OCTAE’s
most recent monitoring reports (issued on or before January 26, 2015), (2) to resolve the
Perkins IV program findings reported in the five States’ 2012 and 2013 single audits, and (3) for
not meeting Perkins IV program performance goals for at least 3 consecutive years. We also
evaluated OCTAE’s process for ensuring that States (1) took steps to address deficiencies noted
in OCTAE’s monitoring report and single audit reports and (2) took steps to improve
performance when Perkins IV program performance goals were not met for more than
3 consecutive years.

To gain an understanding of the Perkins IV program and the requirements for performance data
reporting, we reviewed program information posted on the Department’s Web site, the Carl D.
Perkins Career and Technical Education Improvement Act of 2006, and Perkins IV program
memoranda and nonregulatory guidance. To gain an understanding of how OCTAE managed
Perkins IV program performance data reporting and State monitoring and the environment in
which OCTAE operated, we reviewed OCTAE’s policies and procedures that were related to
Perkins IV program performance data reporting and State monitoring. We also reviewed the
2012 and 2013 single audits for the five States covered by our audit to identify areas of potential
internal control weaknesses related to our audit objectives.

As part of our audit, we visited Arizona and Minnesota but did not visit Arkansas, Vermont, or
West Virginia. We visited Arizona and Minnesota because they received the largest Perkins IV
program allocations of the five selected States and had different risk rankings (see “Sampling
Methodology” in the “Objectives, Scope, and Methodology” section of this report). To gain an
understanding of how Arizona and Minnesota administered the Perkins IV program and the
environment in which they operated, we reviewed each State’s policies and procedures that were
related to Perkins IV program performance data reporting and subrecipient monitoring.

Internal Control
We gained an understanding of the control activities over Perkins IV program performance data
reporting that OCTAE developed and implemented. We interviewed OCTAE employees,
reviewed OCTAE’s organizational chart, reviewed OCTAE’s guidance documents and training
materials, and read OCTAE employees’ review notes on five States’ consolidated annual reports.
For two of the five States (Arizona and Minnesota), we interviewed State officials; reviewed the
States’ organizational charts, written policies and procedures, guidance, and training materials;


5
  When we began the audit, OCTAE had not yet accepted and finalized the five selected States’ 2013–2014
consolidated annual reports. Therefore, we reviewed the five selected States’ 2012–2013 consolidated annual
reports. When we began our work in one of the five selected States, the Department had accepted and finalized the
State’s 2013–2014 consolidated annual report, so we also reviewed that State’s 2013–2014 consolidated annual
report.
Final Report
ED-OIG/A05P0002                                                                      Page 12 of 18

and compared the Perkins IV program performance data in the States’ systems to their
consolidated annual reports.

We also gained an understanding of OCTAE’s control activities over States’ and subrecipients’
implementation of corrective actions to address Perkins IV program findings reported in
OCTAE’s monitoring reports. We also gained an understanding of OCTAE’s control activities
over State’s implementation of corrective actions to address findings reported in single audits
and for States not meeting Perkins IV program performance goals for more than 3 consecutive
years. We interviewed OCTAE officials and reviewed monitoring procedures, schedules, and
forms that OCTAE employees used to conduct and document the results of their monitoring. For
the five States covered by our audit, we reviewed grant awards for the 2014 award year;
OCTAE’s most recent monitoring reports issued on or before January 26, 2015; States’
corrective action plans; OCTAE’s monitoring report closeout letters; and any other relevant
monitoring documentation that OCTAE retained. For two (Arizona and Minnesota) of the
five States, we interviewed State officials and reviewed the procedures, schedules, and forms that
each State used to conduct and document their monitoring of subrecipients. For one State
(Arizona), we confirmed that a sample of subrecipients took corrective actions for Perkins IV
program findings. The most recent OCTAE monitoring report and single audit reports on the
other State (Minnesota) did not contain findings related to our audit objectives; therefore, we did
not select any Minnesota subrecipients.

Our review identified a minor deficiency in OCTAE’s implementation of the control activities
that it designed to provide reasonable assurance that States and subrecipients took corrective
actions when the Department or others identified unreliable Perkins IV program performance
data or inadequate Perkins IV program performance results. We found that OCTAE did not
retain all required monitoring documentation pertaining to the implementation of corrective
actions for Perkins IV program findings that it identified in its monitoring reports (see Finding
No. 1).

Data Reliability
To achieve our objectives, we relied on data from the Department’s Perkins Information
Management System, Arizona’s and Minnesota’s reporting systems, and Arizona’s accounting
system. We assessed the reliability of these data by looking for duplicate entries, invalid
identifiers, missing data, values outside a designated range, or values outside valid periods. We
also reconciled the performance data in Arizona’s and Minnesota’s reporting systems to the
corresponding performance data in the Department’s Perkins Information Management System.
Based on this assessment, we concluded that the data that the Department and two States
provided were sufficiently reliable for our intended use.

Sampling Methodology
To achieve our audit objectives, we judgmentally selected five States (Arizona, Arkansas,
Minnesota, Vermont, and West Virginia) that received Perkins IV program funds for fiscal
year 2014. We also selected a nonstatistical sample of 10 subrecipients (in 1 State) that received
Perkins IV program funds for fiscal year 2013.
Final Report
ED-OIG/A05P0002                                                                                     Page 13 of 18

Selection of States
We judgmentally selected 5 (9 percent) States from the universe of 55 States that received
Perkins IV program funds and received a risk ranking from OCTAE for fiscal year 2014.
OCTAE assigned risk rankings (“low,” “elevated,” or “significant”) to each of the 55 States.
OCTAE based each State’s ranking on factors such as missed performance targets, lapsed funds,
and the last time that OCTAE monitored the State. For fiscal year 2014, OCTAE assigned a
“low” ranking to 40 States and an “elevated” ranking to 15 States. 6 To select the States, we
considered the fiscal year 2014 risk rankings that OCTAE assigned, the amount of Perkins IV
program funds that each State received, and the percentage of each State’s population that was
rural. According to OCTAE’s Program Administration Branch Chief, Division of Academic and
Technical Education, rural States generally faced more challenges administering Perkins IV
programs because their target populations are widely dispersed from education centers. Of the
five States, we judgmentally selected two (Arizona and Minnesota) to visit. We selected Arizona
and Minnesota because they received the largest allocations of the five States and had different
risk rankings. Because we judgmentally selected the States, our sampling results might not be
representative of the entire universe and, therefore, cannot be projected to the universe.

Selection of Subrecipients
To verify that Arizona and its subrecipients implemented corrective actions for Perkins IV
program findings, we randomly selected a nonstatistical sample of 10 (9 percent) of the
109 subrecipients that received Perkins IV program funds for fiscal year 2013. 7 Minnesota’s
most recent monitoring and single audit reports did not contain findings related to our audit
objectives; therefore, we did not select any Minnesota subrecipients.

We conducted this audit from November 2014 through October 2015 in Washington, D.C.;
Phoenix, Arizona; Saint Paul and Roseville, Minnesota; and at our offices in Chicago, Illinois
and Kansas City, Missouri.

We discussed the results of our audit with OCTAE officials on December 2, 2015; with
Arizona officials on November 24, 2015; and with Minnesota officials on November 23, 2015.
We provided OCTAE with a draft of this report on January 25, 2016.

We reviewed OCTAE’s comments on the draft of this report. We also verified the validity of
OCTAE’s explanation for why required monitoring documentation was not available at the time
of our audit work by obtaining and reviewing one State’s revised local application, the second
State’s revised improvement plan, and computer screen shots showing the location of the
recovered documentation on OCTAE’s centralized drive.

We conducted this performance audit in accordance with generally accepted government
auditing standards. Those standards require that we plan and perform the audit to obtain
sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions
based on our audit objectives. We believe the evidence that we obtained provides a reasonable
basis for our findings and conclusions based on our audit objectives.

6
 OCTAE did not assign a risk ranking to American Samoa or the Northern Mariana Islands in fiscal year 2014.
7
 Because of the small sample size, our sampling results might not be representative of all Arizona subrecipients
and, therefore, cannot be projected.
Final Report
ED-OIG/A05P0002                                                                     Page 14 of 18


                            ADMINISTRATIVE MATTERS


Corrective actions proposed (resolution phase) and implemented (closure phase) by your office
will be monitored and tracked through the Department’s Audit Accountability and Resolution
Tracking System. The Department’s policy requires that you develop a final corrective action
plan for our review in the automated system within 30 calendar days of the issuance of this
report. The corrective action plan should set forth the specific action items, and targeted
completion dates, necessary to implement final corrective actions on the findings and
recommendations contained in this final audit report. An electronic copy of this report has been
provided to your Audit Liaison Officer.

In accordance with the Inspector General Act of 1978, as amended, the Office of Inspector
General is required to report to Congress twice a year on the audits that remain unresolved after
six months from the date of issuance.

Statements that managerial practices need improvements, as well as other conclusions and
recommendations in this report, represent the opinions of the Office of Inspector General.
Determinations of corrective action to be taken will be made by the appropriate Department of
Education officials.

In accordance with the Freedom of Information Act (5 U.S.C. §552), reports issued by the Office
of Inspector General are available to members of the press and general public to the extent
information contained therein is not subject to exemptions in the Act.

We appreciate the cooperation given us during this review. If you have any questions, please
contact me at 202-245-6949; Gary Whitman, Regional Inspector General for Audit, at 312-730-
1658; or Jonathan Enslen, Assistant Regional Inspector General for Audit, at 312-730-1625.



Sincerely,

/s/

Patrick Howard
Assistant Inspector General for Audit


Attachments
Final Report
ED-OIG/A05P0002                                                              Page 15 of 18

                                                                       ATTACHMENT 1


             Acronyms, Abbreviations, and Short Forms Used in This Report


Department          U.S. Department of Education

OCTAE               Office of Career, Technical, and Adult Education

Perkins IV          Carl D. Perkins Career and Technical Education Improvement Act of 2006
Final Report
ED-OIG/A05P0002                                                                   Page 16 of 18




        Attachment 2: OCTAE’s Comments on the Draft of This Report

OCTAE provided the following comments on the draft of this report on March 8, 2016. In
addition, on March 16, 2016, and March 18, 2016, OCTAE provided one of the State’s revised
improvement plan, the other State’s revised local application, and computer screen shots
showing the location of the recovered documentation on its centralized drive. We will provide
the documents on request.
Final Report
ED-OIG/A05P0002                                                                     Page 17 of 18




Gary D. Whitman                                                            March 7, 2016
Regional Inspector General for Audit
Office of Inspector General
U. S. Department of Education
Citigroup Center
500 West Madison Street, Suite 1414
Chicago, Illinois 60661

Dear Mr. Whitman:

Thank you for the opportunity to review and comment on the draft audit report, “The U. S.
Department of Education’s Oversight of the Carl D. Perkins Career and Technical Education
Improvement Act of 2006 Program, ED-OIG/A05P0002,” which provides the results of the
Office of Inspector General’s (OIG) audit of selected aspects of the Office of Career, Technical,
and Adult Education's (OCTAE) control activities as of January 26, 2015. The specific purpose
of the audit was to determine whether OCTAE developed and implemented control activities to
provide reasonable assurance that:

   1. States submit reliable Perkins IV program performance data to the Department; and
   2. States and sub-recipients took corrective actions when the Department or others
      identified unreliable Perkins IV program performance data or inadequate Perkins IV
      program performance data.

Below is our response to your finding and recommendations.

Response to Objective 1

We were encouraged by your conclusion that, as of January 26, 2015, OCTAE had developed
and implemented control activities to provide reasonable assurance that States submitted reliable
Perkins IV program data. We were also encouraged that you did not identify any significant
areas for improvement. At the same time, we will consider your suggestion to attain enhanced
assurances that States’ self-certified data are complete and accurate by requesting or encouraging
auditors conducting single audits to evaluate not only a sample of cells on the Consolidated
Annual Report (CAR), but also the adequacy of a State’s system of internal controls, including
information systems controls, over data reporting.
Final Report
ED-OIG/A05P0002                                                                       Page 18 of 18

Response to Objective 2

We were encouraged by your conclusion that, as of January 26, 2015, OCTAE had developed
and implemented control activities that provided reasonable assurance that States and
subrecipients took corrective action when the Department or others identified unreliable
Perkins IV program performance data or inadequate Perkins IV program performance results.
We would like to clarify and update facts related to OIG’s finding on Objective 2. The draft
audit report provides on page 8 that “[a]lthough we could not confirm the reason for the loss of
required documentation, OCTAE could have prevented the loss had it required all documentation
relevant to monitoring visits to be stored on a centralized file server, not on individual
employee’s computers.” Since the audit, we discovered that this file retention failure resulted
from the transfer of files across the Department from a centralized K drive to a cloud-based
server. During this file migration, some of OCTAE’s Perkins files were misplaced. After further
investigation, we were able to locate the missing files, which are now properly filed on
OCTAE’s centralized K drive. We would like to clarify that the loss of files was not due to their
storage on an individual employee’s hard drive, as indicated in the draft audit report, but rather
due to the overall file migration. Accordingly, we request that the OIG revise or adjust its facts
pursuant to this finding.

Thank you for your thoughtful and thorough review of our office’s Perkins IV control activities.
We greatly appreciate the opportunity to look critically at our processes and ensure that all
employees adhere to Department policies and procedures for retaining monitoring and oversight
documentation. We take seriously our fiduciary responsibility in safeguarding Federal funds and
ensuring that States and discretionary grantees effectively comply with Federal laws and
applicable Department regulations.

If you have further input or questions regarding this response to your draft report, please contact
Edward R. Smith, Program Administration Branch Chief, at 202-245-7602 or
edward.smith@ed.gov.

                                      Sincerely,

                                      Johan E. Uvin,
                                      Deputy Assistant Secretary,
                                      Delegated the Duties of the Assistant Secretary for
                                      Career, Technical, and Adult Education