oversight

Review of Title III Program, HEA, Compliance with GPRA Requirements for Implementation of Performance Indicators.

Published by the Department of Education, Office of Inspector General on 2000-06-30.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                    Review of Title III Program, HEA,
                 Compliance with GPRA Requirements for
                 Implementation of Performance Indicators




                    FINAL AUDIT REPORT




                                          ED-OIG / A04-90014
                                              June 2000



Our mission is to promote the efficient                           U.S. Department of Education
and effective use of taxpayer dollars                          Office of Inspector General
in support of American education                               Atlanta, Georgia
                            NOTICE

 Statements that management practices need improvement, as
 well as other conclusions and recommendations in this report,
   represent the opinions of the Office of Inspector General.
 Determination of corrective action to be taken will be made by
          appropriate Department of Education officials.

  In accordance with the Freedom of Information Act (5 U.S.C.
   §552), reports issued by the Office of Inspector General are
  available, if requested, to members of the press and general
public to the extent information contained therein is not subject to
                       exemptions in the Act.
                         UNITED STATES DEPARTMENT OF EDUCATION

                                           OFFICE OF INSPECTOR GENERAL


                                                                                       THE INSPECTOR GENERAL


                                                  June 30, 2000

MEMORANDUM


TO:                Dr. Lee Fritschler
                   Assistant Secretary
                   Office of Postsecondary Education

FROM:              Lorraine Lewis

SUBJECT:           Final Audit Report
                   Review of Title III Program, HEA, Compliance with GPRA
                   Requirements for Implementation of Performance Indicators
                   ED-OIG/A04-90014

We conducted a review to determine if the Title III Program of Higher Education Programs (HEP): (1)
has met the Government Performance and Results Act (GPRA) requirements for developing Title III
Program performance indicators and (2) has developed an accountability system for reporting on the
indicators. This report presents our findings in detail.

The results of our review indicate that HEP may be unable to satisfy the GPRA requirement to report on
the performance of the Title III Program in FY 2000. In addition, HEP officials may be unable to assert
that the data used for performance measurement are reliable and valid.1

We found that performance indicators have been developed along with proposed methods for
measuring the indicators. However, HEP did not follow U.S. Department of Education (ED) guidelines
suggested for developing performance indicators, and the system used to obtain and compile data for
reporting on the indicators is not adequate.


1
 Objective 4.7 of ED’s Strategic Plan, Performance Indicator 30, states that by 2000, all ED program managers will
assert that the data used for their program’s performance measurement are reliable and valid or will have plans for
improvement.


                         400 MARYLAND AVE., S.W. WASHINGTON, D.C. 20202-1510


    Our mission is to ensure equal access to education and to promote educational excellence throughout the Nation.
Although these conditions exist, in part, because of limited administrative resources, there is no
assurance that the performance indicators and data sources will properly measure Title III Program
performance.

We received and reviewed comments from the Director of Institutional Development & Undergraduate
Education Services (IDUES). The comments show concurrence with the findings and recommendations
in our draft report. The Office of Inspector General finds the draft report responses satisfactory to
begin addressing our recommendations.

In accordance with Office of Management and Budget Circular A-50, we will keep this audit report on
the OIG list of unresolved audits until all open issues have been resolved. Any reports unresolved after
180 days from the date of issuance will be shown as overdue in the OIG’s Semiannual Report to
Congress.

Accordingly, please provide the Supervisor, Post Audit Group, Financial Improvement and Post Audit
Operations, Office of the Chief Financial Officer and the Office of Inspector General’s Acting Assistant
Inspector General for Audit Services with semiannual status reports. These reports should address
promised corrective actions until all such actions have been completed or continued follow-up is
unnecessary.

In accordance with the Freedom of Information Act (Public Law 90-23), reports issued by the Office
of Inspector General are available, if requested, to members of the press and general public to the
extent information contained therein is not subject to exemptions in the Act. Copies of this audit report
have been provided to the offices shown on the distribution list enclosed in the report.

We appreciate the cooperation given during the review. If you have any questions or wish to discuss
the contents of this report, please call Carol Lynch, Regional Inspector General for Audit, at (404) 562-
6462. Please refer to the above control number in all correspondence relating to this report.

Attachments

cc: Claudio Prieto, Deputy Assistant Secretary for Higher Education Programs
   Alan Ginsburg, Director of Planning and Evaluation Service
    Judith Winston, General Counsel
    Harold Jenkins, Assistant General Counsel
                                TABLE OF CONTENTS



EXECUTIVE SUMMARY                                                           1

AUDIT RESULTS

  FINDING NO. 1

     Title III Program Officials Did Not Follow Suggested Guidelines
     in Drafting FY 2000 Performance Indicators for the Title III Program   3

  FINDING NO. 2

     Title III Program Did Not Have an Adequate System to Obtain and
     Compile Grantee Data Needed for Measuring Performance for the First
     GPRA Report Due March 31, 2000, and Did Not Meet ED's Internal
     Deadline for Updating Performance Plans                                7

BACKGROUND                                                                  10

OBJECTIVES, SCOPE, AND METHODOLOGY                                          11

STATEMENT ON MANAGEMENT CONTROLS                                            12

ATTACHMENT A

     Pages 144 and 145 of the U.S. Department of Education FY 2000
     Annual Plan, Volume 2

ATTACHMENT B

     Auditee's Response with Attachments
                                   EXECUTIVE SUMMARY

We conducted a review to determine if the Title III Program of Higher Education Programs (HEP) (1)
has met the Government Performance and Results Act (GPRA) requirements for developing Title III
Program performance indicators and (2) has developed an accountability system for reporting on the
indicators.

The results of our review indicate that HEP may be unable to satisfy the GPRA requirement to report on
the performance of the Title III Program in Fiscal Year (FY) 2000. HEP officials may be unable to
assert that the data used for performance measurement are reliable and valid. Specifically, our audit
identified that HEP:

•   Did not follow the Department of Education's (ED) recommended guidelines for establishing a
    working group of essential personnel to develop the FY 2000 GPRA-required performance
    indicators for the Title III Program.

•   Did not have an adequate system to obtain and compile data needed for measuring performance.
    Furthermore, HEP did not have any staff assigned to this task, and the data format in the reports
    may not be conducive to deriving performance statistics.

In addition, ED added several new requirements for the GPRA performance plan and report. ED
asked program offices to set up new targets (performance indicators) for the years 1999, 2000, and
2001 by December 3, 1999. However, the Title III Programs' Institutional Development and
Undergraduate Educational Service (IDUES) did not meet this deadline. IDUES did not submit its
revised plan for its first GPRA performance report until January 2000.

Although these conditions exist, in part, because of limited administrative resources, there is no
assurance that the performance indicators and data sources will properly measure Title III Program
performance.

In order to assure that developed indicators adequately measure Title III Program performance and are
in compliance with GPRA, the Office of Postsecondary Education should require HEP to:

•   Develop future Title III Program performance indicators in accordance with the “Guide to Program
    Outcome Measurement for the U.S. Department of Education.”

•   Document how the Title III Program performance indicators will demonstrate progress toward
    achieving the mission of the Title III Program.




                                                                                ED-OIG / A04-90014
•   Continue development of a strategic plan to (a) ensure that its strategic goals are congruent with
    those contained in OPE’s plan and (b) identify useful benchmarks for what its performance
    indicators should assess.

In order to assure that needed performance data are properly obtained and compiled to demonstrate
performance under GPRA, the Office of Postsecondary Education should require HEP to:

•   Develop a system for collecting and aggregating the data needed for reporting on the Title III
    Program performance indicators in ED’s Annual Performance Plan.

•   Assure that the annual and final grantee performance reports are formatted in a manner conducive to
    providing needed data for the report on Title III Program performance indicators required by
    GPRA.

•   Develop a plan for assuring that the data collected are reliable and valid, and available in a timely
    manner.

•   Identify alternate sources of data, such as a standardized form with information on the performance
    indicators. The form could be included in the annual performance reports from grantees and
    transmitted electronically, or converted to electronic form.


Auditee's Response and Auditor Comments

We received and reviewed comments from the Director of IDUES. The comments show concurrence
with the findings and recommendations in our draft report. The Office of Inspector General finds the
draft report responses satisfactory to begin addressing our recommendations.




                                                  Page 2                            ED-OIG / A04-90014
                                         AUDIT RESULTS


FINDING NO. 1           TITLE III PROGRAM OFFICIALS DID NOT FOLLOW
                        SUGGESTED GUIDELINES IN DRAFTING FY 2000
                        PERFORMANCE INDICATORS FOR THE TITLE III PROGRAM

HEP did not follow ED’s recommended guidelines for establishing a working group of essential
personnel to develop the FY 2000 GPRA-required performance indicators for the Title III Program.
Essential component officials were not involved in developing the performance indicators, including the
current IDUES Director, current Title III Program Office staff, and members of the Program Monitoring
and Information Technology (PMIT) staff.

We noted that there was limited documentation detailing how the Title III Program performance
indicators were developed and how the indicators will demonstrate Title III Program achievement.
Also, ED officials expressed concern over the usefulness of the established indicators. In addition, we
noted that HEP was in the process of developing a strategic plan that contains strategic goals that are
congruent with those contained in OPE’s plan. Such a plan will provide guidance, enumerate the goals
and objectives of the Title III Program, and provide a useful benchmark for what the performance
indicators should assess.

Due to the deficiencies in the process for drafting performance indicators, there is no assurance that the
selected performance indicators will demonstrate Title III Program achievement.

GPRA Requirements. GPRA requires that agencies develop performance indicators to measure or
assess the relevant outputs, service levels, and outcomes for each program activity. GPRA also
required that agencies prepare and submit strategic plans by September 30, 1997. Strategic planning
clearly conveys to employees, customers, stakeholders, and partners the purpose, direction, and plan of
action for an organization.

Guidance for Developing Performance Indicators. In February 1997, ED published a “Guide to
Program Outcome Measurement for the U.S. Department of Education” for use in developing
performance indicators. The guide details a nine-step process for developing and implementing
outcome measurement and includes three essential preliminary steps. A preliminary step is for the
program managers to establish a working group to oversee development of the outcome measurement
process. ED’s guide suggests that the working group be composed of:
       ♦ the program manager who should act as facilitator,
       ♦ members of the program staff,
       ♦ a representative from the relevant Office of the Assistant Secretary,
       ♦ a technical expert familiar with ED’s GPRA efforts, such as someone from
           the Planning and Evaluation Service, Office of Educational Research and



                                                 Page 3                            ED-OIG / A04-90014
          Improvement, or perhaps an outside consultant or contractor, and
        ♦ a representative from Budget Services.

Working Group Not Established and Essential Officials Not Involved

Essential component officials were not included in the working group to develop the Title III Program
performance indicators. The only Title III Program team member included in the working group was
the former IDUES Director. Among those excluded from the group were the IDUES Director, Title III
Program officers, team leaders, grantees, and the PMIT staff.

The involvement of the PMIT staff would have been especially beneficial. Recently, PMIT staff
demonstrated their knowledge and expertise in GPRA and performance measurement by developing the
booklet “Demonstrating Results, An Introduction to the Government Performance and Results Act.”
PMIT sent the booklet to over 3,000 HEP grantees. The PMIT Director also held a training session for
all Title III Program grantees entitled “Title III Program and the Changing Federal Environment,” which
discussed GPRA and performance goals and measurement. However, input on the development of
Title III Program performance indicators was not requested from PMIT.

As of January 1999, the former IDUES Director was still the only Title III Program official participating
in the process to revise the Title III Program performance indicators by the Planning and Evaluation
Service (PES), Office of the Under Secretary. Other members of the group that drafted the Title III
Program performance indicators worked in Quality Improvement and Strategic Planning, OPE, and in
the Office of the Under Secretary.

Documentation Not Maintained

We noted there was limited documentation detailing how the final Title III Program performance
indicators were developed and how the indicators will demonstrate achievement. We asked those
involved in the process for documentation, such as a report or summary, on the selection of the nine
Title III Program performance indicators. The only documentation submitted was an e-mail message to
notify members of changes to the performance indicators. It appears that there is no formal written
documentation detailing the decision-making process used to develop the Title III Program performance
indicators. Furthermore, there is no written documentation reflecting how these nine performance
indicators will demonstrate that the Title III Program is achieving its mission.

Our interviews confirmed that the team members held meetings to discuss revisions to the Title III
Program performance indicators. However, the process appears to have relied on internal e-mail
messages to inform team members of proposed revisions to the performance indicators rather than to
document how the final Title III Program performance indicators were developed and how the
indicators will demonstrate achievement.

ED Officials Indicate that Measures May Not be Adequate



                                                Page 4                           ED-OIG / A04-90014
We obtained three evaluations of the Title III Program performance indicators from senior officials in
HEP and the Office of the Under Secretary’s Planning and Evaluation Service (PES). These officials
were familiar with GPRA and performance measurement and commented on eight of the nine
performance indicators.2 Their comments revealed that the final indicators did not seek numerical
targets beyond maintaining the status quo and did not always follow GPRA guidelines.

For example, one official concluded that the principal goal of the Title III Program was to improve the
capacity of institutions to provide quality programs to low-income and minority students. However, the
same official also pointed out that only Objective 3 (Improve the access of low–income and minority
students to Title III-funded institutions.) and Objective 4 (Increase the number of degrees in under
represented areas.) are directly related to the Title III Program’s principal goal. The official further
stated that the performance indicators could be improved, but the process does not allow for such
improvement.

We also obtained feedback from several HEP Service Area Directors and officials within IDUES and
PES on the results of a three-year performance measurement study of ED's Title III Program,
performed by an outside contractor. One PES official commented that the performance indicators
recommended by the contractor’s study were not especially strong and the team conducting the study
lacked Title III Program knowledge. HEP officials also expressed concerns about the performance
indicators and indicated that the issues raised in the study were not being addressed.

In addition, we reviewed the results of the study and compared HEP’s final performance indicators to
those recommended in the contractor’s study. We found that, out of HEP’s nine Title III Program
performance indicators for FY 2000, five of the indicators matched those recommended in the
contractor’s study. Therefore, it appears that the study influenced HEP’s development of its indicators
even though HEP officials expressed concerns over the study’s results. Based on the HEP officials’
comments and concerns, HEP should carefully consider the concerns expressed regarding the
contractor’s study during future revisions to the performance indicators.

Strategic Plan for HEP Being Developed

We found that although a HEP strategic plan had not been developed, the Deputy Assistant Secretary
appeared to consider this a priority. While GPRA does not require a strategic plan at the program
level, this plan would help HEP: (1) ensure that its strategic goals are congruent with those contained in
OPE’s plan and (2) identify useful benchmarks for what its performance indicators should assess.

RECOMMENDATIONS

In order to assure that developed indicators adequately measure Title III Program performance and are

2
 See Attachment A for a list of these nine performance indicators for the Title III Program. The officials commented
on eight of these nine indicators. The results of their comments are shown in blue strikethrough type in Attachment
A.


                                                      Page 5                              ED-OIG / A04-90014
in compliance with GPRA, the Office of Postsecondary Education should require HEP to:

1.1. Develop future Title III Program performance indicators in accordance with the “Guide to
     Program Outcome Measurement for the U.S. Department of Education.”

1.2. Document how the Title III Program performance indicators will demonstrate progress toward
     achieving the mission of the Title III Program.

1.3. Continue development of a strategic plan to (a) ensure that its strategic goals are congruent with
     those contained in OPE’s plan and (b) identify useful benchmarks for what its performance
     indicators should assess.

Auditee's Response and Auditor Comments

We received and reviewed comments from the Director of IDUES. The comments show concurrence
with the findings and recommendations in our draft report. The Office of Inspector General finds the
draft report responses satisfactory to begin addressing our recommendations.




                                                Page 6                           ED-OIG / A04-90014
FINDING NO. 2           TITLE III PROGRAM DID NOT HAVE AN ADEQUATE
                        SYSTEM TO OBTAIN AND COMPILE GRANTEE DATA
                        NEEDED FOR MEASURING PERFORMANCE FOR THE
                        FIRST GPRA REPORT DUE MARCH 31, 2000, AND DID
                        NOT MEET ED’S INTERNAL DEADLINE FOR UPDATING
                        PERFORMANCE PLANS

Our review found that the Title III Program did not have an adequate system to obtain and compile data
needed for measuring performance. Furthermore, the Title III Program did not have any staff assigned
to this task, and the data format in the reports may not be conducive to deriving performance statistics.
The Title III Program Office does not routinely aggregate data from grantees’ annual and final
performance reports. However, ED uses the data from the reports for GPRA measurement. Until the
Title III Program assigns adequate staff and revises data format in the reports, there is no assurance that
the data obtained will adequately demonstrate Title III Program performance.

In addition, ED added several new requirements for the GPRA performance plan and report. ED
asked program offices to set up new targets (performance indicators) for the years 1999, 2000, and
2001 by December 3, 1999. However, the Title III Program Office (IDUES) did not meet this
deadline. IDUES did not submit its revised plan for its first GPRA performance report until January
2000.

GPRA Requirements for Performance Reports. According to GPRA, Section 4(b) 1116
(Program Performance Reports), “No later than March 31, 2000, and no later than March 31 of each
year thereafter, the head of each agency shall prepare and submit to the President and the Congress a
report on program performance for the previous fiscal year.”

Identified Data Sources May Not Be Adequate for Reporting Performance

According to the FY 2000 Title III Program Performance Plan, annual performance reports from
grantees are shown as a data source for all nine performance indicators. Our discussions with senior
ED officials in PES, PMIT and IDUES revealed that the performance reports might not be adequate for
reporting performance. Officials expressed very strong doubts about using the Title III grantee annual
performance reports as sources of information for GPRA reporting. The reports are narrative driven
and may not provide the data needed to measure performance. One of the officials acknowledged that
the selected information source, annual performance reports, was an “anticipated” data source. During
our fieldwork, the consensus among those interviewed was that it would be difficult for Title III Program
officials to complete the required GPRA report. Respondents suggested that the Title III Program
performance reports need revision to make them more conducive to reporting statistical data, and less
narrative driven.




                                                 Page 7                           ED-OIG / A04-90014
Process and Staffing Level Not Adequate to Compile and Aggregate Needed Data

The use of annual performance reports as the data source poses another problem. During our
fieldwork, we identified that the Title III Program Office had not aggregated data from reports and had
not assigned staff to accomplish this task. We found agreement among senior ED officials in PES,
IDUES and PMIT that the Title III Program Office did not routinely compile the performance data from
grantees’ annual reports. Until data is properly aggregated and adequate staff is assigned, the Title III
Program may have major difficulties in meeting the annual deadline for reporting.

We were also told about additional concerns that may impede HEP from compiling and aggregating
data. One IDUES official stated that the focus of the IDUES staff is getting grant money to the
grantees. The official added that the office’s first priority is assuring that the grantees receive their funds,
not GPRA compliance, and they do not have the staff to compile the data from annual performance
reports for the required GPRA report.

New Revisions of GPRA Performance Plans and Indicators

By December 3, 1999, ED asked program offices to update their annual GPRA performance plans to
cover performance indicators for the years 1999, 2000, and 2001. The Title III Program Office
(IDUES) did not meet this deadline. In January 2000, IDUES submitted a revised plan for its first
GPRA performance report.


RECOMMENDATIONS

In order to assure that needed performance data are properly obtained and compiled to demonstrate
performance under GPRA, the Office of Postsecondary Education should require HEP to:

2.1. Develop a system for collecting and aggregating the data needed for reporting on the Title III
     Program performance indicators in ED’s Annual Performance Plan.

2.2. Assure that the annual and final grantee performance reports are formatted in a manner conducive
     to providing needed data for the report on Title III Program performance indicators required by
     GPRA.

2.3. Develop a plan for assuring that the data collected are reliable and valid and available in a timely
     manner.

2.4. Identify alternate sources of data, such as a standardized form with information on the
     performance indicators. The form could be included in the annual performance reports from
     grantees and transmitted electronically, or converted to electronic form.




                                                   Page 8                             ED-OIG / A04-90014
Auditee's Response and Auditor Comments

We received and reviewed comments from the Director of IDUES. The comments show concurrence
with the findings and recommendations in our draft report. The Office of Inspector General finds the
draft report responses satisfactory to begin addressing our recommendations.




                                               Page 9                         ED-OIG / A04-90014
                                         BACKGROUND

The Government Performance and Results Act of 1993, commonly referred to as “GPRA” or the
“Results Act,” was enacted as the centerpiece of a statutory framework that Congress put in place to
improve federal management and provide a greater focus on results. GPRA seeks to shift the focus of
government decision making and accountability away from a preoccupation with activities such as the
number of grants awarded, to a focus on the results of those activities, such as real gains in ensuring
equal access to education for all individuals.

Under GPRA, agency heads were required to submit a five-year strategic plan to Congress and the
Office of Management and Budget (OMB), no later than September 30, 1997. Updates are required
at least every three years thereafter.

Beginning with fiscal year 1999 (October 1, 1998 to September 30, 1999), and annually thereafter,
each agency must submit to OMB performance plans covering each program activity in the agency’s
budget. Using the agency’s performance plans, OMB must prepare a government-wide performance
plan for inclusion in the President’s annual budget submission to Congress. Each agency’s annual
performance plan must

        1. establish performance goals that define the level of performance to be achieved by a
           particular program activity,
        2. express goals in an objective, quantifiable, and measurable form unless an alternative form is
           approved by OMB,
        3. describe the operational processes, skills, and technology, and the human capital,
           information, or other resources required to achieve performance goals,
        4. establish performance indicators to be used in measuring or assessing the relevant outputs,
           service levels, and outcomes of each program activity,
        5. provide a basis for comparing actual program results with the established performance
           goals, and
        6. describe the means to be used to verify and validate measured values.

In August 1995, the Office of Inspector General issued an audit report (ACN 04-40100-01)
recommending the development of Title III Program performance measures. Based on the audit
recommendations, OPE/HEP worked with an outside contractor to develop performance measures.

In October 1995, a study team began work on a three-year performance measurement study of ED’s
Title III Strengthening Institutions Program. The study team for this project consisted of an outside
contractor and two subcontractors. ED’s Planning and Evaluation Service supervised the study.




                                                Page 10                          ED-OIG / A04-90014
The study design contained the following major components:

•   Survey of all 1995-96 Part A and Part B Title III Program grantees.
•   Case studies consisting of site visits to 21 programs and 19 institutions.
•   Interviews with ED staff and review of documents and other data sources.
•   Advisory team and grantee input.

The outside contractor issued its final report on September 11, 1998. The report outlined
recommendations that, if implemented, could be used to develop a Title III Program performance
measurement system that would be in compliance with GPRA. We found that, out of HEP's nine Title
III Program performance indicators for FY 2000, five of the indicators matched those recommended in
the contractor's study.



                    OBJECTIVES, SCOPE, AND METHODOLOGY

We performed an audit at the Title III Program Office in Washington, D.C. The objectives of our audit
were to determine whether OPE/HEP (1) has met GPRA requirements for developing Title III Program
performance indicators and (2) has developed an accountability system for reporting on the indicators.

To accomplish the audit objectives, we performed the following:

♦ Reviewed applicable Federal regulations.
♦ Interviewed officials from: the Title III Program Office, the Office of the HEP Deputy Assistant
  Secretary, the Program Monitoring and Information Technology staff, the Planning and Evaluation
  Service, the OPE Quality Improvement and Strategic Planning staff, and the Postsecondary, Adult
  and Vocational Education Division.
♦ Reviewed the Title III Program Office’s efforts to disseminate GPRA related information to
  customers and its participation in drafting the FY 2000 performance indicators.
♦ Obtained and analyzed the Title III Program performance indicators for Fiscal Years 1999 and
  2000.
♦ Compared the FY 2000 Title III Program performance indicators to the recommended program
  goal indicators from the outside contractor’s final report.

Our fieldwork was performed from February 8, 1999, through February 26, 1999, at the Title III
Program Office in Washington, D.C. An exit conference took place on May 20, 1999. Program
officials generally concurred with our findings. Our audit was performed in accordance with the
government auditing standards appropriate to the scope of the review described above.




                                               Page 11                           ED-OIG / A04-90014
                   STATEMENT ON MANAGEMENT CONTROLS

Due to the limited scope of our review and audit objectives, we did not review the management control
structures of OPE, HEP, or the Title III Program Office.




                                              Page 12                         ED-OIG / A04-90014
                 ATTACHMENT A



Pages 144 and 145 of the U.S. Department of Education
           FY 2000 Annual Plan, Volume 2

                ED-OIG / A04-90014




                                                ED-OIG / A04-90014
                                                                                                                                                                     Attachment A

Aid for Institutional Development, Title III (HEA)-- $259,825,000 (FY 2000)
Goal:     To assist institutions that have limited resources and that traditionally served large numbers of low-income and minority students to continue to serve
           these students, and to improve the capacity of these institutions to provide on-going, up-to-date quality education in all areas of higher education.
Relationship of Program to Strategic Plan: Title III supports the Department’s overall goal of ensuring access and equity and enabling all students to achieve academic
excellence. Title III serves large numbers of low-income and minority students for whom access, retention, and degree attainment have been elusive. Title III supports strategic
plan Objectives 2.3, 3.1, 3.2, and 3.4.
Objectives                              Indicators                                                                Performance Data          Source, Periodicity, Next
Program improvement objectives
Title III –- Part A (Strengthening Institutions), Part A, sec.316 (American Indian Tribally Controlled Colleges and Universities), Part A, sec.317 (Alaska Native and
Native Hawaiian Serving Institutions), Part B (HBCUs and HBGIs), Part D (HBCU Capital Financing), Part E (Minority Science and Engineering Improvement
Program)
1.   Improve the academic             1.1 1.1 Faculty development.. The _number and            In FY 1996, 43% of faculty at more than          1.1 Performance reports – annual;
     quality of participating             percent of faculty participating in Title III-       half of the institutions participated in              initial comprehensive
     institutions.                        funded development activities will increase over     faculty development.                                  development plan (CDP);
                                          time.                                                                                                      recognition awards; updated
                                                                                                                                                     comprehensive development
                                                                                                                                                     plans; 1999.
                                      1.2 Access to technology. The number and
                                           percentage of students gaining access to            In 1996, approximately 34% of students           1.2 Performance reports – annual;
                                           computers and the Internet due to Title III-        had computer and Internet access. all,                initial comprehensive
                                           funded activities will increase over timer.         1997.                                                 development plan; updated
                                      1.1 .1.3                                                                                                       CDP; 1999

2.   Improve the fiscal stability     2.1 Development offices. The number and pPercent         In FY 1996, approximately 39% of                 2.1 Performance reports – annual;
     of participating institutions.       of funded development offices using grant funds      institutions used grant funds to improve              initial comprehensive
                                          to that show an increase in revenues will increase   development offices.                                  development plan ; updated
                                          oover prior years.                                                                                         CDP; 1999.

                                      2.2 Fiscal balances. The fiscal balance of Title III-    In FY 1996, more than 90% of institutions        2.2 Performance reports – annual;
                                          funded institutions will continue to remain          had positive fiscal balances. .external               initial comprehensive
                                          positive over time.                                  evaluations; comprehensive development                development plan ; updated
                                                                                               plans all, 1997.                                      CDP; 1999.

3.   Improve the access of low-       3.1 Enrollment of low-income minority students.          In FY 1996, 38% of the students under Part       3.1 IPEDS; performance reports –
     income and minority                  The number and percent of low-income and minority    A were minority and 86% under Part B                  annual; 1999.
     students to Title III-funded         students will remain stable or increase              were minority, compared with 20% for
     institutions.                        Over time.After implementating ation, grantees       non-Title III institutions.
                                          Will demonstrate a
                                                                                               Under Part A, 51% of the students were    low-
                                                                                               income, under Part B 48% were low-income.


ED-OIG NOTE: ED-OIG obtained this table on November 19, 1999 from the U.S. Department of Education’s web site containing its FY 2000 Annual Plan located
at http://www.ed.gov/pubs/AnnualPlan/vol2.pdf. This table is the Department’s working document for its Program Performance Plan for the Title III, HEA
Programs as evidenced by the Department’s strikethrough changes. ED-OIG has not made any changes to this document. This Annual Plan is dated February
25, 1999, and the website where it is maintained was last updated on August 11, 1999.
                                        U.S. Department of Education FY 2000 Annual Plan, Volume 2           page 144


                                                                                                                                                          ED-OIG / A04-90014
                                                                                                                                                                                            Attachment A

Aid for Institutional Development, Title III (HEA)-- $259,825,000 (FY 2000)
Goal:    To assist institutions that have limited resources and that traditionally serve large numbers of low-income and minority students to continue to serve these
          students, and to improve the capacity of these institutions to provide on-going, up-to-date quality education in all areas of higher education.
Relationship of Program to Strategic Plan: Title III supports the Department’s overall goal of ensuring access and equity and enabling all students to achieve academic
excellence. Title III serves large numbers of low-income and minority students for whom access, retention, and degree attainment have been elusive. Title III supports strategic plan
Objectives 2.3, 3.1, 3.2, and 3.4.
Objectives                              Indicators                                                                 Performance Data          Source, Periodicity, Next Update

                                                                                                                 In FY 1996, Part A institutions awarded              3.2   IPEDS; annual performance
                                             3.2 Degree attainment. The number and percentage                    34% associate degrees and 38%                               reports; 1999.
                                                 of degrees awarded to minority students at Title                bachelor’s degrees. Part B institutions
                                                 III-funded institutions will remain stable or                   awarded 72% associate degrees and 88%
                                                 increase over time. ering student;                              bachelor’s degrees.

                                                                                                                 In FY 1997, more than 75% of MSIP pre-college        3.3   Initial application; annual
                                             3.3 Improved access to careers in science and                       participants entered and completed MSIP                    performance report; IPEDS;
                                                  engineering. The number of MSIP pre-college                    interventions. Approximately 20%                           1999.
                                                  and undergraduate participants entering and                    of MSIP undergraduate students entered
                                                  completing MSIP interventions will remain                      and completed science and engineering
                                                  stable or increase over time. (benchmark not yet               programs.
                                                 available)After implementing a management                       Ed/IPOS Reports, 19987 and annual.
                                                 information and academic delivery sy,occur in a
                                                 greater number of institutions each year
                                                 (Benchmark not availabl
4. For Part B – HBCU                         4.1 Minority under-representation. The number                       In FY 1997, 3,500 of the 5,177 students              4.1 Performance reports – annual;
   Graduate Program:                             and percentage of advanced degrees in majors in                 enrolled in 20 advanced degree fields were African        initial comprehensive
   Strengthened graduate and                     which African American students are                             American.                                                 development plans; annual
   professional education.                       underrepresented will remain stable or decrease                                                                           updates; 1999.
                                                 over time.

5. Improve physical plants
                                             5.1 Capital projects. The number of capital                         In 1997, 45 inquiries and 1 loan were                5.1   Designated bonding authority
   through grant funding and
                                                 projects constructed, renovated, etc., using                    made, and 4 applications were received.                    updates- monthly; DBA
   low-cost capital for repair,
                                                 HBCU Capital Financing funds will increase                                                                                 annual report; program annual
   renovation, construction or
                                                 over time.                                                                                                                 report; 1999.
   acquisition of capital
   projects.


v    Assist Title III institutions in serving low-income and minority students by disseminating information to institutions on effective practices.
v    Establish a formal mechanism for exchange of information with Title III-related organizations and higher education agencies and associations.
v    Conduct consistent, thorough reviews of performance reports with feedback to grantees.


                                              U.S. Department of Education FY 2000 Annual Plan, Volume 2                                                   page 145




                                                                                                                                                                                ED-OIG / A04-90014
                                            Attachment A




        ATTACHMENT B



Auditee's Response with Attachments

       ED-OIG / A04-90014




                                      ED-OIG / A04-90014
                                                                                                       Attachment A


                           UNITED STATES DEPARTMENT OF EDUCATION
                  INSTITUTIONAL DEVELOPMENT & UNDERGRADUATE EDUCATION SERVICES
                                      OFFICE OF THE DIRECTOR




MEMORANDUM

DATE:             March 31, 2000

TO:               Ms. Carol Lynch
                  Region IV Inspector General-Audit

From:             Margarita Benitez, Director
                  Institutional Development & Undergraduate
                  Education Services

Subject:          Draft Audit Report
                  Review of Title III Program, HEA, Compliance with GPRA Requirements
                   for Implementation of Performance Indicators
                  Audit Control Number ED-OIG/A04-90014

I have reviewed the draft audit report identified in the subject line of this memo. After speaking
with Jim Wiley of your staff, I am submitting responses to the specific recommendation resulting
from your review. These responses are found in Attachment 1.

We are happy to report that data for the Title III program is available for the FY 2000 GPRA
report. In the months following the audit, IDUES staff met several times with Planning and
Evaluation Service (PES), Student Financial Aid, and Policy, Planning & Innovation (PPI) to
devise better indicators and measures, and ascertain data availability and its validity. We were
guided by the PES to the Integrated Postsecondary Education Data System and its feeder
surveys. All of the performance indicators are supported currently and henceforth annually by
one or more of the IPEDS surveys. The revised Title III performance indicators and measures
are found in Attachment 2.

As the new director for IDUES, I am aware that we need more public feedback on the revised
and approved indicators. We will not have the best indicators without significant input of our
publics and stakeholders into their development. I am looking for every opportunity to bring
performance issues to our constituents, receiving information from them, and formulating the
best policies, practices and procedures that will propel the Title III program to the forefront of
successful performance.



                        400 MARYLAND AVE., SW WASHINGTON, D.C. 20202-1510


   Our mission is to ensure equal access to education and to promote educational excellence throughout the Nation.
                                                                                 Attachment A



Our responses also indicate that we have followed the ED guidelines suggested for developing
performance indicators to the extent possible in the time available to address the identified
problems. As we move forward, we will make every effort to use the ED guidelines to the
maximum extent.

Finally, as I indicated to Jim Wiley, I have developed and presented a detailed plan to OPE
management that addresses the concerns of this and previous audits. I have taken the necessary
steps to demonstrate the need for support, and a provided a detailed description of resources
required for implementation of the plan. In our beginning efforts described in our responses and
in the efforts of our action plan, we are collaborating with Program Monitoring and Information
Technology Service to develop a final performance report, monitoring skills, and the capacity of
Title III grantees to administer the grants according to EDGAR. A copy of our Action Plan is
found in Attachment 3.

I hope that you will find our responses satisfactory. If not, my staff and I are available for
further discussion and revisions. I can be reached on (202)502-7794, as you already know.
My assistant, Carolyn Proctor can be reached on (202) 502-7567 or Don Crews on
(202)502-7574.

Cc:     Rich Rasa, OIG
        Jim Wiley, OIG
        Joanne Jones, OIG

        Dr. Claudio Prieto, Deputy Assistant Secretary, HEP/OPE
        Dr. Jowava M. Leggett, Chief of Staff, HEP
        Mr. Alan Schiff, Audit Liaison Officer, HEP
        Lou Venuto, IDUES Senior Advisor
        Carolyn Proctor, IDUES Special Assistant
        Don Crews, IDUES Special Assistant
        Darlene Collins, Program Development Service Team I Leader, IDUES
        Ken Waters, Program Development Service Team II Leader, IDUES




Attachments (3)




                                                2
ATTACHMENT 1            IDUES Response to Draft Audit ED-OIG/A04-90014

FINDING NO. 1           TITLE III PROGRAM OFFICIALS DID NOT FOLLOW
                        SUGGESTED GUIDELINES IN DRAFTING FY 2000
                        PERFORMANCE INDICATORS FOR THE TITLE III
                        PROGRAM

RECOMMENDATIONS

In order to assure that developed indicators adequately measure Title III Program performance
and are in compliance with GPRA, the Office of Postsecondary Education should require HEP
to:

1. Develop future Title III Program performance indicators in accordance with the “Guide to
   Program Outcome Measurement for the U.S. Department of Education.”

2. Document how the Title III Program performance indicators will demonstrate
   progress toward achieving the mission of the Title III Program.

3. Continue development of a strategic plan to (a) ensure that its strategic goals are congruent
   with those contained in OPE’s plan and (b) identify useful benchmarks for what its
   performance indicators should assess.

RESPONSE

1.      The performance indicators and measures for the Title III program were revised in
        January/February, 2000. The revised performance indicators and measures take into
        consideration at least five of the nine Process Development and Implementation Steps
        outlined in the Guide to Program Outcome Measurement for the U.S. Department of
        Education. The remaining four steps are data analysis steps that will occur during
        production of the department’s GPRA report.

2.      The new performance indicators will measure significant changes that occurred at
        grantee institutions in areas of Congressional interest and are directly related to the
        program purpose statement. The revised indicators include academic improvement as
        demonstrated by sustaining or increasing the number of specialized accreditation,
        increased graduation rates, maintaining fiscal balance and increased endowments.

3.      The PMIT Service Director, Dr. Lawrence Grayson, was directed by our Deputy
        Assistant Secretary to develop a draft of the HEP Strategic Plan that is consistent with
        the Department’s Plan. IDUES will participate in all meetings held for this purpose with
        the intent of further developing our performance measures to sufficiently reflect the goals
        of the plan.


                                                1
2
FINDING NO. 2           TITLE III PROGRAM CURRENTLY DOES NOT HAVE AN
                        ADEQUATE SYSTEM TO OBTAIN AND COMPILE
                        GRANTEE DATA NEEDED FOR MEASURING
                        PERFORMANCE FOR THE FIRST GPRA REPORT DUE
                        MARCH 31, 2000 AND DID NOT MEET ED’S INTERNAL
                        DEADLINE FOR UPDATING PERFORMANCE PLANS


RECOMMENDATIONS

In order to assure that needed performance data are properly obtained and compiled to
demonstrate performance under GPRA, the Office of Postsecondary Education should require
HEP to:

1. Develop a system for collecting and aggregating the data needed for reporting on the Title
   III Program performance indicators in ED’s Annual Performance Plan.

2. Assure that the annual and final grantee performance reports are formatted in a manner
   conducive to providing needed data for the report on Title III Program performance
   indicators required by GPRA.

3. Develop a plan for assuring that the data collected are reliable and valid and available in a
   timely manner.

4. Identify alternate sources of data, such as a standardized form with information on the
   performance indicators. The form could be included in the annual performance
   reports from grantees and transmitted electronically or converted to electronic form.

RESPONSES

1.      IDUES, in collaboration with the Planning & Evaluation Service, Office of the
        Undersecretary, has identified sources of data already collected in the Department of
        Education for the performance indicators/measures. The primary source of
        Department’s data collection on colleges and universities is the Integrated
        Postsecondary Education Data System (IPEDS). IDUES has identified a key data
        element, the OPE ID for referencing Title III grantees in IPEDS.

2.      The data needed for the revised performance indicators and measure will be extracted
        from internal departmental sources. IDUES has undertaken the revision of the final
        performance report to ascertain information related to the specific activities supported


                                                3
     by the grant. The report will also contain data that supplements the data that we will
     extract from IPEDS and other internal sources.

3.   IDUES, in collaboration with the Planning & Evaluation Service, carefully crafted
     revised performance indicators and measures that will generate data in a timely manner.
      IDUES expects to receive data from the various IPEDS surveys at least 3-6 months in
     advance of the GPRA Report.

4.   IDUES is currently developing its Final Performance Report in collaboration with
     PMITS. We are trying to avoid data collection that duplicates data that is already
     available in keeping with Paperwork Reduction Act. The Final Performance Report will
     contain data elements that further support or elaborate on data provided through the
     IPEDS surveys. All of our data collection instruments will be included on our Web
     page. As our level of sophistication grows via our Action Plan, we will explore alternate
     means for grantees to submit performance report data.




                                             4
IDUES Response - ATTACHMENT 2
Attachment B
Aid for Institutional Development, Title III (HEA)--$259,825,000 (FY 2000)/Requested budget--$XXXX(FY 2001)
Part A (Strengthening Institutions), Part A, sec.316 (American Indian Tribally Controlled Colleges and Universities), Part A, sec.317 (Alaska Native and Native Hawaiian
Serving Institutions), Part B (Historically Black Colleges and Universities (HBCUs) and Historically Black Graduate Institutions), Part D (HBCU Capital Financing), Part E
(Minority Science and Engineering Improvement Program), and Part V (Developing Hispanic-serving Institutions)
Goal: To assist institutions that have limited resources and that traditionally served large numbers of low-income and minority students to continue to serve these students, and
to improve the capacity of these institutions to provide on-going, up-to-date quality education in all areas of higher education.
Relationship of Program to Volume 1, Department-wide Objectives: The Title III program supports the Department’s overall goal of ensuring access and equity and enabling all
students to achieve academic excellence. More specifically, Title III supports Objectives 3.1 (successful preparation for postsecondary education) and 3.2 (postsecondary
students receive support for high-quality education) by serving large numbers of low-income and minority students for whom postsecondary access, retention, and degree
attainment have been elusive.
     Indicators and Targets                             Performance Data                                  Assessment of Progress                      Source and Data Quality
 Objective 1: Improve the academic quality of participating institutions.
 1.1 Specialized accreditations.     Actual Performance                                           Status:                                          Source: 1999 Higher Education
     The percentage of Title III                                                                  Progress cannot be judged until trend data       Directory.
     institutions having specialized The percentage of Title III institutions having a            are available.
     accreditations, a measure of    specialized accreditation:                                                                                    Frequency: Annual
     academic program quality, will                                                               Explanation:
     be maintained or increased.     1998/99 66%                                                  In the 1998-99 school year, 66 percent of        Next Update: 1999/2000
                                                                                                  Title III institutions had received at least
                                        Performance Targets                                       one specialized accreditation. Specialized       Validation Procedures:
                                        2000: Maintain or increase level.                         accreditations are an indication that the        Data are verified by the publisher
                                        2001: Maintain or increase level.                         quality of an academic program is                by comparing against lists
                                                                                                  sufficiently high to meet the rigorous           maintained by all accrediting
                                                                                                  standards imposed by an independent              agencies recognized by the
                                                                                                  agency.                                          Department of Education.

                                                                                                                                                   Limitations of Data and Planned
                                                                                                                                                   Improvements:
                                                                                                                                                   None.
 1.2 Graduation rates. Completion       Actual Performance                                        Status:                                          Source: Graduation Rate Survey
     rates for all full-time, degree-   The percentage of full-time, degree-seeking students      No 1999 data; progress cannot be judged          (GRS) conducted as part of the
     seeking students in Title III 4-   at Title III institutions completing a four-year degree   until trend data are available.                  Integrated Postsecondary Student
     year and 2-year colleges will      within six years and a two-year degree, certificate,                                                       Aid Study (IPEDS).
     increase over time.                or transferring to a four-year school within three        Explanation:
                                        years.                                                    Approximately one-third of full-time degree-     Frequency: Annual
                                                                                                  seeking students complete a four-year
                                                 Four-year           Two-year                     degree within six years and one-fifth            Next Update: 1998
                                        1997       34%                 22%                        complete a two-year degree, certificate, or
                                                                                                  transfer to a four-year school within three      Validation Procedure:
                                        Performance Targets                                       years. These data only measure the extent        Verified by ED data attestation
                                        1999: continuing increase in rate                         to which students complete their degrees at      process.
                                        2000: continuing increase in rate                         the Title III institution they first attended.

                                                                                       Page 1                                                                ED-OIG / A04-90014
IDUES Response - ATTACHMENT 2
Attachment B
Aid for Institutional Development, Title III (HEA)--$259,825,000 (FY 2000)/Requested budget--$XXXX(FY 2001)
Part A (Strengthening Institutions), Part A, sec.316 (American Indian Tribally Controlled Colleges and Universities), Part A, sec.317 (Alaska Native and Native Hawaiian
Serving Institutions), Part B (Historically Black Colleges and Universities (HBCUs) and Historically Black Graduate Institutions), Part D (HBCU Capital Financing), Part E
(Minority Science and Engineering Improvement Program), and Part V (Developing Hispanic-serving Institutions)
Goal: To assist institutions that have limited resources and that traditionally served large numbers of low-income and minority students to continue to serve these students, and
to improve the capacity of these institutions to provide on-going, up-to-date quality education in all areas of higher education.
Relationship of Program to Volume 1, Department-wide Objectives: The Title III program supports the Department’s overall goal of ensuring access and equity and enabling all
students to achieve academic excellence. More specifically, Title III supports Objectives 3.1 (successful preparation for postsecondary education) and 3.2 (postsecondary
students receive support for high-quality education) by serving large numbers of low-income and minority students for whom postsecondary access, retention, and degree
attainment have been elusive.
     Indicators and Targets                             Performance Data                               Assessment of Progress                    Source and Data Quality
                                       2001: continuing increase in rate                       Therefore, the rates are understated to the    Limitations of data and planned
                                                                                               extent to which students complete their        improvements:
                                                                                               degree at a different institution.             Postsecondary institutions are not
                                                                                                                                              required to report graduation rates
                                                                                                                                              until 2002 (1999 for two-year
                                                                                                                                              institutions). However, data were
                                                                                                                                              voluntarily submitted by
                                                                                                                                              institutions representing 87 percent
                                                                                                                                              of four-year students at Title III
                                                                                                                                              institutions and 73 percent of two-
                                                                                                                                              year students at Title III
                                                                                                                                              institutions.

 Objective 2: Improve the fiscal stability of participating institutions.
 2.1 Fiscal balance. Over 90            Actual Performance.                                    Status:                                        Source: Finance Survey conducted
     percent of Title III institutions  The percentage of Title III institutions having a      No 1999 data; progress toward target is        as part of the Integrated
     will maintain a positive fiscal    positive fiscal balance.                               likely.                                        Postsecondary Student Aid Study
     balance.                                                                                                                                 (IPEDS).
                                                    All institutions Public institutions       Explanation:
                                        1995/96           92%                 92%              Among all Title III institutions, 92 percent   Frequency: Annual
                                        1996/97           N/A                90%               had a positive fiscal balance in the 1995/96
                                                                                               school year. The percentage of public Title    Next Update: 1996-97 for private
                                                                                               III institutions having a positive fiscal      institutions; 1997/98 for all
                                       Performance Targets                                     balance declined slightly from 92 percent in   institutions.
                                       1999: 90%                                               1995/96 to 90 percent in 1996/97.
                                       2000: 90%                                                                                              Validation Procedures:
                                       2001: 90%                                                                                              Data validated by NCES review and
                                                                                                                                              NCES Statistical Standards

                                                                                                                                              Limitations of Data and Planned
                                                                                                                                              Improvements

                                                                                      Page 2                                                            ED-OIG / A04-90014
IDUES Response - ATTACHMENT 2
Attachment B
Aid for Institutional Development, Title III (HEA)--$259,825,000 (FY 2000)/Requested budget--$XXXX(FY 2001)
Part A (Strengthening Institutions), Part A, sec.316 (American Indian Tribally Controlled Colleges and Universities), Part A, sec.317 (Alaska Native and Native Hawaiian
Serving Institutions), Part B (Historically Black Colleges and Universities (HBCUs) and Historically Black Graduate Institutions), Part D (HBCU Capital Financing), Part E
(Minority Science and Engineering Improvement Program), and Part V (Developing Hispanic-serving Institutions)
Goal: To assist institutions that have limited resources and that traditionally served large numbers of low-income and minority students to continue to serve these students, and
to improve the capacity of these institutions to provide on-going, up-to-date quality education in all areas of higher education.
Relationship of Program to Volume 1, Department-wide Objectives: The Title III program supports the Department’s overall goal of ensuring access and equity and enabling all
students to achieve academic excellence. More specifically, Title III supports Objectives 3.1 (successful preparation for postsecondary education) and 3.2 (postsecondary
students receive support for high-quality education) by serving large numbers of low-income and minority students for whom postsecondary access, retention, and degree
attainment have been elusive.
     Indicators and Targets                             Performance Data                                Assessment of Progress                    Source and Data Quality
                                                                                                                                               Data tend to be several years old.
                                                                                                                                               NCES is planning on instituting a
                                                                                                                                               web-based data collection for
                                                                                                                                               IPEDS that should dramatically
                                                                                                                                               reduce the time required for
                                                                                                                                               information to be available.

 2.2 Endowment. The percentage of       Actual Performance                                      Status:                                        Source: Finance Survey conducted
     Title III institutions having an   The percentage of Title III institutions having a       No 1999 data; progress toward target is        as part of the Integrated
     endowment will increase over       positive endowment.                                     likely.                                        Postsecondary Student Aid Study
     time.                                                                                                                                     (IPEDS).
                                                   All institutions   Public institutions       Explanation:
                                        1995/96          57%                51%                 Among all Title III institutions, 57 percent   Frequency: Annual
                                        1996/97          N/A               56%                  had a positive endowment in the 1995/96
                                                                                                school year. The percentage of public Title    Next Update: 1996-97 for private
                                        Performance Targets                                     III institutions having a positive             institutions; 1997/98 for all
                                        1999: continuing increase.                              endowment increased from 51 percent in         institutions.
                                        2000: continuing increase.                              1995/96 to 56 percent in 1996/97.
                                        2001: continuing increase.                                                                             Validation Procedures:
                                                                                                                                               Data validated by NCES review and
                                                                                                                                               NCES Statistical Standards

                                                                                                                                               Limitations of Data and Planned
                                                                                                                                               Improvements
                                                                                                                                               Data tend to be several years old.
                                                                                                                                               NCES is planning on instituting a
                                                                                                                                               web-based data collection for
                                                                                                                                               IPEDS that should dramatically
                                                                                                                                               reduce the time required for
                                                                                                                                               information to be available.


                                                                                       Page 3                                                            ED-OIG / A04-90014
IDUES Response - ATTACHMENT 2
Attachment B


Aid for Institutional Development, Title III (HEA)--$259,825,000 (FY 2000)/Requested budget--$XXXX(FY 2001)
 Goal: To assist institutions that have limited resources and that traditionally served large numbers of low-income and minority students to continue to serve these students, and
 to improve the capacity of these institutions to provide on-going, up-to-date quality education in all areas of higher education.
 Relationship of Program to Volume 1, Department-wide Objectives: More specifically, Title III supports Objectives 3.1 (successful preparation for postsecondary education) and
 3.2 (postsecondary students receive support for high-quality education) by serving large numbers of low-income and minority students for whom postsecondary access,
 retention, and degree attainment have been elusive.
 Key Strategies
 Strategies continued from 1999
 v Assist Title III institutions in serving low-income and minority students by disseminating information to institutions on effective practices.
 v Establish a formal mechanism for exchange of information with Title III-related organizations and higher education agencies and associations.
 v Conduct consistent, thorough reviews of performance reports with feedback to grantees.

 New or Strengthened Strategy(s)
 v Develop a list serve for obtaining grantee feedback on performance indicators.
 v To address the lack of program data on specific outcomes of grantee activities, identify and collect measures of the impact of Title III funds on institutions. In addition,
     identify and collect data on specific outcomes related to the various component programs that make up the Title III program.
 How This Program Coordinates with Other Federal Activities
 v Program works with the White House Initiative on HBCUs, Educational Excellence for Hispanic Americans, and Tribal Colleges to coordinate assistance being provided
     across the federal government to these institutions.
 Challenges to Achieving Program Goal
 v The Inspector General (IG) is expected to release a report on the Title III programs shortly. Once the report has been received, a plan for addressing the IG’s concerns will be
     developed.
 v Title III funds are given to institutions in order to meet specific needs identified in their Comprehensive Development Plan. This flexibility greatly increases the usefulness of
     the program for institutions but makes it difficult to identify common goals and indicators that apply to all schools.
 Indicators that have been Adjusted or Dropped since the 1999 Plan




                                                                                      Page 4                                                               ED-OIG / A04-90014
IDUES Response - ATTACHMENT 2
Attachment B
Aid for Institutional Development, Title III (HEA)--$259,825,000 (FY 2000)/Requested budget--$XXXX(FY 2001)
 Goal: To assist institutions that have limited resources and that traditionally served large numbers of low-income and minority students to continue to serve these students, and
 to improve the capacity of these institutions to provide on-going, up-to-date quality education in all areas of higher education.
 Relationship of Program to Volume 1, Department-wide Objectives: More specifically, Title III supports Objectives 3.1 (successful preparation for postsecondary education) and
 3.2 (postsecondary students receive support for high-quality education) by serving large numbers of low-income and minority students for whom postsecondary access,
 retention, and degree attainment have been elusive.
 From two year old Annual Plan (FY 1999)
 Adjusted
 Old indicator 1.2 was modified to replace institutional persistence rates with the IPEDS GRS data in order to reduce burden on schools since they already have or will have to
 report the IPEDS data. Also, the IPEDS data are likely to be much more uniform since they have standard definitions.
 Old indicator 2.2 was modified to replace institutional endowment information with the IPEDS Finance data in order to reduce burden on schools since they already have to report
 the IPEDS data. Also, the IPEDS data are likely to be much more uniform since they have standard definitions.

 Dropped
 The remaining indicators were dropped for three reasons. Old indicators 1.1, 1.3, 4.1, 4.2, 4.3, 5.1, and 6.1 were dropped because there was no data collection plan
 in place to obtain the needed information and were replaced by indicators based on on-going national data collection systems. Old indicators 2.1, 2.3, 3.1, 3.2, and
 all the indicators listed under management improvement were dropped because these are internal indicators needed to operate and manage the program but are not
 suitable for reporting the outcomes of the program to external constituents. The remainder of the indicators related to the specific component programs of the Title II
 program and were dropped in favor of uniform measures across the Title III program.

 From last year’s Annual Plan (FY 2000)
 Adjusted
 Old indicator 2.2 was modified to reflect more current information available from audited financial statements related to the fiscal strength of institutions.
 Old indicator 3.2 was modified to replace institutional degree attainment rates with the IPEDS GRS data in order to reduce burden on schools since they already have or will have
 to report the IPEDS data. Also, the IPEDS data are likely to be much more uniform since they have standard definitions.

 Dropped
 Old indicators 1.1, 1.2, 2.1, and 3.1, were dropped because there was no data collection plan in place to obtain the needed information and were replaced by
 indicators based on on-going national data collection systems. The remainder of the indicators related to the specific component programs of the Title II program and
 were dropped in favor of uniform measures across the Title III program.
 New
 Indicator 1.1 uses nationally available data on specialized accreditations to address issues regarding improvements in the quality of academic programs.




                                                                                     Page 5                                                                 ED-OIG / A04-90014
IDUES Response - Attachment 3                                                        Attachment B


March 9, 2000

TO:                     Dr. A. Lee Fritschler
                        Assistant Secretary for Postsecondary Education [OPE]

THROUGH: Claudio Prieto
                Deputy Assistant Secretary for Higher Education Programs [HEP]

FROM:                   Margarita Benítez
                        Director of Institutional Development and Undergraduate
                        Program Service [IDUES]

SUBJECT:        URGENT NEED TO IMPLEMENT A MULTI-PRONGED ACTION PLAN
                TO ANSWER AND RESOLVE THE FINDINGS OF FIVE AUDITS OF THE
                INSPECTOR GENERAL WITH REGARD TO IDUES

ISSUE:

Since 1995, the Office of the Inspector General [IG] has conducted a series of audits of the HEP unit
currently known as IDUES, that have focused mainly on the Title III Program. These audits consistently
call for an in-depth reassessment of current procedures at IDUES with regard to the grant award
process, customer service, and program monitoring, as well as for a wide-ranging effort to gather,
analyze, and evaluate information about grantee and program performance in order to assess whether
program goals are being met.

The issues raised in the IG audits have gone unanswered for a number of years. They must be
addressed without further delay.

PROPOSAL:

What follows is a plan that will take the urgent need to respond to the findings of the IG audits as the
point of departure for the conversion of the IDUES Service Area from the Cinderella of OPE into a
more efficient, accountable, technologically adept, and customer focused organization in accordance
with the Department’s Strategic Plan. Strategies include:

•   an ambitious effort to gather, aggregate, and analyze data relevant to program assessment;
•   comprehensive employee training and development in collaboration with other Department offices
    and expert consultants;
•   the development of an annual institutional monitoring plan;
•   further development of a "capacity-building curriculum" in grants management and performance
    evaluation for the various grantee populations;
•   and the design of a process for the development/revision of performance indicators and



                                                  Page 1          ED-OIG / A04-90014
IDUES Response - Attachment 3                                                      Attachment B


    performance reports, which will involve members of the grantee community.

BACKGROUND

In the past five years, the Office of the Inspector General has conducted a number of audits regarding
the operation of programs presently under the jurisdiction of Developing Institutions and Undergraduate
Education Service [IDUES]. Some of the concerns and findings identified therein have been addressed,
but many have not been resolved yet. According to the Office of the Chief Financial Officer [OCFO],
no status reports have gone forward since the submission of the original Corrective Action Plans
[CAPs] some years ago. IDUES Corrective Action Plans should have been implemented and
completed already for the three IG audits that date from 1995 and 1996. OCFO is requesting our
immediate attention to this matter. Updated CAPS for all three outstanding audits have been requested
by February 29.

Old audits with outstanding corrective actions are a matter of great concern to ED Deputy Secretary
Holleman and to the Congress. The Office of Postsecondary Education is listed with three outstanding
audits in the Inspector General’s Semiannual Report to Congress No. 39 [April-September 1999]
under “Recommendations Described in Previous Semiannual Reports on Which Corrective Action Has
Not Been Completed.” The three audits listed all have to do with IDUES. The three audits are:

•   ACN-04-40100 “Helping to Assure Equalized Educational Opportunities with HEA, Title III
    Institutional Aid Funds,” issued 8/28/95. IDUES Corrective Action Plan submitted in November
    1997.
•   ACN-17-30305 “Annual Interest Grants: Improving the Process for Paying the Remaining
    Grants,” issued 9/28/95. Original IDUES Corrective Action Plan submitted in September 1997.
    In this case, all corrective actions were completed by March 31, 1999, and the final CAP was
    submitted in February 2000.
•   ACN-04-60001 “Process Enhancements in the HEA, Title III, Institutional Aid Program Would
    Increase Program Efficiency, Despite Limited Resources," issued 3/27/96. CAP submitted in
    November 1997.

According to the OCFO—some HEP officials disagree--, no status reports were submitted to the IG
after the initial CAPs. This is not to say that no action was taken, but it appears that none was reported
to the appropriate authorities in a 27-month period.

In addition, in November 1998 the IG notified then-Assistant Secretary David A. Longanecker of its
plans “to evaluate whether the program [Title III A and B] is being administered in an efficient and
effective manner, and our focus will be on monitoring, problem resolution process/enforcement,
performance measures, the grant application renewal process for Part A grantees, and the need for
legislative change(s).” The IG conducted this evaluation in 1999. So far, it has produced two draft IG
audits of IDUES which must be addressed as well. They are:




                                                 Page 2          ED-OIG / A04-90014
IDUES Response - Attachment 3                                                      Attachment B



•   A04-90013 “Review of Title III Program Monitoring and Enforcement.” An undated draft
    document is available.
•   A04-90014 “Review of Title III Program, HEA, Compliance with GPRA Requirements for
    Implementation of Performance Indicators.” This was received in draft audit form in February
    2000.

OVERVIEW OF IG FINDINGS

The situation at IDUES described and documented by the IG is widespread and serious. Most of the
audits focus on the Title III Program (Parts A and B), which is the largest program run by IDUES. The
issues raised in ACN 17-30305 with regard to the work of the Institutional Receivables Team are of a
lesser nature and have all been resolved, as documented in the CAP submitted in February 2000.

1. “In our opinion, the Title III Program lacks specific, measurable goals and measures to assess
   overall Title III program achievement.” ACN 04-40100-1. August 1995.
                • “Overall Title III Performance Currently Not Assessable”
                • “Performance Not Currently Globally Measured”
                • “Global Goals and Indicators Needed”
                • GAO’s 1979 study of the Title III program raised a question which, in the IG’s
                     view, remains unanswered two decades later: “Are institutions which are receiving
                     Title III funds moving towards financial stability and being brought into the
                     mainstream of academic life, or is Title III merely a form of continued subsidy to
                     institutions that are no more developed after years of assistance than when their
                     participation first began?”

2. The IG refers to the 1993 National Performance Review [NPR] report “From Red Tape to
   Results: Creating a Government that Works Better and Costs Less,” where the Department of
   Education [ED] was urged to streamline and improve its grant process; improve employee
   development opportunities; and develop a strategy for technical assistance and information
   dissemination. With regard to the Title III Program, the IG found that: “The processes that could
   be enhanced include those of determining eligibility, reviewing applications, granting awards, as well
   as those of providing technical assistance and conducting monitoring.” ACN 04-60001. March
   1996.

3. In a 1999 draft document related to A04-90013 [“Review of Title III Monitoring and
   Enforcement”] the IG states: “The Higher Education Programs (HEP) Office needs a systemic
   approach to monitor Parts A and B of the Title III Program to assure performance and compliance
   by grant recipients. …

        •     A system to obtain needed data such as performance reports, audit reports, accreditation
            reports and follow-up status reports on issues identified during monitoring site visits.



                                                 Page 3          ED-OIG / A04-90014
IDUES Response - Attachment 3                                                       Attachment B


        •    Criteria to apply to data obtained to determine if monitoring is needed.
        •    A formal monitoring plan . . .
        •    A uniform monitoring guide for conducting monitoring reviews; and
        •    Written policies and procedures for providing technical assistance and performing
            monitoring site visits.”

4. IG draft audit A04-90014, released to ED officials in February 2000, makes the following
   recommendations:

        •   Develop a system for collecting and aggregating the data needed for reporting on the Title
            III Program performance indicators in ED’s Annual Performance Plan.
        •   Assure that the annual and final grantee performance reports are formatted in a manner
            conducive to providing needed data for the report on Title III Program performance
            indicators required by GPRA.
        •   Develop a plan for assuring that the data collected are reliable and valid and available in a
            timely manner.
        •   Identify alternate sources of data, such as a standardized form with information on the
            performance indicators.

WHY A MULTI-PRONGED PLAN IS NECESSARY NOW

Irrespective of whether OPE and IDUES management agree or disagree with every IG finding and
recommendation, they cannot go unanswered any longer.

A serious and widespread situation such as the one described by the IG must be addressed by a serious
and wide-ranging effort. A particular urgency is added to the IDUES situation by the fact that most of
the IG audits are four to five years old, and no IDUES responses to the IG audits appear to have been
recorded since 1997-98. IDUES must respond to those audits forthwith, and in a substantive manner.

To set in motion a concerted effort to address and resolve this long-standing situation must be a priority
for the Assistant Secretary for Postsecondary Education [AS/OPE], the Deputy Assistant Secretary for
Higher Education Programs [DAS/HEP], and the newly appointed Director of IDUES, Dr. Margarita
Benítez, who has inherited the situation, and will oversee the effort to respond appropriately to the IG’s
findings.

Not only is it evident that the concerns raised in IG audits since 1995 must be addressed and resolved
without delay. It is also evident that IDUES staff must be thoroughly involved in the realization of these
resolutions, so that they become established practices and procedures, as opposed to stopgap
remedies.

At the present time we are facing two realities that can neither be denied nor postponed. The first one
has already been acknowledged: the issues addressed by the IG audit reports must be addressed



                                                 Page 4           ED-OIG / A04-90014
IDUES Response - Attachment 3                                                         Attachment B


forthwith, particularly the ones where recorded IDUES responses are years behind schedule. The
second reality is that Title III program officers have just entered their busiest period: grant competition,
peer review and funding slates for Title III-A; a March 31 deadline for submission of materials for Title
III-B institutions.

This means that, in the immediate future, the majority of IDUES staff will be heavily involved in the
regular grant award process, where there are a number of upcoming deadlines to be met. ED programs
have been asked by the Deputy Secretary of Education to meet a May 31st deadline for grant awards.
Meeting the grant award deadlines is very important to IDUES, HEP, OPE, and to the institutions we
serve; however, that accomplishment does not begin to address the serious issues raised by the IG
audits.

Since the grant award deadlines must be met, and since the issues raised by the IG audits must be
addressed without further delay, it becomes necessary to outline, develop, and carry out a multi-
pronged plan that will not fly in the face of these realities, but yield both immediate and long term results.
 To develop and carry out such a plan, especially within a tight time frame, calls for expert assistance on
an intense and ongoing basis. Also necessary is the involvement of key offices within OPE and
beyond—most prominently Program Monitoring and Information Technology [PMIT]--, as well as the
increasing participation of IDUES staff once they discharge their present pressing obligations.

PLAN OUTLINE

Goals

This plan seeks to address the findings contained in the IG audits about the operation of IDUES without
delay, and with lasting results. Key goals are:
• To resolve the current lack of substantial and systematic information gathering, analysis, and
    evaluation about: program achievements; customer service; and grantee, program and IDUES
    compliance with GPRA and other legislative requirements.
• To build staff capacity in order to increase IDUES responsiveness and service to institutions.

Objectives:

1. Establish a fast-track timeline for initial results—responding to the IG findings without delay—and a
   long-term timeline for lasting results—integrating effective customer service as well as monitoring
   and assessment procedures as part of IDUES standard operations.
2. Identify resources within and without OPE.
3. Design and develop a combination of qualitative and quantitative approaches to data-gathering that
   will address the information needs pointed out by the IG, and that can be put into practice in the
   near future.
4. Carry out a training needs assessment and develop a training program for IDUES staff in essential
   competencies and required activities as indicated by the audits.



                                                   Page 5          ED-OIG / A04-90014
IDUES Response - Attachment 3                                                        Attachment B



Tasks

1. Define the scope of work and the resources needed to do it, both short-term and long-term.
2. Design, obtain OMB clearance when necessary, and carry out a comprehensive and multifaceted
   supplemental data collection effort about grantee institutions. This data collection effort will include
   the following initiatives:

        a. Design, develop, distribute, and analyze a survey to be circulated among a representative
           sample of Title III-Title V institutions in order to obtain information pertinent to the IG audit
           requirements.
        b. Design, develop, distribute, and analyze a customer satisfaction survey about IDUES
           services to be circulated among grantee institutions. Use PricewaterhouseCoopers IDUES
           customer survey as a point of departure.
        c. Develop a protocol and timeline for site visits of a representative sample of institutions. Use
           PMIT Area Representatives or external resources as needed. Explore the possibility of
           replicating the peer review/capacity building model developed by PMIT for the GAANN
           program. Integrate IDUES staff as they complete their pressing grant award commitments.
        d. Design and carry out an expert analysis of a sample of annual and final grantee performance
           reports for the purpose of identifying what data may be extracted, aggregated, and
           compared among institutions using existing performance reports. Make use of the emerging
           HEPInfo initiative [HEP Information and Communication System].
        e. Conduct a review of other existing data collection instruments and institutional research data
           to identify possible additional sources of data. Make use of HEPInfo.
        f. Develop a protocol and carry out focus group sessions among grantees.
        g. Conduct a longitudinal case study of an aggregate number of institutions that have
           participated in Title III over the years.

3. Conclude a collaborative agreement between IDUES and PMIT to obtain PMIT assistance in
   addressing the tasks mentioned herein.
4. Develop a long-range training plan for IDUES staff which addresses the concerns and needs
   pointed out in the IG report, the PwC report, and at division and team meetings.
5. Develop one or more sets of performance indicators for IDUES programs, and a model or models
   for grantee performance reports.

Key players

•   IDUES Director and IDUES senior staff at the first stage; all IDUES staff thereafter.
•   PMIT senior staff and area representatives.
•   Expert consultants and contractors, as needed.




                                                  Page 6          ED-OIG / A04-90014
IDUES Response - Attachment 3                                                        Attachment B


Roles of the expert consultants

•   Consultants with expertise in data mining and analysis, program and institutional assessment, and
    program evaluation and monitoring, as well as in professional development training in those areas.
    Their combined efforts will bring about the development of a reliable system to collect, aggregate
    and analyze pertinent data about grantee institutions and program accomplishments, as well as the
    development of a training plan for IDUES staff to ensure their continued commitment to and
    competence in this system.
•   A pilot project developed by PMIT for use in performance monitoring the GAANN program could
    be applied here. It applies the peer review model, providing training in performance monitoring to a
    cadre of faculty and university officials from grantee institutions, and sending them out to visit other
    institutions as field reviewers for the federal government. This model makes possible two
    accomplishments: more efficient information gathering, and institutional capacity building through
    professional staff development.
•   Contractors, as needed, to speed up the mailing and data entry of the survey instruments.




                                                 Page 7           ED-OIG / A04-90014
                                             FINAL
                               REPORT DISTRIBUTION SCHEDULE
                                 Control No. ED-OIG/A04-90014

                                                                                  No. Of
                                                                                  Copies
 Auditee

        Dr. Claudio Prieto                                                        Original
        Deputy Assistant Secretary
        Higher Education Programs
        U. S. Department of Education
        1990 K Street, NW, 6th Floor
        Washington, D.C. 20006

Action Official

        Dr. Lee Fritschler                                                                 1
        Assistant Secretary
        Office of Postsecondary Education
        U. S. Department of Education
        1990 K Street, NW, 7th Floor, Room 7115
        Washington, D.C. 20006

Other ED Offices (via e-mail, except PES, OGC & OPA)

        HEP- IDUES Senior Advisor, Director and Special Assistant; Audit
        Liaison Officers; PMIT Director & Program Monitoring Team Leader                   6
        OPE- Policy, Planning and Innovation Staff                                         1
        OUS- Postsecondary, Adult, and Vocational Education
        Division Director                                                                  1
        OUS- Planning and Evaluation Service Director and Staff                            4
        OCFO- Grants Policy and Oversight Staff Director                                   1
        OPE- Policy, Budget, and Analysis Staff                                            1
        OGC- General Counsel and Assistant General Counsel                                 3
        OPA- Acting Director of Public Affairs                                       1
        OCFO- Post Audit Group Supervisor                                                  1

Office of Inspector General (via e-mail)

        Inspector General                                                                  1
        Deputy Inspector General                                                           1
        Acting Assistant Inspector General for Audit                                       1
        Acting Deputy Assistant Inspector General for Audit                                1
        State and Local Advisory & Assistance Director and Staff                           2
        Acting Assistant Inspector General for Analysis & Inspection Services              1




                                                             ED-OIG / A04-90014
Counsel to the Inspector General                                           1
Regional Inspectors General for Audit Services                        1 each




                                                 ED-OIG / A04-90014