oversight

U.S. Department of Education's Administration of the Distance Education Demonstration Program.

Published by the Department of Education, Office of Inspector General on 2004-09-30.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

      U.S. Department of Education’s Administration of the
           Distance Education Demonstration Program




                                 FINAL AUDIT REPORT




                                            ED-OIG/A09-D0010
                                             September 2004




Our mission is to promote the efficiency,                      U.S. Department of Education
effectiveness, and integrity of the                            Office of Inspector General
Department’s programs and operations                           Sacramento, California
                                          Notice

Statements that managerial practices need improvements, as well as other conclusions and
recommendations in this report, represent the opinions of the Office of Inspector General.
     Determinations of corrective action to be taken will be made by the appropriate
                           Department of Education Officials.

In accordance with Freedom of Information Act (5 U.S.C. § 552), reports issued by the
  Office of Inspector General are available to members of the press and general public
 to the extent information contained therein is not subject to exemptions under the Act.
                        UNITED STATES DEPARTMENT OF EDUCATION
                                          OFFICE OF INSPECTOR GENERAL


                                                     September 30, 2004
Memorandum
TO:            Sally Stroup
               Assistant Secretary
               Office of Postsecondary Education
               Lead Action Official
               Theresa Shaw
               Chief Operating Officer
               Federal Student Aid

FROM:          Helen Lew /s/
               Assistant Inspector General for Audit

SUBJECT:       Final Audit Report
               U.S. Department of Education’s Administration of the Distance Education
               Demonstration Program
               Control Number ED-OIG/A09-D0010

Attached is the subject final audit report that covers the results of our review of the Department’s
administration Distance Education Demonstration Program from July 1999 through July 2003. An
electronic copy has been provided to your Audit Liaison Officers. We received your comments in
which you did not concur with our finding that the Department’s Second Report to Congress on the
Distance Education Demonstration Program contained unsupported, incomplete, and inaccurate
statements and did not fully agree with the related recommendation, You generally concurred with the
other findings and recommendations presented in our draft report.

Corrective actions proposed (resolution phase) and implemented (closure phase) by your offices will
be monitored and tracked through the Department’s Audit Accountability and Resolution Tracking
System (AARTS). ED policy requires that you develop a final corrective action plan (CAP) for our
review in the automated system within 30 days of the issuance of this report. The CAP should set forth
the specific action items, and targeted completion dates, necessary to implement final corrective
actions on the findings and recommendations contained in this final audit report.

In accordance with the Inspector General Act of 1978, as amended, the Office of Inspector General is
required to report to Congress twice a year on the audits that remain unresolved after six months from
the date of issuance.

In accordance with the Freedom of Information Act (5 U.S.C. §552), reports issued by the Office of
Inspector General are available to members of the press and general public to the extent information
contained therein is not subject to exemptions in the Act.

We appreciate the cooperation given us during this review. If you have any questions, please call
Gloria Pilotti at 916-930-2399.

Enclosure
                                     400 MARYLAND AVE., S.W., WASHINGTON, D.C. 20202-0500
                                                         www.ed.gov

            Our mission is to ensure equal access to education and to promote educational excellence throughout the Nation.
                                          TABLE OF CONTENTS

                                                                                                                                Page

EXECUTIVE SUMMARY .......................................................................................................... 1


BACKGROUND ........................................................................................................................... 2


AUDIT RESULTS ........................................................................................................................ 3

     FINDING NO. 1 – The Second Report Contained Unsupported, Incomplete, and
                     Inaccurate Statements ...........................................................................4


     FINDING NO. 2 – DEDP Participants Did Not Provide Complete and
                     Consistent Information on Annual Reports ......................................13


     FINDING NO. 3 – Department Has Not Submitted Reports to Congress By the
                     Statutory Due Dates.............................................................................16


OBJECTIVES, SCOPE, AND METHODOLOGY ................................................................. 17


STATEMENT ON MANAGEMENT CONTROLS................................................................ 19


Attachment 1 – Issues Identified in DEDP Reports Concerning Adherence to
               HEA Provisions and Federal Regulations .....................................................20

Attachment 2 – Results of OIG Review of Participants' Programs and Enrollment Data
               Reported on the 1998-1999 to 2001-2002 Annual Reports ..........................25

Attachment 3 – Results of OIG Review of Participants' Student Outcome Data
               Reported on 2001-2002 Annual Reports .......................................................26

Attachment 4 – Department Comments on Draft Report......................................................27
ED-OIG/A09-D0010                                                                              Page 1 of 35




                                EXECUTIVE SUMMARY

The Distance Education Demonstration Program (DEDP), which is jointly administered by the
Department’s Office of Postsecondary Education (OPE) and Federal Student Aid (FSA), has not
met the statutory requirement to provide Congress with information on the specific Higher
Education Act (HEA) and regulatory requirements that should be altered to provide students
greater access to distance education programs.
Even though the Department1 has not met the statutory requirement, it could take steps to
improve the information in reports to Congress. We found that—
        The Second Report to Congress on the Distance Education Demonstration Program
        (Second Report) contained statements that were not supported by information in the
        report or other documents provided for our review. The report also contained incomplete
        and inaccurate statements. Reliance on these statements could adversely affect policy
        decisions made by the Department and Congress.
        DEDP participants did not provide complete and consistent information on their annual
        reports. Without complete and consistent data, the Department cannot properly evaluate
        DEDP participants and the impact of waiving HEA provisions and regulations.
        The Department did not submit reports to Congress by the statutory due dates. The
        Congress was provided two-year-old data in the Second Report and has not received
        the information available from the DEDP participants for years 2001-2002 and
        2002-2003 because the Department has not submitted the required reports to Congress
        for 2003 and 2004.
We recommend that the Department establish a review process for DEDP reports to Congress
that ensures information and conclusions presented in the reports are supported, complete, and
accurate. We also recommend that the Department enhance its efforts to obtain complete and
consistent information in DEDP participants’ annual reports and disclose data deficiencies in its
analyses and reports. In addition, we recommend that the Department establish firm timelines
for the completion of data analyses, and DEDP report development, review, and issuance. We
concluded that the Department provided the statutorily required oversight of DEDP participants.
In its response to the draft report, the Department did not agree with our finding that the Second
Report contained unsupported, incomplete, and inaccurate statements and did not fully agree
with the related recommendation. The Department generally agreed with the other findings and
recommendations. The Department expressed concern with the accuracy of information
presented in Attachment 1 of the report, but did not identify any inaccuracies in its comments.
We thoroughly reviewed the support for our findings and are confident the attachment included
in this final report is accurate. The Department’s comments and our response are summarized at
the end of each finding and the introduction to Attachment 1. The comments are presented in
their entirety in Attachment 4.

1
 To simplify the presentation, the term “Department” is used in this report when referring to actions taken
or to be taken by OPE and/or FSA.
ED-OIG/A09-D0010                                                                                Page 2 of 35




                                          BACKGROUND


The Higher Education Act Amendments of 1998, enacted in October 1998, authorized the
DEDP. One of the purposes of the DEDP is to determine the specific statutory and regulatory
requirements that should be altered to provide greater access to high quality distance education
programs. The legislation authorized the Secretary to exempt participating institutions from
certain provisions in the HEA that inhibited their ability to offer distance education. The
Secretary’s waiver authority included the 50 Percent Rules for telecommunications students and
classes,2 the minimum weeks of instruction in an academic year, and regulations implementing
the general provisions of the HEA in Part G of Title IV.

The HEA allows demonstration programs that are strictly monitored by the Department to test
the quality and viability of expanded distance education programs currently restricted under the
HEA. The HEA mandated that the Department provide oversight and required the Department,
on a continuing basis, to (1) assure compliance of institutions, systems or consortia with the
Title IV requirements that had not been waived; (2) provide technical assistance; (3) monitor
fluctuations in the student population enrolled in participating institutions, systems or consortia;
and (4) consult with appropriate accrediting agencies or associations and appropriate State
regulatory authorities.

Beginning July 1, 1999, the Secretary could waive requirements for up to 15 participants (first
year participants). In the third year of the DEDP, which began July 1, 2001, the Department
could expand the program to include additional participants (third year participants). At the
conclusion of our fieldwork, there were 22, involving over 100 institutions from 20 states and the
District of Columbia.3




2
 An institution becomes ineligible to participate in Title IV programs if, for the latest complete award
year— more than 50 percent of the courses offered were correspondence or telecommunication courses,
or 50 percent or more of its regular students were correspondence students. A regular student enrolled in
one or more telecommunications courses is considered a correspondence student when (1) the institution
exceeds the above limit on courses, or (2) the institution does not offer an associate, bachelor, or graduate
degree program or offers more certificate programs than degree programs.
3
    Attachment 3 of this report lists the 22 DEDP participants.
ED-OIG/A09-D0010                                                                         Page 3 of 35




                                    AUDIT RESULTS

The objectives of our review were to determine if 1) the DEDP is meeting the statutory
requirement to provide information on specific statutory and regulatory requirements that should
be altered for distance education programs, and 2) the Department provided the statutorily
required oversight of DEDP participants. Our review covered the period from July 1999
(the beginning of the DEDP) through July 2003.

The Department took appropriate steps to initiate the DEDP, but has not met the statutory
requirement to provide Congress with information on the specific HEA and regulatory
requirements that should be altered to provide greater access to distance education programs.
The Department issued two reports to Congress on the DEDP—the Report to Congress on the
Distance Education Demonstration Programs (First Report), dated January 2001, and the
Second Report, dated July 2003. Neither report identified changes in specific statutory and
regulatory requirements that were needed to provide greater access to high quality distance
education programs.

The First Report and the Second Report did identify issues related to adherence to the 50 Percent
Rules and financial aid determinations for students who enrolled in a changing mix of courses
delivered by different methods to complete their educational programs. However, as the
Department disclosed in its Second Report, other identified issues related to the application of
HEA and regulatory requirements were not unique to distance education. Attachment 1 provides
a summary of the issues presented in the DEDP reports.

Even though the Department has not yet reached conclusions on specific statutory and regulatory
requirements, it could take steps to improve the information provided in its reports to Congress
on the DEDP. We found that the Second Report contained unsupported, incomplete, and
inaccurate statements; DEDP participants did not provide complete and consistent information
on annual reports; and the Department has not submitted reports to Congress by the statutory due
dates.

We concluded that the Department provided the statutorily required oversight of DEDP
participants, which was to, on a continuing basis, (1) assure compliance of participants with
requirements of Title IV (other than the sections and regulations that are waived), (2) provide
technical assistance to participants, (3) monitor fluctuations in participant student population,
and (4) consult with accrediting agencies and state regulatory agencies.
ED-OIG/A09-D0010                                                                               Page 4 of 35




FINDING NO. 1 –The Second Report Contained Unsupported, Incomplete,
               and Inaccurate Statements


The Department has not met the statutory requirement to provide Congress with information on
the specific HEA and regulatory requirements that should be altered to provide students greater
access to distance education programs. However, the Second Report contained statements that
provide information that could be used in evaluating the need for changes in the HEA and
Federal regulations. We found that several of the statements were not supported by information
in the report or other documents provided for our review. We also found statements that were
incomplete and inaccurate.

The Department issued Information Quality Guidelines that provide quality criteria for principal
offices to use in the review of information products that they plan to disseminate to the public.
The Guidelines are applicable to information that the Department disseminates on or after
October 1, 2002. Two of the factors used by the Guidelines to assess information quality are
utility and objectivity.

        Utility refers to the usefulness of the information to its intended users.... To
        maximize the utility of influential information, care must be taken in the review
        stage to ensure that the information can be clearly understood and, where
        appropriate and to the extent practical, an external user of the information can
        reproduce the steps involved in producing the information.

        Objectivity refers to the accuracy, reliability, and unbiased nature of information.
        It is achieved by using reliable information sources and appropriate techniques to
        prepare information products. Objectivity involves both the content and the
        presentation of the information. Content should be complete, include
        documentation of the source of any information used, as well as, when
        appropriate, a description of the sources of any errors in the data that may affect
        the quality of the information product. The presentation of the information should
        be clear and in a proper context so that users can easily understand its meaning.

Reported Median Retention Rates for Programs
Delivered Solely Onsite and Solely Through
Distance Education Were Meaningless

The segment title “Student Success” included the following statements on median retention
rates4 for baccalaureate and graduate degree programs delivered solely with onsite courses and
those programs delivered solely with distance education courses:


4
  To determine the median rate, the DEDP Director listed participant rates from highest to lowest for each
method of delivery (onsite, distance, and mixed) and selected the rate that fell in the middle of each list.
If the list had an even number of entries, the DEDP Director used the average of the two rates in the
middle of the list.
ED-OIG/A09-D0010                                                                       Page 5 of 35


       Data reported by the nine participants that offer baccalaureate degree
       programs…showed higher median retention rates for students enrolled in onsite
       programs than for distance education program enrollees. However, the gap
       between the two narrowed from ten percentage points after two years
       (66% onsite vs. 56% distance education) to just two percentage points after three
       years (50% vs. 48%). [Page 9 of the Second Report.]
                              *              *              *
       [T]he median retention rate [for graduate degree programs] after two years for
       students enrolled in distance education programs is higher than for those enrolled in
       onsite programs (63.5 % vs. 55%). This persists into the third year, where median
       retention rates were 63% and 51%, respectively. [Page 10 of the Second Report.]

The reported median rates were meaningless because the method used to derive the median rates
presented in the Second Report was flawed, and thus, conclusions drawn from the median rates
were unsupported.

       The participant rates used in the analysis varied significantly among the participants.
       For example, the rates after two years for baccalaureate degrees offered through distance
       education ranged from 20 percent to 99 percent. Due to the variance, the median would
       not be a representative measure of student retention rates.

       The analysis included participants that did not offer both onsite and distance programs
       for the degree type. As a result, the lists used to select the median rate did not have a
       corresponding percentage for the other delivery method.

       The analyses included student groups (cohorts) that had few students in the initial
       enrollment. For example, one student cohort for graduate programs offered onsite
       reported only five students enrolled in the programs.

       The participant rates were calculated using student outcome data that were incomplete
       and inconsistent. (Finding No. 2 of the report provides details on the incomplete and
       inconsistent data identified by our review.)

A more appropriate method for evaluating retention/completion for onsite and distance delivery
methods would be to assess the differences in the retention/completion rates for participants that
offer programs both solely onsite and solely through distance education and limit the analysis to
those participants or report years with complete, consistent data and cohorts that have a
minimum number of students.

Conclusion on Impact of Distance Education Methods
on Student Outcomes Was Unsupported

The Second Report included the following statements on the impact correspondence and other
distance education delivery methods had on student outcomes.
ED-OIG/A09-D0010                                                                        Page 6 of 35



       The mode of [distance education] delivery does not appear to be a salient factor
       in student outcomes [for] baccalaureate degree programs [for demonstration
       program participants]. The institutions offering full degrees through
       correspondence report retention rates well above the median. Those participants
       offering only online courses report retention rates that cluster around the median.
       The retention rates reported by participants offering programs through a mix of
       media (which include courses offered by correspondence, interactive video,
       videotape, and online) show much greater variation across institutions. [Page 9 of
       the Second Report.]
                              *               *               *
       As with baccalaureate degree programs, [distance education] delivery mode does
       not seem to be a significant factor affecting retention in these graduate degree
       programs. [Page 10 of the Second Report.]

We were unable to locate data in the report or other documents provided during our review that
supported the Department’s conclusions. DEDP participants’ annual reports did not provide a
breakdown of outcome data by distance education delivery method (i.e., video, audio, internet,
correspondence, and other). The retention rates for individual participants used as a basis for this
conclusion were derived using student outcome data reported on annual reports for degree
programs offered solely through distance education rather than specific distance education
methods. Also, as noted in the prior section, the method used to derive the median rates was
flawed, and as a result, the median rates were meaningless and conclusions drawn from those
rates were unsupported.

Report’s Conclusions
Lacked Sufficient Details

The “Conclusions” segment of the Second Report stated there was “growing consensus” in the
following policy areas:

       •   The current rules that define education delivered via telecommunications and
           videocassette or disc recordings as correspondence education if the total of
           such courses and correspondence meet or exceed 50% of the courses provided
           need to be revised or eliminated.

       •   The definition of correspondence education needs to be revisited.
                                  *             *             *
       •   The quality of distance education programs should be assessed through the
           same accreditation process that governs on-campus programs.

The report did not explicitly state the Department’s position on these changes or explain “growing
consensus.” Also, the report did not explain the basis or impetus for the above statements.
According to the DEDP Director, the statements reflected the consensus of the Department. The
DEDP Director’s supervisor stated the statements reflected the consensus of the entire higher
education community (i.e., educational institutions, the Department, and Congress). Both stated
ED-OIG/A09-D0010                                                                      Page 7 of 35


that the statements were supported by the Department’s experiences in negotiated rulemaking and
ongoing dialogue with the higher education community, DEDP participants, and other institutions.

Inconsistency Across Title IV Programs
Was Not Identified in the Report

The Executive Summary of the Second Report contained the following statement on lack of
consistency across Title IV programs.

       [T]he lack of consistency across the student financial aid programs make it difficult
       for institutions to effectively manage the Federal student aid programs when they
       offer programs in other than standard terms or in a mix of term structures. [Page iv
       of the Second Report.]

We were unable to locate any information in the report or other documents provided for our
review that identified the inconsistency in Title IV programs referred to in this statement.

Executive Summary Contained Inaccurate
Statement on Risks Identified with the 50 Percent Waiver

The Executive Summary included the following statement about risk to Title IV programs.

       The Department has uncovered no evidence that waiving the 50% rules, or any of
       the other rules for which waivers were provided, has resulted in any problems or
       had negative consequences. [Page iv of the Second Report.]

The Department terminated one institution’s participation in the DEDP and the institution
eventually closed. The Department’s termination letter stated “[a]fter evaluating the actions
and integrity of [the institution] during its operation within the DEDP, the Department has
concluded that [the institution] has failed to demonstrate its trustworthiness as a fiduciary,
and has substantially breached minimally acceptable standards of administrative capability.”
Waiving the 50 Percent Rules would allow rapid growth in programs offered through
distance education methods, and this growth could put stress on an institution’s financial
viability and administrative capacity. Waiving the 50 Percent Rules for the subsequently
terminated institution contributed to the institution’s problems by allowing the institution to
rapidly expand its enrollment in distance education programs and courses. This substantially
increased the number of students and Title IV dollars impacted by the institution’s lack of
administrative capability. The report did not disclose this negative consequence of the
50 Percent waiver.
ED-OIG/A09-D0010                                                                      Page 8 of 35


No Information on Efficiencies in Providing
Title IV Aid That Resulted From Waivers

The Introduction section included the following statement regarding the waivers.
       Participants have received waivers of certain statutory and regulatory provisions
       governing the HEA Title IV student financial assistance programs to enable them
       to provide Title IV aid to distance education students more efficiently and, in
       some instances, to expand their distance education programs beyond otherwise
       applicable statutory limits.
The report provides information on how the waiver of the 50 Percent Rules allowed DEDP
participants to expand their distance education program. However, we did not locate any
information in the report that identified the efficiencies experienced by DEDP participants in
awarding Title IV to distance education students that resulted from waivers.

Department Needs to Improve Procedures for
Review of DEDP Reports to Congress

The DEDP Director provided the following description of the process used for the Second
Report. The Director prepared an outline of the proposed information to be included in the
report and provided the outline to DEDP team members for comment. The information
presented in the Second Report was gleaned from DEDP participants’ annual reports, FSA case
team monitoring reports, and periodic meetings with case team members and DEDP participants.
The Director stated she used insights that the Department gained from its experiences in
administering student financial assistance programs. The Director designed the report format
and consulted with OPE and FSA Program Specialists on the presentation of technical
information. The DEDP Director’s supervisor informed us that he, the OPE Deputy Assistant
Secretary, and the DEDP Director held several conversations regarding the report.

The OPE and FSA Program Specialists and the Director’s supervisor were provided a draft of
the report for their review and comment. The DEDP Director stated that the report was then
provided to OPE’s Assistant Secretary and Deputy Assistant Secretary for review and comment.
The following Department offices also reviewed the report as part of the internal clearance
process: Office of the Secretary, Office of the Under Secretary, Office of the Deputy Secretary,
Office of the General Counsel, and Office of Legislation and Congressional Affairs. The DEDP
Director stated that reviews were generally limited to evaluating the report’s content and
providing editorial comments. The DEDP Director’s supervisor informed us that he did review
the data and conclusions and confirmed that statements were supported. These reviews were not
sufficient to ensure that information and conclusions presented in the report were adequately
supported, complete, and accurate.

Recommendation
1.1    The Assistant Secretary for Postsecondary Education in collaboration with the Chief
       Operating Officer for Federal Student Aid should enhance its review process for DEDP
       reports to Congress to ensures that information and conclusions presented in the reports
       are adequately supported, complete, and accurate and adheres to the Department’s
       Information Quality Guidelines.
ED-OIG/A09-D0010                                                                          Page 9 of 35


Department Comments and OIG Response

Finding 1 of this final report includes material originally presented in an attachment to the draft
report. In response to the Department’s comments, we omitted some material from the
attachment and decided the remaining material should be incorporated into Finding 1, rather than
be included as a separate attachment. Except for the material now omitted, references in the
Department’s comments to Attachment 2 refer to material that now appears in Finding 1.

Department Comment. The Department did not agree with the finding and did not fully agree
with the recommendation. In its comments on the draft report, the Department stated that the
auditors used a narrow definition of data and approached the Second Report as if it were an audit
report. The Department explained that the Second Report drew on all information and data
available and not simply data that were readily available.

OIG Response. The OIG reviewed all information and data made available by the Department.
However, we did not accept as supporting data for statements in the Second Report the
Department staff’s subsequent oral statements to the auditors for which there was no
contemporaneous record of Departmental discussions or analyses.

Contrary to the Department’s suggestion, we did not approach the Second Report as if it should
have been prepared according to generally accepted government auditing standards. While there
are similarities in the Department’s Information Quality Guidelines and generally accepted
government auditing standards, we used only the Information Quality Guidelines to evaluate the
Second Report. The sections of the Guidelines cited earlier state that “where appropriate and to
the extent practical, an external user of the information can reproduce the steps involved in
producing the information.” For an external user to reproduce the steps, the report statements
need to be supported by information in the report or other documents. Also, the earlier cited
sections of the Guidelines state that to ensure information is objective (i.e., accurate, reliable, and
unbiased), the Department should use “reliable information sources and appropriate techniques
to prepare information products.” The cited section also states that “[c]ontent should be
complete” and “information should be clear and in a proper context.”

Department Comment. The Department stated that it has a well-established review and
clearance process, which was followed for both DEDP reports to Congress. The Department
stated the process involves substantive review of content, as well as data analysis. Thus, the
Department did not agree with the recommendation to establish such a process. However, the
Department did agree, to the extent practical, to provide reviewing offices with an annotated
index of data sources used in compiling reports, including disclosure of the known limitations of
the information contained in those data sources.

OIG Response. The OIG acknowledged the Department’s review and clearance process in the
audit report and, based on the Department’s comments, revised the recommendation to reflect
the need to enhance the current process. However, due to the findings from our review of
statements in the Second Report, we have not changed our position that the current process was
not sufficient to ensure that information and conclusions presented in the report were adequately
supported, complete, and accurate. The planned corrective action would improve that process;
however, the process does not have written procedures for ensuring that supervisory or other
reviews have confirmed that reports to Congress meet the Department’s Information Quality
ED-OIG/A09-D0010                                                                     Page 10 of 35


Guidelines. Without written procedures, there is no assurance as to the extent and thoroughness
of the reviews. The Department’s corrective action should include steps to develop and
implement written procedures that ensure either the supervisory review or other review confirms
that reports to Congress adhere to the Department’s Information Quality Guidelines.

The Department’s comments addressed individual sections of the Second Report that were cited
in the Draft Report.

Reported Median Retention Rates for Programs
Delivered Solely Onsite and Solely Through
Distance Education Were Meaningless

Department Comment. The Department agreed that the median retention rates for baccalaureate
and graduate degree programs were not meaningful. The Department stated that the Second
Report quotation included in the audit report omitted cautionary statements on the effect that the
high percentage of part-time students has on the rates that were vital to understanding the issue.
OIG Response. The Department’s comment refers to two sentences in the Second Report that we
did not take exception to, and thus, did not include in the quotation. These two “cautionary
statements” were not sufficient to alert the reader to the fundamental flaws in the median rate
data, which the Department now agrees were not meaningful.

Conclusion on Impact of Distance Education Methods
on Student Outcomes Was Unsupported

Department Comment. The Department stated that it provided the auditors with information
about the delivery modes for programs offered by each participant and that such information was
included in the Second Report. The Department stated that it also drew on information gleaned
from its monitoring reports, DEDP participants’ websites, and the DEDP Director’s interaction
with the participants over a three-year period through onsite visits, telephone calls and emails,
and participants’ meetings.

OIG Response. We agree that information was provided identifying the various delivery
methods used by DEDP participants; however, the Department provided no information on the
retention rates of specific delivery methods.

Report’s Conclusions
Lacked Sufficient Details

Department Comment. The Department stated that it was not appropriate to propose policy
changes in the Second Report. The Department stated that statutory changes must be submitted
to the Office of Management and Budget (OMB) for approval prior to submission to the
Congress and regulatory changes are made through the negotiated rulemaking process. The
Department further stated that it was not the Department’s responsibility to reconcile the OMB
and negotiated rulemaking requirements with the requirements for the DEDP. The Department
stated that the report contained adequate documentation for the opinion on policy changes and
that the opinion was not subject to the Department’s Information Quality Guidelines.
ED-OIG/A09-D0010                                                                        Page 11 of 35


OIG Response. The negotiated rulemaking requirements and OMB legislative clearance process
do not preclude the Department from meeting the Distance Education Demonstration Program
reporting requirements. While Title IV regulatory changes must go through negotiated
rulemaking, these requirements do not prevent the Department from reporting its intention to
hold negotiated rulemaking on areas of the regulations that need to be changed. As for the OMB
clearance process, the Department can have the reports cleared by OMB as part of the report
review process or where timeliness is a concern, issue the report without OMB clearance as
allowed under OMB Circular A-19. As provided for in Circular A-19, “If congressional time
schedules do not allow an agency to send its proposed report to OMB in time for the normal
clearance and advice, the agency shall consult informally with OMB as to the advice to be
included in the proposed report. OMB may advise the agency to state in its report that time has
not permitted securing advice from OMB as to the relationship of the proposed legislation to the
program of the President.”

The Department’s Information Quality Guidelines do not cover “[o]pinions that are clearly
identified as such, and that do not represent facts or the agency’s views. . . .” As we stated in the
audit finding, the Second Report did not clearly identify whose opinion was being expressed in
the statements on policy changes. However, we were informed that the statements did reflect the
consensus of the Department. Since the opinions on policy changes reflect agency views, the
opinions would not be exempt from the Information Quality Guidelines. Based on our review of
the Second Report, we must disagree with the Department’s conclusion that the report contained
adequate documentation for the opinions on policy changes. We found no discussion in the
Second Report explaining the basis for the statements.

Inconsistency Across Title IV Programs
Was Not Identified in the Report

Department Comment. The Department stated that it has provided technical assistance to
schools that experienced difficulty in applying Title IV rules in academic program structures
other than standard terms, and that illustrations of those difficulties were provided in Department
presentations.

OIG Response. The Department’s comments relate to difficulties in applying Title IV rules.
The comments did not address the OIG’s conclusion that the Second Report did not identify the
inconsistencies in Title IV programs referred to in the cited statement from the Second Report.

No Information on Efficiencies in Providing
Title IV Aid That Resulted From Waivers

Department Comment. The Department stated the OIG incorrectly concluded that neither the
Second Report nor other documents provided to the auditors identified efficiencies in providing
Title IV aid that resulted from waivers. The Department stated that monitoring reports included
information showing that the waiver of the definition of a full-time student allowed enrollment
status for correspondence students to be computed in the same manner as other students. The
Department also noted that the waiver of the 12-hour rule eliminated the need for institutions
offering nonterm and nonstandard term programs to document that full-time students were
engaged in the required hours for a week of instruction.
ED-OIG/A09-D0010                                                                       Page 12 of 35


OIG Response. While it is reasonable to anticipate that efficiencies would occur as a result of
certain waivers, the Second Report did not identify the efficiencies (e.g., reduction in waste,
expense, or unnecessary effort) resulting from the full-time student definition, 12-hour rule, or
other waivers. The Second Report would have been more useful if the Department had identified
the actual efficiencies that individual DEDP participants experienced as a result of the waivers.
We omitted the reference to “other documents” from the final report since we did find general
references to efficiencies in effort in monitoring reports reviewed for our sample of DEDP
participants.

Department Comment. The Department disagreed with the OIG conclusions in the draft audit
report that the Second Report incorrectly stated that “the HEA and Department regulations are
based on traditional patterns of higher education.” The Department also disagreed with the OIG
conclusions that certain statements about the ease of administering non-term programs and the
amount of aid received in those programs were unsupported or misleading.

OIG Response. After considering comments received on the draft report and further review of
monitoring reports, we revised our conclusion and eliminated the related sections from the final
report. The HEA and Department regulations may be based on traditional patterns of higher
education, but the regulations implementing the HEA do provide for non-traditional patterns and
methods, such as programs offered in non-standard terms and non-term structures and using
correspondence and telecommunications methods of instruction.

The Second Report did contain a reference to computer systems, which states schools’ decisions
to structure their distance education program in standard terms is driven by the capabilities of
computer systems, which are designed to handle traditional academic structures. The Second
Report contains no mention of the expense of modifying computer systems or the use of manual
rather than electronic processes. However, we did identify some general references in the
monitoring reports we reviewed for the sampled DEDP participants. The Second Report also
mentioned the difficulties that schools using standard terms have as they add courses with start
and end dates that overlap the standard terms, but continue to consider their programs as standard
term programs. The monitoring reports contained evidence that schools limited student flexibility
in order to retain the standard term structure.

The Department is correct that the Second Report provided information on the different
disbursement rules for standard terms and nonterm/nonstandard terms, but this information is not
near the cited statement, which is in the concluding paragraphs of the Second Report. Thus, a
reader could mistakenly conclude that students are eligible for less aid solely due to the fact that
they are enrolled in a nonterm program. Nevertheless, while the statement in the Second Report
could have been written in a better manner, it is technically correct – students sometimes receive
less aid in the nonterm environment as compared to the standard term.

The Department did not comment on the section titled “Executive Summary Contained
Inaccurate Statement on Risks Identified with the 50 Percent Waiver.”
ED-OIG/A09-D0010                                                                            Page 13 of 35




FINDING NO. 2 – DEDP Participants Did Not Provide Complete and
                Consistent Information on Annual Reports


Educational program and related enrollment numbers provided in DEDP participants’ annual
reports were often incomplete. Also, student outcome data were incomplete and inconsistent.5
The HEA § 486(f)(1) requires the Department to evaluate participants’ demonstration programs
on an annual basis. The evaluation is to include a review of “the number and types of students
participating in the programs offered [by each participant], including the progress of
participating students toward recognized certificates or degrees and the extent to which
participation in such programs increased.” To assist the Department in conducting such
evaluations, the participants in the DEDP are required to provide annual reports. Without
complete and consistent data from the annual reports, the Department cannot perform a thorough
evaluation of DEDP participants and the impact of waiving HEA provisions and Federal
regulations.

DEDP Participants Did Not Provide Complete Information on
the Number of Educational Programs and Related Enrollments

The annual report form was designed to provide the number of offered programs and program
enrollments by program type and delivery method. We found that 11 of the 22 DEDP participants
did not report the number of programs and enrollments for one or more certificate, associate,
bachelor, or graduate programs (distance, onsite, or mixed) for which they reported student
outcome data in another section of the annual report. One DEDP participant, which the Second
Report identified as having certificate programs, did not report the number of certificate programs
offered or the number of students enrolled in such programs. Without complete data on programs
and enrollments, the Department cannot fully perform trend analyses to assess the impact that
waivers of the 50 Percent Rules or other requirements may have had on the number of offered
programs and enrollments. The Department has no assurance that it has provided accurate program
and enrollment data in its reports to Congress. Attachment 2 of this report identifies the reporting
deficiencies in program and enrollment data found in the annual reports for the individual DEDP
participants.

DEDP Participants Provided Incomplete and
Inconsistent Student Outcome Data

The annual report required DEDP participants to report on cohorts of students. Students enrolled
in the school year were divided into cohorts based on the program type (certificate, associate,
bachelors, or graduate) and delivery method (distance education courses only, onsite courses
only, or a mix of distance education and onsite courses). Each year new cohorts of students were
added to the report for students enrolled during the latest year. The form was designed to collect

5
  From our limited review, we concluded that data on the number of courses and related enrollments and
student surveys appeared complete and consistent. It also appears that DEDP participants provided
complete and consistent data on distance delivery methods, except for one participant that did not provide
delivery method data for 2001-2002.
ED-OIG/A09-D0010                                                                            Page 14 of 35


the following numbers for each cohort: students enrolled, students remaining in the program or
institution, graduates or students that completed the program, and students that previously
graduated or completed the program. The data was collected for each year that the cohort had
students who continued to be enrolled at the institution.

The DEDP participants that provided incomplete data generally provided enrollment data for the
first year of a cohort but did not provide either the number of students remaining in the program
or institution, or the number of students that graduated/completed, or both. Outcome data were
not always provided for the subsequent years in the cohort and participants did not report student
outcome data for one or more certificate, associate, bachelor, or graduate programs (distance,
onsite, or mixed) for which they reported programs and enrollments in another section of the
annual report. One participant, cited in the Second Report as having certificate programs, did not
provide outcome data for such programs.

We found that only five DEDP participants provided complete and consistent student outcome
data. Of the remaining 17 participants—

          Seven participants provided both inconsistent and incomplete student outcome data,
          Six participants provided inconsistent student outcome data, and
          Four participants provided incomplete student outcome data.

Attachment 3 of this report identifies the deficiencies in student outcome data found in the
annual reports for the individual DEDP participants.

Generally, the inconsistencies identified by our review related to two areas that affected the
retention and/or completion rates for the cohort of students.6

          Eleven participants reported a number for students that previously graduated or
          completed which was inconsistent with numbers reported for graduated or completed in
          earlier years for the cohort. This inconsistency affected the accuracy of the completion
          rate.

          Six participants reported students remaining in the program and institution, previously
          graduated or completed, and graduated or completed that exceeded the total enrollment
          for the cohort of students. Therefore, the retention rate, completion rate, or both would
          be inaccurate depending on where the reporting error occurred.

The completeness and consistency of the student outcome data is essential to accurately calculate
the DEDP participants’ retention and completion rates for each cohort and year. As we
discussed earlier in the report, the Department used the student outcome data to draw
conclusions on retention rates for students taking distance and/or onsite courses for graduate,
baccalaureate, and associate programs.7




6
    Four of the 13 participants with inconsistent data had data problems in both areas.
7
  For these comparisons, the Department used the student outcome data reported by first year participants
in their 2000-2001 annual reports.
ED-OIG/A09-D0010                                                                      Page 15 of 35


Department Needs to Expand
Its Review of Annual Reports

The Department performed only a limited review of the data submitted by DEDP participants on
their annual reports. The DEDP Director stated that she compared current year data with prior
years’ data and followed up two or three times with individual participants whose data were
obviously incomplete or erroneous. This review was not sufficient to ensure that reported data
were complete and consistent.

Recommendation

2.1 The Assistant Secretary for Postsecondary Education in collaboration with the Chief
       Operating Officer for Federal Student Aid should enhance its efforts to obtain complete
       and consistent information in participants’ annual reports. If incomplete or inconsistent
       data are used to evaluate and draw conclusions regarding the DEDP and its participants,
       the data deficiencies should be fully disclosed in the analyses and reports containing the
       results of the analyses or conclusions drawn from them.

Department’s Comments

The Department agreed with our finding and our recommendation, but contends that the data
could not have been significantly improved. The Department stated that Congress asked the
Department to report data in ways that DEDP participants do not maintain the data. The
Department stated that in some instances the DEDP participants could not extract the needed
data from their school systems, for example, data on students who obtained certificates while
enrolled in degree programs and then drop out of the degree programs. Also, consortia and
school systems lacked leverage to require data from schools that did not offer distance education
or were not granted waivers.

The Department stated that while they worked with the DEDP participants to improve the data,
the Department did not have the leverage to require better data. Many DEDP participants did not
need waivers and the only recourse available was to end their participation in the program. The
Department stated that it would continue to work with DEDP participants to obtain the best data
possible and would disclose data deficiencies in analyses and reports containing results of the
analyses and conclusions drawn from them.
ED-OIG/A09-D0010                                                                      Page 16 of 35




FINDING NO. 3 – Department Has Not Submitted Reports to Congress By
                the Statutory Due Dates

The Department met the statutory due date for the First Report when it issued the report in
January 2001, which was 18 months after the initiation of the DEDP. However, the Second
Report was issued late and contained outdated data. Two subsequent reports are overdue. The
Department did not establish firm timelines for preparing data analyses, report development,
review and issuance of the reports to Congress.

The HEA § 486(f)(3)(A) required the Department to report to the Congress 18 months after
initiation of the DEDP. Section 486 (f)(3)(B) required that the Department provide reports to
Congress on an annual basis after the initial report. Since the First Report was issued in January
2001, the Second Report was due January 2002. The Second Report was issued July 2003,
which was 18 months late. Due to the delay in its issuance, the Second Report presented
information and conclusions based on two-year-old participant data derived from participants’
annual reports for 2000-2001. At the time of its issuance, the Department had already received
annual reports for 2001-2002 from DEDP participants and the period covered by the 2002-2003
annual reports had just ended.

At the time of our review, the Department had not yet started to prepare its next report to
Congress or conduct analyses of participant data reported in annual reports for 2001-2002 and
2002-2003. Thus, subsequent reports are also late and, as a result, Congress does not have the
latest information available for policy decision-making.

Recommendation

3.1 The Assistant Secretary for Postsecondary Education in collaboration with the Chief
       Operating Officer for Federal Student Aid should establish a firm timeline for data
       analyses, report development, review, and issuance to ensure that DEDP reports to
       Congress are provided timely.

Department’s Comments

The Department acknowledged that the HEA stipulates annual reporting, but stated it had not
received any requests from Congress for additional reports. The Department explained that the
timing of the reports was driven by the Department’s assessment of having something to report
and that it plans to issue a third report about the 2003 competition for additional DEDP
participants and trend and cost data by December 31, 2004. The Department plans to issue
future reports based on the schedule specified in the HEA, as amended.

OIG Response

To ensure that reports are issued timely, the Department should establish a firm timeline for
completion of data analyses, report development, and review for the third report and other future
reports.
ED-OIG/A09-D0010                                                                          Page 17 of 35




                OBJECTIVES, SCOPE, AND METHODOLOGY

Our audit objectives were to determine if 1) the DEDP is meeting the statutory requirement to
provide information on specific statutory and regulatory requirements that should be altered for
distance education programs, and 2) the Department provided the statutorily required oversight
of DEDP participants. Our review covered the period from July 1999 through July 2003.

To accomplish our objectives, we interviewed the DEDP Director, OPE’s Policy, Budget, and
Analysis staff, and FSA Case Management Oversight (CMO) staff, who had various roles in the
administration of the DEDP. We gained an understanding of the process for the solicitation of
applicants, the application process, the evaluation of applicants, and selection of participants.
We reviewed available documents relevant to the process. We confirmed that a diverse group of
participants was selected and identified the participants’ goals for participation in the program
and the waivers granted to each participant. We evaluated the annual report process for
participants and the Department staff’s analyses of reported data. We performed analyses of the
reported data to assess increases in the number of programs, courses and enrollments and
changes in student outcomes and student survey results.

For a judgmental sample of 9 of the 22 current participants, we reviewed the participants’
applications (including goals), evaluation documents completed as part of the selection process,
use of waivers granted, and monitoring documentation. The universe of 22 participants
consisted of 13 from the first year and 9 from the third year of the DEDP.8 We stratified the
universe by the five types of participants: public, non-profit, for-profit, consortium, and school
system. Based on the distance education enrollments at each of the participants, we decided to
select two public participants, two private non-profit participants, three for-profit participants,
one consortium, and one school system.

The below table lists the five participants in each category with the largest enrollment that were
selected for the sample.
                                                                              Distance
                     Participant Name                        Type of                        Accrediting
                                                                             Education
              (Initial Year of Participation)               Participant                      Agency
                                                                             Enrollment
    University of Maryland University College (1st year)      Public           10,000      Middle States
                            rd
    University of Phoenix (3 year)                           For-Profit        10,000      North Central
                                                 st
    Connecticut Distance Learning Consortium (1 year)       Consortium          1,000      New England
                                       st
    North Dakota University System (1 year)                School System        5,000      North Central
                       rd
    Regis University (3 year)                               Non-Profit          5,000      North Central




8
 Originally, there were 16 participants from the first year of the DEDP. At the time we selected the
sample, one participant had been removed from the program and one participant withdrew voluntarily.
Also, we eliminated Western Governors University from the sampling universe because the university
was covered by specific HEA provisions that allowed additional waivers.
ED-OIG/A09-D0010                                                                         Page 18 of 35


The following table lists the four additional participants selected to obtain the desired number of
participants from each type and ensure that our sample included participants accredited by
different agencies.

                                                                        Distance
                  Participant Name                     Type of                          Accrediting
                                                                       Education
           (Initial Year of Participation)            Participant                        Agency
                                                                       Enrollment
Eastern Oregon University (3rd year)                     Public          3,000           Northwest
Southern Christian University (1st year)              Non-Profit            500           Southern
                      rd
Walden University (3 year)                             For Profit         1,000         North Central
                      st
Capella University (1 year)                            For Profit         3,000         North Central

We assessed the DEDP instructions for conducting monitoring reviews at participating
institutions and reviewed the participant files and monitoring reports for the nine selected
participants. We reviewed the Department’s Report to Congress on the Distance Education
Demonstration Programs, dated January 2001, and Second Report to Congress on the Distance
Education Demonstration Program, dated July 2003. We obtained information on the one
participant that was dropped from the DEDP and the three participants that elected to leave the
program.

To conduct analyses used in achieving our objectives, we relied on annual report data submitted
by the DEDP participants on electronic worksheets. Our assessment of the reliability of the data
on the annual reports was limited to (1) gaining an understanding of the procedures used by the
Department to collect and review the data and (2) confirming that data were complete and
consistent with other data in the annual report and other documentation provided for our review.
We did not compare the reported data to participants’ institutional records. For the 22 DEDP
participants that were participating in the program at the time of our review, we reviewed the
data reported on number of courses and total enrollment in courses,9 number of program and
total enrollment in programs,10 delivery methods for distance education courses,11 student
outcomes,12 and student survey results.13 For data on number of courses, programs, and
enrollments, our review covered the annual reports for 1998-1999, 1999-2000, 2000-2001 and


9
 Annual Report Table 1A: Number of Distance Education Courses and Total Enrollment and Table 1B:
Number of Onsite Courses and Total Enrollment.
10
  Annual Report Table 1C: Number of Distance Education Programs and Total Enrollments, Table 1D:
Number of Onsite Programs and Total Enrollments, and Table 1E: Number of Programs Offered in Both
Distance Education and Onsite Formats and Total Enrollments (Combination of distance education and
onsite courses)
11
     Annual Report Table 2A: Methods of Delivery Currently Used to Deliver Distance Education Courses
12
  Annual Report Table 5A: Student Outcome for Distance Education Programs, Table 5B: Student
Outcome for Onsite Programs, and Table 5C: Student Outcome for Programs Offered Both Onsite and By
Distance Education
13
  Annual Report Table 6A: Summary of Student Survey for Students Enrolled in Distance Education
Programs and Table 6B: Summary of Student Survey for Onsite Students Enrolled in One
or More Distance Education Courses
ED-OIG/A09-D0010                                                                  Page 19 of 35


2001-2002. For data on student outcomes and student survey results, our review covered annual
reports for 2001-2002.

We performed our initial fieldwork at OPE’s offices in Washington, D.C. during the two-week
period from March 31 through April 10, 2003 and performed subsequent analyses in our
Sacramento office. We conducted additional interviews at OPE’s offices on October 30
and 31, 2003. An exit conference was held with Department officials on March 29, 2004. We
performed our audit in accordance with generally accepted government auditing standards
appropriate to the scope of audit described.




            STATEMENT ON MANAGEMENT CONTROLS

As part of our review, we gained an understanding of the management structure for the DEDP.
Our assessment of the Department’s management control structure was limited to those areas of
control weaknesses identified while we gained an understanding of the DEDP and reviewed
related documents. Based on our review, we concluded that the Department could improve its
management of the DEDP by implementing procedures to ensure that report statements are
supported, complete, and accurate; additional reviews of participants’ annual reports are
conducted; and analyses are promptly completed and reports are issued to Congress timely.
These control weaknesses are discussed in the AUDIT RESULTS section of this report.
ED-OIG/A09-D0010                                                                       Page 20 of 35




                                         Attachment 1
                 Issues Identified in DEDP Reports Concerning
              Adherence to HEA Provisions and Federal Regulations

The First Report and Second Report identified issues related to adherence to the 50 Percent Rules
and financial aid determinations for students who enrolled in a changing mix of courses delivered
by different methods to complete their educational programs. However, other issues identified in
the reports were not unique to distance education. Even though DEDP participants experienced
difficulties in these areas, with technical assistance from Department staff, the DEDP participants
were able to implement procedures that met the requirements of the HEA and applicable
regulations.

In its comments on the draft report, the Department stated that Attachment 1 demonstrated
fundamental misunderstandings of the issues and did not correctly describe the issues identified
in its reports to Congress or the HEA provisions or Federal regulations. The Department stated
that the OIG had failed to incorporate most edits suggested from its review of a preliminary
draft. The Department did not identify any specific inaccuracy or misunderstanding in
Attachment 1 in its comments on the draft report.

In response to the Department’s comments on the draft report, we performed a second review of
the Department’s suggested edits to the preliminary draft of the attachment. The Department’s
edits were often matters of style and emphasis and identified only one instance that may have
been technically inaccurate in the attachment included in the draft report. In our draft report, we
stated “The HEA provisions and regulations that comprise the 50 Percent Rules, which we have
condensed above, are contained in HEA § 102(a)(3), and 484(1)(1), 34 C.F.R. § 660.7 and
34 C.F.R. § 668.38.” We eliminated “that comprise the 50 Percent Rules” from the statement as
it appears on page 21 of this report. We accepted the Department’s suggestion that eliminating
the phrase was appropriate because HEA § 484(1)(1) and 34 C.F.R. § 668.38 relate to student
eligibility and, while the sections are used to determine which students to count for purposes of
the 50 Percent Rules, these sections of the HEA and regulations would not technically be
considered part of the 50 Percent Rules, which involves institutional eligibility.

In order to be responsive to the Department, where we deemed appropriate, we made clarifying
changes based on the suggested edits. We are confidant that Attachment 1, as presented in this
final report, fully and accurately describes the issues discussed and correctly describes the laws
and regulations cited.

Title IV Issues Identified That Were
Unique to Distance Education

Application of 50 Percent Rules for Institutional Eligibility. The 50 Percent Rules, which
require institutions to monitor the number of distance education courses and students, could
influence decisions on whether to expand distance education offerings. An institution becomes
ineligible to participate in Title IV programs if, for the latest complete award year—
ED-OIG/A09-D0010                                                                         Page 21 of 35


       More than fifty percent of the courses offered were correspondence or telecommunication
       courses, or

       Fifty percent or more of its regular students were enrolled in one or more correspondence
       courses.

       A regular student enrolled in one or more telecommunications courses is considered
       enrolled in a correspondence course when (1) the institution exceeds the above limit on
       courses, (2) the institution offers more certificate programs than degree programs, or
       (3) the student is enrolled in a certificate program of less than one academic year.

The HEA provisions and regulations, which we have condensed above, are contained in HEA §
102(a)(3), HEA § 484(l)(1), 34 C.F.R. § 600.7, and 34 C.F.R. § 668.38.

The Department reported in the Second Report that seven DEDP participants used waivers of the
50 Percent Rules to continue participation in the Title IV programs. The Department also
reported considerable growth in the number of students enrolled in participants’ courses offered
through distance education methods, especially students who are taking both onsite and distance
education courses. Our review of DEDP participants’ annual reports for 2001-2002 found that
20 of the 22 participants14 reported increases in distance education courses, enrollments, or both.

The Department also cited confusion in interpretation of the interrelation of the regulations at
34 C.F.R.§ 600.7 regarding institutional eligibility and 34 C.F.R. § 668.38 regarding student
eligibility. The regulation at 34 C.F.R. § 668.38 lists additional eligibility requirements for
students enrolled in correspondence and telecommunications courses. Under paragraph (b), a
student enrolled in a telecommunications course is considered enrolled in a correspondence
course unless “at least 50 percent of the programs of study offered by the institution during its
latest completed award year led to an associate, bachelor, or graduate degree.” Thus, an
institution that offers a large number of certificate programs must take care to have one more
degree program than certificate programs.

Determination of Educational Programs as Correspondence Programs. Institutions may
provide students with the option of taking courses offered by correspondence,
telecommunications, or onsite to meet program requirements. The institution determines whether
a program is a correspondence, telecommunications, or onsite program for Title IV purposes
based on the predominant method of instruction. The determination would become more
problematic as institutions use a changing mix of delivery methods for courses offered under the
programs. A program could be identified as a correspondence program in one academic year and
a telecommunications or onsite program in another year depending on the mix of course delivery
methods.

The determination that a program is a correspondence program impacts a student’s eligibility for
Title IV programs and the amount and timing of Title IV funds received by the student. The

14
   One participant, LDS Church Education System, reported decreases in both distance education courses
and enrollments. The other participant was Western Governors University (WGU), which does not offer
its own courses. Students enrolled in programs at WGU take courses at other institutions in order to
prepare for compentency examinations that are required for completion of their degree.
ED-OIG/A09-D0010                                                                       Page 22 of 35


HEA and regulations contain provisions that place the following additional requirements and
limitations on providing Title IV funds to students enrolled in correspondence programs.

       A student enrolled in a correspondence program is not eligible to receive Title IV funds if
       the student is enrolled in a certificate or other non-degree program.

       A student enrolled solely in correspondence study cannot be considered more than a half-
       time student.

       For students enrolled in a correspondence program, the cost of attendance is restricted to
       tuition and fees, and, if required, books and supplies. Travel and room and board costs
       can only be included if they are incurred specifically in fulfilling a required period of
       residential training.

        Before receiving a payment under the Federal Supplemental Educational Opportunity
        Grant Program, a student enrolled in a correspondence program must submit the first
        completed lesson for the course.

        Before receiving a payment under the Federal Pell Grant Program, a student enrolled in a
        correspondence program must complete a specified percentage of lessons. The
        percentage varies depending on whether the course is a term-based or nonterm-based
        course.

Eligibility Determinations for Students Enrolled in Other Than Correspondence Programs
Who Are Taking a Mix of Correspondence, Telecommunications, and Onsite Courses.

Students enrolled in other than correspondence programs who are taking courses through a mix
of delivery methods that include correspondence or telecommunication courses may also be
subject to additional requirements. The combination of courses taken during the award year can
impact the student’s eligibility under Title IV programs.

       A student enrolled in a correspondence course can only receive Title IV funds for the
       correspondence course if the course is part of an educational program that leads to an
       associate, bachelor, or graduate degree.

       A student enrolled in a telecommunications course that is part of a certificate program
       can only receive Title IV funds for the telecommunications course if the certificate
       program is one year or more in length.

The combination of courses taken during the award year by a student can also impact the
student’s enrollment status. The student’s enrollment status (less than half-time, half-time, three-
quarter time, or full-time) affects the amount of Federal Pell Grant funds that a student is eligible
to receive.

       If a correspondence course is taken with other courses, the correspondence course must
       meet the following criteria to be included in determining the student’s enrollment status:
       (1) the course must be applied toward the student’s degree or certificate or must be
ED-OIG/A09-D0010                                                                              Page 23 of 35


        remedial work to help the student in the course of study, and (2) the course must be
        completed during the period required for the student’s regular coursework.

         When combining the number of credit hours of correspondence courses with the number
         of credit hours of other courses to determine a student’s enrollment status for a Pell
         Grant, the amount of correspondence courses counted cannot be more than the number
         of credit hours of the other courses.15

Thus, for students enrolled in programs other than correspondence programs who are taking
courses through a changing mix of delivery methods, the institution must make a determination
each term and payment period on whether the student is enrolled in correspondence or
telecommunications courses and assess the impact on the student’s eligibility for Title IV funds.

Lastly, if the institution happens to offer one more certificate program than degree programs
during the award year, the institution must ensure that the above special conditions for students
taking a correspondence course are also applied to students taking a telecommunications course.

Other Identified Title IV Issues That
Were Not Unique to Distance Education

The Department reported that DEDP participants encountered challenges in administering
Title IV funds in an academic year with standard terms when students enrolled in courses with
different term structures, consortial arrangements, and competency-based models. Also, the
DEDP participants experienced difficulties providing breaks to students in programs with
continuous enrollment. The Department acknowledged in its reports that these difficulties were
not unique to distance education methods.

Standard Term-Based Academic Year. The structure of an academic year established by the
institution for its educational programs affects a student’s eligibility for Title IV funds and the
delivery of those funds. For programs that are offered in standard term-based academic years,
the terms are periods of the same duration during which all classes are scheduled to begin and
end. The term is used when determining the student’s enrollment status (i.e. full-time, half-time,
etc.) and the payment periods for Title IV funds. The Department reported that DEDP
participants experienced difficulties determining the amount of Title IV funds that students were
eligible to receive when the scheduled beginning and ending dates of one or more courses were
not within the standard term. The Department concluded in its report that this difficulty created
barriers to increasing course start dates.




15
  If the student is taking at least a half-time load of correspondence courses, the student would be paid as
at least a half-time student, regardless of the credit hours of regular coursework.
ED-OIG/A09-D0010                                                                      Page 24 of 35


Consortial Arrangements. The enrollment status of a student attending more than one
institution under a consortium agreement is based on all the courses taken that apply to the
educational program. When institutions in the consortium use different academic calendars and
terms, the institution responsible for disbursing Title IV funds to the student must translate the
coursework to its own terms. The Department reported that the administrative burden of the
coursework translations and monitoring student progress at separate institutions was a barrier to
increased use of consortial arrangements.

Competency-Based Models. Under a competency-based model, students only need to
demonstrate that they have acquired the necessary knowledge and skill to receive credit for an
individual course or educational program. Students may take a varying amount of time to
acquire the knowledge and skill and demonstrate their competencies. The Department reported
that institutions using competency-based models, including two DEDP participants, experienced
difficulties establishing the academic year and length of educational programs to comply with
the HEA and regulatory provisions that could not be waived under the DEDP, and determining
students’ enrollment status, cost of attendance, and satisfactory academic progress. The
Department concluded that the annual limits on Title IV funds are a barrier to one of the
objectives of the competency-based model – the acceleration of time to degree for students who
are able to demonstrate their competency on portions of course or program content.

Scheduled Breaks. The academic year established by the institution for its educational
programs may include scheduled breaks between terms. The Department reported that three
DEDP participants did not establish scheduled breaks in the academic years used for their
educational programs since the institutions expected students to be continuously enrolled in their
programs (year around enrollments). The Department reported that these institutions were
challenged to develop policies that allowed students to take breaks that complied with the HEA
and Federal regulations.
  ED-OIG/A09-D0010                                                                                Page 25 of 35




                                                Attachment 2
       Results of OIG Review of Participants’ Programs and Enrollment Data
              Reported on the 1998-1999 to 2001-2002 Annual Reports

  This table identifies the DEDP participants that did not report the number of programs and
  enrollments for one or more certificate, associate, bachelor, or graduate programs for the specified
  delivery system.

                                                                                                   Programs with
                                                               Distance Only      Onsite Only      Mix of Onsite
                       Participants
                                                                 Programs          Programs         and Distance
                                                                                                      Courses
First Year Participants (a)
1     Capella University                                                                X                 X
2     Connecticut Distance Learning Consortium                        X                                   X
3     Franklin University                                                                                 X
4     LDS Church Education System
5     New York University (b)
6     North Dakota University System                                  X                 X                 X
7     Quest Education Corporation/Kaplan College
8     Southern Christian University                                   X                 X
9     Texas Tech University
10 University of Maryland University College                          X                 X                 X(c)
11 Washington Community and Technical College
      System
12 Washington State University                                        X                                   X
13 Western Governors University
Third Year Participants
14      American InterContinental University
15      Brevard Community College                                     X
16      Eastern Oregon University
17      JesuitNet Consortium
18      Marlboro College
19      Regis University                                                                                  X
20      University of Phoenix                                         X
21      United States Sports Academy                                  X                 X
22      Walden University
(a) Three of the first year participants were no longer participating in the DEDP at the time of our review.
    Masters Institute was removed from the DEDP in October 2000. Community Colleges of Colorado and
    Florida State University voluntary withdrew from the program in June 2002 and June 2003, respectively.
(b) After our fieldwork, the DEDP Director informed us that New York University had voluntarily withdrawn
    from the DEDP effective June 2003.
(c) The Second Report identified the institution as having certificate programs, but the institution did not report
    the number of certificate programs and related student enrollment in its annual reports.
ED-OIG/A09-D0010                                                                             Page 26 of 35



                                           Attachment 3
          Results of OIG Review of Participants’ Student Outcome Data
                      Reported on 2001-2002 Annual Reports

This table identifies the DEDP participants that did not report complete and/or consistent student
outcome data on their annual report.

                                                               Incomplete                        Data Both
                                                                              Inconsistent
                       Participants                               Data                         Incomplete &
                                                                                 Data
                                                                                                Inconsistent
First Year Participants (a)
1    Capella University                                                           X (c)
2    Connecticut Distance Learning Consortium                                                      X (d)
3    Franklin University                                                          X (c)
4    LDS Church Education System                                                                  X (c)(d)
5    New York University (b)
6    North Dakota University System                                              X (c)(d)
7    Quest Education Corporation/Kaplan College                                                    X (d)
8    Southern Christian University                                                                 X (c)
9    Texas Tech University                                                                         X (c)
10 University of Maryland University College                        X
11 Washington Community and Technical College                       X
     System
12 Washington State University                                                                     X (c)
13 Western Governors University                                                   X (c)
Third Year Participants
14 American InterContinental University
15 Brevard Community College                                                     X (c)(d)
16 Eastern Oregon University                                        X
17 JesuitNet Consortium                                             X
18 Marlboro College
19 Regis University                                                                                X (c)
20 University of Phoenix                                                         X (c)(d)
21 United States Sports Academy
22 Walden University
     Totals                                                         4               6                7
(a) See table note (a) on Attachment 2.
(b) See table note (b) on Attachment 2.
(c) Participant reported a number for students that previously graduated or completed that was inconsistent
    with numbers reported for graduated or completed in earlier years for the cohort (11 participants).
(d) Participant had reported students remaining in the program and institution, previously graduated or
    completed, and graduated or completed that exceeded the total enrollment for the cohort of students
    (six participants).
ED-OIG/A09-D0010                                                              Page 27 of 35




                                  Attachment 4

                            Department Comments
                               on Draft Report

            The Department’s memorandum refers to page numbers in the draft report.
            Due to the addition of the Department’s comments and our response, the
            page numbers in the memorandum no longer refer to the related sections
            of the report.
                      UNITED STATES DEPARTMENT OF EDUCATION
                                    OFFICE OF POSTSECONDARY EDUCATION

                                                                                                 THE ASSISTANT SECRETARY



MEMORANDUM

DATE:         June 30, 2004

TO:           Gloria Pilotti
              Regional Inspector General for Audit

FROM:         Sally L. Stroup /s/

SUBJECT: Comments on Draft Audit Report - U.S. Department of Education’s
Administration of the Distance Education Demonstration Program (DEDP) (Control
Number ED-OIG/A09-D0010)

The Department of Education has reviewed the Office of the Inspector General’s (OIG)
draft audit report of the U.S. Department of Education’s Administration of the Distance
Education Demonstration Program (Control Number ED-OIG/A09-D0010) and is
pleased to have the opportunity to provide written comments. Our response includes
comments on the draft report’s Attachments 1 and 2.

FINDING NO. 1 – The Second Report Contained Unsupported, Incomplete And
Inaccurate Statements.

For the most part, the Department does not agree with this finding, which it perceives to
be largely a function of two problems with the approach taken by the OIG. The first
problem is that the OIG auditors viewed the Distance Education Demonstration Program
from the other sources of information the Department drew upon to write the Report to
Congress. This resulted in a very narrow definition of “data.” The second problem is
that the OIG approached the Report to Congress and the internal review process, as if it
were an audit report, which it is not. The Report to Congress and all similar reports
developed by the Department in response to congressional directives draw upon all
information and data available and not simply the data that is readily available (such as
the reports submitted by participants under the Distance Education Demonstration Program).

Median retention rates
The OIG concluded that the reported median retention rates for programs delivered solely
through distance education were meaningless. The Department agrees that use of the
median retention rates for baccalaureate and graduate degree programs was not
meaningful and will not use this approach in the future. However, the Department notes
that the OIG report contains only a partial quote about data reported by institutions
offering baccalaureate degree programs (p.4) and that the additional information about



                                      1990 K STREET, N.W. WASHINGTON, D.C. 20006

          Our mission is to ensure equal access to education and to promote educational excellence throughout the Nation.
Page 2 – Gloria Pilotti


the effect of the high percentage of part-time students on the data and the cautionary
statement that follows are vital to understanding the issue (p 9, Second Report to
Congress).

Conclusion on Impact of Distance Education Methods on Student Outcomes was
Unsupported
The draft report states that the Department’s “Conclusion on impact of distance education
methods on student outcomes was unsupported.” (pp. 5-6). The OIG report states that the
auditors were unable to locate data in the report, or other documents, provided that tied
delivery method to retention rates for various institutions and programs. The Department
provided the auditors with copies of all the applications and the Agreements to
Participate in the Distance Education Demonstration Program. These documents have
information about the delivery modes for programs offered by each participant. In
addition, the Second Report to Congress (pp. 3-4) described various delivery modes and
indicated which participants employed each. The Department also drew on information
gleaned from monitoring reports (which were provided to the auditors), from
participants’ Web sites (addresses of which were provided to the auditors), and from the
director’s interaction with the participants over a three-year period through onsite visits,
telephone calls and emails, and participants’ meetings.

Report Conclusions Lacked Sufficient Details
The OIG draft reports states that the “Report’s conclusions lacked sufficient details.” (p.
6-7). The OIG report faults the Department for not explicitly stating its position on three
policy areas. As the Department explained to the OIG staff, it is not appropriate for
policy changes to be proposed in a Report to Congress. The process for making
regulatory changes is through negotiated rulemaking. Statutory changes must be
submitted to the Office of Management and Budget, which has the authority to approve
and authorize the Secretary to submit legislation to Congress. This will be done within
the broader context of the reauthorization of the Higher Education Act (HEA). Further, it
is not the Department’s responsibility to reconcile these requirements with the
requirements in the Distance Education Demonstration Program legislation.

The OIG report further faults the Department for not explaining the use of the term
“growing consensus” in the “Conclusions” section of the Second Report to Congress.
The Department believes there was adequate documentation in the report of this opinion,
which is not subject to the Information Quality Guidelines.

Department Needs to Improve Procedures for Reviews of Distance Education
Demonstration Program Reports to Congress
The draft report states that the “Department needs to improve procedures for reviews of
DEDP Reports to Congress.” The Department has a well-established review and
clearance process. This process was followed for both Reports to Congress. The process
involves substantive review of the content, as well as the data analysis. The OIG
presumes that the Distance Education Demonstration Program is the only source of
information for the Report to Congress. The OIG fails to recognize that the
Demonstration Program exists within a context. The Department’s experience in
Page 3 – Gloria Pilotti


working with a large number of schools over many years allows the Department to look
at the evidence gleaned from the Demonstration Program and come to an understanding
of what it means.

RECOMMENDATION

1.1    The Assistant Secretary for Postsecondary education in collaboration with the
Chief Operating Officer for Federal Student Aid should establish a review process for
Distance Education Demonstration Program reports to Congress that ensures information
and conclusions presented in the reports are adequately supported, complete and accurate,
and adheres to the Department’s Information Quality Guidelines.

RESPONSE

1.1    The Department has a well-established process for reviewing reports to Congress.
The Assistant Secretary and the Chief Operating Officer will assure that this process is followed.
To the extent practical, the reviewing offices will be provided with an annotated index of the
data sources used in compiling the report that notes known limitations of the information
contained in those data sources.

FINDING NO. 2 – Distance Education Demonstration Program Participants Did
Not Provide Complete And Consistent Information On Annual Reports.

The Department agrees that the data provided by Distance Education Demonstration
Program participants has not been complete and consistent. However, for the following
reasons the Department does not agree that the data quality could have been improved
significantly:

   •   Congress asked the Department to report data in ways that institutions do not
       necessarily maintain it. This presented an enormous challenge to participants,
       who do not receive any funds from the Department to help offset the costs of their
       participation. Still, most of the program participants have expended considerable
       resources to extract the requested information.

   •   In some instances, Congress asked for data that schools could not extract. For
       example, one institution offers a large number of certificate programs, which
       consist of a subset of courses that are part of a degree program. Students enroll in
       the degree program, but may drop out after completing one or more certificate
       courses sequences, or may request that they be awarded certificates along with the
       degree. The institution is not able to identify these students as enrolled in
       certificate programs. Thus, institutions are unable to report certificate program
       enrollments.

   •   Several participants were consortia or systems of institutions. Only some of the
       individual institutions in these consortia and systems offer distance education
       programs and were granted waivers, yet all were expected to report data. The
Page 4 – Gloria Pilotti


       lead organization has often lacked leverage to get these institutions to extract and
       report the data in the form requested by Congress.

   •   The Department has been working with participants to improve the quality of the
       data. At some point, the program manager made the determination that the data
       were as good as we would get. Since the Department’s leverage with
       program participants is the granting of waivers and many of the program
       participants do not need any waivers, the only recourse available to the
       Department is to end the participation of some of the program participants. This
       would result in the Department losing access to data, rather than increasing our
       access to data.

RECOMMENDATION

2.1     The Assistant Secretary for Postsecondary Education in collaboration with the
Chief Operating Officer for Federal Student Aid should enhance its efforts to obtain
complete and consistent information in participant’s annual reports. If incomplete or
inconsistent data are used to evaluate or draw conclusions regarding the Distance
Education Demonstration Program and its participants, the data deficiencies should be
fully disclosed in the analyses and reports containing the results of the analyses or
conclusions drawn from them.

RESPONSE

2.1     The program manager and associated staff will continue to work with institutions
to obtain the best data possible given the constraints described above, and will disclose
data deficiencies in the analyses and reports containing results of the analyses or
conclusions drawn from them.

FINDING NO. 3 – Department Has Not Submitted Reports To Congress By The
Statutory Due Dates.

While the statute stipulates annual reporting, the Department has not received any
requests from Congress for additional reports. The Department issued two reports to
congress (one in January 2001 and a second in July 2003) and plans to issue a third report
during 2004. The timing of these reports is driven by the Department’s assessment that
there is something to report.

The first Report to Congress provided details about how the Department is implementing
the program, some baseline information about participants, and an overview of the
student aid issues. The Second Report to Congress updated the participant information to
include the new group of participants and provided detailed information about some of
the challenges institutions face with administering Title IV student financial assistance
programs when they attempt to enhance student access and flexibility. The Department
expects to include information about the 2003 competition and some trend and cost data
in the 2004 report.
Page 5 – Gloria Pilotti


RECOMMENDATION

3.1    The Assistant Secretary for Postsecondary Education in collaboration with the
Chief Operating Officer for Federal Student Aid should establish a firm timeline for data
analyses, report development, review, and issuance to ensure that the Distance Education
Demonstration Program reports to Congress are provided timely.

RESPONSE

3.1    The Department will issue a third report to Congress before the end of the 2004
calendar year. Future reports will be submitted based on the schedule included in the
HEA as amended over the next year through the reauthorization process.


Attachment 1 – Issues Identified in Distance Education Demonstration Program
Reports Concerning Adherence to HEA Provisions and Federal Regulations

The Department is extremely concerned with the contents of Attachment 1, which
contains numerous errors and demonstrates fundamental misunderstandings of the issues
identified in the reports to Congress. After reviewing the preliminary draft, the
Department agreed to edit the material extensively so that it would correctly describe the
issues, HEA provisions, and federal regulations. However, the OIG failed to incorporate
most of these edits.

Attachment 2 – Additional Statements in the Second Report That Were
Unsupported, Incomplete, or Inaccurate

OIG NOTE: The content of Attachment 2 of the Draft Report was incorporated into Finding 1 in
the Final Report.

Attachment 2 provides a listing of statements or conclusions in the Second Report to
Congress that the OIG believes were unsupported, incomplete, or inaccurate. The first
statement, from page iv of the Second Report to Congress, concerns inconsistencies across
Title IV programs. The OIG states it was unable tolocate any information in the report
or other documents to support this statement. The Department noted in the exit
conference that staff regularly provide technical assistance to schools that are
experiencing great difficulty in applying Title IV rules to their academic program
structures that are other than standard term, and that the Department makes presentations
several times a year to financial aid administrators about this topic using an 80-slide
presentation containing numerous illustrations of the difficulties. In failing to
acknowledge this broader context, the OIG inappropriately limits the data sources it
considers to a small subset of the Department’s growing base of knowledge about these
issues.
The statement in Attachment 2 that the Department provided “no information on
efficiencies in providing Title IV aid that resulted from waivers” either in the Report to
Congress or in information in other documents is incorrect. As an example, the
monitoring reports included information about the waiver of a definition of full-time
Page 6 – Gloria Pilotti


student that allowed enrollment status for correspondence students to be computed in the
same manner as that of other students. In addition, the waiver of the 12-hour rule meant
that institutions offering nonterm and nonstandard term distance education programs
were not required to document that full-time students were engaged in 12 hours of
regularly scheduled instruction, examination, or preparation for examination in each
week for it to be counted as a week of instructional time. The burden this regulation
imposed on institutions and students was discussed in the University of Phoenix’s
application and in the Report to Congress on Student Financial Assistance and Non-
Traditional Educational Programs (including “The Twelve-Hour Rule”), which was
posted on the Distance Education Demonstration Program Web site and available to OIG
staff.

OIG NOTE: The remaining portion of the memorandum relates to a section of the Draft Report
that was omitted in the Final Report.

The 12-hour rule is also a good illustration of how “the HEA and Department regulations
are based on traditional patterns of higher education”, the next statement from the Second
Report to Congress that the OIG identifies as inaccurate. The 12-hour rule is directly
derived from standards (the Carnegie unit) used in traditional education, where a certain
amount of classroom instruction (1 hour per semester credit) was estimated to take place
each week. Likewise, the definition of an academic year in the HEA of 30 weeks of
instructional time is based on two 15-week semesters or three 10-week quarters, both
standard term models. An additional example is the different standard students are
expected to meet if they are enrolled in nonterm programs whereby they must
successfully complete all courses for which they have received federal financial aid prior
to getting another disbursement. This is based on the presumption that programs in a
nonterm structure are vocational training programs, not degree programs. Instructional
units in vocational training programs are sequential and build upon prior work. Students
enrolled in such programs are not able to benefit from the instruction if they have not
mastered the prior content.
Attachment 2 concludes by citing two statements from the “Conclusions” section of the
Report to Congress that the OIG claims were unsupported (that administering aid in a
nonterm structure is a “formidable task.”) or misleading (that students sometimes receive
less aid in the nonterm environment as compared to the standard term). The Department
does not agree with the OIG’s judgment on either of these statements for the following
reasons.
              The Report to Congress contained information about the significant challenges
   schools face in administering aid for nonterm programs due to the fact that computer
   systems are designed to support a single (standard term) model. If the nonterm
   structure is the only one used by a school, then the computer system may be modified
   at considerable expense to support it. However, schools that use a mix of academic
   program structures (some standard term, some nonterm or nonstandard term) must
   administer aid for the nonterm and nonstandard term programs manually. Evidence
   was provided in the Report to Congress that some schools have limited the flexibility
   for students in order to retain the standard term structure because of these challenges.
   In addition, the Department’s experience with the University of Phoenix and Western
   Governors University, both of which use nonterm models, has revealed many
Page 7 – Gloria Pilotti


   difficulties with return of Title IV aid and the use of borrower-based academic year
   for loans.

              In discussing the issue of students enrolled in nonterm programs potentially
   receiving less aid than those enrolled in nonstandard and standard term programs, the Report
   to Congress cited the higher standard that students in nonterm programs must meet in
   order to get additional disbursements. These students must successfully complete all
   credits for which they have been paid whereas students in standard and nonstandard
   term programs just have to complete the credits. To meet the “complete” standard, a
   student could withdraw or get an F and then take the same course again and get
   additional financial aid for it. The OIG argues that the Department’s statement is
   misleading since students in a nonterm program can get paid for the same course if
   they complete the education program and re-enroll to take that program again or take
   another program. (This is also true of students in nonstandard and standard term
   programs.) The OIG is pointing to a special instance, which is not typical for
   students. This exception does not undermine the argument made in the Report to
   Congress that there is a lack of parity.

If you need additional information, please contact David Bergeron, Director of Policy and
Budget Development at 202-502-7815

Attachment