United States General Accounting Office GAO Report to the Chairman, Subcommittee on Human Resources, Committee on Government Reform and Oversight, House of Representatives June 1997 PROPRIETARY SCHOOLS Poorer Student Outcomes at Schools That Rely More on Federal Student Aid GAO/HEHS-97-103 United States GAO General Accounting Office Washington, D.C. 20548 Health, Education, and Human Services Division B-276560 June 13, 1997 The Honorable Christopher Shays Chairman, Subcommittee on Human Resources Committee on Government Reform and Oversight House of Representatives Dear Mr. Chairman: Under title IV of the Higher Education Act of 1965, as amended (HEA), the federal government annually spends billions of dollars on various grant and loan programs to assist students seeking postsecondary education and training.1 In the late 1980s and early 1990s, high student loan default rates attracted increased congressional attention. This attention focused in part on proprietary schools—private, for-profit institutions primarily offering vocational training—because their default rates were higher than those for nonprofit postsecondary institutions. For example, in fiscal year 1994, the average student loan cohort default rate2 at proprietary schools was 21 percent, compared with 14 and 7 percent at 2- and 4-year nonprofit colleges, respectively. Each percentage point of proprietary schools’ average default rate costs the government about $5 million annually.3 In response to problems in the proprietary sector, the Congress, in 1992, added a provision to the HEA requiring that proprietary institutions obtain at least 15 percent of their revenues from sources other than title IV student financial aid programs; schools failing to meet the 15-percent threshold lose their title IV eligibility. The rationale behind this provision, known as the “85-15 rule,” is that schools providing a quality educational product should be able to attract a reasonable percentage of their revenues from sources other than title IV. Supporters of the provision said it was intended to “weed out” the “bad” proprietary schools. Given continued concerns about proprietary school performance, you asked us to explore the relationship between school performance and reliance on title IV funds in the proprietary school sector. To meet this objective, we performed a variety of statistical analyses using data from 1 Student financial aid programs authorized under title IV include Pell grants, Federal Family Education Loans (FFEL), Federal Direct Student Loans (FDSL), Perkins loans, and Supplemental Educational Opportunity Grants. 2 The cohort default rate is measured as the percentage of students entering repayment on FFEL and FDSL loans in a fiscal year who default on their loan in that or the succeeding year. We refer to this as the “default rate.” 3 This figure is based on 1992 data, the most recent available. Page 1 GAO/HEHS-97-103 Proprietary Schools and Student Aid B-276560 over 900 proprietary institutions that participated in title IV during 1994 and 1995 to determine whether or not a greater reliance on title IV is associated with poorer school performance measures.4 We sent a questionnaire to these schools to ascertain the percentage of each school’s total revenues received from title IV, a percentage we refer to as the “85-15 measure.” We classified schools as high reliance, medium reliance, or low reliance on the basis of the relative value of their 85-15 measures. As indicators of school performance, we used data on three measures of student outcomes: (1) program completion, (2) training-related placement, and (3) student loan default rates. The Department of Education uses each of these outcomes to some extent as quality measures for gatekeeping—the process of ensuring that students receiving title IV funds attend only schools that provide quality education and training programs. Completion rates generally represent the percentage of students starting an education or training program who complete the program within a designated time period. Placement rates generally represent the percentage of students completing a program who are placed in jobs related to their field of training.5 We conducted our work from May 1996 to April 1997 in accordance with generally accepted government auditing standards. We checked all data for internal consistency, called accrediting agencies and schools in some cases to obtain corrected data, and excluded schools from the analysis in cases where inconsistent data could not be corrected. For a complete discussion of scope and methodological issues, definitions of completion 4 These schools were accredited by five national accrediting agencies that together accredit a large majority of the proprietary schools eligible for title IV programs. Accrediting agencies are nongovernmental, voluntary associations that review educational institutions and their professional programs to ensure a consistent level of performance, integrity, and quality. The five accrediting agencies were (1) the Accrediting Bureau of Health Education Schools (ABHES), which accredits schools training students for jobs in the health professions, such as medical assistants and lab technicians; (2) the Accrediting Council for Continuing Education & Training (ACCET), which accredits schools that train students in a wide variety of fields including computer technology and paralegal and secretarial services; (3) the Accrediting Commission of Career Schools and Colleges of Technology (ACCSCT), which accredits schools that teach paralegal, computer, and electrical technology skills, among many others; (4) the Accrediting Council for Independent Colleges and Schools (ACICS), which accredits schools training students for primarily business-related occupations, such as secretaries and bookkeepers; and (5) the National Accrediting Commission of Cosmetology Arts & Sciences (NACCAS), which accredits schools that train in the cosmetology profession, such as barbers, hair stylists, and manicurists. 5 Because each agency reported completion and placement data differently, our completion and placement rate measures were not defined consistently, requiring us to test the relationship between the 85-15 measure and these measures separately by agency. Because default rates have a standard definition, we tested the relationship between the 85-15 measure and the default rate by aggregating data from all five agencies. Page 2 GAO/HEHS-97-103 Proprietary Schools and Student Aid B-276560 and placement rates for each agency’s schools, and limitations of our study, see appendix I. Proprietary schools that relied more heavily on title IV funds tended to Results in Brief have poorer student outcomes. Our analysis showed that, on average, the higher a school’s reliance on title IV, the lower its students’ completion and placement rates, and the higher its students’ default rates. Although reliance on title IV was a significant factor in explaining completion and default rates, it was not significant in explaining placement rates. Requiring proprietary schools to obtain a higher percentage of their revenues from non-title-IV sources could save millions in default claims. Based on our analysis, however, achieving this result would require a substantial increase to the current 15-percent threshold. This is because, in relative terms, large differences in schools’ 85-15 measures are associated with small differences in outcomes. For example, raising the threshold to 45 percent could improve the average default rate of schools currently relying the most on title IV funds to the level of those that rely the least—3 percentage points lower—for an estimated annual savings of $11 million. However, a standard this high might cause schools to make changes, such as admitting fewer low-income students, that might compromise student access to postsecondary education. Since 1972, when proprietary school students became eligible for the full Background range of title IV grant and loan programs, proprietary schools’ students have consistently accounted for a disproportionate share of defaults. For example, in fiscal year 1991, proprietary school students held 35 percent of loans entering repayment but accounted for 71 percent of those who defaulted in fiscal years 1991 and 1992. Default claims associated with these proprietary school students’ loans totaled $140 million. In response to high default rates, the Congress enacted several legislative requirements proprietary schools must meet for title IV eligibility. One such measure, the 85-15 rule, became part of the HEA in 1992. This rule requires each school to calculate a percentage: The title IV dollars its students receive is the numerator, and total revenues from its educational programs make up the denominator. This percentage cannot exceed 85 percent; an independent accountant must certify to Education that this Page 3 GAO/HEHS-97-103 Proprietary Schools and Student Aid B-276560 calculation is correct.6 The 85-15 rule is similar to one applicable to veterans’ benefits.7 Considerable controversy arose over Education’s implementing regulations that defined “revenues” for the 85-15 calculation and required that schools base their first year’s calculations on the fiscal year prior to the regulations’ publication. Under Education’s regulatory definition, schools cannot include revenues from certain contracts—for example, to train a group of workers for an employer if the course does not meet title IV eligibility criteria—in the denominator. Critics warned that using prior-year data could force many proprietary schools, even those with good student outcomes, to close because it would not provide them ample opportunity to comply with the new rule. In response, the Congress delayed the effective date of the final 85-15 regulations 1 year, until July 1, 1995. Even some lawmakers who supported this delay generally agreed that the basic intent of the 85-15 rule was good and that the concept behind the rule made sense. A few members of the Congress, however, suggested the 85-15 rule needed more study, such as examining the nature of the relationship between revenue sources and school performance. Some observers believe a threshold higher than the current 15 percent would be more effective. Others favor basing regulations on performance measures, such as those already employed as gatekeeping tools. For example, default rates already play a major role in governing program participation: Schools with default rates exceeding 25 percent for 3 successive years can lose eligibility for student loan programs, and schools with rates exceeding 40 percent in a single year can lose eligibility for all title IV aid. In addition, students in short-term programs8 cannot receive title IV aid unless these programs have completion and placement rates of at least 70 percent. 6 As of July 1, 1997, proprietary schools no longer need this attestation but instead must disclose, in their annual audited financial statements, the percentage of their revenues derived from title IV funds. 7 Veterans’ benefits may not be used to pay for postsecondary education instruction when more than 85 percent of program participants have all or part of their education benefits paid for by the educational institution or the Department of Veterans Affairs. As initially proposed, the 85-15 rule would have focused on the percentage of students receiving aid, similar to the veterans’ benefits rule; as ultimately passed, the 85-15 rule focuses instead on the percentage of school revenues coming from title IV programs. 8 Short-term programs are defined as those with fewer than 600 clock hours of instruction. A 60-week program where students meet for 10 hours a week, and a 15-week program where students meet for 40 hours a week, are both 600 clock hour programs. Students cannot receive title IV aid for a program with fewer than 300 clock hours. Page 4 GAO/HEHS-97-103 Proprietary Schools and Student Aid B-276560 Schools that relied more heavily on title IV funds generally had poorer Significant student outcomes. High-reliance schools had lower completion and Relationship Between placement rates and higher default rates than low-reliance schools. Reliance on Title IV Regression analysis substantiated the significance9 of the relationship with completion and default rates but not with placement rates. and Performance Measures Completion rates for schools that relied heavily on title IV funds were lower than for schools that relied on title IV to a lesser extent (see fig. 1). For schools accredited by four of the five accrediting agencies, high-reliance schools had an average completion rate more than 10 percentage points lower than low-reliance schools. Across the board, high-reliance schools had the lowest completion rates. For the four accrediting agencies’ schools, we found significant correlations between reliance on title IV and completion rates; regression analysis confirmed the relationship’s significance. 9 “Significance” refers to statistical significance at the 5-percent confidence level. This significance means that we can be 95 percent certain that a measured association is not due to chance or random variation. Page 5 GAO/HEHS-97-103 Proprietary Schools and Student Aid B-276560 Figure 1: Schools With High Reliance on Title IV Funds Had Lowest Completion Rates Completion Rate (Percent) 100 80 75 75 69 69 68 66 65 60 61 60 57 58 50 50 46 43 40 20 0 ABHES ACCET ACCSCT ACICS NACCAS Accrediting Agency's Schools High-Reliance Schools Medium-Reliance Schools Low-Reliance Schools Note: Definitions of completion rate and low-, medium-, and high-reliance schools vary by agency. Generally, placement rates for schools that relied heavily on title IV funds were slightly lower than low-reliance schools (see fig. 2). Correlations between placement rates and the 85-15 measure were negative and significant for schools from three agencies; for schools from the other two agencies, the correlations were not significant. However, our regression analysis showed that reliance on title IV funds was not a significant factor in explaining placement rates. While correlation analysis examines the relationship of two variables in the absence of information about other influential factors, regression analysis illuminates how other factors exert their own influence on the outcome; accounting for these factors, the relationship was no longer significant. Page 6 GAO/HEHS-97-103 Proprietary Schools and Student Aid B-276560 Figure 2: Schools With High Reliance on Title IV Funds Had Lowest Placement Rates Placement Rate (Percent) 100 87 84 77 79 79 80 74 75 75 74 74 71 71 71 66 68 60 40 20 0 ABHES ACCET ACCSCT ACICS NACCAS Accrediting Agency's Schools High-Reliance Schools Medium-Reliance Schools Low-Reliance Schools Note: Definitions of placement rate and low-, medium-, and high-reliance schools vary by agency. Default rates at schools with high reliance on title IV were higher, on average, than those at schools with medium or low reliance. Schools in the highest one-third of the distribution of the 85-15 measure had an average default rate 4 percentage points higher than schools in the lowest one-third (see fig. 3).10 We found a significant relationship between default rates and schools’ reliance on title IV funds using both correlation and regression analyses. 10 This pattern generally held for each agency separately. Default rates were lowest in the low-reliance group for four of the five agencies and were highest in the high-reliance group for three of the five agencies. Page 7 GAO/HEHS-97-103 Proprietary Schools and Student Aid B-276560 Figure 3: Schools With High Reliance on Title IV Funds Had Highest Default Rates Default Rate (Percent) 30 25 20 20 19 16 15 10 5 0 High-Reliance Medium-Reliance Low-Reliance Schools For more detailed results, including sample sizes, break points for 85-15 measure categories and correlation results for each agency, regression results, and results of sensitivity analyses, see appendix II. Increasing the 85-15 rule’s 15-percent threshold—requiring a higher A Significantly Higher percentage of total revenues from non-title-IV sources—could save 85-15 Threshold millions of dollars annually by reducing default claims. However, because, Would Likely Reduce in relative terms, large differences in schools’ 85-15 measures are associated with small differences in outcomes, it would take a substantial Defaults but Might increase to attain the outcomes demonstrated by schools that rely the Impair Student Access least on title IV. Furthermore, impacts on students’ access to postsecondary education would depend on how schools react. A far more stringent standard would be required to materially improve the effectiveness of the 85-15 rule. Each percentage point difference in a school’s level of reliance on title IV funds is associated with about a 0.27 percentage point difference in its completion rate and about a 0.11 Page 8 GAO/HEHS-97-103 Proprietary Schools and Student Aid B-276560 percentage point difference in the default rate. A significantly higher threshold could save millions in default claims. For illustrative purposes, consider the results achieved by redefining the 85-15 rule to include only schools classified in our sample as low-reliance, or tripling the 15-percent threshold to 45 percent. Take a school that receives 80 percent of its revenues from title IV and has a completion rate of 70 percent. Compare this school to another one identical in all respects to the first, except it receives only 50 percent of its revenues from title IV. Our analysis suggests the second school would have a 78-percent completion rate—8 percentage points higher than the first. Similarly, if the school with the higher reliance on title IV has a default rate of 20 percent, the school with less reliance would be expected to have a 17-percent default rate—3 percentage points lower. If high- and medium-reliance schools’ default rates decreased to the low-reliance school level—that is, if the results illustrated by this example could be achieved across the proprietary school sector—resulting annual default claims savings could be about $11 million. However, the effect of raising the 15-percent threshold on students’ access to postsecondary education would depend on how the affected schools would react to such a change. Two somewhat extreme assumptions illustrate how savings could be achieved without affecting access. One such assumption underlies our savings estimate: all high- and medium-reliance schools in our sample would, among other things, successfully reduce their reliance on title IV and remain eligible for the program, for example by enhancing the quality of their programs and thereby attracting other revenue sources, without changing the characteristics of their student bodies.11 Similar savings would be predicted under a different, also extreme, assumption: All high- and medium-reliance schools become ineligible to participate in title IV, but all their students transfer to other title-IV-eligible proprietary schools.12 On the other hand, meeting a higher standard may cause schools to change their behavior in ways that compromise student access. For example, as a means of reducing revenues from title IV, higher-reliance schools might admit fewer low-income financial aid recipients. Also, if some schools fail to meet the new standard and close, remaining title IV-eligible schools 11 A further assumption is that other characteristics of each school and its students do not change. 12 This example also assumes that the remaining schools have the capacity to absorb these students and the students take on the lower default rates of the new schools to which they transfer. Page 9 GAO/HEHS-97-103 Proprietary Schools and Student Aid B-276560 might not have the capacity to absorb all their students, forcing some students out of higher education altogether. Our results generally support the notion underlying the 85-15 rule—that Conclusions greater reliance on federal financial aid funds by proprietary schools is associated with poorer student outcomes. Overall, the descriptive statistics, the number of significant correlation results, and the regression analysis confirming the correlations for two of the three performance measures indicate students attending proprietary schools that rely heavily on federal student aid as a revenue source fare worse—in terms of completion and default rates—than students at proprietary schools that rely less on student aid. A more stringent standard than the current 85-15 rule could save millions of dollars but also might have unintended consequences. Because a small change to the 15-percent threshold would not materially improve school outcomes, such as lower default rates, a rather large change would be necessary. However, a significantly higher threshold could adversely affect student access because schools may be limited in their ability to reduce reliance on title IV funds without displacing some low-income students. We provided a draft copy of this report to Education for review. We Agency Comments discussed the draft with Education officials, who generally agreed with our findings and conclusions, and we incorporated technical corrections they suggested. We are sending copies of this report to the Secretary of Education, members of relevant congressional committees, and other interested parties. Copies will also be made available to others on request. Page 10 GAO/HEHS-97-103 Proprietary Schools and Student Aid B-276560 This report was prepared under the direction of Wayne B. Upshaw, Assistant Director. If you or your staff have any questions concerning this report, please call me at (202) 512-7014 or James W. Spaulding, Senior Evaluator, at (202) 512-7035. Tim Silva and Dianne Murphy Blank also contributed to the design and implementation of this study. Sincerely yours, Cornelia M. Blanchette Associate Director, Education and Employment Issues Page 11 GAO/HEHS-97-103 Proprietary Schools and Student Aid Contents Letter 1 Appendix I 14 Scope 14 Objective, Scope, and Data Collection 15 Methodology Data Analysis 17 Limitations Analysis 21 Appendix II 23 Definitions of Low, Medium, and High Reliance 23 Detailed Results of Completion Rates 23 Descriptive, Placement Rates 26 Default Rates 29 Correlation, and Sensitivity Analysis 32 Regression Analyses Related GAO Products 36 Tables Table I.1: Response to GAO’s Survey of Proprietary Schools, by 16 Accrediting Agency Table II.1: Categories of Low, Medium, and High Reliance on Title 23 IV Funds, by Accrediting Agency Table II.2: Average Program Completion Rate at Schools With 24 Low, Medium, and High Reliance on Title IV Funds, by Accrediting Agency Table II.3: Correlation Coefficients Between Completion Rates 24 and Title IV Reliance Table II.4: Regression Results for Completion Rates Using 25 ACCSCT Data Table II.5: Regression Results for Completion Rates Using 26 Limited ACICS and ACCSCT Data Table II.6: Average Placement Rate at Schools With Low, 27 Medium, and High Reliance on Title IV Funds, by Accrediting Agency Table II.7: Correlation Coefficients Between Placement Rates and 27 Title IV Reliance Table II.8: Regression Results for Placement Rates Using 28 ACCSCT Data Table II.9: Regression Results for Placement Rates Using Limited 29 ACICS and ACCSCT Data Page 12 GAO/HEHS-97-103 Proprietary Schools and Student Aid Contents Table II.10: Average Default Rate at Schools With Low, Medium, 30 and High Reliance on Title IV Funds, by Accrediting Agency Table II.11: Correlation Coefficients Between Default Rates and 30 Title IV Reliance Table II.12: Regression Results for Default Rates Using ACCSCT 31 Data Table II.13: Regression Results for Default Rates Using Limited 32 ACICS and ACCSCT Data Figures Figure 1: Schools With High Reliance on Title IV Funds Had 6 Lowest Completion Rates Figure 2: Schools With High Reliance on Title IV Funds Had 7 Lowest Placement Rates Figure 3: Schools With High Reliance on Title IV Funds Had 8 Highest Default Rates Abbreviations ABHES Accrediting Bureau of Health Education Schools ACCET Accrediting Council for Continuing Education & Training ACCSCT Accrediting Commission of Career Schools and Colleges of Technology ACICS Accrediting Council for Independent Colleges and Schools EFC expected family contribution GED general equivalency diploma HEA Higher Education Act of 1965, as amended NACCAS National Accrediting Commission of Cosmetology Arts & Sciences Page 13 GAO/HEHS-97-103 Proprietary Schools and Student Aid Appendix I Objective, Scope, and Methodology Our study was designed to explore the relationship between reliance on title IV funds and school performance in the proprietary school sector. To meet this objective, we performed a variety of statistical analyses on data from a substantial number of the proprietary schools that participated in the Higher Education Act of 1965’s title IV programs during 1994 and 1995. The 85-15 rule requires that proprietary schools obtain at least 15 percent Scope of their revenues from sources outside of title IV funding. The rule applies only to proprietary schools—for-profit institutions that provide postsecondary education and training programs in a wide variety of fields, many for 2 years or less but some for 4 years. Our analysis treated individual proprietary schools as the unit of analysis. We used school data from 1994 and 1995. We obtained our data on proprietary schools from five nationally recognized accrediting agencies: the Accrediting Bureau of Health Education Schools (ABHES); the Accrediting Council for Continuing Education & Training (ACCET); the Accrediting Commission of Career Schools and Colleges of Technology (ACCSCT); the Accrediting Council for Independent Colleges and Schools (ACICS); and the National Accrediting Commission of Cosmetology Arts & Sciences (NACCAS). Together, these five agencies accredit a large majority of all proprietary schools that participate in title IV programs. Each agency requires member schools to submit annual reports that provide information on various aspects of school operations. For example, schools typically report the number of students who (1) matriculated in their programs, (2) completed programs, and (3) were placed in training-related jobs. All the schools in our study met two criteria. First, each school had a title IV institution code number assigned by the Department of Education, signifying the school’s eligibility for title IV programs. Second, each school was a main campus, not a branch campus or additional location.13 Regulations require the 85-15 calculation to be performed at the 13 The terms “branch campus” and “additional location” are often used interchangeably. They refer to school operations that are under the administrative control of a main campus but are located elsewhere. For example, a main campus in Los Angeles might have branch campuses in San Diego and Phoenix. All federal financial aid for students attending branch campuses is administered through the institution’s main campus. Page 14 GAO/HEHS-97-103 Proprietary Schools and Student Aid Appendix I Objective, Scope, and Methodology institutional level, which includes one main campus and all of its branch campuses and additional locations.14 Because Education does not yet require schools to disclose the results of Data Collection 85-15 calculations15 in their certified financial statements, we conducted a confidential mail survey of schools from the five accrediting agencies. Our questionnaire asked school officials to report the results of their institution’s 85-15 calculation for the first fiscal year that ended after June 30, 1995.16 It also asked them to identify all other affiliated campuses—such as branch campuses or additional locations—whose revenue data were included in the institution’s 85-15 calculation. This information enabled us to (1) eliminate from our analyses any schools that performed the 85-15 calculation using revenue data from more than one main campus and (2) make sure we included information on school performance and characteristics from all the additional campuses that the institution included in its 85-15 calculation. Thus, we would not be comparing the results of an 85-15 calculation from a main campus and its branch campuses with student outcome data from the main campus alone.17 We use the term “school” hereafter to refer to a respondent, or a main campus plus any associated branch campuses. The accrediting agencies helped us identify schools for our survey and assisted in following up on survey responses. 14 We used one additional criterion in selecting NACCAS schools for this study. While other accrediting agencies collect student-outcome data from each campus individually, NACCAS collects student-outcome data by program, across all campuses under the same ownership. Thus, if an owner filed one annual report to NACCAS covering two main campuses, both of which offered the same course, it was impossible to determine separately the placement rate for students taking the course at each of the two schools. We included in our study only those main campuses whose annual report contained data for a single main campus. As a result of this necessary step, our analysis of NACCAS data does not include some schools that are part of multicampus chains; that is, schools that share the same name and are owned or operated by the same individual(s) or corporation. Of 997 records in the database NACCAS provided us, we identified 314 cases in which annual reports combined data from two or more main campuses. We cannot determine whether our results would have been different if such schools had been included in our analysis. 15 The 85-15 calculation produces a percentage. The numerator is “Title IV, HEA program funds the institution used to satisfy tuition, fees, and other institutional charges to students.” The denominator is “the sum of revenues generated by the institution from: Tuition, fees, and other institutional charges for students enrolled in eligible programs . . .; and activities conducted by the institution, to the extent not included in tuition, fees, and other institutional charges, that are necessary for the education or training of its students who are enrolled in those eligible programs.” See 34 C.F.R. Sec. 600.5(d)(1). New rules going into effect July 1, 1997, require proprietary institutions to disclose this percentage as a footnote to their financial statement audits. 16 The 85-15 regulation became effective on July 1, 1995. 17 Schools are required to calculate their 85-15 measure by combining main and branch campus revenue data. Page 15 GAO/HEHS-97-103 Proprietary Schools and Student Aid Appendix I Objective, Scope, and Methodology We sent questionnaires to 1,624 schools, with an initial mailing in October 1996 and follow-up mailings in December 1996 and January 1997. Of the 1,624 schools we surveyed, 81 were ineligible for our study, yielding an “adjusted” population of 1,543. We categorized schools as ineligible if (1) they had closed, (2) they were actually nonprofit institutions, or (3) they were not currently participating in title IV programs. We received responses from 1,181 of the 1,543 schools in our adjusted sample, a 77-percent response rate. The response pattern for schools from each accrediting agency is shown in table I.1. Table I.1: Response to GAO’s Survey of Proprietary Schools, by Accrediting Agency ABHES ACCET ACCSCT ACICS NACCAS Total Number of schools surveyed 42 93 503 341 645 1,624 Number of schools determined ineligible 2 3 40 15 21 81 Adjusted size of population 40 90 463 326 624 1,543 Number of questionnaires returned 34 70 358 253 466 1,181 Response rate 85.0% 77.8% 77.3% 77.6% 74.7% 76.5% For each accrediting agency, we compared respondents with nonrespondents using data on school size and student outcomes from the agency’s annual report database. For schools accredited by four of the five agencies, including the three agencies accrediting the largest number of schools, schools that responded were slightly larger, on average, than nonrespondents. Because there were no systematic differences in completion and placement rates, however, we concluded that our respondents did not differ substantially from nonrespondents. Therefore, because we surveyed the population of schools that met our selection criteria in each accrediting agency, we assumed that the information provided by our respondents gives a representative picture of all proprietary schools participating in title IV programs accredited by the five agencies.18 The number of schools accredited by each agency included in most of our statistical analyses, however, was somewhat lower than the number of usable returns listed in table I.1, because some respondents did not answer particular items in the questionnaire or gave nonvalid responses. For example, if respondents indicated they did not know the result of their 85-15 calculation, we excluded them from our main analyses. Similarly, if 18 We are less confident of this conclusion with NACCAS member schools because, as described earlier, in selecting schools for our study, we excluded those who filed a single annual report for more than one main campus. Page 16 GAO/HEHS-97-103 Proprietary Schools and Student Aid Appendix I Objective, Scope, and Methodology school officials indicated they did the 85-15 calculation using revenue data from more than one main campus, we ruled it a nonvalid response.19 Our completion and placement rate calculations for schools varied by Data Analysis accrediting agency because of variations in the data the agencies collected. We performed separate but similar analyses on schools by agency. We used descriptive statistics and correlation analysis to explore the relationship between school performance indicators and reliance on title IV funds for schools from all five agencies. For ACCSCT and ACICS schools, we also used regression analysis. Completion and Placement For schools accredited by ACCSCT, the completion rate was the number of Rate Calculations students that graduated from a program within a specified time divided by the number that started, adjusted for transfers in and out of the school. The completion rate for schools accredited by ACCET and NACCAS was the number completing a program within a specified time divided by the number scheduled to complete in that year. For schools accredited by ABHES and ACICS, the completion rate was the number of students who graduated (or completed) in the program year divided by the number of students that left the school through graduation (or completion), dismissal, or withdrawal. Because neither of the latter two agencies had cohort-based data, and because the schools often had programs lasting longer than 1 year, we could not simply divide the number of graduates by the number of students starting the program that year. The placement rate was some measure of the number of graduating or completing students placed in jobs divided by the number that graduated or completed that year. For schools accredited by ABHES and ACICS, the numerator was the number of students placed in the field of training or a related field; for schools accredited by ACCET, the numerator was the number placed in training-related employment. For schools accredited by ACCSCT, the numerator was the number of graduates who were employed in the field of training. For schools accredited by NACCAS, the numerator was the number who had found jobs. 19 In addition, we excluded schools from our analyses if they reported that their 85-15 calculation included revenue data from a branch campus or additional location that we could identify as not affiliated with the main campus. For NACCAS schools, we also ruled a school’s response invalid if it indicated that the 85-15 calculation did not include revenue data from a branch campus or additional location that was included in its annual report. We could not, however, take these same steps for our analyses using default rates, because we could not identify the branch campuses or additional locations included in an institution’s default rate. While we did exclude invalid 85-15 results, we could not be certain whether valid 85-15 results were based on data from the same set of campuses that contributed to the default rate. Page 17 GAO/HEHS-97-103 Proprietary Schools and Student Aid Appendix I Objective, Scope, and Methodology Descriptive Statistics We initially examined the relationship between title IV reliance and school performance using simple descriptive statistics. Within each accrediting agency, we sorted schools from low to high based on the extent to which they relied on title IV funds as a revenue source. We divided the schools into three roughly equal groups—categorized as low-reliance, medium-reliance, and high-reliance schools—and computed the mean value of the three outcome variables for schools in each category. This approach yielded descriptive statistics for schools with low, medium, and high reliance on title IV. Correlation Analysis We used correlation analysis to determine the direction and strength of association between reliance on title IV and each outcome variable. We examined whether this relationship was in the direction predicted by the theory underlying the 85-15 rule—that is, as reliance on financial aid revenues increases, outcomes worsen. The statistic measuring correlation, the correlation coefficient, may vary between –1 and 1. Direction of association refers to whether the values of two variables tend to move in the same direction (a positive correlation) or in opposite directions (a negative correlation). For example, if higher levels of reliance on title IV funds is generally associated with higher student loan default rates, we would say that the two variables are positively correlated. Strength of association refers to how tightly the scores on one variable are distributed, on average, given particular values on the other variable. When this range is wide, the correlation is weak; when it is narrow, the correlation is strong. The farther the correlation coefficient is from 0 (zero), the stronger the association. Thus, a correlation coefficient for two variables of 0.78 indicates a stronger association than if the same variables had a correlation coefficient of 0.13, and a correlation coefficient of –0.78 is stronger than one of –0.13. However, a correlation coefficient of 0.78 for two variables cannot be compared to one of 0.13 for two other variables. To guard against the possibility that our findings were due to chance, we tested for statistical significance at the 5-percent level, a standard practice in this type of research. Thus, we report a correlation as statistically significant only if the probability of getting that result by chance is less than 5 in 100. We used a one-tailed significance test, because the legislation presumes that high values of the 85-15 measure are associated with unfavorable outcomes, that is, low completion and placement rates and high default rates. Page 18 GAO/HEHS-97-103 Proprietary Schools and Student Aid Appendix I Objective, Scope, and Methodology Finally, it is important to note that correlation does not indicate causality; that is, just because two variables are correlated does not mean that one “causes” the other. When correlation analysis shows two variables are related, a third, unmeasured variable may really explain the observed relationship. In the prior example, the level of poverty among a school’s students might “cause” both reliance on title IV funds and student loan default rates to be high. Regression Analysis Regression analysis is a method for exploring how a dependent variable is affected by a number of independent variables. We performed several regressions to isolate the unique influence of one particular independent variable (extent of reliance on title IV funds) on a series of dependent variables (completion rate, placement rate, and default rate) while holding constant the influence of various other independent variables. As with our correlation analyses, we used tests of statistical significance to determine the likelihood that our regression analysis results were due to chance. We accounted for • the number of students at the school; • the percentages of students who were female; were black; were Hispanic; were under age 25; were age 45 or older; were admitted under the ability-to-benefit provision, that is, with no high school diploma or general equivalency diploma (GED); were admitted with a GED; were admitted with some prior postsecondary education; received Pell grants; received Stafford loans; had an expected family contribution (EFC) of zero, that is, were not required to contribute from their own resources toward the cost of education;20 and attended part time; • the ratio of students to faculty; • the faculty turnover rate; • the number of years—since its founding or 1972, whichever is later—that the school operated before participating in title IV programs; • the number of years the education director and the placement director have held their positions; • the average years of tenure for all instructors; • weighted average program length, in weeks; • weighted average cost of tuition and fees, in thousands of dollars; 20 The EFC is determined by a formula that accounts for family income and assets and is used in awarding financial aid. Page 19 GAO/HEHS-97-103 Proprietary Schools and Student Aid Appendix I Objective, Scope, and Methodology • weighted average starting salary for school graduates, in thousands of dollars;21 • the unemployment rate of the area where the school is located; and • the percentage of gross tuition income spent on new equipment and teaching aids. Our regression model specified a particular relationship between the three outcome variables and the independent variables. Our model was recursive—completion rates (and the full set of independent variables) were modeled to influence placement rates, and completion and placement rates (and the full set of independent variables) were modeled to influence default rates. We believe knowing a school’s completion rate helps predict its placement rate and knowing both completion and placement rates helps predict its default rate. For example, a school with low completion and low placement rates might be expected to have a high default rate, because many of its students would either leave without completing their education or complete but not find a job. Both types of students might be at higher risk than average of defaulting, thus the school’s default rate could be higher than average. We performed our baseline regression analysis on schools accredited by ACCSCT. ACCSCT was the only agency that had data on the requisite independent variables. ACICS had data on some but not all of the independent variables. We also performed regressions on the ACICS data to try to determine whether the results obtained from the ACCSCT data could be replicated with a different data set. We then performed new regressions on the ACCSCT data, using independent variables available for ACICS, and compared the results. In these regressions, we accounted for • the number of students at the school; • the percentages of students who were female; were minority;22 were admitted under the ability-to-benefit provision, that is, with no high school diploma or GED; were admitted with some prior postsecondary schooling; had an EFC of zero; and attended part time;23 and • the ratio of students to faculty. 21 Data on program length, tuition and fees, and starting salary of graduates were provided for each of a school’s programs. We weighted the figure for each program by the number of students in that program to determine an average for the school. 22 For schools accredited by ACCSCT, this variable includes only black and Hispanic students. 23 For schools accredited by ACICS, this variable covered students enrolled in less than a full program, which may be different from students who were part time. Page 20 GAO/HEHS-97-103 Proprietary Schools and Student Aid Appendix I Objective, Scope, and Methodology Detailed results of our analyses appear in appendix II. Our study could not fully assess the impact of the 85-15 rule because of Limitations Analysis certain data limitations. For example, we could not measure qualitative factors involved in schools’ vocational training processes. Accrediting agencies’ data typically pertain to easily measurable inputs, such as student or faculty characteristics, or outcomes, such as completion and placement rates. We could not directly assess the quality of instruction or schools’ equipment, to give just two examples of key aspects of the training process that may influence outcomes like program completion or training-related placement rates. Also, our findings cannot be generalized to all proprietary schools participating in title IV. The schools that we included in our study, though they make up a large proportion of title-IV-eligible proprietary schools, are not necessarily representative of all such schools in the nation. In addition, as noted previously, not all schools that responded to our survey knew the value of their 85-15 measure or computed it correctly. We did not verify schools’ computations. Finally, variables in our analyses came from different time periods. Our measure of school reliance on title IV funds—the 85-15 measure—pertains to each school’s first fiscal year ending after June 30, 1995, which for many schools covered the period of January 1, 1995, to December 31, 1995. Thus, our key independent variable typically represents a time period slightly later than, though usually overlapping with, the period that was the basis for most of our dependent and other independent variables, which came from accrediting agency annual report data and whose time periods differed by agency. At the time of our study, the most recently available student loan default data were for 1994, reflecting the percentage of loans in default among each school’s borrowers who entered repayment in fiscal year 1994. Such students would have attended school at least 1 year prior to the time period for which annual report data were collected and the fiscal year for which officials did the 85-15 calculation. These students’ experiences at a given school thus do not necessarily represent the experiences of students who were enrolled during the time period for which accrediting agencies collected annual report data. However, we do not believe the mismatching time periods raise significant questions about the results of our analyses using default rates as the Page 21 GAO/HEHS-97-103 Proprietary Schools and Student Aid Appendix I Objective, Scope, and Methodology dependent variable. A researcher who previously analyzed the relationship between default rates and various school and student characteristics among ACCSCT schools reported that using default rates and annual report data for matching time periods yielded results “virtually identical to those obtained with the time-lagged data.”24 24 Morgan V. Lewis, “Analysis of Annual Report Data for School Years 1990 to 1993,” study prepared for ACCSCT, Center on Education and Training for Employment (Columbus, Oh.: The Ohio State University, Nov. 1994), p. 25. Page 22 GAO/HEHS-97-103 Proprietary Schools and Student Aid Appendix II Detailed Results of Descriptive, Correlation, and Regression Analyses This appendix presents technical detail and results of our analyses of the relationship between reliance on title IV funds and school performance. It includes sample sizes, standard deviations or standard errors, and significance levels for many of our results, as well as sensitivity tests for some of the assumptions we made in conducting our analyses. We ranked the schools accredited by each agency by their 85-15 measure Definitions of Low, and grouped them into three categories, which we refer to as low-reliance, Medium, and High medium-reliance, and high-reliance schools. For each agency, each Reliance category contained roughly one-third of the schools. Table II.1 shows the break points for each agency and the number of schools falling into each category. Table II.1: Categories of Low, Medium, and High Reliance on Title IV Funds, by Accrediting Agency Category of 85-15 measure ABHES ACCET ACCSCT ACICS NACCAS Low Range of measure 23%-65% 4%-58% 1%-59% 12%-64% 2%-40% Number of schools in category 10 18 110 73 138 Medium Range of measure 67%-77% 61%-76% 60%-75% 65%-77% 41%-61% Number of schools in category 10 19 107 85 138 High Range of measure 78%-85% 77%-84% 76%-85% 78%-85% 62%-85% Number of schools in category 10 17 114 71 135 Some of the analyses, however, used fewer schools than shown in table II.1 because some schools had missing data for a particular outcome. Schools with high reliance on title IV, on average, had lower completion Completion Rates rates than schools with low or medium reliance. The differences between the high one-third and low one-third of schools ranged from 12 to 18 percentage points for schools accredited by four of the five agencies. Schools from the fifth agency, NACCAS, showed virtually no difference in completion rates across the three categories. Table II.2 shows means and standard deviations, as well as sample sizes, for completion rates for schools in low, medium, and high title IV reliance categories. Page 23 GAO/HEHS-97-103 Proprietary Schools and Student Aid Appendix II Detailed Results of Descriptive, Correlation, and Regression Analyses Table II.2: Average Program Completion Rate at Schools With Low, Medium, and High Reliance on Title IV Funds, by Accrediting Agency Numbers in percent ABHES ACCET ACCSCT ACICS NACCAS (30 schools) (54 schools) (262 schools) (229 schools) (411 schools) Standard Standard Standard Standard Standard 85-15 category Mean deviation Mean deviation Mean deviation Mean deviation Mean deviation Low 60 24 75 11 75 12 58 20 68 14 Medium 50 16 69 24 66 16 50 17 69 14 High 43 19 57 17 61 15 46 18 65 14 The correlation coefficients between completion rates and reliance on title IV were negative for schools from all five agencies. The coefficients were significantly different from zero25 for four of the five. Table II.3 shows correlation coefficients, standard errors, and sample sizes for these analyses. Table II.3: Correlation Coefficients Between Completion Rates and Title IV ABHES ACCET ACCSCT ACICS NACCAS Reliance Correlation coefficient –0.36a –0.29a –0.41a –0.23a –0.07 P-value 0.03 0.02 0.00 0.00 0.07 Number of cases 30 54 262 229 411 a Significant at 5-percent level. Regression analysis on schools accredited by ACCSCT confirmed the statistically significant negative relationship between completion rates and title IV reliance (see table II.4). Even accounting for other factors, the 85-15 measure—our measure of title IV reliance—was statistically significant. The coefficient indicated that for each 10-percentage-point increase in title IV reliance, completion rates were 2.7 percentage points lower. The regression showed that five other factors were statistically significant: the number of students at the school, the percentage of students who received Pell grants, the faculty turnover rate, the average length of the school’s program, and the average starting salary of a school’s graduates. In addition, the constant term, which we included in each regression rather than forcing the regression line’s intercept to equal zero, was significant. 25 We used one-tailed significance tests for our correlation results throughout this report because the 85-15 rule presumes that high values of the 85-15 variable are associated with bad outcomes, that is, low completion and placement rates and high default rates. We conducted significance tests based on this presumption. Page 24 GAO/HEHS-97-103 Proprietary Schools and Student Aid Appendix II Detailed Results of Descriptive, Correlation, and Regression Analyses Table II.4: Regression Results for Completion Rates Using ACCSCT Data Variable Coefficient Standard error 85-15 measure –0.2747a 0.0698 a Number of students –0.01468 0.00438 Percentages of students who Were female 0.01547 0.0341 Were black –0.08514 0.0572 Were Hispanic 0.01018 0.0721 Were under age 25 0.00004117 0.0558 Were age 45 or older 0.06918 0.241 Did not have a high school diploma or GED –0.03939 0.118 Had a GED 0.2112 0.144 Had some prior postsecondary education 0.006901 0.0537 Received Pell grants –0.1508a 0.0568 Received Stafford loans 0.04981 0.0466 Had an expected family contribution of zero 0.03271 0.0501 Attended part time –0.1184 0.0735 Student-faculty ratio 0.1244 0.115 Faculty turnover rate –0.1321a 0.0590 Years school operated before participating in title IV –0.08259 0.229 Years of experience of education director 0.02447 0.138 Years of experience of placement director 0.02926 0.173 Average years of tenure of all instructors –0.5154 0.342 Average program length –0.2390a 0.0680 Average tuition and fees 0.5622 0.301 Average starting salary of graduates 0.1709a 0.0532 Unemployment rate in school’s local area 0.3690 0.283 Percentage of revenues spent on new equipment 0.2633 0.194 Constant 93.26a 6.71 Note: Sample size was 187. a Significant at 5-percent level. We also performed regressions of completion rates on the 85-15 measure and a limited set of independent variables for schools from ACICS. The results were similar—the coefficient on the 85-15 measure was negative and significant. When we replicated this regression using the ACCSCT data—that is, regressed completion rates on the same set of independent Page 25 GAO/HEHS-97-103 Proprietary Schools and Student Aid Appendix II Detailed Results of Descriptive, Correlation, and Regression Analyses variables in ACCSCT data that we used for ACICS—the results were again consistent. Table II.5 shows the results for both regressions. Table II.5: Regression Results for Completion Rates Using Limited ACICS and ACCSCT Data ACCSCT data ACICS data Variable Coefficient Standard error Coefficient Standard error 85-15 measure –0.3361a 0.0571 –0.3055a 0.0951 a a Number of students –0.01743 0.00396 –0.004033 0.00200 Percentages of students who Were female –0.005707 0.0338 0.02571 0.0734 Were minority –0.06444 5.05 –0.01911 0.0640 Did not have a high school diploma or GED 0.07422 0.115 –0.1573 0.224 Had some prior postsecondary education 0.04783 0.0511 –0.1479a 0.0733 Had an expected family contribution of zero 0.01855 0.0492 0.06821 0.0768 Attended part time –0.09079 0.0704 –0.3562 0.187 Student-faculty ratio 0.1328 0.117 0.1110 0.142 Constant 90.93a 3.86 75.53a 8.98 Note: Sample sizes were 195 for ACCSCT and 160 for ACICS. a Significant at 5-percent level. Schools with high reliance on title IV had slightly lower placement rates Placement Rates than schools with low or medium reliance, but the differences were much smaller than for completion rates. The differences between the high one-third and low one-third of schools were only 3 to 8 percentage points for schools from four of the five agencies. Schools from the other agency, ACICS, showed no difference in placement rates. Table II.6 shows means and standard deviations, as well as sample sizes, for placement rates for schools in low, medium, and high title IV reliance categories. Page 26 GAO/HEHS-97-103 Proprietary Schools and Student Aid Appendix II Detailed Results of Descriptive, Correlation, and Regression Analyses Table II.6: Average Placement Rate at Schools With Low, Medium, and High Reliance on Title IV Funds, by Accrediting Agency Numbers in percent ABHES ACCET ACCSCT ACICS NACCAS (29 schools) (54 schools) (262 schools) (229 schools) (411 schools) Standard Standard Standard Standard Standard 85-15 category Mean deviation Mean deviation Mean deviation Mean deviation Mean deviation Low 77 9 74 16 79 15 71 15 84 15 Medium 75 18 68 18 75 13 71 12 87 13 High 74 8 66 14 74 13 71 13 79 17 As with the descriptive statistics, the correlation analysis showed a weaker relationship between title IV reliance and placement rates than it did for completion rates. Only three of the five correlation coefficients were significant and negative; the other two were insignificant. Table II.7 details the results. Table II.7: Correlation Coefficients Between Placement Rates and Title IV ABHES ACCET ACCSCT ACICS NACCAS Reliance Correlation coefficient –0.01 –0.26a –0.14a 0.01 –0.13a P-value 0.49 0.03 0.01 0.43 0.00 Number of cases 29 54 262 229 411 a Significant at 5-percent level. Regression analysis showed that the relationship between placement rates and title IV reliance was not statistically significant when accounting for other factors that could affect placement rates (see table II.8). The only factors that were significant besides the constant term were the number of students, the student-faculty ratio, and the unemployment rate in the school’s local area. Page 27 GAO/HEHS-97-103 Proprietary Schools and Student Aid Appendix II Detailed Results of Descriptive, Correlation, and Regression Analyses Table II.8: Regression Results for Placement Rates Using ACCSCT Data Variable Coefficient Standard error 85-15 measure –0.02117 0.0753 Completion rate 0.01967 0.0812 Number of students –0.01173a 0.00467 Percentages of students who Were female 0.01573 0.0351 Were black 0.02753 0.0594 Were Hispanic –0.02689 0.0743 Were under age 25 –0.005373 0.0575 Were age 45 or older 0.4787 0.248 Did not have a high school diploma or GED –0.07588 0.122 Had a GED 0.2700 0.150 Had some prior postsecondary education –0.04097 0.0554 Received Pell grants 0.04122 0.0598 Received Stafford loans 0.05588 0.0482 Had an expected family contribution of zero –0.02750 0.0517 Attended part time –0.06594 0.0764 Student-faculty ratio 0.3472a 0.119 Faculty turnover rate –0.06935 0.0617 Years school operated before participating in title IV 0.2769 0.236 Years of experience of education director 0.02647 0.143 Years of experience of placement director 0.2177 0.178 Average years of tenure of all instructors 0.1943 0.355 Average program length 0.03623 0.0727 Average tuition and fees –0.07255 0.313 Average starting salary of graduates 0.001333 0.0565 Unemployment rate in school’s local area –0.9791a 0.293 Percentage of revenues spent on new equipment 0.2916 0.201 a Constant 67.01 10.3 Note: Sample size was 187. a Significant at 5-percent level. Placement rate regressions using the more limited set of independent variables from ACICS also showed that the coefficient on the 85-15 measure was not significant. Furthermore, regressions on the same set of variables Page 28 GAO/HEHS-97-103 Proprietary Schools and Student Aid Appendix II Detailed Results of Descriptive, Correlation, and Regression Analyses for ACCSCT confirmed that reliance on title IV did not significantly affect placement rates (see table II.9). Table II.9: Regression Results for Placement Rates Using Limited ACICS and ACCSCT Data ACCSCT data ACICS data Variable Coefficient Standard error Coefficient Standard error 85-15 measure –0.07563 0.0612 0.09922 0.0722 Completion rate 0.005950 0.0724 0.04107 0.0599 Number of students –0.01515a 0.00410 –0.001468 0.00149 Percentages of students who Were female 0.01745 0.0333 0.1007 0.0539 Were minority –0.05108 0.0499 –0.08013 0.0470 Did not have a high school diploma or GED –0.1292 0.113 –0.1803 0.165 Had some prior postsecondary education –0.06564 0.0504 –0.0853 0.0546 Had an expected family contribution of zero –0.02387 0.0485 –0.04739 0.0565 Attended part time –0.009300 0.0696 - 0.09208 0.139 Student-faculty ratio 0.4260a 0.116 0.04753 0.104 a a Constant 81.43 7.60 61.20 8.00 Notes: Sample sizes were 195 for ACCSCT and 160 for ACICS. a Significant at 5-percent level. Schools with high reliance on title IV had higher default rates than schools Default Rates with low or medium reliance for three of the five agencies. The differences between the high one-third and low one-third of schools were only 6 to 7 percentage points for these agencies, but these differences are large relative to the values of the default rates. For example, high-reliance schools from NACCAS had default rates of 22 percent, about half again as high as the 15-percent rate for low-reliance schools. Table II.10 shows means and standard deviations, as well as sample sizes, for default rates for schools in low, medium, and high title IV reliance categories. Page 29 GAO/HEHS-97-103 Proprietary Schools and Student Aid Appendix II Detailed Results of Descriptive, Correlation, and Regression Analyses Table II.10: Average Default Rate at Schools With Low, Medium, and High Reliance on Title IV Funds, by Accrediting Agency Numbers in percent ABHES ACCET ACCSCT ACICS NACCAS (25 schools) (43 schools) (230 schools) (203 schools) (352 schools) Standard Standard Standard Standard Standard 85-15 category Mean deviation Mean deviation Mean deviation Mean deviaiton Mean deviation Low 13 10 18 13 15 11 14 9 15 12 Medium 19 7 15 10 18 10 16 8 20 17 High 16 6 16 12 22 13 20 9 22 18 The correlation between default rates and reliance on title IV was positive for four agencies; for three of these agencies it was statistically significant (see table II.11). Table II.11: Correlation Coefficients Between Default Rates and Title IV ABHES ACCET ACCSCT ACICS NACCAS Reliance Correlation coefficient 0.07 –0.19 0.21 a 0.18 a 0.19a P-value 0.37 0.11 0.00 0.01 0.00 Number of cases 25 43 230 203 352 a Significant at 5-percent level. Our regression analysis confirmed that schools with high reliance on title IV had high default rates. The coefficient on the 85-15 measure was positive and significant; it indicated that a 10-percentage-point increase in reliance on title IV was associated with a 1.1-percentage-point increase in the default rate. Besides the 85-15 measure, other factors associated with higher default rates include the percentage of students who were black or age 45 or older, and a high student-faculty ratio. Three factors negatively affected default rates: a high placement rate and a high percentage of students who were women or received Stafford loans. Page 30 GAO/HEHS-97-103 Proprietary Schools and Student Aid Appendix II Detailed Results of Descriptive, Correlation, and Regression Analyses Table II.12: Regression Results for Default Rates Using ACCSCT Data Variable Coefficient Standard error 85-15 measure 0.1088a 0.0539 Completion rate 0.03715 0.0581 Placement rate –0.1296a 0.0565 Number of students –.0005475 0.00341 Percentages of students who Were female –0.06055a 0.0252 Were black 0.2216a 0.0425 Were Hispanic 0.02875 0.0532 Were under age 25 –0.02427 0.0411 a Were age 45 or older 0.3665 0.179 Did not have a high school diploma or GED 0.02615 0.0870 Had a GED 0.01321 0.108 Had some prior postsecondary education –0.01811 0.0397 Received Pell grants 0.06405 0.0428 Received Stafford loans –0.09434a 0.0346 Had an expected family contribution of zero 0.07023 0.0370 Attended part time 0.004140 0.0547 a Student-faculty ratio 0.2186 0.0875 Faculty turnover rate 0.02639 0.0443 Years school operated before participating in title IV –0.2848 0.170 Years of experience of education director –0.01353 0.102 Years of experience of placement director –0.1146 0.128 Average years of tenure of all instructors –0.2236 0.254 Average program length 0.01765 0.0520 Average tuition and fees –0.08231 0.224 Average starting salary of graduates 0.003740 0.0404 Unemployment rate in school’s local area –0.2229 0.217 Percentage of revenues spent on new equipment 0.2410 0.145 Constant 14.96 8.26 Note: Sample size was 187. a Significant at 5-percent level. Default rate regressions using the more limited set of independent variables from ACICS showed the only result inconsistent with our baseline analyses. In the limited default rate regressions, on both ACCSCT and ACICS Page 31 GAO/HEHS-97-103 Proprietary Schools and Student Aid Appendix II Detailed Results of Descriptive, Correlation, and Regression Analyses data, the coefficient on the 85-15 measure was not significant (see table II.13). Table II.13: Regression Results for Default Rates Using Limited ACICS and ACCSCT Data ACCSCT data ACICS data Variable Coefficient Standard error Coefficient Standard error 85-15 measure 0.07628 0.0436 0.05851 0.0412 Completion rate –0.001833 0.0513 –0.01933 0.0340 Placement rate –0.1023 0.0523 –0.03611 0.0464 Number of students –0.001682 0.00301 0.0007948 0.000845 Percentages of students who Were female –0.05948a 0.0237 –0.04922 0.0309 Were minority 0.1532a 0.0355 0.02449 0.0269 a Did not have a high school diploma or GED 0.1386 0.0808 0.3696 0.0939 Had some prior postsecondary education 0.002262 0.0359 0.008830 0.0312 a a Had an expected family contribution of zero 0.08619 0.0344 0.07270 0.0321 Attended part time 0.002495 0.0494 0.1203 0.0790 a Student-faculty ratio 0.2075 0.0850 0.09847 0.0592 Constant 12.41 6.87 12.31a 5.35 Note: Sample sizes were 195 for ACCSCT and 160 for ACICS. a Significant at 5-percent level. In any quantitative analysis of this kind, the results may be sensitive to the Sensitivity Analysis definition and measurement of the variables used. If there is any uncertainty about how well the variables capture the concept they are intended to represent, or about the accuracy of the data, it is important to test to what extent the results are sensitive to those factors. For example, variables we used could have been defined and measured in more than one way. Therefore, where possible, we conducted analyses to explore whether or how much our results were sensitive to methodological decisions we made. We tested sensitivity to three factors: • the definition of placement rates for each agency, • the time frames within which our data were defined, and • the types of programs included for each school. Page 32 GAO/HEHS-97-103 Proprietary Schools and Student Aid Appendix II Detailed Results of Descriptive, Correlation, and Regression Analyses Definitions of Placement Placement rate definitions varied by agency. Our general definition was Rates the number of graduates placed in their field of training, or a related field, divided by the number of graduates. For schools accredited by ABHES and ACICS, we knew both the number of graduates placed in the field of training and the number of graduates placed in a related field. For schools accredited by ACCET and ACCSCT, we knew the number of graduates who went on for further education or were otherwise unavailable for employment; furthermore, for ACCSCT schools, we knew the number employed in the field of training who had not actually graduated. We tested variations on the placement rate definition for these agencies. We computed a new placement rate for ABHES and ACICS schools by deleting those placed in a related field from the numerator, yielding a lower placement rate. We computed a new placement rate for ACCET schools, excluding students unavailable for employment from the denominator, yielding a higher rate. For ACCSCT schools, we computed two new measures, one excluding those unavailable for placement from the denominator and the other including those employed in their field, but who did not graduate, in the numerator, both yielding higher rates. The results of the correlation analyses between these new measures and the 85-15 measure were similar to those for our baseline analyses. For each agency with an insignificant correlation coefficient in our baseline analyses, the new coefficient remained insignificant. For each agency with a significant correlation coefficient, the new coefficient remained significant, with one exception: for schools accredited by ACCET, the correlation coefficient became insignificant when students ineligible for placement were excluded. Time Frames for Data We performed sensitivity analyses to explore the implications of using Definitions data from differing time periods. To carry this out, we analyzed only the subset of schools with 6 or more months of overlap between the time periods for their annual report and their 85-15 calculation, for four of the five agencies,26 and compared the results to the analysis for all schools. Our sample sizes decreased somewhat because, for some agencies, many schools had less than a 6-month overlap. However, the correlations that were significant in our baseline analyses were always of the same sign, and nearly always significant, in the sensitivity analyses. 26 Virtually none of the schools accredited by NACCAS had more than a 6-month overlap. Page 33 GAO/HEHS-97-103 Proprietary Schools and Student Aid Appendix II Detailed Results of Descriptive, Correlation, and Regression Analyses Types of Programs Schools calculate the 85-15 measure by incorporating only title-IV-eligible Included for Each School programs. Students in programs that are shorter than 300 clock hours cannot receive title IV aid for those programs. Ideally, our data would always cover title-IV-eligible programs only, to match the coverage of the 85-15 rule. However, three of the accrediting agencies—ABHES, ACCSCT, and ACICS—provided data on schools with either all data aggregated up to the school level or program-level data that did not include the number of hours per program for all relevant variables. Some of the programs at those schools might have been shorter than 300 clock hours; thus, students in those programs would not be eligible for title IV aid. However, we could not exclude students in those short programs from our analysis because we could not separate them from the rest of the programs the schools offered. For schools from agencies that provided data at the program level, including length of program—ACCET and NACCAS—we performed two sets of analyses. Our baseline analysis, the results of which we discuss throughout this report, excluded programs shorter than 300 clock hours. We tested sensitivity of the analysis to this exclusion, that is, we performed all our analyses anew for these two agencies by including all programs each school offered. When we compared the results for eligible programs only with results for all programs, for schools accredited by ACCET and NACCAS, we found the results did not change substantially. We thus feel confident that our results for schools accredited by ABHES, ACCSCT, and ACICS would not change materially if we had the data to exclude ineligible programs. Page 34 GAO/HEHS-97-103 Proprietary Schools and Student Aid Page 35 GAO/HEHS-97-103 Proprietary Schools and Student Aid Related GAO Products High-Risk Series: Student Financial Aid (GAO/HR-97-11, Feb. 1997). Department of Education: Status of Actions to Improve the Management of Student Financial Aid (GAO/HEHS-96-143, July 12, 1996). Higher Education: Ensuring Quality Education From Proprietary Institutions (GAO/T-HEHS-96-158, June 6, 1996). Defaulted Student Loans: Analysis of Defaulted Borrowers at Schools Accredited by Seven Agencies (GAO/HRD-90-178FS, Sept. 12, 1990). (104851) Page 36 GAO/HEHS-97-103 Proprietary Schools and Student Aid Ordering Information The first copy of each GAO report and testimony is free. Additional copies are $2 each. Orders should be sent to the following address, accompanied by a check or money order made out to the Superintendent of Documents, when necessary. VISA and MasterCard credit cards are accepted, also. Orders for 100 or more copies to be mailed to a single address are discounted 25 percent. Orders by mail: U.S. General Accounting Office P.O. Box 6015 Gaithersburg, MD 20884-6015 or visit: Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders may also be placed by calling (202) 512-6000 or by using fax number (301) 258-4066, or TDD (301) 413-0006. Each day, GAO issues a list of newly available reports and testimony. To receive facsimile copies of the daily list or any list from the past 30 days, please call (202) 512-6000 using a touchtone phone. A recorded menu will provide information on how to obtain these lists. For information on how to access GAO reports on the INTERNET, send an e-mail message with "info" in the body to: email@example.com or visit GAO’s World Wide Web Home Page at: http://www.gao.gov PRINTED ON RECYCLED PAPER United States Bulk Rate General Accounting Office Postage & Fees Paid Washington, D.C. 20548-0001 GAO Permit No. G100 Official Business Penalty for Private Use $300 Address Correction Requested
Proprietary Schools: Poorer Student Outcomes at Schools That Rely More on Federal Student Aid
Published by the Government Accountability Office on 1997-06-13.
Below is a raw (and likely hideous) rendition of the original report. (PDF)