oversight

Military Education: DOD Needs to Enhance Performance Goals and Measures to Improve Oversight of Military Academies

Published by the Government Accountability Office on 2003-09-10.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                 United States General Accounting Office

GAO              Report to the Subcommittee on
                 Defense, Committee on Appropriations,
                 House of Representatives


September 2003
                 MILITARY
                 EDUCATION
                 DOD Needs to
                 Enhance Performance
                 Goals and Measures to
                 Improve Oversight of
                 Military Academies




GAO-03-1000
                                                September 2003


                                                MILITARY EDUCATION

                                                DOD Needs to Enhance Performance
Highlights of GAO-03-1000, a report to the      Goals and Measures to Improve
Subcommittee on Defense, Committee on
Appropriations, House of Representatives        Oversight of Military Academies



Graduates of the service academies              The Office of the Under Secretary of Defense for Personnel and Readiness
operated by the Army, Navy, and                 (OUSD/P&R), the services, and the academies’ boards of visitors conduct
Air Force currently make up                     considerable oversight of the academies’ operations and performance, but
approximately 18 percent of the                 they lack a complete oversight framework. A complete oversight framework
officer corps for the nation’s armed            includes performance goals and measures against which the academies’
services. The academies represent
the military’s most expensive
                                                performance could be better assessed. OUSD/P&R and the services use the
source of new officers. The                     number and type of commissioned officers as the primary measure of
Department of Defense (DOD) pays                academy performance. OUSD/P&R requires and receives reports on
the full cost of a student’s 4-year             academy performance from the services. While data submitted in these
education at the academies; and                 reports provide perspective on current performance compared with past
the related cost has increased over             performance, without stated performance goals and measures, these reports
the past 4 years. Admission to the              do not offer OUSD/P&R or the services as good an insight into the academies
academies is highly competitive.                performance as they could. Additionally, though the academy boards of
The academies use a “whole                      visitors serve as an external oversight mechanism to focus attention on a
person” method to make admission                wide range of issues, they also do not assess the academies’ performance
decisions. Recent studies by the                against established performance goals and measures.
Air Force raised questions about
possible adverse effects of whole
person admissions policies on                   The academies do not grant waivers from academic criteria or have absolute
student quality. GAO was asked to               minimum scores for admission. However, under the whole person approach,
review all three service academies              the academies can admit some applicants whose academic scores are lower
and specifically address the extent             than might normally be competitive for admission, but who in their totality
to which (1) DOD oversees the                   (academics, physical aptitude, and leadership) are evaluated by academy
service academies, (2) applicants               officials as being capable of succeeding at the academy.
are granted waivers of academic
standards, and (3) various groups               In our review of the academy classes that started in 1998 (class of 2002), we
of students differ in admissions                found that despite differences among various groups of students in their
scores and academy performance.                 admissions scores and similar differences in their performance while at the
                                                academies, the differences in performance were not sizable. Some groups,
                                                such as females, performed better in some categories than the class as a
GAO recommends that the                         whole and worse in others. Some groups (minorities, preparatory school
Secretary of Defense direct the                 graduates, recruited athletes, and students in the lower 30 percent of their
Office of the Under Secretary of                class in terms of academic admissions scores) performed at lower levels on
Defense for Personnel and                       average in all categories than the class as a whole.
Readiness (OUSD/P&R), in concert
with the services, to further
                                                Academy Operating Costs and Cost Per Graduate, Fiscal Years 1999-2002
enhance performance goals and
measures to improve oversight of                 Academy       Cost category           FY 1999       FY 2000        FY 2001       FY 2002
                                                               Total operating
the operations and performance of                Military      costs               $301,058,452 $330,603,820    $336,416,716 $364,971,975
the service academies. In                        Academy
                                                               Cost per graduate       312,150       320,120        339,318       349,327
comments on a draft of this report,                            Total operating
DOD concurred with GAO’s                         Naval         costs                245,749,679   253,817,467    273,809,865   292,696,358
recommendation.                                  Academy
                                                               Cost per graduate       254,983       256,931        266,033       275,001
                                                               Total operating
www.gao.gov/cgi-bin/getrpt?GAO-03-1000.          Air Force     costs                277,639,005   314,972,559    321,335,152   333,056,023
                                                 Academy
To view the full product, including the scope                  Cost per graduate       305,945       305,133        313,456       322,750
and methodology, click on the link above.       Source: DOD.
For more information, contact Derek Stewart
at (202) 512-5559 or stewartd@gao.gov.
Contents


Letter                                                                                            1
                       Results in Brief                                                          3
                       Background                                                                6
                       DOD Lacks a Complete Framework for Oversight of the Academies            15
                       Whole Person Approach Allows Academies Flexibility to Admit
                         Students with a Range of Qualifications                                19
                       No Significant Differences in Admissions and Academy
                         Performance between Various Groups of Students                         21
                       Conclusion                                                               26
                       Recommendation for Executive Action                                      26
                       Agency Comments                                                          26

Appendix I             Scope and Methodology                                                    28



Appendix II            Results of Statistical Analysis of Class of 2002
                       Admissions and Academy Performance Scores                                31
                       Admissions and Performance Scores                                        31
                       Relationships between Admissions and Performance Scores                  34

Appendix III           Comments from the Department of Defense                                  37



Appendix IV            GAO Contact and Staff Acknowledgments                                    39



Related GAO Products                                                                            40



Tables
                       Table 1: Civilian and Military Faculty at the Service Academies            7
                       Table 2: Academy Operating Costs and Cost Per Graduate, Fiscal
                                Years 1999-2002.                                                  8
                       Table 3: Average Admissions Scores for the Selected Groups in the
                                Class That Started in 1998 at the U.S. Military Academy         22
                       Table 4: Average Admissions Scores for the Selected Groups in the
                                Class That Started in 1998 at the U.S. Naval Academy            22




                       Page i                                        GAO-03-1000 Military Education
          Table 5: Average Admissions Scores for the Selected Groups in the
                   Class That Started in 1998 at the U.S. Air Force Academy        22
          Table 6: Percentage of the Selected Groups Making Up the Lower
                   30 percent of the Classes in Terms of Their Academic
                   Admissions Scores, by Academy                                   23
          Table 7: Student Performance for Selected Groups at the U.S.
                   Military Academy                                                24
          Table 8: Student Performance for Selected Groups at the U.S.
                   Naval Academy                                                   24
          Table 9: Student Performance for Selected Groups at the U.S. Air
                   Force Academy                                                   25
          Table 10: Admissions and Academy Performance Scores for the
                   Class of 2002                                                   31
          Table 11: Number of Students Graduating and Graduation Rates for
                   the Class of 2002                                               31
          Table 12: Admissions and Performance Scores for the Class of 2002
                   at the U.S. Military Academy                                    32
          Table 13: Admissions and Performance Scores for the Class of 2002
                   at the U.S. Naval Academy                                       33
          Table 14: Admissions and Performance Scores for the Class of 2002
                   at the U.S. Air Force Academy                                   33
          Table 15: Regression Coefficients (Standardized Coefficients) from
                   Linear Regression Models Testing Correlations between
                   Academic and Whole Person Admissions Scores with
                   Cumulative GPA, Cumulative MPA, and Order of Merit for
                   the Class of 2002 at the Service Academies                      35
          Table 16: Regression Coefficients (Standardized Coefficients) from
                   Logistic Regression Models Testing Correlations Between
                   Academic and Whole Person Admissions Scores and the
                   Likelihood of Graduation for the Class of 2002 at the
                   Service Academies                                               36


Figures
          Figure 1: Basic Steps in the Academy Admissions Process                    9
          Figure 2: Areas and Their Weights Considered in the U.S. Military
                   Academy’s Whole Person Admissions Process                       11
          Figure 3: Areas and Their Weights Considered in the U.S. Naval
                   Academy’s Whole Person Admissions Process                       12
          Figure 4: Areas and Their Weights Considered in the U.S. Air Force
                   Academy’s Whole Person Admissions Process                       13
          Figure 5: Categories of Academy Nominations                              14


          Page ii                                       GAO-03-1000 Military Education
Abbreviations

ACT               American College Testing
DOD               Department of Defense
GPA               grade point average
MPA               military performance average
OUSD/P&R          Office of the Under Secretary of Defense for Personnel
                   and Readiness
ROTC              Reserve Officer Training Corps
SAT               Scholastic Aptitude Test




This is a work of the U.S. government and is not subject to copyright protection in the
United States. It may be reproduced and distributed in its entirety without further
permission from GAO. However, because this work may contain copyrighted images or
other material, permission from the copyright holder may be necessary if you wish to
reproduce this material separately.




Page iii                                                GAO-03-1000 Military Education
United States General Accounting Office
Washington, DC 20548




                                   September 10, 2003

                                   The Honorable Jerry Lewis
                                   Chairman
                                   The Honorable John P. Murtha
                                   Ranking Minority Member
                                   Subcommittee on Defense
                                   Committee on Appropriations
                                   House of Representatives

                                   Graduates of the service academies operated by the Army, Navy, and Air
                                   Force make up approximately 18 percent of the officer corps for the
                                   nation’s armed services.1 The academies represent the most expensive
                                   source of new officers, compared with other sources for officers, such as
                                   Reserve Officer Training Corps (ROTC) programs at colleges and
                                   universities or officer candidate/training schools for individuals who
                                   already have college degrees. The Department of Defense (DOD) pays the
                                   full cost of providing the 4-year programs of academic education, military
                                   training, physical conditioning, and pay for each student.2 In fiscal year
                                   2002, DOD reported costs per graduate for the U.S. Military Academy, the
                                   U.S. Naval Academy, and the U.S. Air Force Academy were approximately
                                   $349,000, $275,000, and $333,000, respectively. These costs have increased
                                   over the past 4 years. To ensure the best value for the investment in the
                                   academies, effective management principles are critical. Such principles
                                   include a complete oversight framework, with clear roles and
                                   responsibilities, as well as performance goals and measures against which
                                   to objectively assess performance.

                                   With each academy accepting about 1,200 of its more than 10,000
                                   applicants a year, admission to the academies is highly competitive.
                                   Applicants must be selected or obtain a nomination, such as from a
                                   senator, representative, the President, or the Vice President, based on the



                                   1
                                    The Marine Corps does not have its own academy. The Naval Academy graduates both
                                   Navy and Marine Corps officers.
                                   2
                                    Students attending the U.S. Military Academy at West Point, New York, and the U.S. Air
                                   Force Academy in Colorado Springs, Colorado, are called “cadets,” while those attending
                                   the U.S. Naval Academy, in Annapolis, Maryland, are called “midshipmen.” We refer to
                                   cadets and midshipmen collectively as “students.”



                                   Page 1                                                  GAO-03-1000 Military Education
categories established by law.3 Most nominations are reserved for
Congress, which, therefore, has a central role in admitting students to the
academies. In addition to basic age and medical qualifications, the
academies’ admissions process involves an assessment of applicants’
academic achievement (e.g., Scholastic Aptitude Test—SAT—scores and
grade point averages), physical aptitude, and extracurricular activities
(i.e., leadership potential). Academy officials combine these assessments
into a “whole person” admissions score that is used to determine an
applicant’s potential to graduate from an academy and potential fitness as
a commissioned officer. Applicants compete for admission based on these
scores.

Air Force studies have raised questions about possible adverse effects of
whole person admissions policies on student quality. For example, the Air
Force found that its whole person assessments and resulting admissions
scores have led the Air Force Academy to admit an increasing number of
students whose academic qualifications are below academic minimums, as
well as to admit an increasing number of students recruited largely to
participate in varsity intercollegiate athletics.

The House of Representatives report on defense appropriations for fiscal
year 20034 directed that we perform reviews of all three service academies
and their respective preparatory schools.5 As part of the review of the
service academies, we were also directed to obtain student and faculty
perceptions of various aspects of student life at the academies. Issues
associated with the academy preparatory schools and the results of
surveys on aspects of student life are addressed in separate reports.6




3
    10 U.S.C. §§ 4342, 6954, and 9342.
4
    H.R. Rept. 107-532, at 14-15 (2002).
5
 The academy preparatory schools exist to prepare selected students who are not ready
academically to attend one of the academies.
6
 U.S. General Accounting Office, Military Education: DOD Needs to Align Academy
Preparatory Schools’ Mission Statements with Overall Guidance and Establish
Performance Goals, GAO-03-1017 (Washington, D.C.: September 2003); and Military
Education: Student and Faculty Perceptions of Student Life at the Military Academies,
GAO-03-1001 (Washington, D.C.: September 2003).




Page 2                                                 GAO-03-1000 Military Education
                   As agreed with your offices, this report addresses the following questions,
                   to what extent

                   (1) does DOD oversee the academies’ operations and performance?

                   (2) are applicants granted waivers from academic criteria for admissions?

                   (3) do various groups of students differ in admissions scores and academy
                       performance?

                   In addition to reviewing documents and interviewing officials at all three
                   academies, the service headquarters, the Office of the Under Secretary of
                   Defense for Personnel and Readiness (OUSD/P&R), and the academies’
                   boards of visitors, we reviewed admissions policies and procedures and
                   observed their use by academy officials in evaluating applications being
                   considered for the incoming class of 2007. We also obtained and analyzed
                   admissions and performance data for the student class that graduated in
                   2002. To compare student admissions qualifications and performance at
                   the academies, we identified six major groups of students common to all
                   academies: females, minorities, academy preparatory school graduates,
                   recruited athletes, prior enlisted personnel, and students whose academic
                   admission scores fell in the lower 30 percent of the entering class.7 Data on
                   student performance included academic grade point average; military
                   performance average, which is similar to a performance evaluation for
                   commissioned officers; and class rank.8 It also included graduation rate.
                   Other issues, such as recent controversies associated with alleged sexual
                   assault, did not fall within the scope of this review. Further details on our
                   scope and methodology are in appendix I. We conducted our work from
                   October 2002 through May 2003 in accordance with generally accepted
                   government auditing standards.


                   The Office of the Under Secretary of Defense for Personnel and Readiness
Results in Brief   (OUSD/P&R), the services, and the academies’ boards of visitors conduct
                   considerable oversight of the academies’ operations and performance, but
                   they lack a complete oversight framework. In 1991, our report concluded
                   that better oversight of the academies was needed and made
                   recommendations to improve DOD oversight. Since then, DOD has taken


                   7
                       Each group may contain members of the other groups.
                   8
                       Class rank is referred to as “order of merit” by the academies.




                   Page 3                                                       GAO-03-1000 Military Education
measures to address these issues, including establishing guidance on
oversight of the academies and uniform academy cost reporting. However,
DOD has not established a complete oversight framework, which would
include not only clear roles and responsibilities, but also performance
goals and measures against which to objectively assess performance.
OUSD/P&R, the services, and the academies’ boards of visitors have
different oversight roles, but largely conduct oversight activities without
the benefit of formalized performance goals and measures. OUSD/P&R
and the services use the number and types of commissioned officers as the
primary measure of academy performance. OUSD/P&R requires and
receives reports on academy performance from the services.

While data submitted in these reports provide perspective on such
performance measures as graduation rates, admissions trends for women
and minorities, and information on the quality of admitted students,
without stated performance goals and measures, these data do not offer
OUSD/P&R or the services as good an insight into the academies’
performance as they could. For example, the data collected by the
academies show that the graduation rates have increased in the last 10
years; however, there is no stated goal for graduation rate against which to
judge whether this rate of increase is adequate. Other data collected by the
academies indicate that the percentage of females and minorities has
fluctuated over the last 3 years, but apart from admissions targets used by
the Military Academy, there are no stated goals against which to measure
the adequacy of these admissions trends. Additionally, academy officials
regularly analyze data on student performance to determine the extent to
which admissions standards can be changed to improve overall student
performance at the academies. However, there are no stated goals for
student body performance, apart from minimum graduation standards
such as the cumulative academic grade point average, that might help the
academies and other oversight bodies assess overall student performance.
Additionally, each academy’s board of visitors—an external oversight
mechanism—focuses attention and actions on a wide range of operational
and quality of life issues at the academies. However, the boards do not
evaluate academy performance against established performance goals and
measures. Without formal goals and measures that are, moreover, linked
to mission statements, oversight bodies do not have sufficient focus for
their efforts and cannot systematically assess an organization’s strengths
and weaknesses nor identify appropriate remedies that would help them
achieve the best value for the nation’s investment in the academies.

The academies do not grant waivers from academic criteria or have
absolute minimum scores for admission. Under the whole person


Page 4                                          GAO-03-1000 Military Education
approach, the academies can admit some applicants whose academic
scores are lower than might normally be competitive for admission, but
who in their totality (academics, physical aptitude, and leadership) are
evaluated by academy officials as being qualified and capable of
succeeding at the academy. The only admissions criteria with an absolute
minimum score for qualifying for admissions is physical aptitude. The
academic and leadership criteria have a range of qualifying scores based
on what general levels of ability are considered competitive during the
admissions process. If an applicant’s score is lower than the competitive
range in academics, then admission officials have some flexibility in
(1) further considering the applicant by re-examining the student’s record
for information that can produce further insight about his or her academic
achievement and (2) weighing the extent to which the leadership
component of the whole person score may offset the low component. It is
possible for students to be admitted whose academic scores were not as
competitive as some of their peers who may not have been admitted. The
applicant is considered a risk and is evaluated through a deliberative
process by academy officials on the basis of their judgment of whether the
applicant is fully qualified and capable of succeeding at that academy. The
subjective nature of this approach is consistent with the intent of the
whole person concept, by which the academies want to admit students
who also demonstrate leadership characteristics that cannot be quantified
by purely objective scoring methods. Academy officials do not consider
this approach to represent an academic waiver, but instead their judicious
assessment of the whole person.

In our review of the academy classes that started in 1998 (class of 2002),
we found differences among various groups of students in their
admissions scores and similar differences in their performance while at
the academies; the differences in performance were not sizable. For the
class data we reviewed, minorities, academy preparatory school
graduates, recruited athletes, and prior enlisted students9 all had lower
average admissions scores than the average for the class as a whole. Of
those students in the lower 30 percent of the class in terms of academic
admissions scores, about 44 percent were recruited athletes, between 25
and 31 percent were minorities, and between 20 and 34 percent were
preparatory school graduates. Regarding performance, we found
differences at the academies between selected groups (i.e., females,
minorities, preparatory school graduates, recruited athletes, prior enlisted


9
    Each of these groups can contain members from other groups.




Page 5                                                   GAO-03-1000 Military Education
             students, and students in the lower 30 percent of the class in terms of
             academic admissions scores) and the class as a whole. Those differences
             varied but were generally not sizable. For example, females at one
             academy had a lower graduation rate than the class as a whole but a
             higher average academic grade point average and a higher average class
             rank. Some groups at all academies—such as minorities, preparatory
             school graduates, recruited athletes, and students in the lower 30 percent
             of their class in terms of academic admissions scores—performed at lower
             levels on average in all categories than the class as a whole, but these
             differences were not significant. For example, one of the lowest average
             academic grade point averages among the groups we reviewed was 2.61,
             whereas the average for the class as a whole at that academy was 2.93. A
             2.0 grade point average is required to graduate. The lowest graduation rate
             for the class we reviewed was 65 percent for the students in the lower 30
             percent of their class in terms of academic admissions scores at one
             academy. The average graduation rate for the class as a whole at that
             academy was 74 percent.

             We are making a recommendation to improve DOD’s oversight of
             operations and performance at the academies through the enhancement of
             performance goals and measures. In comments on a draft of this report,
             DOD concurred with our recommendation.


             The Departments of the Army, the Navy, and the Air Force each have their
Background   own educational institutions (academies) to produce a portion of each
             branch’s officer corps:10

             •    U.S. Military Academy (West Point, N.Y.), established in 1802;
             •    U.S. Naval Academy (Annapolis, Md.), established in 1845; and
             •    U.S. Air Force Academy (Colorado Springs, Colo.), established in 1954.

             The academies are structured to provide a curriculum critical to the
             development of successful future officers in academic, military, and
             physical areas of achievement. Additionally, the academies emphasize the
             moral and ethical development of students through their respective honor
             codes and concepts.



             10
               Other sources for commissioned officers include ROTC programs at colleges and
             universities and officer candidate/training schools for individuals who already have college
             degrees.




             Page 6                                                    GAO-03-1000 Military Education
There are approximately 4,000 students enrolled at each of the three
service academies at any given time, each comprising four classes. In
December 2002, Congress authorized an annual increase of up to 100
students until the total number reaches 4,400 for each academy. 11 In 2002
the Military Academy graduated 968 students; the Naval Academy 977
students; and the Air Force Academy 894 students. Faculty at the U.S.
Military Academy and the U.S. Air Force Academy are comprised
predominantly of military officers (79 and 75 percent, respectively), while
at the U.S. Naval Academy 59 percent of the faculty are civilians. Table 1
shows the composition of the faculty at the service academies.

Table 1: Civilian and Military Faculty at the Service Academies

                                        Total       Total number of            Total number of
                                     number          civilian faculty           military faculty
 Service academy                   of faculty         (% of faculty)             (% of faculty)
 U.S. Military Academy                    622              131 (21%)                  491 (79%)
 U.S. Naval Academy                       555              326 (59%)                  229 (41%)
 U.S. Air Force
 Academy                                  490              123 (25%)                  367 (75%)
 Total                                  1,667              580 (35%)                 1,087 (65%)
Source: DOD.

Note: Faculty information is based on a snapshot of each academy in February 2003.


DOD reports that the total cost to operate all three academies in fiscal
year 2002 was $990.7 million. Table 2 shows the reported operating costs
and cost per graduate for each academy from fiscal year 1999 through
fiscal year 2002. We did not independently verify these costs.




11
  Pub. L. 107-314, Bob Stump National Defense Authorization Act for Fiscal Year 2003, §
532, December 2, 2002.




Page 7                                                       GAO-03-1000 Military Education
Table 2: Academy Operating Costs and Cost Per Graduate, Fiscal Years 1999-2002

                                                      Fiscal year          Fiscal year           Fiscal year           Fiscal year
 Academy              Cost Category                         1999                 2000                  2001                  2002
 Military Academy     Total operating costs         $301,058,452         $330,603,820          $336,416,716          $364,971,975
                      Cost per graduate                  312,150               320,120              339,318                349,327
 Naval Academy        Total operating costs          245,749,679           253,817,467          273,809,865            292,696,358
                      Cost per graduate                  254,983               256,931              266,033                275,001
 Air Force Academy    Total operating costs          277,639,005           314,972,559          321,335,152            333,056,023
                      Cost per graduate                  305,945               305,133              313,456                322,750
Source: DOD.



                                          Prospective students must meet basic eligibility requirements for
                                          appointment to an academy. They must (1) be unmarried, (2) be a U.S.
                                          citizen, (3) be at least 17 years of age and must not have passed their
                                          twenty-third birthday on July 1 of the year they enter an academy, (4) have
                                          no dependents, and (5) be of good moral character.12

                                          After determining eligibility, a candidate submits an application to a
                                          preferred academy or academies. Each submitted application is required
                                          to include information such as, but not limited to, the candidate’s (1) SAT
                                          scores (or American College Testing—ACT—examination scores); (2) high
                                          school grade point average (and class rank, if possible); (3) physical
                                          aptitude scores; (4) medical examination results; and (5) extracurricular
                                          activities. The academies admit those candidates that have secured a
                                          nomination and who represent, in the opinion of academy officials, the
                                          best mixture of attributes (academic, physical, and leadership) necessary
                                          to ensure success at the academies and as military officers.

                                          The military academies use a “whole person” method to assess potential
                                          candidates in three major areas: (1) academics, (2) physical aptitude, and
                                          (3) leadership potential. Each academy uses the same basic approach.
                                          Admissions assessments are weighted toward academic scores that
                                          include objective tests and high school performance. Leadership potential
                                          is measured by assessing athletic and non-athletic extracurricular
                                          activities. Subjective assessments of potential candidates in these major
                                          areas also contribute to final admissions “scores.” Such assessments



                                          12
                                           10 U.S.C. §§ 4346, 6958, and 9346; and Department of Defense, Directive 1322.22, Service
                                          Academies, § 4.3, August 24, 1994.




                                          Page 8                                                  GAO-03-1000 Military Education
include interviews with prospective candidates, teacher/coach
evaluations, and analyses of writing samples. Though medical criteria
differ between services, the medical examinations are conducted
according to the same standards, under a joint DOD Medical Examination
Review Board that manages the medical examination process and records
for applicants to all academies.13

Each academy is authorized to permit up to 60 foreign students to attend
at any given time on a reimbursable basis by their country of origin.14 This
number does not count against the authorized student strength of the
academies. The admission of foreign students is covered by separate
policies and procedures. Our review was limited to the policies and
procedures for admitting U.S. citizens to the academies. Figure 1 shows
the basic steps in the admissions process for all U.S. applicants.

Figure 1: Basic Steps in the Academy Admissions Process




13
  See Department of Defense, Directive 5154.25, DOD Medical Examination Review
Board, June 11, 1981; Directive 6130.3, Physical Standards for Appointment, Enlistment,
or Induction, December 15, 2000; and Instruction 6130.4, Criteria and Procedure
Requirements for Physical Standards for Appointment, Enlistment, or Induction in the
Armed Forces, December 14, 2000.
14
     10 U.S.C. §§ 4344, 6957, and 9344.




Page 9                                                 GAO-03-1000 Military Education
                         Students who are disenrolled from an academy after the start of their third
                         year may be required to complete a period of active duty enlisted service
                         of up to 4 years or may be required to reimburse the federal government
                         for the cost of their education. Those who are disenrolled in their first 2
                         years do not incur an active service or reimbursement obligation.15


United States Military   The United States Military Academy’s admissions evaluation considers
Academy Admissions       academics, leadership, and physical aptitude. Academic considerations
Process                  include above-average high school or college academic records as well as
                         strong performance on SAT/ACT. Additionally, the Military Academy
                         considers recommendations from English, mathematics, and science
                         teachers. The leadership potential considers demonstrations of leadership
                         and initiative in sports, school, community, or church activities and strong
                         recommendations from faculty and community leadership and is a more
                         subjective assessment of character. Physical aptitude is based on a scored
                         standardized test. This test is made up of pull-ups for men or the flexed-
                         arm hang for women, push-ups, standing long jump, basketball throw, and
                         shuttle run. Figure 2 shows the areas considered and the weights assigned
                         to each area in the U.S. Military Academy’s whole person admissions
                         process.




                         15
                           Department of Defense, Directive 1332.23, Service Academy Disenrollment, §§ 6.1 and
                         6.2, February 19, 1988.




                         Page 10                                                GAO-03-1000 Military Education
                      Figure 2: Areas and Their Weights Considered in the U.S. Military Academy’s Whole
                      Person Admissions Process




United States Naval   The United States Naval Academy’s admissions evaluation considers
Academy Admissions    academics, leadership, physical aptitude, and technical interest. Academic
Process               considerations include above-average high school or college academic
                      records as well as strong performance on SAT/ACT. Additionally, the
                      Naval Academy considers recommendations from English and
                      mathematics teachers. Assessment of leadership potential represents a
                      subjective evaluation of character in which the academy considers
                      demonstrations of leadership in terms of extracurricular activities in
                      sports, school, community, or church and strong recommendations from
                      faculty and community leadership. Physical aptitude is based on a scored,
                      standardized test consisting of pull-ups for men or the flexed-arm hang for
                      women, push-ups, standing long jump, basketball throw, and shuttle run.
                      Additionally, the Naval Academy considers the technical interest of a
                      prospective student, which is measured through a questionnaire in the
                      application packet and used to gauge interest in pursuing a technical
                      degree. The intent of this requirement is to admit students that are
                      interested in pursuing technical degrees, specifically nuclear and maritime
                      engineering. The admissions board can also apply further points to an
                      applicant’s overall whole person score based on further consideration of
                      an applicant’s record, including such things as the results of the evaluation
                      form filled out by the Naval Academy representative who interviewed the



                      Page 11                                            GAO-03-1000 Military Education
                          applicant. Figure 3 shows the areas considered and the weights assigned
                          to each area in the U.S. Naval Academy’s whole person admissions
                          process.

                          Figure 3: Areas and Their Weights Considered in the U.S. Naval Academy’s Whole
                          Person Admissions Process




United States Air Force   The United States Air Force Academy’s admissions evaluation considers
Academy Admissions        academics, leadership, and an assessment by the selections panel.
Process                   Academic considerations include above-average high school or college
                          academic records as well as strong performance on SAT/ACT.
                          Additionally, the Air Force Academy considers recommendations from
                          English and mathematics teachers. Under leadership potential, the
                          academy considers extracurricular activities in sports, school, community,
                          or church and strong recommendations from faculty and community
                          leadership. Finally, the Air Force Academy Selections Panel makes an
                          assessment of all potential students. This assessment is composed of a
                          pass/fail score from the physical aptitude examination and the evaluation
                          of the academy’s liaison officer evaluation, made after interviewing the
                          applicant. The physical aptitude examination is made up of pull-ups for
                          men or the flexed-arm hang for women, push-ups, standing long jump,
                          basketball throw, and shuttle run. The leadership potential area and the
                          admissions board include the more subjective assessments of a potential


                          Page 12                                           GAO-03-1000 Military Education
                 student. Figure 4 shows the areas considered and the weights assigned to
                 each area in the U.S. Air Force Academy’s whole person admissions
                 process.

                 Figure 4: Areas and Their Weights Considered in the U.S. Air Force Academy’s
                 Whole Person Admissions Process




Nomination and   The President of the United States alone appoints candidates to the
Appointment of   academies.16 Before receiving an appointment, all candidates must secure
Candidates       one or more nominations according to the following categories: 17

                 •      congressional (including a U.S. senator, representative, delegate, or the
                        Vice President);

                 •      service-connected (including, among others, children of disabled
                        veterans, enlisted personnel in the active or reserve components, and
                        students from ROTC programs or other designated honor school
                        graduates); and




                 16
                      10 U.S.C. §§ 4341a, 6953, and 9341a.
                 17
                      10 U.S.C. §§ 4342, 6954, and 9342.




                 Page 13                                             GAO-03-1000 Military Education
                   •    other (including the academy superintendents’ nominees and other
                        nominees to bring the incoming class to full strength).

                   Figure 5 shows the approximate distribution of categories of academy
                   nominations, based on the types and numbers of nominees per category
                   allowed by law.

                   Figure 5: Categories of Academy Nominations




                   Note: Percentages do not add to 100 due to rounding.




Oversight of the   Oversight of the academies is the responsibility of three principal
Academies          organizations: OUSD/P&R, the service headquarters, and the board of
                   visitors of each academy. According to Department of Defense Directive
                   1322.22 (Service Academies),18 OUSD/P&R serves as the DOD focal point
                   for matters affecting the academies and has responsibility to assess
                   academy operations and establish policy and guidance for uniform
                   oversight and management of the military academies. The military
                   departments perform the primary DOD oversight function for their
                   respective academies. The superintendent of each academy reports



                   18
                    Department of Defense, Directive 1322.22, Service Academies § 5.1, August 24, 1994.
                   DOD is currently revising this directive.




                   Page 14                                                 GAO-03-1000 Military Education
                       directly to the uniformed head of his respective service (the Chiefs of Staff
                       for the Army and the Air Force and the Chief of Naval Operations for the
                       Navy), in accordance with the chain of command for each service. Each
                       academy also has a board of visitors, mandated by law,19 that is comprised
                       of congressional members and presidential appointees. These boards
                       focus attention and action on a wide range of operational and quality of
                       life issues at the academies.

                       As educational institutions, the service academies are also overseen by
                       several nongovernmental organizations that are outside DOD purview.
                       Each academy undergoes periodic review by a higher-education
                       accreditation body associated with its region of the country, 20 usually
                       involving a full review every 10 years with an interim review every 5 years.
                       The accreditation bodies review such areas as core curriculum, strategic
                       planning, self-assessments, diversity of faculty and students, and faculty
                       credentials. The athletic programs of the academies are also subject to
                       periodic certification by the National Collegiate Athletic Association. This
                       body reviews academy athletics in terms of such issues as finances and
                       impact on the education mission of the academies. We limited our review
                       of oversight of the academies to DOD organizations and the boards of
                       visitors.


                       The OUSD/P&R, the services, and the academies’ boards of visitors
DOD Lacks a            conduct many oversight activities, but they lack a complete oversight
Complete Framework     framework. A complete oversight framework includes not only clear roles
                       and responsibilities, but also performance goals and measures against
for Oversight of the   which to objectively assess performance. Such elements embody the
Academies              principles of effective management in which achievements are tracked in
                       comparison with plans, goals, and objectives and the differences between
                       actual performance and planned results are analyzed. Without formal
                       goals and measures, oversight bodies do not have sufficient focus for their
                       efforts and cannot systematically assess an organization’s strengths and
                       weaknesses nor identify appropriate remedies that would permit DOD to
                       achieve the best value for the investment in the academies. In a prior




                       19
                            10 U.S.C. §§ 4355, 6968, and 9355.
                       20
                         Commission on Higher Education of the Middle States Association of Colleges and
                       Schools (Military and Naval Academies) and Commission of Institutions of Higher
                       Education of the North Central Association of Colleges and Schools (Air Force Academy).




                       Page 15                                                GAO-03-1000 Military Education
report,21 GAO concluded that better external oversight of the academies
was needed to provide useful guidance and suggestions for improvement.
The report recommended that DOD improve oversight of the academies
through such measures as establishing a focal point for monitoring
academy issues in the Office of the Secretary of Defense and establishing
guidance on uniform cost reporting.

OUSD/P&R and the services have established clear roles and
responsibilities for oversight of the academies, with the former serving as
the focal point for issues affecting all academies and the latter having
direct oversight authority over their respective academies. DOD
established guidance in 1994 for the oversight of the academies22 and for
uniform reporting of costs and resources.23 OUSD/P&R is directly involved
in those policy issues that affect all academies and require DOD-level
attention and legislative matters. For example, the office was recently the
DOD focal point on the issue of increasing authorized enrollment at the
academies from 4,000 to 4,400. With respect to the academies, the office is
chiefly concerned with monitoring the degree to which the services are
meeting their goals for the accession of new officers.24 The office also
coordinates major studies that affect the academies, such as a November
1999 report on the career progression of minority and women officers.

The services are responsible for direct oversight of their respective
academies; and the academies are treated similarly to major military
commands. The superintendents of the academies are general/flag officers
who report directly to the uniformed heads of their services (the Chiefs of
Staff for the Army and the Air Force and the Chief of Naval Operations for
the Navy). In addition to overseeing the academies’ budget through the
same approval process as a major command activity, the services oversee
the academies’ operations and performance primarily through the
academies’ goal of meeting service officer accession targets. The


21
  U.S. General Accounting Office, DOD Service Academies: Improved Cost and
Performance Monitoring Needed, GAO/NSIAD-91-79 (Washington, D.C.: July 16, 1991).
22
  Department of Defense, Directive 1322.22, Service Academies, August 24, 1994. DOD is
currently revising this directive.
23
   Department of Defense, Instruction 1025.4, Service Academy Resources Report, October
18, 1994.
24
  The academies are one of the sources for officers. The others include reserve officer
training programs at colleges and universities, officer candidate/training schools, and direct
commissioning programs.




Page 16                                                    GAO-03-1000 Military Education
superintendents are responsible for meeting those targets and, in so doing,
are given wide discretion in such areas as modifying their specific
admissions objectives and the process for matching graduates with service
assignments. The service headquarters use a number of mechanisms to
oversee academy performance. For example, each service headquarters
provides officer accession targets to the academies so that the assignment
of graduates and the make up of incoming student classes can be modified
as necessary. In addition to general numbers of officers, each service also
has a number of specialty officer fields that need to be filled, and the
services also monitor the extent to which the academies will be able to
meet those accession goals.

The services also directly oversee the academies by requiring the
superintendents to report on and discuss their operations. For example,
the Air Force uses an annual forum of the most senior Air Force officers to
focus on the Air Force Academy with respect to how it is meeting the
needs of the operational Air Force. The Navy uses similar senior officer
conferences and frequent interaction between the superintendent and
Navy headquarters to conduct oversight. The Army uses the U.S. Military
Academy Forum, comprised of senior Army officers, to address academy
operations issues. The superintendents of the three academies also hold
annual meetings to discuss issues common to all academies. These
mechanisms have resulted in such academy actions as curriculum changes
to increase the number of technical degree majors, increasing language
requirements, and increasing the number of students attending the
academies.

While OUSD/P&R and the services conduct a wide variety of oversight
activity, there are few stated performance goals against which to measure
academy operations and performance. Each of the academies has a
strategic plan that is focused on providing quality military and professional
training and education in order to commission highly capable junior
officers. These plans are approved by the service headquarters but are not
generally used by the services as benchmarks against which to measure
academy performance, and they do not contain specific goals against
which to measure student performance. OUSD/P&R is required to assess
and monitor academy operations based on the information provided in
annual reports it requires from the service secretaries.25 These reports
provide data on various aspects of performance, such as student


25
     DOD Directive 1322.22 §§ 5.1.2 and 6.1.




Page 17                                         GAO-03-1000 Military Education
demographics and trends, student quality, admissions and attrition trends,
compensation for students and faculty, leadership and honor systems, and
incidents of indiscipline.

The reports provide OUSD/P&R and the services with information on
current and past performance for academy operations, but apart from
officer accession goals, neither OUSD/P&R nor the services have specific
stated performance goals against which to compare the information
provided in the assessment reports, thus they do not have an explicit basis
for judging the adequacy of their performance. For example, the data
collected by the academies show that graduation rates have increased in
the last 10 years; however, there is no stated goal for a graduation rate
against which to judge whether this rate of increase is adequate. Other
data collected by the academies indicate that the percentage of females
and minorities has fluctuated over the last 3 years, but apart from
admissions targets used by the U.S. Military Academy, there are no stated
goals against which to assess these trends. Additionally, academy officials
regularly analyze data on student body performance to determine the
extent to which admissions standards can be changed to affect student
body performance. However, there are no stated goals for student body
performance, apart from minimum graduation standards, that might help
the academies and other oversight bodies assess overall student
performance.

The oversight efforts of each academy’s board of visitors are similarly
limited by the absence of sufficient performance goals and measures. Each
of the academies has a board of visitors, mandated by law26 and comprised
of Members of Congress and presidential appointees, that is outside the
DOD chain of command. The boards have a broad legal mandate to inquire
into all aspects of academy operations.27 The boards meet several times a
year to be briefed on and discuss academy operations and must conduct
an annual visit to their respective academies. During these visits, the
boards are briefed by academy staff on such issues as admissions,
curriculum, recruiting, athletics, morale and welfare, and construction
programs; they also interview students to obtain their perceptions of life at
the academies. The boards also address inquiries to academy staff, which
are usually followed up at subsequent meetings, and they make
suggestions to improve operations or quality of life at the academies. For


26
     10 U.S.C. §§ 4355, 6968, and 9355.
27
     10 U.S.C. §§ 4355, 6968, and 9355.




Page 18                                         GAO-03-1000 Military Education
                        example, boards of visitors have recommended increased recruiting of
                        qualified minority applicants from various congressional districts and
                        increased surveying of students on quality of life issues.

                        The boards submit annual reports to the President on the status of and
                        issues at the academies but do not evaluate academy operations and
                        performance against established performance goals. The boards of visitors
                        do not have dedicated staffs to conduct their work, and though board
                        members may inquire into any aspect of academy operations, the agenda is
                        set largely by the briefings presented to the boards by academy officials.
                        Academy officials with whom we spoke were generally satisfied with the
                        oversight provided by the boards of visitors, though there were concerns
                        at the Air Force Academy about poor attendance by board members
                        during annual visits to the academy.


                        The academies do not grant waivers from academic criteria but do not
Whole Person            have absolute minimum scores for admission. Under the whole person
Approach Allows         approach, the academies can admit some applicants whose academic
                        scores are lower than might normally be competitive for admission, but
Academies Flexibility   who in their totality (academics, physical aptitude, and leadership
to Admit Students       potential) are deemed an acceptable risk and qualified to attend an
                        academy. This admissions approach is consistent with the intent of the
with a Range of         academies to admit students who also demonstrate leadership and
Qualifications          initiative characteristics, which cannot be quantified by purely objective
                        scoring methods.

                        When conducting their admissions processes, the academies do not set
                        absolute minimum scores for academic ability. Rather, they establish a
                        range of scores that would be considered competitive, based on past
                        incoming class performance and academy research on the overall quality
                        of the applicant pool. Prior to 2002, the Air Force Academy set absolute
                        minimum academic scores, and a waiver was required to further consider
                        an applicant who fell below that minimum, no matter how high his or her
                        scores in the leadership area. However, the Air Force Academy no longer
                        has absolute minimums and uses the same competitive range approach as
                        the other academies. Under this approach, if an applicant’s academic score
                        is lower than the competitive range guidelines, academy officials have
                        some flexibility to further consider the applicant. Academy officials will
                        re-examine the applicant’s record for information that might provide
                        further insight about his or her academic achievement. For example,
                        officials may contact high school teachers to inquire about the types and
                        difficulty of the classes the applicant has been taking and his or her


                        Page 19                                         GAO-03-1000 Military Education
performance in those classes. Academy officials will also weigh the extent
to which the leadership component of the applicant’s whole person score
offset the low component. The applicant is considered a risk and is
evaluated through a deliberative process by academy officials on the basis
of their judgment of whether the applicant is fully qualified and capable of
succeeding at that academy. The subjective nature of this approach is
consistent with the intent of the whole person concept, by which the
academies want to admit students who also demonstrate leadership
characteristics that cannot be quantified by purely objective scoring
methods. Academy officials do not consider these judgments to constitute
a waiver of academic standards, but rather a judicious assessment of the
whole person. The process for assessing those applicants whose academic
scores are lower than might normally be competitive is nonetheless similar
to the former Air Force Academy process for granting waivers.

With over 10,000 applicants28 for each academy each year and about 1,200
students admitted, the academic standards are high. Academy data show
that the academic quality of the applicants has remained high over the past
4 years, and the competitive ranges for academic scores used by the
academies have remained the same or have increased during this time.
However, it is possible for students to be admitted whose academic scores
were not as competitive as some other applicants who may not have been
admitted. Senators, representatives, and delegates may submit up to 10
nominees for each student vacancy available to him or her per academy.
They may choose to designate one as a principal nominee.29 If an applicant
receives a principal nomination and is in all other respects qualified, the
academies must admit that applicant, even over an applicant on the same
senator’s, delegate’s, or representative’s nomination list with higher
academic and/or whole person scores. The other nominated names
become alternates for possible admission later in the admissions process.

Though some academies award credit for the extent to which an applicant
surpasses the standards of the physical aptitude examination, there are
minimum standards for the physical test that must be met. None of the
academies uses a system of “waivers,” except for medical conditions. An
applicant can be waived for a medical condition, based on the deliberation
and judgment of DOD medical personnel and the academy superintendent.


28
  This includes the total number of students who applied and not the number that received
a nomination.
29
     10 U.S.C. §§ 4342, 6954, and 9342.




Page 20                                                 GAO-03-1000 Military Education
                      For example, an applicant who is disqualified due to a vision condition
                      may apply for and receive a waiver, based on subsequent surgical vision
                      correction or determination by the academy superintendent that the
                      applicant would be able to serve on active duty without the vision
                      condition being a problem.


                      In our review of the academy classes that started in 1998 (class of 2002),
No Significant        we found differences among various groups of students in their
Differences in        admissions scores and similar differences in their performance while at
                      the academies, but the differences were not significant in magnitude. In
Admissions and        terms of performance after admission to the academies,30 differences
Academy               between these student groups and the class as a whole were also not
                      sizable. We reviewed data for the following distinct groups: 31
Performance between
Various Groups of     •     overall class,
                      •     females,
Students              •     minorities,
                      •     academy preparatory school graduates,
                      •     recruited athletes,
                      •     prior enlisted, and
                      •     lower 30 percent of class by academic admissions scores.

                      For the class data we reviewed, minorities, academy preparatory school
                      graduates, recruited athletes, and prior enlisted students all had lower
                      average admissions scores than the average for the class as a whole,
                      though these differences varied. The differences between groups and the
                      class as a whole were not sizable, generally falling within 5 percent. Those
                      differences that were statistically significant and outside the 5 percent
                      range were still generally less than 10 percent of the class as a whole.
                      Tables 3, 4, and 5, show the average admissions scores for the selected
                      groups in the class that started in 1998 at the Military, Naval, and Air Force
                      Academies, respectively. Although each academy uses the same
                      fundamental whole person approach, they use different scales to calculate
                      scores. Therefore, the academic and whole person scores cannot be
                      compared across academies.



                      30
                        We used the following performance factors to measure student performance at the
                      academies: cumulative grade point average, cumulative military performance average,
                      order of merit (class rank), and graduation rate (for each group of students).
                      31
                           Each of these groups can contain members from other groups.




                      Page 21                                                  GAO-03-1000 Military Education
Table 3: Average Admissions Scores for the Selected Groups in the Class That Started in 1998 at the U.S. Military Academy

                                                                                                                                               Lower
                                                                                         Preparatory                                    30 percent of
                                                                                             school           Recruited                  admissions
 Average                                   Overall      Females         Minorities        graduates            athletes Prior enlisted          class
 admissions score                          (1,246)         (192)            (269)              (184)              (279)            (31)         (377)
                                                                                                        b                a                                      b
 Academic score                                 600           603               583               546               558                 594                  532
                                                                                                        a                                                       a
 Whole person score                          6,006           6,022            5,865             5,645              5,814              5,861            5,609
Source: GAO analysis, from Military Academy sources.
                                                        a
                                                         Denotes a group average or percentage with a statistically significant difference greater than 5%
                                                        from the overall average or percentage.
                                                        b
                                                         Denotes a group average or percentage with a statistically significant difference greater than 10%
                                                        from the overall average or percentage.



Table 4: Average Admissions Scores for the Selected Groups in the Class That Started in 1998 at the U.S. Naval Academy

                                                                                                                                              Lower
                                                                                        Preparatory                                    30 percent of
                                                                                            school           Recruited                  admissions
 Average                                  Overall       Females         Minorities       graduates            athletes Prior enlisted          class
 admissions score                         (1,226)          (190)            (221)             (146)              (380)            (76)         (368)
                                                                                                       b                                   a                       b
 Academic score                                 618           624               594               545                596               570                   544
                                                                                                       a                                   a                       a
 Whole person Score                        65,732           65,719          63,769            61,254             64,233            62,256             61,404
Source: GAO analysis, from Naval Academy sources.
                                                        a
                                                         Denotes a group average or percentage with a statistically significant difference greater than 5%
                                                        from the overall average or percentage.
                                                        b
                                                         Denotes a group average or percentage with a statistically significant difference greater than 10%
                                                        from the overall average or percentage.



Table 5: Average Admissions Scores for the Selected Groups in the Class That Started in 1998 at the U.S. Air Force Academy

                                                                                                                                              Lower
                                                                                        Preparatory                                    30 percent of
                                                                                            school           Recruited                  admissions
 Average                                  Overall       Females         Minorities       graduates            athletes Prior enlisted          class
 admissions score                         (1,216)          (190)            (229)             (157)              (312)            (44)         (366)
                                                                                                                                                                   b
 Academic Score                              3,202           3,216            3,123             3,112              3,043              3,188            2,863
 Whole person Score                             798           805               782               774                773                792                  751a
Source: GAO analysis, from Air Force Academy sources.
                                                        a
                                                         Denotes a group average or percentage with a statistically significant difference greater than 5%
                                                        from the overall average or percentage.
                                                        b
                                                         Denotes a group average or percentage with a statistically significant difference greater than 10%
                                                        from the overall average or percentage.




                                                        Page 22                                                         GAO-03-1000 Military Education
                                             Of those students in the lower 30 percent of the class in terms of academic
                                             admissions scores, about 44 percent were recruited athletes, between 25
                                             and 31 percent were minorities, and between 20 and 34 percent were
                                             preparatory school graduates. Table 6 shows the percentage of the
                                             selected groups making up the lower 30 percent of the classes in terms of
                                             their academic admissions scores, by academy.

Table 6: Percentage of the Selected Groups Making Up the Lower 30 percent of the Classes in Terms of Their Academic
Admissions Scores, by Academy

 Numbers in percent
                                                                                Preparatory               Recruited
 Academy                                  Females            Minorities    school graduates                athletes         Prior enlisted
 Military Academy                              16                    31                     34                    44                     3
 Naval Academy                                 13                    29                     32                    45                     13
 Air Force Academy                             14                    25                     20                    44                     5
Source: GAO analysis, from DOD sources.

                                             Note: Numbers do not add to 100 percent because each of the groups can contain members of
                                             another group.


                                             We also found differences in performance after admission to the
                                             academies between selected groups and the class as a whole. For
                                             example, females at the Naval Academy had a lower graduation rate than
                                             the class as a whole, but they had a higher average academic grade point
                                             average (cumulative GPA) than the class as a whole and higher average
                                             class rank (order of merit). The differences in performance between the
                                             selected groups and the class as a whole were not sizable, generally falling
                                             within 5 percent. Those differences that were statistically significant and
                                             outside the 5 percent range were still generally less than 10 percent of the
                                             class as a whole. Tables 7, 8, and 9 show how the selected groups
                                             performed at the Military, Naval, and Air Force Academies, respectively.
                                             See appendix II for further information on comparisons of performance by
                                             defined student groups.




                                             Page 23                                                     GAO-03-1000 Military Education
Table 7: Student Performance for Selected Groups at the U.S. Military Academy

                                                                                                                                                       Lower
                                                                                                  Preparatory                                   30 percent of
                                                                                                      school       Recruited                     admissions
                                              Overall          Females       Minorities            graduates        athletes    Prior enlisted          class
 Performance score                            (1,246)             (192)          (269)                  (184)          (279)               (31)         (377)
                                                                                                               a                                                   a
 Average cumulative                                 2.99           2.99              2.82               2.61            2.81              3.14              2.66
 GPA
 Average cumulative                                 3.28           3.26              3.21                3.26           3.20              3.37                  3.21
 MPA
                                                                                              a                a                                                   a
 Average order of merit                             3.03           3.04             2.86                2.75            2.90              3.06              2.78
                                                                                              a                                                                    a
 Graduation rate                                  78%             76%               71%                  72%            76%               71%               71%
Source: GAO analysis, from Military Academy sources.
                                                           a
                                                            Denotes a group average or percentage with a statistically significant difference greater than 5%
                                                           from the overall average or percentage.
                                                           b
                                                            Denotes a group average or percentage with a statistically significant difference greater than 10%
                                                           from the overall average or percentage.



Table 8: Student Performance for Selected Groups at the U.S. Naval Academy

                                                                                                                                                     Lower
                                                                                                  Preparatory                                 30 percent of
                                                                                                      school        Recruited                  admissions
                                             Overall           Females      Minorities             graduates         athletes Prior enlisted          class
 Performance score                           (1,226)              (190)         (221)                   (146)           (380)            (76)         (368)
                                                                                          a                    a                                                   a
 Average cumulative                              2.97              3.01            2.82                 2.67             2.86             3.02              2.67
 GPA
 Average cumulative                              3.12              3.16             3.02                 2.99            3.08             3.19                  3.00
 MPA
 Average order of merit                             489            456a             590b                 658b            551b              453                  661b
 Graduation rate                                 80%              71%b             75%                   77%             79%              72%               76%a
Source: GAO analysis, from Naval Academy sources.
                                                           a
                                                            Denotes a group average or percentage with a statistically significant difference greater than 5%
                                                           from the overall average or percentage.
                                                           b
                                                            Denotes a group average or percentage with a statistically significant difference greater than 10%
                                                           from the overall average or percentage.




                                                           Page 24                                                          GAO-03-1000 Military Education
Table 9: Student Performance for Selected Groups at the U.S. Air Force Academy

                                                                                                                                                      Lower
                                                                                        Preparatory                                            30 percent of
                                                                                            school        Recruited                             admissions
                                            Overall     Females        Minorities        graduates         athletes      Prior enlisted                class
 Performance score                          (1,216)        (190)           (229)              (157)           (312)                 (44)               (366)
                                                                                    a               b                                                           a
 Average cumulative                              2.93       2.97             2.78              2.61              2.79               2.89                 2.64
 GPA
 Average cumulative                              2.90       2.93              2.89              2.83             2.81               2.93                     2.84
 MPA
                                                                                    b               b                b                                          b
 Average order of merit                          469         440              545               663              568                 499                     646
                                                                                                                                                                b
 Graduation rate                                74%         75%               71%               69%              71%                66%                  65%
Source: GAO analysis, from Air Force Academy sources.
                                                        a
                                                         Denotes a group average or percentage with a statistically significant difference greater than 5%
                                                        from the overall average or percentage.
                                                        b
                                                         Denotes a group average or percentage with a statistically significant difference greater than 10%
                                                        from the overall average or percentage.


                                                        Some groups—such as minorities, preparatory school graduates, recruited
                                                        athletes, and students in the lower 30 percent of their class in terms of
                                                        academic admissions scores—performed at lower levels on average in all
                                                        categories than the class as a whole, but these differences varied between
                                                        academies and by category and were not sizable. For example, one of the
                                                        lowest average academic grade point averages for the groups we reviewed
                                                        was 2.61 and the average for the class as a whole at that academy was 2.93.
                                                        A 2.0 grade point average is required to graduate for academic and military
                                                        averages. Similarly, the lowest graduation rate for the class we reviewed
                                                        was 65 percent for the students in the lower 30 percent of their class in
                                                        terms of academic admissions scores at one academy. The average
                                                        graduation rate for the class as a whole was 74 percent.

                                                        Our analysis of data for the students who entered the academies in 1998
                                                        (class of 2002) indicates that admissions scores are generally good
                                                        predictors of performance at the academies. Of the admissions scores, the
                                                        academic component of the whole person scores was often the best
                                                        predictor of academic performance at the academies, and the whole
                                                        person scores in their entirety were often the best predictors of military
                                                        performance at the academies. Both academic and whole person
                                                        admissions scores were good predictors of class rank. In general, whole
                                                        person admissions scores were better predictors of graduation rate than
                                                        the academic admissions scores alone.




                                                        Page 25                                                          GAO-03-1000 Military Education
                     Although the service academies receive oversight from a number of
Conclusion           organizations and have established guidance for that oversight that
                     includes the reporting of a wide range of data on academy operations,
                     without clear and agreed-upon performance goals, there is no objective
                     yardstick against which to fully measure academy performance and
                     operations, apart from the officer accessions goals currently used.
                     Establishment of such performance goals is consistent with the principles
                     of effective management and would enhance the quality of oversight
                     already performed by OUSD/P&R, the services, and the academy boards of
                     visitors, permitting them to more clearly note those areas in which the
                     academies excel, highlight areas where improvement is warranted, and
                     achieve the best value for the nation’s investment in the academies.


                     To improve DOD oversight of the operations and performance of the
Recommendation for   service academies, we recommend that the Secretary of Defense direct the
Executive Action     OUSD/P&R, in concert with the services, to further enhance performance
                     goals and measures whereby the information required in annual
                     assessment reports can be better evaluated. These performance goals
                     should be developed for each academy and, where appropriate, in
                     common for all academies. The specific goals should coincide with
                     performance elements agreed upon by the services and OUSD/P&R and
                     might include such things as graduation rates, demographic composition
                     of student classes, assessments of officer performance after graduation,
                     and other performance information already collected by the academies,
                     including performance characteristics of various groups of students.


                     In comments on a draft of this report, DOD agreed with our
Agency Comments      recommendation to further enhance performance goals and measures for
                     the service academies whereby the information required in annual
                     assessment reports can be better evaluated. DOD further stated that the
                     Office of the Under Secretary of Defense for Personnel and Readiness
                     OUSD/P&R will (1) monitor development of improved goals and measures
                     by the service academies, to include facilitating the development of
                     common performance goals where appropriate and (2) update DOD




                     Page 26                                       GAO-03-1000 Military Education
Directive 1322.22, Service Academies, as required. DOD’s written
comments are included in their entirety in appendix III.


We are sending copies of this report to the appropriate congressional
committees; the Secretaries of Defense, the Army, the Navy, and the Air
Force; and the Director, Office of Management and Budget. We will also
make copies available to others upon request. In addition, the report will
be available at no charge on the GAO Web site at http://www.gao.gov.

Please contact me on (202) 512-5559 if you or your staff have any
questions concerning this report. Key contributors are listed in
appendix V.




Derek B. Stewart
Director
Defense Capabilities and Management




Page 27                                         GAO-03-1000 Military Education
             Appendix I: Scope and Methodology
Appendix I: Scope and Methodology


             To assess the extent to which DOD oversees the service academies’
             operations and performance, we interviewed officials at the Office of the
             Under Secretary of Defense for Personnel and Readiness; the Army, Navy,
             and Air Force headquarters; and the U.S. Military, U.S. Naval, and U.S. Air
             Force Academies. We reviewed documents on service and DOD oversight
             criteria and structures, reporting mechanisms, academy strategic plans,
             academy annual reports on operations and performance, boards of
             visitors’ minutes and reports, and superintendents conference reports. We
             also attended a U.S. Naval Academy Board of Visitors meeting at the Naval
             Academy in December 2002 and a U.S. Military Academy Board of Visitors
             meeting in Washington, D.C., in March 2003. Additionally, we reviewed
             criteria on the principles of effective management, such as those found in
             Standards for Internal Control in the Federal Government.1

             To assess the extent to which academy applicants are granted waivers
             from academic admissions criteria, we interviewed officials from the
             Military, Naval, and Air Force Academies and reviewed documents on
             admissions policies, standards, and practices. We discussed with academy
             officials their execution of the whole person approach, including how they
             assess applicants’ records, the weights applied to the various components
             of the whole person score (academic, leadership, and physical aptitude),
             and the justification for points given to various aspects of an applicant’s
             scores. We also reviewed data from each academy on trends in academic
             admissions scores. During site visits to each academy, we observed the
             evaluation of applicant packages for the incoming class of 2007 by
             academy officials, including how the whole person approach was applied
             for admissions scores. We also observed meetings of senior officials at
             each academy where applicants’ records were evaluated and final
             admissions decisions were made.

             To assess the extent to which admissions and academy performance
             scores differ between various groups of students, we analyzed admissions
             scores and academy performance scores for all students who started at
             the three academies in 1998 and should have graduated in 2002. This
             represented the most recent group of students for which complete data
             were available. We requested and received from each academy a database
             that included data on both admission scores and information about
             students’ performance while attending the academy. We did not



             1
              U.S. General Accounting Office, Standards for Internal Control in the Federal
             Government, GAO/AIMD-00-21.3.1 (Washington, D.C.: November 1999).




             Page 28                                                 GAO-03-1000 Military Education
Appendix I: Scope and Methodology




independently assess data reliability, but we obtained assurances about
data completeness, accuracy, and reliability from academy officials
responsible for maintaining data at each academy. We analyzed these data
separately for each academy since each academy calculated admission
scores or performance scores somewhat differently. We identified six
major groups of students common to all academies: females, minorities,
academy preparatory school graduates, recruited athletes, prior enlisted
personnel, and students whose academic admission scores fell in the
lower 30 percent of the entering class (we chose the latter group in order
to capture information on students whose academic admissions scores
may have been lower than might normally be competitive). Information
specifying a student’s membership in each of these groups was provided in
the databases from the academies. To assess differences, we first
compared the mean performance scores for each group to the overall
mean for each performance measure for the entire class. See appendix II
for details on the results of our analysis of the relationships between
admissions and performance scores.

In addition, we assessed the relationship between admissions scores and
performance at the academies by using the whole person admission score
and the academic component of the admissions score. We estimated the
effects of those scores on four measures of performance for students at
the academies: (1) cumulative grade point average (GPA), (2) cumulative
military performance average (MPA), (3) order of merit (class standing),
and (4) graduation rate. We used cumulative GPA upon graduation as an
indicator of academic performance at the academies and military
performance averages upon graduation as an indicator of military
performance at the academies. Order of merit is a measure of class
standing at each academy that combines academic and military grade
performance and is a final rank for each graduating student. At both the
Air Force Academy and the Naval Academy, order of merit is an actual
class rank number. At the Military Academy, however, order of merit
could range between 0 and 4.0 and was given on the same scale as grade
point averages. For each academy, we analyzed the association of both the
academic component scores and whole person admission scores with
each of the performance scores using regression models. Relationships
between the admissions scores and cumulative GPA, cumulative MPA, and
order of merit were estimated using linear regression models. The
relationships between these two admissions scores and the likelihood of
graduating were estimated using logistic regression models. See appendix
II for more details on the results of those analyses.




Page 29                                       GAO-03-1000 Military Education
Appendix I: Scope and Methodology




Issues related to alleged sexual assaults at the academies fell outside the
scope of our objectives. We conducted our work from October 2002
through May 2003 in accordance with generally accepted government
auditing standards.




Page 30                                         GAO-03-1000 Military Education
                                                            Appendix II: Results of Statistical Analysis of
Appendix II: Results of Statistical Analysis of             Class of 2002 Admissions and Academy
                                                            Performance Scores


Class of 2002 Admissions and Academy
Performance Scores
                                                            This appendix provides the results of our analyses of both admissions and
                                                            performance scores for the class of 2002 at the U.S. Military Academy, the
                                                            U.S. Naval Academy, and the U.S. Air Force Academy.


                                                            We obtained data from all three service academies that included
Admissions and                                              information on admissions scores (academic and whole person),
Performance Scores                                          performance scores while at the academy (cumulative academic grade
                                                            point average, military performance average, and order of merit), attrition
                                                            information where applicable, and various demographic characteristics for
                                                            all students entering each academy in 1998. Table 10 shows the minimum,
                                                            maximum and average admissions and performance scores for students at
                                                            each academy. Table 11 shows graduation rates at each academy.

Table 10: Admissions and Academy Performance Scores for the Class of 2002

                                          Military Academy                               Naval Academy                             Air Force Academy
                                     Average         Min              Max        Average             Min            Max       Average           Min         Max
 Academic                                  600       430              791              618            440           788           3,202       2,492       4,005
 Admissions score
 Whole person score                       6,006     4,587          7,188           65,732         51,651         82,250             798         655          931
 Cum. GPA                                  2.99      1.97             4.19            2.97           2.03           4.00           2.93         2.06        3.97
 Cum. MPA                                  3.28      2.09             3.99            3.12           2.17           3.85           2.90         2.32        3.92
 Order of merit                            3.03      1.30             3.92             489              1           977             469            1         929
Source: GAO analysis, from DOD sources.

                                                            Note: For the U.S. Air Force Academy, an additional step during the selection panel process results in
                                                            a lower whole person score than the component parts.



Table 11: Number of Students Graduating and Graduation Rates for the Class of 2002

                                          Military Academy                          Naval Academy                                Air Force Academy
                                          Number            Percent                      Number          Percent                        Number          Percent
 Graduation rate                              968               78%                           977            80%                             894             74%
Source: GAO analysis, from DOD sources.



                                                            Next, we compared the average admissions scores, performance scores,
                                                            and graduation rates of the six student groups to these overall scores and
                                                            rates. Tables 12, 13, and 14 show the average admission scores and the
                                                            four measures of student performance for the overall sample, and for the
                                                            six student groups, for each of the academies. Because we have data for
                                                            the population of students in this class and there is no sampling error, the



                                                            Page 31                                                         GAO-03-1000 Military Education
                                                             Appendix II: Results of Statistical Analysis of
                                                             Class of 2002 Admissions and Academy
                                                             Performance Scores




                                                             standard error of these estimates are small and differences that could be
                                                             considered small in magnitude may in fact be statistically significant. In
                                                             the tables below, differences that are statistically significant (p<.05) and
                                                             exceed 5 percent are considered meaningful and noted, though such
                                                             differences may not be practically significant when compared with class
                                                             performance requirements overall. For example, at the Naval Academy the
                                                             overall average academic admissions score is 618, 5 percent of 618 is
                                                             about 31. Only those group average academic admissions scores that are
                                                             statistically significant and more than 31 points below 618 are noted with
                                                             an “a.” Differences that are greater than 10 percent are marked with a “b.”

Table 12: Admissions and Performance Scores for the Class of 2002 at the U.S. Military Academy

                                                                                                                                                         Lower 30
                                                                                                                                                        percent of
                                                                                              Prep school           Recruited Prior enlisted           admissions
                                                       Overall   Females Minorities             graduates            athletes    personnel                  class
                                                       (1,246)      (192)         (269)                (184)             (279)               (31)              (377)
                                                                                                              b                a                                       b
 Academic admissions                                      600          603          583                 546               558                594                532
                                                                                                              a
 Whole person admissions score                          6,006       6,022         5,865               5,645              5,814             5,861              5,609a
 Four performance measures
 1. Cumulative GPA                                       2.99          2.99        2.82                 2.61a             2.81               3.14              2.66a
 2. Cumulative MPA                                       3.28          3.26        3.21                 3.26              3.20               3.37               3.21
                                                                                        a                     a                                                        a
 3. Order of merit                                       3.03          3.04        2.86                 2.75              2.90               3.06              2.78
                                                                                        a                                                                              a
 4. Graduation rate                                      78%         76%           71%                  72%               76%                71%               71%
Source: GAO analysis, from Military Academy sources.
                                                             a
                                                              Denotes a group average or percentage with a statistically significant (p<.05) difference greater than
                                                             5% from the overall average or percentage.
                                                             b
                                                              Denotes a group average or percentage with a statistically significant (p<.05) difference greater than
                                                             10% from the overall average or percentage.




                                                             Page 32                                                          GAO-03-1000 Military Education
                                                              Appendix II: Results of Statistical Analysis of
                                                              Class of 2002 Admissions and Academy
                                                              Performance Scores




Table 13: Admissions and Performance Scores for the Class of 2002 at the U.S. Naval Academy

                                                                                                                                                          Lower 30
                                                                                                                                                         percent of
                                                                                               Prep school          Recruited Prior enlisted            admissions
                                                        Overall   Females Minorities             graduates           athletes    personnel                   class
                                                        (1,226)      (190)         (221)                (146)             (380)               (76)              (368)
                                                                                                               b                                  a                     b
 Academic admissions                                       618          624          594                 545                596               570                544
                                                                                                               a                                  a                     a
 Whole Person admissions score                          65,732     65,719         63,769              61,254            64,233            62,256              61,404
 Four performance measures
 1. Cumulative GPA                                        2.97          3.01        2.82a               2.67a              2.86               3.02              2.67a
 2. Cumulative MPA                                        3.12          3.16        3.02                 2.99              3.08               3.19               3.00
                                                                           a             b                     b                b
 3. Order of merit                                         489          456         590                  658               551                453                661b
                                                                           b                                                                                            a
 4. Graduation rate                                       80%        71%            75%                  77%               79%                72%               76%
Source: GAO analysis, from Naval Academy sources.
                                                              a
                                                               Denotes a group average or percentage with a statistically significant (p<.05) difference greater than
                                                              5% from the overall average or percentage.
                                                              b
                                                               Denotes a group average or percentage with a statistically significant (p<.05) difference greater than
                                                              10% from the overall average or percentage.



Table 14: Admissions and Performance Scores for the Class of 2002 at the U.S. Air Force Academy

                                                                                                                                                          Lower 30
                                                                                                                                                         percent of
                                                                                                Prep school          Recruited Prior enlisted           admissions
                                                        Overall   Females Minorities              graduates           athletes    personnel                  class
                                                        (1,216)      (190)         (229)                 (157)             (312)              (44)              (366)
                                                                                                                                                                        b
 Academic admissions                                     3,202       3,216         3,123                3,112             3,043             3,188              2,863
 Whole Person admissions score                             798          805          782                  774                773              792                751a
 Four performance measures
 1. Cumulative GPA                                        2.93          2.97        2.78a                2.61b              2.79             2.89               2.64a
 2. Cumulative MPA                                        2.90          2.93        2.89                  2.83              2.81             2.93                2.84
                                                                                         b                     b                 b                                      b
 3. Order of merit                                         469          440         545                   663               568               499                646
 4. Graduation rate                                       74%         75%           71%                  69%                71%              66%                65%b
Source: GAO analysis, from Air Force Academy sources.
                                                              a
                                                               Denotes a group average or percentage with a statistically significant (p<.05) difference greater than
                                                              5% from the overall average or percentage.
                                                              b
                                                               Denotes a group average or percentage with a statistically significant (p<.05) difference greater than
                                                              10% from the overall average or percentage.




                                                              Page 33                                                          GAO-03-1000 Military Education
                     Appendix II: Results of Statistical Analysis of
                     Class of 2002 Admissions and Academy
                     Performance Scores




                     Regression models were used to assess the relationship between
Relationships        admission scores and performance at the three academies. We used linear
between Admissions   regression models to examine relationships between admission scores and
                     GPA, MPA, and order of merit. To examine the relationship between
and Performance      admission scores and the likelihood of graduating we used a logistic
Scores               regression model. Both the academic admission score and the whole
                     person score were included as independent variables in each model. We
                     estimated separate regression models for each academy. The results of
                     these regressions are shown in tables 15 and 16.

                     The tables show both regression coefficients and standardized
                     coefficients. In general, regression coefficients are interpreted as the
                     predicted change in the dependent variable for every unit change in the
                     independent variables. Here, we have scaled the admissions scores so that
                     the regression coefficients in the table can be interpreted as the predicted
                     change in the relevant measure of success for every 100-point increase in
                     the academic or “whole person” admission score. For example, overall at
                     the U.S. Air Force Academy, for every 100-point increase in the academic
                     admission score we expect to see a 0.06 increase in GPA. For every 100-
                     point increase in the “whole person” score, we expect to see a 0.18
                     increase in GPA. Both relationships are statistically significant, meaning
                     that both the academic score and the “whole person” score are significant
                     predictors of cumulative GPA at the academy.

                     We cannot compare the size of these coefficients across the three
                     academies, though, because the academic and “whole person” scores are
                     on different scales. Because the size of the unstandardized regression
                     coefficients is affected by the scale of the independent variables (the
                     admissions scores), we use standardized regression coefficients to
                     compare them. These appear in parentheses in the tables. To estimate
                     these coefficients, all of the coefficients are standardized by dividing the
                     regression coefficient by the ratio of the standard deviation of the success
                     measure to standard deviation of the admission score. The standardized
                     regression coefficients, therefore, represent the change in the measure of
                     success for each change of one standard deviation in admission scores.
                     Using standardized coefficients, one can conclude that the coefficient that
                     is larger in magnitude has a greater effect on the measures of success.
                     Using the same U.S. Air Force Academy example, we see that while the
                     relationships between both academic and “whole person” scores and GPA
                     are significant, the relationship between academic scores and GPA is
                     actually a stronger one than the relationship between the “whole person”
                     score and GPA. Overall, while the academic scores are often a better
                     predictor of academic performance at the academies (GPA), the “whole


                     Page 34                                           GAO-03-1000 Military Education
                                                               Appendix II: Results of Statistical Analysis of
                                                               Class of 2002 Admissions and Academy
                                                               Performance Scores




                                                               person” scores are often better predictors of military performance (MPA).
                                                               The academic admissions scores have no effect on MPA at the Military and
                                                               Air Force Academies and the whole person scores, not the academic
                                                               admissions scores, predict likelihood of graduating at all three academies.
                                                                                                 2
                                                               We also used the R statistic to estimate how much of the variation in each
                                                               performance score can be explained by both academic and whole person
                                                               admission scores. The admission scores explained about 30 percent of the
                                                               variation in GPAs at both the Naval and Air Force Academies and about 40
                                                               percent of the variation in GPAs at the Military Academy. The admission
                                                               scores explained between a quarter and a third of the variation in order of
                                                               merit across the three academies. However, admission scores did not
                                                               explain as much of the variation in either military performance scores or
                                                               graduation rates. Therefore, while both types of admission scores are
                                                               significant predictors of performance at the academy, they only explain
                                                               between 7 and 40 percent of the variation in performance at the
                                                               academies, and only a very small percentage of the variability in the
                                                               likelihood of graduating. Other factors not studied here, such as the
                                                               military training and academic environment students experience at the
                                                               academies, may contribute to performance more than just students’
                                                               admissions scores do.

Table 15: Regression Coefficients (Standardized Coefficients) from Linear Regression Models Testing Correlations between
Academic and Whole Person Admissions Scores with Cumulative GPA, Cumulative MPA, and Order of Merit for the Class of
2002 at the Service Academies

                                          Cumulative GPA                                Cumulative MPA                                         Order of Merit
                              Academic        Whole person                     Academic        Whole person                 Academic                   Whole person score
                              admission score score                            admission score score                        admission score
 Military Academy                         .42                      .01                -.04                       .03               .18                           .02
 class overall                       (.56)
                                                a
                                                               (.09)                 (-.11)                     (.44)
                                                                                                                        a
                                                                                                                                  (.34)
                                                                                                                                          a
                                                                                                                                                                (.29)
                                                                                                                                                                        a



                                                    R2 = .42                                         R2 = .12                                     R2 = .37
 Naval Academy                            .28                  .002                    .09                      .001             -157.62                        -1.13
 class overall                       (.38)      a
                                                               (.18)                 (.18)   a
                                                                                                                (.19)   a
                                                                                                                                  (-.35)   a
                                                                                                                                                              (-.17) a
                                                     2                                                2                                            2
                                                    R = .30                                          R = .13                                      R = .26
 Air Force Academy                        .06                      .18                -.01                       .21              -27.89                      -147.29
 class overall                       (.38)      a
                                                               (.20)
                                                                         a
                                                                                     (-.14)                     (.37)
                                                                                                                        a
                                                                                                                                  (-.30)
                                                                                                                                           a
                                                                                                                                                              (-.25)
                                                                                                                                                                        a


                                                     2                                                2                                            2
                                                    R = .31                                          R = .07                                      R = .29
Source: GAO analysis, from DOD sources.

                                                               Note: Because of the difference in scales for admissions scores between academies, the size of the
                                                               coefficients cannot be compared across academies.
                                                               a
                                                                   Denotes statistically significant (p<.05) relationships.




                                                               Page 35                                                                GAO-03-1000 Military Education
Appendix II: Results of Statistical Analysis of
Class of 2002 Admissions and Academy
Performance Scores




Table 16: Regression Coefficients (Standardized Coefficients) from Logistic
Regression Models Testing Correlations Between Academic and Whole Person
Admissions Scores and the Likelihood of Graduation for the Class of 2002 at the
Service Academies

                                                                 Graduation
                                          Academic admission score      Whole person score
    Military Academy class                               -.11                     .11
    overall                                             (-.03)                   (.24) a
                                                                   2
                                                                  R = .02
    Naval Academy class                                  -.36                     .01
    overall                                             (-.13)                   (.23) a
                                                                   2
                                                                  R = .01
    Air Force Academy class                              .01                      .75
    overall                                             (.02)                    (.20)
                                                                                         a


                                                                    2
                                                                  R = .03
Source: GAO analysis, from DOD sources.

Note: Because of the difference in scales for admissions scores between
academies, the size of the coefficients cannot be compared across academies.
a
    Denotes statistically significant (p<.05) relationships.




Page 36                                                            GAO-03-1000 Military Education
                   Appendix III: Comments from the Department
Appendix III: Comments from the
                   of Defense



Department of Defense




         Page 37                                           GAO-03-1000 Military Education
          Appendix III: Comments from the Department
          of Defense




Page 38                                           GAO-03-1000 Military Education
                  Appendix IV: GAO Contact and Staff
Appendix IV: GAO Contact and Staff
                  Acknowledgments



Acknowledgments

                  Sandra F. Bell (202) 512-8981
GAO Contact
                  In addition to the individual named above, Gabrielle M. Anderson, Herbert
Staff             I. Dunn, Brian G. Hackett, Joseph W. Kirschbaum, Wendy M. Turenne, and
Acknowledgments   Susan K. Woodward also made key contributions to this report.




                  Page 39                                       GAO-03-1000 Military Education
             Related GAO Products
Related GAO Products


             Military Education: DOD Needs to Align Academy Preparatory Schools’
             Mission Statements with Overall Guidance and Establish Performance
             Goals. GAO-03-1017. Washington, D.C.: September 2003.

             Military Education: Student and Faculty Perceptions of Student Life at
             the Military Academies. GAO-03-1001. Washington, D.C.: September 2003.

             DOD Service Academies: Problems Limit Feasibility of Graduates
             Directly Entering the Reserves. GAO/NSIAD-97-89. Washington, D.C.:
             March 24, 1997.

             DOD Service Academies: Comparison of Honor and Conduct
             Adjudicatory Processes. GAO/NSIAD-95-49. Washington, D.C.: April 25,
             1995.

             DOD Service Academies: Academic Review Processes. GAO/NSIAD-95-57.
             Washington, D.C.: April 5, 1995.

             DOD Service Academies: Update on Extent of Sexual Harassment.
             GAO/NSIAD-95-58. Washington, D.C.: March 31, 1995.

             Coast Guard: Cost for the Naval Academy Preparatory School and Profile
             of Minority Enrollment. GAO/RCED-94-131. Washington, D.C.: April 12,
             1994.

             Military Academy: Gender and Racial Disparities. GAO/NSIAD-94-95.
             Washington, D.C.: March 17, 1994.

             DOD Service Academies: Further Efforts Needed to Eradicate Sexual
             Harassment. GAO/T-NSIAD-94-111. Washington, D.C.: February 3, 1994.

             DOD Service Academies: More Actions Needed to Eliminate Sexual
             Harassment. GAO/NSIAD-94-6. Washington, D.C.: January 31, 1994.

             Academy Preparatory Schools. GAO/NSIAD-94-56R. Washington, D.C.:
             October 5, 1993.

             Air Force Academy: Gender and Racial Disparities. GAO/NSIAD-93-244.
             Washington, D.C.: September 24, 1993.

             Military Education: Information on Service Academies and Schools.
             GAO/NSIAD-93-264BR. Washington, D.C.: September 22, 1993.



             Page 40                                      GAO-03-1000 Military Education
           Related GAO Products




           Naval Academy: Gender and Racial Disparities. GAO/NSIAD-93-54.
           Washington, D.C.: April 30, 1993.

           DOD Service Academies: More Changes Needed to Eliminate Hazing.
           GAO/NSIAD-93-36. Washington, D.C.: November 16, 1992.

           DOD Service Academies: Status Report on Reviews of Student
           Treatment. GAO/T-NSIAD-92-41. Washington, D.C.: June 2, 1992.

           Service Academies: Historical Proportion of New Officers During
           Benchmark Periods. GAO/NSIAD-92-90. Washington, D.C.: March 19, 1992.

           DOD Service Academies: Academy Preparatory Schools Need a Clearer
           Mission and Better Oversight. GAO/NSIAD-92-57. Washington, D.C.:
           March 13, 1992.

           Naval Academy: Low Grades in Electrical Engineering Courses Surface
           Broader Issues. GAO/NSIAD-91-187. Washington, D.C.: July 22, 1991.

           DOD Service Academies: Improved Cost and Performance Monitoring
           Needed. GAO/NSIAD-91-79. Washington, D.C.: July 16, 1991.

           Review of the Cost and Operations of DOD’s Service Academies. GAO/T-
           NSIAD-90-28. Washington, D.C.: April 4, 1990.




(350270)
           Page 41                                     GAO-03-1000 Military Education
                         The General Accounting Office, the audit, evaluation and investigative arm of
GAO’s Mission            Congress, exists to support Congress in meeting its constitutional responsibilities
                         and to help improve the performance and accountability of the federal
                         government for the American people. GAO examines the use of public funds;
                         evaluates federal programs and policies; and provides analyses,
                         recommendations, and other assistance to help Congress make informed
                         oversight, policy, and funding decisions. GAO’s commitment to good government
                         is reflected in its core values of accountability, integrity, and reliability.


                         The fastest and easiest way to obtain copies of GAO documents at no cost is
Obtaining Copies of      through the Internet. GAO’s Web site (www.gao.gov) contains abstracts and full-
GAO Reports and          text files of current reports and testimony and an expanding archive of older
                         products. The Web site features a search engine to help you locate documents
Testimony                using key words and phrases. You can print these documents in their entirety,
                         including charts and other graphics.
                         Each day, GAO issues a list of newly released reports, testimony, and
                         correspondence. GAO posts this list, known as “Today’s Reports,” on its Web site
                         daily. The list contains links to the full-text document files. To have GAO e-mail
                         this list to you every afternoon, go to www.gao.gov and select “Subscribe to e-mail
                         alerts” under the “Order GAO Products” heading.


Order by Mail or Phone   The first copy of each printed report is free. Additional copies are $2 each. A
                         check or money order should be made out to the Superintendent of Documents.
                         GAO also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a
                         single address are discounted 25 percent. Orders should be sent to:
                         U.S. General Accounting Office
                         441 G Street NW, Room LM
                         Washington, D.C. 20548
                         To order by Phone:     Voice:    (202) 512-6000
                                                TDD:      (202) 512-2537
                                                Fax:      (202) 512-6061


                         Contact:
To Report Fraud,
                         Web site: www.gao.gov/fraudnet/fraudnet.htm
Waste, and Abuse in      E-mail: fraudnet@gao.gov
Federal Programs         Automated answering system: (800) 424-5454 or (202) 512-7470


                         Jeff Nelligan, Managing Director, NelliganJ@gao.gov (202) 512-4800
Public Affairs           U.S. General Accounting Office, 441 G Street NW, Room 7149
                         Washington, D.C. 20548