oversight

Military Transformation: Progress and Challenges for DOD's Advanced Distributed Learning Programs

Published by the Government Accountability Office on 2003-02-28.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                United States General Accounting Office

GAO             Report to Congressional Committees




February 2003
                MILITARY
                TRANSFORMATION
                Progress and
                Challenges for DOD's
                Advanced Distributed
                Learning Programs




GAO-03-393
                                                 February 2003


                                                 MILITARY TRANSFORMATION

                                                 Progress and Challenges for DOD's
 Highlights of GAO-03-393, a report to
                                                 Advanced Distributed Learning Programs
 Congressional Committees




The Department of Defense (DOD)                  DOD has set high expectations for ADL. They expect the programs to provide
spends more than $17 billion                     new learning opportunities and technologies across a wide range of training
annually for military schools that               areas. Ultimately, a key benefit of ADL is expected to be improved readiness
offer nearly 30,000 military training            through reengineering of training and enhancing service members’ skills.
courses to almost 3 million military
personnel and DOD civilians. DOD                 DOD, the services, and Joint Staff are generally in the early stages of
is transforming its forces, including            implementing their ADL programs and have made progress in several areas.
the way it trains, to favor more                 OSD, with its three ADL co-laboratories; the services; and the Joint Staff chose
rapid and responsive deployment.                 an industry-wide ADL standard for content interoperability and collaboration
DOD’s training transformation                    across the services. They promoted experimentation with new technology and
strategy emphasizes the use of                   working with private industry. The services’ programs generally focus on
advanced distributed learning                    distribution infrastructure and service-specific content development. According
(ADL) programs, such as Internet-                to ADL program officials, OSD, the Joint Staff, and the services have achieved
based training, as critical to
                                                 some ADL successes. For example, OSD, in collaboration with the co-
achieving its training and
                                                 laboratories, developed successful course content prototypes; and the Army’s
overarching transformation goals.
                                                 Battle Staff Noncommissioned Officer course resulted in annual savings while
ADL is instruction that does not
require an instructor’s presence;                maintaining student performance. However, it is too early to fully assess the
can use more than one media; and                 extent of each program’s effectiveness.
emphasizes the use of reusable
content, networks, and learning                  DOD faces cultural, technological, policy and financial challenges that affect the
management systems.                              ADL programs’ ability to fully achieve the benefits of enhanced learning and
                                                 performance and of improved readiness. Key challenges are summarized below.
Because of ADL’s importance to
DOD’s transformation efforts and                 Challenges Affecting DOD’s ADL Programs
pursuant to GAO’s basic legislative              Challenge           Description
responsibilities, we initiated this              Cultural            •   Organizational culture is resistant to change.
review to create a baseline                                          •   Senior leadership commitment varies: preference is for the more
                                                                         traditional schoolhouse-focused learning.
document that describes the status
                                                                     •   Service schoolhouses are reluctant to change since funding and
of DOD’s ADL programs. GAO                                               infrastructure are closely tied to numbers of in-resident students.
reviewed these programs to                       Technological       •   Bandwidth issues and unresolved network security concerns stifle utility.
determine (1) DOD’s expectations                                     •   The development, fielding of, and access to military skills-related content
for the programs; (2) the                                                is more difficult and costly than anticipated.
implementation status of those                   Policy              •   OSD is in the early stage of formulating policy that specifically addresses
programs; and (3) major challenges                                       the use of ADL, consequently some of the military services’ and ’s training
                                                                         and education regulations are outdated and awaiting a definitive policy.
affecting program implementation.                Financial           •   Budget and funding issues for the long-term use of ADL are unresolved.
GAO did not assess the programs’                                     •   Allocated funds—around $431 million, less than 1.3 percent of its training
effectiveness at this time because                                       budget for fiscal years 1999 through 2002—did not always meet
most are in the early stages of                                          requirements.
implementation.                                                      •   Projected program needs—$2.2 billion for fiscal years 2003 through
                                                                         2007—is about $600 million more than currently programmed.
DOD reviewed a draft of this report              Source: DOD.
and concurred with its contents.
                                                 Note: GAO analysis of OSD, Joint Staff and service data.


 www.gao.gov/cgi-bin/getrpt?GAO-03-393.

 To view the full report, including the scope
 and methodology, click on the link above.
 For more information, contact Neal P. Curtin,
 at (757) 552-8100 or e-mail curtinn@gao.gov.
Contents


Letter                                                                              1



Briefing Section I      Background on DOD’s Advanced Distributed
                        Learning Programs                                          12



Briefing Section II     DOD’s Expectations for Advanced Distributed
                        Learning Programs                                          14




Briefing Section III    Implementation Status of DOD’s Advanced
                        Distributed Learning Programs                              18



Briefing Section IV     Major Challenges Affecting DOD’s Advanced
                        Distributed Learning Programs                              28



Briefing Section V      Conclusions                                                37



Briefing Section VI     Army’s Advanced Distributed Learning Programs              38



Briefing Section VII    Navy’s Advanced Distributed Learning Programs              44



Briefing Section VIII   Marine Corps’ Advanced Distributed Learning
                        Programs                                                   48




                        Page i                          GAO-03-393 DOD's ADL Programs
Briefing Section IX   Air Force’s Advanced Distributed Learning
                      Programs                                                             52



Briefing Section X    Joint Staff’s Advanced Distributed Learning Programs 56


Appendix I            Scope and Methodology                                                60



Appendix II           Timeline of Key Events, Directives and Guidance for
                      DOD’s ADL Programs                                                   62



Appendix III          Comments from the Department of Defense                              63




                      Abbreviations

                      ADL         Advanced Distributed Learning
                      AEC         Automated Electronic Classrooms
                      AETC        U.S. Air Force Air Education and Training Command
                      AFIADL      Air Force Institute for Advanced Distributed Learning
                      BA3         Budget Activity 3
                      CBT         Computer Based Training
                      CNET        U.S. Navy Chief of Naval Education and Training
                      C4I         Command, Control, Communications, Computers and
                                   Intellegence
                      DAU         Defense Acquisition University
                      DL          Distance Learning
                      DLRC        Deployable Learning Resource Centers
                      DOD         Department of Defense
                      DTF         Digital Training Facilities
                      DTTP        Distributive Training Technology Project


                      Page ii                                   GAO-03-393 DOD's ADL Programs
DUSD®             Deputy Under Secretary of Defense for Readiness
ECP               Extension Course Program
ETSC              Education and Training Steering Committee
IRR               Individual Ready Reserve
ITV               Interactive Television
JCLE              Joint Collaborative Learning Environment
JPME II           Joint Professional Military Education II
LMS               Learning Management System
MC                Marine Corps
MCDLP             Marien Corps Distance Learning Program
MOS               Military Occupational Specialty
NATO              North Atlantic Treaty Organization
NCO               Noncommissioned Officer
NIPRNET           Non-Classified Internet Protocol Router Network
NG                National Guard
NGB               National Guard Bureau
NMCI              Navy and Marine Corps Intranet
NSIAD             National Security and International Affairs Division
O&M               Operations and Maintenance
OSD               Office of the Secretary of Defense
PME               Professional Military Education
POM               Program Objective Memorandum
QDR               Quadrennial Defense Review
R,D,T&E           Research, Development, Test and Evaluation
SCORM             Sharable Content Object Reference Model
SIPRNET           Secret Internet Protocol Router Network
TADLP             The Army Distributed Learning Program
TFADLAT           Total force Advanced Distributed Learning Action Team
TRADOC            U.S. Army Training and Doctrine Command
VTC               Video Teleconference
VTT               Video Teletraining




This is a work of the U.S. Government and is not subject to copyright protection in the
United States. It may be reproduced and distributed in its entirety without further
permission from GAO. It may contain copyrighted graphics, images or other materials.
Permission from the copyright holder may be necessary should you wish to reproduce
copyrighted materials separately from GAO’s product.




Page iii                                                GAO-03-393 DOD's ADL Programs
United States General Accounting Office
Washington, DC 20548




                                   February 28, 2003

                                   The Honorable John Ensign
                                   Chairman
                                   The Honorable Daniel K. Akaka
                                   Ranking Minority Member
                                   Subcommittee on Readiness and Management Support
                                   Committee on Armed Services
                                   United States Senate

                                   The Honorable Joel Hefley
                                   Chairman
                                   The Honorable Solomon P. Ortiz
                                   Ranking Minority Member
                                   Subcommittee on Readiness
                                   Committee on Armed Services
                                   House of Representatives

                                   The Department of Defense (DOD) spends more than $17 billion1 annually
                                   for military schools that offer nearly 30,000 military training courses to
                                   almost 3 million military personnel and DOD civilians, much of it to
                                   maintain readiness. 2 To better meet the diverse defense challenges of the
                                   future, DOD is transforming its forces, including its training, for a post-
                                   Cold War environment that favors more rapid deployment and
                                   responsiveness. DOD’s Training Transformation Strategy3 emphasizes the
                                   use of advanced distributed learning (ADL) programs such as Internet-
                                   based training, as critical to achieving the department’s training and
                                   overarching transformation goals and to deliver the highest quality training
                                   cost-effectively anytime, anywhere, whether active duty, reserve, or
                                   civilian personnel. ADL is instruction that does not require an instructor’s



                                   1
                                    This amount includes the cost of conducting school training, including instructor’s pay;
                                   classroom availability and operation; course development; and student’s military pay,
                                   billeting cost, and temporary duty costs.
                                   2
                                    Generally, formal military training and education occurs at centralized training facilities
                                   and lasts weeks or months.
                                   3
                                    Office of the Under Secretary of Defense for Personnel and Readiness, Strategic Plan for
                                   Transforming DOD Training, March 1, 2002. In this plan the definition of “training” is
                                   expanded to include training, education, and job performance aiding. OSD’s training
                                   transformation implementation plan should be completed by March 2003.



                                   Page 1                                                    GAO-03-393 DOD's ADL Programs
             presence; can use more than one media; and emphasizes the use of
             reusable content, networks, and learning management systems.4

             We initiated this review of DOD’s ADL programs, pursuant to our basic
             legislative responsibilities, because of the importance DOD has placed on
             them as a key to achieving the department’s transformation efforts.
             Specifically, we addressed the following questions: (1) What are DOD’s
             expectations for the programs? (2) How is DOD managing ADL and what
             progress is being made in implementing the programs? (3) What major
             challenges are affecting the programs’ implementation? We did not assess
             the effectiveness of the programs at this time because most are in the early
             stages of implementation; thus our objective was to provide a baseline
             document concerning the focus, status, and magnitude of DOD’s ADL
             programs.

             In late August and early September 2002, because of your continuing
             interest in the readiness and training of U.S. armed forces, we briefed your
             offices and those of Representatives John McHugh and Adam Smith on the
             results of our work. This report summarizes and updates the major
             observations provided at our briefings. (See briefing sections I through X.)

             We conducted our review from February 2002 through August 2002 in
             accordance with generally accepted government auditing standards.
             Appendix I describes our scope and methodology.


             The increased rate of deployments in recent years of DOD’s forces, which
Background   often involve rapid, unplanned movements to locations around the world,
             highlights the need for the services to provide training on demand to
             soldiers and units deployed worldwide. Accordingly, because of more
             demanding deployment criteria and other time-sensitive constraints, DOD
             recognized that yesterday’s framework “right time, right place” learning,
             with its use of set times and places for training, may not meet future
             military requirements. It also recognizes that providing “anytime,
             anywhere” instruction is essential to maintaining military readiness in the
             information age, where future forces and their support activities need to
             be highly adaptive to meet threats effectively and rapidly.


             4
              Reusable content includes, but is not limited to, courseware, tutorials, and case studies;
             networks are Intra- or Internet based; and learning management systems are operating
             systems that provide access to “content objects” and help register, track, and administer
             courses to a given student population.




             Page 2                                                    GAO-03-393 DOD's ADL Programs
In response to the DOD 1997 Quadrennial Defense Review,5 the
department developed a DOD-wide strategy to use learning and
information technologies to modernize education and training. The initial
effort in that development was the ADL Initiative. Its intent was to set
forth a new framework to provide DOD personnel access to high quality
education and training, tailored to individual needs and delivered cost-
effectively, whenever and wherever it is required. DOD envisioned using
the Internet and other virtual or private wide-area networks, distributed
learning experts, learning management, and diverse support tools to
ensure a “learner-centric” ADL system that delivers high quality training,
education, and job performance aiding. DOD sees ADL programs as part of
a continuum6 of learning that encompasses many learning methodologies,
as shown in table 1.




5
 William S. Cohen, Secretary of Defense, Report of the Quadrennial Defense Review, May
1997.
6
 A continuum is defined as a whole characterized as a collection, sequence, or progression
of elements varying by minute degrees.




Page 3                                                  GAO-03-393 DOD's ADL Programs
Table 1: Continuum of Learning Methods

                    Right time, right place                          Anytime, anywhere
    Classroom               Distance/distributed             Advanced distributed
    delivery method         learning delivery methods        learning delivery methods
    •Instructor-led         •Video tele-training             •Integrated networked systems
    training                •Embedded training               •Integrated platforms
                            •Computer conferencing           •Reusable learning objects
                            •Interactive television          •Widespread collaboration
                            •Electronic classrooms           •Global knowledge databases
                            •Interactive multimedia          •Intelligent tutoring systems
                            •Computer-based training         •Performance aiding
                            •Audio-graphics                  •Digital knowledge repositories
                            •Audiotapes/videotapes           •Internet-based instruction
                            •Correspondence courses          •Virtual libraries
                                                             •Simulations
                                                             •Virtual classrooms

Source: Defense Acquisition University.

Note: The data displayed in the table is based on data provided in the Defense Acquisition
University’s Strategic Plan 2002-2009 Training Transformation (T2), The DAU Road Map for e-
Learning and On-line Performance Support.


In April 1999, DOD issued its ADL strategy7 in response to the 1997 DOD
Quadrennial Defense Review. The strategy also responded to (1) the
directive in the National Defense Authorization Act for Fiscal Year 19998
for DOD to develop a strategic plan to guide and expand distributed
learning initiatives and (2) Executive Order 13,1119 that tasked DOD to
provide guidance to Defense agencies and advise civilian agencies in
developing and implementing collaborative distance learning standards.
DOD’s strategic plan defined ADL as a way to leverage the power of
computer, information, and communication technologies through the use
of common standards in order to provide learning that can be tailored to
individual needs and delivered anytime, anywhere, in either training or
education environments. It also includes establishing an interoperable
“computer-managed instruction” environment to support the needs of
developers, learners, instructors, administrators, managers, and family. An


7
 Office of the Under Secretary of Defense for Personnel and Readiness, Report to the 106th
Congress, Department of Defense Strategic Plan for Advanced Distributed Learning, Apr.
30, 1999.
8
 Public Law 105-261, sec. 378, Oct. 17, 1998.
9
Exec. Order 13,111, Using Technology to Improve Training Opportunities for Federal
Government Employees, sec. 4 (c), Jan. 12, 1999.




Page 4                                                      GAO-03-393 DOD's ADL Programs
          ADL implementation plan followed in May 2000 to provide a federal
          framework. It described the department’s approach to carrying out its
          strategic plan and provided an update on each of the services’ and the
          Joint Staff’s programs. 10 Since 1995, OSD, the services, and the Joint Staff
          have established ADL programs in concert with key executive,
          congressional, and departmental guidance discussed above. See appendix
          II for a timeline of key events.

          OSD’s March 2002 Training Transformation Strategy emphasizes the use of
          ADL programs as critical to achieving the department’s training and
          overarching transformation goals and ensuring that training is readily
          available to both active and reserve military personnel, regardless of time
          and place. The training transformation strategy and soon to be released
          implementation plan are intended to reengineer training; enhance service
          members’ skills; and provide capabilities-based training to support service,
          joint, interagency, intergovernmental, and multinational operations.


          Officials from OSD, the services, and the Joint Staff have set high
Summary   expectations for ADL. They expect the programs, which include the
          various delivery methods cited in table 1, to provide new learning
          opportunities and technologies and improved readiness. In terms of new
          learning opportunities and technologies, DOD expects

          •    increased accessibility to training for personnel,

          •    interoperability of instruction components in varied locations by
               different services,

          •    reusability in multiple applications,

          •    durability, despite changes in technology, and

          •    affordability.




          10
           Office of the Deputy Under Secretary of Defense (Readiness), Director for Readiness and
          Training, Department of Defense Implementation Plan for Advanced Distributed
          Learning, May 19, 2000.




          Page 5                                                GAO-03-393 DOD's ADL Programs
With regard to improved readiness, DOD expects ADL to improve
readiness by

•   supporting the training transformation initiative and the combatant
    commanders,

•   enhancing training opportunities for joint assignments,

•   enhancing training opportunities for reserve personnel,

•   improving mission performance through anytime, anywhere, and just-
    in-time assignment-oriented and job performance enhancement
    training, and

•   improving manning by reducing personnel’s nonavailability and unit
    turbulence and reducing time for in-resident training with large return-
    on-investment for temporary duty costs, while increasing retention and
    quality-of-life enrichment. (See briefing section II.)

OSD, the services, and the Joint Staff are generally in the early stages of
implementing their ADL programs and have made progress in several
areas. OSD’s Office of the Deputy Under Secretary of Defense for
Readiness provides executive policy and programmatic oversight and
guidance for the department’s ADL implementation. That office also leads
a collaborative effort to produce ADL policy, plans, and procedures for
developing and implementing ADL technologies across the department.
This collaboration involves the services, Joint Staff, other DOD
components, the ADL collaborative laboratories (co-labs), the Coast
Guard, and the Department of Labor. For example, OSD in collaboration
with its partners, chose an industry-wide ADL standard for content
interoperability to be used throughout DOD, which allows for
collaboration of course content across the services. The standard,
Sharable Content Object Reference Model (SCORM), is an evolving set of
technical specifications designed to ensure the interoperability,
accessibility, and reusability of on-line courseware. The Joint Staff and the
services agree that future course content will be designed to conform to
SCORM. OSD, with the National Guard Bureau (NGB) and the Department
of Labor, established three ADL co-labs to experiment with new
technology and leverage experience between private industry and military
components. It also participates in an international partnership co-lab in
Telford, England, to promote collaboration and global e-learning. The
services’ and Joint Staff’s programs—individual programs designed by and
tailored for the specific needs of each service or joint position—share a



Page 6                                         GAO-03-393 DOD's ADL Programs
similar vision of providing learner-centric (i.e., on demand, “anytime,
anywhere”) training and focus on, among other aspects, distribution
infrastructure and service-specific content development. (See briefing
section III.)

OSD, the Joint Staff, and the services note that they have achieved some
ADL successes, such as the following:

•    OSD—with the co-labs, military services, Joint Staff, and co-sponsors—
     developed successful content prototypes, including one joint
     professional military education course.11

•    The Joint Staff’s Joint Collaborative Learning Environment prototype
     established an initial joint personnel tracking and portal capability.

•    The Army’s Battle Staff Noncommissioned Officer course conversion to
     an ADL format resulted in a $2.9 million annual cost avoidance while
     maintaining student performance.

•    The Navy—to promote interoperability, ease of access to DOD Internet
     sites, and reduce training time—established both .mil and .com access
     to ADL courses.

•    The Marine Corps’ distance learning application in terrorism awareness
     reduced training time from 11 hours to 6 hours and increased the
     average exam scores by 7 percentage points.

•    The Air Force developed CD-ROM training for hazardous material
     incident response for DOD firefighters and law enforcement personnel
     that reportedly resulted in a significant increase of certified responders
     and a projected $16.6 million cost avoidance.

Additionally, the Defense Acquisition University’s (DAU) ADL program is
cited by DOD ADL program officials as a success and an example of “best
practices.” 12 According to university officials, since 1998 on-line



11
 Joint professional military education is a Joint Chief of Staff-approved body of objectives,
policies, procedures, and standards supporting the educational requirements for joint
officer development.
12
  Defense Acquisition University, the “corporate university” for DOD, provides the
acquisition, technology, and logistics community with learning products and services. Its
distance learning program currently provides 19 on-line courses.




Page 7                                                   GAO-03-393 DOD's ADL Programs
                instructional time increased from 15,750 hours to 1.4 million hours;
                graduates attending on-line training courses increased 38 percent; and the
                on-line program management curriculum reduced annual student training
                weeks from 36,120 to 10,000—a real savings of 300 annual work years or
                $17.4 million. The university’s program was awarded the U.S. Distance
                Learning Association Award for Excellence in Government in 2001 and
                2002 for the quality of its on-line offerings.

                A number of cultural, technological, policy, and financial challenges affect
                OSD’s, the services’ and the Joint Staff’s ability to execute programs that
                achieve the attainable benefits of enhanced learning and performance and
                improved readiness in concert with DOD’s ADL vision and training
                transformation strategy. According to DOD officials, there is a strong
                interrelationship among the challenges and that a solution for one
                challenge may have an impact on the others.

Cultural        A major cultural barrier, according to DOD ADL program officials, is the
                varying level of commitment of senior military and civilian leadership in
                the military. The consensus view of the ADL program officials we
                contacted was that not all senior military and civilian leadership is
                committed to ADL, preferring the traditional, schoolhouse-focused
                approach to learning. Hesitance to embrace ADL is also explained as a
                function of less familiarity and comfort by senior officials with computers,
                advanced technologies, and emerging policies. Similarly, ADL program
                officials told us that the military services’ schoolhouses are reluctant to
                change, in large part because their funding and infrastructure are tied so
                closely to the number of students actually trained on-site.

Technological   According to DOD officials, the services are all moving toward Web- or
                Internet-based access to course content in support of DOD’s vision of
                “anytime, anywhere” delivery of training. The officials stated that much
                progress has been made to enable this type of access. However, according
                to OSD and service officials, bandwidth is generally insufficient to support
                interactive, multimedia learning content and simulations; and unresolved
                network security concerns stifle utility. For example, we recently reported




                Page 8                                         GAO-03-393 DOD's ADL Programs
         that the National Guard Bureau cannot ensure that GuardNet13 will
         perform as intended or provide its users with reliable and secure services
         because the requirements, configuration, and security processes for
         managing the network are ineffective. 14 DOD ADL officials acknowledge
         the same issue exists throughout DOD. Perhaps more significantly, the
         development of, fielding of, and access15 to military skills-related course
         content that could most positively impact readiness continue to be more
         difficult than anticipated, leading to higher costs and slower content
         availability than forecasted.

Policy   Some of DOD’s training policies are obsolete; consequently, some of the
         military services’ training regulations do not reflect the availability or use
         of new ADL technologies. 16 For example, according to DOD officials, DOD
         is in the early stage of formulating policy that specifically addresses the
         use of ADL. DOD officials believe that without an OSD-specific ADL
         policy, many of DOD’s policies and guidance documents will require
         updating, so as to provide a requirement for the military service’s in turn,
         to update their training and education regulations that address the use of
         ADL. Also, the Army’s primary training regulation17 has been awaiting




         13
           National Guard Bureau’s GuardNet, the NGB’s wide-area network, was initially
         established to support Web-based distance learning for its units in the states, the U.S.
         territories, and the District of Columbia. GuardNet, a network of interconnected federal
         and state military networks across the United States, can connect to a defense network
         operated by the Defense Information Systems Agency, and through this network to the
         Internet. GuardNet has recently been used to support homeland security activities such as
         emergency command and control functions, airport security activities coordination, and
         public service announcements.
         14
          U.S. General Accounting Office, National Guard: Effective Management Processes
         Needed for Wide-Area Network, GAO-02-959 (Washington, D.C.: Sept. 24, 2002).
         15
          For this report, access refers to the availability and ability to access computer hardware,
         sufficient bandwidth to support multimedia, interactive course content, and/or available
         duty time to accomplish ADL.
         16
           DOD policies and regulations include, but may not be limited to, Department of Defense
         Directive 1200.16, Contracted Civilian-Acquired Training (CCAT) for Reserve
         Components, May 30,1990; Department of Defense Directive 1322.18, Military Training,
         Jan. 9, 1987; Department of Defense Directive 1430.13, Training Simulators and Devices,
         Aug. 22, 1986; Department of Defense Directive 8320.1, DOD Data Administration, Sept.
         26, 1991; Department of Defense Directive 8000.1, Management of DOD Information
         Resources and Information Technology, Feb. 27, 2002; and Department of Defense
         Instruction 5200.40, DOD Information Technology Security Certification and
         Accreditation Process, Dec. 30, 1997.
         17
          Department of the Army, Army Regulation 350-1, Army Training, Aug. 1, 1983.



         Page 9                                                   GAO-03-393 DOD's ADL Programs
                  revision for 3 years, in part, due to a lack of consensus on integrating new
                  technologies, including ADL, with traditional training approaches.

Financial         Funding and budgeting issues similar to those we reported for DOD’s
                  distance learning programs in 1997 remain unresolved.18 Funding
                  allocations of more than $431 million for fiscal years 1999 through 2002
                  (less than 1.3 percent of its training budget during that period) did not
                  always meet program requirements, which were difficult to determine for
                  a new program where standards were evolving and the technology
                  changing rapidly. It is not likely that planned funding levels will meet
                  future expected requirements. DOD program officials project that over
                  $2.2 billion will be needed for ADL programs through fiscal year 2007 but
                  currently have programmed about $1.6 billiona more than $600 million
                  funding gap. Furthermore, according to DOD program officials, in some
                  cases, anticipated training savings attributable to ADL implementation
                  were removed from the budget as savings before they were realized.
                  According to service officials, some training facility commanders continue
                  to be concerned that ADL will reduce their resources because of the
                  decrease in the number of students receiving traditional schoolhouse
                  training. Finally, the Joint Staff and the services are still considering how
                  to budget for the long-term use of ADL. (See briefing section IV.)


                  The Deputy Under Secretary of Defense (Readiness) provided written
Agency Comments   comments on a draft of this report, which are reprinted in their entirety in
                  appendix III. In its comments, DOD concurred with the content of the
                  report. DOD also provided technical comments to the draft, which we
                  have incorporated as appropriate.




                  18
                    U.S. General Accounting Office, Distance Learning: Opportunities Exist for DOD to
                  Capitalize on Services’ Efforts, GAO/NSIAD-98-63R (Washington, D.C.: Dec. 18, 1997). We
                  reported that the resolution of funding and budgeting issues would benefit the services’
                  distance learning initiatives. These issues are the (1) extent of investment that will be
                  needed to convert selected courses and delivery infrastructures; (2) dollar savings that can
                  be realized; (3) impact on the current training infrastructure, in terms of requirements for
                  instructors, training developers, training equipment, course maintenance, and training
                  facility operations; and (4) process for budgeting for long-term use of distance learning.
                  Distance learning is structured training that can take place almost anywhere and anytime
                  without the physical presence of an instructor and may use one or more media but, unlike
                  ADL, does not emphasize the use of reusable objects, networks, and learning management
                  systems.




                  Page 10                                                  GAO-03-393 DOD's ADL Programs
We are sending copies of this report to Representatives John McHugh and
Adam Smith and other congressional members as appropriate. We will
also send copies to the Secretary of Defense; the Secretaries of the Army,
the Navy, and the Air Force; and the Commandant of the Marine Corps.
We will make copies available to others on request. In addition, the report
will be available at no charge on the GAO Web site at http://www.gao.gov .

If you or your staff have any questions, please call me on (757) 552-8100 or
Clifton Spruill, Assistant Director, on (202) 512-4531. Major contributors to
this report were Claudia Dickey, Arnett Sanders, James Walker, M. Jane
Hunt, Susan Woodward, and Scott Gannon.




Neal P. Curtin
Director, Defense Capabilities
 and Management




Page 11                                        GAO-03-393 DOD's ADL Programs
              Briefing Section I: Background on DOD’s
Briefing Section I: Background on DOD’s
              Advanced Distributed Learning Programs



Advanced Distributed Learning Programs




              Page 12                                   GAO-03-393 DOD's ADL Programs
                                                               Briefing Section I: Background on DOD’s
                                                               Advanced Distributed Learning Programs




Source: Washington Headquarters Service Directorate for Information Operations and Reports and Defense Manpower Data Center.

                                                               Notes:       Data is as of Apr. 2002.

                                                                            Reserve Component numbers include Selective Reserve Personnel,
                                                                            Individual Ready Reserve (IRR) and Standby Reserve personnel.




                                                               Page 13                                                         GAO-03-393 DOD's ADL Programs
               Briefing Section II: DOD’s Expectations for
Briefing Section II: DOD’s Expectations for
               Advanced Distributed Learning Programs



Advanced Distributed Learning Programs




               Page 14                                       GAO-03-393 DOD's ADL Programs
Briefing Section II: DOD’s Expectations for
Advanced Distributed Learning Programs




Page 15                                       GAO-03-393 DOD's ADL Programs
Briefing Section II: DOD’s Expectations for
Advanced Distributed Learning Programs




Page 16                                       GAO-03-393 DOD's ADL Programs
Briefing Section II: DOD’s Expectations for
Advanced Distributed Learning Programs




Page 17                                       GAO-03-393 DOD's ADL Programs
               Briefing Section III: Implementation Status of
Briefing Section III: Implementation Status of
               DOD’s Advanced Distributed Learning
               Programs


DOD’s Advanced Distributed Learning
Programs




               Page 18                                          GAO-03-393 DOD's ADL Programs
Briefing Section III: Implementation Status of
DOD’s Advanced Distributed Learning
Programs




Page 19                                          GAO-03-393 DOD's ADL Programs
Briefing Section III: Implementation Status of
DOD’s Advanced Distributed Learning
Programs




Page 20                                          GAO-03-393 DOD's ADL Programs
Briefing Section III: Implementation Status of
DOD’s Advanced Distributed Learning
Programs




Page 21                                          GAO-03-393 DOD's ADL Programs
Briefing Section III: Implementation Status of
DOD’s Advanced Distributed Learning
Programs




Page 22                                          GAO-03-393 DOD's ADL Programs
               Briefing Section III: Implementation Status of
               DOD’s Advanced Distributed Learning
               Programs




Source: DOD.

               Note:     GAO analysis of OSD, Joint Staff, and military service data.

                         The Joint Staff reportedly added $650,000 per year to its fiscal years 2003 through
                         2007 POM after we completed our audit work.




               Page 23                                                        GAO-03-393 DOD's ADL Programs
Briefing Section III: Implementation Status of
DOD’s Advanced Distributed Learning
Programs




Page 24                                          GAO-03-393 DOD's ADL Programs
Briefing Section III: Implementation Status of
DOD’s Advanced Distributed Learning
Programs




Page 25                                          GAO-03-393 DOD's ADL Programs
Briefing Section III: Implementation Status of
DOD’s Advanced Distributed Learning
Programs




Page 26                                          GAO-03-393 DOD's ADL Programs
Briefing Section III: Implementation Status of
DOD’s Advanced Distributed Learning
Programs




Page 27                                          GAO-03-393 DOD's ADL Programs
              Briefing Section IV: Major Challenges
Briefing Section IV: Major Challenges
              Affecting DOD’s Advanced Distributed
              Learning Programs


Affecting DOD’s Advanced Distributed
Learning Programs




              Page 28                                 GAO-03-393 DOD's ADL Programs
Briefing Section IV: Major Challenges
Affecting DOD’s Advanced Distributed
Learning Programs




Page 29                                 GAO-03-393 DOD's ADL Programs
Briefing Section IV: Major Challenges
Affecting DOD’s Advanced Distributed
Learning Programs




Page 30                                 GAO-03-393 DOD's ADL Programs
Briefing Section IV: Major Challenges
Affecting DOD’s Advanced Distributed
Learning Programs




Page 31                                 GAO-03-393 DOD's ADL Programs
Briefing Section IV: Major Challenges
Affecting DOD’s Advanced Distributed
Learning Programs




Page 32                                 GAO-03-393 DOD's ADL Programs
               Briefing Section IV: Major Challenges
               Affecting DOD’s Advanced Distributed
               Learning Programs




Source: DOD.

               Notes:    GAO analysis of OSD, Joint Staff and the military service budgetary data.

                         Total Training amount includes all component O&M training funding allocated as reported
                         in the DOD budget for Budget Activity 3 (BA3) for the indicated fiscal years. Budget Activity
                         3 funds all training and recruiting programs.

                         Reserve component funding is included within the active duty component totals.




               Page 33                                                        GAO-03-393 DOD's ADL Programs
               Briefing Section IV: Major Challenges
               Affecting DOD’s Advanced Distributed
               Learning Programs




Source: DOD.

               Notes:    GAO’s analysis of OSD, Joint Staff, and military service budgetary data.

                         Reserve component funding amounts were included with the active component
                         funding data.




               Page 34                                                       GAO-03-393 DOD's ADL Programs
Briefing Section IV: Major Challenges
Affecting DOD’s Advanced Distributed
Learning Programs




Page 35                                 GAO-03-393 DOD's ADL Programs
               Briefing Section IV: Major Challenges
               Affecting DOD’s Advanced Distributed
               Learning Programs




Source: DOD.

               Notes:    GAO analysis of OSD, military service, and Joint Staff budgetary data.

                         The DOD bars reflect the total requirements of the services, OSD and Joint Staff.

                         Total requirements include both infrastructure and content requirements.

                         Joint Staff requirements are included in the “OSD and Joint Staff” total because Joint Staff
                          receives, funding for ADL projects from the funds allocated to OSD for ADL projects. The
                         Joint Staff, along with the military services, competes for funds allocated to OSD for ADL
                         prototypes projects.




               Page 36                                                        GAO-03-393 DOD's ADL Programs
               Briefing Section V: Conclusions
Briefing Section V: Conclusions




               Page 37                           GAO-03-393 DOD's ADL Programs
              Briefing Section VI: Army’s Advanced
Briefing Section VI: Army’s Advanced
              Distributed Learning Programs



Distributed Learning Programs




              Page 38                                GAO-03-393 DOD's ADL Programs
Briefing Section VI: Army’s Advanced
Distributed Learning Programs




Page 39                                GAO-03-393 DOD's ADL Programs
Briefing Section VI: Army’s Advanced
Distributed Learning Programs




Page 40                                GAO-03-393 DOD's ADL Programs
Briefing Section VI: Army’s Advanced
Distributed Learning Programs




Page 41                                GAO-03-393 DOD's ADL Programs
Briefing Section VI: Army’s Advanced
Distributed Learning Programs




Page 42                                GAO-03-393 DOD's ADL Programs
Briefing Section VI: Army’s Advanced
Distributed Learning Programs




Page 43                                GAO-03-393 DOD's ADL Programs
              Briefing Section VII: Navy’s Advanced
Briefing Section VII: Navy’s Advanced
              Distributed Learning Programs



Distributed Learning Programs




              Page 44                                 GAO-03-393 DOD's ADL Programs
Briefing Section VII: Navy’s Advanced
Distributed Learning Programs




Page 45                                 GAO-03-393 DOD's ADL Programs
Briefing Section VII: Navy’s Advanced
Distributed Learning Programs




Page 46                                 GAO-03-393 DOD's ADL Programs
Briefing Section VII: Navy’s Advanced
Distributed Learning Programs




Page 47                                 GAO-03-393 DOD's ADL Programs
              Briefing Section VIII: Marine Corps’ Advanced
Briefing Section VIII: Marine Corps’
              Distributed Learning Programs



Advanced Distributed Learning Programs




              Page 48                                         GAO-03-393 DOD's ADL Programs
Briefing Section VIII: Marine Corps’ Advanced
Distributed Learning Programs




Page 49                                         GAO-03-393 DOD's ADL Programs
Briefing Section VIII: Marine Corps’ Advanced
Distributed Learning Programs




Page 50                                         GAO-03-393 DOD's ADL Programs
Briefing Section VIII: Marine Corps’ Advanced
Distributed Learning Programs




Page 51                                         GAO-03-393 DOD's ADL Programs
               Briefing Section IX: Air Force’s Advanced
Briefing Section IX: Air Force’s Advanced
               Distributed Learning Programs



Distributed Learning Programs




               Page 52                                     GAO-03-393 DOD's ADL Programs
Briefing Section IX: Air Force’s Advanced
Distributed Learning Programs




Page 53                                     GAO-03-393 DOD's ADL Programs
Briefing Section IX: Air Force’s Advanced
Distributed Learning Programs




Page 54                                     GAO-03-393 DOD's ADL Programs
Briefing Section IX: Air Force’s Advanced
Distributed Learning Programs




Page 55                                     GAO-03-393 DOD's ADL Programs
               Briefing Section X: Joint Staff’s Advanced
Briefing Section X: Joint Staff’s Advanced
               Distributed Learning Programs



Distributed Learning Programs




               Page 56                                      GAO-03-393 DOD's ADL Programs
Briefing Section X: Joint Staff’s Advanced
Distributed Learning Programs




Page 57                                      GAO-03-393 DOD's ADL Programs
Briefing Section X: Joint Staff’s Advanced
Distributed Learning Programs




Page 58                                      GAO-03-393 DOD's ADL Programs
Briefing Section X: Joint Staff’s Advanced
Distributed Learning Programs




Page 59                                      GAO-03-393 DOD's ADL Programs
             Appendix I: Scope and Methodology
Appendix I: Scope and Methodology


             We reviewed the Department of Defense’s (DOD) Advanced Distributed
             Learning (ADL) programs to determine the programs’ expectations,
             implementation status, and major challenges. We collected, reviewed, and
             analyzed relevant program information and conducted interviews with
             DOD officials responsible for distance learning programs and from the
             Office of the Deputy Under Secretary of Defense for Readiness and
             Training; Advanced Distributed Learning Co-Laboratory, Alexandria,
             Virginia; Joint Advanced Distributed Learning Co-Laboratory, Orlando,
             Florida; Department of the Army, Deputy Chief of Staff for Operations—
             Training; the Army Distance Learning Program, U.S. Army Training and
             Doctrine Command; U.S. Army National Guard Bureau, Distributed
             Training Technology Project; Department of the Navy, Office of the Chief
             of Naval Operations—Education; U.S. Naval Education and Training
             Command, Office of Naval Education and Training; U.S. Marine Corps
             Training and Education Command, Distance Learning Center; Department
             of the U.S. Air Force, Office of the Deputy Chief of Staff for Personnel,
             Learning and Force Development; U.S. Air Force Air Education and
             Training Command, Air Force Institute for Advanced Distributed Learning;
             U.S. Air Force Office of Air Force Reserve, Education, Training, Readiness
             Policy; U.S. Air National Guard, Distributed Learning Program; Office of
             the Chairman of the Joint Chiefs of Staff, Joint Doctrine Education and
             Training Division; and Department of Defense, Defense Acquisition
             University.

             To determine DOD’s expectations for its programs, we reviewed
             executive, congressional and departmental guidance related to developing
             DOD-wide ADL programs. We reviewed and analyzed the Office of the
             Secretary of Defense’s (OSD), the military services’, and the Joint Staff’s
             ADL strategy and implementation plans and OSD’s Training
             Transformation Plan. We interviewed OSD, service, and Joint Staff ADL
             program personnel to obtain their views about OSD’s and their service- or
             Joint Staff-specific ADL program expectations.

             To determine the implementation status of OSD’s, the services’, and Joint
             Staff’s ADL programs, we provided OSD, service, and Joint Staff ADL
             program officials a detailed list of questions concerning program vision,
             strategy, implementation status, number of ADL courses, program
             successes, and challenges. We reviewed their written responses, if
             provided, and followed up with face-to-face interviews to clarify or obtain
             additional information if necessary. We reviewed, and compared OSD’s,
             the services’, and Joint Staff’s ADL strategies and implementation plans.
             We interviewed ADL program officials and collected other documents as
             necessary to determine the status of the programs as compared to their


             Page 60                                       GAO-03-393 DOD's ADL Programs
Appendix I: Scope and Methodology




ADL program implementation plans. Additionally, for fiscal years 1999
through 2002, we obtained, analyzed, and compared information about the
amount of funding OSD, the services, and the Joint Staff reportedly
received for their ADL programs. For the same fiscal years, we obtained
and reviewed the amount of funding DOD and the services received as
reported for Budget Activity 3 in each of the components Operations and
Maintenance budgets (BA3 funds all training and recruiting programs) and
compared the overall training budgets to the amount of funding each
reportedly allocated for ADL programs. In addition, we obtained and
analyzed the amount of funding that OSD, the services, and the Joint Staff
reported that they need and have programmed for future ADL
requirements for fiscal years 2003 through 2007. We compared the
amounts reported as needed to implement program plans with the
amounts included in OSD’s, the services’, and the Joint Staff ‘s program
objective memorandums for fiscal years 2003 through 2007. The dollar
amounts shown in this report are as of August 31, 2002. We did not
independently verify the dollar amounts reported in OSD’s and the
services’ budgets, nor did we independently verify the amount of funding
OSD, the services, and the Joint Staff reportedly allocated for their ADL
programs.

To determine major challenges affecting OSD’s, the services’ and the Joint
Staff’s ADL program implementation, we provided ADL program officials a
detailed list of questions that included specific questions related to
challenges ADL program managers face that affect their ability to execute
programs that achieve their expectations. We reviewed their written
responses, if provided, and followed up with face-to-face-interviews to
clarify or obtain additional information as necessary. We did a
comparative analysis of the comments they provided. We compiled a list of
challenges for OSD, each service, and the Joint Staff. We provided the lists
to each for their review and verification. The challenges cited by ADL
officials were grouped into four basic categories. During our exit briefing,
we provided ADL representatives from OSD, the services, and the Joint
Staff with a summary of the challenges noted during our review and asked
for their comments. It was the consensus of those ADL program officials
that the challenges we identified are valid.

We did not assess the effectiveness of the programs at this time because
most are in the early stages of implementation.




Page 61                                       GAO-03-393 DOD's ADL Programs
              Appendix II: Timeline of Key Events,
Appendix II: Timeline of Key Events,
              Directives and Guidance for DOD’s ADL
              Programs


Directives and Guidance for DOD’s ADL
Programs




              Page 62                                 GAO-03-393 DOD's ADL Programs
              Appendix III: Comments from the Department
Appendix III: Comments from the
              of Defense



Department of Defense




(350172)
              Page 63                                      GAO-03-393 DOD's ADL Programs
                         The General Accounting Office, the audit, evaluation and investigative arm of
GAO’s Mission            Congress, exists to support Congress in meeting its constitutional responsibilities
                         and to help improve the performance and accountability of the federal
                         government for the American people. GAO examines the use of public funds;
                         evaluates federal programs and policies; and provides analyses,
                         recommendations, and other assistance to help Congress make informed
                         oversight, policy, and funding decisions. GAO’s commitment to good government
                         is reflected in its core values of accountability, integrity, and reliability.


                         The fastest and easiest way to obtain copies of GAO documents at no cost is
Obtaining Copies of      through the Internet. GAO’s Web site (www.gao.gov) contains abstracts and full-
GAO Reports and          text files of current reports and testimony and an expanding archive of older
                         products. The Web site features a search engine to help you locate documents
Testimony                using key words and phrases. You can print these documents in their entirety,
                         including charts and other graphics.
                         Each day, GAO issues a list of newly released reports, testimony, and
                         correspondence. GAO posts this list, known as “Today’s Reports,” on its Web site
                         daily. The list contains links to the full-text document files. To have GAO e-mail
                         this list to you every afternoon, go to www.gao.gov and select “Subscribe to daily
                         E-mail alert for newly released products” under the GAO Reports heading.


Order by Mail or Phone   The first copy of each printed report is free. Additional copies are $2 each. A
                         check or money order should be made out to the Superintendent of Documents.
                         GAO also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a
                         single address are discounted 25 percent. Orders should be sent to:
                         U.S. General Accounting Office
                         441 G Street NW, Room LM
                         Washington, D.C. 20548
                         To order by Phone:     Voice:    (202) 512-6000
                                                TDD:      (202) 512-2537
                                                Fax:      (202) 512-6061


                         Contact:
To Report Fraud,
                         Web site: www.gao.gov/fraudnet/fraudnet.htm
Waste, and Abuse in      E-mail: fraudnet@gao.gov
Federal Programs         Automated answering system: (800) 424-5454 or (202) 512-7470


                         Jeff Nelligan, managing director, NelliganJ@gao.gov (202) 512-4800
Public Affairs           U.S. General Accounting Office, 441 G Street NW, Room 7149
                         Washington, D.C. 20548