oversight

Next Generation Enterprise Network: Navy Implementing Revised Approach, but Improvement Needed in Mitigating Risks

Published by the Government Accountability Office on 2012-09-19.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                 United States Government Accountability Office

GAO              Report to Congressional Requesters




September 2012
                 NEXT GENERATION
                 ENTERPRISE
                 NETWORK
                 Navy Implementing
                 Revised Approach, but
                 Improvement Needed
                 in Mitigating Risks




GAO-12-956
                                                 September 2012

                                                 NEXT GENERATION ENTERPRISE NETWORK
                                                 Navy Implementing Revised Approach, but
                                                 Improvement Needed in Mitigating Risks
Highlights of GAO-12-956, a report to
congressional requesters




Why GAO Did This Study                           What GAO Found
DON, a component of the Department               While the Department of the Navy (DON) has revised its acquisition approach for
of Defense (Defense), is replacing its           its new network system, the Next Generation Enterprise Network (NGEN), it still
existing network system with NGEN.               has not shown that it has selected the most cost-effective approach for acquiring
Capabilities for the new system include
                                                 NGEN capabilities. Cost effectiveness is shown by comparing life-cycle costs
secure transport of voice and data,
                                                 and quantifiable and nonquantifiable benefits among alternatives, which can be
data storage, and e-mail, at a cost of
about $38 billion through fiscal year            accomplished by conducting a thorough analysis of alternatives. GAO previously
2024. In March 2011, GAO reported                identified weaknesses with the NGEN analysis of alternatives related to cost
that the approach for acquiring NGEN             estimates and analysis of operational effectiveness and made associated
was not grounded in a reliable analysis          recommendations. However, DON did not revisit the analysis of alternatives to
of alternatives, the execution of NGEN           address the weaknesses previously identified, nor did it conduct any other
was not based on a reliable schedule,            analysis that would show whether its revised approach is the most cost effective.
and acquisition decisions were not               For example, while DON developed a draft economic analysis in February 2012,
always performance- and risk-based.              the analysis assessed only the status quo and revised approach, and not other
GAO recommended that Defense,                    alternatives. As a result, GAO remains concerned with the analysis measuring
among other things, reconsider its               NGEN cost effectiveness and DON does not know whether its revised approach
approach. The department has not yet             for acquiring NGEN is the most cost effective.
fully implemented GAO’s
recommendations but revised its                  Even though DON lacks assurance that it is pursuing the most cost-effective
approach to include acquiring certain            approach to acquiring NGEN capabilities, it has moved forward with
NGEN services simultaneously instead             implementing its revised approach. For example, the agency has completed
of staggering their implementation.              activities to support the acquisition and transition to NGEN, prepared plans and
                                                 analyses required for program initiation, and conducted oversight reviews to
GAO was asked to review the revised              support the release of the request for proposals for transport and enterprise
approach to determine (1) the extent to
                                                 services (secure data and e-mail services, among other things). However, the
which DON’s selected approach to
                                                 program’s schedule for acquiring NGEN capabilities has been delayed, thus
acquire NGEN is the most cost
effective and (2) the current status of          making it more likely that DON will not be able to fully transition by the end of the
and plans for acquiring NGEN. To do              continuity of services contract in April 2014. For example, the release of the
this, GAO reviewed analyses                      request for proposals was delayed, and upcoming milestones, such as contract
supporting the cost effectiveness of the         award and program initiation, have slipped (see table for major delays). Program
acquisition approach, the program’s              officials attributed the delays to the need for additional planning and to revisions
revised acquisition strategy, integrated         to the request for proposals. Compounding this situation is that identified risks
master schedule, key milestone                   are not being adequately mitigated. For example, not all mitigation plans are
decisions, and other relevant                    comprehensive because they do not always include all the elements of an
documents.                                       effective plan (e.g., identification of resources needed) nor do they always
                                                 contain the current status of the mitigation actions. According to program
What GAO Recommends                              officials, weaknesses in these mitigation plans were due, in part, to the lack of a
GAO is recommending that Defense                 priority in establishing and maintaining comprehensive and current mitigation
develop comprehensive risk mitigation            plans. As a result, the program faces an increased probability that transition from
plans for program-wide risks. In its             its existing system to NGEN will face further delays and cost overruns.
comments, Defense concurred with
GAO’s recommendation and noted that              Major Delays
it will continue to build on efforts to           Milestone                                                  Status                Delay
improve NGEN risk mitigation.                     Transport services request for proposals release           Completed             17 months
                                                  Enterprise services request for proposals release          Completed             9 months
                                                  Program initiation                                         Not yet occurred      15 months
View GAO-12-956. For more information,            Transport services contract award                          Not yet occurred      14 months
contact Valerie C. Melvin at (202) 512-6304 or    Enterprise services contract award                         Not yet occurred      6 months
melvinv@gao.gov.
                                                 Source: GAO analysis of DON data.

                                                                                                     United States Government Accountability Office
Contents


Letter                                                                                      1
               Background                                                                   2
               DON Has Not Reevaluated Alternatives to Ensure It Is Pursuing the
                 Most Cost-Effective NGEN Acquisition Approach                            13
               DON Is Proceeding with the NGEN Acquisition, but Is
                 Experiencing Schedule Delays and Not Adequately Mitigating
                 Risks                                                                    15
               Conclusions                                                                21
               Recommendation for Executive Action                                        22
               Agency Comments                                                            22

Appendix I     Objectives, Scope, and Methodology                                         24



Appendix II    Comments from the Department of Defense                                    26



Appendix III   GAO Contact and Staff Acknowledgments                                      28



Tables
               Table 1: Key Purpose of Each DON Gate Review                                 6
               Table 2: Summary and Comparison of the Four Alternatives in the
                        NGEN AOA                                                            7
               Table 3: Comparison of Previous and Current Plans for Acquiring
                        Segments                                                          11
               Table 4: Remaining Key NGEN Program Activities                             17
               Table 5: NGEN Program Critical Risks, as of July 2012                      20


Figure
               Figure 1: NGEN Major Milestone Delays                                      18




               Page i                            GAO-12-956 Next Generation Enterprise Network
Abbreviations

AOA                        analysis of alternatives
Defense                    Department of Defense
DON                        Department of the Navy
IT                         information technology
MAIS                       Major Automated Information System
Marine Corps               United States Marine Corps
NGEN                       Next Generation Enterprise Network
NMCI                       Navy Marine Corps Intranet
USN                        United States Navy




This is a work of the U.S. government and is not subject to copyright protection in the
United States. The published product may be reproduced and distributed in its entirety
without further permission from GAO. However, because this work may contain
copyrighted images or other material, permission from the copyright holder may be
necessary if you wish to reproduce this material separately.




Page ii                                    GAO-12-956 Next Generation Enterprise Network
United States Government Accountability Office
Washington, DC 20548




                                   September 19, 2012

                                   Honorable Susan M. Collins
                                   Ranking Member
                                   Committee on Homeland Security and Governmental Affairs
                                   United States Senate

                                   Honorable Claire McCaskill
                                   Chair
                                   Subcommittee on Contracting Oversight
                                   Committee on Homeland Security and Governmental Affairs
                                   United States Senate

                                   The Department of the Navy’s (DON) Next Generation Enterprise
                                   Network (NGEN) program is expected to replace and improve information
                                   technology (IT) services provided by the Navy Marine Corps Intranet
                                   (NMCI), which currently provides about 382,000 computer workstations to
                                   approximately 700,000 users across 2,500 United States Navy (USN) and
                                   United States Marine Corps (Marine Corps) locations around the world.
                                   As envisioned, NGEN is to provide secure data and IT services, such as
                                   data storage, e-mail, and video teleconferencing, and its capabilities are
                                   to be incrementally acquired through multiple providers. The first
                                   increment is planned to provide comparable NMCI capabilities, as well as
                                   enhanced information assurance and increased government control over
                                   network operations. This increment is expected to be fully operational by
                                   March 2014 and is estimated to cost approximately $38 billion through
                                   fiscal year 2024.

                                   In March 2011, we reported weaknesses in DON’s NGEN acquisition,
                                   noting, for example, that its approach for acquiring the network
                                   capabilities was not grounded in a reliable analysis of alternatives (AOA). 1
                                   As a result, we recommended, among other things, that the department
                                   conduct an interim review to reconsider the acquisition approach. DON
                                   subsequently reported that it had made changes to the NGEN acquisition
                                   approach.



                                   1
                                    GAO, Information Technology: Better Informed Decision Making Needed on Navy’s Next
                                   Generation Enterprise Network Acquisition, GAO-11-150 (Washington, D.C.: Mar. 11,
                                   2011).




                                   Page 1                                  GAO-12-956 Next Generation Enterprise Network
             At your request, we conducted a study of DON’s revised approach for the
             NGEN acquisition. Our specific objectives were to determine (1) the
             extent to which DON’s selected approach to acquire NGEN is the most
             cost effective and (2) the current status of and plans for this acquisition.

             To determine the extent to which the selected acquisition approach is the
             most cost effective, we reviewed DON documentation describing the cost
             effectiveness of the acquisition approach, including the draft economic
             analysis and other analyses supporting specific acquisition approach
             changes. In addition, we interviewed cognizant DON and Office of the
             Secretary of Defense officials about the use of these analyses in
             acquisition decision making to ensure that NGEN capabilities are
             acquired in the most cost-effective manner.

             To determine the current status of and plans for acquiring NGEN, we
             analyzed the program’s revised acquisition approach, integrated master
             schedule, performance assessments, risk reports, and executive
             acquisition decision briefings and meeting minutes, among other things.
             We also discussed progress made on NGEN acquisition efforts with DON
             program officials. Additional details on our objectives, scope, and
             methodology are discussed in appendix I.

             We conducted this performance audit from November 2011 to September
             2012 in accordance with generally accepted government auditing
             standards. Those standards require that we plan and perform the audit to
             obtain sufficient, appropriate evidence to provide a reasonable basis for
             our findings and conclusions based on our audit objectives. We believe
             that the evidence obtained provides a reasonable basis for our findings
             and conclusions based on our audit objectives.


             DON is a major component of Department of Defense (Defense), and
Background   consists of USN and Marine Corps service components. It is a large and
             complex organization, whose primary mission is to organize, train,
             maintain, and equip combat-ready naval forces capable of winning wars,
             deterring aggression from foes, preserving freedom of the seas, and
             promoting peace and security for the nation. To support this mission,
             DON performs a variety of interrelated and interdependent IT-dependent
             functions. In fiscal year 2012, DON’s IT budget was approximately $7.8
             billion for 841 investments. NGEN, with a budget of $1.7 billion in fiscal
             year 2012, is one such system investment.




             Page 2                              GAO-12-956 Next Generation Enterprise Network
Overview of NGEN   NGEN is to replace and improve the enterprise network and services
                   provided by NMCI, which were delivered through a DON-wide network
                   services contract with a single service provider (Hewlett Packard
                   Enterprise Services) that ended in September 2010. 2 To bridge the time
                   between the end of the NMCI contract and the full transition to the first
                   increment of NGEN, DON awarded a $3.4 billion continuity of services
                   contract 3 to Hewlett Packard Enterprise Services, which is scheduled to
                   run from October 2010 through April 2014. In addition to providing
                   continuity of network services, the contract includes transition services
                   and the transfer of NMCI infrastructure and intellectual property to DON.

                   When implemented, NGEN is to provide secure data and IT services,
                   such as data storage, e-mail, and video teleconferencing to USN and the
                   Marine Corps. It is also intended to provide the foundation for DON’s
                   future Naval Networking Environment. 4 The network is to be developed
                   incrementally, with the first increment expected to inherit the same
                   architecture and design, and provide the same capabilities and services
                   as does NMCI. In addition, NGEN is to provide increased DON control
                   over network operations and additional mandatory information assurance
                   capabilities to meet new Defense security requirements and the
                   implementation of an independent security validation function. Future
                   increments of the network have yet to be defined.

                   While NGEN’s first increment capabilities are not expected to differ from
                   those of NMCI, the operational environment for the network is expected to
                   change—from the contractor-owned and contractor-operated model
                   previously used by both services to a government-owned and contractor-
                   operated model for USN and to a government-owned and government-



                   2
                    NMCI was composed of transport infrastructure such as cables, routers, and switches;
                   end-user equipment such as computers, monitors, and keyboards; and software. It
                   provided, among other things, data storage, e-mail, transport of voice and data, and video
                   teleconferencing.
                   3
                    In August 2012, DON issued a sole source modification notice to increase the contract
                   ceiling to $5.5 billion. According to DON officials, the increase is needed because
                   additional funds will be necessary to maintain NMCI services through the transition to
                   NGEN. Officials stated that the modification had not yet been approved.
                   4
                    The Naval Networking Environment is to be an iterative set of integrated, phased
                   programs that share a common enterprise architecture and standards. It includes NGEN
                   and the Consolidated Afloat Networks and Enterprise Services program, among others,
                   and is to be in place by 2016.




                   Page 3                                     GAO-12-956 Next Generation Enterprise Network
                            operated model for the Marine Corps. In particular, USN plans to have
                            ownership and oversight of network operations while it relies on
                            contractors to execute and provide NGEN services. The Marine Corps
                            also plans to have ownership and oversight of network operations, but will
                            serve as its own service provider and obtain supplemental contractor
                            support as needed. The different operational models are intended to allow
                            USN and Marine Corps to operate their respective domains in the manner
                            best suited to support their different mission needs.


Oversight and Acquisition   To manage the acquisition and deployment of NGEN, DON established a
Processes for NGEN          program management office within the Program Executive Office for
                            Enterprise Information Systems. In February 2011, DON merged the
                            NGEN program management office with the NMCI program management
                            office to form the Naval Enterprise Networks program management office.
                            This office manages the program’s cost, schedule, and performance and
                            is responsible for ensuring that the program meets its objectives. In
                            addition, various Defense and DON organizations share program
                            oversight and review responsibilities. These key entities include the

                            •   Under Secretary of Defense for Acquisition, Technology, and
                                Logistics. Serves as the Milestone Decision Authority, which is the
                                individual designated with overall responsibility for the program, to
                                include approving the program to proceed through its acquisition cycle
                                on the basis of, for example, the acquisition strategy, an
                                independently evaluated economic analysis, and the acquisition
                                program baseline. The Milestone Decision Authority is accountable for
                                cost, schedule, and performance reporting, including reporting to
                                Congress.

                            •   Assistant Secretary of the Navy, Research, Development, and
                                Acquisition. Serves as DON’s acquisition oversight organization for
                                the program, to include implementation of Under Secretary of Defense
                                for Acquisition, Technology, and Logistics policies and procedures.
                                Determines when all key milestones are ready to be submitted to the
                                Milestone Decision Authority.

                                Department of the Navy, Program Executive Office for Enterprise
                                Information Systems. Oversees a portfolio of large-scale projects
                                and programs designed to enable common business processes and
                                provide standard capabilities. Reviews the acquisition strategy,
                                economic analysis, and the acquisition program baseline prior to
                                approval by the Milestone Decision Authority.



                            Page 4                             GAO-12-956 Next Generation Enterprise Network
•   Department of the Navy Chief Information Officer. Supports DON’s
    planning, programming, budgeting, and execution processes by
    ensuring that the program has achievable and executable goals and
    conforms to financial management regulations and to DON, Defense,
    and federal IT policies in several areas (e.g., security, architecture,
    and investment management). Works closely with the program office
    during milestone review assessments.
NGEN is designated as a Major Automated Information System (MAIS) 5
and is subject to both the Office of the Secretary of Defense’s and DON’s
MAIS acquisition policy and guidance, 6 which require the program to
comply with defense acquisition system requirements. The defense
acquisition system consists of five key program life-cycle phases and
three related milestone decision points: (1) materiel solution analysis, (2)
technology development (milestone A held prior to entering this phase),
(3) engineering and manufacturing development (milestone B held prior to
entering this phase), (4) production and deployment (milestone C held
prior to entering this phase), and (5) operations and support. The
Milestone Decision Authority is to review the initial capabilities document,
which defines operational goals and needed capabilities, and authorize
the phase in which a MAIS program is to enter the defense acquisition
system.

In May 2010, the Under Secretary of Defense for Acquisition, Technology,
and Logistics authorized the NGEN program to enter the defense
acquisition system at production and deployment. NGEN was approved to
enter at this later phase because the technology was considered mature
and already operational under NMCI. Prior to entering the production and
deployment phase, a milestone C review must be held to review the
capability production document and the test and evaluation master plan,
among other things, and to authorize limited deployment to support
operational testing. The purpose of the phase is to achieve an operational
capability that satisfies the mission needs and is verified through
independent operational test and evaluation, and to implement the


5
 Defense uses acquisition categories, where programs of increasing dollar value and
management interest are subject to increasing levels of oversight. MAIS programs are the
highest level category for automated information systems.
6
 Department of Defense Instruction 5000.02, Operation of the Defense Acquisition
System (Dec. 8, 2008), and Secretary of the Navy Instruction 5000.2E, Implementation
and Operation of the Defense Acquisition System and the Joint Capabilities Integration
and Development System (Sept. 1, 2011).




Page 5                                    GAO-12-956 Next Generation Enterprise Network
system at all applicable locations. At milestone C, the NGEN program is
planned to be initiated and the acquisition program baseline to be
approved, establishing the cost, schedule, and performance thresholds
and objectives for the program. On approval of milestone C, DON will
proceed with award of the NGEN contracts for the transport and
enterprise services segments.

In addition to the defense acquisition system requirements, DON
guidance and policy require all MAIS programs to go through a “two-
pass/six-gate” acquisition review process. 7 The first pass, which consists
of gates 1 through 3, is focused on requirements development and
validation and is led by the Chief of Naval Operations or the Commandant
of the Marine Corps. The second pass, which consists of gates 4 through
6, is focused on developing and delivering a solution via systems
engineering and acquisition and is led by the Assistant Secretary of the
Navy (Research, Development and Acquisition). In addition to meeting
specific criteria for passing a given gate and proceeding to the next gate,
all gate reviews are to consider program health (i.e., satisfactory cost and
schedule performance, known risks, and budget adequacy) in deciding
whether to proceed. Table 1 lists the key purpose of each gate review.

Table 1: Key Purpose of Each DON Gate Review

Gate     Purpose
1        Approve the initial capabilities document and validate AOA guidance and
         assumptions.
2        Review the AOA and approve the preferred alternatives resulting from it.
3        Authorize the capability development document, which defines the system’s key
         performance parameters and includes information necessary to develop an
         affordable system or system increment.
4        Approve the system design specification, which specifies the system
         requirements.
5        Approve release of a request for proposals.
6        Assess overall program health through multiple reviews, following award of a
         contract and satisfactory completion of an integrated baseline reviewa and
         approve the capability production document and program health prior to and after
         milestone C and the full deployment decision review. Sufficiency (sustainment)
         reviews focus on system performance and comparing actual performance and
         cost data to estimates.




7
Secretary of the Navy Instruction 5000.2E.




Page 6                                     GAO-12-956 Next Generation Enterprise Network
                                           Source: GAO analysis of DON documentation.
                                           a
                                            An integrated baseline review is performed to obtain stakeholder agreement on a contractor’s
                                           performance measurement baseline, which represents the cumulative value of planned work and
                                           serves as the baseline against which variances are calculated.


Prior GAO Review                           In March 2011, 8 we evaluated DON’s AOA for NGEN, which had
Identified Weaknesses in                   examined four acquisition alternatives. All of the alternatives were
DON’s Approach for                         assumed to deliver the same NMCI capabilities and the technology
                                           considered for each alternative was assumed to be substantially the
Acquiring NGEN
                                           same. As a result, DON officials stated that the AOA was not intended to
                                           be a traditional analysis to determine a system solution, but rather was an
                                           analysis of alternative acquisition approaches. The primary differences
                                           among the alternatives related to how NGEN was to be acquired,
                                           managed, and operated. Specifically, the alternatives varied in terms of
                                           the number of contracts to be awarded and in the scope of government
                                           versus contractor responsibilities. Table 2 provides a description and
                                           comparison of each alternative that was examined in the AOA.

Table 2: Summary and Comparison of the Four Alternatives in the NGEN AOA

                                                                                                                     Fiscal years 2011 to
                                                                                                         Contractual 2015 estimated cost       Relative
Alternative     Description                                                                            relationships          (in billions)    risk
Alternative 1   A recompete of the NMCI contract as a commercial item under                                               3           $10.3    Least
(status quo)    Federal Acquisition Regulation Part 12, in which the contractor
                was to be responsible for end-to-end integration of services and
                control of the network.
Alternative 2   A single contract similar to NMCI but as a noncommercial item                                             3           $10.8    More
(enhanced       with contract terms and conditions that were to address known
status quo)     NMCI deficienciesa and give the agency more control. The
                contractor was to be responsible for end-to-end integration of
                services and the government would control the network.
Alternative 3   Multiple contracts with different vendors, using the same                                               10            $10.8    Greater
variant         negotiated approach as alternative 2. The government was to be
(segmentedb     responsible for end-to-end integration of services and control of
approach)       the network.
Alternative 3   A greater number of contracts than alternative 3 variant. The                                           15            $10.7    Greatest
(segmentedb     government was to be responsible for end-to-end integration of
approach)       services and control of the network.
                                           Source: NGEN Analysis of Alternatives report, Center for Naval Analyses, April 2009.




                                           8
                                               GAO-11-150.




                                           Page 7                                                          GAO-12-956 Next Generation Enterprise Network
a
 See, for example, GAO, Information Technology: DOD Needs to Ensure That Navy Marine Corps
Intranet Program Is Meeting Goals and Satisfying Customers, GAO-07-51 (Washington, D.C.: Dec. 8,
2006), for information on NMCI deficiencies.
b
 A segment represents an allocation of IT services, functions, tools, and roles and responsibilities
associated with end-to-end service delivery. Segmentation of the network creates interfaces, which
DON refers to as “seams,” among the different contractors and government entities that must be
managed effectively to ensure successful delivery and continuity of services.


However, we reported that the approach pursued by DON did not match
any of the alternatives assessed in the AOA, and it was riskier and
potentially costlier than the alternatives assessed because it included a
higher number of contractual relationships. In particular, the chosen
approach was one that included more contracts, a different segmentation
scheme, and a different transition timeline than any of the alternatives
that had been assessed. We also reported that DON’s November 2009
risk-adjusted preliminary program life-cycle cost estimate for the
approach for fiscal years 2011 through 2015 showed that this approach
would cost at least an estimated $4.7 billion more than the alternatives
assessed in the AOA. 9 Moreover, DON had not analyzed the impact of
these differences in terms of how they compared to the original
alternatives. Further, we identified key weaknesses in the cost estimates
and operational effectiveness analysis included in the NGEN AOA.
Specifically, we reported that, while the AOA cost estimates were
substantially well documented, they were not substantially accurate, and
they were neither comprehensive nor credible. Additionally, we reported
that, while the AOA identified program capabilities and goals, it did not
sufficiently assess the alternatives’ ability to satisfy the capabilities and
goals.

Given our findings, we recommended that Defense reconsider the
acquisition approach based on a meaningful analysis of all viable
alternative acquisition approaches. The department did not fully concur
with our recommendation and stated that it had concluded that DON’s
AOA was sufficient and that the analysis had been approved by the Office
of the Secretary of Defense, Cost Assessment and Program Evaluation.
The department added that it would complete an economic analysis for
milestone C, which would include a follow-on independent cost estimate


9
 According to program documentation, the November 2009 risk-adjusted preliminary
program life-cycle cost estimate of approximately $50 billion was developed when
program definition was still evolving and program documentation was very recent or did
not exist. The NGEN life-cycle cost estimate has since been refined and is now
approximately $38 billion through fiscal year 2024.




Page 8                                           GAO-12-956 Next Generation Enterprise Network
and an updated determination of the most cost-effective solution.
However, in response, we pointed out that DON planned to assess only
the status quo and the current approach in the economic analysis, not
other alternatives such as those that had been included in the AOA, and
we maintained that without a meaningful analysis of alternatives, the
department would be unable to determine the most cost-effective
solution.

We also reported that DON’s schedule for NGEN did not adequately
satisfy key schedule estimating best practices by, for example,
establishing the critical path (the sequence of activities that, if delayed,
impacts the planned completion date of the project) and assigning
resources to all work activities. Because it did not satisfy these practices,
the schedule did not provide a reliable basis for program execution.
According to program officials, schedule estimating had been constrained
by staffing limitations. However, these weaknesses contributed to delays
in the completion of NGEN events and milestones, including multiple
major acquisition reviews and program plans. Accordingly, we
recommended that Defense ensure that the NGEN schedule substantially
reflect key schedule estimating practices. The department partially agreed
with our recommendation.

Additionally, we reported that NGEN acquisition decisions were not
always performance- and risk-based. In particular, senior executives had
approved the program’s continuing progress in the face of known
performance shortfalls and risks. For example, in November 2009, the
program was approved at a key acquisition review despite the lack of
defined requirements, which officials recognized as a risk that would
impact the completion of other key documents, such as the test plan.
According to DON officials, the decision to proceed was based on their
view that they had sufficiently mitigated known risks and issues. We
recommended that the department ensure future NGEN acquisition
reviews and decisions fully reflect the state of the program’s performance
and its exposure to risks. The department agreed with our
recommendation.




Page 9                              GAO-12-956 Next Generation Enterprise Network
DON Revised the NGEN   Subsequent to the issuance of our March 2011 report, 10 DON
Acquisition Approach   reconsidered and made certain changes to the NGEN acquisition
                       approach. Specifically, in April 2012, the Office of the Secretary of
                       Defense approved NGEN acquisition approach changes that were
                       intended to support program executability and reduce program risk for
                       USN. Like the original approach, the revised approach emphasized
                       segmentation of the network, with the same five segments that had been
                       defined in the previous acquisition approach: the two primary segments
                       are enterprise services and transport services and the remaining three
                       segments are end user hardware; enterprise software licenses; and
                       verification, validation, and reporting (see table 3 for details on these
                       segments). Further, each segment is expected to be delivered by either a
                       contractor or government provider, with multiple competitive awards.
                       However, DON made changes to how certain NGEN segments are to be
                       acquired and transitioned. For example, it plans to solicit transport and
                       enterprise services using a single request for proposals and has said it
                       may award a combined contract for both segments; in addition, it plans to
                       transition both segments to the new provider(s) simultaneously instead of
                       staggering their implementation. According to DON officials, these
                       changes were made primarily because the transport and enterprise
                       services segments were integrally related under NMCI, so acquiring them
                       simultaneously would potentially reduce labor costs and administrative
                       burden, and reduce risk. Another change is that USN is expected to
                       acquire end user hardware as a service from the enterprise services
                       contractor rather than purchase the equipment and provide it as
                       government-furnished property to the contractor. According to program
                       officials, this change was made to mitigate a critical NGEN risk that the
                       program may not be fully funded if end user hardware must be purchased
                       in fiscal year 2014 and, in the long term, procuring the end user hardware
                       as a service is not more expensive than government-purchased
                       equipment. As an additional change to the acquisition approach, USN is
                       no longer expected to award a contract for the verification, validation, and
                       reporting segment because it now has an internal entity—the Tenth Fleet
                       Cyber Command—that is to perform this function. Table 3 summarizes
                       the previous and current plans for acquiring NGEN.




                       10
                        GAO-11-150.




                       Page 10                             GAO-12-956 Next Generation Enterprise Network
Table 3: Comparison of Previous and Current Plans for Acquiring Segments

NGEN
segment         Purpose of segment                               Previous acquisition approach Revised acquisition approach
Enterprise      Provide the enterprise service desk, seat        Award contract for enterprise       Potentially award a combined contract
services        services supporting end user devices,            services and transition to new      with transport services and solicit the
                and data center services such as storage         provider separately from other      contract using a single request for
                and e-mail, along with hardware and              segments, for both USN and          proposals with transport services and
                software specific to enterprise services         Marine Corps.                       transition to new provider
                that are not covered under the end user                                              simultaneously with transport services,
                hardware and enterprise software                                                     for both USN and Marine Corps.
                licenses segments.
Transport       Provide for the operation and                    Award contract for transport        Potentially award a combined contract
services        sustainment of the transport                     services and transition to new      with enterprise services and solicit the
                infrastructure, associated services, and         provider separately from other      contract using a single request for
                level-of-effort support for those services.      segments for both USN and           proposals with enterprise services and
                It includes technology refresh of cable          Marine Corps.                       transition to a new provider
                plant, routers, and switches; some                                                   simultaneously with enterprise services,
                leasehold improvements; and moveable                                                 for both USN and Marine Corps.
                infrastructure associated with local
                network operations.
End user        Provide end user equipment such as               Award contracts for end user        USN will acquire end user hardware as
hardware        computers, monitors, and keyboards.              hardware with DON ownership of      a service from the enterprise services
                                                                 equipment, for both USN and         contractor. While it may purchase the
                                                                 Marine Corps.                       initial end user hardware and provide it
                                                                                                     as government-furnished property to the
                                                                                                     enterprise services contractor, it will
                                                                                                     acquire end user hardware as a service
                                                                                                     from the contractor once the initial
                                                                                                     equipment requires technology refresh.
                                                                                                     No change for Marine Corps from
                                                                                                     previous approach.
Enterprise      Provide software licenses to meet DON-           Award contract for end user         No change from previous approach.
software        wide requirements.                               software licenses for both USN
licenses                                                         and Marine Corps.
Verification,   Provide independent third-party security         Award contract for independent      USN will perform its own security
validation,     assessments of NMCI and NGEN to                  security assessment services for    function.
and reporting   support all DON networks within the              USN.                                No change for Marine Corps from
                naval network environment.                       Marine Corps will perform its own   previous approach.
                                                                 security function.
                                              Source: GAO analysis of DON data.



                                              DON’s planned transition from NMCI’s contractor-owned and contractor-
                                              operated model to NGEN’s government-owned and contractor-operated
                                              model for USN and government-owned and operated model for the
                                              Marine Corps includes establishing government control over network
                                              operations and transferring responsibility for network services from the
                                              incumbent provider to new service providers (contractor or government)
                                              in phases. DON is working to establish government control before award



                                              Page 11                                        GAO-12-956 Next Generation Enterprise Network
of the transport and enterprise services contract(s) to help reduce risk
during the transition to new provider(s). Specifically, it has defined early
transition activities as discrete efforts that are intended to establish
government management capabilities, allow for greater participation in
operational decisions, and help expedite the transition time. In addition,
USN plans to demonstrate that it has the people, processes, and tools in
place to execute command and control and governance in a segmented
environment without service degradation through a series of four
government readiness reviews. Government readiness review 0 is
planned to assess general government readiness prior to milestone C,
including determining whether USN has clearly defined the operating
model and whether the program office is ready to manage and execute
the transition with an acceptable level of risk. Government readiness
review 1 is planned to assess readiness after milestone C and prior to
award of the transport and enterprise services contract(s) to integrate IT
service management capabilities. Government readiness reviews 2 and 3
are planned to assess readiness after contract award to transition
remaining aspects, such as network and security operation centers. 11
Further, USN and Marine Corps have identified two transition completion
milestones—initial transition complete and final transition complete. DON
is working towards ensuring that all of these transition milestones are met
by the end of the continuity of services contract in April 2014.




11
  In addition to the government readiness reviews, USN plans to conduct a series of
accompanying reviews with the contractor(s) to validate contractor readiness to transition.




Page 12                                    GAO-12-956 Next Generation Enterprise Network
                         According to cost estimating and acquisition guidance, 12 cost
DON Has Not              effectiveness is shown by a comparative analysis of all life-cycle costs
Reevaluated              and quantifiable and nonquantifiable benefits among the competing
Alternatives to Ensure   alternatives. Such an analysis should be used to examine viable
                         alternatives to inform acquisition decision making on the most promising
It Is Pursuing the       solution, without assuming a specific means of achieving the desired
Most Cost-Effective      result. For example, an AOA is initiated to examine potential solutions
                         with the goal of identifying the most promising option and can
NGEN Acquisition         subsequently be updated, as needed, to refine the proposed solution and
Approach                 reaffirm the rationale in terms of cost effectiveness. 13 Additionally, an
                         economic analysis assesses net costs and benefits of the proposed
                         solution relative to the status quo and can identify and examine additional
                         alternatives that are considered feasible methods of satisfying the
                         objective.

                         Even after having revised its acquisition approach, DON has not yet
                         shown that it is pursuing the most cost-effective approach for acquiring
                         NGEN capabilities because it did not revisit the AOA to address the
                         weaknesses we previously identified, 14 nor did it conduct any other
                         analysis that would show that the current approach is the most cost
                         effective. Officials told us they believe the approach they are now
                         pursuing remains consistent with the AOA we previously assessed.
                         However, the revised approach DON is now pursuing was not one of the
                         alternatives assessed because it differs from the AOA alternatives in
                         terms of transition timeline, segmentation scheme, and potentially the
                         number of contracts and the AOA still contains the issues we identified in
                         our previous report.

                         Additionally, while DON developed a draft economic analysis for NGEN in
                         February 2012, the analysis assesses only the status quo (i.e., NMCI)
                         and the current acquisition approach for NGEN. As was the concern
                         noted in our previous report, this analysis does not show whether DON’s



                         12
                           GAO, GAO Cost Estimating and Assessment Guide: Best Practices for Developing and
                         Managing Capital Program Costs, GAO-09-3SP (Washington, D.C.: March 2009);
                         Defense Acquisition University, Defense Acquisition Guidebook (accessed January 10,
                         2012).
                         13
                           Defense Acquisition Guidebook and DON Acquisition and Capabilities Guidebook (May
                         2012).
                         14
                          GAO-11-150.




                         Page 13                                 GAO-12-956 Next Generation Enterprise Network
approach is the most cost-effective solution. Further, according to
program officials, the draft economic analysis is to be refined and updated
based on a revised service cost position, 15 and is not expected to be final
until the acquisition program baseline is to be approved and about 3
months before the planned time frame for awarding the primary NGEN
contracts for transport and enterprise services, and thus, would be limited
in its ability to inform decision makers on the best NGEN approach to
pursue. Program officials agreed that the final economic analysis would
not be able to show the most cost-effective solution; they stated that the
economic analysis is being prepared because it is a required document
for program initiation (milestone C review).

DON also developed analyses to support changes to its acquisition
approach by examining whether a specific change to a particular segment
would be more cost effective. For example, DON examined whether it
should release one request for proposals instead of two for the transport
and enterprise services segments. Additionally, USN examined whether it
should acquire existing end user hardware owned by the incumbent and
provide it to the enterprise services contractor as government-furnished
property or acquire the end user hardware from the enterprise services
contractor as a service. However, because these analyses focus on
specific changes, they do not provide an understanding of whether DON’s
overall acquisition approach is the most cost effective. Without a
meaningful analysis of acquisition alternatives, DON does not know
whether its approach for acquiring NGEN capabilities and meeting NGEN
goals is the most cost effective among other viable alternatives.




15
   The service cost position is DON’s first official cost estimate for NGEN, reflecting total
life-cycle NGEN costs of about $38 billion through fiscal year 2024, and was developed by
reconciling the NGEN program life-cycle cost estimate with an independent cost estimate
developed by the Naval Center for Cost Analysis.




Page 14                                     GAO-12-956 Next Generation Enterprise Network
                            Notwithstanding the lack of assurance that it is pursuing the most cost-
DON Is Proceeding           effective acquisition, DON nonetheless has moved forward with its
with the NGEN               revised approach for acquiring NGEN. In this regard, the department has
                            undertaken activities to support its acquisition and transition to NGEN,
Acquisition, but Is         prepared plans and analyses required for program initiation at milestone
Experiencing                C, and conducted oversight reviews to support the release of the request
Schedule Delays and         for proposals for transport and enterprise services. However, the
                            program’s schedule for acquiring NGEN capabilities has been delayed,
Not Adequately              resulting in a compressed timeline for transitioning to the new network
Mitigating Risks            and increased risks associated with transitioning to the new network
                            before the end of the continuity of services contract. Compounding this
                            situation is the fact that identified risks that can further impact schedule
                            delays are not being adequately mitigated.


Execution of the NGEN       DON has undertaken activities to support its acquisition and transition to
Program Is Proceeding,      NGEN, prepared plans and analyses required for program initiation at
but Major Milestones Have   milestone C, and conducted oversight reviews to support the release of
                            the request for proposals for transport and enterprise services.
Slipped                     Specifically,

                            •   As of December 2011, DON had completed early transition activities,
                                such as developing IT service management strategies, processes,
                                procedures, and tools to serve as the overarching governance
                                framework for delivering NGEN capabilities; analyzing and validating
                                the current NMCI infrastructure inventory; and conducting job task
                                analyses and assessing learning tools for contractor technical
                                representatives. Additionally, the Marine Corps assumed control of
                                the NMCI infrastructure currently supporting its operations and
                                awarded the Marine Corps Common Hardware Suite contract to
                                procure NGEN end user hardware in May 2012. Also in May 2012,
                                DON reached agreement on the first of 12 planned enterprise
                                software license agreements. Further, the department released NGEN
                                documents and technical data to industry to ensure all competitors
                                have full access to NMCI technical data and to reduce the potential for
                                a protest. DON also released multiple requests for information and
                                solicited input from industry on a draft request for proposals in order to
                                better understand the capabilities of the current IT marketplace with
                                respect to NGEN requirements. Finally, DON released the request for
                                proposals for transport and enterprise services in May 2012.

                            •   DON has also prepared several plans and analyses required for
                                program initiation at milestone C, when the acquisition program



                            Page 15                              GAO-12-956 Next Generation Enterprise Network
     baseline is to be approved. In particular, in October 2011, DON
     approved the cost analysis requirements description, which defines
     the programmatic and technical features of NGEN increment 1 and
     serves as the basis for estimating program costs. Additionally, the
     Naval Center for Cost Analysis developed the service cost position,
     which was based on the reconciliation of a completed program life-
     cycle cost estimate and an independent cost estimate. DON also
     approved the systems engineering plan and the test and evaluation
     master plan, which describes the overall test and evaluation strategy
     for how the network’s capabilities will be assessed. Subsequently, in
     November 2011, the program office developed the capability
     production document, which clarified and solidified the capabilities for
     NGEN increment 1 and became the primary source requirements
     document for the program. Finally, the revised acquisition strategy,
     which was required prior to release of the transport and enterprise
     services request for proposals, was approved in April 2012.

•    According to Defense and DON policy, acquisition programs must
     proceed through a series of gate and milestone reviews (as described
     earlier in this report in table 1). Since our prior report, 16 DON has
     conducted two gate reviews and an Office of the Secretary of
     Defense-level decision review to support the release of the request for
     proposals for transport and enterprise services. In particular, in
     October 2011, it completed an acquisition gate review to endorse the
     NGEN increment 1 capability production document. Subsequently,
     DON conducted a second NGEN acquisition gate review in January
     2012 to approve the transport and enterprise services request for
     proposals, during which it reviewed the current status and health of
     the program including the key activities remaining to release the
     request for proposals. Subsequently, in April 2012, the Office of the
     Secretary of Defense, Milestone Decision Authority, reviewed the
     NGEN program to approve the updated acquisition strategy and
     authorize the release of the transport and enterprise services request
     for proposals.

While DON has made progress on these efforts to acquire and transition
to NGEN, key program activities remain to be completed. For example,


16
  We previously reported in March 2011 that DON had completed gates 1 through 4 and
conducted a gate 5 review in October 2010 of its transport services request for proposals
but had not yet exited the gate because approval to proceed was conditional based on
satisfactory completion of open action items (GAO-11-150).




Page 16                                    GAO-12-956 Next Generation Enterprise Network
DON will need to demonstrate that it is prepared to execute control and
governance of the network through four government readiness reviews;
baseline the program by establishing cost, schedule, and performance
thresholds and objectives; award the primary NGEN contract(s); and
transition to the new NGEN provider(s). Table 4 lists planned completion
dates for these remaining key activities.

Table 4: Remaining Key NGEN Program Activities

    Activity                                                                  Planned date
    Government readiness review 0                                             September 2012
    Gate 6 (sufficiency) review                                               October 2012
    Milestone C review (NGEN program initiation)                              November 2012
    Government readiness review 1                                             December 2012
    Award of transport and enterprise services contract(s)                    February 2013
    Government readiness review 2                                             April 2013
    USN initial transition complete                                           May 2013
                                                    a
    Finalization of software licensing agreements                             June 2013
    Marine Corps final transition complete                                    June 2013
    Government readiness review 3                                             July 2013
    USN final transition complete                                             March 2014
Source: DON data.
a
 DON plans to enter into 12 enterprise software license agreements and this date represents
finalization of the last agreement.


However, a number of acquisition activities are facing schedule delays,
even though the incumbent is scheduled to end service delivery in April
2014. Specifically, while the request for proposals for transport and
enterprise services was issued, as we previously stated, it was delayed
by 17 months and 9 months, respectively, resulting in current delays in
NGEN program milestones, including the dates for conducting the
milestone C review and awarding the contract(s) for transport and
enterprise services. Additionally, the schedule for assessing USN’s
readiness to transition (i.e., government readiness reviews) is tied to
milestone C review and contract award, meaning that they are expected
to occur a certain number of days before or after their associated event;
thus, the government readiness reviews would also be impacted by
delays in milestone C and contract award. Moreover, these delays have
compressed the timeline for and increased the risks associated with
transitioning to the new network before the end of the continuity of
services contract. For example, the date for USN’s initial transition from
the current service provider to the new service provider(s) has slipped by



Page 17                                        GAO-12-956 Next Generation Enterprise Network
                                        5 months and final transition is scheduled for March 2014, thus
                                        compressing the period for shutting down network services with the
                                        incumbent and transitioning them to the new NGEN contractor(s) by
                                        about 5 months. Further, USN has identified a number of factors that
                                        could impact transition and increase the risk that NGEN may not be
                                        completed on time and may experience cost overruns, such as proposals
                                        not meeting NGEN requirements and lack of coordination among
                                        contractors and the government in operating the network.

                                        Program officials attributed these schedule delays to the department’s
                                        need to conduct more detailed planning before issuing the transport and
                                        enterprise services request for proposals and for addressing industry
                                        comments on the draft request for proposals to reduce the potential of a
                                        bid protest. Figure 1 illustrates the delays in major NGEN milestones.

Figure 1: NGEN Major Milestone Delays




                                        Notes: The schedules, considered to be informal baseline schedules, have been used to manage the
                                        overall scope of work to be conducted within the program. An official baseline schedule is expected to
                                        be approved as part of the acquisition program baseline at milestone C.
                                        Delays were rounded to the nearest month.
                                        a
                                         According to Marine Corps officials, new criteria for the initial and final transition complete milestones
                                        resulted in new dates for planned completion in the April 2012 schedule.




                                        Page 18                                            GAO-12-956 Next Generation Enterprise Network
                            Our prior report highlighted the significance of DON not having a reliable
                            schedule for executing NGEN and its contribution to delays in key
                            program milestones. 17 The lack of a reliable schedule, as we previously
                            noted, and the continuing delays in DON’s efforts to complete the network
                            transition as planned, raise concerns that it will be unable to complete the
                            transition within the time frames of the current continuity of services
                            contract. As a result, ensuring that the NGEN schedule substantially
                            reflects the key estimating practices, as we discussed and recommended
                            in our previous report, continues to be a vital step for DON to take.


Program Risk Mitigation     According to industry best practices, 18 an effective risk management
Plans Have Not Been Fully   process identifies potential problems before they occur, so that risk-
Defined                     handling activities may be planned and invoked, as needed, across the
                            life of the product and project in order to mitigate adverse impacts on
                            achieving objectives. Key activities of a comprehensive risk management
                            process include (1) identifying and analyzing risks, (2) escalating key
                            risks to the attention of senior management, and (3) developing risk
                            mitigation plans and milestones for key mitigation deliverables. In
                            particular, effective plans for risk mitigation should be developed for the
                            most important risks to the project, which includes a period of
                            performance, identification of resources needed, and responsible parties.
                            In addition, the status of each risk should be monitored periodically to
                            determine whether established thresholds have been exceeded and risk
                            mitigation plans should be implemented as appropriate to ensure that
                            systems will operate as intended.

                            NGEN program-wide and project-specific risks are managed by different
                            offices, with the program office identifying and tracking program-wide
                            risks—those that affect the overall NGEN program. In accordance with
                            best practices, the NGEN program identifies and analyzes program-wide
                            risks, by assigning a severity rating to risks, tracking these risks in a
                            database, and planning response strategies for each risk in the database.
                            In addition, NGEN program officials escalate these risks by reviewing and
                            evaluating these risks during monthly program risk management board
                            meetings. As of July 2012, the program office had identified eight


                            17
                             GAO-11-150.
                            18
                              Carnegie Mellon Software Engineering Institute, Capability Maturity Model® Integration
                            for Services (CMMI-SVC), version 1.3 (November 2010).




                            Page 19                                   GAO-12-956 Next Generation Enterprise Network
                                           program risks that it considered critical (moderate- or high-level risks) and
                                           that could result in schedule delays and cost increases. These risks
                                           included potential delays in transition from the incumbent to the new
                                           service provider(s) and in contract award for the transport and enterprise
                                           services, as well as the potential lack of coordination among contractors
                                           and the government in operating the network. Table 5 describes the
                                           program-identified critical risks for NGEN as of July 2012.

Table 5: NGEN Program Critical Risks, as of July 2012

                                                                                                                          Program-
                                                                                                                          identified
Program-identified critical risk    Program risk description from program documentation                                   risk level
NGEN service request tool           The current tools for ordering requests for services are not prepared within the      High
                                    NGEN environment. If the development of a functional NGEN ordering tool is not
                                    complete in time to support the NGEN transition, then ordering will rely on error-
                                    prone manual processes. Without automated ordering, service delivery will
                                    degrade and staffing costs to perform ordering will increase significantly.
Transition delays for USN           Significant delays could impact the ability to transition from the incumbent to the    High
                                    new service provider(s) before the continuity of services contract performance
                                    period ends in April 2014. If USN does not fully transition in time, it will have to
                                    extend the continuity of services contract, resulting in cost increases and additional
                                    schedule delays.
Seam management                     Roles and relationships among segments and government functions have not been High
                                    defined or agreed on. If these seams are not accurately identified and
                                    characterized, DON will not be able to manage them effectively and the transition
                                    will take longer.
Purchase of end user hardware for   The NGEN program will not be fully funded if end user hardware must be                Moderate
USN                                 purchased in fiscal year 2014 for approximately $150 million.
Delayed transport and enterprise    A day-to-day slip in completing the final transition will be realized if award of     Moderate
contract(s) award                   transport and enterprise service contract(s) is delayed.
Systems engineering                 Service performance may degrade, systems integration costs may increase, and          Moderate
                                    time to resolve engineering issues may increase if the program is unable to
                                    execute design authority and problem management.
Service offering descriptions       Service availability and performance will degrade if DON cannot effectively           Moderate
                                    manage the level of service quality and direct the resources in support of service
                                    delivery.
Potential protest is sustained      A protest against any of the NGEN contract awards may impact the schedule for         Moderate
                                    transition completion by as much as 6 months, resulting in an extension of the
                                    continuity of services contract.
                                           Source: DON data.



                                           While DON is working to mitigate seven of the eight program risks, its
                                           mitigation plans did not always include all the elements of an effective
                                           plan (e.g., identification of resources needed, responsible parties, and
                                           period of performance). Specifically, the reported mitigation strategies did
                                           not fully identify the resources needed, such as the staff and funds, nor


                                           Page 20                                       GAO-12-956 Next Generation Enterprise Network
              fully identify organizations that are responsible and accountable for
              accomplishing risk mitigation activities. Additionally, while five of the
              seven mitigation plans had activities with planned completion dates, most
              did not include an estimated start date; thus, the plans did not fully define
              the period of performance to ensure that the mitigation activities are being
              implemented appropriately. Moreover, three of the seven plans did not
              identify the status of activities for which completion dates had already
              occurred. In particular, to mitigate the risk of potential lack of coordination
              among contractors and the government in operating the network, DON
              was to develop, implement, and automate key processes by February 15,
              2012. However, the plan does not reflect whether this activity has been
              completed or, otherwise discuss its status.

              Additionally, two of the seven plans did not fully reflect the current status
              of the program. For example, to mitigate transition risks, DON officials
              identified that the enterprise and transport services contract(s) must be
              awarded no later than December 2012 in order to ensure continuous
              network availability during the transition from the continuity of services
              contract to the NGEN contract(s). However, the current mitigation plan
              does not document this milestone or reflect the current status of the
              program, which now plans to award the contract(s) in February 2013.

              Further, according to program documentation, a mitigation plan was
              required and was being updated for the service offering descriptions risk;
              however, according to other program documentation, the plan to mitigate
              service offering descriptions is still under development, even though it has
              been identified as a program risk since August 2011. According to
              program officials, weaknesses in these mitigation plans were due, in part,
              to the lack of a priority in establishing and maintaining comprehensive
              and current plans.

              Several of the risks identified are significant to ensure that NGEN
              transition occurs as planned and within the estimated costs. Therefore it
              is essential to ensure that for a given risk, techniques and methods will be
              invoked to avoid, reduce, and control the probability of occurrence.


              Even though DON does not know whether it is pursuing the most cost-
Conclusions   effective approach to acquiring NGEN capabilities, it has proceeded to
              implement its revised acquisition approach, completed various plans and
              analyses including the first official program life-cycle cost estimate, and
              held oversight reviews to support the issuance of the request for
              proposals for the two primary NGEN segments. While these steps have


              Page 21                              GAO-12-956 Next Generation Enterprise Network
                     been taken, DON faces delays in upcoming milestones, which have
                     resulted in a compressed transition timeline and increased risks
                     associated with transitioning to the new network before the end of the
                     continuity of services contract. Compounding this are weaknesses in
                     DON’s risk mitigation efforts that could further impact schedule delays
                     and result in cost increases. Without a well-defined schedule, as we
                     previously reported, and adequate risk mitigation, DON cannot ensure
                     that needed NGEN capabilities will be in place in time to ensure that
                     services will continue to operate when the incumbent is scheduled to shut
                     down its services.


                     To strengthen risk mitigation activities for the NGEN program, we
Recommendation for   recommend that the Secretary of Defense direct the Secretary of the
Executive Action     Navy to develop comprehensive mitigation plans and strategies for
                     program-wide critical risks that identify the mitigation period of
                     performance, resources needed, and responsible parties, and that fully
                     reflect the current status of the program.


                     The Department of Defense provided written comments on a draft of this
Agency Comments      report, signed by the Deputy Assistant Secretary of Defense (C3 and
                     Cyber), and reprinted in appendix II. In its comments, the department
                     agreed with our recommendation and noted that the program office will
                     continue to build on efforts to improve NGEN’s risk management and
                     mitigation process. For example, the department stated that it plans to
                     increase the speed at which NGEN risk management board action items
                     are closed.

                     We are sending copies of this report to the appropriate congressional
                     committees; the Director, Office of Management and Budget; the
                     Congressional Budget Office; the Secretary of Defense; and the
                     Secretary of the Navy. In addition, the report is available at no charge on
                     the GAO website at http://www.gao.gov.

                     If you or your staff have any questions about this report, please contact
                     me at (202) 512-6304 or melvinv@gao.gov. Contact points for our Offices
                     of Congressional Relations and Public Affairs may be found on the last




                     Page 22                             GAO-12-956 Next Generation Enterprise Network
page of this report. GAO staff who made major contributions to this report
are listed in appendix III.




Valerie C. Melvin
Director, Information Management and Technology Resources Issues




Page 23                            GAO-12-956 Next Generation Enterprise Network
Appendix I: Objectives, Scope, and
              Appendix I: Objectives, Scope, and
              Methodology



Methodology

              Our objectives were to determine (1) the extent to which the Department
              of the Navy’s (DON) selected approach to acquiring the Next Generation
              Enterprise Network (NGEN) is the most cost effective and (2) the current
              status of and plans for acquiring NGEN.

              To determine the extent to which DON’s approach to acquiring NGEN is
              the most cost effective, we reviewed our prior work evaluating the NGEN
              analysis of alternatives 1 and analyzed current documentation that DON
              had completed to describe and justify the cost effectiveness of its
              acquisition approach. These included the draft economic analysis and
              analyses to support specific acquisition approach changes. We assessed
              DON’s supporting analyses against relevant Department of Defense
              guidance and our Cost Estimating and Assessment Guide. 2 In this regard,
              we evaluated the purpose and use of these analyses in examining viable
              alternatives to inform acquisition decision making on the most cost-
              effective solution. We also interviewed cognizant DON program officials
              and Office of the Secretary of Defense Cost Assessment and Program
              Evaluation officials about the use of these analyses in acquisition decision
              making.

              To determine the current status of and plans for acquiring NGEN, we
              analyzed the revised NGEN acquisition strategy, integrated master
              schedule, program performance assessments, risk reports, transport and
              enterprise services request for proposals, planned system requirements,
              cost estimates, draft operational readiness plan, and executive acquisition
              decision briefings and meeting minutes, among other things. We also
              reviewed these documents to determine how the program had changed
              since our prior review of NGEN. 3 To assess the status of risk
              management for the NGEN program initiative, we compared the risk
              management plans and supporting documentation against leading
              practices, such as Carnegie Mellon Software Engineering Institute’s



              1
               GAO, Information Technology: Better Informed Decision Making Needed on Navy’s Next
              Generation Enterprise Network Acquisition, GAO-11-150 (Washington, D.C.: Mar. 11,
              2011).
              2
               GAO, GAO Cost Estimating and Assessment Guide: Best Practices for Developing and
              Managing Capital Program Costs, GAO-09-3SP (Washington, D.C.: March 2009);
              Defense Acquisition University, Defense Acquisition Guidebook (accessed January 10,
              2012); and DON Acquisition and Capabilities Guidebook (May 2012).
              3
               GAO-11-150.




              Page 24                                 GAO-12-956 Next Generation Enterprise Network
Appendix I: Objectives, Scope, and
Methodology




Capability Maturity Model® Integration for Services, 4 to determine
whether such practices had been specified in the plans. Further, we
interviewed relevant DON program officials and Office of the Secretary of
Defense Cost Assessment and Program Evaluation officials to clarify
information in documents we reviewed and to more fully understand the
program’s progress to acquire NGEN. To assess the reliability of the data
that we used to support the findings in this report, we reviewed relevant
program documentation to substantiate evidence obtained through
interviews with agency officials. We determined that the data used in this
report are sufficiently reliable. We have also made appropriate attribution
indicating the sources of the data used.

We conducted this performance audit from November 2011 to September
2012 in accordance with generally accepted government auditing
standards. Those standards require that we plan and perform the audit to
obtain sufficient, appropriate evidence to provide a reasonable basis for
our findings and conclusions based on our audit objectives. We believe
that the evidence obtained provides a reasonable basis for our findings
and conclusions based on our audit objectives.




4
 Carnegie Mellon Software Engineering Institute, Capability Maturity Model® Integration
for Services (CMMI-SVC), version 1.3 (November 2010).




Page 25                                   GAO-12-956 Next Generation Enterprise Network
Appendix II: Comments from the Department
             Appendix II: Comments from the Department
             of Defense



of Defense




             Page 26                                     GAO-12-956 Next Generation Enterprise Network
Appendix II: Comments from the Department
of Defense




Page 27                                     GAO-12-956 Next Generation Enterprise Network
Appendix III: GAO Contact and Staff
                  Appendix III: GAO Contact and Staff
                  Acknowledgments



Acknowledgments

                  Valerie C. Melvin, (202) 512-6304, or melvinv@gao.gov
GAO Contact
                  In addition to the individual named above, key contributors to this report
Staff             were Eric Winter (assistant director), Nabajyoti Barkakati, Harold Brumm,
Acknowledgments   Neil Doherty, Nancy Glover, Madhav Panwar, Jeanne Sung, and Niti
                  Tandon.




(310977)
                  Page 28                               GAO-12-956 Next Generation Enterprise Network
GAO’s Mission         The Government Accountability Office, the audit, evaluation, and
                      investigative arm of Congress, exists to support Congress in meeting its
                      constitutional responsibilities and to help improve the performance and
                      accountability of the federal government for the American people. GAO
                      examines the use of public funds; evaluates federal programs and
                      policies; and provides analyses, recommendations, and other assistance
                      to help Congress make informed oversight, policy, and funding decisions.
                      GAO’s commitment to good government is reflected in its core values of
                      accountability, integrity, and reliability.

                      The fastest and easiest way to obtain copies of GAO documents at no
Obtaining Copies of   cost is through GAO’s website (http://www.gao.gov). Each weekday
GAO Reports and       afternoon, GAO posts on its website newly released reports, testimony,
                      and correspondence. To have GAO e-mail you a list of newly posted
Testimony             products, go to http://www.gao.gov and select “E-mail Updates.”

Order by Phone        The price of each GAO publication reflects GAO’s actual cost of
                      production and distribution and depends on the number of pages in the
                      publication and whether the publication is printed in color or black and
                      white. Pricing and ordering information is posted on GAO’s website,
                      http://www.gao.gov/ordering.htm.
                      Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
                      TDD (202) 512-2537.
                      Orders may be paid for using American Express, Discover Card,
                      MasterCard, Visa, check, or money order. Call for additional information.
                      Connect with GAO on Facebook, Flickr, Twitter, and YouTube.
Connect with GAO      Subscribe to our RSS Feeds or E-mail Updates. Listen to our Podcasts.
                      Visit GAO on the web at www.gao.gov.
                      Contact:
To Report Fraud,
Waste, and Abuse in   Website: http://www.gao.gov/fraudnet/fraudnet.htm
                      E-mail: fraudnet@gao.gov
Federal Programs      Automated answering system: (800) 424-5454 or (202) 512-7470

                      Katherine Siggerud, Managing Director, siggerudk@gao.gov, (202) 512-
Congressional         4400, U.S. Government Accountability Office, 441 G Street NW, Room
Relations             7125, Washington, DC 20548

                      Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800
Public Affairs        U.S. Government Accountability Office, 441 G Street NW, Room 7149
                      Washington, DC 20548




                        Please Print on Recycled Paper.