oversight

Military Readiness: Navy Needs to Assess Risks to Its Strategy to Improve Ship Readiness

Published by the Government Accountability Office on 2012-09-21.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                 United States Government Accountability Office

GAO              Report to Congressional Committees




September 2012
                 MILITARY
                 READINESS
                 Navy Needs to Assess
                 Risks to Its Strategy to
                 Improve Ship
                 Readiness




GAO-12-887
                                             Date

                                             MILITARY READINESS
                                             Navy Needs to Assess Risks to Its Strategy to
                                             Improve Ship Readiness
Highlights of GAO-12-887, a report to
congressional committees




Why GAO Did This Study                       What GAO Found
In 2010, the Navy concluded that             Recent data show variations in the material readiness of different types of ships,
decisions it made to increase                but do not reveal any clear trends of improvement or decline for the period from
efficiencies of its surface force had        2008 to 2012. The Navy uses a variety of means to collect, analyze, and track
adversely affected ship readiness and
                                             the material readiness of its surface combatant and amphibious warfare ships.
service life. To improve ship readiness
                                             Three data sources the Navy uses to provide information on the material
the Navy developed a new strategy,
which includes several initiatives.          readiness of ships are: casualty reports, which reflect equipment malfunctions;
House Report 112-78, accompanying a          Defense Readiness Reporting System-Navy (DRRS-N) reports; and Board of
proposed bill for the Fiscal Year 2012       Inspection and Survey (INSURV) material inspection reports. These data sources
National Defense Authorization Act           can be viewed as complementary, together providing data on both the current
(H.R.1540), directed GAO to review           and life cycle material readiness of the surface force. INSURV and casualty
the recent Navy initiatives. GAO             report data show that the material readiness of amphibious warfare ships is lower
assessed 1) how the Navy evaluates           than that of frigates and destroyers. However, there is no clear upward or
the material readiness of its surface        downward trend in material readiness across the entire Navy surface combatant
combatant and amphibious warfare             and amphibious warfare ships. From 2010 to March 2012, INSURV data
ships and the extent to which data           indicated a slight improvement in the material readiness of the surface combatant
indicate trends or patterns in the           and amphibious warfare fleet, but over that period casualty reports from the ships
material readiness of these ships, and       increased, which would indicate a decline in material readiness. DRRS-N data
2) the extent to which the Navy has          also show differences in material readiness between ship types, but the precise
taken steps to improve the readiness         differences are classified and therefore are not included in this report.
of its surface combatant and
amphibious warfare ships, including          The Navy has taken steps to improve the readiness of its surface combatant and
implementing its new readiness               amphibious warfare ships, including a new strategy to better integrate
strategy. GAO analyzed Navy policies,        maintenance actions, training, and manning, but it faces risks to fully
material and readiness data from             implementing its strategy and has not assessed these risks or developed
January 2008—two years prior to the          alternatives to mitigate them. In March 2012, near the end of a year-long pilot,
release of the Navy’s 2010 report on         the Navy issued its Surface Force Readiness Manual, which calls for integrating
the degradation of surface force
                                             and synchronizing maintenance, training and manning among multiple
readiness—through March 2012, two
                                             organizations. The Navy expects this strategy to provide a standard, predictable
years after the release of the report,
and interviewed headquarters and
                                             path for ships to achieve and sustain surface force readiness, but certain factors,
operational officials and ship crews.        such as high operational tempos and supporting organizations’ staffing levels,
                                             could delay the entry of some ships into the strategy and the execution of the
What GAO Recommends                          strategy. For example, one supporting organization reported needing an
                                             additional 680 personnel to fully execute the strategy. As of August 2012, the
GAO recommends that the Navy
                                             Navy plans to reflect its funding needs for 410 personnel in its fiscal year 2014
conduct a comprehensive assessment
                                             budget request and the remaining 270 in subsequent requests. Also, due to high
of the risks the new strategy faces and
develop alternatives to mitigate these       operational tempos the phased implementation of some ships into the strategy
risks. DOD partially concurred, but felt     may be delayed. Furthermore, ships that do not execute the strategy’s
that current assessments sufficiently        maintenance periods as planned will have lifecycle maintenance actions
identify risks. GAO continues to believe     deferred. GAO has previously reported that risk assessment can inform effective
that a comprehensive assessment that         program management by helping managers make decisions about the allocation
takes into account the full range of risk    of finite resources, and alternative courses of action. However, the Navy has not
to the overall strategy is needed.           undertaken a comprehensive assessment of risks to the implementation of its
                                             strategy, nor has it developed alternatives to mitigate its risks. GAO believes
                                             operational tempo, supporting organizations’ staffing levels, and other risks may
                                             hinder the Navy’s full implementation of its surface force readiness strategy. If
View GAO-12-887. For more information,
contact Sharon Pickup at (202) 512-9619 or   not addressed, this could lead to deferrals of lifecycle maintenance, which have
pickups@gao.gov.                             in the past contributed to increased maintenance costs, reduced readiness, and
                                             shorter service lives for some ships.
                                                                                     United States Government Accountability Office
Contents


Letter                                                                                    1
               Background                                                                 3
               Data Indicate Differences in Material Readiness Between Ship
                 Types, but Do Not Reveal Readiness Trends Due to Limitations
                 in the Data                                                              7
               Navy Has Acted to Improve Readiness but Not Assessed Risks to
                 Achieving Full Implementation of Its Recent Strategy                   13
               Conclusions                                                              19
               Recommendations for Executive Action                                     19
               Agency Comments and Our Evaluation                                       20

Appendix I     Scope and Methodology                                                    23



Appendix II    Comments from the Department of Defense                                  29



Appendix III   GAO Contacts and Staff Acknowledgments                                   32



Tables
               Table 1: Classes of Surface Ships Reviewed                                 3
               Table 2: The 27-month Fleet Response Plan for Surface Combatant
                        and Amphibious Warfare Ships                                      5
               Table 3: Navy’s Material Readiness Data Sources                            8
               Table 4: INSURV and CASREP Data by Ship Type: January 2008 -
                        March 2012                                                        9
               Table 5: Average INSURV Equipment Operational Capability Scores
                        for All Surface Combatant and Amphibious Warfare Ships
                        from January 2008 through March 2012                            12
               Table 6: Navy Ships Planned For Early Retirement                         17
               Table 7: Summary of Regression Model Parameters                          26


Figures
               Figure 1: Administrative Chain of Command                                  4
               Figure 2: Average Number of Casualty Reports, per Ship, of Surface
                        Combatant and Amphibious Warfare Ships by Quarter
                        from January 2008 through March 2012                            11


               Page i                                          GAO-12-887 Military Readiness
Figure 3: Notional 27-month Fleet Response Plan Cycle                                     15
Figure 4: INSURV Process for Determining Overall Material
         Inspection Ratings                                                               24




This is a work of the U.S. government and is not subject to copyright protection in the
United States. The published product may be reproduced and distributed in its entirety
without further permission from GAO. However, because this work may contain
copyrighted images or other material, permission from the copyright holder may be
necessary if you wish to reproduce this material separately.




Page ii                                                     GAO-12-887 Military Readiness
United States Government Accountability Office
Washington, DC 20548




                                   September 21, 2012

                                   The Honorable Carl Levin
                                   Chairman
                                   The Honorable John McCain
                                   Ranking Member
                                   Committee on Armed Services
                                   United States Senate

                                   The Honorable Howard P. “Buck” McKeon
                                   Chairman
                                   The Honorable Adam Smith
                                   Ranking Member
                                   Committee on Armed Services
                                   House of Representatives

                                   In support of national interests, the Navy maintains a large surface force
                                   to meet its current missions and long-term obligations, including ensuring
                                   sea control, projecting power, and providing maritime security. The
                                   Navy’s plan, which is to grow its current fleet of 286 ships to about 300 by
                                   2019, is dependent both on its ability to acquire new ships and to
                                   maintain current ships for their expected service lives. The costs of
                                   procuring new ships and maintaining current ships can both be
                                   significant. For example, the cost of a new destroyer is more than $1.6
                                   billion, and in fiscal year 2011, the Navy spent an average of $11 million
                                   maintaining each of its 62 destroyers. In the past, when faced with the
                                   high costs of maintaining ships’ material conditions and keeping them
                                   mission-ready, the Navy elected to retire ships early—before they
                                   reached their expected service lives. For example, in January 2012, the
                                   Navy announced plans to retire seven cruisers and two amphibious ships
                                   early, in 2013 and 2014. Navy officials later testified that the service
                                   would redirect the savings from these early retirements to fund the
                                   maintenance of its remaining ships.

                                   In 2010, a Navy report found that the material readiness of its surface
                                   force had declined over the previous ten years and was well below the
                                   levels necessary to support reliable, sustained operations at sea and
                                   achieve expected ship service lives. Among other things, the report found
                                   that the declines in material readiness were attributable to reductions in
                                   the number of assessments and inspections, deferrals of scheduled
                                   maintenance, and reductions in the length of major repair periods from 15
                                   to 9 weeks. In response, the Navy developed several maintenance


                                   Page 1                                            GAO-12-887 Military Readiness
initiatives and a new readiness strategy that it expects will improve the
material condition of its ships and help them achieve their expected
service lives. The strategy and initiatives are in various stages of
implementation.

In House Report 112-78, accompanying a bill for the Fiscal Year 2012
National Defense Authorization Act (H.R. 1540), the House Armed
Services Committee directed GAO to review the Navy’s initiatives to
improve the material condition of its surface ship fleet. 1 This report
analyzes 1) how the Navy evaluates the material readiness of its surface
combatant and amphibious warfare ships and the extent to which data
indicate trends or patterns in the material readiness of these ships, and 2)
the extent to which the Navy has taken steps intended to improve the
readiness of its surface combatant and amphibious warfare ships,
including efforts to implement its recent strategy.

To address our first objective, we analyzed Navy policies and procedures
for determining surface force material readiness, as well as various
studies and reports on the Navy’s material readiness process. We also
interviewed officials from Navy commands that are responsible for
maintaining, assessing, evaluating, and inspecting the material condition
of the Navy’s surface ships. In addition, we analyzed readiness,
maintenance, and inspection data to determine any changes in readiness
from 2008—two years prior to the release of the Navy’s 2010 report on
the degradation of surface force readiness—through March 2012, two
years after the release of the report. Specifically, we analyzed data for the
Navy’s guided-missile cruisers (CG 47 class), guided-missile destroyers
(DDG 51 class), frigates (FFG 7 class), amphibious assault ships (LHA 1
and LHD 1 classes), amphibious transport dock ships (LPD 4 and LPD 17
classes), and dock landing ships (LSD 41 and LSD 49 classes). To
address our second objective, we reviewed relevant Navy instructions on
Navy material readiness, including the Navy’s Surface Force Readiness
Manual, and prior GAO work on risk management. We also interviewed
Navy training and maintenance officials to discuss how the Navy is
implementing its new readiness strategy. Finally, we met with ship
personnel to discuss possible challenges in implementing the new
readiness strategy and efforts.




1
    H.R. Rep. No-112-78, at 110 (2011).




Page 2                                             GAO-12-887 Military Readiness
                                        We conducted this performance audit from July 2011 to September 2012
                                        in accordance with generally accepted government auditing standards.
                                        Those standards require that we plan and perform the audit to obtain
                                        sufficient, appropriate evidence to provide a reasonable basis for our
                                        findings and conclusions based on our audit objectives. We believe that
                                        the evidence obtained provides a reasonable basis for our findings and
                                        conclusions based on our audit objectives. We discuss our scope and
                                        methodology in more detail in appendix I.


                                        The Navy’s fleet includes aircraft carriers, cruisers, destroyers, frigates,
Background                              littoral combat ships, submarines, amphibious warfare, mine warfare,
                                        combat logistics, and fleet support ships. Our review focused on surface
                                        combatant and amphibious warfare ships, which constitute slightly less
                                        than half of the total fleet. Table 1 shows the classes of surface ships we
                                        reviewed along with their numbers, expected service lives, and current
                                        average ages.

Table 1: Classes of Surface Ships Reviewed

                                                                                      Expected Service       Current Average Age of
Ship Class                                       Ship Numbers                Life of Ship Class (years)     Active Ship Class (years)
Surface Combatants                                               107
 Cruisers (CG 47)                                                  22                               35                             22
 Destroyers                                                        28                               35                             16
•   DDG 51(I and II)                                               34                               40                              7
•   DDG 51 (IIA)
 Frigates (FFG 7)                                                  23                               30                             28
Amphibious Warfare Ships                                           28
 LHA 1                                                               1                              35                             32
LPD 4                                                                1                              35                             44
LHD 1                                                                8                              40                             15
LPD 17                                                               6                              40                              4
LSD 41                                                               8                              40                             24
LSD 49                                                               4                              40                             16
Total                                                            135
                                        Source: GAO analysis of Navy data.



                                        Figure 1 shows the administrative chain of command for Navy surface
                                        ships. The U.S. Pacific Fleet and U.S. Fleet Forces Command organize,
                                        man, train, maintain, and equip Navy forces, develop and submit budgets,
                                        and develop required and sustainable levels of fleet readiness, with U.S.



                                        Page 3                                                            GAO-12-887 Military Readiness
Fleet Forces Command serving as the lead for fleet training requirements
and policies to generate combat-ready Navy forces. The Navy’s surface
type commanders—Commander, Naval Surface Force, U.S. Pacific Fleet
and Commander, Naval Surface Force, Atlantic have specific
responsibilities for the maintenance, training, and readiness of their
assigned surface ships. 2

Figure 1: Administrative Chain of Command




To meet the increased demands for forces following the events of
September 2001, the Navy established a force generation model—the
Fleet Response Plan—and in August 2006 the Navy issued a Fleet
Response Plan instruction. 3 The plan seeks to build readiness so the
Navy can surge a greater number of ships on short notice while
continuing to meet its forward-presence requirements. As depicted in
table 2, there are four phases in the Fleet Response Plan 27-month cycle
that applies to surface combatant and amphibious warfare ships. The four
Fleet Response Plan phases are (1) basic, or unit-level training; (2)
integrated training; (3) sustainment (which includes deployment); and (4)
maintenance.


2
 The Navy also has two air type commanders responsible for aircraft and aircraft carriers
assigned to their geographic areas of responsibility and two submarine type commanders.
3
  Office of the Chief of Naval Operations Instruction 3000.15, Fleet Response Plan (FRP)
(Aug. 31, 2006).




Page 4                                                     GAO-12-887 Military Readiness
Table 2: The 27-month Fleet Response Plan for Surface Combatant and Amphibious Warfare Ships

Phase        Basic                          Integrated                             Sustainment               Maintenance
Activities   Sea trials, ammunition         Multi-ship training         Continued advanced multi-ship        Includes major maintenance
             loading and unloading, unit-   exercises up to the carrier training, including carrier strike   overhauls
             level training                 strike group level          group exercises after
                                                                        deployment
                                              Source: GAO analysis of Navy data.



                                              In September 2009, the Commanders of U.S. Pacific Fleet and U.S. Fleet
                                              Forces directed Vice Admiral Balisle, USN-Ret., to convene and lead a
                                              Fleet Review Panel to assess surface force readiness. The Panel issued
                                              its report in February 2010. It stated that Navy decisions made to
                                              increase efficiencies throughout the fleet had adversely affected surface
                                              ship current readiness and life cycle material readiness. 4 Reducing
                                              preventative maintenance requirements and the simultaneous cuts to
                                              shore infrastructure were two examples of the detrimental efficiencies
                                              cited in the report. The report also stated that if the surface force stayed
                                              on the present course, surface ships would not reach their expected
                                              service lives. For instance, it projected that destroyers would achieve 25-
                                              27 years of service life instead of the 35-40 years expected. The report
                                              concluded that each decision to improve efficiency may well have been
                                              an appropriate attempt to meet Navy priorities at the time, but there was
                                              limited evidence to identify any changes that were made with surface
                                              force readiness as the top priority—efficiency was sought over
                                              effectiveness. The Fleet Review Panel made several maintenance,
                                              crewing, and training recommendations that it stated should be
                                              addressed not in isolation but as a circle of readiness. According to the
                                              report, it will take a multi-faceted, systematic solution to stop the decline
                                              in readiness, and begin recovery.

                                              We have previously reported on the Navy’s initiatives to achieve greater
                                              efficiencies and reduce costs. In June 2010, we issued a report regarding
                                              the training and crew sizes of cruisers and destroyers. In it we found that
                                              changes in training and reductions in crew sizes had contributed to
                                              declining material conditions on cruisers and destroyers. We



                                              4
                                               Fleet Review Panel, Final Report, Fleet Review Panel of Surface Force Readiness (Feb.
                                              26, 2010). For the purposes of this report, we are defining current readiness as ships’
                                              abilities to meet near term operational requirements and life cycle material readiness as
                                              ships’ abilities to achieve their expected service lives.




                                              Page 5                                                         GAO-12-887 Military Readiness
recommended that the Navy reevaluate its ship workload requirements
and develop additional metrics to measure the effectiveness of Navy
training. DOD agreed with these recommendations. 5 Also, in July 2011
we reported 6 on the training and manning information presented in the
Navy’s February 2011 7 report to Congress regarding ship readiness. The
Navy’s report included information on ships’ ability to perform required
maintenance tasks, pass inspection, and any projected effects on the
lifespan of individual ships. We concluded that the Navy’s report did not
provide discussion of data limitations or caveats to any of the information
it presented, including its conclusions and recommendations. However,
we found that the Navy did outline specific actions that it was taking or
planned to take to address the declines in readiness due to manning and
crew changes.

In January 2011, the commanders of U.S. Fleet Forces Command and
U.S. Pacific Fleet jointly instructed their type commanders to develop a
pilot program to “establish a sequenced, integrated, and building block
approach” to achieve required readiness levels. This pilot program began
in March 2011, and in March 2012, near the end of the pilot, the Navy
issued its Surface Force Readiness Manual, which details a new strategy
for optimizing surface force readiness throughout the Fleet Response
Plan. 8 The strategy calls for integrating and synchronizing maintenance,
training, and resources among multiple organizations such as Afloat
Training Groups and Regional Maintenance Centers.




5
  GAO, Military Readiness: Navy Needs to Reassess Its Metrics and Assumptions for Ship
Crewing Requirements and Training, GAO-10-592 (Washington, D.C.: June 2010).
6
 GAO, Military Readiness: Navy’s Report to Congress on the Impact of Training and
Crew Size on Surface Force Material Readiness, GAO-11-746R (Washington, D.C.: July
2011).
7
 Department of the Navy, Report to Congress: Impact of Training and Crew Size on
Surface Force Material Readiness (Washington, D.C.: February 2011).
8
 Commander, Naval Surface Force U.S. Pacific Fleet/Commander, Naval Surface Force
Atlantic Instruction 3502.3. Surface Force Readiness Manual (Mar. 9, 2012).




Page 6                                                   GAO-12-887 Military Readiness
                        For the period from 2008 to 2012, available data show variations in
Data Indicate           material readiness between different types of ships—such as material
Differences in          readiness differences between amphibious warfare ships and surface
                        combatants—but data limitations prevent us from making any conclusions
Material Readiness      concerning improvements or declines in the overall readiness of the
Between Ship Types,     surface combatant and amphibious warfare fleet during the period.
but Do Not Reveal       Through a variety of means and systems, the Navy collects, analyzes,
                        and tracks data that show the material condition of its surface ships—in
Readiness Trends Due    terms of both their current and life cycle readiness. Three of the data
to Limitations in the   sources the Navy uses to provide information on the material condition of
                        ships are casualty reports 9; Defense Readiness Reporting System – Navy
Data                    (DRRS-N) reports; and Board of Inspection and Survey (INSURV)
                        material inspection reports. None of these individual data sources are
                        designed to provide a complete picture of the overall material condition of
                        the surface force. However, the data sources can be viewed as
                        complementary and, when taken together, provide data on both the
                        current and life cycle material readiness of the surface force. For
                        example, some casualty report data must be updated every 72 hours and
                        provides information on individual pieces of equipment that are currently
                        degraded or out of commission. DRRS-N data is normally reported
                        monthly and focuses on current readiness by presenting information on
                        broader capability and resource areas, such as ship command, control,
                        and communications, rather than individual equipment. INSURV data is
                        collected less frequently—ships undergo INSURV inspections about once
                        every 5 years—but the data is extensive, and includes inspection results
                        for structural components, individual pieces of equipment, and broad
                        systems, as well as assessments of a ship’s warfighting capabilities. The
                        INSURV data is used to make lifecycle decisions on whether to retain or
                        decommission Navy ships. Casualty reports, DRRS-N data, and INSURV
                        reports are all classified when they identify warfighting capabilities of
                        individual ships. However, when casualty reports and INSURV
                        information is consolidated and summarized above the individual ship
                        level it is unclassified. Even summary DRRS-N data is classified, and
                        therefore actual DRRS-N data is not included in this unclassified report.
                        Table 3 provides additional details on each of the data sources.




                        9
                          Casualty reports reflect equipment malfunctions that impact a ship’s ability to support
                        required mission areas and suggest a deficiency in mission essential equipment. We drew
                        casualty report data from the Maintenance Figure of Merit program.




                        Page 7                                                     GAO-12-887 Military Readiness
Table 3: Navy’s Material Readiness Data Sources


 Data Source                 INSURV                                   DRRS-N                              Casualty Reports
 Data Owner                  Navy Board of Inspection and             Office of the Chief of Naval        United States Fleet Forces
                             Survey                                   Operations                          Command and United States
                                                                                                          Pacific Fleet
 Purpose and Types of Data   Determines fitness of Navy               Measures current readiness in       Provides a measure of the
 Collected                   ships for further service;               terms of capabilities, training,    material condition of a ship, in
                             assesses material condition to           and resources (people,              terms of deficiencies.
                             include equipment operating              equipment, supplies, and
                             capability and demonstrational           ordnance).
                             testing.
 Limitations                 Inspections may be delayed               Focus on ability to meet current    Casualty report data has not
                             due to deployment,                       mission requirements may not        been consistently reported and
                             maintenance requirements, or             reflect elements of material        can vary among ships due to
                             other factors such as                    condition requiring action in the   differences in leadership
                             coordination of schedules.               long term, such as corrosion.       philosophy, crew experience and
                                                                                                          training, and the number of
                                                                                                          independent assessments.
 Frequency of Reports        Generally, every 48 to 60                Monthly/Within 24 hours of a        As often as every 72 hours
                             months                                   significant change
                                        Source: GAO analysis of Navy data.

                                        Legend: INSURV = Board of Inspection and Survey and DRRS-N = Defense Readiness Reporting
                                        System–Navy.


                                        INSURV and casualty report data from January 2008 through March 2012
                                        consistently show differences in material readiness between different
                                        types of ships. As illustrated in Table 4, there are differences between
                                        frigates, destroyers, cruisers, and amphibious warfare ships in their
                                        overall INSURV ratings—which reflect ship abilities to carry out their
                                        primary missions; their INSURV Equipment Operational Capability
                                        scores—which reflect the material condition of 19 different functional
                                        areas; and their average numbers of casualty reports—which reflect
                                        material deficiencies in mission essential equipment. The differences
                                        within the average Equipment Operational Capability and casualty reports
                                        were found to be statistically significant. See additional details regarding
                                        the statistical significance of average Equipment Operational Capability
                                        scores and the average number of casualty reports in Appendix I.




                                        Page 8                                                                 GAO-12-887 Military Readiness
Table 4: INSURV and CASREP Data by Ship Type: January 2008 - March 2012

                                                                                                                                    Amphibious
 January 2008 to March 2012                                        Frigates         Destroyers                  Cruisers                 Ships
 INSURV Overall   Number of ships inspected                                  20                38                       18                      18
 Rating           ‘Unsatisfactory’                                           5%               3%                     22%                       6%
                  ‘Degraded’                                                 5%              14%                     11%                      28%
                  ‘Satisfactory’                                         90%                 84%                     67%                      67%
 INSURV Average EOC Scores                                             0.806               0.829                    0.786                    0.746
 Average Number of Casualty Reports per Ship                             11.8                14.5                    18.0                     22.3
                                        Source: GAO analysis of Navy data.

                                        Notes: Percentages may not add to 100 percent due to rounding.
                                        INSURV overall ratings are assigned based on the ship’s ability to materially carry out its missions. A
                                        rating of ‘satisfactory’ indicates an ability to carry out all missions, ‘degraded’ indicates an inability to
                                        carry out one mission; a rating of ‘unsatisfactory’ indicates an inability to carry out more than one
                                        mission.
                                        INSURV assigns Equipment Operational Capability scores to different functional areas of inspected
                                        ships’ systems. The scores range from 0 to 1 with 1 being the best score. The average Equipment
                                        Operational Capability scores shown in table 4 are based on GAO analysis of INSURV scores for all
                                        19 functional areas (18 areas in the case of amphibious ships). Differences in Equipment Operational
                                        Capability scores between ship types are statistically significant at the 95 percent confidence level.
                                        Differences in Average Number of Casualty Reports per Ship between ship types are statistically
                                        significant at the 95 percent confidence level. Differences in INSURV Overall Ratings were not
                                        assessed statistically. See additional statistical details in Appendix I.


                                        For example, the data in table 4 shows that, for the time period covered,
                                        the material condition of amphibious ships is generally lower than that of
                                        frigates and destroyers. For example, a lower percentage of amphibious
                                        warfare ships received overall “satisfactory” ratings in INSURV
                                        inspections than destroyers and frigates; likewise, amphibious ships had
                                        lower average INSURV Equipment Operational Capability scores than
                                        those two types of ships. Amphibious warfare ships also have on average
                                        more casualty reports per ship than destroyers and frigates. According to
                                        Navy officials, some of these differences may result from differences in
                                        the size, complexity, and age of the various types of ships. Likewise,
                                        cruisers have a lower material condition than that of destroyers. The data
                                        show that 22 percent of cruisers were rated “unsatisfactory” compared to
                                        3 percent of destroyers, and the average cruiser Equipment Operational
                                        Capability score of 0.786 was also lower than the destroyer score of
                                        0.829. Finally, the average of 18 casualty reports per cruiser was about
                                        24 percent higher than the 14.5 casualty reports per destroyer. DRRS-N
                                        data also show that there are readiness differences between the Navy’s
                                        different types of ships but the precise differences are classified and
                                        therefore are not included in this report.




                                        Page 9                                                                   GAO-12-887 Military Readiness
Material readiness data show some clear differences between types of
ships as shown in table 4. However, when we considered the surface
combatant and amphibious warfare ships in aggregate, we were unable
to make any conclusions concerning trends in the overall readiness of
these ships. One readiness measure—casualty reports—indicates that
the material readiness of these ships has declined but other readiness
measures show both upward and downward movement. Because of the
relatively small number of INSURV inspections conducted each year, it is
not possible to draw a conclusion about trends in the material readiness
of surface combatant and amphibious warfare ships from January 2008 to
March 2012 based on INSURV data.

Casualty report data from January 2008 to March 2012 show that there is
a significant upward trend in the average daily number of casualty reports
per ship for both surface combatants and amphibious warfare ships,
which would indicate declining material readiness. Specifically, the
average daily numbers of casualty reports per ship have been increasing
at an estimated rate of about 2 and 3 per year, respectively. Furthermore,
for both ship types, there is not a statistically significant difference in the
trend when comparing the periods before February 2010—when the Fleet
Review Panel’s findings were published—and after February 2010.
According to Navy officials, increases in casualty reports could be
reflective of the greater numbers of material inspections and evaluations
than in the past, which is likely to identify more material deficiencies and
generate more casualty reports. Figure 2 shows the increases in casualty
reports over time.




Page 10                                             GAO-12-887 Military Readiness
Figure 2: Average Number of Casualty Reports, per Ship, of Surface Combatant and Amphibious Warfare Ships by Quarter
from January 2008 through March 2012




                                       Table 5 shows the summary data for all the INSURV inspections of
                                       surface combatant and amphibious warfare ships that were conducted
                                       from January 2008 through March 2012. 10 Throughout the period, the
                                       data fluctuate in both an upward and downward direction. For example,
                                       the proportion of surface combatant and amphibious warfare ships rated
                                       ‘satisfactory’ fell 11 percent from 83 percent in 2008 to 72 percent in
                                       2010, and then increased to 77 percent in 2011. Average Equipment
                                       Operational Capability scores also fluctuated throughout the period—
                                       increasing in 2011 and declining in 2009, 2010, and 2012. As previously
                                       noted, because of the relatively small number of INSURV inspections
                                       conducted each year, it is not possible to draw a conclusion about trends
                                       in the material readiness of surface combatant and amphibious warfare
                                       ships between 2008 and 2012 based on INSURV data.


                                       10
                                          The data includes all first inspection data but does not include data from any re-
                                       inspections that were conducted following an “unsatisfactory” inspection.




                                       Page 11                                                       GAO-12-887 Military Readiness
Table 5: Average INSURV Equipment Operational Capability Scores for All Surface Combatant and Amphibious Warfare Ships
from January 2008 through March 2012

                                          Total Number and Percent of Surface                                  Average Equipment
                                        Combatant and Amphibious Warfare Ships                                         Operational
                                                                                                              Capability Scores for
                                                                                                               Surface Combatant
INSURV by                                                                                           ‘Unsatis-     and Amphibious
Calendar Year               Inspected             ‘Satisfactory’              ‘Degraded’             factory’       Warfare Ships
2008                                29                          24                    3                     2                     0.796
                                                             (83%)                (10%)                  (7%)
2009                                23                          19                    2                     2                     0.795
                                                             (83%)                 (9%)                  (9%)
2010                                25                          18                    5                     2                     0.785
                                                             (72%)                (20%)                  (8%)
                                     a
2011                               13                           10                    2                     1                     0.842
                                                             (77%)                (15%)                  (8%)
    b
2012                                 4                           3                    1                     0                     0.836
                                                             (75%)                (25%)                  (0%)
                                         Source: GAO analysis of Navy data.

                                         Note: Percentages may not add to 100 percent due to rounding.
                                         a
                                          Because the Board of Inspection and Survey conducts INSURV inspections of all Navy ships, the
                                         number of INSURV inspections of surface combatant and amphibious warfare ships can vary widely
                                         from year to year.
                                         b
                                         Data for 2012 goes through March 2012.


                                         The casualty report and INSURV data that we analyzed are consistent
                                         with the findings of the Navy’s Fleet Review Panel, which found that the
                                         material readiness of the Navy’s ships had been declining prior to 2010.
                                         Our analysis showed a statistically significant increase in casualty reports
                                         between 2008 and 2010 which would indicate a declining material
                                         condition. Although the statistical significance of the INSURV data from
                                         2008 to 2010 could not be confirmed due to the small number of ships
                                         that were inspected during this time period, that data showed declines in
                                         both the percentage of satisfactory inspections and average Equipment
                                         Operational Capability scores.




                                         Page 12                                                         GAO-12-887 Military Readiness
                          The Navy has taken steps intended to improve the readiness of its
Navy Has Acted to         surface combatant and amphibious warfare ships. However, it faces risks
Improve Readiness         to achieving full implementation of its recent strategy and has not
                          assessed these risks or developed alternative implementation
but Not Assessed          approaches to mitigate risks.
Risks to Achieving
Full Implementation
of Its Recent Strategy

Navy Steps to Improve     The Navy has taken several steps to help remedy problems it has
Surface Force Readiness   identified in regard to maintaining the readiness of its surface combatant
                          and amphibious warfare ships. In the past, material assessments,
                          maintenance, and training were carried out separately by numerous
                          organizations, such as the Regional Maintenance Centers and Afloat
                          Training Groups. According to the Navy, this sometimes resulted in
                          overlapping responsibilities and duplicative efforts. Further, the Navy has
                          deferred maintenance due to high operational requirements. The Navy
                          recognizes that deferring maintenance can affect readiness and increase
                          the costs of later repairs. For example, maintenance officials told us that
                          Navy studies have found that deferring maintenance on ballast tanks to
                          the next major maintenance period will increase costs by approximately
                          2.6 times, and a systematic deferral of maintenance may cause a
                          situation where it becomes cost prohibitive to keep a ship in service. This
                          can lead to early retirements prior to ships reaching their expected
                          service lives.

                          In the past few years the Navy has taken a more systematic and
                          integrated approach to address its maintenance requirements and
                          mitigate maintenance problems. For example, in November 2010 it
                          established the Surface Maintenance Engineering Planning Program,
                          which provides life cycle management of maintenance requirements,
                          including deferrals, for surface ships and monitors life cycle repair work.
                          Also, in December 2010 the Navy established Navy Regional
                          Maintenance Center headquarters, and began increasing the personnel
                          levels at its intermediate maintenance facilities in June 2011. More
                          recently, in March 2012, the Navy set forth a new strategy in its Surface




                          Page 13                                            GAO-12-887 Military Readiness
Force Readiness Manual. 11 This strategy is designed to integrate material
assessments, evaluations, and inspections with maintenance actions and
training and ensure that surface ships are (1) ready to perform their
current mission requirements and (2) able to reach their expected service
lives. 12 The manual addresses the need for the organizations involved in
supporting ship readiness to take an integrated, systematic approach to
eliminate redundancy, build training proficiency to deploy at peak
readiness, and reduce costs associated with late identified work.

According to the Surface Force Readiness Manual, readiness is based
upon a foundation of solid material condition that supports effective
training. In line with this integrated maintenance and training approach,
the new strategy tailors the 27-month Fleet Response Plan by adding a
fifth phase that is not included in the Fleet Response Plan, the
shakedown phase. This phase allows time between the end of the
maintenance phase and the beginning of the basic phase to conduct a
material assessment of the ship to determine if equipment conditions are
able to support training. In addition, the new strategy shifts the cycle’s
starting point from the basic phase to the sustainment phase to support
the deliberate planning required to satisfactorily execute the maintenance
phase and integrate maintenance and training for effective readiness.
Under the new strategy, multiple assessments, which previously certified
ship readiness all throughout the Fleet Response Plan cycle, will now be
consolidated into seven readiness evaluations at designated points within
the cycle. Because each evaluation may have several components, one
organization will be designated as the lead and will be responsible for
coordinating the evaluation with the ship and other assessment teams,
thereby minimizing duplication and gaining efficiencies through
synchronization. Figure 3 shows the readiness evaluations that occur
within each phase of the strategy’s notional 27-month cycle.




11
   Commander, Naval Surface Force U.S. Pacific Fleet/Commander, Naval Surface Force
Atlantic Instruction 3502.3, Surface Force Readiness Manual (Mar. 9, 2012).
12
  The Surface Force Readiness Manual applies to most surface ships in the fleet.
However, aircraft carriers and littoral combat ships are governed by separate instructions.




Page 14                                                      GAO-12-887 Military Readiness
Figure 3: Notional 27-month Fleet Response Plan Cycle




                                        As previously noted, development of the Navy’s new strategy began with
                                        a pilot program. The pilot was conducted on ships from both the East and
                                        West coasts beginning in March 2011. Initial implementation of the new
                                        strategy began in March 2012 and is currently staggered, with ships’
                                        schedules being modified to support the strategy’s integration of training,
                                        manning, and maintenance efforts. 13 Ships that were not involved in the
                                        pilot program will begin implementing the strategy when they complete
                                        the maintenance phase of the Fleet Response Plan cycle. The Navy
                                        plans to fully implement the new strategy in fiscal year 2015 (i.e. to have
                                        all surface ships operating under the strategy and resources needed to
                                        conduct the strategy’s required tasks in place). While the Surface Force
                                        Readiness Manual states that providing a standard, predictable path to
                                        readiness is one of the tenets of the Navy’s new strategy, it also
                                        acknowledges that circumstances may arise that will require a deviation
                                        from the notional 27-month cycle.


Navy Has Not Assessed                   Certain factors could affect the Navy’s ability to fully implement its
Risks to Implementing its               strategy, but the Navy has not assessed the risks to implementation or
Strategy as Planned or                  developed alternatives. As we have previously reported, 14 risk
                                        assessment can provide a foundation for effective program management.
Developed Alternatives
                                        Risk management is a strategic process to help program managers make



                                        13
                                          Currently, no ships have applied the strategy throughout an entire 27-month Fleet
                                        Response Plan cycle.
                                        14
                                          GAO, Standards for Internal Control in the Federal Government, GAO/AIMD-00-21.3.1
                                        (Washington, D.C.: November 1999).




                                        Page 15                                                    GAO-12-887 Military Readiness
decisions about assessing risk, allocating finite resources, and taking
actions under conditions of uncertainty. To carry out a comprehensive risk
assessment, program managers need to identify program risks from both
external and internal sources, estimate the significance of these risks,
and decide what steps should be taken to best manage them. Although
such an assessment would not assure that program risks are completely
eliminated, it would provide reasonable assurance that such risks are
being minimized.

As the Navy implements its new surface force readiness strategy one risk
we identified involves the tempo of operations. While the strategy
acknowledges circumstances may arise that require a deviation from the
27-month Fleet Response Plan cycle, it also states that predictability is
necessary in order to synchronize the maintenance, training, and
operational requirements. However, the tempo of operations is currently
higher than planned for in the Fleet Response Plan. According to Navy
officials, this makes execution of the strategy challenging. High
operational tempos pose challenges because they could delay the entry
of some ships into the strategy as well as the movement of ships through
the strategy. For example, some ships that have been operating at
increased tempos, such as the Navy’s ballistic missile defense cruisers
and destroyers, have not followed the Navy’s planned 27-month cycle.
Navy officials told us that requirements for ballistic missile defense ships
are very high leading to quick turnarounds between deployments. They
said, in some cases, ships may not have time for the maintenance or full
basic and integrated/advanced training phases. The manual notes that
ships without an extended maintenance period between deployments will
remain in the sustainment phase. According to Navy guidance, the
maintenance phase is critical to the success of the Fleet Response Plan
since this is the optimal period in which lifecycle maintenance activities—
major shipyard or depot-level repairs, upgrades, and modernization
installations—occur. Thus, ships with a high operational tempo that do not
enter the maintenance phase as planned will have lifecycle maintenance
activities deferred, which could lead to increased future costs. Further,
ships that do not enter the maintenance phase may be delayed entering
into the strategy. This delay would be another risk to the implementation
of the Navy’s new readiness strategy and ships’ lifecycle readiness.

In addition, the Navy’s plan to decrease the number of surface combatant
and amphibious warfare ships through early retirements is likely to
increase operational tempos even further for many ships that remain in
the fleet. DOD’s fiscal year 2013 budget request proposes the early
retirement of seven Aegis cruisers and two amphibious ships in fiscal


Page 16                                           GAO-12-887 Military Readiness
                                        years 2013 and 2014. When fewer ships are available to meet a given
                                        requirement, ships must deploy more frequently. Table 6 shows the ships
                                        that the Navy plans to retire early, their ages at retirement, and their
                                        homeports.

Table 6: Navy Ships Planned For Early Retirement

                                                                                          Age at
Hull No.         Name                        Year to be Retired               Retirement (years)      Homeport
CG 63            Cowpens                     FY 13                                             22     Yokosuka, Japan
CG 68            Anzio                       FY 13                                             21     Norfolk, VA
CG 69            Vicksburg                   FY 13                                             21     Mayport, FL
CG 73            Port Royal                  FY 13                                             19     Pearl Harbor, HI
CG 64            Gettysburg                  FY 14                                             23     Mayport, FL
CG 65            Chosin                      FY 14                                             23     Pearl Harbor, HI
CG 66            Hue City                    FY 14                                             23     Mayport, FL
LSD 41           Whidbey Island              FY 14                                             29     Norfolk, VA
LSD 46           Tortuga                     FY 14                                             24     Sasebo, Japan
                                        Source: GAO analysis of Navy data.

                                        Legend: FY = Fiscal Year


                                        Also, recent changes in national priorities, 15 which call for an increased
                                        focus on the Asia-Pacific region that places a renewed emphasis on air
                                        and naval forces, make it unlikely that operational tempos will decline. At
                                        the same time, DOD will still maintain its defense commitments to Europe
                                        and other allies and partners.

                                        In addition to the risks posed by high operational tempos, several
                                        supporting organizations currently have staffing levels that are below the
                                        levels needed to fulfill their roles in the new integrated readiness strategy.
                                        For example, Navy Afloat Training Group officials have identified the
                                        staffing levels required to fully support the strategy, and reported that they
                                        need an additional 680 personnel to fully execute the new strategy. As of
                                        August 2012, the Navy plans to reflect its funding needs for 410 of the
                                        680 personnel in its fiscal year 2014 budget request and for the remaining
                                        270 in subsequent requests. Under the new strategy, the Afloat Training



                                        15
                                          Department of Defense, Sustaining U.S. Global Leadership: Priorities For 21st Century
                                        Defense (Jan. 3, 2012).




                                        Page 17                                                     GAO-12-887 Military Readiness
Groups provide subject matter experts to conduct both material, and
individual and team training. Previously the Afloat Training Groups used a
“Train the Trainer” methodology, which did not require the same number
of trainers because ships’ crews included their own system experts to
train the crew and the Afloat Training Groups just trained the ships’
trainers. Afloat Training Group Pacific officials told us that there are times
when the training events that can be offered—to ships currently under the
strategy and/or ships that have not yet implemented the strategy—are
limited because of their staffing level gaps. Current staffing allows
executing all portions of the Basic Phase in select mission areas only.
Other mission areas are expected to gain full training capability as staffing
improves over the next several years. Until then, the Afloat Training
Group officials plan to schedule training events within the limited
capability mission areas based on a prioritized hierarchy.

Further, Surface Maintenance Engineering Planning Program officials told
us they are also short of staff. They said they need 241 staff to perform
their requirements, but currently have 183 staff. They stated that while
current budget plans include funding to reach the 241 staffing level in
2013, it will be reduced below the 241 requirement in 2014.

As with the Afloat Training Groups and Surface Maintenance Engineering
Planning Program, officials at the Navy Regional Maintenance Center
headquarters told us they currently lack the staff needed to fully execute
the ship readiness assessments called for in the new strategy. Ship
readiness assessments evaluate both long-term lifecycle maintenance
requirements (e.g. preservation to prevent structural corrosion) and
maintenance to support current mission requirements (e.g. preventative
and corrective maintenance for the Aegis Weapons System). According
to the officials, ship readiness assessments allow them to deliberately
plan the work to be done during major maintenance periods and prioritize
their maintenance funds. The goal is for ships to receive all the prescribed
ship readiness assessments in fiscal year 2013. However, Navy officials
stated that they are evaluating the impact of recent readiness assessment
revisions on changes in the Regional Maintenance Center’s funding and
personnel requirements.

The Navy has not undertaken a comprehensive assessment of the impact
of high operational tempos, staffing shortages, or any other risks it may
face in implementing its new readiness strategy, nor has it developed
alternatives to mitigate any of these risks. The Navy does recognize in its
strategy that circumstances may arise that require ships to deviate from
the 27-month Fleet Response Plan cycle and has considered the


Page 18                                            GAO-12-887 Military Readiness
                      adjustments to training that would need to take place in such a case.
                      However, the strategy does not discuss, nor identify plans to mitigate,
                      maintenance challenges that could arise from delays in full
                      implementation. We believe the risks we identified may delay full
                      implementation, which could lead to continued deferrals of lifecycle
                      maintenance, increasing costs and impacting the Navy’s ability to achieve
                      expected service lives for its ships.


                      Today’s fleet of surface combatant and amphibious warfare ships
Conclusions           provides core capabilities that enable the Navy to fulfill its missions. In
                      order to keep this fleet materially and operationally ready to meet current
                      missions and sustain the force for future requirements, the Navy must
                      maximize the effective use of its resources and ensure that its ships
                      achieve their expected service lives. Full implementation of its new
                      strategy, however, may be delayed if the Navy does not account for the
                      risks it faces and devise plans to mitigate against those risks. Navy
                      organizations have taken individual steps to increase their staffing levels,
                      but the Navy has yet to consider alternatives if the integration of
                      assessment, maintenance, and training under the strategy is delayed.
                      Without an understanding of risks to full implementation and plans to
                      mitigate against them, the Navy is likely to continue to face the challenges
                      it has encountered in the past, including the increased costs that arise
                      from deferring maintenance and the early retirement of ships. This could
                      impact the Navy’s ability to meet its long-term commitments. Further,
                      ongoing maintenance deferrals—and early retirements that increase the
                      pace of operations for the remaining surface force—could potentially
                      impact the Navy’s ability to meet current missions.


                      To enhance the Navy’s ability to implement its strategy to improve surface
Recommendations for   force material readiness, we recommend that the Secretary of Defense
Executive Action      direct the Secretary of the Navy to take the following two actions:

                      •   Develop a comprehensive assessment of the risks the Navy faces in
                          implementing its Surface Force Readiness Manual strategy, and
                          alternatives to mitigate risks. Specifically, a comprehensive risk
                          assessment should include an assessment of risks such as high
                          operational tempos and availability of personnel.
                      •   Use the results of this assessment to make any necessary
                          adjustments to its implementation plan.




                      Page 19                                           GAO-12-887 Military Readiness
                     In written comments on a draft of this report, DOD partially concurred with
Agency Comments      our recommendations. Overall, DOD stated it agrees that risk assessment
and Our Evaluation   is an important component of program management, but does not agree
                     that a comprehensive assessment of the risks associated with
                     implementation of the Navy’s Surface Force Readiness strategy is either
                     necessary or desirable. It also stated that existing assessment processes
                     are sufficient to enable adjustments to implementation of the strategy.
                     DOD also noted several specific points. For example, according to DOD,
                     a number of factors impact surface ship readiness and some of those
                     factors, such as budgetary decisions, emergent operational requirements,
                     and unexpected major ship repair events are outside of the Navy’s direct
                     control. DOD further stated that the strategy, and the organizations that
                     support the strategy, determine and prioritize the full readiness
                     requirement through reviews of ship material condition and assess the
                     risk of any gaps between requirements and execution, as real world
                     events unfold. DOD also noted that the Surface Ship Readiness strategy
                     has a direct input into the annual Planning, Programming, Budgeting, and
                     Execution (PPBE) process. It stated that its position is that execution of
                     the strategy and PPBE process adequately identify and mitigate risks.
                     DOD further believes that a separate one-time comprehensive
                     assessment of risks, over and above established tracking mechanisms, is
                     an unnecessary strain on scarce resources. Moreover, DOD stated that
                     the Navy now has the technical resources available, using a disciplined
                     process, to inform risk-based decisions that optimize the balance
                     between current operational readiness and future readiness tied to
                     expected service life through the standup of its Surface Maintenance
                     Engineering Planning Program and Commander Navy Regional
                     Maintenance Centers. Specifically, DOD noted documenting and
                     managing the maintenance requirement is now a fully integrated process.
                     According to DOD, the Navy’s Surface Type Commanders identify and
                     adjudicate risks to service life and this approach is consistent with
                     fundamental process discipline and risk management executed by the
                     submarine and carrier enterprises. Finally, according to DOD, the Navy is
                     continually assessing progress in achieving the strategy and has the
                     requisite tools in place to identify changes in force readiness levels that
                     may result from resource constraints, and will adjust the process as
                     necessary to ensure readiness stays on track.

                     As described in our report, we recognize that the Navy has taken a more
                     systematic and integrated approach to address its maintenance
                     requirements and mitigate problems, and specifically cite the Surface
                     Readiness strategy, and actions such as standing up Surface
                     Maintenance Engineering Planning Program and Commander Navy


                     Page 20                                          GAO-12-887 Military Readiness
Regional Maintenance Centers. We also recognize that the Navy
conducts various assessments of ship readiness and considers resource
needs associated with implementing the strategy as part of the budget
process. However, we do not agree that any of the current assessments
or analyses offer the type of risk assessment that our report recommends.
For example, the PPBE process does not address the specific risk that
high operational tempos pose to implementation of the strategy nor does
it present alternatives for mitigating this risk. Also, despite the ongoing
efforts by Surface Maintenance Engineering Planning Program and
Commander Navy Regional Maintenance Centers officials to document
and manage the maintenance requirement of the surface force in an
integrated process, both organizations are currently under staffed. The
challenges identified in our report, including high operational tempos and
current organizational staffing levels, have hindered the Navy’s ability to
achieve the desired predictability in ships’ operations and maintenance
schedules, as called for in its strategy. Given factors such as the Navy’s
plan to decrease the number of ships as well as changes in national
priorities that place a renewed emphasis on naval forces in the Asia
Pacific region, these challenges we identified are unlikely to diminish in
the near future, and there could be additional risks to the strategy’s
implementation. Without an understanding of the full range of risks to
implementing its strategy and plans to mitigate them, the Navy is likely to
continue to face the challenges it has encountered in the past, including
increased costs that arise from deferring maintenance and the early
retirement of ships. Therefore, we continue to believe that a
comprehensive risk assessment is needed.


We are sending copies of this report to appropriate congressional
committees, the Secretary of Defense, the Secretary of the navy, and
other interested parties. In addition, the report is available at no charge
on the GAO website at http://www.gao.gov

If you or your staff have any questions about this report, please contact
me at (202) 51209619. Contact points for our Offices of Congressional




Page 21                                            GAO-12-887 Military Readiness
Relations and Public Affairs may be found on the last page of this report.
GAO staff who made key contributions to this report are listed in
appendix III.




Sharon Pickup
Director
Defense Capabilities and Management




Page 22                                           GAO-12-887 Military Readiness
Appendix I: Scope and Methodology
             Appendix I: Scope and Methodology




             To assess how the Navy evaluates the material readiness of its surface
             combatant and amphibious warfare ships and the extent to which data
             indicate trends or patterns in the material readiness of these ships, we
             interviewed officials from the Commander Naval Surface Force, U.S.
             Pacific Fleet, Commander Naval Surface Force, U.S. Atlantic Fleet, as
             well as visiting a number of ships, to include the USS Leyte Gulf (CG 55),
             USS Arleigh Burke (DDG 51), USS San Antonio (LPD 17), and USS
             Higgins (DDG-76). We obtained and analyzed Navy policies and
             procedures for determining surface force readiness, as well as various
             studies and reports on the Navy’s material readiness process. We
             obtained and analyzed material readiness data from the Navy’s Board of
             Inspection and Survey (INSURV) as well as the United States Fleet
             Forces Command (USFF). We also met with Navy officials from the
             Board of Inspection and Survey and the United States Fleet Forces
             Command to complement our data analysis, and observed the INSURV
             material inspection of the USS Cole (DDG 67).

             We limited our data analysis to the period from January 2008 to March
             2012 in order to cover a period of approximately two years prior to, and
             two years following, publication of the Fleet Review Panel of Surface
             Force Readiness report. Specifically, we analyzed data for the Navy’s
             guided-missile cruisers (CG 47 class), guided-missile destroyers (DDG 51
             class), frigates (FFG 7 class), amphibious assault ships (LHA 1 and LHD
             1 classes), amphibious transport dock ships (LPD 4 and LPD 17 classes),
             and dock landing ships (LSD 41 and LSD 49 classes).

             We analyzed data from three of the primary data sources the Navy uses
             to provide information on the material condition of ships: casualty reports;
             Board of Inspection and Survey (INSURV) material inspection reports;
             and the Defense Readiness Reporting System – Navy (DRRS-N) reports.
             None of these individual data sources are designed to provide a complete
             picture of the overall material condition of the surface force.

             From the Board of Inspection and Survey we met with INSURV officials
             and observed an INSURV inspection onboard the USS Cole (DDG 67)
             conducted on December 12, 2011 and December 14, 2011. We obtained
             all INSURV initial material inspection reports dating from 2008 through
             2012 for cruisers, destroyers, frigates, and amphibious warfare ships. We
             then extracted relevant data from those reports, including INSURV’s
             overall assessment of the material condition of these surface ships
             (satisfactory, degraded, unsatisfactory), Equipment Operational Capability
             scores for the different functional areas of ships systems (on a 0.00 to
             1.00 scale), and dates when these ships were inspected. Although


             Page 23                                           GAO-12-887 Military Readiness
Appendix I: Scope and Methodology




INSURV provides an overall assessment, we included Equipment
Operational Capability scores to provide additional insight into the
material condition of a ship’s systems. Overall assessments focus on a
ship’s material readiness to perform primary missions. As such, while
multiple individual systems may be in an unsatisfactory condition
(Equipment Operational Capability scores below 0.80 are considered
“degraded,” while those below 0.60 are considered “unsatisfactory”), the
ship may receive an overall rating of “satisfactory” due to its material
readiness to meet its primary missions. Figure 4 below shows the process
for determining INSURV ratings, with that segment for determining
Equipment Operational Capability scores highlighted.

Figure 4: INSURV Process for Determining Overall Material Inspection Ratings




We analyzed both INSURV overall ratings and Equipment Operational
Capability scores to identify differences in material readiness between
types of ships. To determine if there were statistically significant
differences in the Equipment Operational Capability scores among four
types of ships (cruisers, destroyers, frigates, and amphibious ships), we
took the average of the various Equipment Operational Capability scores
for each ship and conducted a one-way analysis of variance (ANOVA). In
addition, we conducted post-hoc multiple comparison means tests to
determine which ship types, if any, differed. Based on the results of this
analysis, we concluded that there were statistically significant differences
in the average Equipment Operational Capability score between the four
ship types (p-value < 0.0001). Specifically, the average for amphibious
ships was significantly lower, at the 95 percent confidence level, than the
average scores for cruisers, destroyers, and frigates and the average for
cruisers was significantly lower than the average for destroyers.




Page 24                                               GAO-12-887 Military Readiness
Appendix I: Scope and Methodology




In presenting our results, we standardized relevant data where necessary
in order to present a consistent picture. For example, in 2010, the Board
of Inspection and Survey moved from rating those ships with the worst
material condition as “unfit for sustained combat operations” to rating
them as “unsatisfactory.” We have treated both these ratings as
“unsatisfactory” in this report.

We obtained casualty report data for the same set of ships from the
United States Fleet Forces Command office responsible for the Navy’s
Maintenance Figure of Merit program. Casualty report data provided
average daily numbers of casualty reports per ship for cruisers,
destroyers, frigates, and amphibious warfare ships. We then used these
daily averages to identify differences between ship types and to calculate
and analyze changes in these daily averages from month to month and
quarter to quarter.

We assessed the reliability of casualty report data presented in this
report. Specifically, the Navy provided information based on data
reliability assessment questions we provided, which included information
on an overview of the data, data collection processes and procedures,
data quality controls, and overall perceptions of data quality. We received
documentation about how the systems are structured and written
procedures in place to ensure that the appropriate material readiness
information is collected and properly categorized. Additionally, we
interviewed the Navy officials to obtain further clarification on data
reliability and to discuss how the data were collected and reported into
the system. After assessing the data, we determined that the data were
sufficiently reliable for the purposes of assessing the material condition of
Navy surface combatant and amphibious warfare ships, and we discuss
our findings in the report.

To determine if there were statistically significant differences in the daily
averages among the four types of ships (cruisers, destroyers, frigates,
and amphibious warfare ships), we conducted a one-way analysis of
variance (ANOVA), followed by post-hoc multiple comparison means
tests to determine which ship types, if any, differed. Based on the results
of this analysis we concluded that there were statistically significant
differences in the daily averages between the four ship types (p-value <
0.0001), and specifically, the daily average for amphibious warfare ships
was significantly higher, at the 95 percent confidence level, than the daily
average for cruisers, destroyers, and frigates.




Page 25                                            GAO-12-887 Military Readiness
                                      Appendix I: Scope and Methodology




                                      Next we analyzed the changes in the daily averages to determine if there
                                      was an increasing, decreasing, or stationary trend from month to month.
                                      We did this separately for surface combatant ships (cruisers, destroyers,
                                      and frigates) and amphibious warfare ships. To estimate the trends, we
                                      conducted a time-series regression analysis to account for the correlation
                                      in the average daily scores from month to month. We then tested the
                                      estimated trends for significant changes after February 2010 — when the
                                      Fleet Review Panel’s findings were published – using the Chow test for
                                      structural changes in the estimated parameters. We fit a time-series
                                      regression model with autoregressive errors (AR lag of 1) to monthly data
                                      for both surface combatants and amphibious ships to account for the
                                      autocorrelation between monthly observations. The total R-squared, a
                                      measure that reflects how well the model predicts the data, was 0.9641
                                      for the surface combatant ships model and 0.9086 for the amphibious
                                      warfare ships model which indicate both models fit the data well. A
                                      summary of the model parameters is given in the table below.

Table 7: Summary of Regression Model Parameters

                                   Model:                    Model:
Model:                    Total R-Squared      Regression R-Squared             Parameter       Estimate             p-value
Surface Combatants                 0.9641                              0.6063   Intercept          9.8221            <0.0001
                                                                                Month (trend)      0.1770            <0.0001
Amphibious Ships                   0.9086                              0.4013   Intercept         16.1106            <0.0001
                                                                                Month (trend)      0.2438            <0.0001
                                      Source: GAO analysis of Navy data.



                                      We observed statistically significant positive trends in the daily average
                                      for both models. Specifically, the estimated trend for the daily average
                                      number of casualty reports per ship increased at a rate of about 2 per
                                      year (0.1770 * 12 months) for surface combatant ships and about 3 per
                                      year (0.2438 * 12 months) for amphibious warfare ships. In addition,
                                      neither of the tests for significant structural changes in the model
                                      parameters after February 2010 were significant at the 95 percent
                                      confidence level. Based on this, we concluded that there is not enough
                                      evidence to suggest there were significant changes in the estimated
                                      trends after February 2010 for either ship type.

                                      We analyzed data from the Defense Readiness Reporting System-Navy
                                      (DRRS-N), which contains data that is normally reported monthly and
                                      focuses on current readiness by presenting information on broader
                                      capability and resource areas. We obtained classified DRRS-N readiness



                                      Page 26                                                   GAO-12-887 Military Readiness
Appendix I: Scope and Methodology




data for all surface combatant and amphibious warfare ships from
January 2008 through March 2012. DRRS-N data showed upward and
downward movements between 2008 and 2012, but we did not evaluate
the statistical significance of these movements.

To determine the extent to which the Navy has taken steps intended to
improve the readiness of its surface combatant and amphibious warfare
ships including efforts to implement its recent strategy, we reviewed
relevant Navy instructions on Navy material readiness, including the
strategy—the Surface Force Readiness Manual—to identify the policies
and procedures required by the Navy to ensure its surface ships are
ready to perform their current mission requirements and reach their
expected service lives. We also reviewed prior GAO work on risk
management and collected and analyzed data on the resources needed
to implement the strategy, and interviewed relevant officials.

To gain a better understanding of how the Navy’s independent
maintenance, training, and manning initiatives will be integrated into the
new strategy, we collected data on the staffing resources needed to
implement the strategy and met with officials from the Commander Navy
Regional Maintenance Center, the Surface Maintenance Engineering
Planning Program, and the Afloat Training Group Pacific. We focused
primarily on the Navy’s maintenance initiatives because we have
previously reported on its training and manning initiatives. 1

In addition, we met with personnel on board four Navy ships to obtain
their views on the impact of the Navy’s maintenance initiatives, such as
readiness assessments and material inspections, on the readiness of
these ships. Specifically, we visited the USS Leyte Gulf (CG 55), USS
Arleigh Burke (DDG 51), USS San Antonio (LPD 17), and USS Higgins
(DDG 76). We also discussed initial implementation of the new strategy
with personnel on board the USS Higgins.

We also met with officials from the Commander Naval Surface Force,
U.S. Pacific Fleet who are responsible for administering the strategy for



1
  Footnote numbers start over at “1” in appendices. GAO, Military Readiness: Navy’s
Report to Congress on the Impact of Training and Crew Size on Surface Force Material
Readiness, GAO-11-746R (Washington, D.C.: July 7, 2011) and Military Readiness: Navy
Needs to Reassess Its Metrics and Assumptions for Ship Crewing Requirements and
Training, GAO-10-592 (Washington, D.C.: June 9, 2010).




Page 27                                                 GAO-12-887 Military Readiness
Appendix I: Scope and Methodology




surface ships on the West coast and in Hawaii and Japan to discuss
timeframes for transitioning ships into the strategy, challenges
implementing the strategy, and plans to address any risks that may occur
during the strategy’s implementation. Additionally, we obtained written
responses to our questions from these officials and from officials at the
Commander Naval Surface Force, U.S. Atlantic Fleet who administer the
strategy for surface ships on the East coast.

Finally, we reviewed prior GAO work on risk assessment as well as Navy
testimony on the readiness of its ships and aircraft and Department of
Defense strategic guidance on the key military missions the department
will prepare for and budget priorities for fiscal years 2013-2017.

We conducted this performance audit from July 2011 to September 2012,
in accordance with generally accepted government auditing standards.
Those standards require that we plan and perform the audit to obtain
sufficient, appropriate evidence to provide a reasonable basis for our
findings and conclusions based on our audit objectives. We believe that
the evidence obtained provides a reasonable basis for our findings and
conclusions based on our audit objectives.




Page 28                                          GAO-12-887 Military Readiness
Appendix II: Comments from the Department
             Appendix II: Comments from the Department
             of Defense



of Defense




             Page 29                                     GAO-12-887 Military Readiness
Appendix II: Comments from the Department
of Defense




Page 30                                     GAO-12-887 Military Readiness
Appendix II: Comments from the Department
of Defense




Page 31                                     GAO-12-887 Military Readiness
Appendix III: GAO Contacts and Staff
                  Appendix III: GAO Contacts and Staff
                  Acknowledgments



Acknowledgments

                  Sharon L. Pickup, (202) 512-9619 or pickups@gao.gov
GAO Contacts
                  In addition to the contact named above, key contributors to this report
Staff             were Michael Ferren (Assistant Director), Jim Ashley, Mary Jo Lacasse,
Acknowledgments   David Rodriguez, Michael Silver, Amie Steele, Nicole Volchko, Erik
                  Wilkins-McKee, Nicole Willems, and Ed Yuen.




(351621)
                  Page 32                                         GAO-12-887 Military Readiness
GAO’s Mission         The Government Accountability Office, the audit, evaluation, and
                      investigative arm of Congress, exists to support Congress in meeting its
                      constitutional responsibilities and to help improve the performance and
                      accountability of the federal government for the American people. GAO
                      examines the use of public funds; evaluates federal programs and
                      policies; and provides analyses, recommendations, and other assistance
                      to help Congress make informed oversight, policy, and funding decisions.
                      GAO’s commitment to good government is reflected in its core values of
                      accountability, integrity, and reliability.

                      The fastest and easiest way to obtain copies of GAO documents at no
Obtaining Copies of   cost is through GAO’s website (http://www.gao.gov). Each weekday
GAO Reports and       afternoon, GAO posts on its website newly released reports, testimony,
                      and correspondence. To have GAO e-mail you a list of newly posted
Testimony             products, go to http://www.gao.gov and select “E-mail Updates.”

Order by Phone        The price of each GAO publication reflects GAO’s actual cost of
                      production and distribution and depends on the number of pages in the
                      publication and whether the publication is printed in color or black and
                      white. Pricing and ordering information is posted on GAO’s website,
                      http://www.gao.gov/ordering.htm.
                      Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
                      TDD (202) 512-2537.
                      Orders may be paid for using American Express, Discover Card,
                      MasterCard, Visa, check, or money order. Call for additional information.
                      Connect with GAO on Facebook, Flickr, Twitter, and YouTube.
Connect with GAO      Subscribe to our RSS Feeds or E-mail Updates. Listen to our Podcasts.
                      Visit GAO on the web at www.gao.gov.
                      Contact:
To Report Fraud,
Waste, and Abuse in   Website: http://www.gao.gov/fraudnet/fraudnet.htm
                      E-mail: fraudnet@gao.gov
Federal Programs      Automated answering system: (800) 424-5454 or (202) 512-7470

                      Katherine Siggerud, Managing Director, siggerudk@gao.gov, (202) 512-
Congressional         4400, U.S. Government Accountability Office, 441 G Street NW, Room
Relations             7125, Washington, DC 20548

                      Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800
Public Affairs        U.S. Government Accountability Office, 441 G Street NW, Room 7149
                      Washington, DC 20548




                        Please Print on Recycled Paper.