oversight

Defense Management: Actions Needed to Evaluate the Impact of Efforts to Estimate Costs of Reports and Studies

Published by the Government Accountability Office on 2012-05-10.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

United States Government Accountability Office
Washington, DC 20548



           May 10, 2012

           The Honorable Carl Levin
           Chairman
           The Honorable John McCain
           Ranking Member
           Committee on Armed Services
           United States Senate

           The Honorable Howard P. “Buck” McKeon
           Chairman
           The Honorable Adam Smith
           Ranking Member
           Committee on Armed Services
           House of Representatives

           Subject: Defense Management: Actions Needed to Evaluate the Impact of Efforts to
           Estimate Costs of Reports and Studies

           Citing long-term fiscal challenges affecting the federal government, in May 2010, the
           Secretary of Defense directed the Department of Defense (DOD) to undertake a
           departmentwide initiative to assess how the department is staffed, organized, and
           operated with the goal of reducing excess overhead costs and reinvesting these
           savings in sustaining DOD’s current force structure and modernizing its weapons
           portfolio. 1 The Secretary’s initiative targeted both shorter- and longer-term
           improvements and set specific goals and targets for achieving cost savings and
           efficiencies. The initiative was organized along four tracks, each of which had a
           different focus (see enc. I). The fourth track focused on specific areas where DOD
           could take immediate action to reduce inefficiencies and overhead, in particular, to
           reduce headquarters and support bureaucracies and to instill a culture of cost
           consciousness and restraint in the department. As part of the fourth track, the
           Secretary of Defense announced a number of specific initiatives, including actions
           intended to address the need to reduce or eliminate reporting requirements for DOD
           reports and studies. For example, in his August 9, 2010, speech announcing the
           overall efficiency initiative, the Secretary of Defense stated that the department is
           “awash in taskings for reports and studies” and directed several specific actions that,
           according to the press release accompanying the announcement, were intended to



           1
               Remarks as delivered by former Secretary of Defense Robert M. Gates, Abilene, Kansas, May 8, 2010.


           1                                                             GAO-12-480R Defense Management
“combat the enormous amounts of taskings for reports and studies.” 2 In his remarks,
the Secretary noted that there is little basis to determine whether the value gained is
worth the considerable time and resources expended to generate reports and
studies. With respect to specific actions, the Secretary directed DOD and its
components to track the approximate cost of preparing DOD reports and studies and
publish the cost on the front cover of each report or study. He also called for a
comprehensive review of all oversight reports and use of the results to reduce the
volume of reports and studies generated internally while engaging with Congress on
ways to meet their needs while working together to reduce the number of reports.


In September 2010, the Secretary of Defense tasked DOD’s Cost Assessment and
Program Evaluation (CAPE) office with implementing the Secretary’s requirement to
publish the cost of preparing DOD reports and studies. By November 2010, the
CAPE office had developed a cost estimating tool and corresponding guidance on
using the tool, which described the categories of costs to be estimated and identified
the types of reports and studies for which costs were to be estimated. 3 The following
month, the Secretary of Defense issued a memorandum requiring the use of the tool
beginning on February 1, 2011. 4


In House Report Number 112-78, which accompanied a bill for the National Defense
Authorization Act for Fiscal Year 2012, 5 the House Committee on Armed Services
commended DOD for implementing a process for collecting an estimate of resources
required to generate internally and externally required reports and agreed that
additional transparency would be useful for decision makers when determining the
utility of various reporting requirements. The committee also observed that any tool
used to collect costs was only as useful as the inputs received and directed GAO to
conduct an assessment of DOD’s methodology and tools for collecting cost data on
both internally and externally required reports and to submit a report, including any
recommendations needed to improve the data collection, transparency, and utility of
the tool. Specifically, we evaluated DOD’s approach to estimate the costs of
selected reports and studies, including (1) whether DOD entities are capturing and
presenting costs in a manner consistent with relevant cost estimating best practices;
(2) the status of DOD’s efforts to track whether organizations are developing cost
estimates for all required types of reports and studies; and (3) whether DOD has
evaluated the usefulness of its efforts to estimate the costs of reports and studies as
one of the means for achieving the Secretary’s intended purpose of increasing the
transparency of costs, reducing or eliminating reporting requirements, and instilling a
culture of cost consciousness across DOD.
2
  Remarks as delivered by former Secretary of Defense Robert M. Gates, The Pentagon, August 9, 2010, and
DOD News Release No. 706-10, “Sec. Gates Announces Efficiency Initiatives” (Aug. 9, 2010).
3
  Guidance for using the tool has been updated and the most recent version can be found in DOD Cost
Guidance Group, Studies and Reports Guidance Document: DOD Cost Guidance Portal (Sept. 16, 2011), and
DOD Cost Guidance Group, Cost Guidance Frequently Asked Questions: DOD Cost Guidance Portal (Sept. 16,
2011).
4
  Secretary of Defense Memorandum, Consideration of Costs in DOD Decision-Making (Dec. 27, 2010).
5
  H.R. Rep. No.112-78 (2011), which accompanied a bill for the National Defense Authorization Act for Fiscal
Year 2012, Pub. L. No. 112-81 (2012).


    2                                                       GAO-12-480R Defense Management
To determine whether the tool is being used to capture and present costs in a
manner consistent with relevant cost estimating best practices, we reviewed the cost
estimating tool and compared it to characteristics of cost estimating best practices
identified in GAO’s Cost Estimating and Assessment Guide 6 and related managerial
cost accounting guidance. 7 GAO’s Cost Estimating and Assessment Guide reflects
best practices for developing and managing cost estimates for capital programs.
Because DOD is estimating noncapital costs, we assessed DOD’s tool used to
generate cost estimates against the best practices most applicable to noncapital
costs as well as against relevant managerial cost accounting standards and
guidance, which we determined are consistent with the selected best practices.
Specifically, we determined the extent to which the estimates resulting from use of
the tool were “comprehensive,” “well documented,” and “accurate”—essential
elements for high-quality, reliable cost estimates. We also interviewed DOD officials
to corroborate our understanding of their use of the tool and its corresponding
guidance. All data elements we assessed are sufficiently reliable for the purpose of
assessing DOD’s cost estimating tool against best practices. We had originally
intended to review a representative sample of DOD reports and studies for which
cost estimates had been generated and published. However, as described in this
report, DOD does not currently have the ability to distinguish between real estimates
and estimates that were developed while individuals were using the tool for training
purposes. As a result, DOD could not provide us with a comprehensive list of all
reports and studies for which costs had been estimated. However, DOD officials
were able to provide us information on nine reports prepared for Congress for which
cost estimates had been generated in 2011. We reviewed these nine reports and
interviewed the key officials responsible for developing the cost estimates for them.
As described in this report, in some cases, officials were not able to readily retrieve
documentation; therefore, we were unable to fully evaluate the extent to which they
consistently applied DOD’s guidance for estimating costs. To describe the status of
DOD’s efforts to track whether organizations are developing cost estimates for all
required types of reports and studies, we reviewed guidance and interviewed DOD
officials to assess the nature of their tracking efforts. To determine whether DOD has
evaluated the usefulness of its efforts to estimate the costs of reports and studies as
one of the means for achieving the Secretary’s intended purpose of increasing the
transparency of costs, reducing or eliminating reporting requirements, and instilling a
culture of cost consciousness across DOD, we reviewed DOD guidance. We also
interviewed DOD officials responsible for developing and using the tool, including the
individuals who generated the nine reports that we reviewed to determine any steps


6
  GAO, GAO Cost Estimating and Assessment Guide: Best Practices for Developing and Managing Capital
Program Costs, GAO-09-3SP (Washington, D.C: March 2009). GAO developed this guide in order to establish a
consistent methodology that can be used across the federal government for developing, managing, and
evaluating cost estimates.
7
  Federal Accounting Standards Advisory Board, Statement of Federal Financial Accounting Standards (SFFAS)
No. 4, Managerial Cost Accounting Concepts and Standards for the Federal Government (Washington, D.C.:
July 1995), and DOD Financial Management Regulation 7000.14R, Accounting Policy and Procedures, vol.
4 (2012), see ch. 19, “Managerial Cost Accounting,” and ch. 22, “Cost Finding.”




    3                                                      GAO-12-480R Defense Management
taken to evaluate the impact of the cost estimating effort, including whether they had
obtained feedback from internal and external decision makers.


We conducted this performance audit between October 2011 and May 2012 in
accordance with generally accepted government auditing standards. Those
standards require that we plan and perform the audit to obtain sufficient, appropriate
evidence to provide a reasonable basis for our findings and conclusions based on
our audit objectives. We believe that the evidence obtained provides a reasonable
basis for our findings and conclusions based on our audit objectives. More detailed
information on our scope and methodology is provided in enclosure II.


Results in Brief


DOD is estimating and publishing approximate costs for selected types of internally
and externally required reports, but in some cases its approach is not fully consistent
with relevant cost estimating best practices and cost accounting standards.
Specifically, DOD entities have been directed to use the cost estimating tool to
capture marginal costs of activities associated with completing a report or study that
would not have been performed otherwise. These costs consist of certain manpower
costs (such as the prorated salaries of military and civilian personnel based on the
time they spent) and nonlabor costs (such as contract services, travel, or printing). In
comparing DOD’s approach to (1) GAO’s Cost Estimating and Assessment Guide—
which states that high-quality, reliable cost estimates should be comprehensive, well
documented, and accurate—and (2) relevant accounting standards, we found the
following.

Comprehensive. GAO’s cost guide states that cost estimates should include all
costs, but allows flexibility for the estimates to exclude costs where information is
limited as long as steps are taken to clearly define and document what costs are
included or excluded. DOD’s guidance on using the tool defines and documents
decisions to include certain manpower and all nonlabor costs in its calculation. In
addition, it documents the decision to exclude other manpower costs, such as those
for health care and training expenses. It also provides broad examples of some
types of activities to consider in estimating costs, but leaves it up to the discretion of
the individuals generating the cost estimate to decide which types of activities to
include or exclude. As a result, based on the nine reports we reviewed, we found
inconsistencies in the types of activities individuals decided to include when
developing their cost estimates. For example, we found that for six of the nine
reports we reviewed, some individuals calculated the manpower costs for activities
associated with coordinating the report through the final review while others did not.
In addition, according to relevant accounting standards, appropriate procedures and
practices should be established to enable, among other things, the interpretation and
communication of cost information. However, when presenting the cost estimate on
the front cover of a report or study, the language DOD uses does not provide


   4                                             GAO-12-480R Defense Management
information on what costs are included or excluded. Specifically, it does not indicate
that the estimate reflects only certain marginal manpower and nonlabor costs.
Without further explanation, this wording could be subject to misinterpretation such
that recipients of the reports or studies may assume that all costs are included in the
estimate when that is not the case.

Well documented. GAO’s Cost Estimating and Assessment Guide states that cost
estimates should be easily traceable to source documents. While DOD’s guidance
on using the tool states that individuals should be prepared to explain how they
developed the cost estimate, it does not include a requirement for individuals to
retain source documentation. In practice, we found that of the nine reports we
reviewed, individuals were able to easily retrieve documentation for the estimates
prepared for three reports, but not for the other six. As a result, DOD may not have
the information needed so that others can understand how a cost estimate was
derived and readily replicate it.

Accurate. GAO’s Cost Estimating and Assessment Guide further states that cost
estimates should have all cost inputs checked to verify that they are accurate.
However, we found that DOD’s guidance on using the tool has no requirement to
independently verify the costs used to generate the cost estimate. In practice, the
individual generating the cost estimate is responsible for ensuring its accuracy. As a
result, without independent verification, DOD cannot ensure that cost estimates are
accurate and reliable.

DOD currently lacks a means to ensure that organizations are developing cost
estimates for all required types of reports and studies, but is taking steps to enhance
its ability to monitor the preparation of reports and studies to satisfy reporting
requirements, including those for which a cost estimate is required to be generated.
DOD’s guidance on using the tool identified 10 specific types of reports or studies
that require a cost estimate. The guidance states that the DOD component preparing
a report or study is responsible for ensuring that a cost estimate, if required, is
included. However, the guidance does not include any process or requirement to
track whether organizations are developing required cost estimates. According to
CAPE officials who developed the tool, they were not directed to ensure that cost
estimates have been developed for all required reports and studies, and they added
that the cost estimating tool was not designed to track reports and studies that
require a cost estimate. These same officials noted that the magnitude of DOD’s
reporting requirements makes it challenging to identify the universe of these
requirements and to track the completion of reports and studies to meet them,
including whether cost estimates have been generated. In the past year, DOD has
initiated efforts to improve its visibility over its internal and external reporting
requirements. In March 2011, the Secretary of Defense issued a memorandum
requiring the Assistant Secretary of Defense for Legislative Affairs to enhance that
office’s database capability and further automate the process of tracking external
reports to Congress and required the Office of the Director of Administration and
Management to track internal reports. The Office of Legislative Affairs enhanced its
Congressional Hearings and Reporting Requirements Tracking System by adding a


   5                                            GAO-12-480R Defense Management
data field to record the estimated costs of reports or studies generated by DOD
components. To gain greater visibility of internal reporting requirements, the Office of
the Director of Administration and Management tasked Washington Headquarters
Services with developing a repository for tracking internal reports. According to a
Washington Headquarters Services official, the repository has a field to capture
whether an internal report or study has a cost estimate. Currently, the repository is in
development and has been populated with reports that date back 3 years.

DOD has not evaluated and currently does not plan to evaluate the usefulness of its
efforts to estimate the costs of selected reports and studies as one of the means for
achieving the Secretary of Defense’s intended purpose of increasing the
transparency of costs, reducing or eliminating reporting requirements, and instilling a
culture of cost consciousness across DOD. According to GAO’s Standards for
Internal Control in the Federal Government, 8 activities such as assessments need to
be established to monitor performance, and managers need to compare actual
performance against targets and then analyze any significant differences so that
appropriate action can be taken to ensure that organizational goals are met.
However, under current guidance, no requirement exists for DOD to evaluate the
impact of its cost estimating efforts, such as whether these efforts have prompted
internal or external decision makers to consider cost as a factor in deciding whether
to establish a reporting requirement for DOD. Without evaluating the impact of its
efforts, including seeking the views of internal and external decision makers, DOD
does not have the information it needs to assess whether the time and effort it is
investing to develop cost estimates is having the desired effect of achieving greater
transparency, reducing reporting requirements, and raising cost consciousness, as
intended by the Secretary.


We are making a recommendation to the Secretary of Defense to take steps to
evaluate DOD’s effort to estimate costs to determine whether that effort is having the
desired effect of achieving greater transparency, reducing or eliminating reporting
requirements, and raising cost awareness. Based on the results of this evaluation, if
DOD plans to continue the effort to estimate costs for selected reports and studies,
we recommend that the Secretary modify the current guidance or otherwise take
steps to improve DOD’s cost estimating approach. In commenting on a draft of our
report, DOD partially concurred with our recommendations, except it disagreed with
one of the steps we recommended to improve its cost estimating approach. Our
comments on specific points made in DOD’s letter begin on page 14 and DOD’s
comments are reprinted in enclosure V. The department also provided technical
comments on our draft report, which we incorporated as appropriate.




8
 GAO, Standards for Internal Control in the Federal Government, GAO/AIMD-00-21.3.1 (Washington, D.C.:
November 1999).


    6                                                     GAO-12-480R Defense Management
Background


In September 2010, the CAPE office, in conjunction with the military services,
established the Cost Guidance Group to implement the Secretary of Defense’s
requirement to estimate the cost of preparing DOD reports and studies and publish
that cost on the front cover. This group, consisting of representatives from the Office
of the Director of Administration and Management, the Joint Staff, and the military
departments, among others, developed the guidance and tool that the Secretary of
Defense directed DOD and its components to begin using by February 1, 2011.
Figure 1 provides a timeline of key events related to the CAPE office’s
implementation of the Secretary’s requirement.

Figure 1: Timeline of the Development and Implementation of DOD’s Cost Estimating Tool




The tool, also referred to as a calculator, is web based and maintained on an
unclassified DOD website (see enc. III for a snapshot of the tool). DOD entities use
the tool for training purposes as well as to generate cost estimates that will be
published on reports and studies. The corresponding guidance for the tool is also
maintained on the website and identifies the types of reports and studies that require
a cost estimate, instructs individuals on how to access and utilize the tool, and
identifies manpower and nonlabor costs that could be considered for inclusion in the
cost estimate, among other things. The guidance and the tool are updated
incrementally with both minor and major changes.

Guidance on using the tool identifies 10 types of reports or studies that require a
cost estimate. Specifically, a cost estimate is required if the report or study is

    •   to be provided to the Secretary of Defense,
    •   to be provided to the Deputy Secretary of Defense,
    •   to be provided to Congress,
    •   to be included in the Defense Technical Information Center holdings,
    •   executed by DOD contractors,


    7                                                  GAO-12-480R Defense Management
    •    executed by federally funded research and development centers,
    •    executed by a board created by DOD,
    •    executed by a board created by law,
    •    executed by a commission created by DOD, or
    •    executed by a commission created by law.


In addition, although it is not required, guidance on using the tool recommends that a
cost estimate be generated for all reports and studies.

DOD Captures Certain Costs for Selected Reports and Studies, but Its
Approach Does Not Fully Reflect Relevant Cost Estimating Best Practices and
Cost Accounting Standards

DOD is estimating and publishing approximate costs for selected internally and
externally required reports, but in some cases, its approach is not fully consistent
with relevant cost estimating best practices and cost accounting standards.

DOD Estimates and Publishes Certain Marginal Costs as Directed by the Secretary
of Defense

DOD entities use the cost estimating tool to capture marginal costs—costs for
activities that would not otherwise have been performed had the report or study not
been developed—associated with preparing and completing a report or study. Costs
to be captured consist of (1) manpower costs, such as the prorated salaries based
on time spent by military personnel and DOD civilians to develop and publish a
report or study, 9 and (2) nonlabor costs, such as costs for travel, contractor-related
services, and printing. Once the estimate is generated, individuals include the dollar
amount of the estimate on the front cover of the corresponding report or study.
Specifically, the language on the front cover states that “Preparation of this
report/study cost the Department of Defense a total of approximately $_______ in
Fiscal Year _____” and includes the date that the cost estimate was generated and
a unique reference identification number for the report or study. CAPE officials
responsible for developing the tool stated that the Secretary of Defense approved
this approach for estimating and publishing the costs of reports and studies and
emphasized that the Secretary’s intent was to develop an easy to use,
nonburdensome approach for estimating costs. As a result, the tool was not
designed with the level of precision of a more formal cost estimating process.

DOD’s Approach for Estimating Costs Is Not Fully Consistent with Relevant Best
Practices and Cost Accounting Standards

We evaluated DOD’s approach for estimating costs against some of the best
practices contained in GAO’s Cost Estimating and Assessment Guide, which states

9
 The tool automatically calculates salary costs for military personnel based on composite rates and civilian
personnel based on the General Schedule, neither of which includes all costs associated with manpower.


    8                                                          GAO-12-480R Defense Management
that high-quality and reliable cost estimates should be comprehensive, well
documented, and accurate. In addition, we compared DOD’s approach with
Statement of Federal Financial Accounting Standards No. 4 and relevant portions of
DOD’s Financial Management Regulation, which state that appropriate procedures
and practices should be established to enable, among other things, the interpretation
and communication of cost information. We found that DOD’s approach in using the
tool to estimate costs is not fully consistent with relevant best practices and cost
accounting standards, as discussed below.

         Comprehensive

While GAO’s Cost Estimating and Assessment Guide states that cost estimates
should include all costs, it allows flexibility for the estimates to exclude costs where
information is limited. 10 In such cases, the decisions on what costs are to be
included or excluded should be clearly defined and documented in the ground rules
and assumptions used to develop the cost estimate. 11 Consistent with this guide,
DOD’s guidance on using the tool defines and documents decisions to include or
exclude certain manpower costs and to include all nonlabor costs. For example, it
identifies specific manpower costs that are excluded from the calculation, such as
costs associated with health care and training. Additionally, the guidance provides
broad examples of some types of activities to consider in estimating costs, but states
that individuals should use their best judgment in determining what costs to include.
In calculating manpower costs, the guidance indicates that if individuals developing
a report or study collected data or took other actions that they would have done even
if the report or study was not required, the costs associated with these activities
should not be included in the cost estimate. For example, the guidance states
activities that are part of an individual’s normal job activities—such as attending
meetings; coordinating reports, studies, or proposals; and completing security
reviews of documents—should not be included in the manpower calculation. On the
other hand, if a study or report required activities that would not have otherwise been
undertaken, costs associated with these activities should be included in the estimate
of manpower costs. The guidance also provides examples of nonlabor activities
individuals could consider in developing the cost estimate, such as costs associated
with contractors, printing, software or hardware purchases, lease or rental fees for
space, and travel.

Ultimately, the guidance leaves it up to the individuals generating the cost estimate
to decide which types of activities to include or exclude in the cost estimate. As a
result, in our review of nine reports that DOD prepared for Congress, we found
inconsistencies in the types of activities individuals decided to include when
developing their cost estimates (see enc. IV for a description of these reports, which
ranged in estimated costs from about $3,000 to over $2.3 million). For example, for
six of the nine reports we reviewed, we found that individuals included in their
manpower calculations the time they spent coordinating the report for review and

10
  Similarly, SFFAS No. 4 permits flexibility when measuring and reporting costs for special purpose cost studies.
11
  Grounds rules represent a common set of agreed-on estimating standards that provide guidance and minimize
conflicts in definitions. Assumptions represent a set of judgments about past, present, or future conditions
postulated as true in the absence of positive proof.


     9                                                        GAO-12-480R Defense Management
approval and the time that their supervisors spent finalizing it. For the other reports
we reviewed, costs for this activity were not included. One of the officials that
decided to include time spent on coordinating the report in the cost estimate stated
that DOD’s guidance was unclear as to whether coordination was limited to the
review and approval of a report. Therefore, given the significant amount time he
spent coordinating the development of the report (e.g., time spent clarifying
guidance or coordinating inputs), he decided to include those costs in the cost
estimate. We also found instances where costs for nonlabor activities that DOD’s
guidance states individuals could consider in developing estimates were not
included even though costs were incurred. Specifically, we found that for four of the
reports we reviewed, officials told us that costs had been incurred for printing but
were not included in the cost estimate. They stated that they did not include them
because they believed their printing costs were so small that the level of effort
needed to calculate those costs outweighed the value of including them in the
estimate. In some cases, officials were not able to readily retrieve documentation;
therefore, we were unable to fully evaluate the extent to which they consistently
applied DOD’s guidance for estimating costs.

In addition, according to DOD’s Financial Management Regulation and incorporated
portions of Statement of Federal Financial Accounting Standards No. 4, appropriate
procedures and practices should be established to enable, among other things, the
interpretation and communication of cost information. While DOD has developed
boilerplate language for use when presenting the cost estimate on the front cover of
a report or study, the language does not provide information on what costs are
included or excluded. Specifically, it does not indicate that the estimate reflects only
certain marginal manpower and nonlabor costs. Rather, DOD includes a general
statement that says “Preparation of this report/study cost the Department of Defense
a total of approximately $_______ in Fiscal Year _____.” Without further
explanation, this wording could be subject to misinterpretation such that recipients of
the reports or studies may assume that all costs are included in the estimate when
that is not the case.

        Well Documented

GAO’s Cost Estimating and Assessment Guide states that high-quality, reliable cost
estimates should be easily traceable to source documents. While DOD’s guidance
on using the tool states that individuals should be prepared to explain how they
developed a cost estimate, it does not include a requirement for individuals to retain
source documentation. According to DOD officials responsible for the development
of the tool, the tool is not a database and therefore was never intended to be used to
capture references to supporting documents. When talking with officials responsible
for developing cost estimates, we found that for three of the nine reports, they were
able to easily retrieve the documentation they used to generate the cost estimate.
For example, some individuals were able to provide us with e-mails containing data
on the amount of time individuals spent working on the report or study. Officials
responsible for three of the other six reports stated that they could not easily provide
the information because they had moved on to other positions and no longer had



   10                                           GAO-12-480R Defense Management
access to their records. The remaining officials stated that with time, they might be
able to find and provide the source documentation.

        Accurate

GAO’s Cost Estimating and Assessment Guide further states that high-quality,
reliable cost estimates should have all cost inputs independently checked to verify
that they are accurate. The guidance on using the tool does not include any process
or requirement for another individual to independently verify the manpower and
nonlabor costs used to generate the cost estimate. In the absence of any guidance,
CAPE and service officials responsible for administering the tool stated that they had
not been assigned responsibility for validating or vetting inputs used to develop the
cost estimate. According to these same officials, the individual generating the cost
estimate is responsible for ensuring its accuracy. In some cases, individuals who
were responsible for generating cost estimates for the nine reports we reviewed
stated that they included the information used to develop the cost estimates when
they submitted the reports to their supervisors for approval.

In the absence of clearly defined parameters as to which activities should be
covered, full disclosure of the scope of the estimates, the availability of source
documentation, and a means to independently verify the estimates, DOD does not
have the information it needs to readily replicate the estimates and assure itself that
the estimates are consistent, accurate, and reliable.

DOD Does Not Currently Have a Means to Ensure That Required Types of
Reports and Studies Have Cost Estimates, but Has Efforts Under Way to
Improve Its Visibility


DOD currently lacks a means to ensure that organizations are developing cost
estimates for all required types of reports and studies, but is taking steps to enhance
its ability to monitor the preparation of reports and studies to satisfy reporting
requirements, including those for which a cost estimate is required to be generated.
DOD’s guidance on using the tool identified 10 specific types of reports or studies
that require a cost estimate. The guidance also states that the DOD office preparing
the report or study is responsible for ensuring that a cost estimate, if required, is
included. However, the guidance does not include any process or requirement to
track whether organizations are developing required cost estimates.


According to CAPE officials who developed the tool, they were not directed to
ensure that cost estimates have been developed for all required reports and studies.
They added that the cost estimating tool was not designed to track reports and
studies that require a cost estimate. For example, officials stated that while the tool
stores information each time a user generates a cost estimate, they cannot
distinguish between real estimates and those that were developed while individuals
were using the tool for training purposes. Therefore, when we requested a list of all
the reports and studies for which individuals had prepared cost estimates, they were


   11                                           GAO-12-480R Defense Management
unable to provide us such a list. CAPE officials also noted that the magnitude of
DOD’s reporting requirements makes it challenging to identify the universe of these
requirements and to track the completion of reports and studies to meet them,
including whether cost estimates have been generated.


In the past year, DOD has taken steps to improve the department’s visibility over its
internal and external reporting requirements. For example, in March 2011, the
Secretary of Defense issued a memorandum requiring the Assistant Secretary of
Defense for Legislative Affairs to enhance that office’s database capability and
further automate the process of tracking external reports to Congress, and required
the Office of the Director of Administration and Management to track internal
reports. 12 The Office of Legislative Affairs enhanced its Congressional Hearings and
Reporting Requirements Tracking System 13 by adding a data field to record the
estimated costs of reports or studies generated by DOD components. To gain
greater visibility of internal reporting requirements, the Office of the Director of
Administration and Management tasked Washington Headquarters Services with
developing a repository for tracking internal reports. According to a Washington
Headquarters Services official, the repository has a field to capture whether an
internal report or study has a cost estimate. Currently, the repository is in
development and has been populated with reports that date back 3 years. However,
officials stated that this only captures reporting requirements involving multiple DOD
components and does not reflect all of DOD’s internal reporting requirements. The
official noted that Washington Headquarters Services is planning to send out a data
call to all of DOD in an effort to capture the universe of internal reporting
requirements.

DOD Lacks a Plan to Evaluate the Impact of Its Efforts to Estimate Costs


DOD has not evaluated and currently does not plan to evaluate the usefulness of its
efforts to estimate the costs of selected reports and studies as one of the means for
achieving the Secretary of Defense’s intended purpose of increasing the
transparency of costs, reducing or eliminating reporting requirements, and instilling a
culture of cost consciousness across DOD. According to GAO’s Standards for
Internal Control in the Federal Government, activities such as assessments need to
be established to monitor performance and managers need to compare actual
performance against targets and then analyze any significant differences so that
appropriate action can be taken to ensure that organizational goals are met.
However, under current guidance, no requirement exists for DOD to evaluate the
impact of its cost estimating efforts, such as whether it has prompted internal or
external decision makers to consider cost as a factor in deciding whether to
establish a reporting requirement for DOD. In addition, CAPE officials responsible for
developing and administering the tool stated they have not been otherwise directed
to take steps nor have they taken steps to conduct any such evaluation, including
12
   Secretary of Defense Memorandum, Track Four Efficiency Initiatives Decisions (Mar. 14, 2011).
13
   The Congressional Hearings and Reporting Requirements Tracking System is DOD’s official system used to
track congressional processes.


     12                                                    GAO-12-480R Defense Management
reaching out to internal and external decision makers to obtain their views. CAPE
officials stated that anecdotally they have heard some officials remark that they
believe the tool has increased the transparency of costs and improved cost
awareness. However, most of the DOD officials who generated cost estimates for
the nine reports we reviewed stated that they had not sought or received any
feedback from internal or external decision makers about the cost estimates
published on the front covers of the reports or studies. Without evaluating the impact
of its efforts, including seeking the views of internal and external decision makers,
DOD does not have the information it needs to assess whether the time and effort it
is investing to develop cost estimates is having the desired effect of achieving
greater transparency, reducing reporting requirements, and raising cost
consciousness, as intended by the Secretary.

Conclusions

When the Secretary of Defense directed DOD to track the cost of preparing reports
and studies, his intention was to generate an approximate cost that could raise
awareness of the costs associated with the level of effort DOD expends in
generating reports or studies. In response to the Secretary’s directions, officials from
the CAPE office, in conjunction with officials from other DOD offices, expended time
and effort to develop an approach to estimate the cost of reports and studies. DOD
components are also expending time and effort to use the tool to develop cost
estimates for selected reports and studies. Because of this investment, it is
important that DOD officials have an understanding of the impact publishing a cost
estimate is having on internal and external decision makers. Further, there are
actions that DOD can take to improve its approach so that it better aligns with key
characteristics of cost estimating best practices—actions that would help DOD
ensure that it has the information it needs to readily replicate the estimate and
assure itself and the recipients of the reports or studies that the estimates are clearly
defined and consistent, accurate, and reliable. Without first evaluating the impact its
cost estimating approach is having, DOD officials will not be positioned to determine
whether the resources invested in generating and publishing the approximate costs
are having the desired effect or whether they should take steps to improve their cost
estimating approach.

Recommendations for Executive Action

To determine whether its effort to estimate costs is having the desired effect of
achieving greater transparency, reducing or eliminating reporting requirements, and
raising cost awareness, we recommend that the Secretary of Defense take steps to
evaluate the effort, including reaching out to internal and external decision makers to
obtain their views on how it has affected their decisions to establish internal and
external reporting requirements and whether they have a common understanding of
what types of costs are being reported.

Based on the results of this evaluation, if DOD plans to continue the effort to
estimate costs for selected reports and studies, we recommend that the Secretary


   13                                            GAO-12-480R Defense Management
modify the current guidance or otherwise take steps to improve DOD’s cost
estimating approach, including
    • clarifying those activities that should be included in the estimate as they are
      incurred, including time spent at every level to review and approve the report
      or study;
    • requiring that the language on the front cover of reports and studies make
      clear that the estimate is based on certain marginal manpower and nonlabor
      costs;
    • requiring that source documentation used to develop the cost estimate is
      retained and easily accessible for review purposes; and
    • establishing and implementing a verification process to provide reasonable
      assurance of consistency and completeness of cost inputs used to develop
      the cost estimate.

Agency Comments and Our Evaluation

DOD provided written comments on a draft of this report. The full text of DOD’s
written comments is reprinted in enclosure V. The department also offered technical
comments, which we incorporated as appropriate.

Overall, DOD stated that it largely agreed with GAO’s recommendation to evaluate
the effectiveness of estimating and publishing the costs of DOD studies and reports
and that the department remains committed to improving the existing cost estimating
guidance, methods, and tools associated with this policy. However, DOD expressed
concern that we had inaccurately characterized the objectives of its cost estimating
policy. For example, DOD believed that the title of our review—Department of
Defense’s (DOD) Reporting Cost Data Collection—suggested that the department
was collecting cost data on reports. DOD also stated that our draft report
inaccurately stated that an objective of the current policy is to reduce or eliminate
reporting requirements. According to DOD, the department’s effort to reduce or
eliminate reporting requirements is a separate endeavor entailing the formal
submission of legislation to Congress, and is inappropriately linked in our draft report
to the initiative to estimate and publish the costs of DOD studies and reports.

We disagree with DOD’s view that we have inappropriately linked the initiative to
estimate and publish costs to the objective of eliminating and reducing reporting
requirements. We recognize that DOD has undertaken a specific effort to review all
reports stemming from both external and internal reporting requirements, and have
clarified our report accordingly. However, in our review of various DOD documents,
including prepared remarks made by the Secretary and a press release from his
office, it is clear that the review cited by DOD was one of several actions, which
included the effort to estimate and publish costs, that were intended to address the
volume of reporting requirements. We, therefore, modified the report to provide
additional contextual information from these sources to amplify the Secretary’s
intent. In the introductory paragraphs of the report, we now include information
contained in the Secretary of Defense’s August 9, 2010, speech announcing the
efficiency initiative as well as the accompanying press release further describing his
remarks. Specifically, we note the following: In his August 9, 2010 speech


   14                                           GAO-12-480R Defense Management
announcing the overall efficiency initiative, the Secretary of Defense stated that the
department is “awash in taskings for reports and studies” and directed several
specific actions that according to the press release accompanying the
announcement, were intended to “combat the enormous amounts of taskings for
reports and studies.” 14 In his remarks, the Secretary noted that there is little basis to
determine whether the value gained is worth the considerable time and resources
expended to generate reports and studies. With respect to specific actions, the
Secretary directed DOD and its components to track the approximate cost of
preparing DOD reports and studies and publish the cost on the front cover of each
report or study. He also called for a comprehensive review of all oversight reports
and use of the results to reduce the volume of reports and studies generated
internally while engaging Congress on ways to meet needs while working together to
reduce the number of reports. With regard to the title of our review, this title was
used for administrative purposes, including its use in our letter to notify DOD of our
work. We note that in several places throughout the draft report, we describe DOD’s
effort as a cost estimating approach rather than an effort to collect cost data.

With regard to our recommendation that DOD evaluate its effort to estimate the
costs of reports and studies, including reaching out to internal and external decision
makers to obtain their views on how it has affected their decisions to establish
internal and external reporting requirements and whether they have a common
understanding of what types of costs are being reported, DOD partially concurred.
DOD agreed that there could be value gained in evaluating its efforts but stated that
any such evaluation must avoid linking the efforts of this policy to objectives that are
unrelated to its original intent. Specifically, DOD stated that its policy was not
designed to reduce or eliminate reporting requirements, which it characterized as a
separate effort. In addition, DOD stated that its policy was not designed to apply cost
estimating best practices from GAO’s Cost Estimating and Assessment Guide. It
noted that DOD’s guidance and methodology for estimating costs differ from criteria
used to estimate costs for other DOD decision making. It further stated that DOD
would not support a change to the rigor associated with estimating the costs of DOD
studies and reports unless it is guaranteed that the marginal value gained by
increasing the level of accuracy outweighs the additional efforts required to produce
more accurate cost estimates.

With regard to engaging with internal and external decision makers to better
understand the effectiveness of the current cost estimating policy, DOD stated that
this process had already begun. Specifically, DOD noted that members of the
department have communicated internally with DOD employees and externally with
non-DOD employees to understand ways in which the value gained by the policy
could be improved. In its comments, DOD stated that it had received evidence that
the value gained by the policy is significant and cited specific examples to illustrate
its point. For example, DOD noted that contact had occurred between the Chairman
of the House Armed Services Committee and the Secretary of Defense regarding
the costs associated with a particular DOD report, which led to agreement that the

14
 Remarks as delivered by former Secretary of Defense Robert M. Gates, The Pentagon, August 9, 2010, and
DOD News Release No. 706-10, “Sec. Gates Announces Efficiency Initiatives” (Aug. 9, 2010).


     15                                                   GAO-12-480R Defense Management
report’s quality required improvement and could be produced at lower costs. In
addition, DOD cited actions taken by DOD decision makers to expand the scope of
the policy to include additional types of reports or to establish mechanisms to ensure
that reports to Congress are not transmitted without a cost estimate. DOD also noted
that DOD decision makers have contacted CAPE staff to commend the Secretary’s
effort to raise cost awareness and improve transparency through the use of the cost
estimating guidance, methods, and tools. Finally, DOD stated that it will continue to
monitor the effectiveness of the current policy by engaging with DOD employees,
senior-level DOD staff, and individuals outside of DOD.

We are encouraged that the department agrees that there could be value gained in
evaluating efforts to raise cost awareness and improve transparency of costs by
estimating and publishing costs of the DOD studies and reports. However, we
disagree with DOD’s characterization of the underlying intent of the evaluation that
we recommended. As discussed previously, the Secretary included the effort to track
and publish costs for reports and studies among efforts intended to address the
volume of taskings for reports and studies. Therefore, we believe that in any
evaluation of the effort, it would be essential to specifically obtain the views of
decision makers on the benefits of estimating report costs, including how such
efforts affected their decision making on whether to call for future reports or studies.
As stated in our report and as reflected in the examples that DOD described in its
response, we recognize that DOD has obtained anecdotal feedback from internal
and external sources regarding the cost estimating effort. Given the time and effort
that DOD components are expending to use the tool to develop cost estimates, we
continue to believe that DOD needs to develop an evaluation approach that will give
it a more complete picture of how decision makers have been influenced by the cost
estimating policy. Without such information, it will continue to be difficult for DOD to
assess the value of the policy, including whether the time and effort it is expending
to estimate costs is a worthwhile investment.

We further note that our recommendation that DOD evaluate its cost estimating
effort did not suggest that such an evaluation would assess whether DOD should
apply cost estimating best practices. Rather, we recommended that DOD first
evaluate whether its effort is achieving its desired effect. Based on the results of this
evaluation, we recommended that if DOD decided to continue the effort, it should
modify existing guidance or otherwise take steps to improve its cost estimating
approach so that it better aligns with key characteristics of cost estimating best
practices. We note that DOD specifically states in its written comments that the
department intends to continue the current policy. Therefore, as discussed below in
our response to DOD’s specific comments regarding that part of our
recommendations, we continue to believe that opportunities exist for DOD to
improve its cost estimating approach to make it more consistent with relevant best
practices.

With regard to our recommendations that DOD modify the current guidance or
otherwise take steps to improve DOD’s cost estimating approach, DOD partially
concurred with three of our proposed recommendations and did not concur with the
fourth one.


   16                                            GAO-12-480R Defense Management
•    DOD partially concurred with our recommendation to clarify those activities
     that should be included in the estimate as they are incurred, including time
     spent at every level to review and approve the report or study. DOD stated
     that providing clarity to the overall guidance will continue to be a priority,
     adding that the direction to exclude coordination efforts in the cost estimate is
     explicitly documented in guidance. As we noted in the draft report, we
     recognize that DOD’s guidance states that manpower costs should not be
     included when participation only involves normal job activities, such as
     attending meetings and coordinating studies, reports, or proposals. However,
     we found that some of the individuals who generated cost estimates found the
     guidance confusing with regard to whether to include or exclude costs
     associated with coordination activities that are beyond an individual’s normal
     job activity. As a result, we found inconsistencies in how these individuals
     decided to account for this activity when generating their cost estimates.
•    DOD partially concurred with our recommendation requiring that source
     documentation used to develop the cost estimate is retained and easily
     accessible for review purposes. DOD stated that its Cost Guidance Portal,
     which includes the tool, has several features to assist users in obtaining
     source documentation of their cost estimates, including an automatic e-mail
     with a detailed summary report of the cost estimate and a hyperlink to each
     cost estimate. We recognize that these features enable DOD to retrieve the
     information that was input into the cost estimating tool. However, it does not
     include a reference to the actual source documents. As we discuss in the
     report, some individuals responsible for generating the cost estimates that we
     reviewed had difficulty retrieving the source documentation that they used to
     develop the estimates. We continue to believe that without specifically
     requiring individuals to retain source documentation, DOD may not have the
     information needed so that others can understand how a cost estimate was
     derived and readily replicate it.
•    DOD did not concur with our recommendation to establish and implement a
     verification process to provide reasonable assurance of consistency and
     completeness of cost inputs used to develop the cost estimate. The
     department stated that it currently has the ability to pull detailed cost
     estimates for specific studies and reports, when necessary, and to contact
     each DOD employee who prepares a cost estimate, which DOD believes
     provides a reasonable level of accountability. DOD also stated that
     implementing a verification process similar to the one proposed by GAO
     would be a costly endeavor. For example, verifying the amount of time spent
     by each DOD employee preparing a study or report would require the
     department to reduce time spent on other, higher-priority missions. Rather
     than implementing a costly verification process for review of cost estimates
     after they have been submitted, the department will continue to train users
     prior to their preparation and submission of cost estimates. We agree that the
     department can pull information for a specific report or study and contact
     DOD employees regarding estimates. However, we disagree that DOD’s
     ability to take these steps constitutes the same type of accountability that
     would be present if DOD had a verification process in place. We are not


17                                            GAO-12-480R Defense Management
        suggesting that such a process would require DOD to verify the time spent by
        each employee to prepare a report or study. Rather, we would expect that
        any verification effort would include steps to review whether the estimate was
        developed in a manner consistent with DOD’s guidance. Such a process
        would provide a greater assurance of consistency and completeness of the
        cost inputs used to develop the cost estimate. In addition, while we agree that
        training is important, we note that training is not currently required before an
        individual is able to use the tool, and individuals may therefore continue to
        develop cost estimates without the benefit of any training.
   •    DOD partially concurred with our recommendation requiring that the language
        on the front cover of reports and studies make clear that the estimate is
        based on certain marginal manpower and nonlabor costs. DOD stated that
        while the department is open to modifying the phrase that is published on the
        front cover of reports or studies, the objective should not be to detail the
        specific costs included or excluded on the front cover and that the cost on the
        front cover should always represent the marginal costs borne by DOD in
        preparing the study or report. We agree that DOD does not need to detail all
        the specific costs included or excluded in the cost estimate on the front cover.
        Rather, DOD should modify the language on the front cover of reports and
        studies to make it clear that the estimate is based on certain marginal
        manpower and nonlabor costs.

                                         -----

We are sending copies of this report to appropriate congressional committees and
the Secretary of Defense. In addition, this report is available at no charge on the
GAO website at http://www.gao.gov.

If you or your staff have any questions about this report, please contact me at (202)
512-9619 or pickups@gao.gov. Contact points for our Offices of Congressional
Relations and Public Affairs may be found on the last page of this report. GAO staff
who made key contributions to this report are listed in enclosure VI.




Sharon L. Pickup
Director
Defense Capabilities and Management

Enclosures - 6




   18                                            GAO-12-480R Defense Management
Enclosure I: Description of the Four Tracks of the Secretary of Defense’s
Efficiency Initiative

In May 2010, the Secretary of Defense directed the Department of Defense (DOD)
to undertake a departmentwide initiative to assess how the department is staffed,
organized, and operated with the goal of reducing excess overhead costs and
reinvesting these savings into sustaining DOD’s current force structure and
modernizing its weapons portfolio. The Secretary’s initiative targeted both shorter-
and longer-term improvements and set specific goals and targets for achieving cost
savings and efficiencies. The Secretary’s initiative was organized along four tracks—
each of which had a different focus, as shown in table 1. 15


Table 1: Description of the Four Tracks of the Secretary of Defense’s Efficiency Initiative
Track                     Purpose
Track 1                   Military services and other components: Tasked to find savings of about $100 billion in
                          overhead costs that are to be reinvested over 5 years, starting with DOD’s fiscal year
                          2012 budget.
Track 2                   External and internal suggestions: Sought ideas, suggestions, and proposals for
                          efficiencies from outside normal, official DOD channels (e.g., “think tanks,” industry,
                          external boards, and DOD employees).
Track 3                   Departmentwide review of certain areas: Assessed personnel and health care policies,
                          logistics, and acquisition to identify efficiencies.
Track 4                   Specific issues identified by the Secretary: Focused on specific areas where DOD could
                          take immediate action to reduce inefficiencies and overhead, in particular, to reduce
                          headquarters and support bureaucracies and to instill a culture of cost consciousness
                          and restraint in the department.
Source: DOD.
Note: Other components include various offices within the Office of the Secretary of Defense, the combatant commands, DOD
agencies, and DOD field activities.




15
  GAO, Streamlining Government: Key Practices from Select Efficiency Initiatives Should Be Shared
Governmentwide, GAO-11-908 (Washington, D.C.: Sept. 30, 2011).


     19                                                            GAO-12-480R Defense Management
Enclosure II: Scope and Methodology

Our objectives were to evaluate DOD’s efforts to implement an approach to
estimating the costs of selected reports and studies, including (1) whether DOD
entities are capturing and presenting costs in a manner consistent with relevant cost
estimating best practices; (2) the status of DOD’s efforts to track whether
organizations are developing cost estimates for all required types of reports and
studies; and (3) whether DOD has evaluated the usefulness of its efforts to estimate
the cost of reports and studies as one of the means for achieving the Secretary’s
intended purpose of increasing the transparency of costs, reducing or eliminating
reporting requirements, and instilling a culture of cost consciousness across DOD.


To determine whether DOD entities are capturing and presenting costs in a manner
consistent with relevant cost estimating best practices, we reviewed the cost
estimating tool used by DOD entities to generate cost estimates and compared it to
several best practices recognized in GAO’s Cost Estimating and Assessment Guide
and relevant managerial cost accounting guidance—Statement of Federal Financial
Accounting Standards No. 4 and relevant sections of DOD’s Financial Management
Regulation. GAO’s Cost Estimating and Assessment Guide reflects best practices
for developing and managing cost estimates for capital programs. Because DOD’s
cost estimating tool covers noncapital costs, we assessed the tool against the best
practices in the guide that we identified as most applicable to these types of cost.
We compared the tool to three characteristics—comprehensive, well documented,
and accurate—in the guide to determine if the estimate generated by the tool is of
high quality and reliable. We scored each best practice as not met (the tool provided
no evidence that satisfies any of the criterion), minimally met (the tool provided
evidence that satisfies a small portion of the criterion), partially met (the tool
provided evidence that satisfies about half of the criterion), substantially met (the tool
provided evidence that satisfies a large portion of the criterion), and met (the tool
provided complete evidence that satisfies the entire criterion). We determined the
overall assessment by assigning an individual rating to the best practice: not met =
1, minimally met = 2, partially met = 3, substantially met = 4, and met = 5. To
perform this analysis, we reviewed documents provided by the Cost Assessment
and Program Evaluation (CAPE) office, including guidance on using the tool and the
tool itself, and conducted interviews with DOD officials to corroborate our
understanding of the tool and its corresponding guidance. In addition, we compared
the cost estimating tool with Statement of Federal Financial Accounting Standards
No. 4 and relevant sections of DOD’s Financial Management Regulation to
determine the extent to which the tool reflects relevant cost accounting standards
and cost management guidance. All data elements we assessed are sufficiently
reliable for the purpose of assessing the reliability of DOD’s cost estimating tool.
Further, we had originally intended to review a representative sample of DOD
reports and studies with cost estimates published on the front cover. However, DOD
could not readily provide us with a comprehensive list of all reports and studies that
included cost estimates from which we could select reports and studies to review
because DOD could not distinguish between real cost estimates and estimates that
were developed while individuals were using the tool for training purposes. DOD was


   20                                            GAO-12-480R Defense Management
able to provide us information on nine reports prepared for Congress on which
Congress had requested more information, and identify the individuals responsible
for generating the cost estimates for these reports. Therefore, we reviewed these
nine reports. Using semistructured interviews, we held discussions with the key
officials responsible for developing the cost estimates for these reports to gain an
understanding of their experience with using the tool and corresponding guidance
and their level of confidence in the cost estimates generated, among other things. In
addition, we reviewed supporting documentation for some of the estimates as well
as the snapshot of the data that was entered into the tool. In some cases, officials
were not able to readily retrieve documentation; therefore, we were unable to fully
evaluate the extent to which they consistently applied DOD’s guidance for estimating
costs. We were unable to assess the reliability of the data used to generate the cost
estimates that were included on these nine reports, or to generalize these findings to
all reports, but we feel that the review of these nine reports is sufficient for providing
context about the process, and we are making a recommendation about future data
collection efforts.


To describe DOD’s efforts to determine whether organizations are developing cost
estimates for all required types of reports and studies, we reviewed guidance and
interviewed DOD officials to assess the nature of their tracking efforts. Specifically,
we conducted interviews, gathered information and documents regarding internal
and external report tracking mechanisms from officials in the CAPE office; the Office
of the Assistant Secretary of Defense for Legislative Affairs; the Office of the Director
of Administration and Management; Washington Headquarters Services; and the
legislative affairs offices of the Departments of the Army, Navy, and Air Force.


To determine whether DOD has evaluated the usefulness of its efforts to estimate
the cost of reports and studies as one of the means for achieving the Secretary’s
intended purpose of increasing the transparency of costs, reducing or eliminating
reporting requirements, and instilling a culture of cost consciousness across DOD,
we reviewed relevant DOD guidance and GAO’s Standards for Internal Control in
the Federal Government. We also interviewed DOD officials responsible for
administering and using the tool, including the individuals who generated the nine
reports for Congress, to determine if they had taken steps to obtain feedback from
internal and external decision makers regarding the impact of estimating costs and
publishing the cost estimate on the front covers of the reports and studies.

We conducted this performance audit between October 2011 and May 2012 in
accordance with generally accepted government auditing standards. Those
standards require that we plan and perform the audit to obtain sufficient, appropriate
evidence to provide a reasonable basis for our findings and conclusions based on
our audit objectives. We believe that the evidence obtained provides a reasonable
basis for our findings and conclusions based on our audit objectives.




   21                                            GAO-12-480R Defense Management
Enclosure III: Snapshot of Fields in DOD’s Cost Estimating Tool

The tool designed by DOD consists of fields with drop-down menus to allow users to
choose the category of costs they wish to input, among other types of information.
Additionally, the tool contains information such as the definition of what constitutes a
report or study and a real-time calculator. (See fig. 2.)

Figure 2: Fields in DOD’s Cost Estimating Tool




    22                                           GAO-12-480R Defense Management
Enclosure IV: Description of Nine Reports Prepared for Congress That GAO
Reviewed

We reviewed information on nine reports DOD prepared for Congress for which cost
estimates had been generated in 2011 and interviewed the key officials responsible
for developing the cost estimates. The reports we reviewed were prepared by
organizations within the Office of the Secretary of Defense, the Navy, and the Air
Force. Four of the cost estimates consisted solely of manpower costs, and the other
five consisted of manpower and nonlabor costs. (See table 2.)

Table 2: Description of Nine DOD Reports Prepared for Congress That GAO Reviewed

Organization
                                                                          Cost estimate (dollars in thousands)
that prepared     Report                                   Date
the report
                                                                          Manpower       Nonlabor        Total
                  Integration of Current and Future        January 12,
Navy                                                                               $3            $0         $3
                  Unmanned Payloads into Submarines        2011
Office of the
Under Secretary
of Defense for    Panel on Contracting Integrity 2010      January 28,
                                                                                   18             0        $18
Acquisition,      Report to Congress                       2011
Technology and
Logistics
                  Report to Congress on Repair of Naval    February 8,
Navy                                                                               11             0        $11
                  Vessels in Foreign Shipyards             2011
Office of the
                  Report on the Joint Use of Federal
Under Secretary                                            February 9,
                  Forest Fire Fighting Assets/C-130 Fire                           51          120       $171
of Defense for                                             2011
                  Fighting Capabilities
Policy
Office of the
Secretary of
Defense, Cost     A Review of the Army's Modular Force     February 16,
                                                                                 149           460       $609
Assessment and    Structure                                2011
Program
Evaluation
                  Expeditionary Combat Support System
                  (ECSS) Increment 1 Major Automated       February 18,
Air Force                                                                      1,232          1,146    $2,378
                  Information System (MAIS) Critical       2011
                  Change Report
                  Strategic Plan for Homeporting the       February 24,
Navy                                                                               11             0        $11
                  Littoral Combat Ship                     2011
Office of the
Under Secretary   Department of Defense Foreign
of Defense for    Language Proficiency Fiscal Year 2010    March 2011              78            23      $101
Personnel and     Report to Congress
Readiness
                  Littoral Combat Ship (LCS) Monthly       March 24,
Navy                                                                                2             1         $3
                  Cost Report to Congress                  2011
Source: DOD.




     23                                                    GAO-12-480R Defense Management
Enclosure V: Comments from the Department of Defense




  24                                   GAO-12-480R Defense Management
25   GAO-12-480R Defense Management
26   GAO-12-480R Defense Management
27   GAO-12-480R Defense Management
28   GAO-12-480R Defense Management
29   GAO-12-480R Defense Management
Enclosure VI: GAO Contact and Staff Acknowledgments

GAO Contact

Sharon L. Pickup, (202) 512-9619 or pickups@gao.gov

Staff Acknowledgments

In addition to the contact named above, key contributors to this report were
James A. Reynolds, Assistant Director; Bruce Brown; Frances Abe Dymond; Mae
Jones; Terry Richardson; Karen Richey; Amber Lopez Roberts; Robert Sharpe;
Amie Steele; Stacey Steele; Sonja S. Ware; and Karen Nicole Willems.




(351661)



   30                                      GAO-12-480R Defense Management
This is a work of the U.S. government and is not subject to copyright protection in the
United States. The published product may be reproduced and distributed in its entirety
without further permission from GAO. However, because this work may contain
copyrighted images or other material, permission from the copyright holder may be
necessary if you wish to reproduce this material separately.
                      The Government Accountability Office, the audit, evaluation, and
GAO’s Mission         investigative arm of Congress, exists to support Congress in meeting its
                      constitutional responsibilities and to help improve the performance and
                      accountability of the federal government for the American people. GAO
                      examines the use of public funds; evaluates federal programs and
                      policies; and provides analyses, recommendations, and other assistance
                      to help Congress make informed oversight, policy, and funding decisions.
                      GAO’s commitment to good government is reflected in its core values of
                      accountability, integrity, and reliability.

                      The fastest and easiest way to obtain copies of GAO documents at no
Obtaining Copies of   cost is through GAO’s website (www.gao.gov). Each weekday afternoon,
GAO Reports and       GAO posts on its website newly released reports, testimony, and
                      correspondence. To have GAO e-mail you a list of newly posted products,
Testimony             go to www.gao.gov and select “E-mail Updates.”

Order by Phone        The price of each GAO publication reflects GAO’s actual cost of
                      production and distribution and depends on the number of pages in the
                      publication and whether the publication is printed in color or black and
                      white. Pricing and ordering information is posted on GAO’s website,
                      http://www.gao.gov/ordering.htm.
                      Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
                      TDD (202) 512-2537.
                      Orders may be paid for using American Express, Discover Card,
                      MasterCard, Visa, check, or money order. Call for additional information.
                      Connect with GAO on Facebook, Flickr, Twitter, and YouTube.
Connect with GAO      Subscribe to our RSS Feeds or E-mail Updates. Listen to our Podcasts.
                      Visit GAO on the web at www.gao.gov.
                      Contact:
To Report Fraud,
Waste, and Abuse in   Website: www.gao.gov/fraudnet/fraudnet.htm
                      E-mail: fraudnet@gao.gov
Federal Programs      Automated answering system: (800) 424-5454 or (202) 512-7470

                      Katherine Siggerud, Managing Director, siggerudk@gao.gov, (202) 512-
Congressional         4400, U.S. Government Accountability Office, 441 G Street NW, Room
Relations             7125, Washington, DC 20548

                      Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800
Public Affairs        U.S. Government Accountability Office, 441 G Street NW, Room 7149
                      Washington, DC 20548




                        Please Print on Recycled Paper.