oversight

OMB Circular A-76: DOD's Reported Savings Figures Are Incomplete and Inaccurate

Published by the Government Accountability Office on 1990-03-15.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                         C;nited States   General   Accounting         Office
                        Report to the Chairman? Subcommittee
GAO                     on Federal Services, Post Office, and
                        Civil Service, Committee on
                        Governmental Affairs, U.S. Senate

March   1990
                         OMB CIRCULAR A-76
                         DOD’s Reported
                         Savings Figures Are
                         Incomplete and
                         Inaccurate




                EEsTBI(=TED --No% to be relea&d&t$ide            the
                Generzsl Aeeounting Office anless tsptic&-
                appmved bythe Off&e of Cmgressimal
                Relations.



GAO/GGD-90-58
united   states
General Accounting Office
Washington, D.C. 20648

General Government Division

B-234046

March 15,199O
The Honorable David Pryor
Chairman, Subcommittee on
  Federal Services,Post Office,
  and Civil Service
Committee on Governmental Affairs
United States Senate
Dear Mr. Chairman:

This report responds in part to your request that we evaluate federal
agency implementation of the Office of Managementand Budget (OMB)
Circular A-76, “Performance of Commercial Activities.” A principal pur-
pose of OMB’S A-76 program is to achieve economy in government
operations.
As agreed, we focused on the savings the Department of Defense(DOD)
attributes to its A-76 efforts. As part of our evaluation, we assessedthe
(1) accuracy of A-76 savings data reported to OMBand (2) availability of
A-76 administrative cost information. Although our work was limited to
DOD's cost and savings data, it provides an insight into the accuracy of
OMB’S  reported aggregatesavings in government operations achieved by
the A-76 program becausemost of the reported A-76 savings were made
by DOD. A more detailed discussion of our objectives, scope,and method-
ology appears in appendix II. Other aspectsof your request have been
addressedin our previous reports.’

OMBCircular A-76 applies to federal agencies’commercial activities-
functions such as custodial services,data processing,and vehicle main-
tenance. (Inherently governmental functions-those intimately related
to the public interest-must continue to be done by federal employees.)
Commercial activities are to be studied to determine whether they could
be done more economically by nonfederal sources-contractors-or by
government employees.
The first step in an A-76 cost study involves the agency precisely defin-
ing the performance standards of the work it is doing and expects to do
in the future.




Page 1                                        GAO/GGD~OMB-A-76
                       B-224046




                       As part of the A-76 study, the agency does a managementreview to
                       determine how the in-house workforce should be organized and
                       equipped to most efficiently accomplish the performance standards. The
                       agency next determines how much it would cost to operate according to
                       the revised in-house organization. This cost is the in-house bid to accom-
                       plish the work.
                       The agency also solicits contractors’ bids for the work to be done. When
                       all bids are received, the agency comparesthe in-house bid to the con-
                       tractors’ bids. In general, the agency is to award a contract to the lowest
                       bidder whom the agencyjudges to be able to meet all the standards. In
                       instances where there are no qualified contractor bids or the contractor
                       bids exceedthe in-house bid, the activity will be kept in-house to be
                       done by government employees,who are to implement the revised in-
                       house organization within 6 months of winning the bid.

                       Agencies are required to report to OMBquarterly the status of their A-76
                       studies. Estimated savings figures for completed studies are reported
                       annually. Agencies are to calculate annual savings from each study by
                       subtracting the anticipated new cost of the activity (based on the win-
                       ning bid) from the estimated original cost to operate the function. Sav-
                       ings figures, therefore, are projections and are not basedon actual
                       experience.

                       To track A-76 studies on a boBwide basis, DOD’S A-76 office usesa com-
                       puterized data base.Installations provide required data to their compo-
                       nent headquarters, who report the information quarterly to the Defense
                       Manpower Data Center (DMDC). DMDC compiles reports and other infor-
                       mation, including data reports for OMB.


Results in Brief        According to OMB,annual A-76 savings for fiscal year 1988 totaled over
                        $133 million. More than 80 percentof these reported governmentwide
                        savings are attributable to DOD. However, our evaluation of DOD'S sav-
                        ings data shows that OMB’S  figures do not accurately reflect the extent to
                        which economy in government operations is being achieved.Specifically:
                   .    DOD estimates expected cost savings from
                                                               individual studies on the basis
                     of standardized assumptions, not on the best available cost data.
                   . DOD doesnot routinely collect and analyze cost information to monitor
                     actual operations after a cost study has been made.
                   . The computerized data basethat DOD usesto accumulate information on
                     expected cost savings contains inaccurate and incomplete information.
                                         .
                        Page 2                                         GAO/GGD(KM(IOMBcircularA-76
                              l   DOD'S automated system miscalculates total   annual expected cost
                                  savings.
                              l   DOD'S system does not contain reliable information on the cost of imple-
                                  menting DOD'S A-76 program, including the cost of doing the studies.

                                  Therefore, neither DOD nor OMBhas reliable information on which to
                                  assessthe soundnessof savings estimates or knows the extent to which
                                  expected savings are realized.

                                  Complete and accurate savings information would help to reduce some
                                  of the controversy surrounding the A-76 program. As discussedin our
                                  previous reports, agenciesare not embracing the program and Congress
                                  has legislatively prohibited A-76 cost studies of certain functions. There
                                  is no unanimity that the A-76 program is the appropriate way to get
                                  governmentwide efficiencies. The processis characterized as burden-
                                  someand time-consuming, agenciessay the goals assignedthem by OMB
                                  are unrealistic, and Congresshas not fully acceptedthe program.


Principal Findings

Origin ~1 f%usr~fir
     K&A   “pLc&,dg   costs
                                  Projected cost savings from individual A-76 studies are estimated cen-
 A- n
1WC uetermined by
                                  trally by DOD'S A-76 office. Rather than basing savings on original oper-
                                  ating costs as estimated by the installation doing the study, DOD'S A-76
1Formula                          office uses a formula to compute savings. DOD'S formula assumesthat
                                  nonpersonnel costs vary directly with the number of people working in
                                  an activity, an assumption that is not valid for sometypes of costs, such
                                  as materials for housing repairs. In general, the cost of repair materials
                                  would be determined by the amount of repair material required, not by
                                  the number of people working on the repair.

                                  DOD A-76  officials said they use the formula becausenot all accounting
                                  systems at military installations can identify the actual operating costs
                                  of individual activities of that installation. OMB A-76 officials do not sup-
                                  port the use of a formula by DOD'S A-76 office and said that installations
                                  could, from existing information, prepare a reasonably accurate esti-
                                  mate of original costs.Service officials said service headquarters could
                                  assumeresponsibility for reporting original activity costs,on the basis
                                  of available information from installations. They said they did not want
                                  to add another requirement on the already burdened installations’ A-76
                                  offices.


                                   Page 3                                         GAO/GGDM      OMB Circular A-76
                                                                                                       -
                              B234046




DOD Lacks Information on      DODdoes not know the extent to which the actual costs incurred after a
                              cost study is completed differ from anticipated costs becauseit doesnot
Modifications Made After      analyze the information the reporting system collects, and the system
the Cost Study                doesnot collect sufficient information. Sometimes,contracts are modi-
                              fied, contract administration costs are higher than originally estimated,
                              or in-house reorganizations do not occur as planned. Without adequate
                              information for post-study analysis, however, DODcannot determine the
                              extent to which projected savings are realized nor detect cost-growth
                              problems in functions that have been studied.

                              Adequate post-study analysis requires the collection of at least three
                              types of information that DODdoesnot now have.

                            . Detailed information on contract cost increases:DODnow collects only
                              the information on total (annual) amounts paid to contractors. Informa-
                              tion on the reasonsfor cost increases,and related dollar amounts, is not
                              collected. Such data could be used to identify the extent of problems,
                              such as errors or omissions in work statements.
                            . Estimated contract administration staffing: A comparison of staffing
                              estimated at the time of contracting out and actual staffing would show
                              the extent to which a changefrom the estimated staffing level has
                              affected projected savings. DODplans to collect information on actual
                              staffing, but not on the estimated (baseline) staffing.
                            . Revised in-house organization actual costs:DOD’S    A-76 office doesnot
                               collect any information about the implementation of the revised in-
                               house organization, including whether it is saving as much money as
                               projected. Although DODdoesnot collect this information, OMBrequire-
                               ments state that all agenciesshould.


DOD’s A-76 Data Base           DOD’S A-76 savings reports for fiscal years 1986 through 1988 did not
Contains Inaccuracies and      include about 13 percent of the studies it completed in those years. Some
                               studies were excluded becausea DODreporting instruction directed that
Incomplete Savings-            if there were no qualified contractor bids, an installation should not
Related Data                   report its in-house bid. Therefore, data that DODwould need to calculate
                               the savings resulting from implementation of the revised in-house
                               organization were not available. DODofficials speculated that these cost
                               data involved what could be consideredto be proprietary information
                               that could not be disclosed.However, they acknowledged that such pro-
                               tection is not neededoncethe period for any appeals and protests has
                               passed.DODexcluded other studies from its totals becausecost-related
                               data required to be reported to DODwere missing or inconsistent. The
                               two types of excluded studies involved a total of more than $36 million


                               Page 4                                        GAO/GGIMM6 OMB CireoLarA-76
                           in annual (recurring) savings from studies completed in fiscal years
                           1986 through 1988.
                           One reason for not detecting the missing or inconsistent data-as well as
                           for problems found with data that are not cost related-is flawed com-
                           puterized edit checks and error correction proceduresof the DOD A-76
                           reporting system. The system lacks comprehensiveedit checksat the
                           points where data are initially entered into computerized systems,and
                           errors in the data base are not corrected in a timely manner. A majority
                           of the suspectederrors that DOD identified recently in its data basewere
                           for studies completed at least a year earlier-studies that DOD had, or
                           should have, already included in its annual savings calculations. These,
                           therefore, affected the annual savings figures reported to OMB.


Special Computer Progw n   The total annual A-76 savings figures we calculated by adding the sav-
Miscalculates Reported     ings from individual DUD cost studies did not match the figures produced
                           by a special computer program that DoD developedto generate its total
Total Annual Savings       annual savings figures. One phase of this computer program miscalcu-
                           lates the savings figures. The averageyearly miscalculation for fiscal
                           years 1986 through 1988 was more than $2 million each year. (For 2 of
                           the years DOD'S program produced lower savings figures, and for 1 year
                           it produced a higher figure.) DOD had no compelling reason for using that
                           phase of the computer program. Moreover, if DOD continues to use that
                           phase of the program, OMB and DOD will have different annual savings
                           figures, becauseOMBplans to compute its own annual figures by sum-
                           ming individual study savings reported to it.


Lack of Information on     Neither DOD nor OMB had much information on the cost of administration
A-76 Program Costs         of the A-76 program. The overall cost of the program, including doing
                           the studies, has been estimated by the Deputy Assistant Secretary of
                           Defensefor Installations at between $150 million and $300 million per
                           year. None of these costs, such as those incurred in doing the studies,
                           are considered in estimating savings. The costs of administering the
                           A-76 program have been a congressionalconcern. For example, discus-
                           sions of DOD’S appropriation for fiscal year 1990 have focused on the
                           costs of DOD’S A-76 program.

                           DOD requires itscomponentsto report staff hours expended to do indi-
                           vidual studies. However, we found that most of the data are not being




                           Page5                                          GAO/GGIMBO-5gOlKEclrcularA-76
                       reported or do not appear to be reasonable,and DOD plans to stop collect-
                       ing the information. OMB doesnot require agenciesto collect this infor-
                       mation. OMB believes that routinely collecting information on the cost of
                       the A-76 program is not appropriate becausethe A-76 program is aimed
                       at achieving managementefficiencies, which is what all government
                       managers should be doing. While we believe that information on study
                       costs could be useful for getting a clearer picture of the costs and bene-
                       fits from the A-76 program, OMB, DOD, and service officials noted diffi-
                       culties in getting installations to report reliable information,


A-76 Program Remains   According to OMB, the A-76 program has been achieving economy in gov-
Controversial          ernment operations. The President’s report, Managementof the United
                       States Government, Fiscal Year 1990, stated that annual A-76 savings in
                       fiscal year 1988 totaled over $133 million. More than 80 percent of these
                       reported savings were from DOD’S A-76 studies.

                       In 1988 and 1989 reports2and testimony,3 we discussedcontroversies
                       surrounding the A-76 program. Proponents believe it can achieve sub-
                       stantial savings and improve government productivity. Others are con-
                       cerned about its potential effect on the anxious employeeswho spend
                       time looking for new jobs and on federal managerswho are left to
                       accomplish essential functions with fewer workers. Moreover, there is
                       no unanimity that the A-76 program is the appropriate way to get
                       governmentwide efficiencies because(1) the processis characterized as
                       burdensomeand time-consuming, (2) agenciessay the goals assigned
                       them by OMBare unrealistic, and (3) Congresshas not fully acceptedthe
                       program. In turn, agenciesare not embracing the program, and Congress
                       has legislatively prohibited A-76 cost studies of certain functions. Con-
                       gress also has been concernedabout the costs to administer the program.
                       In our opinion, accurate and complete program data would help resolve
                       someof the controversy and stimulate more informed debate on the
                       issue by providing clearer insight into the costs and benefits of federal
                       agencies’efforts.




                       ‘Federal Productivity: DOD’sExperiencein C+tr   Out Gxnmercially Available Activities
                       (GAO/GGD89-6), NOV.28,19&B;and Managmg‘l??%ernment: RevisedApproachCould Improve
                       OMEVs Effectiveness(GAO/GGD-89-66),May 4, 19:9.
                       3Statementof GeneL. Dodaro,beforethe President’sCommissionon Privatization (GAO-T-
                       GGD-88-7,Jan. 7,1988).



                       Page 6                                                   GAO/GGD9068 OME Circular A-76
Conclusions   which the A-76 program is meeting the objective of achieving economy
              in government operations. Estimates of annual recurring savings have
              been reported, and program costs have also been estimated. However,
              neither the reported savings figures nor cost estimates are supported by
              complete and accurate data. Accordingly, someof the controversy sur-
              rounding the A-76 program is understandable-Congress questions
              whether projected savings are real, and agenciesare not sure of the ben-
              efits of implementing the program.
              A number of factors contribute to the inaccuracy of DOD'S savings esti-
              mates. DOD estimates expected cost savings from individual studies on
              the basis of standardized assumptions,not on the best available cost
              data. Further, DOD'S A-76 office doesnot routinely collect and analyze
              cost information to monitor actual operations after a cost study has
              been made, nor doesit assessthe extent to which actual post-study costs
              differ from anticipated costs.Also, DOD'S instructions for its A-76 report-
              ing system causenonreporting of someneededdata. Moreover, that sys-
              tem lacks comprehensiveedit checks at the points where data are
              initially entered into computerized systems.As a result, data reported
              by DOD services are often incomplete or inconsistent. Those errors also
              are not being corrected in a timely manner. Further, DOD'S program used
              to calculate total annual savings miscalculates the reported savings
              figures. We believe DOD can correct each of these problems.

              Regarding the costs of administering the A-76 program, neither DOD nor
              OMB has much information.     Such information, together with complete
              and accurate savings data, could present a more comprehensivepicture
              of program costs and benefits and help to determine the most cost-
              effective use of resources.However, A-76 program officials at OMB,DOD,
              and the services believe that there are difficulties in obtaining reliable
              information from installations on the hours expended in doing A-76
              studies, and DOD plans to stop collecting this information. In general,
              these officials believe the relative costs would outweigh the benefits of
              collecting program cost information. Moreover, OMB officials believe that
              doing reviews to determine the most efficient way to accomplish opera-
              tions is a management responsibility that should be carried out routinely
              and thus should not be counted as a cost against A-76 savings.
              We recognizethat obtaining reliable information on A-76 administrative
              costs may be difficult. Not all military accounting systems automatically
              provide such information, and staff time and effort would be neededto



              Page 7                                         GAO/GGD8O-58   OMB CircnLar A-76
                      develop it. Even so, we believe more needsto be done to measureA-76
                      administrative costs given the following:
                      The contentious differences between proponents and opponents of A-76
                      programs can affect program implementation.
                      Neither proponents nor opponents have hard data to support their
                      positions.
                      Reliable data could help resolve the controversy and stimulate more
                      informed debate on the costs and benefits of A-76.
                      Congressionalfocus on A-76 administrative costs is increasing.

                      As a first step, DODshould design and pilot test an approach for reliably
                      measuring its A-76 administrative costs.On the basis of the pilot-test
                      information-the costs versus the benefits-oMB should be in a position
                      to accurately determine the level of A-76 administrative costs and
                      whether it would be worthwhile to collect these costs governmentwide.


                      To improve DOD'S oversight of its A-76 program and the reliability of
Recommendations       A-76 program savings reported by DOD,we recommendthat the Secre-
                      tary of Defensetake the following actions:

                  l Make DODcomponentsresponsible for estimating the original cost of
                    activities being studied, using standardized budgetary and actual cost
                    information, rather than using a formula to calculate this, and use these
                    cost figures in computing and reporting savings.
                  l Improve DOD’S ability to analyze cost changesthat occur after cost stud-
                    ies are completed by requiring DOD'S A-76 office to collect information on
                    (1) reasonsfor contract cost changesand related dollar amounts,
                    (2) estimated contract administration costs (baseline costs), and
                    (3) revised in-house organization costs (activity costs after the revised
                    in-house organization is implemented).
                  . Report to OMB actual costs for revised in-house organizations, as
                    required.
                  . ChangeDOD reporting instructions to require reporting of the in-house
                    organization bid for all completed cost studies.
                  . Direct the Secretariesof the Air Force, Army, and Navy to ensure that
                    there are comprehensiveedits on A-76 data at the point where they are
                    initially entered into a computerized system, in order to reduce the
                    number of errors in DOD'S A-76 data base.
                  l Place increased emphasis on correcting data errors in the DOD A-76 data
                    base as soon as they are found by error-identification programs.



                      Page 6                                        GAO/GGLWO-SS
                                                                               OMB circular A-76
. Changethe procedure for calculating total A-76 savings for a fiscal year
  by summing savings acrossstudies. The portion of the computer pro-
  gram currently used to produce the annual estimated savings total
  should be deleted to avoid the possibility of errors in reported
  information.
  To help resolve the controversy surrounding the A-76 program, we rec-
  ommend that the Secretary of Defensedesign and pilot test an approach
  for reliably measuring A-76 program administration costs at DOD, with
  OMBmonitoring the progress of the effort. On the basis of the pilot test,
  OMBshould be in a position to accurately determine the level of A-76
  administrative costs and whether it would be worthwhile to require
  their collection governmentwide.


   OMEI commentedon a draft of this report and generally agreed with our
   findings and concurred with our recommendations.It said it will moni-
   tor DOD’S progress in pilot testing an approach to reliably measuring
   A-76 administrative costs.

   However, OMBexpressedconcern about what it called the report’s
   “inference” that becauseDOD'S savings figures are suspect,the A-76 pro-
   gram may not be achieving economy in government operations. This was
   not our intent. We could not determine, from the available data, the
   impact the program has had or whether the reported economieswere
   overstated or understated. Thus, we have modified our report’s lan-
   guageto minim& the possibility that readers would draw other infer-
   ences.Cur principal messageis that accurate A-76 savings figures are
   neededto address long-standing and growing congressionalconcerns
   about the A-76 program’s value and achievements.While we agreethat
   the program is appealing conceptually, as long as the data supporting its
   cost effectiveness are open to question, the program will continue to be
   vulnerable to doubts about its efficacy in practical application.

   OMBalso questioned whether our review, which was confined to DOD, has
   governmentwide applicability. We agreethat our review provides no
   indication whether the inaccuracies found in DOD'S data system exist in
   other agencies.However, becauseDOD was responsible for more than 80
   percent of estimated governmentwide savings in fiscal year 1988, we
   believe that cleaning up its data basewill result in more complete and
   accurate estimates of governmentwide A-76 savings. We have modified
   the report to make this point more clearly.



   Page 9                                        GAO/GGD89.69OMB CircnLar A-76
B-234046




We requested written comments from DOD, but none were provided.
However, we obtained the department’s views during an exit confer-
ence, and these are reflected throughout the report. DOD officials gener-
ally agreed with our recommendations.However, as discussedon
page 17 of this report, they believe the responsibilities of the A-76 office
end with a cost study’s completion and do not involve gathering further
information.

As arranged with the Subcommittee,unless you publicly announceits
contents earlier, we plan no further distribution of this report until 30
days after its issue date. At that time, we will send copies to other
appropriate congressionalcommittees and Members,the Secretary of
Defense,and the Director of the Office of Managementand Budget. We
will also send copies to other interested parties upon request.
As agreed with the Subcommittee,we have also included as appendix IV
a list of recent GAO reports that addressissuesrelated to this report.
Major contributors to this report are listed in appendix V. Pleasecontact
me at 275-8676 if you or your staff have any questions concerning the
report.

Sincerely yours,




L. Nye Stevens
Director, Government Business
   Operations Issues




 Page 10                                        GAO/GGlMO-KSOlKB Cii     A-76
Page 11   GAO/GGDo(MB OMB circular A-76
Contents


Letter                                                                                       1

Appendix I                                                                                 14
DOD’s Reported A-76     Original Operating CostsAre Determined by Formula
                        DOD’sA-76 Office Lacks Information on Modifications
                                                                                           16
                                                                                           17
Savings Figures Are         Made After the Cost Study
Incomplete and          DOD’sA-76 Data BaseContains Inaccuracies and                       20
Inaccurate                  Incomplete Savings-RelatedData
                        Special Computer Program Miscalculates Reported Total              23
                            Annual Savings
                        Lack of Information on A-76 Program Costs                          24

Appendix II                                                                                26
Objectives, Scope,and
Methodology
Appendix III                                                                               28
Comments From the
Office of Management
and Budget
Appendix IV                                                                                30
Recent GAO Reports
Addressing Related
Issues
Appendix V                                                                                  32
Major Contributors to   General Government Division, Washington, D.C.                       32
                        Norfolk Regional Office                                             32
This Report
Tables                  Table I. 1: Projected Recurring Savings Excluded by DOD             21
                            Reporting Instruction
                        Table 1.2:Projected Recurring Savings Excluded Because              22
                            of Data Problems
                        Table 1.3:Differences in Projected Recurring Savings-               24
                            Special Computer Program vs. Summing
                                               .

                        Page 12                                    GAO/GGD-9049OME clrcnlu A-76
Table 1.4:Completed Studies Involving 1 to 10 Positions               25




Abbreviations

AF+MEA Air Force ManagementEngineering Agency
DMDC   DefenseManpower Data Center
DOD    Department of Defense
OMB    Office of Managementand Budget
OPM    Office of PersonnelManagement


Page!13                                      GAO/GGIM6-63 OMB fZXrdar A-76
Appendix I

DOD’s Reported A-76 SavingsFigures Are
Incompleteand Inaccurate

              ohmCircular A-76 applies to federal agencies’commercial activities,
              such as custodial services,data processing,and vehicle maintenance.
              The A-76 processrequires that agenciesidentify all activities being done
              by government workers that could be done by nonfederal sourcesand
              schedulethose activities for review. As the initial step in the actual A-76
              cost study process,the agency must precisely define the performance
              standards (quality, timeliness, quantity) of the work it is doing and
              expects to do in the future. These standards are incorporated in a writ-
              ten document called the performance work statement. This work state-
              ment is the basis for developing both the in-house cost estimate and
              contractors’ bids.
               As part of the A-76 process,the agency does a managementstudy to
               determine how the in-house workforce should be organized and
               equipped to most efficiently accomplish the performance standards
               specified in the work statement. The in-house workforce’s most efficient
               organization is designedto enable the government workers to be more
               competitive with private sector contractors in bidding for the work. The
               agency next determines how much it would cost to operate according to
               the revised in-house organization. This cost becomesthe in-house bid to
               accomplish the commercial activity.

               The agency solicits contractors’ bids for the work to be done. When all
               bids are received, the agency comparesthe in-house bid to the contrac-
               tors’ bids. The agency awards a contract to the lowest-bidding contrac-
               tor who is judged by the agency to be able to meet all the government’s
               quality, timeliness, and quantity standards, provided that (1) the total
               cost of contract performance is lessthan the in-house bid, and (2) the
               margin of difference between the total cost of contract performance and
               the in-house bid exceeds10 percent of the personnel costsof the in-
               house bid. This lo-percent margin is included in the cost comparison to
               take into account such factors as temporary decreasein efficiency and
               effectiveness, the cost of retained grade and pay, and other unpredict-
               able risks that may occur as a result of the conversion to contract. The
               total cost of contract performance includes, in addition to the contrac-
               tor’s bid price, the government’s estimated costsof severancepay, relo
               eating and retraining the government’s workers, and administering the
               contract.

               If the conditions listed above are not met, the activity will be retained
               in-house and be done by government workers, who must implement the
               revised in-house organization within 6 months of winning the bid.
               Affected parties can fiie an appeal as a safeguard to help ensure that


               Page 14                                       GAO/GGIMO-M OM8 Circular A-76
        Appendix I
        DOD%Beported A-76 Savinga F’l@uw Am
        Incomplete and Inaccumte




        the decision is equitable and in accordancewith A-76 procedures, Pro-
        jected savings from each cost study are computed by DOD’S A-76 office,
        which reports to OMB the projected savings from all its components’ cost
         studies.
        In January 1988, the Deputy Assistant Secretary of Defensefor Installa-
        tions testified that A-76 cost comparisons result in an averagesavings of
        30 percent of the original cost of operations regardlessof whether the
        activity is retained in-house or contracted out. Savings are defined as
        the difference between the anticipated cost of the activities (basedon
        the winning bid) and the estimated original cost of the activities before
        the studies began. For A-76 studies completed in fiscal years 1986
        through 1988, DOD reports showed average annual (recurring) savings of
        about $93 million.

        DOD’S experience with A-76 cost studies is the most extensive of all fed-
        eral agencies.DOD records show that from fiscal year 1984 through
        1988, DOD completed cost comparisons covering about 40,000 employee
        positions.’ DOD uses a computerized data base to track and monitor these
         studies. Installations provide required data to component headquarters,
         which in turn report quarterly on the status of their A-76 studies to the
         DefenseManpower Data Center (DMDC). DMDC prepares reports and other
         output as requested by DOD A-76 officials, including data for OMB’S A-76
         tracking system.

         We found that the savings figures DOD’S A-76 office reports to OMB are
         not accurate, and DOD’s A-76 office doesnot have a complete picture of
         A-76 efforts for the following reasons:

l        DOD’S A-76  office estimates savings from individual cost studies using a
         formula, rather than having the servicesproject the savings using
         installations’ known operating costs.
.        DOD’S A-76 office lacks information on modifications made to contracts
         or to the revised in-house organization after a cost-study decision has
         been made.
    .    DOD’S A-76 data base contains inaccurate and incomplete information.
    .    The computer program DOD usesto calculate total annual estimated A-76
         savings miscalculates the savings figures.
    .    DOD lacks information on A-76 program costs.



         ‘OMB reportedthat all other federal agencies’studiescovereda t&al of ahout 16,000positionsdur-
         ing this period.



         Page 15                                                     GAO/GGIMO-56OMB Circular A-76
                                           .


                      Appendix I
                      DOD3 Reported A-76Saving13Pigtuw Are
                      Incomplete and Ixmumrate




                      Projected annual (recurring) savings from individual A-76 cost studies
Original Operating    are estimated centrally by DOD’S A-76 office, using a formula, rather
CoSti he Determined   than by service headquarters or the installation doing the study, using
by Formula            standardized budgetary and actual cost information. DOD’S A-76 office
                      projects savings from each cost study by subtracting the anticipated
                      new cost of the activity (based on the winning bid) from the estimated
                      original operating cost of the studied function. To do this, the A-76
                      office usesa formula to estimate the original operating costs of activities
                      and usesthe estimated cost figures to compute the A-76 savings figures
                      reported to OMB. DOD A-76 officials said that they use the formula
                      becausenot all accounting systems at military installations can identify
                      the actual operating costs of the individual activities being studied.
                      OMB’S  A-76 official said that original costs should be estimated by instal-
                       lation officials who have much of the actual cost information rather
                       than be calculated by DOD’s A-76 staff using a formula. Service officials
                       said, however, that installations’ A-76 staffs already have too many
                       responsibilities. They said that the services’ headquarters staffs could
                       assumeresponsibility for reporting original activity costs.

                      DOD’S formula  for estimating the original cost of an activity is basedon
                      the assumption that the operating cost per full-time employee to do that
                      activity remains constant, regardlessof the number of people doing the
                      work in-house after the reorganization. Specifically, to obtain its esti-
                      mated original cost of the activity, DOD calculates the per-person cost of
                      the revised in-house organization determined by the managementstudy
                      and multiplies that cost by the number of people neededfor that activ-
                      ity before the study began2
                       DOD’S formula   implicitly assumesthat if the number of people staffing
                       an activity decreases,then all coststhat are being bid on for that activ-
                       ity-including materials and supplies, overhead, and equipment-also
                       decreaseby the sameproportion. For certain types of costs (such as tele-
                       phones or office supplies), this assumption may be valid. However,
                       other types of costs, such as materials and supplies for housing repairs
                       or the equipment costs of a computer center, are largely independent of
                       the number of people who staff an activity. A DOD A-76 official said that
                       validation of DOD’S formula was done someyears ago using a small sam-
                       ple of activities, but he was unable to provide us with any
                       documentation.

                       2Forexample,if the in-housebid is $600,000and proposesstaffing of 10 people,the per-personcost
                       is $60,000.If 16 full-time peoplewere neededfor an activity beforethe study began,DOD’sformula
                       would computethe estimatedoriginal costof the activity to be $SOO,OOO  ($60,000x 16).



                       Page 16                                                    GAO/GGIMW5SOMB Circular A-76
                            Appendix I
                            DOD’sReported A-76 Sa-     Fignree Are
                            Incomplete and Inaccwati




                            OMB officialssaid that they do not agreethat DOD needsto use a formula
                            to estimate original costs.They believe that installation officials doing a
                            cost study can prepare a reasonably accurate estimate of the original
                            operating cost of an activity on the basis of the installation management
                            study and on actual personnel, travel, materials, and supplies budgets.


                            Sufficient data to monitor A-76 savings projections are not being rou-
DOD’s A-76 Office           tinely collected. DOD’S reporting system collects only a few items of
Lacks Information on        information about performance of activities after the cost study decision
Modifications Made          has been made to contract out or keep the function in-house.Without an
                            adequate monitoring and evaluation system in place, DOD does not know
After the Cost Study        the extent to which projected costs or savings were realized. DOD does
                            not try to detect cost growth problems in activities after the studies
                            have been completed, and neither DOD nor OMBhas a complete view of
                            DOD’S A-76 program results. Projected savings do not reflect (1) modifi-
                            cations to contracts that affect the fiial contract price, (2) differences
                            between estimated and actual contract administration costs,or (3) modi-
                            fications to the revised in-house organization when activities remain in-
                            house.
                            A DOD A-76 official said that his office doesnot attempt to evaluate the
                            data it collects or determine the reasonsfor contract cost increases
                            reported to it, becauseonce an A-76 study has been completed, the
                            activities involved are no longer A-76 issuesand are therefore not
                            within his area of responsibility. Similarly, DOD’S A-76 office does not
                            want to collect additional information about the results of completed
                            studies. We believe that DOD’S position on this issue renders DOD’S sav-
                            ings figures questionable and prevents DOD from having a complete view
                            of A-76 program results.


Modifications to Contract   Cost increasessometimes are due to errors or omissions in work state-
Costs Not Analyzed          ments or other parts of the contract. Contract modifications to correct
                            such errors can causethe projected savings for a study not to be real-
                            ized. We reported in November 19883that work statements were often
                            incomplete or lacked the specificity of requirements and tasks needed
                            by the in-house work force to estimate costs and to enable contractors to
                            develop accurate bids. An Air Force survey completed in October 1988
                            of 11 larger studies completed in fiscal year 1986 found that of the 6

                            3Federalproductivity: DOD’sExperiencein contracting Out hnmercWy Available Activities
                            C-M,             Nov. 28, 19W.
                                                         .

                             Page 17                                                 GAo/GGD9oM OMB Circuku A-76
                           DOD’s Reported A-76 Savinga F’i@uw Are
                           Incomplete and Inacmrab




                           studies won by contractors, contracts for 2 were terminated becauseof
                           disagreementsabout the scopeof work defined by the contract. Another
                           of the six studies resulted in higher-than-expected contractor payments
                           (11 percent higher in fiscal year 1986,ZOpercent in 1987) for which the
                           Air Force survey was unable to determine the reason.

                           DOD’S A-76  reporting system collects only figures for the total contract
                           costs for each of the first 3 years of each contract, and DOD reports these
                           data to OMB’S A-76 office. Two of the military services have decided that
                           they need additional information on contract cost changes.Officials at
                           the Air Force Management Engineering Agency (AF’MEA)       said that Air
                           Force installations will, in the future, be required to report specific rea-
                           sons for cost increasesof 10 percent or more. The Army now requires its
                           installations to identify, by dollar amounts, the reasonsfor changesin
                           contract costs for 5 years after a cost comparison final decision and is
                           evaluating the extent to which it will ask for retroactive information
                           about such cost changes.


Estimated Contract         In a cost comparison, an estimate for contract administration costs is
Administration Costs Not   added to the contractors’ bids. This is an additional cost to the govern-
                           ment if a contractor wins the competition. OMB’S    A-76 Cost Comparison
Captured                   Handbook sets out authorized contract administration staffing levels,
                           based on the number of employeesof the revised in-house organization.
                           If the installation believes it will need more staff than specified by OMB
                           standards, it must request a waiver from its headquarters organization.

                           Contract administration costs can increase after a cost study has been
                           completed. For example, the Army Audit Agency reported in 1983 that
                           the average contract administration costs for 12 A-76 contracts it
                           reviewed were more than double the estimates used in the cost compari-
                           son. In a June 1989 report, the Army Audit Agency found that the con-
                           tract administration costs for 10 contracted-out activities were about
                           $2.3 million higher than originally estimated by the government.
                           DOD’S A-76  office lacks the information to compare actual to estimated
                           costs to determine if projected savings have been affected by a change
                           from the estimated staffing level. The office plans to add, as a data ele-
                           ment in its reporting system and data base,the number of personnel
                           performing contract administration duties for each completed study
                           won by a contractor. However, the office is not planning to include in its
                           A-76 data base any information on the estimated staffing level used in



                           Page 18                                         GAO/GGlMMhW3OMBCiiA-76
                             Appendix I
                             DOD’sReported A-76Savings Pigurea Are
                             Incomplete and Inaccurate




                             the cost comparison. Without such baseline information, increasesin
                             contract administration costs cannot be identified.


Information on               For in-house wins, problems have been reported with the implementa-
Modifications to In.-House   tion of revised in-house organizations. For example, in a 1984 report on
                             25 commercial activities that remained in-house,the Army Audit
Staffing
I--      -and Costs Not      Agency found 8 activities for which the revised in-house organization
Collected                    had not been implemented promptly or in the manner prescribed by the
                             managementstudy. It reported that the costs of in-house operations
                             were higher than projected. The Air Force survey completed in October
                              1988 (of 11 larger studies completed in fiscal year 1985) found that of
                             the 5 installations with in-house wins, 1 had not implemented the
                             revised in-house organization and 1 could not, except with considerable
                             research, determine whether actual costs after the win were above,
                             below, or the same as the winning in-house bid.

                             The services have established different ways to monitor the implemen-
                             tation of the revised in-house organization. A DOD Inspector General offi-
                             cial told us that the Navy controls implementation through its budget,
                             and the Air Force controls through billets or position staffing systems.
                             The Army now requires, for in-house wins, that the installation report
                             actual in-house organization costs and staffing for 5 years after the cost
                             comparison final decision. Service officials told us that it can be difficult
                             to track in-house activity costs over time becausean activity’s functions
                             often do not remain constant.

                             DOD's reporting system does not collect information (such as actual
                             costs) about the implementation of revised in-house organizations. A DOD
                             A-76 official said his office doesnot collect these costs becausethey do
                             not consider events occurring after an A-76 study has been completed to
                             be A-76 issues.However, becausethe OMBsupplement to Circular A-76
                             requires the revised in-house organization be implemented within 6
                             months of the cost-study decision, we believe collection of these data is
                             appropriate under the A-76 program.

                             DOD is not reporting required information on in-house costs to OMB. OMB'S
                             A-76 tracking system is designedto collect revised in-house organization
                             costs for three performance periods (up to 3 years) following the cost
                             competition final decision. Becauseit doesnot collect the data, DOD can-
                             not report the information to OMB.




                              Page 19                                         GAO/GGDs(Ms   OMB -      A-76
                                DOD’s Reported A-76 !%I-   pienres   Are
                                Incomplete and Inaccnmti




                            computerized A-76 data baseis the source from which DOD pro-
DOD’s A-76 Data Base DOD'S
                      vides program information to OMB and to Congress.When we examined
Contains Inaccuracies the data used to report savings,we found that DODexcluded about 13
and Incomplete        percent of all studies completed in fiscal years 1986 through 1988 from
                      its savings reports. These excluded studies represent at least $36 million
Savings-RelatedData h annual    (recurring)  savings.
                                 DOD'S data base,with information as of December31, 1988, showed that
                                 489 studies were completed during fiscal years 1986 through 1988. The
                                 DOD computer program used to calculate savings excluded 63 of these
                                 from its calculations. Of these 63, the program excluded 19 becauseof a
                                 problem causedby a DOD reporting instruction. The other 44 studies
                                 were excluded becausethe data base did not contain cost-related data
                                 neededto calculate savings, or it contained inconsistent data.
                                 In reviewing the cause of the missing and inconsistent data, we found
                                 that the computerized edit checks and error correction procedures of the
                                 DOD A-76 reporting system need improvement. Comprehensiveedit
                                 checks do not exist at the point where data are first entered into a com-
                                 puterized system- at the service level. Also, errors contained in DoD'S
                                 error identification listings are not being corrected in a timely manner.
                                 As a result, DOD'S reported savings continue to be inaccurate and remain
                                 uncorrected.


DOD Reporting Instruction        DOD'S instructions for reporting cost study information affects reported
CausesNeeded                     savings figures. Included in DOD'S September 1986 A-76 reporting proce-
                                 dures (DOD14100.33) is an instruction that if no contractor bids are
Information to Remain            received during a cost study, or all bidders are disqualified, then the
Unreported                       estimated cost of the revised in-house organization is not to be reported
                                 to DOD. DOD usesthis cost figure in the formula that calculates projected
                                 annual savings from a study. Nonreporting causesDODto exclude such
                                 studies from its calculation of total projected savings for the year, erro-
                                 neously reducing DOD’S   total reported savings.

                                 DOD’S  data base shows 19 of these studies were reported to DODas hav-
                                 ing been completed during fiscal years 1986 through 1988. To identify
                                 the effect of DOD’S reporting instruction, we calculated what DOD’S
                                 approximate savings estimates would have been for 18 of these 19 stud-
                                 ies (for 1 study, there were insufficient data to calculate savings). Table




                                 Pyre20                                         GAO/GGDgoM    OMB -      A-76
                                         AppendLxI
                                         DOD% Jleported   A-76   Savhign   FIJJIUW   Are
                                         Incomplete and Inaumrati




                                         I.1 shows approximately how much these studies would have changed
                                         DOD’S total projected savings4

Table 1.1: Projected Recurring Savings
Excluded by DOD Reporting Instruction    Fiscal vear                                 Number of studies      Excluded rwoiected savinas
                                         1986                                                        8                      $1.7 million
                                         1987                                                        8                       12.1 million
                                         1988                                                        2                        2.0 million
                                         Total                                                      18                    $15.8 million


                                         DOD’s reporting  instruction is inconsistent with OMB’S A-76 reporting pro-
                                         cedures,which do not provide any exception for the reporting of the
                                         cost of the revised in-house organization. DOD A-76 officials speculated
                                         that the information could be consideredproprietary and should there-
                                         fore be kept confidential. They agreed,however, that once the period
                                         for any appeals and protests has passed,such information can be dis-
                                         closed and should be reported.


Other Errors With                        To compute savings figures, DOD’S computer program requires that
Savings-Related Data                     installations report original (baseline) staffing for an activity, the bid
                                         (cost) and planned staffing for the revised in-house organization, and
                                         the cost of contracting out. If any of the neededdata are missing, or are
                                         inconsistent (for example, the in-house bid is reported as less than the
                                         contractor bid, yet the bid winner is identified-in another data field-
                                         as being the contractor), then the program excludes the study in ques-
                                         tion. For studies reported as completed in fiscal years 1986 through
                                          1988, DOD’S computer program excluded 44 studies from DOD savings
                                         reports becauseof problems with missing data or inconsistent cost-
                                         related data. For 26 of these 44 studies, enough information was availa-
                                         ble in the data base for us to calculate a minimum approximate pro-
                                         jected savings from a study and thus approximate the effect of these
                                         data errors. Table I.2 shows what we were able to identify as additional
                                         unreported savings estimates.




                                          4Themethodologyusedfor tables1.1and L2 is describedin appendix II.



                                          Page 21                                                        GAO/GGD9oM)   OMB circnlar A-76
                                          Appendix     I
                                          DOD'rhportedA-70Sa~FignreaAre
                                          incomplete       and Inacmuate




Table 1.2: Projected Recurring Saving8
Excluded Because of Data Problems         Fiscal year                          Number of studies   Excluded projected savings
                                          1986                                                 7                  $15.5 million
                                          1987                                                 5                      .5 million
                                          1988                                                13                    4.5 million
                                          Total                                               25                 520.5 million




Edit Checks and Error                      Installations initially report information in the data baseto their respec-
Corrections Need                           tive service headquarters, which in turn report the information to DOD.
                                           Each service has created and maintains its own computerized A-76 data
Improving                                  base and management information system, containing both non-required
                                           data elements (such as the study announcement date) and service-
                                           unique data elements (for example, the date when development of the
                                           work statement began). Data reported to each service by the installa-
                                           tions are subject to various checks by the servicesto ensure accuracy
                                           and completeness.DMDC also prepares a quarterly feedback report that
                                         . lists suspectederrors.

                                          We identified two problems with the existing system. First, comprehen-
                                          sive edit checks (checking for all incorrect data entries) often do not
                                          exist where most data are fii<ntered      into a computerized system-at
                                          the service level. As a result, when the data reach DOD headquarters
                                          level, large numbers of suspectederrors are being found. For example,
                                          the secondquarter 1989 feedback report listed 1,006 suspectederrors,
                                          such as an unreported cost comparison period or a missing or incorrect
                                          appeal result.

                                          Second,errors noted on DMDC’S error identification listings are not being
                                          corrected in a timely manner. We found 799 of the 1,006 suspected
                                          errors were for data involving 448 completed studies-most of which
                                          have been completed for more than a year. We noted that one service,
                                          with 662 completed studies in the data base, had only 26 suspected
                                          errors listed for those studies; another service, with 866 completed stud-
                                          ies, had 627 suspectederrors. Since OMB does not make retroactive
                                          changesto the cumulative savings figures it reports annually, the
                                          effects of untimely error correction include erroneous yearend savings
                                          reports to OMBand erroneous cumulative savings reports by OMB.

                                           Earlier in this report, we noted the extent of problems with data used to
                                           calculate savings. The quality of the remaining data is also important    .

                                                                           .


                                           Page 22
                        DOD’s Reprted  A-70 Savin@ F’i@uea Are
                        Incomplete and Knaccumte




                        for managementpurposes, For example, a changein severancepay pro-
                        cedures, effective March 28,1990, permits severancepay for federal
                        employees displaced by A-76 cost studies who go to work for successor
                        contractors. DOD already collects information on employeesdisplaced
                        becauseof A-76 contracting out. However, after reviewing fiscal year
                        1986 through 1988 data in the A-76 data base,we concluded that errors
                        in the data would prevent DOD from accurately reporting on or reaching
                        correct conclusions about these data-including whether OPM'S changeis
                        accomplishing its intended purpose of encouraging displaced federal
                        employeesto go to work for winning contractors.

                        DOD'S A-76  staff plans to begin checking service-provided information
                        using new, microcomputer-basederror routines. DOD'S A-76 official said
                        the first error check-checking cost study dates-was started in Sep-
                        tember 1989. He said when the system is fully operational, his office
                        would supply the error lists to the servicesquarterly, replacing the DMDC
                        feedback report.


                        About 6 years ago, DOD officials responsible for the A-76 program cre-
Special Computer        ated a special computer program to calculate annual (recurring) savings
Program Miscalculates   for studies completed each year. One phase of the program generatesa
Reported Total Annual   total annual savings figure that cannot be matched to the sum of the
                        savings from each individual study for that year. There is no clear ratio-
Savings                 nale for the use of this part of the special program, and continuing its
                        use will lead, in the future, to OMBand DOD having different total savings
                        figures.
                        To check the accuracy of the portion of the DOD computer program used
                        to calculate annual savings, we summed the savings estimates for the
                        426 studies of fiscal years 1986 through 1988 that DOD included in its
                        calculations, using DOD'S savings formula for each individual study. In
                        comparing our results with those of the computer program, we found
                        that part of the computer program miscalculates total savings. The dif-
                        ference in total savings figures produced by that portion of the com-
                        puter program and by summing is shown in table 1.3.DOD A-76 officials
                        could not explain why annual savings were not simply calculated on an
                        individual study basis and then summed, and they were not aware of the
                        magnitude of the difference.




                         Page23                                       GAO/GGD-30-53   OMB Circuhr   A-76
                                         Appendix I
                                         DOD’s Reported A-70 Savh@      Pi@ueu Are
                                         Iucomplete and Inaccnate




Table 1.3:Differences in Projected
Recurring Savings-Special     Computer                            Projected savings,         Projected swings,
Program vs. Summing                      Fiscal year                  DOD program’                    summing                     Difference
                                         1966                             $64.6 million               $88.9 million             + $4.3 million
                                         1987                              $85.7 million              $87.0 million             + $1 .3 million
                                         1988                            $129.3 million              $128.0 million              - $1.3 million

                                         aAt our request, DMDC provided calculated totals for studies completed in fiscal years 1986 through
                                         1988 using information as of December 31, 1988. These totals do not match those reported at the end of
                                         each fiscal year by DOD ($67 million, $76 million, and $114.5 million) because of changes to data made
                                         by services and agencies dunng quarterly updates subsequent to the year-end reports.


                                         Since the first quarter of fiscal year 1989, DOD has been providing com-
                                         puterized individual study information to OMB. OMBA-76 officials said
                                         that they intend to use this information in the future to calculate total
                                         DOD savings by summing acrossindividual studies. Thus, if DOD contin-
                                         ues to use its special computer program, total savings figures as calcu-
                                         lated by DOD and OMB will no longer match.


                                         Neither DOD nor OMBhas much information about the overall administra-
Lack of Information                      tive costs of the A-76 program, so that neither knows whether the pro-
on A-76 Program Costs                    gram is cost beneficial. The issue of A-76 program costs has becomea
                                         concern to several congressionalcommittees. For example, expenditures
                                         to administer the program have been a focus of congressionaldiscussion
                                         on DOD’S appropriation for fiscal year 1990.

                                         DOD plans to  stop collecting data on the hours expendedto do individual
                                         A-76 studies. While such data could be useful in determining the cost
                                         effectiveness of the program, A-76 officials at all levels noted difficul-
                                         ties with its collection.

                                         We found that little is known about actual DOD A-76 program costs. In
                                         January 1988, in testimony before the President’s Commissionon Priva-
                                         tization, the Deputy Assistant Secretary of Defensefor Installations said
                                         that DOD spends about $160 million to $300 million per year on the A-76
                                         program, and there are probably large costs in addition to that which
                                         DOD could not quantify.6 DOD’S A-76 and DMDC staffs could not provide us
                                         with any additional information on overall costs.An OMB A-76 official
                                         said that OMB’S only information on DOD costs was from about 1986,
                                         when DOD reported that there were more than 1,700 personnel working

                                          6A-76programcostsand savingsfigures should be comparedwith caution. Costsduring a year are
                                          one-timeand involve both studiescompletedthat year and studiesstill in progress.A-76 savings
                                          during year result from (1) studiescompletedin prior years(such studiesgeneratea streamof
                                          recurr@-annual-savings), and (2) studiescompletedin that year.



                                          Page 24                                                         GAO/GGMO-63       OMB circplar    A-76
                                           DOD’r Re~rted   A-76 Savings Fl@res    Are
                                           Incomplete and Inaccumte




                                           full time on the A-76 program throughout DOD. OMB’S A-76 official said
                                           that OMB doesnot believe information on the cost of the A-76 program
                                           should be routinely collected becausethe A-76 program is aimed at
                                           achieving managementefficiencies, which is what all government mana-
                                           gers should be doing. OMBhas said that managerswho meet what should
                                           be normal performance standards will not encounter any new require-
                                           ments in doing an A-76 cost study.

                                           DOD now  requires its componentsto collect and report the number of
                                           staff hours spent to do each cost study, but it is planning to eliminate
                                           this requirement. We believe that such information could help evaluate
                                           whether the studies being done represent the best use of A-76 resources.
                                           For example, someservice A-76 officials told us that for activities with
                                           10 or fewer full-time positions, the costs of doing an A-76 study
                                           exceeded,on average,the expected savings from such a study.6 Accord-
                                           ing to DOD’S data base, 189 (39 percent) of the 489 studies completed in
                                           fiscal years 1986 through 1988 involved 1 to 10 positions (table 1.4).
Table 1.4: Completed Studies involving 1
to 10 Positions (“Small Studies”)                                   All completed A-76 studies                      Small studies
                                                                         No. ot                                  No. ot
                                           Fiscal vear                 studies Positions studied               studies Positions studied
                                           1986                            182                    6,769              72                431
                                           1987                            148                    7,570              52                328
                                           1988                            159                   10,766              85                365
                                           Total                           499                  25.105              189              1.124


                                           DOD and service officials said that they do not use the data on hours
                                           expended, and the data being reported are unreliable becauseinstalla-
                                           tions are not using the samecriteria to measure the number of staff
                                           hours. OMB’S A-76 tracking system does not require the reporting of
                                           hours or costs to do individual A-76 studies. OMB considers study costs to
                                           be a normal cost of business.OMB’S   A-76 officials also noted the diffi-
                                           culty of obtaining reliable data. Our analysis of the data in the DOD A-76
                                           data base on hours expended found that most installations are either not
                                           reporting hours or are reporting numbers that do not appear to be com-
                                           parable to other studies7


                                           ‘%fficient data were not available for us to evaluatethe validity of this statement.
                                           7Wecategorizedthe hours of study that were reportedas “not comparable”if the total represented
                                           either lessthan 26 hours per position in the rev&xl in-houseorganization,or morethan 1,000hours
                                           per position.



                                           Page 25                                                        GAO/GGDo(MB OMB CYXrdar A-76
Appendix II

Objectives,Scope,and Methodology


                  We did this assignment.at the request of the Chairman of the Subcom-
                  mittee on Federal Services,Post Office, and Civil Service, SenateCom-
                  mittee on Governmental Affairs. As agreedwith the Subcommittee,we
                  focused on the A-76 efforts of the DOD, which has the most extensive
                  experience with A-76 cost studies. We did not review any other agencies.
                  Our objectives were to assess(1) the accuracy of DOD'S A-76 savings
                   data reported to OMB and (2) the availability of information on the
                   administrative costs of DOD'S A-76 program. We did not examine the part
                   of the A-76 program involving conversion of an activity to a contract
                   status without a full cost study, becauseDOD'S reported savings do not
                  include these conversions.

                  To assessthe accuracy of reported savings, we reviewed DOD'S proce-
                  dures for calculating and reporting A-76 program savings. We also
                  reviewed the completenessand consistency of the data on which DOD has
                  basedits savings reports, using a computer file extracted from DOD'S
                  A-76 data base,which we obtained from the DefenseManpower Data
                  Center, Rosslyn, Virginia. To assessthe availability of information on
                  the administrative cost of DOD'S A-76 program, we reviewed DOD report-
                  ing instructions and the administrative cost information in DOD'S A-76
                  data base.

                  We obtained background information from, and discussedour findings
                  and conclusions with, A-76 officials at OMB,DOD, and headquarters of the
                  Army, Navy, and Air Force. Sincethe Air Force ManagementEngineer-
                  ing Agency (AF+MF,A) at Randolph Air Force Basein Texas managesthe
                  Air Force A-76 managementinformation system, we also met with
                  AF’MJU officials. Becausethese three services account for more than 95
                  percent of all DOD A-76 reported savings, we did not interview officials
                  of other DOD components.We also reviewed prior reports on the A-76
                  program issued by GAO, DOD, and the servicesand other background
                  information.

                  To estimate approximate (and minimum) savings for the studies com-
                  pleted in fiscal years 1986 through 1988 that were excluded from DOD'S
                  savings reports, we used the following methodology to develop tables I. 1
                  and 1.2.This methodology is generally consistent with DOD'S formula for
                  estimating savings, but its application is dependent on the DOD data base
                  containing sufficient data.

              l   For studies where the bid was won by the contractor, we counted as
                  savings the difference between the bid of the revised in-house organiza-
                  tion and the cost of contracting out.


                  Page26                                        GAO/GGB~OMBcircularA-76
    Appendix II
    Objectives, !kope, and Methodology




l   For studies where the staffing of the revised in-house organization was
    reported as being equal to or lower than the staffing of the original
    (baseline) organization, but no cost of the revised in-house organization
    was reported, we basedour savings estimates on the reduction in the
    number of positions. We assumedthat the savings from each position
    reduction was the sameas the averagecost, per person, of the revised
    in-house organizations for all other studies completed during the same
    fiscal year that DOD included in its savings figures and which were won
    in-house.
l   For studies where there was no qualified contractor bid but the cost of
    the proposed in-house organization was reported despite the DOD report-
    ing instruction not to do so, we used the standard DOD formula to esti-
    mate savings.

    We restricted our analysis to information for studies completed in fiscal
    years 1986 though 1988 becauseDOD issued the most recent reporting
    procedures for its A-76 data base in September 1985, and becauseOMB
    Circular A-76 was significantly revised in August 1983. The extract
    from the DOD A-76 data basethat we used contained information on
    completed and in progress cost comparisonsas of December31, 1988.
    We did not verify information in the data baseby comparing it to source
    documents. We did our work between April 1988 and September 1989,
    in accordancewith generally acceptedgovernment auditing standards.
    We obtained comments on our report from OMB.Formal written com-
    ments were also requested from DOD but were not received.




                                   .

     Page 27                                       GAO/GGIMO-66OMB Circular A-76
Appendix III

CommentsFrom the Office of Management
and Budget


                                 EXECUTIVE     OFFICE     OF THE      PRESIDENT
                                    OFFICE   OF MANAGEMENT        AND BUDGET
                                              WASHINGTON   DC    20503

                                              February          12,   1990




               Mr. Richard L. Fogel
               Assistant Comptroller General
               United States General Accounting         Office
               Washington, DC 20548


               Dear Mr. Fogel:

                       Thank you for the opportunity  to provide comments on the draft
               report, OMR CIRCULAR A-76: DOD’s Reoorted S ‘nes Figures Are N
               Reliable. In providing our comments, we would :ye to draw a distinzion
               between the inaccuracies found with DoD’s data system and the inferences
               drawn from those inaccuracies.    The report assumes that (1) DoD savings are
               suspect and, as a consequence, (2) it is questionable whether the A-76
               program is achieving economy in government operations.

                      With respect to the inaccuracies, we recognize that there are aspects of
               the DoD system which can be improved.       The system was first instituted in
               the 1970’s and many of the identified weaknesses are holdovers from a time
               when the A-76 program was less stringent than today. For example, the use
               of a formula to derive original operating costs stems from a time when the A-
               76 program did not require that original costs be identi8ed.    Likewise, the
               requirement to track actual costs, both in-house and contract, after the
               completion of a cost study is less than two years old.

                       We concur, therefore, with GAO’s recommendations      that DoD use
               actual cost information to calculate the original costs of an organization and
               the revised costs of the in-house organization, and that it collect information
               on in-house and contract cost changes. We also concur with the
               recommendations     that DoD report all completed cost studies, conduct
               comprehensive edit checks on A-76 data, and place increased emphasis on
               correcting data errors. In response to the recommendation       that DoD and
               OMB jointly design a pilot to test an approach for reliably measuring A-76
               program administration costs, DoD has agreed to design such a pilot for
               DoD. We will monitor the progress of this effort.




                   Page 28                                                        GAO/GGIMO-66   OMB Circular   A-76
       While we agree with the need to improve the accuracy of the
system, we are particularly          concerned about the report's        inference
that as a consequence of system inaccuracies,                the DoD savings are
suspect and, by extension,             that the A-76 program may not be
achieving economy in government OperatiOnS.                  The report    findings
fail to support such an inference.               In fact, the examples in the
report demonstrate that the system tends to underestimate                         the
savings generated by the A-76 program.                   Without a sensitivity
analysis      reflecting     the impact of the inaccuracies          found to the
total    savings estimate,         it is impossible        to determine if the
magnitude of the inaccuracies            are so significant        as to make the
data unreliable         or the data insignificant.       We recommend that such
an analysis          be conducted to determine            if    the inaccuracies
significantly        affect the level of savings reported by DOD.
      We also question whether a review of the DODA-76 data system
"provides    insight    into OMB's use of the A-76 program to achieve
economy rn government operationa.lS          The inaccuracies      in the DoD
data do not necessarily          exist in other agencies' systems.         The
savings rates reported by the civilian          agencies, while varying by
agency and type of function,          are of the same magnitude      aa those
reported by DOD. Moreover, the savings generated by the A-76
program are quaLitative        as well as quantitative.       The report doea
not address the lqsavingsn generated through improved contract
administration,      quality,   management reporting and decision-making,
or the benefits       of establishing    a measurable baseline for future
performance evaluation.          We do not believe that they can be used
to support an argument that the A-76 program              does   not generate
savings either for DODor for civilian           agencies.
                                              sincerely,


                                              Peter J. Basso
                                              Assistant Director
                                              for General Management




                                          2




        Page29                                                 GAO/GGI)-BG68OMBCbcalarA-76
RecentGAO ReportsAddressing RelatedIssues


              1. NAVY CONTRACTING:Contract Administration Staffing Require-
              ments for Navy A-76 Studies, GA~/NSIAD~&~~~BR, July 26, 1988, for the
              Chairman, Subcommitteeon Government Efficiency, Federalism, and
              the District of Columbia, SenateCommittee on Governmental Affairs.

              We concluded that the projected contract administration staffing levels
              for the A-76 studies we reviewed were basedon specific work load data
              identified in the quality assuranceplans and were reviewed and
              approved as required by appropriate Navy guidance. The objectives of
              the review and approval processwere to eliminate any unnecessary
              staffing levels and to ensure that the staffing levels requested by the
              instalIations were adequately justified. Contract administration staffing
              levels that exceedthe ohin-authorizedlevels have not significantly
              affected the outcome of the Navy’s A-76 studies. While we did not
              review the appropriateness of OMB'S guidelines, Navy contract adminis-
              tration experience indicates the obu+authorizedstaffing levels may not
              be sufficient to administer A-76 contracts that involved multiple func-
              tions and technically complex tasks.
              2. FEDERAL PRODUCTIVITY: DOD'S Experience in Contracting Out Com-
              mercially Available Activities, GAO~XD-~XM, November 281988, for the
              Chairman, Subcommitteeon Federal Services,Post Office, and Civil Ser-
              vice, SenateCommittee on Governmental Affairs.

              We said that the A-76 processhas led to savings and encouragedcompe-
              tition for providing quality goodsand services.Nearly half of DOD'S A-76
              cost comparisons have resulted in decisionsto continue operating com-
              mercial activities with a government workforce. Many DOD managers
              support the concept of government/private sector competition and rec-
              ognize its objectives of seeking efficiencies and cost savings.

              However, major concernsraised by federal employeesand unions
              regarding various implementation issuescontinually cloud the A-76 cost
              study process.DOD A-76 cost studies often have beentime-consuming,
              difficult, disruptive, and threatening to activity managersand
              employees.
              Agencies have achieved only a fraction of past goals set by OMB for
              studying positions. OMB has recently set new goals, however, and these
              are nearly double those established in the past. Basedon agencies’past
              progress in meeting OMB’S    goals, and given the length of time cost studies
              take, it is unlikely that the new goals will be met.



              Page 30                                         GAO/GGD~     OlKB Circular   A-76
    Appedi~     IV
    Beant     GAO Repma Addressing
    Related   IMIles




    3. MANAGING THE GOVERNMENT:RevisedApproach Gould Improve
         Effectiveness, GAO/GGD-~Q-G,May 4,1989, to the Congress.
    OMES’S
    In the section discussing A-76, we said obstaclesagenciesidentified as
    hindering A-76 successinclude
l lack of relevancy of the program to agency operations;
. unrealistic study goals;
l a burdensome,time-consuming cost study process;
. unclear program objectives;
. limited OMB program resources;and
l poor congressionalrelations.




      Page 31                                     GAO/GGlMt6-63OMB circdar A-76
Appendix V

Major Contributors to This Report


A      ~, n --~------ -_   William M. Hunt, Assistant Dirtxtor
benem     wvernment        Nancy A. Patterson, Evaluator-in-Charge
Division, Washington,
                  V    I
                           John Broughton, Evaluator
                           Joanne M. Parker, Technical Adviser


                           David A. Schmitt, Regional Assignment Manager
Norfolk Regional
Office




                                                .

(41063B)                   Page 32                                   GAO/GGDs(Ms   OMB Circular   A-76