oversight

Defense Transportation: Monitoring Costs and Benefits Needed While Implementing a New Program for Moving Household Goods

Published by the Government Accountability Office on 2003-04-18.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

             United States General Accounting Office

GAO          Report to Congressional Committees




April 2003
             DEFENSE
             TRANSPORTATION
             Monitoring Costs and
             Benefits Needed While
             Implementing a New
             Program for Moving
             Household Goods




GAO-03-367
                                                 April 2003


                                                 DEFENSE TRANSPORTATION

                                                 Monitoring Costs and Benefits Needed
Highlights of GAO-03-367, a report to            While Implementing a New Program for
Congressional Committees.
                                                 Moving Household Goods



 The Department of Defense (DOD)                 The recommendations in DOD’s report to Congress have the potential to resolve
 spends more than $1.7 billion each              several long-standing problems found in the current personal property program,
 year to move and store over                     which manages the transportation and storage of household goods. The
 600,000 household goods                         recommendations, if implemented, would
 shipments when relocating military                  • reengineer the claims process to reduce the length of time it currently
 personnel. It conducted and                              takes to resolve claims for lost, destroyed, or damaged household goods
 evaluated several pilot program                          and increase the reimbursement rates that military personnel currently
 studies aimed at fixing its problem-                     receive for their losses;
 plagued program and, in 2002,
                                                     • use performance-based service contracts to improve the generally low
 issued a report to Congress with
 three recommendations. The 1997                          quality of service that DOD currently gets from the moving industry; and
 Defense Appropriations Act                          • put in place new information technology with interface capabilities to
 Conference Report directed GAO                           enable program managers and users to monitor in-transit shipments and
 to validate the results achieved by                      track the number and cost of shipments processed each year.
 the pilot programs. In response,
 GAO examined the extent to which                The recommendations in DOD’s report to Congress are supported by the
 DOD’s recommendations to                        Transportation Command’s evaluation of the pilot programs’ findings and should
 Congress (1) offer solutions to                 be implemented within budget constraints. DOD’s approach in conducting the
 long-standing problems in the                   evaluation was methodologically sound: It developed an evaluation plan to guide
 current program and (2) are                     its work and adjusted the plan when necessary to address differences in the pilot
 supported by the evaluation’s                   programs’ approaches. While the shipments included in the evaluation do not
 findings and should be                          represent all shipment types managed annually by DOD, GAO believes that the
 implemented. GAO also assessed                  evaluation results provide sufficient information to allow DOD to initiate actions
 the soundness of methodologies                  to improve its current personal property program.
 used by DOD to develop cost
 estimates to implement the                      GAO found that the soundness of methodologies used to develop DOD’s cost
 recommendations.                                estimates varied. Therefore, DOD’s ability to implement changes to the existing
                                                 program within the cost estimates DOD reported to Congress is uncertain. GAO
                                                 found that the estimate to implement the information technology
                                                 recommendation was $7 million rather than the $4 million to $6 million estimate
GAO recommends that DOD                          DOD reported to Congress. In developing cost estimates for the remaining
implement the recommendations                    recommendations, DOD did not provide the same level of evidentiary support
within budget constraints, quantify              for one of the three adjustments it used to align the pilot programs’ costs to
the risk associated with achieving               current program costs. As a result, GAO questions the extent to which these
its cost estimates, monitor costs                recommendations can be implemented within DOD’s estimated 13 percent
during the implementation phase,                 increase over current program costs. While DOD believes it used a conservative
and assess the new program to                    approach in developing this 13 percent estimate, it has not quantified the risk
determine if anticipated                         associated with the projection, which could provide the military services and
improvements were achieved at a                  Congress information needed to develop and review future budget requests for
reasonable cost. DOD agreed with                 this program. Without providing the range of possible cost increases and the risk
three recommendations, but did not               regarding the likelihood of achieving this 13 percent projection within that
agree with the need to quantify the              range, DOD may find a repeat of what occurred during the pilots, where the
risk associated with achieving its               military services terminated participation in one of the pilot programs due to
cost estimates.                                  costs exceeding projections.
www.gao.gov/cgi-bin/getrpt?GAO-03-367.
                                                 GAO also found that without carefully monitoring costs during the
To view the full report, including the scope     implementation phase and assessing costs and benefits from a period
and methodology, click on the link above.
For more information, contact William M. Solis
                                                 succeeding full implementation of the recommendations, DOD would not have
at (202) 512-8365 or solisw@gao.gov.             the information needed to determine if anticipated improvements in the program
                                                 are being achieved at a reasonable cost.
Contents


Letter                                                                                            1
                       Results in Brief                                                           2
                       Background                                                                 5
                       DOD’s Recommendations Offer Solutions to Long-Standing
                         Problems                                                                 8
                       Transportation Command’s Evaluation of Pilot Programs Supports
                         DOD’s Three Recommendations                                            13
                       Ability to Implement New Program within Cost Estimates Reported
                         to Congress Is Uncertain                                               16
                       Conclusions                                                              21
                       Recommendations for Executive Action                                     22
                       Agency Comments and Our Evaluation                                       23

Appendix I             Scope and Methodology                                                    28



Appendix II            Overview of Current Personal Property Program
                       and Pilot Programs                                                       32
                       Current Program                                                          34
                       Military Traffic Management Command’s Reengineered Personal
                         Property Program                                                       36
                       The Department of Defense’s Full Service Moving Project                  39
                       Navy’s Service Member Arranged Move Pilot Program                        42

Appendix III           Comments from the Department of Defense                                  45



Related GAO Products                                                                            49



Tables
                       Table 1: Claims-Related Features of the Current Personal Property
                                Program and Pilot Programs                                      32
                       Table 2: Quality of Service-Related Features of the Current
                                Personal Property Program and Pilot Programs                    33
                       Table 3: Data Reliability-Related Features of the Current Personal
                                Property Program and Pilot Programs                             34




                       Page i                                     GAO-03-367 Defense Transportation
This is a work of the U.S. Government and is not subject to copyright protection in the
United States. It may be reproduced and distributed in its entirety without further
permission from GAO. It may contain copyrighted graphics, images or other materials.
Permission from the copyright holder may be necessary should you wish to reproduce
copyrighted materials separately from GAO’s product.




Page ii                                              GAO-03-367 Defense Transportation
United States General Accounting Office
Washington, DC 20548




                                   April 18, 2003

                                   Congressional Committees

                                   Military personnel and their families can expect to relocate many times
                                   during a service member’s career. As the moving industry’s single largest
                                   customer, the Department of Defense (DOD) spends more than $1.7 billion
                                   annually for its personal property program, which provides household
                                   goods transportation and storage services for military personnel and their
                                   families when they relocate. The program manages more than 600,000
                                   personal property shipments each year. DOD has experienced long-
                                   standing problems with its current personal property program, including
                                   excessive loss or damage to property, high claims costs incurred by the
                                   government, and poor quality of service from moving companies.
                                   Moreover, the program’s data management system does not provide
                                   reliable information on the status of individual shipments or on the types
                                   of shipments and their costs.

                                   In an effort to test alternative approaches and improve the quality of its
                                   personal property program, DOD has carried out several initiatives over
                                   the past 8 years. In 2000, the U.S. Transportation Command1 began to
                                   collect data from one of the three pilot programs to evaluate alternative
                                   approaches for improving the current program.2 The Transportation
                                   Command compared the features of the current program with those of the
                                   three pilot programs, and in June 2002 it submitted a report to the Deputy
                                   Under Secretary of Defense (Logistics) outlining its evaluation strategy,
                                   findings, and recommendations. In its November 12, 2002, report to
                                   Congress, DOD included the three recommendations resulting from the
                                   evaluation and the estimated additional costs required to implement
                                   program improvements based on a subsequent assessment of pilot and
                                   current programs’ operations.




                                   1
                                    The Deputy Under Secretary of Defense (Logistics) tasked the U.S. Transportation
                                   Command to evaluate alternatives and develop recommendations to enhance the
                                   department’s current personal property program.
                                   2
                                    The three pilots included in the evaluation are the Military Traffic Management
                                   Command’s Reengineered Personal Property Program, the Department of Defense’s Full
                                   Service Moving Project, and the Navy’s Service Member Arranged Move Pilot Program.



                                   Page 1                                              GAO-03-367 Defense Transportation
                   Our involvement in this issue stems from the conference report on the
                   1997 Defense Appropriations Act, which directed us to review the data
                   collected from the pilot programs and validate the results before DOD
                   expands any of the pilot programs, and a subsequent request from the
                   Subcommittee on Readiness of the House Committee on Armed Services.
                   This report supplements our previous testimony3 on DOD’s efforts to
                   enhance its current program, as well as our status report4 and status
                   briefings to Congress on DOD’s efforts to implement a methodologically
                   sound evaluation of the pilot programs. As agreed with your offices, for
                   this report we assessed the extent to which the recommendations in
                   DOD’s November 2002 report (1) offer solutions to long-standing problems
                   in the personal property program and (2) are supported by the evaluation’s
                   findings and should be implemented. In addition, we assessed the
                   soundness of the methodology used by DOD to develop the cost estimates
                   for implementing the recommendations.

                   To conduct our assessment of DOD’s three recommendations to Congress,
                   we reviewed DOD and GAO prior reports on the personal property
                   program, interviewed DOD officials and private-sector contractors
                   involved in the current and pilot personal property programs and the
                   Transportation Command’s evaluation, examined the methodology and
                   findings of the Transportation Command’s evaluation of the three pilot
                   programs, reviewed the methodology and data generated by each of the
                   three pilot programs, and assessed the methodologies used to develop the
                   cost estimates for implementing DOD’s recommendations. We did not
                   make an assessment of whether the anticipated benefits to be derived
                   from implementing the three recommendations would warrant the
                   additional costs DOD projects will be required to fund these
                   improvements. The scope and methodology we used in our review are
                   described in further detail in appendix I.


                   The three recommendations in DOD’s report to Congress offer solutions to
Results in Brief   several long-standing problems in the current personal property program.
                   Specifically, the recommendations address previously identified problems
                   with the liability/claims process (including the lengthy claims process,



                   3
                    U.S. General Accounting Office, Defense Transportation: Efforts to Improve DOD’s
                   Personal Property Program, GAO/T-NSIAD-99-106 (Washington, D.C.: Mar. 18, 1999).
                   4
                    U.S. General Accounting Office, Defense Transportation: Final Evaluation Plan Is
                   Needed to Assess Alternatives to the Current Personal Property Program,
                   GAO/NSIAD-00-217R (Washington, D.C.: Sept. 27, 2000).




                   Page 2                                             GAO-03-367 Defense Transportation
reimbursement rates that may not fully compensate service members for
loss and damage incurred during their moves, and high claims costs to the
government), the low quality of shipping services, and the inability to track
shipments and their costs. If implemented, DOD’s first recommendation—
to reengineer the claims process—has the potential to reduce the length of
time it currently takes to resolve claims for lost, destroyed, or damaged
household property and increase the reimbursement rates that military
personnel currently receive for their losses. The recommendation also has
the potential to reduce the high cost of claims incurred by DOD. The
second recommendation—to use performance-based service contracts—
has the potential to improve the generally low quality of service that DOD
currently receives from moving companies. The third recommendation—
to put in place new information technology with interface capabilities—
has the potential to enable the personal property program managers and
other users of the program to reliably monitor in-transit shipments and
track the number and the cost of shipments processed each year so that
accurate reporting can be provided to DOD and Congress.

The recommendations in DOD’s report to Congress are supported by the
Transportation Command’s evaluation of the pilot programs’ findings and
should be implemented within budget constraints. We found that the
Transportation Command’s approach in conducting the evaluation was
methodologically sound: It developed an evaluation plan to guide its work
and adjusted the plan when necessary to address differences in the pilot
programs’ approaches. While the shipments included in the evaluation do
not represent all shipment types managed annually by DOD, we believe
that the evaluation results provide sufficient information to allow DOD to
initiate actions to improve its current personal property program.

Our review showed that the soundness of the methodologies used by DOD
to develop cost estimates for implementing the three recommendations
varied. Therefore, DOD’s ability to implement changes to the existing
program within the cost estimates reported to Congress is uncertain. Our
review found that the total initial cost for implementing the information
technology improvements recommendation would more likely be
$7 million rather than DOD’s $4 million to $6 million estimate. We agreed
that the premise of two of three adjustments DOD used to develop the
13 percent cost increase to implement the claims process and
performance-based service contract recommendations was sound.
However, we are less assured about the extent to which the projected cost
savings related to a third adjustment may occur because the adjustment
was not supported by historical experience or by the same quality of data
provided for the other adjustments. While DOD believes it can incorporate
the three recommendations into a new program within its proposed



Page 3                                       GAO-03-367 Defense Transportation
13 percent increase due to the conservative approach it took in developing
this estimate, we believe that by quantifying the risk associated with this
projection, DOD could provide the military services and Congress
information needed to develop and review future budget requests for this
program. The need for this type of information is further supported based
on the long-standing problems associated with the current program and
the large increase in costs contributing to the military services’ decision to
terminate participation in one of the pilot programs. Further, without
carefully monitoring costs during the implementation process and
assessing costs and benefits from a period succeeding full implementation
of the recommendations, DOD will not have the information needed to
determine if anticipated improvements in the program are being achieved
at a reasonable cost. Currently, DOD is beginning planning efforts to
implement the recommendations. These efforts do not include monitoring
and evaluating costs and benefits during the implementation phase and
post implementation of the recommendations in a new program.

We are recommending that DOD initiate actions that will implement the
recommendations contained in its report to Congress within budget
constraints, quantify the risk associated with achieving its cost estimates,
monitor costs during the implementation phase to ensure that the
proposed changes are being achieved within an acceptable and a
predefined range, and assess the personal property program after the
recommendations have been implemented to determine whether
anticipated improvements are being achieved at a reasonable cost.

In commenting on a draft of this report, DOD concurred with three of our
four recommendations, including initiating actions to implement the
recommendations contained in its report to Congress, monitoring costs
during the implementation phase, and assessing the program after DOD’s
recommendations have been implemented. DOD partially concurred with
the remaining recommendation, i.e., to provide the military services and
Congress with additional information to quantify the risk associated with
achieving the projected 13 percent cost estimate. In its response, DOD
stated that the 5 percent reduction that it made to the pilot programs’
average costs to adjust for economies of scale/program efficiencies was
reasonable and very conservative. Further, DOD continues to believe that
the program can be implemented within the 13 percent increase and noted
that one of the military services validated this estimate. Therefore, DOD
did not see the value added in providing the military services or Congress
a formal risk assessment. We find that these statements still do not
provide a basis for the 5 percent reduction and do not indicate the level of
risk associated with implementing the recommendation within this
estimate. We continue to believe that this information needs to be



Page 4                                        GAO-03-367 Defense Transportation
             developed to help the military services prepare their budgets. The military
             services terminated participation in one of the pilot programs because
             actual costs were significantly greater than the projections provided to
             them for budgetary purposes. Providing a measure of risk for the
             13 percent projection could help prevent a repeat of what occurred during
             the pilot programs. Without this risk information, the military services will
             have to wait until after the moving industry submits bids to find out if
             DOD’s projection was reliable.


             DOD’s personal property program is managed centrally by the Military
Background   Traffic Management Command headquarters and administered locally by
             about 200 military service and DOD transportation offices around the
             world. The program relies on more than 1,200 domestic commercial
             carriers and 150 freight forwarders for international shipments to provide
             household goods transportation and storage services for military
             personnel and their families when they relocate. The military services pay
             shipment and storage-related costs from their military personnel accounts
             and loss and damage claims and personal property shipment office
             expenses through their operations and maintenance accounts. The
             program has remained virtually unchanged for nearly 40 years. It involves
             a complex process of qualifying carriers, soliciting rates, distributing
             moves, evaluating transportation providers’ performance, paying invoices,
             and settling claims. Among the program’s many challenges is ensuring that
             the moving industry provides adequate year-round capacity, especially
             during the summer peak-moving season when most service members, as
             well as the general public, schedule their moves.

             In prior reports, both DOD and GAO have identified problems related to
             the loss and damage claims process and the low quality of service from
             movers. In designing and implementing its evaluation plan, the
             Transportation Command also noted that weaknesses in the current
             program’s data management system precluded DOD from being able to
             track shipments in transit and from being able to extract reliable data on
             the number and types of shipments managed annually and their associated
             costs.

             In response to the long-standing problems, DOD has undertaken a number
             of pilot program studies to find ways to improve the process of shipping
             service members’ household goods. In August 1996, the Deputy Under
             Secretary of Defense (Logistics) tasked the Transportation Command with
             evaluating alternative approaches to the current program and
             recommending changes in the program based on the results of its
             evaluation. The Transportation Command identified three ongoing or



             Page 5                                       GAO-03-367 Defense Transportation
planned pilot programs to include in its evaluation and began to collect
data for its analysis from one of them in 2000. These three pilot programs
shared some common features, such as testing performance-based service
contracts and providing full replacement (rather than depreciated) value
for loss or damage. Each one also had some distinctive features, such as
allowing service members to participate in selecting their movers and
contracting out installation personal property shipment office functions to
private-sector move manager companies.

The three pilot programs are summarized below. Further information on
each program, as well as DOD’s current personal property program, is
provided in appendix II.

•   The Military Traffic Management Command’s Reengineered Personal
    Property Program operated from the military services’ and the Coast
    Guard’s installations located in North Carolina, South Carolina, and
    Florida. It used military installation personal property shipment office
    personnel, as the current program does, and developed a new data
    management system that tracked both the movement of individual
    shipments and information on the number and cost of shipments.

•   The Department of Defense’s Full Service Moving Project operated
    from the military services’ and the Coast Guard’s installations located
    in the National Capital Region (the Washington, D.C., metropolitan
    area), Georgia, and North Dakota. It contracted the management of
    shipments to private-sector companies and offered optional relocation
    services, such as referrals for rental assistance and purchase and sale
    of real estate services, to personnel participating in the pilot program.

•   The Navy’s Service Member Arranged Move Pilot Program operated
    from Navy installations located in the states of California, Connecticut,
    Virginia, and Washington. It designated current staff within the
    installation personal property shipment offices as “move coordinators”
    to provide assistance, allowed participants to pre-select transportation
    providers, and paid for moves through government purchase cards.

In June 2002, the Transportation Command submitted a report containing
its evaluation results and proposed three recommendations to the Deputy
Under Secretary of Defense (Logistics). After reviewing the results and
receiving comments from the military services, DOD submitted its report,
dated November 12, 2002, to Congress. DOD’s report contained the same
three recommendations contained in the Transportation Command’s
report. DOD also provided cost estimates for implementing the
recommendations. The three recommendations were to



Page 6                                        GAO-03-367 Defense Transportation
•   reengineer the liability/claims process by adopting commercial
    practices of minimum valuation, simplifying the filing of claims, and
    providing direct settlement with the carrier;

•   change the acquisition process to implement performance-based
    service contracts; and

•   implement information technology improvements, which could
    interface functions across such areas as personnel, transportation,
    financial, and claims.

In its report to Congress, DOD estimated that reengineering the
liability/claims process and changing the acquisition process to implement
performance-based service contracts would increase the current
program’s estimated $1.7 billion cost by 13 percent. Implementing the
information technology improvements to enhance its data management
capabilities and to provide training to users was estimated at an additional
$4 million to $6 million.

DOD also estimated that efforts to implement the changes to the current
program would be completed by the first quarter of fiscal year 2005. DOD
has developed a plan of action and milestones for designing the new
personal property program. This initial effort identifies several teams,
which are exploring the following issues: the acquisition/solicitation
process, quality assurance, the liability/claims process, information
systems technology, and electronic billing and payment. Four of these
issues address the recommendations included in DOD’s November 2002
report to Congress. The plan identifies a list of essential activities needed
to carry out the responsibilities required to build the future personal
property program. It also includes time lines and identifies a process to
monitor problems and delays. However, it does not include monitoring
costs and benefits during the implementation phase and the extent the
proposed changes are being achieved within an acceptable and a
predefined range. Further, it does not include evaluating the extent the
benefits from the pilot programs will be achieved after the new program is
implemented to determine whether the anticipated improvements were
achieved at a reasonable cost.




Page 7                                       GAO-03-367 Defense Transportation
                            The three recommendations in DOD’s report to Congress offer solutions to
DOD’s                       several of the current program’s long-standing problems, such as the
Recommendations             liability/claims process and the low quality of service. These problems
                            have been identified in DOD and GAO’s prior reports, as well as in surveys
Offer Solutions to          conducted as part of the pilot program evaluation. The inability to monitor
Long-Standing               shipments and shipping information has been long recognized and was
                            highlighted as an additional problem during DOD’s evaluation. If
Problems                    implemented, the first recommendation (i.e., reengineering the
                            liability/claims process by adopting commercial practices of minimum
                            valuation, simplifying the filing of claims, and providing direct settlement
                            with the carrier) has the potential to help reduce the length of time it
                            currently takes to resolve claims for lost, destroyed, or damaged
                            household goods because the carrier recovery time would be eliminated
                            for most moves, increase the reimbursement rates military personnel
                            receive for their losses, and reduce DOD’s claims-related costs. The
                            second recommendation (i.e., changing the acquisition process to
                            implement performance-based service contracts) has the potential to help
                            improve the generally low quality of service that DOD currently receives
                            from the moving industry. The third recommendation (i.e., implementing
                            information technology improvements, which could interface functions
                            across areas such as personnel, transportation, financial, and claims) has
                            the potential to improve the program’s ability to reliably monitor and
                            collect data on the status and costs of shipments so that accurate
                            reporting can be provided to DOD and Congress.


Reengineered                As part of its evaluation, the Transportation Command cited that one of
Liability/Claims Process    the long-standing problems with military household goods shipments is
                            the liability/claims process, including the (1) length of time it takes to
Has the Potential to        resolve claims, (2) low reimbursement rates, and (3) high cost of claims
Shorten Time, Increase      that DOD must pay.
Reimbursement Rates, and
Reduce Claims-Related
Costs to DOD

Length of Time to Resolve   In a study conducted in 1999, the Military Traffic Management Command
Claims                      reported that 146 days are expended between the time a claim is filed by a
                            service member to recovery of costs from the carrier by the government.
                            During this period, military personnel file their claims for lost, destroyed,
                            or damaged household goods with their respective military service’s
                            claims offices and receive settlements (this occurs, on average, within




                            Page 8                                       GAO-03-367 Defense Transportation
                              23 days), and then these offices file the claims against the carriers to
                              recover the costs (this step is completed within the 146 day period).5 In the
                              Transportation Command’s pre-evaluation survey completed in 2000,
                              responses from military personnel who had recently moved indicated that
                              one of the lowest performance ratings involved the time required to settle
                              a claim.

                              Based on the Transportation Command’s evaluation of the claims process
                              under the pilot programs, one of the results from implementing the pilot
                              programs was the 146 day average required under the current program to
                              settle claims and recover costs was reduced to an average of 30 days since
                              the service member filed directly with the carrier and the military services
                              did not have to recover costs. Under each pilot program, military
                              personnel settled claims directly with the carriers. Service members who
                              were not satisfied with offers made by the carriers could file their
                              unresolved claims directly with DOD. Military services worked these
                              claims with the carriers and if a claim was justified, the service member
                              received just settlement under the pilot programs (i.e., if the items were
                              lost or destroyed, the member received full replacement value, while
                              damaged items were repaired).

Low Reimbursement Rates for   In its pre-evaluation survey, the Transportation Command found that
Loss and Damage               military personnel cited low reimbursement amounts that typically do not
                              cover the loss or damage of household goods as a major concern during
                              their moves. Under the current program, a carrier’s liability is limited to
                              $1.25 per pound multiplied by the shipment weight. Personnel receive only
                              the depreciated value of lost, destroyed, or damaged items, up to a
                              maximum of $40,000 per move.6 When arranging their moves under the
                              current program, military personnel can buy increased insurance coverage
                              from their carrier, up to a full replacement value limit of $3.50 per pound
                              times the shipment weight, at a cost of 85 cents per $100 of the stated
                              value of the shipment. However, only military personnel making moves



                              5
                               DOD reported that the 146 day recovery time is due to the fact that all current DOD
                              contracts for shipment and storage of household goods give the carriers 120 days from
                              receipt of the government’s demand to pay, deny, or make a final written offer on the
                              claims. While many claims are settled in less that 120 days, mailing time and negotiations to
                              resolve disputes result in a slightly higher average settlement time.
                              6
                                For example, a 10,000-pound shipment would have a maximum carrier liability for loss
                              and damage of $12,500. Service members can make shipments exceeding their weight
                              allowance by paying the extra shipping and storage costs for the overage in weight.
                              However, the government’s liability is limited to $40,000 per move regardless of the excess
                              weight shipped and stored.




                              Page 9                                                GAO-03-367 Defense Transportation
                               within the continental United States can buy this additional coverage; it is
                               not available to those moving to or from overseas posts. Another option
                               that military personnel have to increase their protection for loss and
                               damage is to buy additional coverage from private-sector sources.

                               Unlike the current program, the pilot programs provided full replacement
                               value for lost and destroyed goods, with maximum amounts ranging from
                               $63,000 to $75,000. Damaged items were repaired. Two of the three pilot
                               programs reported that their cost per pound times the shipment weight
                               rates were $3.50 for $63,000 maximum coverage and $6.00 for $75,000
                               maximum coverage, respectively. The remaining pilot program did not
                               give a cost per pound, stating only that its maximum coverage rate was
                               $72,000.

High Costs to Government for   DOD has reported that, historically, approximately 35 percent of all moves
Claims                         result in loss or damage claims. A 1997 Military Traffic Management
                               Command survey of 3,000 moves revealed that while 65 percent of
                               shipments had loss or damage, only 35 percent resulted in claims being
                               filed. DOD pays approximately $100 million a year in claims but recovers
                               only 60 to 65 percent of the amounts paid to military personnel from the
                               moving industry. These figures understate the actual loss and damage,
                               since all military personnel do not file claims, apparently because the
                               process takes a long time and reimbursement rates do not always cover
                               the losses. DOD incurs these losses due to the structure of its current
                               program. The military claims offices assist service members by arranging
                               to pay their claims and then submitting the claims to the respective
                               movers for reimbursement. As indicated above, DOD receives only partial
                               reimbursement from the moving industry.

                               If the recommendation is implemented, DOD expects to reduce a
                               substantial portion of the estimated $100 million it currently pays in claims
                               each year to service members and eliminate much of the 35 to 40 percent
                               in losses it incurs from settling claims with the moving industry because
                               service members will be settling claims directly with their carriers. DOD
                               also expects additional savings because fewer demands would be placed
                               on military claims officials to manage the claims process. DOD believes
                               that these savings will help offset the higher costs of providing full
                               replacement value to service members for any loss and damage incurred
                               during the shipment and storage of their personal property.




                               Page 10                                      GAO-03-367 Defense Transportation
Change to Performance-     Our work has shown that another long-standing problem with the current
Based Service Contracts    personal property program is the poor quality of moving services provided
                           to military personnel. The high number of loss and damage claims that
Could Improve Quality of   military personnel file underscores this problem. According to the two
Moving Services            pre-evaluation surveys cited in the Transportation Command’s evaluation,
                           around 55 to 65 percent of respondents reported suffering some loss or
                           damage of household goods during a recent move. Moreover, in the
                           pre-evaluation survey conducted by the Transportation Command, the top
                           four factors identified by service members as being of greatest importance
                           to them in the moving process were the quality of packing, the care in
                           handling personal property, the condition of their property upon receipt at
                           the end of the move, and the receipt of fair payment for any losses or
                           damages they suffered. In individual comments obtained during the
                           pre-evaluation survey, the Transportation Command reported that some
                           service members also cited the lack of professionalism and quality of
                           customer service on behalf of moving crews as a concern.

                           The problem stems primarily from the current program placing greater
                           emphasis on costs (i.e., the lowest bids) than on the quality of service that
                           carriers provide when moving shipments of military household goods.
                           While the current program established its Total Quality Assurance
                           Program to measure quality, data collected to develop scores for each
                           carrier includes three measures (timeliness of pickup, timeliness of
                           delivery, and reported loss and damage), which are not collected for all
                           household goods shipments. The best indication of quality, customer
                           satisfaction, is not measured in the current program. The problem of
                           quality is further exacerbated by the program’s use of a 20 year-old tariff
                           schedule that carriers use in developing their bids. This tariff contains
                           lower rates than the current commercial tariff used during the pilot
                           programs.

                           Unlike the current program, the pilot programs screened carriers that
                           wanted to participate in their programs by emphasizing the quality of
                           carriers’ prior performance rather than the amount of their bids. For
                           example, the Full Service Moving Project contracted a financial services
                           company to conduct a financial and performance assessment of potential
                           movers. The pilot program emphasized best value and placed more
                           emphasis on performance (70 percent) than cost (30 percent) in
                           determining which providers were awarded shipments. The pilot programs
                           showed that these types of contracts could allow the government to
                           pre-screen carriers for financial viability and, more importantly, to
                           institute and maintain a quality assurance process to reduce losses and
                           improve service.




                           Page 11                                      GAO-03-367 Defense Transportation
                              In addition to prescreening carriers for quality control purposes, the pilot
                              programs also surveyed military personnel who participated in the
                              programs and used the results to distribute future shipments to carriers
                              that received the best performance scores. To address concerns about the
                              obsolete tariff schedule, the pilot programs adopted current commercial
                              tariffs for carriers to use in establishing their bids.

                              The pilot programs also showed that the solicitation process could be
                              streamlined by eliminating detailed statements of work and that the pilot
                              programs could place responsibility for successful performance on
                              carriers, allowing the government to focus on outcomes, rather than
                              processes. Finally, the pilot programs demonstrated that using these types
                              of performance-based service contracts did not have an adverse effect on
                              small business participation, a major concern of the moving industry. On
                              the basis of the total dollar value of shipments, each pilot program
                              exceeded the Small Business Administration’s goal of 23 percent
                              participation for the industry. Specifically, 48 percent of the Military
                              Traffic Management Command’s Reengineered Personal Property
                              Program’s revenues, 74 percent of the Full Service Moving Project’s
                              revenues, and 100 percent of the Navy’s Service Member Arranged Move’s
                              revenues went to small businesses.


Implementing Information      Another ongoing problem with the current personal property program is
Technology Improvements       its inability to provide reliable data on the status of in-transit shipments or
                              on the number and associated costs of shipments managed by DOD each
to Address Data Reliability   year. Because of the lack of reliable data on shipments and costs, program
Problems                      managers have no way of knowing the actual costs of moving military
                              personnel’s household goods. In addition, they have no access to real-time
                              tracking data that they could use to manage transportation and storage
                              costs and to help cut down on the need for temporary storage by reducing
                              the number of failed deliveries.

                              Two of the pilot programs included features to address the problems
                              associated with the current program’s stand-alone data management
                              system. The pilot programs each developed a Web-based data management
                              system to enhance the visibility of individual shipments and provide more
                              reliable data on shipments and costs. For example, the Military Traffic
                              Management Command’s Reengineered Personal Property Program’s data
                              management system provided in-transit visibility. This made it possible to
                              track the status of individual shipments and gave real-time access to those
                              sections of the shipment records that various parties involved in the
                              relocation process needed for data entry or status review. The pilot
                              program’s data management system provided a complete picture of the



                              Page 12                                       GAO-03-367 Defense Transportation
                         service member’s move from start (the move application process) to finish
                         (the claims submission and resolution process). In addition, the data
                         management system demonstrated the potential to provide information to
                         personnel in various functional areas involved in the service members’
                         relocation process (such as personnel, transportation, financial, and
                         claims). Finally, the data management system demonstrated the potential
                         to provide data for planning and budgeting purposes on the types of
                         shipments made annually across DOD and their costs. The Full Service
                         Moving Project’s data management system was developed but not fully
                         implemented because the military services terminated their participation
                         in the pilot program due to its high costs. While the Navy’s pilot program
                         developed a database near the end of the Transportation Command’s
                         evaluation, the database was not fully implemented nor assessed as part of
                         the evaluation.


                         Our analysis indicated that DOD’s three recommendations are supported
Transportation           by the results of the Transportation Command’s evaluation of the three
Command’s                pilot programs. The Transportation Command adopted a sound
                         methodology to conduct its evaluation, and it adjusted this methodology
Evaluation of Pilot      when circumstances warranted. The results of the Transportation
Programs Supports        Command’s evaluation are based on data collected from a limited number
                         of geographical areas. While the shipments included in the evaluation do
DOD’s Three              not represent all shipment types managed annually by DOD, we believe
Recommendations          that the evaluation results provide sufficient information to allow DOD to
                         initiate actions to improve its current personal property program within
                         budget constraints.


Transportation Command   We found that the Transportation Command used a methodologically
Implemented a            sound approach to evaluate the results of the three pilot programs and
                         make its recommendations. Before it started the evaluation process, the
Methodologically Sound   Transportation Command considered some lessons learned that had
Evaluation               emerged from our review of the Hunter Pilot Program in 1999,7 and it
                         followed through with several of them. For example, it obtained assistance
                         from a contractor to design an evaluation plan that met professional
                         standards. The Transportation Command identified four aspects, or
                         factors, of the property program that served as the focus of its evaluation
                         (i.e., quality of life, total costs, small business participation, and process


                         7
                          U.S. General Accounting Office, Defense Transportation: The Army’s Hunter Pilot
                         Project Is Inconclusive but Provides Lessons Learned, GAO/NSIAD-99-129 (Washington,
                         D.C.: June 23, 1999).




                         Page 13                                           GAO-03-367 Defense Transportation
                             improvements). The evaluation plan also prescribed that only one quality
                             of life survey be administered to each participating service member in
                             order to avoid survey “fatigue” that can result from subjecting a person to
                             multiple surveys, and thus avoid the resulting potential for questionable
                             results. In designing the evaluation plan, the Transportation Command
                             incorporated a number of important evaluation features. These features
                             included assessing the four factors consistently across all three pilot
                             programs, ensuring that the evaluation received data from the pilot
                             programs during the same time period to avoid the need to make
                             adjustments due to potential changes in carrier operations and costs,
                             conducting a survey of service members using the current program to
                             establish a baseline from which to measure the pilot program results, and
                             developing a method to provide estimates of what DOD would have paid
                             for comparable shipments under the current program for those shipments
                             completed under the pilot programs.

                             The Transportation Command made appropriate adjustments to the
                             evaluation plan when it learned that the three pilot programs would not be
                             underway at the same time and that they would not provide all of the
                             information originally outlined in the plan. For example, the Full Service
                             Moving Project began later and terminated earlier than was expected, the
                             Navy’s Service Member Arranged Move Pilot Program failed to conduct a
                             quality of life survey and collect cost data as outlined in the evaluation
                             plan, and none of the three pilot programs provided costs associated with
                             individual process improvements. The Transportation Command included
                             qualitative analytical techniques so that it could include as much
                             information on each pilot program as possible in its evaluation while also
                             dealing appropriately with data limitations. The Transportation Command
                             also shifted the evaluation focus from the individual pilot programs to
                             specific features from the three programs, such as full replacement value
                             for loss and damage and the screening process for carrier participation.


Survey and Analytical Data   Our work indicated that the Transportation Command’s analysis of data
Support Command’s            collected from the three pilot programs supports the three
                             recommendations that DOD included in its report to Congress. The
Recommendations              Transportation Command’s analysis of household goods shipment data
                             from the pilot programs showed that the average amount of time that
                             service members and DOD spend to settle claims and recover costs from
                             carriers fell dramatically in all three pilot programs. In comparison with
                             the current program’s 146-day average, it took only 30 days, on average, to
                             settle a claim under the Reengineered Personal Property Program and the
                             Full Service Moving Project and fewer than 14 days under the Navy’s
                             program. Survey results indicated that full replacement (rather than



                             Page 14                                     GAO-03-367 Defense Transportation
depreciated) value, direct claims settlements, and anticipated
improvements in the claims process accounted for the highest increases in
satisfaction. Based on experiences during the pilot programs, DOD
believes that direct claims settlement between service members and
carriers should reduce claims costs DOD currently incurs. Under the
current program, DOD must collect from the carriers after it has paid the
service members’ claims. DOD expects that this step will be eliminated in
most instances because it is anticipated that service members will be
resolving most of their claims directly with their carriers.

The Transportation Command’s analysis of process improvement data,
interviews and observations during site visits, and survey results from the
pilot programs supported DOD’s recommendation to use performance-
based service contracts to improve the quality of services that the moving
industry provides to the military. The process of prescreening carriers
desiring to participate in the pilot programs on the basis of their financial
viability and past performance helped to eliminate poor performers.
Furthermore, the pilot programs’ use of post-move surveys allowed them
to get immediate and continuous feedback on the carriers’ performance
and to use this information to distribute future work to those carriers with
the highest performance ratings and best value. In addition, two of the
pilot programs reduced the amount of paperwork associated with
soliciting proposals and approving carriers.

Finally, the Transportation Command’s review and observations of two of
the pilot programs’ Web-based data management systems supported
DOD’s recommendation to overhaul the current personal property
program’s computer system (the Transportation Operational Personal
Property Standard System). The Transportation Command found that the
Reengineered Personal Property Program’s data management system
significantly improved communications between the various DOD offices
and the moving industry. The system gave real-time access to shipment
records to DOD’s personal property shipment offices, certifying officers,
prepayment auditors, military service headquarters, and military service
claims offices and finance centers, as well as moving industry participants.
Similar results occurred with the Full Service Moving Project’s Best Value
Distribution Database system, but the military services terminated their
participation in this pilot program before the system’s full potential could
be demonstrated.

While the shipments included in the evaluation do not represent all the
shipment types managed annually by DOD, we believe that the evaluation
results provide sufficient information to allow DOD to initiate actions to
improve its current personal property program.



Page 15                                      GAO-03-367 Defense Transportation
                       Our review found that the estimates DOD reported to Congress might
Ability to Implement   understate the total initial cost for implementing the information
New Program within     technology improvements recommendation and contain a questionable
                       adjustment for costs associated with the claims and contracting process
Cost Estimates         recommendations. Also, DOD did not quantify the risk associated with
Reported to Congress   implementing these latter recommendations within its projected
                       13 percent increase over the current program’s cost. Therefore, the ability
Is Uncertain           to implement changes to the existing program within the cost estimates
                       reported to Congress is uncertain.

                       Based on our discussions with Military Traffic Management Command
                       officials and review of available documents, we concluded that the total
                       initial cost to implement the information technology improvements
                       recommendation will more likely be $7 million rather than the $4 million
                       to $6 million estimate that DOD previously reported to Congress. In its
                       response to a draft of this report, DOD maintained that the costs to
                       implement a new Web-based data management system would fall within
                       its initial cost estimate of $4 million to $6 million. DOD’s projected cost
                       estimate includes $5 million for development and implementation of the
                       new system and $500,000 each for user training and system verification
                       and validation testing. At a minimum, based on these projected cost
                       estimates, the initial cost to implement the information technology
                       improvements recommendation would more likely be $6 million.

                       While we concur with the premise of two of the three adjustments used to
                       develop the 13 percent cost increase to implement the remaining
                       recommendations, we are less assured in the extent to which the projected
                       savings related to the third adjustment may occur. We found that the first
                       two adjustments were based on historical data. However, we question the
                       rationale DOD used to develop the third adjustment, as the savings
                       associated with this adjustment are based on assumed cost reductions
                       resulting from changes in program operations. Also, these reductions lack
                       the same quality of evidentiary support as DOD provided for the other two
                       adjustments.

                       DOD believes it took a conservative approach in developing the savings in
                       each of the three adjustments; therefore, it assumes that the proposed
                       changes to claims and the contracting process can be achieved within the
                       13 percent increase over the current program’s costs. Due to the long-
                       standing problems with this program and the high pilot program costs that
                       contributed to the military services’ early termination of participating in
                       one of the pilot programs, we believe that by quantifying the risk
                       associated with this projection, DOD could provide the military services
                       and Congress information needed to develop and review future budget



                       Page 16                                      GAO-03-367 Defense Transportation
                         requests for this program. Further, without carefully monitoring costs
                         during the implementation process and assessing costs and benefits from a
                         period succeeding full implementation of the recommendations, DOD will
                         not have the information needed to determine if anticipated improvements
                         in the program are being achieved at a reasonable cost. Currently, DOD is
                         beginning planning efforts to implement the recommendations. These
                         efforts do not include monitoring and evaluating costs and benefits during
                         the implementation phase and post implementation of the
                         recommendations in a new program.


Costs to Implement       The information DOD has provided on costs to implement the information
Information Technology   technology improvements recommendation varies. Information provided
                         during our review indicated that the total initial cost to improve the
Improvements Vary        current data management system would be higher than the $4 million to
                         $6 million DOD included in its report to Congress. DOD worked with the
                         contractor who developed the Reengineered Personal Property Program’s
                         Web-based data management system to develop an estimate of the cost to
                         expand the capabilities tested during the pilot program. Also included in
                         this estimate were funds to provide training for users of the new system.
                         Based on our discussions with officials from the Military Traffic
                         Management Command and our review of available documents, we
                         concluded that these costs would more likely be $6 million, as the data
                         management system development cost was projected to be $5 million with
                         an additional $1 million for user training. The need for this training as part
                         of a new personal property program was identified during DOD’s
                         evaluation of the pilot programs. We increased our overall projections for
                         the cost of the new system to $7 million when we learned that DOD
                         planned to continue spending at least another $1 million annually for
                         independent verification and validation testing and contractor support.
                         This latter expense was identified to us during discussions following
                         DOD’s submission of its report to Congress.

                         In its response to a draft of this report, DOD maintained that the costs to
                         implement a new Web-based data management system would fall within
                         its initial cost estimate of $4 million to $6 million. It projected a cost of
                         $5 million for system development and implementation and $500,000 each
                         for user training and initial system validation. At a minimum, based on
                         these projected cost estimates, the initial cost to implement the
                         information technology improvements recommendation would more likely
                         be $6 million. Because we did not assess the sufficiency of DOD’s original
                         estimates of $1 million each for training and validation testing, we are
                         unable to assess the impact of the reduction on the improvements in
                         information technology across DOD.



                         Page 17                                       GAO-03-367 Defense Transportation
                               Based on our discussion with DOD officials, we learned that the plan is to
                               implement this recommendation regardless of the status of the other two
                               recommendations because managers and users of the program need more
                               reliable information to manage the program’s shipments and their costs.
                               Funds to implement this recommendation would come from the military
                               services’ operations and maintenance accounts.


Soundness of Estimates to      The soundness of the three adjustments the Military Traffic Management
Develop Cost for Changing      Command used to develop its estimated 13 percent increase over the
                               current program costs to implement the remaining recommendations—the
Claims and Contracting         claims process and performance-based service contracts—varies. We
Processes Varies               found that two of these adjustments are based on reasonable assumptions
                               and are supported by historical experience and by data. The savings
                               associated with the third adjustment are based on assumed cost
                               reductions resulting from changes in program operations and lack the
                               same quality of evidentiary support as DOD provided for the other two
                               adjustments. Therefore, we are less assured in the extent to which the
                               savings associated with this adjustment may occur. Finally, we found that
                               in its report to Congress, DOD did not quantify the risk of achieving these
                               recommendations within the projected 13 percent increase. This
                               information is important to the military services as they develop their
                               military personnel and operations and maintenance budget requests and to
                               Congress as it assesses the reasonableness of these requests.

Estimate Includes a            In developing the 13 percent estimate, the Military Traffic Management
Questionable Cost Adjustment   Command determined that three adjustments to the average costs for the
                               pilot programs were required to develop the cost for the full rollout of a
                               new personal property program. The first two adjustments (i.e., reducing
                               the average weight of shipments and reducing costs to adjust for a mix of
                               small and large businesses) were made to offset differences between the
                               pilot programs’ shipments and those more typically managed across DOD.
                               The third adjustment was made to reduce the pilot programs’ costs to
                               reflect anticipated savings based on economies of scale.8 In developing
                               these adjustments, the Military Traffic Management Command worked
                               with a contractor and consulted with officials from the military services
                               and moving industry associations.



                               8
                                In later discussions, department officials stated that “economies of scale” should be
                               changed to “program efficiencies” to reflect a more efficient program with timely and
                               accurate management data; member counseling; and reduced loss, storage, and indirect
                               costs.




                               Page 18                                             GAO-03-367 Defense Transportation
While we believe that the shipment weight and small business mix
adjustments are reasonable, we question the extent to which the
economies of scale or program efficiencies adjustment may be achieved.
For the weight adjustment, the Military Traffic Management Command
determined that the average weights of moves in the two pilot program
areas were higher than those experienced in typical departmentwide
moves. As a result, the Military Traffic Management Command reduced the
pilot programs’ average weights to reflect the lower, more typical weights
to be used in calculating a total cost for a departmentwide program. This
adjustment resulted in a 12 percent drop in average costs. We found the
approach of using historical data to more accurately reflect the typical
shipment weights to be reasonable.

Next, the Military Traffic Management Command further lowered the pilot
programs’ average costs because the pilot programs had higher small
business participation rates than the departmentwide average, and small
businesses are typically more expensive than large businesses. Small
businesses accounted for 48 percent of the cost of all moves under the
Reengineered Personal Property Program and 73 percent under the Full
Service Moving Project. In addition, small businesses were 14 percent
more expensive per shipment in the Reengineered Personal Property
Program and 74 percent more expensive in the Full Service Moving Project
than what each pilot program paid to large businesses. In developing its
departmentwide estimate, the Military Traffic Management Command
used a small business participation target rate of 30 percent. This
30 percent target rate is higher than the Small Business Administration’s
23 percent goal for government agencies conducting business with this
industry. On the basis of this lower participation rate, the Military Traffic
Management Command reduced the pilot programs’ average costs further
by 8 percent. We agree that this adjustment in costs based on differences
in the pilot programs’ small business participation rate and the new
30 percent goal is a reasonable way to reflect the differences between the
pilot programs’ costs and the departmentwide-projected costs.

We found that the third adjustment that the Military Traffic Management
Command made—to reduce the cost of departmentwide shipments
because of economies of scale or program efficiencies—was not
adequately supported based on either historical experience or data that
DOD later provided. The Military Traffic Management Command reduced
the pilot programs’ average costs by 5 percent on the assumption that

•   the pilot programs’ shipments involved only a limited number of
    providers;




Page 19                                      GAO-03-367 Defense Transportation
                                •     the pilot programs only included a limited number of shipments while
                                      the current program manages over 600,000 shipments annually;

                                •     more accurate and timely management data that includes service
                                      member counseling, reduced losses, and storage and indirect costs will
                                      result in a more efficient program; and

                                •     overhead and operating costs will be spread due to a larger volume of
                                      shipments.

                                While recognizing that some changes may result from these anticipated
                                program efficiencies, the effect of these changes on potential cost savings
                                is uncertain at this time. The Military Traffic Management Command did
                                not provide the same level of evidentiary support that it provided on the
                                other two adjustments. Further, we believe that only time will determine if
                                DOD’s assumption for this adjustment, in particular, proves to be correct.

DOD Did Not Quantify the Risk   We found that DOD has not provided a level of assurance to the military
for Its Cost Estimate           services and Congress that its projected 13 percent increase over the
                                current program’s cost can be achieved. Quantifying the risk associated
                                with this projection could provide the military services assurance of the
                                viability of the projected 13 percent increase as they prepare budgets to
                                support the increased cost for this program. Congress could also use this
                                information as it reviews DOD’s requests for additional funds to
                                implement changes in this program. The need for this type of information
                                is further supported based on the long-standing problems associated with
                                the current program and the fact that shipment and storage costs under
                                the pilot programs were significantly higher than those that DOD
                                estimated it would have paid under its current program in the same
                                geographical areas. These costs ranged from 31 to 32 percent higher under
                                the Reengineered Personal Property Program and from 51 to 54 percent
                                higher under the Full Service Moving Project.9 These higher-than-
                                anticipated costs contributed to the military services’ decision to
                                terminate their participation in the Full Service Moving Project before its
                                test period ended.

                                While DOD did not quantify the risk, per se, it believes a conservative
                                approach was taken in developing the savings in each of the three
                                adjustments. As a result, DOD assumes that the proposed changes to the
                                claims and contracting processes can be achieved with its projected



                                9
                                    We believe that the methodology and data used to develop these cost estimates are sound.




                                Page 20                                                GAO-03-367 Defense Transportation
              increase of 13 percent over the current program’s budget. We still believe
              that the Military Traffic Management Command could have quantified the
              risk and provided this additional information to the military services and
              Congress as additional assurance of the likelihood of implementing the
              two recommendations within its projected 13 percent increase. The need
              for this information is further supported based on the long-standing
              problems DOD has experienced in this program, the fact that the military
              services terminated participation in one of the pilot programs due to the
              high cost increases, and the need to determine whether the proposed
              additional funds from military personnel and operations and maintenance
              accounts will be sufficient to implement the recommendations.

              In addition to the information that could be gained from quantifying the
              risk of its cost projection, we believe that only by careful monitoring
              during the implementation phase will DOD be able to ensure that the
              proposed changes are being achieved within an acceptable and a
              predefined range. Further, while we believe that the evaluation results
              support implementing plans to enhance the current program, it should be
              noted that the pilot programs’ shipments included in the evaluation were
              not typical of all types of shipments managed annually. Therefore, DOD
              was precluded from projecting the extent to which the recommended
              improvements can be achieved DOD-wide. Unless a subsequent evaluation
              is undertaken after the recommendations have been implemented, DOD
              will not be able to assess the extent to which the projected benefits are
              being achieved for military personnel, their families, and DOD, and
              whether the benefits are being achieved at a reasonable cost. Selecting an
              evaluation period to include the peak-moving season would also provide
              DOD with the information its needs to determine if the proposed changes
              can be achieved during the summer, when the demand for moving services
              by DOD and the private sector is at its highest.


              The three recommendations DOD developed from its evaluation of the
Conclusions   current and pilot programs, if implemented successfully, could enhance
              the quality of life for relocating service members and their families; reduce
              claims-related costs to DOD; and resolve problems related to the reliability
              of management information on the status of shipments and on the
              quantity, types, and costs of shipments that DOD and the military services
              manage annually. Delaying implementation of the recommendations only
              prolongs problems military personnel, their families, and DOD experience
              under the current program.

              DOD has not quantified the risk associated with achieving its projected
              13 percent increase over the current program’s costs to implement the



              Page 21                                      GAO-03-367 Defense Transportation
                      claims process and performance-based service contract recommendations.
                      Without quantifying the risk, the military services and Congress cannot be
                      assured that these recommendations can be achieved within this estimate
                      or whether additional funding or trade-offs may be needed. Further,
                      without careful monitoring during the implementation phase, DOD will not
                      be able to ensure that the proposed changes are being achieved within an
                      acceptable and a predefined range.

                      Because the pilot programs’ shipments included in the evaluation were not
                      typical of all types of shipments managed annually, it was not possible for
                      DOD to project the extent to which the recommended improvements can
                      be achieved departmentwide. Without evaluating the program following
                      implementation of the recommendations, DOD will be unable to assess the
                      extent to which the projected benefits for military personnel, their
                      families, and DOD are being achieved and, if so, whether they are being
                      achieved within a reasonable cost. Also, if DOD does not select an
                      evaluation period that includes the peak-moving season, it will not have
                      the information needed to determine if the proposed changes can be
                      achieved in the summer, when the demand for moving services is at its
                      highest.


                      To improve the personal property program for military personnel, their
Recommendations for   families, and program administrators, we recommend that the Secretary of
Executive Action      Defense direct the Commander, U.S. Transportation Command, to

                      •   initiate actions to implement the three recommendations contained in
                          DOD’s report to Congress within budget constraints,

                      •   provide the military services and Congress additional information to
                          quantify the risk associated with achieving the projected 13 percent
                          cost estimate before the claims process and performance-based service
                          contracts recommendations are implemented to provide the military
                          services with information needed for budgeting purposes,

                      •   monitor costs for all recommendations during the implementation
                          phase to ensure that the proposed changes are being achieved within
                          an acceptable and a predefined range, and

                      •   assess the effects of the three recommendations on the personal
                          property program (to be carried out after the summertime peak-moving
                          season once the recommendations have been implemented) to
                          determine whether the anticipated improvements in the program are
                          being achieved at a reasonable cost.



                      Page 22                                     GAO-03-367 Defense Transportation
                     In commenting on a draft of this report, DOD concurred with three of our
Agency Comments      four recommendations. For the first of these recommendations, DOD
and Our Evaluation   stated that it is developing a plan to implement those recommendations it
                     reported to Congress and anticipates its recommendations will be
                     implemented by the end of the first quarter of fiscal year 2006, assuming
                     the military services receive the additional funds needed to fund program
                     enhancements. In response to our recommendation to monitor costs
                     during the implementation phase, DOD stated that rolling out the new
                     program will require monitoring of costs to determine if the moving
                     industry partners are submitting bids that will allow DOD to enhance this
                     program within the projected 13 percent cost increase. Further, DOD plans
                     to include a process to conduct a rate reasonableness analysis upon
                     receipt of the rates. For rates found to be outside the range of
                     reasonableness, carriers will be given one opportunity to resubmit their
                     rates. DOD plans to only use those rates determined to be reasonable in
                     the new program. DOD also plans to include metrics, target/benchmark
                     performance indicators, and a methodology for data collection in an
                     updated program of action and milestone plan. For our recommendation,
                     i.e., assess the effects of the three DOD recommendations on the personal
                     property program to determine whether the anticipated improvements in
                     the program are being achieved at a reasonable cost, DOD plans to collect
                     data needed to determine if anticipated improvements have been achieved
                     on a continuing basis. DOD plans to use customer satisfaction surveys in
                     developing carrier performance ratings, which will be established
                     quarterly, with the exception of the peak season, when performance
                     ratings will be established monthly. If properly implemented, we believe
                     the proposed DOD actions will sufficiently address these
                     recommendations.

                     DOD partially concurred with the remaining recommendation, i.e., provide
                     the military services and Congress with additional information to quantify
                     the risk associated with achieving the projected 13 percent cost estimate
                     to provide the military services with information needed for budgeting
                     purposes. DOD continues to believe that the 5 percent reduction it made
                     to pilot programs’ average costs to adjust for economies of scale/program
                     efficiencies was reasonable and very conservative and that the program
                     can be implemented within the projected 13 percent increase over current
                     program costs. DOD also reported that one of the military services
                     validated the 13 percent cost increase following our audit. Further, DOD
                     stated that it did not see value added in providing the military services or
                     Congress a formal risk assessment but will continue to work with the
                     military services as execution progresses to make sure they have all
                     information required for budget purposes. Additionally, while not part of
                     this recommendation, DOD also said it did not concur with our finding



                     Page 23                                     GAO-03-367 Defense Transportation
that the cost estimate for implementing its information technology
improvements recommendation would be $7 million.

In reviewing the response, we found that DOD still did not provide any
data to support its assumption of a 5 percent cost savings from economies
of scale/program efficiencies. DOD stated that the new program will be
about 200 times larger than the pilot programs and that the resulting
increase in volume will lower the cost per unit, a standard and accepted
law of economics. While we agree that the cost may decrease, it may also
increase or remain unchanged. Moreover, the cost may decrease by less
than 5 percent. Without specific data showing the per move costs will
decrease as the scale of operations increase, we continue to question the
basis for DOD’s assumption of a 5 percent reduction.

We believe that the validation effort completed by one of the military
services, along with the calculations and assumptions DOD used in
developing the 13 percent cost estimate, does not provide the military
services and Congress with information needed to reliably develop and
review budget requests to fund enhancements to the current program. We
continue to believe that DOD needs to qualify this estimate with a measure
of the risk associated with implementing its recommendations. Without
providing the range of possible cost increases and the risk regarding the
likelihood of achieving this 13 percent projection within that range, DOD
may encounter a repetition of its experience with one of the pilot
programs, which had to be terminated because actual costs exceeded
projected costs. Absent this risk information, the military services will
have to wait until after the transportation providers submit their bids in
order to learn whether the recommendations can be implemented within
the 13 percent projection. Should the bids result in costs that exceed this
estimate, DOD and the military services will need to make adjustments to
ensure that the recommendations are implemented within funding limits.
Therefore, we continue to believe that our recommendation has merit.

Our finding that the implementation of the information technology
improvements recommendation would likely cost $7 million rather than
the $4 million to $6 million that DOD projected was based on information
we received from DOD during the audit. Specifically, we calculated that
the costs to develop and implement the new system would be about
$5 million and that training for users of the enhanced system would cost
an additional $1 million. DOD had identified the need for this training
during its evaluation of the pilot programs. After DOD submitted its report
to Congress, it identified another potential cost—an additional $1 million
for independent verification and validation testing of the system. Our
$7 million estimated included all three of these cost elements. In its



Page 24                                     GAO-03-367 Defense Transportation
response to a draft of this report, DOD maintained that its costs estimate
would fall within its initial cost estimate of $4 million to $6 million,
including $5 million for system development and implementation and an
additional $500,000 each for user training and system verification and
validation testing. At a minimum, based on these projected cost estimates,
the initial cost to implement the information technology improvements
recommendation would more likely be $6 million. However, since we did
not originally assess the sufficiency of the $1 million estimates for training
and testing, we are unable to assess what impact DOD’s reduction for
these costs to $500,000 would have on the implementation of the system
across DOD. We have reflected DOD’s changes in the body of our report.

DOD’s comments are reprinted in appendix III. DOD also provided
technical comments, and we revised our report to reflect them where
appropriate.

We performed our review from April 2002 through February 2003 in
accordance with generally accepted government auditing standards.
Appendix I contains the scope and methodology for this report. DOD’s
comments are reprinted in their entirety in appendix III.


We are sending copies of this report to the appropriate congressional
committees; the Secretary of Defense; the Commander, U.S.
Transportation Command; and the Director, Office of Management and
Budget. We will also make copies available to others upon request. In
addition, the report will be made available at no charge on the GAO Web
site at http://www.gao.gov.




Page 25                                       GAO-03-367 Defense Transportation
Please contact me at (202) 512-8365 or Lawson Gist, Jr., at (202) 512-4478
if you or your staff have any questions concerning this report. Key
contributors to this assignment were Robert L. Self, Jacqueline S. McColl,
Arthur L. James, Jr., Charles W. Perdue, and Nancy L. Benco.




William M. Solis
Director
Defense Capabilities and Management




Page 26                                     GAO-03-367 Defense Transportation
List of Congressional Committees

The Honorable John Warner
Chairman
The Honorable Carl Levin
Ranking Minority Member
Committee on Armed Services
United States Senate

The Honorable Ted Stevens
Chairman
The Honorable Daniel K. Inouye
Ranking Minority Member
Subcommittee on Defense
Committee on Appropriations
United States Senate

The Honorable Duncan Hunter
Chairman
The Honorable Ike Skelton
Ranking Minority Member
Committee on Armed Services
House of Representatives

The Honorable Joel Hefley
Chairman
The Honorable Solomon P. Ortiz
Ranking Minority Member
Subcommittee on Readiness
Committee on Armed Services
House of Representatives

The Honorable Jerry Lewis
Chairman
The Honorable John P. Murtha
Ranking Minority Member
Subcommittee on Defense
Committee on Appropriations
House of Representatives




Page 27                            GAO-03-367 Defense Transportation
             Appendix I: Scope and Methodology
Appendix I: Scope and Methodology


             To assess the extent to which the recommendations in the Department of
             Defense’s (DOD) November 2002 report to Congress addressed major
             problems in the personal property program, we took the following steps:

             •     To identify the major problems facing the current personal property
                   program, we reviewed DOD and GAO reports addressing this program.
                   These reports identified problems associated with quality of service
                   and claims. We also conducted interviews with personal property
                   program officials and their contractors to gain an understanding of the
                   current data management system’s limitations and the long-standing
                   problems involving the lack of reliable information on shipments and
                   their costs.

             •     To determine whether the proposed recommendations in DOD’s report
                   to Congress addressed the major problems of the current program, we
                   tracked the recommendations back to the U.S. Transportation
                   Command’s report on its evaluation results and assessed the extent to
                   which the recommendations are linked to and have the potential to
                   address problems.

             To assess whether the recommendations in DOD’s report to Congress
             were supported by DOD’s evaluation findings and should be implemented,
             we took the following steps:

             •     To determine if the Transportation Command developed a
                   methodologically sound evaluation plan, we assessed the command’s
                   efforts against the findings and recommendations contained in our
                   report1 on the Army’s Hunter Pilot Program results and against
                   professional standards2 we would use if conducting a comparable
                   evaluation. These sources addressed issues such as (1) seeking advice
                   in designing a methodologically sound evaluation plan, (2) developing
                   the evaluation plan prior to testing, (3) identifying factors to be
                   assessed and the data required for analyses to develop findings and
                   recommendations, (4) limiting quality of life surveys to only one for
                   each participant to preclude survey “fatigue,” and (5) conducting
                   simultaneous testing of the pilot and current programs.




             1
                 GAO/NSIAD-99-129.
             2
              U.S. General Accounting Office, Developing and Using Questionnaires, GAO/PEMD-
             10.1.7 (Washington, D.C.: Oct. 1993) and U.S. General Accounting Office, Using Structured
             Interviewing Techniques, GAO/PEMD-10.1.5 (Washington, D.C.: June 1991).




             Page 28                                             GAO-03-367 Defense Transportation
Appendix I: Scope and Methodology




•   To determine if the Transportation Command implemented an effective
    evaluation strategy during the data collection phase of its evaluation,
    we reviewed the pilot programs’ efforts to collect data for the four
    factors as prescribed in the Transportation Command’s evaluation
    plan. We also assessed the adjustments the Transportation Command
    made in its evaluation strategy to address issues that could affect the
    soundness of the results. An example of the issues addressed included
    developing a constructed cost methodology to provide better estimates
    of what DOD would have paid under the current program for
    shipments made by the pilot programs.

•   To assess the Transportation Command’s development of findings and
    recommendations to improve the current personal property program,
    we reviewed the evaluation techniques (quantitative and qualitative
    analyses) used to analyze data collected for the four factors. Further,
    we assessed the extent to which the Transportation Command adjusted
    the evaluation techniques to make up for differences in the way that
    the pilot programs provided data for the evaluation.

To assess the methodology that DOD used to develop cost estimates for
implementing the recommendations, we took the following steps:

•   To determine the reliability of the cost estimates for the pilot programs
    and for the proposed recommendations, we reviewed the cost
    projection methodologies used by the Transportation Command and by
    the Military Traffic Management Command.

•   To determine the reliability of pilot program shipment-related costs
    used in the report, we reviewed the data collection efforts used by each
    pilot program for the transportation and storage of household goods
    included in the Transportation Command’s evaluation. Further, we
    reviewed the constructed cost methodology used to develop the
    estimates of what DOD would have paid to make comparable
    shipments under the current program in the pilot programs’ test areas.

•   To determine the reasonableness of the assumptions and sources of
    data used to develop cost estimates for implementing
    recommendations for the personal property program, we met with
    officials from the Military Traffic Management Command and their
    contractor to discuss the methodology. We also reviewed the contents
    of their briefing on the cost estimate work for implementing changes to
    the claims process and performance-based service contracts and




Page 29                                      GAO-03-367 Defense Transportation
Appendix I: Scope and Methodology




    additional information the Military Traffic Management Command
    provided on the costs to implement information technology
    improvements.

We did not make an assessment of whether the anticipated benefits to be
derived from implementing the three recommendations would warrant the
additional costs DOD projects will be required to fund these
improvements. Furthermore, we did not independently test the reliability
of data DOD extracted from its data system to develop costs. We found
that the department placed proper caveats on their use of such data, and
in the case of comparing pilot programs’ shipment costs to current
program costs, developed a constructed cost methodology to address
current program data management system weaknesses.

During this and prior3 reviews of DOD’s evaluation efforts, we met with
officials and obtained documents from the Office of the Assistant Deputy
Under Secretary of Defense (Transportation Policy), Washington, D.C.; the
U.S. Transportation Command, Scott Air Force Base, Illinois; the Military
Traffic Management Command, Alexandria, Virginia; the Department of
Defense Inspector General, Full Service Moving Project, and Hay Group
(Transportation Command Contractor), Arlington, Virginia; American
Management Systems (Transportation Command contractor),
PricewaterhouseCoopers (Military Traffic Management Command
contractor), and Systems Research and Applications (Military Traffic
Management Command contractor), Fairfax, Virginia; Logistics
Management Institute (Military Traffic Management Command
contractor), McLean, Virginia; the Navy’s Service Member Arranged Move
Pilot Program, Mechanicsburg, Pennsylvania; The Gallup Organization
(Full Service Moving Project contractor), Omaha and Lincoln, Nebraska;
and Parsifal Corporation (Military Management Traffic Command
contractor), Palm Bay, Florida. In addition to these agency meetings and
documents, we drew upon information contained in a testimony




3
  Our prior reviews focused on the department’s efforts (1) to develop a methodologically
sound evaluation plan and (2) to collect data according to the plan for future analysis and
development of recommendations for an improved departmentwide program.




Page 30                                               GAO-03-367 Defense Transportation
Appendix I: Scope and Methodology




statement, in reports, and in status briefings resulting from our prior
reviews of this program.

Our work for this review was performed from April 2002 through February
2003 in accordance with generally accepted government auditing
standards.




Page 31                                      GAO-03-367 Defense Transportation
                                             Appendix II: Overview of Current Personal
Appendix II: Overview of Current Personal    Property Program and Pilot Programs



Property Program and Pilot Programs

                                             The Transportation Command evaluated three pilot programs to assess
                                             alternative approaches that might address long-standing problems with its
                                             current personal property program. The following tables provide features
                                             of the current program and the three pilot programs. As the tables show,
                                             the pilot programs had several features that provided enhancements to
                                             military personnel and their families and to DOD that are not offered by
                                             the current program.

                                             Table 1 compares claims-related features. Specifically, the pilot programs
                                             provided full replacement value rather than depreciated value for loss and
                                             damage and guaranteed claims settlement with 45 to 60 days of filing the
                                             claims.

Table 1: Claims-Related Features of the Current Personal Property Program and Pilot Programs

                                                              Military Traffic
                                                              Management                 Department of
                                                              Command’s                  Defense’s Full       Navy’s Service
                                       Current personal       Reengineered Personal      Service Moving       Member Arranged
 Program features                      property program       Property Program           Project              Move Pilot Program
 Loss and damage claims
 Reimbursements:
 —Basis for valuing property           Depreciated value      Full replacement value     Full replacement     Full replacement
                                                                                         value                value
 —Maximum dollar value per move        $40,000                $63,000                    $75,000              $72,000
 Guaranteed claims settlement          Not specified          Within 60 days             Within 45 days       Within 60 days
Sources: DOD (data); GAO (analysis).




                                             Page 32                                            GAO-03-367 Defense Transportation
                                          Appendix II: Overview of Current Personal
                                          Property Program and Pilot Programs




                                          Table 2 compares the quality of service-related features. Some of the
                                          comparable features included emphasizing performance over cost in
                                          selecting transportation providers and prescreening of transportation
                                          providers.




Table 2: Quality of Service-Related Features of the Current Personal Property Program and Pilot Programs

                                                     Military Traffic
                                                     Management
                                                     Command’s                  Department of              Navy’s Service Member
                          Current personal           Reengineered Personal      Defense’s Full Service     Arranged Move Pilot
 Program features         property program           Property Program           Moving Project             Program
 Counseling and arranging shipment services
 Single relocation        No                         No                         Yes                        Yes
 coordinator
 Program management Installation personal            Installation personal      Private-sector move        Installation personal
 and counseling           property shipping office   property shipping office   managers                   property shipping office
 services provider
 Screening process for transportation providers
 Program emphasis in      Lowest cost                Performance                Performance                Performance
 selecting transportation
 providers
 Prescreening of          Very limited               Yes                        Yes                        Yes
 transportation providers
 Customer satisfaction    Surveys are not            Survey company calls       Survey company calls       Service member returns
 surveys conducted and conducted                     member                     member                     survey via mail
 method
 Visibility of shipments during relocation process
 Methods used to          None                       Toll-free number to        Toll-free number to move   Toll-free number to move
 increase service                                    transportation provider    manager and                coordinator and
 members’ visibility of                                                         transportation provider    transportation provider,
 shipments during move                                                                                     and member has pager
 process
Sources: DOD (data); GAO (analysis).




                                          Page 33                                              GAO-03-367 Defense Transportation
                                          Appendix II: Overview of Current Personal
                                          Property Program and Pilot Programs




                                          Table 3 compares data reliability-related features. As noted, only one of
                                          the pilot programs had a data management system that provided reliable
                                          information to track individual shipments in transit and provide overall
                                          data on shipments and their associated costs.




Table 3: Data Reliability-Related Features of the Current Personal Property Program and Pilot Programs

                                                       Military Traffic
                                                       Management
                                                       Command’s               Department of              Navy’s Service Member
                             Current personal          Reengineered Personal   Defense’s Full Service     Arranged Move Pilot
 Program features            property program          Property Program        Moving Project             Program
 Availability and reliability of data on household goods shipments
 Reliable data               No (current system is not Yes                     Not determined             Not determined
 management system to        designed to track                                 (developed but not fully   (developed a database
 track individual            shipments nor provide                             operational–data           by end of pilot program
 shipments in transit and reliable shipment and                                management system          but was not fully
 to provide overall data on cost data)                                         needed additional          implemented or
 shipments and                                                                 refinement)                evaluated)
 associated costs
Sources: DOD (data); GAO (analysis).



                                          Additional information on the current program and on each pilot program
                                          and its unique features follows.


                                          The current DOD personal property program, valued at over $1.7 billion
Current Program                           annually, moves more than 600,000 shipments each year for military
                                          personnel and their families from the military services, Defense agencies,
                                          and the Coast Guard. DOD is the moving industry’s single largest
                                          customer. Managed centrally by the headquarters office of the Military
                                          Traffic Management Command and administered locally by about 200
                                          military and DOD transportation offices around the world, this program
                                          relies on over 1,200 domestic commercial carriers and more than 150
                                          forwarders for international traffic to provide moving and storage services.


Loss and Damage Claims                    When loss and damage occur, military personnel can submit claims to
                                          their respective military service claims office. Based on depreciated
                                          values, the reimbursement rate is $1.25 per pound multiplied by the
                                          shipment weight, with a maximum amount of $40,000 per move. Military
                                          personnel have up to 2 years after receiving their shipments to file claims
                                          but must submit notice of loss and damage within 70 days of delivery. The



                                          Page 34                                             GAO-03-367 Defense Transportation
                           Appendix II: Overview of Current Personal
                           Property Program and Pilot Programs




                           current program does not have a specified time period in which the claims
                           are to be settled.


Counseling and Arranging   The current program provides counseling services and arranges the
Shipment Services          shipment and storage of household goods and unaccompanied baggage
                           through government representatives, who are available to assist military
                           personnel and their families at the origin and destination points of their
                           moves. The current program does not have a real-time tracking system for
                           shipments nor does it provide a single point of contact to manage the
                           entire moving process; therefore, military personnel may interact with
                           several people at the origin and destination offices during their relocation.


Screening and Shipment     The current system is not designed to select transportation providers on
Distribution Process for   the basis of quality service; rather, transportation providers offering a
                           minimally acceptable level of quality are generally selected based on the
Transportation Providers   lowest rates. The program uses the Total Quality Assurance Program to
                           develop quality scores for each transportation provider. Each local
                           military installation distributes its traffic using a traffic distribution roster.
                           Transportation providers are placed on the rosters for each channel
                           (origin and destination areas) by order of rate level and quality score.
                           Transportation providers who participate in the domestic part of the
                           current program submit their rates as a percentage of the government
                           tariff, which is nearly 20 years old. The providers who participate in the
                           international part of the current program submit single factor or fixed
                           rates per hundredweight of the shipments.

                           The current program does not use customer satisfaction surveys as a
                           means to evaluate transportation provider performance. To remain in the
                           program, a provider must maintain a minimally acceptable level of quality–
                           a 90 percent score. Three factors are measured: on-time pickup, on-time
                           delivery, and reported loss and damage to determine if points should be
                           deducted from transportation providers and allocation of shipments
                           should be reduced or terminated.


Visibility of Shipments    The current program does not provide service members with real-time
during the Relocation      visibility of shipments during the relocation process.
Process




                           Page 35                                         GAO-03-367 Defense Transportation
                               Appendix II: Overview of Current Personal
                               Property Program and Pilot Programs




Availability and Reliability   Personnel at origin and destination personal property shipping offices
of Data on Household           enter information on shipments to their respective Transportation
                               Operational Personal Property Standard Systems. However, data in these
Goods Shipments                individual systems does not include all shipments that occur during the
                               year, and the systems are not accessible to all parties involved in the
                               relocation process. Destination personal property shipment offices are
                               forwarded information on shipments via the current system; however,
                               payment data on these shipments is maintained in a separate system.

                               In addition to not providing information on all aspects of individual
                               shipments, the current program’s data management system does not
                               provide DOD and the military services information about the types of
                               shipments and related costs for planning and budgeting purposes. The
                               following are examples of the current system’s limitations:

                               •   the format of the system makes compiling data from multiple sites
                                   difficult;
                               •   not all data is captured promptly; and
                               •   not all data and costs are captured/updated in the system.


Funding Sources                Under the current program, the military services reimburse carriers and
                               forwarders for shipment-related costs from military personnel accounts.
                               Personal property shipment office expenses and claims filed with the
                               government are funded from the military services’ operations and
                               maintenance accounts.


                               Sponsored by the Military Traffic Management Command, the
Military Traffic               Reengineered Personal Property Program included outbound shipments
Management                     for military and Coast Guard personnel departing from installations
                               located in North Carolina, South Carolina, and Florida (excluding Tyndall
Command’s                      Air Force Base). The pilot program ran concurrently with the existing
Reengineered                   program at these installations. The pilot program’s goal was to include
                               50 percent of eligible moves from the above installations to continental
Personal Property              United States and European locations. The remaining shipments were to
Program                        be moved under the existing program. The Reengineered Personal
                               Property Program was initiated in January 1999 and operated for 12
                               months before data was submitted to the Transportation Command for
                               evaluation.


Loss and Damage Claims         Reimbursement for loss and damage claims was increased from
                               depreciated value to full replacement value, and the dollar amounts per



                               Page 36                                     GAO-03-367 Defense Transportation
                           Appendix II: Overview of Current Personal
                           Property Program and Pilot Programs




                           move increased from $40,000 under the current program to $63,000 under
                           the Reengineered Personal Property Program. Additionally, the pilot
                           program provided direct claims settlement between military personnel and
                           their transportation providers and a requirement that transportation
                           providers settle claims within 60 days of receiving claims forms from
                           military personnel.


Counseling and Arranging   Like the current program, the Reengineered Personal Property Program
Shipment Services          relied on personnel in the personal property shipping offices to provide
                           counseling services and arrange for shipment and storage of household
                           goods and unaccompanied baggage. A central contact point in these
                           offices was not designated to manage the entire moving process; therefore,
                           military personnel may have interacted with several people at the origin
                           and destination offices during the relocation process. However, to improve
                           customer service, the program’s Pilot Transportation Operational Personal
                           Property Standard System provided real-time worldwide tracing
                           capability.


Screening and Shipment     Greater emphasis was placed on performance in awarding shipments to
Distribution Process for   transportation providers. Evaluation of financial status, elimination of
                           high-risk companies, and consideration of providers’ past performance,
Transportation Providers   rather than lowest bid, played the dominant role in selecting initial
                           transportation providers to participate in this pilot program.
                           Transportation providers who participated in the pilot program submitted
                           their bids for various origin and destination routes as a discount from the
                           commercial tariff. Prices were fixed for a year, with no provision for rate
                           increases during the contract period. Awards were made only to
                           transportation providers whose offers conformed to the solicitation and
                           represented the best overall value to the government.

                           The Military Traffic Management Command evaluated company
                           performance quarterly and compliance with terms and conditions of the
                           contracts annually. Subsequent performance reviews were conducted
                           based on customer satisfaction surveys and claims data. After
                           transportation providers received their minimum guarantee of business for
                           the year ($25,000), future awards were offered to the best performers.
                           Feedback was provided monthly to transportation providers, and those
                           that became poor performers were no longer offered household goods and
                           unaccompanied baggage shipments.




                           Page 37                                     GAO-03-367 Defense Transportation
                               Appendix II: Overview of Current Personal
                               Property Program and Pilot Programs




Visibility of Shipments        The Reengineered Personal Property Program provided the transportation
during the Relocation          provider’s toll-free number to military personnel to enhance visibility over
                               their shipments throughout the relocation process.
Process

Availability and Reliability   The Reengineered Personal Property Program implemented its central,
of Data on Household           Web-based Pilot Transportation Operational Personal Property Standard
                               System in part to address problems associated with visibility of and
Goods Shipments                availability of information on shipments during the relocation process. The
                               pilot program’s data management system provided real-time access to
                               both shipment and payment records. Access to the various modules of the
                               system was granted to personal property shipment office personal at
                               origin and destination locations, transportation providers, invoice
                               certifying officers, prepayment auditors, military service headquarters, and
                               military service claims offices and finance centers, based on each party’s
                               need for the information.

                               The system’s design allowed for entry of current address and telephone
                               numbers of military personnel to improve the process of delivering
                               household goods to a new residence. Data reliability was enhanced under
                               the Reengineered Personal Property Program, but one problem noted
                               during the evaluation was the need for military personnel to ensure that
                               their contact information (phone number and address) was current during
                               the relocation process. This had an effect on deliveries of household goods
                               and the quality of life survey contractor’s ability to reach military
                               personnel to ascertain their opinions about their relocation experience.

                               In addition to providing information on all aspects of individual shipments,
                               the Reengineered Personal Property Program’s data management system
                               demonstrated the potential to provide DOD and the military services with
                               information about the types of shipments and related costs managed
                               annually for planning and budgeting purposes.


Other Reengineered             The Reengineered Personal Property Program achieved stronger
Personal Property              transportation provider commitment with long-term contracts, and it used
                               contractor support to conduct quality of life surveys with military
Program Features               personnel moving under the pilot program and to perform audits of each
                               invoice submitted by transportation providers.


Funding Sources                Like the current program, the military services reimbursed carriers and
                               forwarders for shipment-related costs from their military personnel
                               accounts. Transportation office expenses and any claims filed with the



                               Page 38                                      GAO-03-367 Defense Transportation
                           Appendix II: Overview of Current Personal
                           Property Program and Pilot Programs




                           government were funded from the services’ operations and maintenance
                           accounts.


                           Sponsored by the Office of the Assistant Deputy Under Secretary of
The Department of          Defense (Transportation Policy), the Full Service Moving Project included
Defense’s Full Service     outbound shipments for military and Coast Guard personnel and DOD
                           civilian departing from locations in the National Capital Region, Georgia
Moving Project             (excluding Robins Air Force Base), and Minot Air Force Base, North
                           Dakota. The pilot program’s goal was to include 90 percent of the moves
                           from these locations to continental United States and to European and
                           Asian-Pacific locations. The remaining shipments were to be moved under
                           the current program. The Full Service Moving Project began in January
                           2001 and continued until its early termination in September 2001. Due to
                           continuing delays in implementing this pilot program and DOD’s decision
                           to terminate the pilot program in September 2001, the Full Service Moving
                           Project had limited operational experience before submitting data to the
                           Transportation Command.


Loss and Damage Claims     Reimbursement for loss and damage claims was increased from
                           depreciated value to full replacement value, with the dollar amounts
                           increasing from $40,000 per move under the current program to $75,000
                           per move under the Full Service Moving Project. Additionally, the pilot
                           program provided for direct claims settlement between military personnel
                           and their transportation providers and a requirement that the responsible
                           party (transportation providers or move managers) settle claims within 45
                           days of receiving claim forms from military personnel.


Counseling and Arranging   Unlike the current program and other pilot programs, the Full Service
Shipment Services          Moving Project tested the use of private-sector relocation companies
                           (move managers) for outsourcing traditional transportation services
                           (counseling and arranging for the shipment and storage of household
                           goods and unaccompanied baggage) performed by origin and destination
                           personal property shipping offices. The pilot program’s goal was to
                           provide a single point of contact (move manager) for military personnel
                           and transportation providers to contact throughout the relocation process.


Screening and Shipment     The Full Service Moving Project made major changes to the existing
Distribution Process for   transportation provider approval, rate solicitation, and traffic distribution
                           processes. The pilot program emphasized best value and placed more
Transportation Providers   weight on performance (70 percent) than cost (30 percent) in determining



                           Page 39                                      GAO-03-367 Defense Transportation
                          Appendix II: Overview of Current Personal
                          Property Program and Pilot Programs




                          which transportation providers would be awarded shipments. The pilot
                          program contracted with a financial services company to conduct financial
                          and performance assessments of transportation providers and move
                          manager companies that wanted to participate in the pilot program. For
                          approved transportation providers, rates were established for a 1-year
                          cycle. The providers submitted their rates as a discount from the
                          commercial tariff for domestic shipments and negotiated single rate
                          factors for the overseas locations. Approved move managers were
                          awarded 2-year contracts with 1-year options. The move management
                          companies competitively bid their fees as flat rates, depending on whether
                          they were responsible for claims settlement or the transportation provider
                          carried this liability. Also, different fees were established for domestic and
                          international shipments.

                          The Full Service Moving Project used survey data from all personnel
                          participating in the pilot program to determine future percentages of
                          shipments that would be allocated to the transportation providers. The
                          pilot program also planned to use survey data on move manager
                          performance to determine future participation in the pilot program and
                          incentive payments. The Full Service Moving Project’s Web-based Best
                          Value Distribution Database maintained the transportation providers’
                          quality and cost scores based on survey information and costs associated
                          with prior shipments. Move managers used this data to assign future
                          shipments. However, in some instances (i.e., for group moves, when
                          meeting small business requirements, when there was a lack of
                          transportation provider capacity to handle shipments offered, for multiple
                          shipments to a single transportation provider, and for international
                          shipments to areas without an established rate), move managers were told
                          to deviate from the information provided by the data management system.


Visibility of Shipments   One of the goals of incorporating move managers into the relocation
during the Relocation     process was to provide real-time information to military personnel and to
                          transportation providers regarding the status of household goods
Process                   shipments. The move managers, unlike the current program’s personal
                          property shipping office personnel, were responsible for the entire
                          relocation process from the point of origin in establishing entitlements,
                          arranging for transportation providers, and handling other personnel-
                          related issues, to the destination in overseeing deliveries, approving
                          storage, and either settling claims or assisting military personnel with
                          issues involving settling claims with the transportation providers if the
                          liability fell with the providers. Working with both military personnel and
                          transportation providers, the move managers used contact information to
                          keep military personnel informed of their shipments’ status and to



                          Page 40                                      GAO-03-367 Defense Transportation
                               Appendix II: Overview of Current Personal
                               Property Program and Pilot Programs




                               coordinate the delivery of the shipments at the destination. Additionally,
                               as part of the pilot program, all participants were provided a toll-free
                               number to maintain visibility over their shipments throughout the process.


Availability and Reliability   In addition to move managers, the Full Service Moving Project’s Web-
of Data on Household           based Best Value Distribution Database was implemented to address
                               problems associated with visibility of shipments during the relocation
Goods Shipments                process. The pilot program’s data management system had access to both
                               shipment and payment records via interface with US Bank’s PowerTrack
                               and the move managers’ systems. Access to the pilot program’s data
                               management system was granted to move managers, invoice certifying
                               officers, military service headquarters, and military service claims offices
                               and finance centers, based on each party’s requirements.

                               Move managers were responsible for keeping the status of the shipments
                               current in the pilot program’s data management system. However, the
                               move managers did not always update this information in the system.
                               Further, the ability of the move manager to contact the service member
                               was directly affected by the information provided by the member.

                               The Full Service Moving Project’s Web-based Best Value Distribution
                               Database was anticipated to provide DOD and the military services
                               information on the types of shipments and related costs managed annually
                               for planning and budgeting purposes. Unlike the Reengineered Personal
                               Property Program where various parties in the relocation process entered
                               data into that pilot program’s data management system, the majority of the
                               data in the Full Service Moving Project’s data management system was
                               predicated on the move managers gathering and entering the information.


Other Full Service Moving      The Full Service Moving Project achieved stronger transportation provider
Project Features               commitment with long-term contracts and faster payment of invoices; it
                               offered binding cost estimates for shipments; it used contractor support to
                               conduct quality of life surveys with military personnel moving under the
                               pilot program and to perform audits of each invoice submitted by the
                               transportation providers; and it offered optional relocation referral
                               assistance for activities such as the purchase and sale of service members’
                               residences.

                               Move managers were required to perform prepayment audits and business
                               rules were established for an automatic payment method. Payment
                               methodology was predicated on the move manager entering the expected
                               invoice into PowerTrack and the transportation provider submitting a



                               Page 41                                      GAO-03-367 Defense Transportation
                         Appendix II: Overview of Current Personal
                         Property Program and Pilot Programs




                         notice of delivery and invoice. Payment timeliness was also driven by the
                         timeliness of documentation submitted by the transportation providers.
                         On some invoices, the contracting representative had to review and certify
                         payment in PowerTrack. This occurred when the match showed a
                         difference of more than $1.00.


Funding Sources          For this pilot program, the military services reimbursed carriers and
                         forwarders for shipment-related costs from military personnel accounts.
                         These accounts were also used to fund move manager expenses. Any
                         claims that might have been filed with the government would have been
                         funded from the military services’ operations and maintenance accounts.


                         Sponsored by the Navy, the Service Member Arranged Move Pilot Program
Navy’s Service           included only domestic outbound intrastate and interstate shipments for
Member Arranged          Navy personnel moving from its installations located at Puget Sound,
                         Washington; San Diego, California; Norfolk, Virginia; New London,
Move Pilot Program       Connecticut; and Whidbey Island, Washington. One of this program’s
                         objectives was to offer Navy military personnel a set of moving choices to
                         meet their specific needs. This pilot program was one of three choices
                         offered. Military personnel moving from the above locations could choose
                         to move under the current personal property program, move their own
                         household goods, or participate in the pilot program. The pilot program
                         was initiated in April 1997 and began operations in January 1998. Because
                         the Navy decided not to scope the Service Member Arranged Move Pilot
                         Program comparable to other pilot programs (i.e., operational at multiple
                         military services) and the pilot program did not provide data as outlined
                         by the Transportation Command’s evaluation plan, its inclusion in the
                         Transportation Command’s evaluation was limited to a qualitative
                         assessment.


Loss and Damage Claims   Reimbursement for loss and damage claims was increased from
                         depreciated value to full replacement value, with the dollar amounts per
                         move increasing from $40,000 under the current program to $72,000 under
                         the Service Member Arranged Move Pilot Program. Additionally, the pilot
                         program provided direct claims settlement between military personnel and
                         their transportation providers and a requirement that transportation
                         providers settle claims within 60 days of receiving claims forms from
                         military personnel.




                         Page 42                                     GAO-03-367 Defense Transportation
                           Appendix II: Overview of Current Personal
                           Property Program and Pilot Programs




Counseling and Arranging   Like the current program, the Service Member Arranged Move Pilot
Shipment Services          Program also relied on personnel in the personal property shipping offices
                           to provide counseling services and arrange shipment and storage of
                           household goods and unaccompanied baggage. However, the shipping
                           office personnel at the origin installations participating in this pilot
                           program served as the single point of contact coordinating the service
                           members’ moves and remained available throughout the move to handle
                           all issues, including claims. Unlike those participating in the current
                           program and other pilot programs, service members participating in this
                           pilot program identified the transportation provider they desired to handle
                           their household goods shipments after they completed their reviews of
                           participating providers’ vendor quality books (containing provider
                           information and marketing materials) and of surveys completed by
                           previous pilot program participants. The personal property office
                           coordinator assigned to the service member had to concur with the
                           member’s request, and the coordinator made actual arrangements with the
                           carrier.


Screening and Shipment     Staff in the program management office and personal property shipping
Distribution Process for   offices participating in the Service Member Arranged Move Pilot Program
                           initially screened transportation providers that wished to participate in the
Transportation Providers   pilot program based on providers’ performance rather than low cost.
                           Letters of agreement were adopted to streamline the contracting process
                           and improve the quality of the move for Navy personnel. According to pilot
                           program officials, these letters of agreement provided commercial best
                           practices and enabled lessons learned from prior pilot program efforts and
                           industry to be incorporated into the Navy pilot program. Actual contract
                           awards were made on a case-by-case basis based on the best value
                           decision for each move. Transportation providers used commercial tariffs
                           in developing their bids for each move.

                           Transportation providers approved to participate in the pilot program
                           submitted their bids for various origin and destination channels using
                           commercial tariffs. Bids were rejected if they did not fall within acceptable
                           percentage discounts. Feedback was provided monthly to transportation
                           providers, and those that became poor performers were no longer offered
                           household goods and unaccompanied baggage shipments. Subsequently,
                           service members who were planning their upcoming moves relied on
                           information contained in a nine-question survey that other service
                           members had completed after their moves and claims process ended.
                           Service members who volunteered to participate in this pilot program had
                           to manually review carrier books, which included documents provided by
                           the carriers and prior surveys completed by service members who had



                           Page 43                                      GAO-03-367 Defense Transportation
                               Appendix II: Overview of Current Personal
                               Property Program and Pilot Programs




                               been moved by the carriers. According to pilot program officials, six
                               carriers were terminated or canceled from the pilot program–four for
                               providing poor service and two for price gouging.


Visibility of Shipments        The Service Member Arranged Move Pilot Program relied upon the
during Relocation Process      shipping office personnel, who served as the single point of contact
                               coordinating the service members’ moves, to maintain visibility of
                               shipments during the relocation process. In addition, the pilot program
                               provided both the personal property shipping office’s and transportation
                               provider’s toll-free numbers, as well as a pager to service members to
                               enhance the members’ visibility of their shipments during the relocation
                               process.


Availability and Reliability   The Service Member Arranged Move Pilot Program did not initially
of Data on Household           develop an alternative data management system to capture data on
                               shipments and payment records. By the end of the pilot program, the Navy
Goods Shipments                had developed a database to capture shipment data; however, the system
                               was not fully implemented or evaluated.

                               Unlike the other pilot programs, the Navy pilot program used local
                               personal property program personnel rather than third parties to review
                               all invoices for payment. Navy personnel who participated in the pilot
                               program completed their own surveys, mailing the paper forms to their
                               respective personal property program offices.


Other Service Member           The Service Member Arranged Move Pilot Program was designed to offer
Arranged Move Pilot            all shipments to small businesses, to provide direct claims settlements
                               between Navy personnel and the transportation providers, to make faster
Program Features               payments to transportation providers through government purchase cards,
                               and to establish a stronger commitment from transportation providers by
                               offering long-term contracts.


Funding Sources                Like the current program, the Navy reimbursed carriers and forwarders for
                               shipment-related costs from its military personnel account. Personal
                               property shipment office expenses were funded from the Navy’s
                               operations and maintenance account. While information on claims filed
                               with the government was not provided, under this pilot program such
                               expenses would also be funded from the operations and maintenance
                               account.




                               Page 44                                     GAO-03-367 Defense Transportation
                   Appendix III: Comments from the Department
Appendix III: Comments from the
                   of Defense



Department of Defense




         Page 45                                       GAO-03-367 Defense Transportation
          Appendix III: Comments from the Department
          of Defense




Page 46                                       GAO-03-367 Defense Transportation
          Appendix III: Comments from the Department
          of Defense




Page 47                                       GAO-03-367 Defense Transportation
          Appendix III: Comments from the Department
          of Defense




Page 48                                       GAO-03-367 Defense Transportation
           Related GAO Products
Related GAO Products


           Defense Transportation: Final Evaluation Plan Is Needed to Assess Alternatives to
           the Current Personal Property Program. GAO/NSIAD-00-217R. Washington, D.C.:
           September 27, 2000.

           Defense Transportation: The Army’s Hunter Pilot Project Is Inconclusive but
           Provides Lessons Learned. GAO/NSIAD-99-129. Washington, D.C.: June 23, 1999.

           Defense Transportation: Plan Needed for Evaluating the Navy Personal Property
           Pilot. GAO/NSIAD-99-138. Washington, D.C.: June 23, 1999.

           Defense Transportation: Efforts to Improve DOD’s Personal Property Program.
           GAO/T-NSIAD-99-106. Washington, D.C.: March 18, 1999.

           Defense Transportation: The Army’s Hunter Pilot Project to Outsource Relocation
           Services. GAO/NSIAD-98-149. Washington, D.C.: June 10, 1998.

           Defense Transportation: Reengineering the DOD Personal Property Program.
           GAO/NSIAD-97-49. Washington, D.C.: November 27, 1996.




(350161)
           Page 49                                   GAO-03-367 Defense Transportation
                         The General Accounting Office, the audit, evaluation and investigative arm of
GAO’s Mission            Congress, exists to support Congress in meeting its constitutional responsibilities
                         and to help improve the performance and accountability of the federal
                         government for the American people. GAO examines the use of public funds;
                         evaluates federal programs and policies; and provides analyses,
                         recommendations, and other assistance to help Congress make informed
                         oversight, policy, and funding decisions. GAO’s commitment to good government
                         is reflected in its core values of accountability, integrity, and reliability.


                         The fastest and easiest way to obtain copies of GAO documents at no cost is
Obtaining Copies of      through the Internet. GAO’s Web site (www.gao.gov) contains abstracts and full-
GAO Reports and          text files of current reports and testimony and an expanding archive of older
                         products. The Web site features a search engine to help you locate documents
Testimony                using key words and phrases. You can print these documents in their entirety,
                         including charts and other graphics.
                         Each day, GAO issues a list of newly released reports, testimony, and
                         correspondence. GAO posts this list, known as “Today’s Reports,” on its Web site
                         daily. The list contains links to the full-text document files. To have GAO e-mail
                         this list to you every afternoon, go to www.gao.gov and select “Subscribe to daily
                         E-mail alert for newly released products” under the GAO Reports heading.


Order by Mail or Phone   The first copy of each printed report is free. Additional copies are $2 each. A
                         check or money order should be made out to the Superintendent of Documents.
                         GAO also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a
                         single address are discounted 25 percent. Orders should be sent to:
                         U.S. General Accounting Office
                         441 G Street NW, Room LM
                         Washington, D.C. 20548
                         To order by Phone:     Voice:    (202) 512-6000
                                                TDD:      (202) 512-2537
                                                Fax:      (202) 512-6061


                         Contact:
To Report Fraud,
                         Web site: www.gao.gov/fraudnet/fraudnet.htm
Waste, and Abuse in      E-mail: fraudnet@gao.gov
Federal Programs         Automated answering system: (800) 424-5454 or (202) 512-7470


                         Jeff Nelligan, managing director, NelliganJ@gao.gov (202) 512-4800
Public Affairs           U.S. General Accounting Office, 441 G Street NW, Room 7149
                         Washington, D.C. 20548