oversight

Defense Transportation: Efforts to Improve DOD's Personal Property Program

Published by the Government Accountability Office on 1999-03-18.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                     United States General Accounting Office

GAO                  Testimony
                     Before the Subcommittee on Military Readiness,
                     Committee on Armed Services, House of Representatives




For Release
Expected at
11:00 a.m., EST
                     DEFENSE
Thursday,
March 18, 1999       TRANSPORTATION

                     Efforts to Improve DOD’s
                     Personal Property Program
                     Statement by David R. Warren, Director, Defense
                     Management Issues, National Security and International
                     Affairs Division




GAO/T-NSIAD-99-106
                   Mr. Chairman and Members of the Subcommittee:

                   We are pleased to be here today to discuss the Department of Defense’s
                   (DOD) efforts to improve its program for relocating military personnel and
                   their families. This has been a long, complicated process. Since 1994, DOD
                   has been engaged in initiatives to reengineer the personal property
                   program to simplify current processes, control program costs, ensure
                   quality of service by adopting commercial business processes
                   characteristic of world-class businesses, and relieve carriers of
                   DOD-unique terms and conditions. The U.S. Transportation Command
                   (USTRANSCOM) is tasked with evaluating the pilots to determine which
                   pilots, or portions thereof, could provide better long-term results for DOD.1
                   The statement of managers in the conference report on the DOD
                   Appropriations Act for 1997 directed us to validate the results and savings
                   achieved from any personal property program before DOD proposes
                   further expansion of such programs.2 My testimony today will focus on our
                   ongoing work related to the Army’s Hunter pilot and DOD’s plans to
                   evaluate other ongoing and planned pilot programs. In assessing the
                   Army’s evaluation of the Hunter Pilot, we focused on (1) the evaluation
                   methodology and (2) the Army Audit Agency’s (AAA) validation of the test
                   and baseline data.



Results in Brief   We agree that DOD needs to reengineer its personal property moving
                   process to improve the poor quality of service military members are
                   receiving. We support the use of pilots as a means to test new concepts.
                   Although this process has been ongoing since the early 1990s, DOD is not
                   yet in a position to determine what changes are needed.

                   We were unable to validate the reported results of the Army’s evaluation of
                   the Hunter pilot because of weaknesses in the evaluation methodology and
                   the data. However, the lessons learned from the pilot do provide
                   information that should be useful to DOD as it assesses and conducts its
                   pilot efforts. Further, the Hunter pilot provides military personnel with


                   1The  mission of USTRANSCOM, which is DOD’s single manager of all defense transportation services,
                   is to provide global air, land, and sea transportation to meet national security needs, both in time of
                   peace and time of war. USTRANSCOM executes its mission through three component commands:
                   Military Traffic Management Command (MTMC) for land transportation and port operations, the
                   Military Sealift Command for sea transport, and the Air Mobility Command for air transport.

                   2
                       House Report 104-863, Sept. 28, 1996.




                   Page 1                                                                          GAO/T-NSIAD-99-106
                        services that were not previously available during the moving process,
                        including personal move counseling and coordination, full replacement
                        value for lost or damaged household goods, and visibility of the shipment
                        throughout the move.

                        Key pilot results that we were unable to validate included the level of
                        customer satisfaction and costs. Although customer surveys were
                        conducted, the results were inconclusive because of the methods used.
                        While the Army indicated that the estimated pilot cost was higher than the
                        baseline cost, we could not validate the extent to which the pilot cost
                        exceeded the baseline cost. Further, we could not calculate the baseline
                        and pilot costs due to weaknesses in the Army’s methodology and data
                        reliability. As for the participation of small businesses, we confirmed that
                        33 percent of the pilot shipments were awarded by Cendant Mobility to
                        small business carriers and agents.

                        DOD has three pilot programs underway to improve its current personal
                        property program and is proposing a fourth pilot. The general plan is to
                        evaluate the results of these pilots and use that information to develop a
                        redesigned Department-wide relocation program. To achieve this
                        objective, it is imperative that DOD develop a well thought out strategy
                        with clear time lines for testing each of the approaches, and more
                        importantly, an evaluation methodology that will produce credible and
                        accurate information to be used in making a final decision. However, plans
                        for accomplishing these tasks and milestones for implementing a new
                        process have not yet been finalized.



Background              DOD has long been concerned about the quality service members receive
                        from its nearly $3 billion annual program to transport, store, and manage
                        the household goods and unaccompanied baggage of its personnel. Past
                        problems include poor service from its movers, excessive incidence of loss
                        or damage to service members’ property, and high claims costs to the
                        government. All of these problems contribute to a poor quality of service
                        for persons using the system.


MTMC’s Reengineering    DOD first proposed reengineering its personal property program 5 years
Efforts Began in 1994   ago. On June 21, 1994, the Deputy Commander in Chief, USTRANSCOM,
                        directed MTMC, the Army component of USTRANSCOM and program
                        manager for DOD’s Personal Property Shipment and Storage Program, to
                        reengineer the personal property program. On June 15, 1995, the House


                        Page 2                                                     GAO/T-NSIAD-99-106
                          Committee on National Security, concurring that DOD must pursue a
                          higher level of service, directed that DOD undertake a pilot program to
                          implement commercial business practices and standards of service.3
                          MTMC planned to award contracts for the new program pursuant to the
                          competitive acquisition system provisions (chapter 137 of title 10 of the
                          U.S. Code and the primary implementing regulation contained in the
                          Federal Acquisition Regulation). Expressing congressional concerns about
                          the impact that the competition system and any nonstandard commercial
                          business practice requirements might have on small businesses, the
                          statement of managers accompanying the 1997 DOD Authorization Act
                          directed that DOD report on the impact of the pilot program on small
                          business. After reviewing the defense reports on small business impact,
                          the House Committee on National Security was still concerned that
                          MTMC’s pilot program did not satisfactorily address issues raised by the
                          small moving companies and directed that the Secretary of Defense
                          establish a working group of military and industry representatives to
                          develop an alternative pilot program.


MTMC Pilot Commenced in   Although the working group reached a consensus on many issues,
1999                      including a set of program goals, it could not reach agreement on the
                          approach to take for the pilot test. Consequently, the two sides presented
                          separate proposals. In November 1996, we reported that in our assessment
                          MTMC’s proposal met the goals of reengineering the personal property
                          program to a greater extent than the industry plan.4

                          The MTMC pilot program implementation, delayed by numerous bid
                          protests, finally commenced in January 1999.5 The pilot reengineers the
                          existing program for 50 percent of the moves originating in three states:
                          North Carolina, South Carolina, and Florida. The pilot will run
                          concurrently with the existing MTMC-managed program at installations in
                          the three states and will involve approximately 18,500 annual moves. The


                          3
                              The House Committee on National Security is now called the House Armed Services Committee.

                          4See Defense Transportation: Reengineering the DOD Personal Property Program (GAO/NSIAD-97-49,
                          Nov. 27, 1996) for more detailed information on the MTMC pilot effort.

                          5
                            Solicitation disputes in the following decisions: Aalco Forwarding, Inc., et al. (B-277241.8, B-277241.9,
                          Oct. 21, 1997); Aalco Forwarding, Inc. et al. (B-277241.12, B-277241.13, Dec. 29,1997); Aalco Forwarding,
                          Inc. et al. (B-277241.15, Mar. 11, 1998); Aalco Forwarding, Inc. et al. (B-277241.16, Mar. 11, 1998); Aalco
                          Forwarding, Inc., et al. (B-277241, 277241.14, June 8, 1998); and Aalco Forwarding, Inc.,
                          et al. (B-277241, B-277241.20, B-277241.21, July 1, 1998).




                          Page 3                                                                            GAO/T-NSIAD-99-106
                               key features of the pilot program include selecting carriers based on
                               service member satisfaction and past performance rather than simply
                               price; achieving stronger carrier commitment with long-term contracts; and
                               offering full replacement value protection and direct claims settlement to
                               users. MTMC’s pilot will run for a 3-year test period (1 year with two 1-year
                               option periods), which will end in September 2002.


Hunter Pilot Effort Began in   Separately from the MTMC pilot program, the Army decided to determine
1996                           whether the commercial business practice of outsourcing could alleviate
                               known problem areas. The pilot was initiated in February 1996 as a
                               quality-of-life effort to improve the relocation process and to test
                               commercial business practices in a military environment at Hunter Army
                               Airfield, Savannah, Georgia. On January 31, 1997, the Army selected PHH
                               Relocation, a relocation/move management company (today named
                               Cendant Mobility, located in Bethesda, Maryland), as the contractor for the
                               pilot. In July 1997, Cendant Mobility began relocation operations at Hunter
                               Army Airfield, consisting of point-to-point move management, personal
                               move counseling and coordination, assistance in buying/selling a residence,
                               24-hour in-transit visibility of the shipment, direct claims settlement with
                               the service member, full replacement value, and a single point of contact
                               for the member, that were not previously available under the existing
                               system. The contract will end on September 30, 1999.


Navy Pilot Effort Began in     The Navy initiated a separate pilot effort in 1997 to test the option of
1997                           allowing service members to select their carrier, giving them more control
                               over the relocation process to meet their specific needs. The pilot, which
                               commenced in January 1998, is not intended to replace the existing
                               MTMC-managed program and affects a very small number of military
                               personnel. Presently, the option is offered for shipments originating in the
                               Puget Sound, Washington; San Diego, California; Norfolk, Virginia; and
                               Groton, Connecticut areas. Participation is limited to Navy military
                               members with permanent change of station orders from the participating
                               sites. Eligible participants must limit shipments to at least 3,000 pounds
                               and to costs between $2,500 and $25,000 (and it excludes shipments from
                               nontemporary storage or warehouse). The service member can select a
                               mover from a list of carriers that are self-certified as small businesses and
                               approved by MTMC. As of March 1999, 169 Navy service members had
                               selected this option. This pilot does not have a specific end date.




                               Page 4                                                     GAO/T-NSIAD-99-106
DOD Pilot Effort Proposed   On February 12, 1999, DOD stated that it intended to begin a fourth test,
in 1999                     called the Full Service Moving Project, with certain modifications based on
                            lessons learned from the Army Hunter test. The test will involve a larger
                            volume of moves and will include the National Capital Region (17,000
                            annual moves), Georgia (26,000 annual moves), and North Dakota (2,000
                            annual moves). Like the Army pilot, it is intended to replace the existing
                            MTMC-managed program by using a contractor or contractors to provide
                            both transportation and move management services. Presently, there is no
                            official start date for this project.


USTRANSCOM to Evaluate      To oversee all personal property pilots, the Deputy Under Secretary of
All Pilot Programs          Defense (Logistics) has tasked USTRANSCOM to ensure consistency in
                            evaluation criteria and assessment. USTRANSCOM plans to evaluate the
                            pilots to determine which pilots or tests, or portions thereof, could provide
                            better long-term results. It will then recommend the follow-on course of
                            action and time lines for implementation throughout DOD. Presently,
                            USTRANSCOM is working with the services to develop a common set of
                            data measures to evaluate the pilot projects’ results.



Results of Army Hunter      Because of the weaknesses in the Army’s evaluation methodology and data,
                            we were unable to validate all reported results of the Hunter pilot program.
Pilot Are Inconclusive      Moreover, the Army’s evaluation plan methodology was not an effective
                            tool for collecting and analyzing the pilot results. However, the lessons
                            learned from the pilot program do provide useful information to DOD as it
                            conducts and assesses its pilot efforts. Also, the pilot program is providing
                            services and benefits that were not previously available, including
                            point-to-point move management, personal move counseling and
                            coordination, direct claims settlement, assistance in buying/selling a
                            residence, and visibility of the shipment throughout the move.


The Army’s Assessment and   Through the first 12 months of operations, the Army reported that the
Evaluation Approach         Hunter pilot was successful in demonstrating that commercial practices
                            can be applied to the military relocation process. That is, Army officials
                            stated that they were able to contract with a relocation company to provide
                            services similar to those available in the private sector. To evaluate the
                            pilot, in January 1997, the Army began developing a test evaluation plan
                            that defined roles and responsibilities, test factors, and the process for
                            capturing data related to each test factor, and stated how test data would



                            Page 5                                                     GAO/T-NSIAD-99-106
be compared to that from the existing program. The plan stated that AAA
would be responsible for validating the baseline and test data and for
providing the Army with a report on the results of the test. AAA conducted
a review and its findings are incorporated in our evaluation. We reported in
June 1998 that the Army needed to further define the various factors and
measurements.6 The Army provided more clarity and a revised
methodology in the evaluation plan dated August 12, 1998. This
represented the third such revision, which was completed more than a year
after the pilot had begun. The modified plan indicated that the pilot results
would be reviewed on the basis of three factors: quality of life, cost, and
impact on small business—and would use a 5-point scoring system.
Quality of life was the key element the Army used in its evaluation,
consisting of 1 point each for its three sub-factors—customer satisfaction,
average claims settlement time, and percentage of direct deliveries—along
with 1 point each for the other two factors—total cost and impact on small
business, for a total of 5 points.

In its October 1998 evaluation report and February and March 1999
supplements, the Army reported to us that the pilot project earned a
passing score on each of the test factors that it measured over the 12-month
period ending June 30, 1998. The Army determined that the Hunter pilot
was a success because the factor points exceeded the minimum threshold
of 3.0 based on a 5.0-point scale. The Hunter pilot test earned 3.75 points,
as follows:

• customer satisfaction improved by over 11 percent (0.25 point),
• the initial offer to settle claims averaged 9 days (1 point),
• 100 percent of eligible shipments were delivered directly to the service
  member without storage in transit (1 point),
• Pilot project costs exceeded baseline costs by 18.6 percent (0.50 point),
  and
• 33 percent of the shipments were awarded to small businesses (1 point).

The Army stated that it also considered other factors, generally related to
transportation process improvements, and did not rely on test scores alone
to determine the pilot a success. These benefits included simplifying the
price structure and claims process, providing pre-audit and certification
services, and minimizing government-unique requirements. Each service


6
 Defense Transportation: The Army’s Hunter Pilot Project to Outsource Relocation Services
(GAO/NSIAD-98-149, June 10, 1998).




Page 6                                                                      GAO/T-NSIAD-99-106
                            member worked with one personal move coordinator who was
                            accountable for direct delivery at destination, timely service, and prompt
                            resolution of any problems. This is in contrast to the current process
                            where a service member may need to contact four to five different offices
                            over the course of a single move. Neither AAA nor we reviewed these
                            benefits cited by the Army because they were not included as measurable
                            factors in the evaluation plan.


Evaluation Weaknesses Led   We identified a number of shortfalls with the Army’s evaluation
to Inconclusive Results     methodology and data. In several instances, the data provided to support
                            the measurements did not demonstrate achievement of the reported
                            results. Most importantly, we could not confirm that customer satisfaction
                            improved by 11.5 percent. Further, we could not confirm that all eligible
                            shipments were delivered directly without incurring in-transit storage
                            costs, that offer of claims settlement time averaged 9 days, and that the
                            pilot program cost 18.6 percent more than the baseline cost. We reviewed
                            the methodology and data for each factor—quality of life (consisting of
                            three sub-factors: customer satisfaction, claims settlement time, and direct
                            deliveries); costs; and impact on small business.

Inconclusive Results for    Because service members were surveyed multiple times and the survey
Customer Satisfaction       methodology was flawed, the Army’s reported results on customer
                            satisfaction were compromised. Therefore, we could not validate that
                            customer satisfaction improved by 11.5 percent. However, data gathered in
                            these surveys did provide indicators of customer satisfaction as well as
                            provide lessons learned for DOD, USTRANSCOM, and the Army that may
                            be useful for conducting their pilot efforts.

                            To measure customer satisfaction, the Army said it would use its contractor
                            (Battelle’s Pacific Northwest National Laboratory) to survey members at
                            Hunter and those making similar moves at Fort Stewart, Georgia.
                            Responses from service members at both locations would be compared.
                            The Army told us that Battelle would conduct the survey, but instead, the
                            Army used USTRANSCOM’s survey, conducted by Electronic Data Systems
                            (EDS), to evaluate and score the pilot project. Initially, the Army informed




                            Page 7                                                    GAO/T-NSIAD-99-106
us that it did not use Battelle’s results because the draft report was difficult
to interpret and inconclusive. It subsequently expressed concern over a
disclaimer on the survey results.7

Nonetheless, service members were surveyed multiple times, possibly by
four different entities. EDS and Battelle (the survey contractors), Cendant
Mobility (which was required by its contract to survey a sample of
customers), and the carrier (for its own quality control) each may have
contacted the same service members to query them on their move
experience. As a result, the customer satisfaction results were
compromised and produced inconclusive results. EDS reported that some
service members refused to respond to the survey and that it does not
know how many of those who did respond only gave perfunctory answers.
In a caveat in its report, Battelle stated that it was aware that some
individuals were interviewed multiple times.

In addition, the contractors’ survey results varied significantly. While EDS
reported that 11.5 percent of respondents were more satisfied with their
current relocation experience (the pilot), Battelle reported that only
3 percent of its respondents were more satisfied.8 We are uncertain of the
specific reasons for these differences. However, we do know that the
question asked by the contractors to determine customer satisfaction was
not identical, and that the number of respondents in the two surveys varied
significantly. This may have contributed to the differences. EDS had a
much lower response rate, 53 percent, than Battelle, which achieved an
89-percent response from service members on which their results were
calculated. Further, EDS used a 75-percent significance level, while
Battelle calculated responses using a 95-percent significance level. The
latter is a standard used in social science research.9



7
 The Army said it subsequently learned that the Department of Energy requires such a disclaimer in all
studies performed for government entities and that the disclaimer had no relevance to the results
reported by Battelle.

8Battellealso reported that the 3-percent figure was “statistically insignificant,” meaning that the
variance in responses from the two groups (personnel at Hunter and at Fort Stewart) was not great
enough to conclude that their move experiences differed significantly.

9
 Significance level is the likelihood that a true population value may be rejected by a statistical test. In
the case of the two surveys, for example, Battelle used a 95-percent and EDS used a 75-percent
probability that the differences in relocation satisfaction levels were due to something other than
random occurrence. Battelle found no significant difference in satisfaction levels, with a 5-percent
chance of reaching an incorrect conclusion. EDS did find a significant difference but with a 25-percent
chance of reaching an incorrect conclusion.




Page 8                                                                             GAO/T-NSIAD-99-106
                         Other data obtained by the surveys, but not measured by the Army, provide
                         indicators of customer satisfaction and highlights areas that could be
                         addressed in future evaluations. For example, the Battelle survey found
                         that 61 percent of the Hunter respondents were more satisfied with their
                         most recent (pilot) moving experience than with their prior move.
                         Eighty-seven percent of these respondents were satisfied or very satisfied
                         with the carrier’s responsiveness. Both surveys also indicated that service
                         members relocating within the continental United States were generally
                         more satisfied with the pilot process than personnel moving overseas were.
                         The Army cited several examples of problems with overseas moves that
                         indicated greater dissatisfaction by those service members, and stated that
                         it has taken action to correct some of the problems.

                         The surveys also showed that service members liked some unique features
                         the pilot offered, such as one-on-one counseling and the simplified claims
                         process. However, few service members took advantage of many of the
                         additional services that Cendant Mobility offered, particularly assistance in
                         house hunting. Moreover, officials representing The Military Coalition,
                         which represents five million active duty, retired, reserve, and National
                         Guard personnel and their families, told us that more could be done to
                         tailor the range of services offered by the relocation company to meet
                         unique military needs. Such services, they said, might include more
                         assistance in finding rental property because military personnel tend to
                         rent rather than purchase, and providing assistance for spouses seeking
                         employment.

                         Another lesson learned related to access to survey data by outside
                         reviewers. AAA and we were unable to verify the survey data because the
                         independent survey contractors, citing privacy act restrictions, refused to
                         grant access to the original survey documentation. The Army stated that, in
                         future pilot projects, it would ensure that service members are not
                         subjected to multiple surveys and would establish a better survey method
                         for determining customer satisfaction.

Limited Data on Claims   Due to weaknesses in the Army’s data collection and analysis, we could not
Settlement Time          validate that the initial offer to settle claims occurred on average in
                         9 days after a completed claim form was received by Cendant Mobility.
                         While the claims settlement time is one approach to measuring the claims
                         process, it does not capture the entire claims process and does not
                         measure the extent the process represented an improvement over the
                         existing system. The pilot process, however, does offer benefits to the
                         service member in terms of full replacement value protection. For



                         Page 9                                                     GAO/T-NSIAD-99-106
                                 example, household effects are insured for their replacement value (up to
                                 $75,000) rather than their depreciated value (which had been capped at
                                 $40,000). The pilot process also requires less work for the service member
                                 since the contractor provides estimates on lost and damaged household
                                 goods, and the contractor pays the service member directly.

                                 The claims settlement time measurement is based on the contract
                                 requirement that the contractor offer settlement to a member within
                                 30 days after receiving a claim for damaged or lost property. Consequently,
                                 the Army developed a threshold for claims settlement of 30 days. Under
                                 the current process, we were told that it takes an average 30 days for the
                                 U.S. Army Claims Service to review a completed claim form and offer
                                 settlement, and it takes the Defense Finance and Accounting Service an
                                 additional day to process the payment. Service members may appeal to the
                                 contractor for reconsideration by providing additional information or file
                                 an appeal with the Army if resolution with the contractor could not be
                                 achieved—a process that can take more than 2 years to complete. While
                                 we found that some claims remained unpaid and unsettled because the
                                 claims were put “on hold” by the service member, the Army reported that
                                 no one under the Hunter pilot filed an appeal as of January 1999.

                                 The Army reported that Cendant Mobility took an average 9 days to offer
                                 settlement for lost or damaged property, but AAA and we could not confirm
                                 this because the Army had not collected the necessary data. AAA
                                 determined, and we concur, that Cendant Mobility may have taken as long
                                 as an average 28 days to offer settlement. This figure is based on
                                 measuring the time between the date the claim was signed and the date the
                                 member was offered settlement. The difference between the two estimates
                                 is primarily due to an average 19-day period between the date the claim was
                                 signed and the date the contractor reported that it entered the claim into its
                                 database. Although Cendant Mobility officials stated that its standard
                                 company practice is to enter a claim into the database the same day a claim
                                 is received, delays could be caused by (1) the form being mailed some time
                                 after it is dated, (2) mail delays, and (3) contractor delays in entering the
                                 claim into the database. Army officials stated that, due to the difficulties in
                                 measuring claims settlement time, they would change their data collection
                                 and analysis of this sub-factor in future pilot programs.

Direct Delivery Results May Be   We could not confirm that all eligible direct deliveries were made without
Overstated                       in-transit storage because the necessary data was not collected. Direct
                                 delivery is defined by the Army as delivery of a shipment to a service
                                 member’s residence without storage in transit. A shipment is eligible for



                                 Page 10                                                     GAO/T-NSIAD-99-106
                                direct delivery only if it has a destination address before the shipment is
                                offered for delivery. Direct delivery is a contract performance
                                measurement, and the contractor is paid an incentive for maintaining a
                                direct delivery rate of over 60 percent. Consequently, the Army threshold
                                for this measurement is 60 percent. Measuring direct delivery in this
                                manner does not provide the Army sufficient information to determine
                                whether the results represent an improvement over the current system.

                                Neither AAA nor we could confirm that the shipments that were directly
                                delivered to service members were the only ones eligible for direct delivery
                                because the necessary information on all shipments was not collected.
                                That is, other shipments may have been candidates (eligible) under the
                                Army criteria; however, AAA and we could not determine which shipments
                                had addresses prior to being offered for delivery (and were thus eligible)
                                but ended up requiring in-transit storage. The Army acknowledged the
                                difficulty in validating eligible direct deliveries and stated that in the future,
                                it would consider using in-transit storage costs as a test factor. In the
                                February 5, 1999, supplemental information provided to us, the Army
                                stated that 74 percent of Army-wide shipments required in-transit storage,
                                compared to 66 percent requiring temporary storage during the Army pilot.

                                Service members benefit from having household goods delivered directly
                                to their homes because the practice limits additional handling of their
                                property, reducing opportunities for loss and damage. Also, the
                                government avoids temporary, in-transit storage, which is costly and hard
                                for the Army to control. Historically, household goods shipments
                                frequently require temporary storage because service members often do
                                not know at the time of shipment their new address and/or service
                                members will not immediately move to the new duty station due to
                                vacations or military requirements.

Cost Data Were Inaccurate and   We could not validate that 18.6 percent represented the additional cost of
Incomplete                      the pilot project over the baseline cost due to weaknesses in the Army’s
                                methodology and the reliability of overhead cost data. As we reported in
                                June 1998, developing overhead costs historically has been difficult in the
                                government, including DOD, because such data are often unreliable and
                                unavailable (see app. I for a detailed breakdown of pilot and baseline
                                costs).

                                Neither AAA nor we could validate some overhead costs. Specifically, we
                                could not confirm the accuracy of the overhead costs attributed to MTMC
                                because the Army used data from fiscal year 1994. MTMC has changed



                                Page 11                                                        GAO/T-NSIAD-99-106
significantly since that time due to downsizing, but AAA could not obtain
updated costs from MTMC representing the current organization to
determine the reasonableness of its overhead cost. AAA and we also agree
that the costs associated with reducing the cost of processing
documentation (claims, invoices, and inbound) would not result in a cost
reduction to the government unless the activities that perform these
functions, such as the Defense Finance and Accounting Service, take
corresponding action to reduce their costs (for example, eliminating
personnel to reflect the drop in the workload).

AAA and we also identified other costs that were not included in the Army’s
analysis, such as the costs related to awarding the Hunter contract and the
contractor’s use of foreign-flag vessels. The Army could not provide an
estimated dollar value associated with the cost of developing the
solicitation, reviewing offers, and resolving bid protests. While we
recognize that some of these estimated costs may be infrequent or one-time
costs, they should be considered. Also, the Army did not factor the
contractor’s use of foreign-flag vessels into the pilot project’s
transportation costs.10 While individually, costs such as these are probably
of low dollar value in relation to other costs in the analysis; collectively,
they could have an impact on the difference between the baseline and pilot
costs.

According to the Army, some of the reasons for the higher estimated cost of
the pilot project can be attributed to (1) the difficulty in calculating
accurate baseline costs, which the Army believes are understated; (2) the
low volume of moves—1,400—which did not provide enough leverage to
negotiate better rates and discounts; (3) higher-than-expected cost of
overseas shipments; (4) relatively high unaccompanied baggage shipment
rates, which could have been lower using negotiated rates; (5) the packing
allowance for “do-it-yourself” moves was incorrectly calculated using
commercial rates during part of the 12-month test period, which resulted in
a higher rate; and (6) the fact that, generally, quality moving services cost
more. In regard to the high cost of overseas shipments, the Army provided
us with additional details of pilot project costs that showed a 44-percent
increase for overseas household goods shipments and a 2-percent increase
for domestic shipments over the baseline cost.




10
     Foreign-flag vessels are those ships registered in foreign countries.




Page 12                                                                      GAO/T-NSIAD-99-106
Impact on Small Business   Both AAA and we validated that Cendant Mobility awarded 33 percent of
                           the shipments to small businesses—10 percent to small business carriers
                           and 23 percent to small business agents. The Army measured the impact of
                           the pilot on two types of small business providers (carriers and agents) and
                           established a threshold of 23 percent to demonstrate successful
                           participation by small businesses.11 The Army based this factor on section
                           15(g) of the Small Business Act, which establishes a governmentwide goal
                           for participation by small business concerns at not less than 23 percent of
                           the value of all prime contracts. Of the 790 billed Hunter shipments during
                           the 12-month pilot, the Army reported that 261 (33 percent) were awarded
                           by Cendant Mobility to small businesses.



Status of Plans to         USTRANSCOM is still in the process of finalizing the evaluation plan for
                           other ongoing and planned relocation pilot programs. It is proposing to
Evaluate the Pilot         evaluate the pilots on the same three factors that the Army used in its
Programs                   evaluation, except the factors will be defined differently and will use a
                           more expansive point scale. Unresolved issues, as of February 1999,
                           include the development of a method that recognizes the unique
                           characteristics and/or process improvements of each pilot program, and
                           the validation of the baseline indirect costs that will be used for each of the
                           pilot programs. The evaluation plan does not currently include an
                           evaluation of the Hunter pilot, but has used lessons learned in the Hunter
                           pilot to help develop the plan. We provided comments to USTRANSCOM in
                           this process, and among other things, encouraged them to seek expert
                           methodological advice before finalizing the evaluation plan to enhance the
                           quality of USTRANSCOM’s assessment.



Summary                    Improving DOD’s personal property program has been a slow, complex
                           process. DOD and the services have spent a large amount of time and
                           effort to dramatically change the quality of service their military customers
                           receive. We support these efforts. However, before any final conclusions
                           can be reached, DOD must have accurate and credible data to determine
                           the type and extent of changes that should be made. To facilitate a timely
                           completion of this process, DOD needs to (1) develop a comprehensive
                           strategy for testing a finite number of approaches, (2) specify time lines for


                           11
                             In the motor freight and transportation industry, firms with annual gross revenues of $18.5 million or
                           less are classified as small businesses. See Federal Acquisition Regulation 19.102.




                           Page 13                                                                         GAO/T-NSIAD-99-106
implementation and completion, and (3) ensure that it has a
methodologically sound evaluation plan to assess each pilot’s attributes in
a comparable manner. As it has been in the past, it is important for DOD to
continue taking into consideration the views of the moving and relocation
companies that will be affected by changes to the current program.

Mr. Chairman, this concludes my prepared statement. I would be pleased
to answer any questions you or members of the Subcommittee might have
at this time.




Page 14                                                   GAO/T-NSIAD-99-106
Appendix I

Pilot and Baseline Costs Calculated by the
Department of the Army (Office of the Deputy
Chief of Staff for Logistics) and Validated by Army
Audit Agency                                                                                                      AppeIx
                                                                                                                       ndi




                                                                          Estimated                      Estimated
                 Cost elements                                            pilot costs                baseline costs
                 Transportation costs
                 1. Transportation                                        $3,005,229                    $2,102,384
                 2. Accessorials                                               25,767                       30,655
                  3. Storage                                                 210,706                       211,655
                 Total transportation costs                               $3,241,702                    $2,344,694


                 Claims paid by Army
                 4. Claims paid by Army                                              0                     $66,268


                 Overhead costs
                 5. Personnel                                                $79,802                      $221,721
                 6. Management price                                         132,700                             0
                 7. MTMC headquartersa                                               0                      63,510
                 8. Automation                                                       0                      52,612
                 9. Voucher processing                                          4,614                       90,658
                 10. Inbound processing                                              0                      46,834
                 11. Claims processing                                               0                      11,510
                 12. Pay for performance                                       14,756                            0
                 13. Building overhead                                         28,321                       43,286
                 14. Telephone and copier                                       1,281                       10,106
                 15. Consumables                                                3,760                        5,189
                 Total overhead costs                                       $265,234                      $545,426


                 Total cost                                               $3,506,936                    $2,956,388


                                                                            Baseline                     Percent of
                                                          Pilot costs         costs        Difference    difference
                 Percent pilot costs exceeded
                 baseline costs                           $3,506,936      $2,956,388         $550,548        18.6%
                 aMilitary   Traffic Management Command (MTMC)
                 Source: Office of the Deputy Chief of Staff for Logistics, Department of the Army




                 Page 15                                                                       GAO/T-NSIAD-99-106
Related GAO Products


                   Defense Transportation: The Army’s Hunter Pilot Project to Outsource
                   Relocation Services (GAO/NSIAD-98-149, June 10, 1998).

                   Defense Outsourcing: Better Data Needed to Support Overhead Rates for
                   A-76 Studies (GAO/NSIAD-98-62, Feb. 27, 1998).

                   Defense Transportation: Reengineering the DOD Personal Property
                   Program (GAO/NSIAD-97-49, Nov. 27, 1996).

                   Household Goods: Administrative Changes Would Improve DOD’s
                   Do-It-Yourself Moving Program (GAO/NSIAD-94-226, Sept. 27, 1994).

                   DOD Commercial Transportation: Savings Possible Through Better Audit
                   and Negotiation of Rates (GAO/NSIAD-92-61, Dec. 27, 1991).

                   Household Goods: Competition Among Commercial Movers Serving DOD
                   Can Be Improved (GAO/NSIAD-90-50, Feb. 12, 1990).




(709408)   Leter   Page 16                                                GAO/T-NSIAD-99-106
Ordering Information

The first copy of each GAO report and testimony is free.
Additional copies are $2 each. Orders should be sent to the
following address, accompanied by a check or money order made
out to the Superintendent of Documents, when necessary, VISA and
MasterCard credit cards are accepted, also.

Orders for 100 or more copies to be mailed to a single address are
discounted 25 percent.

Orders by mail:

U.S. General Accounting Office
P.O. Box 37050
Washington, DC 20013

or visit:

Room 1100
700 4th St. NW (corner of 4th and G Sts. NW)
U.S. General Accounting Office
Washington, DC

Orders may also be placed by calling (202) 512-6000
or by using fax number (202) 512-6061, or TDD (202) 512-2537.

Each day, GAO issues a list of newly available reports and
testimony. To receive facsimile copies of the daily list or any list
from the past 30 days, please call (202) 512-6000 using a touchtone
phone. A recorded menu will provide information on how to obtain
these lists.

For information on how to access GAO reports on the INTERNET,
send an e-mail message with “info” in the body to:

info@www.gao.gov

or visit GAO’s World Wide Web Home Page at:

http://www.gao.gov
United States                       Bulk Rate
General Accounting Office      Postage & Fees Paid
Washington, D.C. 20548-0001           GAO
                                 Permit No. GI00
Official Business
Penalty for Private Use $300

Address Correction Requested