United States General Accounting Office GAO Testimony Before the Subcommittee on Military Readiness, Committee on Armed Services, House of Representatives For Release Expected at 11:00 a.m., EST DEFENSE Thursday, March 18, 1999 TRANSPORTATION Efforts to Improve DOD’s Personal Property Program Statement by David R. Warren, Director, Defense Management Issues, National Security and International Affairs Division GAO/T-NSIAD-99-106 Mr. Chairman and Members of the Subcommittee: We are pleased to be here today to discuss the Department of Defense’s (DOD) efforts to improve its program for relocating military personnel and their families. This has been a long, complicated process. Since 1994, DOD has been engaged in initiatives to reengineer the personal property program to simplify current processes, control program costs, ensure quality of service by adopting commercial business processes characteristic of world-class businesses, and relieve carriers of DOD-unique terms and conditions. The U.S. Transportation Command (USTRANSCOM) is tasked with evaluating the pilots to determine which pilots, or portions thereof, could provide better long-term results for DOD.1 The statement of managers in the conference report on the DOD Appropriations Act for 1997 directed us to validate the results and savings achieved from any personal property program before DOD proposes further expansion of such programs.2 My testimony today will focus on our ongoing work related to the Army’s Hunter pilot and DOD’s plans to evaluate other ongoing and planned pilot programs. In assessing the Army’s evaluation of the Hunter Pilot, we focused on (1) the evaluation methodology and (2) the Army Audit Agency’s (AAA) validation of the test and baseline data. Results in Brief We agree that DOD needs to reengineer its personal property moving process to improve the poor quality of service military members are receiving. We support the use of pilots as a means to test new concepts. Although this process has been ongoing since the early 1990s, DOD is not yet in a position to determine what changes are needed. We were unable to validate the reported results of the Army’s evaluation of the Hunter pilot because of weaknesses in the evaluation methodology and the data. However, the lessons learned from the pilot do provide information that should be useful to DOD as it assesses and conducts its pilot efforts. Further, the Hunter pilot provides military personnel with 1The mission of USTRANSCOM, which is DOD’s single manager of all defense transportation services, is to provide global air, land, and sea transportation to meet national security needs, both in time of peace and time of war. USTRANSCOM executes its mission through three component commands: Military Traffic Management Command (MTMC) for land transportation and port operations, the Military Sealift Command for sea transport, and the Air Mobility Command for air transport. 2 House Report 104-863, Sept. 28, 1996. Page 1 GAO/T-NSIAD-99-106 services that were not previously available during the moving process, including personal move counseling and coordination, full replacement value for lost or damaged household goods, and visibility of the shipment throughout the move. Key pilot results that we were unable to validate included the level of customer satisfaction and costs. Although customer surveys were conducted, the results were inconclusive because of the methods used. While the Army indicated that the estimated pilot cost was higher than the baseline cost, we could not validate the extent to which the pilot cost exceeded the baseline cost. Further, we could not calculate the baseline and pilot costs due to weaknesses in the Army’s methodology and data reliability. As for the participation of small businesses, we confirmed that 33 percent of the pilot shipments were awarded by Cendant Mobility to small business carriers and agents. DOD has three pilot programs underway to improve its current personal property program and is proposing a fourth pilot. The general plan is to evaluate the results of these pilots and use that information to develop a redesigned Department-wide relocation program. To achieve this objective, it is imperative that DOD develop a well thought out strategy with clear time lines for testing each of the approaches, and more importantly, an evaluation methodology that will produce credible and accurate information to be used in making a final decision. However, plans for accomplishing these tasks and milestones for implementing a new process have not yet been finalized. Background DOD has long been concerned about the quality service members receive from its nearly $3 billion annual program to transport, store, and manage the household goods and unaccompanied baggage of its personnel. Past problems include poor service from its movers, excessive incidence of loss or damage to service members’ property, and high claims costs to the government. All of these problems contribute to a poor quality of service for persons using the system. MTMC’s Reengineering DOD first proposed reengineering its personal property program 5 years Efforts Began in 1994 ago. On June 21, 1994, the Deputy Commander in Chief, USTRANSCOM, directed MTMC, the Army component of USTRANSCOM and program manager for DOD’s Personal Property Shipment and Storage Program, to reengineer the personal property program. On June 15, 1995, the House Page 2 GAO/T-NSIAD-99-106 Committee on National Security, concurring that DOD must pursue a higher level of service, directed that DOD undertake a pilot program to implement commercial business practices and standards of service.3 MTMC planned to award contracts for the new program pursuant to the competitive acquisition system provisions (chapter 137 of title 10 of the U.S. Code and the primary implementing regulation contained in the Federal Acquisition Regulation). Expressing congressional concerns about the impact that the competition system and any nonstandard commercial business practice requirements might have on small businesses, the statement of managers accompanying the 1997 DOD Authorization Act directed that DOD report on the impact of the pilot program on small business. After reviewing the defense reports on small business impact, the House Committee on National Security was still concerned that MTMC’s pilot program did not satisfactorily address issues raised by the small moving companies and directed that the Secretary of Defense establish a working group of military and industry representatives to develop an alternative pilot program. MTMC Pilot Commenced in Although the working group reached a consensus on many issues, 1999 including a set of program goals, it could not reach agreement on the approach to take for the pilot test. Consequently, the two sides presented separate proposals. In November 1996, we reported that in our assessment MTMC’s proposal met the goals of reengineering the personal property program to a greater extent than the industry plan.4 The MTMC pilot program implementation, delayed by numerous bid protests, finally commenced in January 1999.5 The pilot reengineers the existing program for 50 percent of the moves originating in three states: North Carolina, South Carolina, and Florida. The pilot will run concurrently with the existing MTMC-managed program at installations in the three states and will involve approximately 18,500 annual moves. The 3 The House Committee on National Security is now called the House Armed Services Committee. 4See Defense Transportation: Reengineering the DOD Personal Property Program (GAO/NSIAD-97-49, Nov. 27, 1996) for more detailed information on the MTMC pilot effort. 5 Solicitation disputes in the following decisions: Aalco Forwarding, Inc., et al. (B-277241.8, B-277241.9, Oct. 21, 1997); Aalco Forwarding, Inc. et al. (B-277241.12, B-277241.13, Dec. 29,1997); Aalco Forwarding, Inc. et al. (B-277241.15, Mar. 11, 1998); Aalco Forwarding, Inc. et al. (B-277241.16, Mar. 11, 1998); Aalco Forwarding, Inc., et al. (B-277241, 277241.14, June 8, 1998); and Aalco Forwarding, Inc., et al. (B-277241, B-277241.20, B-277241.21, July 1, 1998). Page 3 GAO/T-NSIAD-99-106 key features of the pilot program include selecting carriers based on service member satisfaction and past performance rather than simply price; achieving stronger carrier commitment with long-term contracts; and offering full replacement value protection and direct claims settlement to users. MTMC’s pilot will run for a 3-year test period (1 year with two 1-year option periods), which will end in September 2002. Hunter Pilot Effort Began in Separately from the MTMC pilot program, the Army decided to determine 1996 whether the commercial business practice of outsourcing could alleviate known problem areas. The pilot was initiated in February 1996 as a quality-of-life effort to improve the relocation process and to test commercial business practices in a military environment at Hunter Army Airfield, Savannah, Georgia. On January 31, 1997, the Army selected PHH Relocation, a relocation/move management company (today named Cendant Mobility, located in Bethesda, Maryland), as the contractor for the pilot. In July 1997, Cendant Mobility began relocation operations at Hunter Army Airfield, consisting of point-to-point move management, personal move counseling and coordination, assistance in buying/selling a residence, 24-hour in-transit visibility of the shipment, direct claims settlement with the service member, full replacement value, and a single point of contact for the member, that were not previously available under the existing system. The contract will end on September 30, 1999. Navy Pilot Effort Began in The Navy initiated a separate pilot effort in 1997 to test the option of 1997 allowing service members to select their carrier, giving them more control over the relocation process to meet their specific needs. The pilot, which commenced in January 1998, is not intended to replace the existing MTMC-managed program and affects a very small number of military personnel. Presently, the option is offered for shipments originating in the Puget Sound, Washington; San Diego, California; Norfolk, Virginia; and Groton, Connecticut areas. Participation is limited to Navy military members with permanent change of station orders from the participating sites. Eligible participants must limit shipments to at least 3,000 pounds and to costs between $2,500 and $25,000 (and it excludes shipments from nontemporary storage or warehouse). The service member can select a mover from a list of carriers that are self-certified as small businesses and approved by MTMC. As of March 1999, 169 Navy service members had selected this option. This pilot does not have a specific end date. Page 4 GAO/T-NSIAD-99-106 DOD Pilot Effort Proposed On February 12, 1999, DOD stated that it intended to begin a fourth test, in 1999 called the Full Service Moving Project, with certain modifications based on lessons learned from the Army Hunter test. The test will involve a larger volume of moves and will include the National Capital Region (17,000 annual moves), Georgia (26,000 annual moves), and North Dakota (2,000 annual moves). Like the Army pilot, it is intended to replace the existing MTMC-managed program by using a contractor or contractors to provide both transportation and move management services. Presently, there is no official start date for this project. USTRANSCOM to Evaluate To oversee all personal property pilots, the Deputy Under Secretary of All Pilot Programs Defense (Logistics) has tasked USTRANSCOM to ensure consistency in evaluation criteria and assessment. USTRANSCOM plans to evaluate the pilots to determine which pilots or tests, or portions thereof, could provide better long-term results. It will then recommend the follow-on course of action and time lines for implementation throughout DOD. Presently, USTRANSCOM is working with the services to develop a common set of data measures to evaluate the pilot projects’ results. Results of Army Hunter Because of the weaknesses in the Army’s evaluation methodology and data, we were unable to validate all reported results of the Hunter pilot program. Pilot Are Inconclusive Moreover, the Army’s evaluation plan methodology was not an effective tool for collecting and analyzing the pilot results. However, the lessons learned from the pilot program do provide useful information to DOD as it conducts and assesses its pilot efforts. Also, the pilot program is providing services and benefits that were not previously available, including point-to-point move management, personal move counseling and coordination, direct claims settlement, assistance in buying/selling a residence, and visibility of the shipment throughout the move. The Army’s Assessment and Through the first 12 months of operations, the Army reported that the Evaluation Approach Hunter pilot was successful in demonstrating that commercial practices can be applied to the military relocation process. That is, Army officials stated that they were able to contract with a relocation company to provide services similar to those available in the private sector. To evaluate the pilot, in January 1997, the Army began developing a test evaluation plan that defined roles and responsibilities, test factors, and the process for capturing data related to each test factor, and stated how test data would Page 5 GAO/T-NSIAD-99-106 be compared to that from the existing program. The plan stated that AAA would be responsible for validating the baseline and test data and for providing the Army with a report on the results of the test. AAA conducted a review and its findings are incorporated in our evaluation. We reported in June 1998 that the Army needed to further define the various factors and measurements.6 The Army provided more clarity and a revised methodology in the evaluation plan dated August 12, 1998. This represented the third such revision, which was completed more than a year after the pilot had begun. The modified plan indicated that the pilot results would be reviewed on the basis of three factors: quality of life, cost, and impact on small business—and would use a 5-point scoring system. Quality of life was the key element the Army used in its evaluation, consisting of 1 point each for its three sub-factors—customer satisfaction, average claims settlement time, and percentage of direct deliveries—along with 1 point each for the other two factors—total cost and impact on small business, for a total of 5 points. In its October 1998 evaluation report and February and March 1999 supplements, the Army reported to us that the pilot project earned a passing score on each of the test factors that it measured over the 12-month period ending June 30, 1998. The Army determined that the Hunter pilot was a success because the factor points exceeded the minimum threshold of 3.0 based on a 5.0-point scale. The Hunter pilot test earned 3.75 points, as follows: • customer satisfaction improved by over 11 percent (0.25 point), • the initial offer to settle claims averaged 9 days (1 point), • 100 percent of eligible shipments were delivered directly to the service member without storage in transit (1 point), • Pilot project costs exceeded baseline costs by 18.6 percent (0.50 point), and • 33 percent of the shipments were awarded to small businesses (1 point). The Army stated that it also considered other factors, generally related to transportation process improvements, and did not rely on test scores alone to determine the pilot a success. These benefits included simplifying the price structure and claims process, providing pre-audit and certification services, and minimizing government-unique requirements. Each service 6 Defense Transportation: The Army’s Hunter Pilot Project to Outsource Relocation Services (GAO/NSIAD-98-149, June 10, 1998). Page 6 GAO/T-NSIAD-99-106 member worked with one personal move coordinator who was accountable for direct delivery at destination, timely service, and prompt resolution of any problems. This is in contrast to the current process where a service member may need to contact four to five different offices over the course of a single move. Neither AAA nor we reviewed these benefits cited by the Army because they were not included as measurable factors in the evaluation plan. Evaluation Weaknesses Led We identified a number of shortfalls with the Army’s evaluation to Inconclusive Results methodology and data. In several instances, the data provided to support the measurements did not demonstrate achievement of the reported results. Most importantly, we could not confirm that customer satisfaction improved by 11.5 percent. Further, we could not confirm that all eligible shipments were delivered directly without incurring in-transit storage costs, that offer of claims settlement time averaged 9 days, and that the pilot program cost 18.6 percent more than the baseline cost. We reviewed the methodology and data for each factor—quality of life (consisting of three sub-factors: customer satisfaction, claims settlement time, and direct deliveries); costs; and impact on small business. Inconclusive Results for Because service members were surveyed multiple times and the survey Customer Satisfaction methodology was flawed, the Army’s reported results on customer satisfaction were compromised. Therefore, we could not validate that customer satisfaction improved by 11.5 percent. However, data gathered in these surveys did provide indicators of customer satisfaction as well as provide lessons learned for DOD, USTRANSCOM, and the Army that may be useful for conducting their pilot efforts. To measure customer satisfaction, the Army said it would use its contractor (Battelle’s Pacific Northwest National Laboratory) to survey members at Hunter and those making similar moves at Fort Stewart, Georgia. Responses from service members at both locations would be compared. The Army told us that Battelle would conduct the survey, but instead, the Army used USTRANSCOM’s survey, conducted by Electronic Data Systems (EDS), to evaluate and score the pilot project. Initially, the Army informed Page 7 GAO/T-NSIAD-99-106 us that it did not use Battelle’s results because the draft report was difficult to interpret and inconclusive. It subsequently expressed concern over a disclaimer on the survey results.7 Nonetheless, service members were surveyed multiple times, possibly by four different entities. EDS and Battelle (the survey contractors), Cendant Mobility (which was required by its contract to survey a sample of customers), and the carrier (for its own quality control) each may have contacted the same service members to query them on their move experience. As a result, the customer satisfaction results were compromised and produced inconclusive results. EDS reported that some service members refused to respond to the survey and that it does not know how many of those who did respond only gave perfunctory answers. In a caveat in its report, Battelle stated that it was aware that some individuals were interviewed multiple times. In addition, the contractors’ survey results varied significantly. While EDS reported that 11.5 percent of respondents were more satisfied with their current relocation experience (the pilot), Battelle reported that only 3 percent of its respondents were more satisfied.8 We are uncertain of the specific reasons for these differences. However, we do know that the question asked by the contractors to determine customer satisfaction was not identical, and that the number of respondents in the two surveys varied significantly. This may have contributed to the differences. EDS had a much lower response rate, 53 percent, than Battelle, which achieved an 89-percent response from service members on which their results were calculated. Further, EDS used a 75-percent significance level, while Battelle calculated responses using a 95-percent significance level. The latter is a standard used in social science research.9 7 The Army said it subsequently learned that the Department of Energy requires such a disclaimer in all studies performed for government entities and that the disclaimer had no relevance to the results reported by Battelle. 8Battellealso reported that the 3-percent figure was “statistically insignificant,” meaning that the variance in responses from the two groups (personnel at Hunter and at Fort Stewart) was not great enough to conclude that their move experiences differed significantly. 9 Significance level is the likelihood that a true population value may be rejected by a statistical test. In the case of the two surveys, for example, Battelle used a 95-percent and EDS used a 75-percent probability that the differences in relocation satisfaction levels were due to something other than random occurrence. Battelle found no significant difference in satisfaction levels, with a 5-percent chance of reaching an incorrect conclusion. EDS did find a significant difference but with a 25-percent chance of reaching an incorrect conclusion. Page 8 GAO/T-NSIAD-99-106 Other data obtained by the surveys, but not measured by the Army, provide indicators of customer satisfaction and highlights areas that could be addressed in future evaluations. For example, the Battelle survey found that 61 percent of the Hunter respondents were more satisfied with their most recent (pilot) moving experience than with their prior move. Eighty-seven percent of these respondents were satisfied or very satisfied with the carrier’s responsiveness. Both surveys also indicated that service members relocating within the continental United States were generally more satisfied with the pilot process than personnel moving overseas were. The Army cited several examples of problems with overseas moves that indicated greater dissatisfaction by those service members, and stated that it has taken action to correct some of the problems. The surveys also showed that service members liked some unique features the pilot offered, such as one-on-one counseling and the simplified claims process. However, few service members took advantage of many of the additional services that Cendant Mobility offered, particularly assistance in house hunting. Moreover, officials representing The Military Coalition, which represents five million active duty, retired, reserve, and National Guard personnel and their families, told us that more could be done to tailor the range of services offered by the relocation company to meet unique military needs. Such services, they said, might include more assistance in finding rental property because military personnel tend to rent rather than purchase, and providing assistance for spouses seeking employment. Another lesson learned related to access to survey data by outside reviewers. AAA and we were unable to verify the survey data because the independent survey contractors, citing privacy act restrictions, refused to grant access to the original survey documentation. The Army stated that, in future pilot projects, it would ensure that service members are not subjected to multiple surveys and would establish a better survey method for determining customer satisfaction. Limited Data on Claims Due to weaknesses in the Army’s data collection and analysis, we could not Settlement Time validate that the initial offer to settle claims occurred on average in 9 days after a completed claim form was received by Cendant Mobility. While the claims settlement time is one approach to measuring the claims process, it does not capture the entire claims process and does not measure the extent the process represented an improvement over the existing system. The pilot process, however, does offer benefits to the service member in terms of full replacement value protection. For Page 9 GAO/T-NSIAD-99-106 example, household effects are insured for their replacement value (up to $75,000) rather than their depreciated value (which had been capped at $40,000). The pilot process also requires less work for the service member since the contractor provides estimates on lost and damaged household goods, and the contractor pays the service member directly. The claims settlement time measurement is based on the contract requirement that the contractor offer settlement to a member within 30 days after receiving a claim for damaged or lost property. Consequently, the Army developed a threshold for claims settlement of 30 days. Under the current process, we were told that it takes an average 30 days for the U.S. Army Claims Service to review a completed claim form and offer settlement, and it takes the Defense Finance and Accounting Service an additional day to process the payment. Service members may appeal to the contractor for reconsideration by providing additional information or file an appeal with the Army if resolution with the contractor could not be achieved—a process that can take more than 2 years to complete. While we found that some claims remained unpaid and unsettled because the claims were put “on hold” by the service member, the Army reported that no one under the Hunter pilot filed an appeal as of January 1999. The Army reported that Cendant Mobility took an average 9 days to offer settlement for lost or damaged property, but AAA and we could not confirm this because the Army had not collected the necessary data. AAA determined, and we concur, that Cendant Mobility may have taken as long as an average 28 days to offer settlement. This figure is based on measuring the time between the date the claim was signed and the date the member was offered settlement. The difference between the two estimates is primarily due to an average 19-day period between the date the claim was signed and the date the contractor reported that it entered the claim into its database. Although Cendant Mobility officials stated that its standard company practice is to enter a claim into the database the same day a claim is received, delays could be caused by (1) the form being mailed some time after it is dated, (2) mail delays, and (3) contractor delays in entering the claim into the database. Army officials stated that, due to the difficulties in measuring claims settlement time, they would change their data collection and analysis of this sub-factor in future pilot programs. Direct Delivery Results May Be We could not confirm that all eligible direct deliveries were made without Overstated in-transit storage because the necessary data was not collected. Direct delivery is defined by the Army as delivery of a shipment to a service member’s residence without storage in transit. A shipment is eligible for Page 10 GAO/T-NSIAD-99-106 direct delivery only if it has a destination address before the shipment is offered for delivery. Direct delivery is a contract performance measurement, and the contractor is paid an incentive for maintaining a direct delivery rate of over 60 percent. Consequently, the Army threshold for this measurement is 60 percent. Measuring direct delivery in this manner does not provide the Army sufficient information to determine whether the results represent an improvement over the current system. Neither AAA nor we could confirm that the shipments that were directly delivered to service members were the only ones eligible for direct delivery because the necessary information on all shipments was not collected. That is, other shipments may have been candidates (eligible) under the Army criteria; however, AAA and we could not determine which shipments had addresses prior to being offered for delivery (and were thus eligible) but ended up requiring in-transit storage. The Army acknowledged the difficulty in validating eligible direct deliveries and stated that in the future, it would consider using in-transit storage costs as a test factor. In the February 5, 1999, supplemental information provided to us, the Army stated that 74 percent of Army-wide shipments required in-transit storage, compared to 66 percent requiring temporary storage during the Army pilot. Service members benefit from having household goods delivered directly to their homes because the practice limits additional handling of their property, reducing opportunities for loss and damage. Also, the government avoids temporary, in-transit storage, which is costly and hard for the Army to control. Historically, household goods shipments frequently require temporary storage because service members often do not know at the time of shipment their new address and/or service members will not immediately move to the new duty station due to vacations or military requirements. Cost Data Were Inaccurate and We could not validate that 18.6 percent represented the additional cost of Incomplete the pilot project over the baseline cost due to weaknesses in the Army’s methodology and the reliability of overhead cost data. As we reported in June 1998, developing overhead costs historically has been difficult in the government, including DOD, because such data are often unreliable and unavailable (see app. I for a detailed breakdown of pilot and baseline costs). Neither AAA nor we could validate some overhead costs. Specifically, we could not confirm the accuracy of the overhead costs attributed to MTMC because the Army used data from fiscal year 1994. MTMC has changed Page 11 GAO/T-NSIAD-99-106 significantly since that time due to downsizing, but AAA could not obtain updated costs from MTMC representing the current organization to determine the reasonableness of its overhead cost. AAA and we also agree that the costs associated with reducing the cost of processing documentation (claims, invoices, and inbound) would not result in a cost reduction to the government unless the activities that perform these functions, such as the Defense Finance and Accounting Service, take corresponding action to reduce their costs (for example, eliminating personnel to reflect the drop in the workload). AAA and we also identified other costs that were not included in the Army’s analysis, such as the costs related to awarding the Hunter contract and the contractor’s use of foreign-flag vessels. The Army could not provide an estimated dollar value associated with the cost of developing the solicitation, reviewing offers, and resolving bid protests. While we recognize that some of these estimated costs may be infrequent or one-time costs, they should be considered. Also, the Army did not factor the contractor’s use of foreign-flag vessels into the pilot project’s transportation costs.10 While individually, costs such as these are probably of low dollar value in relation to other costs in the analysis; collectively, they could have an impact on the difference between the baseline and pilot costs. According to the Army, some of the reasons for the higher estimated cost of the pilot project can be attributed to (1) the difficulty in calculating accurate baseline costs, which the Army believes are understated; (2) the low volume of moves—1,400—which did not provide enough leverage to negotiate better rates and discounts; (3) higher-than-expected cost of overseas shipments; (4) relatively high unaccompanied baggage shipment rates, which could have been lower using negotiated rates; (5) the packing allowance for “do-it-yourself” moves was incorrectly calculated using commercial rates during part of the 12-month test period, which resulted in a higher rate; and (6) the fact that, generally, quality moving services cost more. In regard to the high cost of overseas shipments, the Army provided us with additional details of pilot project costs that showed a 44-percent increase for overseas household goods shipments and a 2-percent increase for domestic shipments over the baseline cost. 10 Foreign-flag vessels are those ships registered in foreign countries. Page 12 GAO/T-NSIAD-99-106 Impact on Small Business Both AAA and we validated that Cendant Mobility awarded 33 percent of the shipments to small businesses—10 percent to small business carriers and 23 percent to small business agents. The Army measured the impact of the pilot on two types of small business providers (carriers and agents) and established a threshold of 23 percent to demonstrate successful participation by small businesses.11 The Army based this factor on section 15(g) of the Small Business Act, which establishes a governmentwide goal for participation by small business concerns at not less than 23 percent of the value of all prime contracts. Of the 790 billed Hunter shipments during the 12-month pilot, the Army reported that 261 (33 percent) were awarded by Cendant Mobility to small businesses. Status of Plans to USTRANSCOM is still in the process of finalizing the evaluation plan for other ongoing and planned relocation pilot programs. It is proposing to Evaluate the Pilot evaluate the pilots on the same three factors that the Army used in its Programs evaluation, except the factors will be defined differently and will use a more expansive point scale. Unresolved issues, as of February 1999, include the development of a method that recognizes the unique characteristics and/or process improvements of each pilot program, and the validation of the baseline indirect costs that will be used for each of the pilot programs. The evaluation plan does not currently include an evaluation of the Hunter pilot, but has used lessons learned in the Hunter pilot to help develop the plan. We provided comments to USTRANSCOM in this process, and among other things, encouraged them to seek expert methodological advice before finalizing the evaluation plan to enhance the quality of USTRANSCOM’s assessment. Summary Improving DOD’s personal property program has been a slow, complex process. DOD and the services have spent a large amount of time and effort to dramatically change the quality of service their military customers receive. We support these efforts. However, before any final conclusions can be reached, DOD must have accurate and credible data to determine the type and extent of changes that should be made. To facilitate a timely completion of this process, DOD needs to (1) develop a comprehensive strategy for testing a finite number of approaches, (2) specify time lines for 11 In the motor freight and transportation industry, firms with annual gross revenues of $18.5 million or less are classified as small businesses. See Federal Acquisition Regulation 19.102. Page 13 GAO/T-NSIAD-99-106 implementation and completion, and (3) ensure that it has a methodologically sound evaluation plan to assess each pilot’s attributes in a comparable manner. As it has been in the past, it is important for DOD to continue taking into consideration the views of the moving and relocation companies that will be affected by changes to the current program. Mr. Chairman, this concludes my prepared statement. I would be pleased to answer any questions you or members of the Subcommittee might have at this time. Page 14 GAO/T-NSIAD-99-106 Appendix I Pilot and Baseline Costs Calculated by the Department of the Army (Office of the Deputy Chief of Staff for Logistics) and Validated by Army Audit Agency AppeIx ndi Estimated Estimated Cost elements pilot costs baseline costs Transportation costs 1. Transportation $3,005,229 $2,102,384 2. Accessorials 25,767 30,655 3. Storage 210,706 211,655 Total transportation costs $3,241,702 $2,344,694 Claims paid by Army 4. Claims paid by Army 0 $66,268 Overhead costs 5. Personnel $79,802 $221,721 6. Management price 132,700 0 7. MTMC headquartersa 0 63,510 8. Automation 0 52,612 9. Voucher processing 4,614 90,658 10. Inbound processing 0 46,834 11. Claims processing 0 11,510 12. Pay for performance 14,756 0 13. Building overhead 28,321 43,286 14. Telephone and copier 1,281 10,106 15. Consumables 3,760 5,189 Total overhead costs $265,234 $545,426 Total cost $3,506,936 $2,956,388 Baseline Percent of Pilot costs costs Difference difference Percent pilot costs exceeded baseline costs $3,506,936 $2,956,388 $550,548 18.6% aMilitary Traffic Management Command (MTMC) Source: Office of the Deputy Chief of Staff for Logistics, Department of the Army Page 15 GAO/T-NSIAD-99-106 Related GAO Products Defense Transportation: The Army’s Hunter Pilot Project to Outsource Relocation Services (GAO/NSIAD-98-149, June 10, 1998). Defense Outsourcing: Better Data Needed to Support Overhead Rates for A-76 Studies (GAO/NSIAD-98-62, Feb. 27, 1998). Defense Transportation: Reengineering the DOD Personal Property Program (GAO/NSIAD-97-49, Nov. 27, 1996). Household Goods: Administrative Changes Would Improve DOD’s Do-It-Yourself Moving Program (GAO/NSIAD-94-226, Sept. 27, 1994). DOD Commercial Transportation: Savings Possible Through Better Audit and Negotiation of Rates (GAO/NSIAD-92-61, Dec. 27, 1991). Household Goods: Competition Among Commercial Movers Serving DOD Can Be Improved (GAO/NSIAD-90-50, Feb. 12, 1990). (709408) Leter Page 16 GAO/T-NSIAD-99-106 Ordering Information The first copy of each GAO report and testimony is free. Additional copies are $2 each. Orders should be sent to the following address, accompanied by a check or money order made out to the Superintendent of Documents, when necessary, VISA and MasterCard credit cards are accepted, also. Orders for 100 or more copies to be mailed to a single address are discounted 25 percent. Orders by mail: U.S. General Accounting Office P.O. Box 37050 Washington, DC 20013 or visit: Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders may also be placed by calling (202) 512-6000 or by using fax number (202) 512-6061, or TDD (202) 512-2537. Each day, GAO issues a list of newly available reports and testimony. To receive facsimile copies of the daily list or any list from the past 30 days, please call (202) 512-6000 using a touchtone phone. A recorded menu will provide information on how to obtain these lists. For information on how to access GAO reports on the INTERNET, send an e-mail message with “info” in the body to: email@example.com or visit GAO’s World Wide Web Home Page at: http://www.gao.gov United States Bulk Rate General Accounting Office Postage & Fees Paid Washington, D.C. 20548-0001 GAO Permit No. GI00 Official Business Penalty for Private Use $300 Address Correction Requested
Defense Transportation: Efforts to Improve DOD's Personal Property Program
Published by the Government Accountability Office on 1999-03-18.
Below is a raw (and likely hideous) rendition of the original report. (PDF)