United States General Accounting Office GAO Report to the Chairman, Subcommittee on National Security, Emerging Threats and International Relations, Committee on Government Reform, House of Representatives March 2003 MILITARY READINESS DOD Needs to Better Manage Automatic Test Equipment Modernization GAO-03-451 March 2003 MILITARY READINESS DOD Needs to Better Manage Automatic Highlights of GAO-03-451, a report to the Chairman, Subcommittee on National Test Equipment Modernization Security, Emerging Threats and International Relations, House Committee on Government Reform The services have billions of DOD and the services face growing concerns regarding obsolete automatic dollars worth of outdated and test equipment, given the high costs of modernizing or replacing it and its obsolete automatic test equipment potential effect on aircraft readiness. The Navy and Air Force, for example, (ATE) used to test components estimate that they will spend billions of dollars to modernize or replace on military aircraft or weapon this equipment, much of which was acquired in the 1970s and 1980s. In the systems. Department of Defense (DOD) policy advocates the meantime, the aging testers are becoming increasingly out of date and more development and acquisition of test difficult to support. When the testers do not work properly, maintenance can equipment that can be used on suffer and readiness can be adversely affected. multiple types of weapon systems and aircraft and used Since 1994, DOD policy has advocated the acquisition of test equipment interchangeably between the that can be used on multiple weapon systems and aircraft and can be used services. interchangeably between the services; progress in this regard has been slow. For example, although the Navy set out in 1991 to replace 25 major tester At the request of the types with one standard tester by 2000, budget cuts and delays in developing Subcommittee’s Chairman, software have resulted in delays in completing the replacement of these GAO examined the problems that obsolete testers until 2008. The Air Force has only recently initiated a test the Air Force, Navy, and Marine Corps are facing with this aging equipment modernization plan. However, little evidence suggests that equipment and their efforts to consideration is being given to the acquisition of equipment that would have comply with DOD policy. common utility for more than one weapon system as DOD policy advocates. For procurement of new weapon systems, the Air Force is giving little consideration to the use of a common tester, while a common tester is planned for use as the primary tester for the Joint Strike Fighter. GAO recommends that the Secretary of Defense reemphasize Although DOD tasked the Navy as its Executive Agent for automatic test the policy and reconsider the equipment in 1994, the agent has made only limited progress in achieving organizational placement and authority of the Executive Agent compliance across all the services with DOD policy advocating the for ATE. development of common systems. While the Executive Agent can point to some successes in individual systems, its officials acknowledged that the DOD concurred with GAO’s organization does not have sufficient authority or resources to fully recommendations and agreed that implement the policy and achieve the maximum commonality possible. its Executive Agent for ATE should be given the authority and resources to more effectively fulfill the Department’s oversight responsibilities. www.gao.gov/cgi-bin/getrpt?GAO-03-451. To view the full report, including the scope and methodology, click on the link above. The Navy’s Consolidated Automated Support System will minimize unique types of testers. For more information, contact Neal Curtin at (757) 552-8100 or email@example.com. Contents Letter 1 Results in Brief 3 Background 4 Aging ATE Presents Major Challenges to DOD 5 DOD Has Had Limited Success in Fostering Commonality 7 Conclusions 15 Recommendations for Executive Action 16 Agency Comments and Our Evaluation 16 Appendix I Scope and Methodology 18 Appendix II Comments from the Department of Defense 20 Related GAO Products 23 Figures Figure 1: Schematic of an Automatic Test System, Including ATE and Test Program Set Components 2 Figure 2: CASS Station 9 Abbreviations ATE automatic test equipment CASS Consolidated Automated Support System DOD Department of Defense GAO General Accounting Office JSF Joint Strike Fighter This is a work of the U.S. Government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. It may contain copyrighted graphics, images or other materials. Permission from the copyright holder may be necessary should you wish to reproduce copyrighted materials separately from GAO’s product. Page i GAO-03-451 DOD Automated Test Equipment Modernization United States General Accounting Office Washington, DC 20548 March 31, 2003 The Honorable Christopher Shays Chairman, Subcommittee on National Security, Emerging Threats and International Relations Committee on Government Reform House of Representatives Dear Mr. Chairman: One of the major challenges facing the military services is that of modernizing billions of dollars’ worth of aging and increasingly obsolete automatic test equipment1 (ATE) used to troubleshoot and diagnose components of aircraft or weapon systems. Since 1994, Department of Defense (DOD) policy has advocated a reduction in the proliferation of testers that are unique to one type of weapon system or aircraft, favoring the development or acquisition of testers that (1) are interoperable2 within a service and between the services and (2) can be used on many different components of multiple types of aircraft and weapon systems. As the services modernize ATE, they are challenged to adhere to DOD policy and reduce the number of unique testers. ATE with its test program sets, including test software, an interface device that connects the ATE to the item being tested, and documentation make up an automatic test system. (See fig. 1.) 1 The test hardware and software of an integrated assembly of stimulus, measurement, and switching components under computer control that is capable of processing software routines designed specifically to test a particular item or group of items. 2 “Interoperability” is the ability of systems to provide data or material to and accept the same from other systems and to operate effectively together. Page 1 GAO-03-451 DOD Automated Test Equipment Modernization Figure 1: Schematic of an Automatic Test System, Including ATE and Test Program Set Components ATE is used by the services at all maintenance levels (from flight lines for routine maintenance to depots for major overhauls and factories for production and acceptance testing) to test electronic systems and components that are difficult or impossible to test manually, to isolate system malfunctions, and to verify that systems are operating properly. These testers can be made to examine a single aircraft system, various components of an aircraft, or multiple components of different aircraft. Because of your concerns regarding DOD’s continued reporting of spare parts shortages and the potential impact that ATE obsolescence could have on the readiness of military aircraft, you asked us to determine whether DOD and the services are giving adequate attention to ATE modernization efforts. Specifically, our objectives were to identify (1) what problems the Air Force and Navy3 are facing with their ATE 3 The term “Navy,” as presented in this report, represents Naval Aviation, which includes the Marine Corps. Page 2 GAO-03-451 DOD Automated Test Equipment Modernization and (2) how successful DOD, the Air Force, and the Navy have been in addressing the proliferation of unique testers. Our review included ATE for aircraft managed by the Air Force and the Navy and included information on ATE acquisition for two fighter aircraft currently under development: the multiservice Joint Strike Fighter (JSF) and the Air Force’s F/A-22. Our scope and methodology are described in more detail in appendix I. We performed our review from January 2002 through March 2003 in accordance with generally accepted government auditing standards. DOD and the services face growing concerns regarding obsolete ATE, Results in Brief given the high costs of modernizing or replacing this type of equipment and its potential adverse impact on aircraft readiness. ATE acquired in the 1970s and 1980s is becoming increasingly out-of-date and more difficult to support. These obsolescence issues are further aggravated by new technologies that, in some cases, make ATE obsolete even before the new testers can be fully fielded. Also, older testers are kept much longer than initially planned because the weapon systems they support are being kept longer. Repair parts for older ATE are becoming increasingly scarce, as more contractors discontinue their support. Although exact cost figures are not available, the services estimate that they will need several billion dollars in the coming years either to acquire new testers or modernize existing ones. Although the services do not maintain data that allow them to measure the extent to which obsolete ATE affects readiness, according to DOD readiness reports, only 28 percent of Air Force, Navy, and Marine Corps key aircraft models met their readiness goals in 2002. Although a combination of factors affects readiness goals, the availability of spare parts is a key contributor to readiness, and the performance of ATE significantly affects the supply of spare parts. For years, DOD’s policy has aimed to minimize the acquisition of ATE that is unique to a particular weapon system; however, the implementation of this policy has been slow. In 1994, DOD appointed the Navy as its Executive Agent to oversee policy implementation; however, according to Executive Agent officials, the Navy has not had the authority or resources to effectively carry out this oversight. The services lack ATE Page 3 GAO-03-451 DOD Automated Test Equipment Modernization modernization plans, and there is no DOD-wide approach to ensure that all ATE acquisitions and modernizations are identified in an early enough stage to ensure that commonality4 and interoperability are adequately considered. Without sufficient information concerning the magnitude of the services’ modernization efforts or a departmentwide approach to accomplish ATE modernization, the department faces a very expensive and time-consuming ATE modernization effort, with the continued proliferation of unique testers and no assurance that resources are allocated in the most effective manner. As a result, some ATE modernization and acquisition planning is being done with little consideration to commonality. For example, Executive Agent officials said that they have not had contact with the Air Force’s F/A-22 project office concerning ATE since 1994, and it does not appear that commonality is being considered or that unique ATE development will be minimized. We are making several recommendations aimed at reinforcing DOD’s stated goal of achieving more commonality of test equipment and strengthening the department’s oversight. DOD concurred with our recommendations and agreed to reemphasize its policy that common automatic test equipment be developed to the maximum extent possible. In addition, DOD agreed that its Executive Agent for ATE should be given the authority and resources to more effectively fulfill its oversight responsibilities. DOD’s comments on our report are reprinted in their entirety in appendix II. It is estimated that DOD employs more than 400 different tester types. Background This equipment is used to diagnose problems in aircraft avionics and weapon system components so that the component can be repaired and replaced on the aircraft or put into the supply system for future use. For example, testers may be used to diagnose problems with aircraft radars, guidance and control systems, or weapon systems. According to DOD, the department spent over $50 billion in its acquisition and support of ATE from 1980 through 1992, and the procurement was characterized by the proliferation of testers designed to support a specific weapon system or component. These testers are quickly becoming obsolete and more difficult and costly to maintain because they may no longer be in production and parts may not be readily available. Over the years, various studies have criticized the continued proliferation of unique ATE and 4 Test equipment that can be used on multiple airframes and weapon systems. Page 4 GAO-03-451 DOD Automated Test Equipment Modernization highlighted the need for the development and acquisition of testers that can be used to test more than one system or component. In September 1993, the House Appropriations Committee recommended that the Secretary of Defense develop a DOD-wide policy requiring ATE commonality among the services, along with a formal implementation mechanism with sufficient authority, staffing, and funding to ensure compliance.5 In 1994, DOD established a policy stating that managers of DOD programs should select families of testers or commercial off-the-shelf components to meet all ATE acquisition needs and that the introduction of unique testers should be minimized. DOD designated the Navy at that time as its Executive Agent to oversee policy implementation in all services, and identified a goal of reducing life-cycle costs and providing greater ATE commonality and interoperability. Additional DOD guidance published in 1996 and 1997 required that all ATE acquisitions be part of the approved families of testers or commercial off-the-shelf. DOD faces major challenges with aging and increasingly obsolete ATE. Aging ATE Presents These problems include the high costs of maintaining and replacing Major Challenges ATE and the declining availability of spare parts for the aging testers. In addition, several DOD organizations, including the Navy Inspector to DOD General, have suggested that aging and obsolete ATE may adversely affect aviation readiness. Modernization Costs Departmentwide estimates of funds needed for ATE modernization and Are Substantial acquisition are not readily available. However, according to Air Force and Navy ATE managers, most of the services’ ATE is obsolete and will need to be upgraded or replaced over the next several years. Our study confirmed that replacement and modernization costs would be substantial. The Navy, for example, spent about $1.5 billion from fiscal years 1990 through 2002 for the acquisition of its primary family of testers and plans to spend an additional $430 million through fiscal year 2007. Additionally, the Navy estimates that it plans to spend $584 million through fiscal year 2007 to adapt existing test program sets necessary to perform specific tests of the various aircraft components supported by this family of testers. The Navy also anticipates spending an additional $584 million to develop program test sets for new weapon system requirements. 5 House Report No 103-254, Sept. 22, 1993. Page 5 GAO-03-451 DOD Automated Test Equipment Modernization Information on the Air Force’s spending for ATE modernization is somewhat sketchy, as limited data are available centrally for individual weapon systems. According to a recent study done for the Air Force, the service has not developed a plan that allows modernization funding requirements to be determined. However, estimates are available for selected systems. The F-15 fighter program office, for example, is spending approximately $325 million on just one tester that will be fielded in 2004. It also plans to upgrade its electronic warfare tester, which is one of seven primary testers for the aircraft, at a cost of over $40 million. A 2002 study of B-52 bomber ATE identified obsolescence issues associated with six of the aircraft’s seven major testers that will require more than $140 million in the near future. Similarly, the upgrade of a unique B-1 bomber tester is expected to exceed $15 million, even though the Air Force is considering replacing this tester and has already begun planning the acquisition. The latest estimate for the new tester is $190 million. Current ATE estimates for the F/A-22, which is still under development, are not available. However, estimates made early in the development phase exceeded $1.5 billion. Readiness Could Be ATE is becoming increasingly out-of-date and more difficult to support. Adversely Affected And, according to service officials, using this outdated equipment to perform required tests in a timely manner is becoming increasingly challenging. Although the services could not quantify the extent that tester problems affect readiness, service officials noted that without adequate test equipment to diagnose problems, components cannot be repaired in a timely manner and the mission capability of military aircraft can be adversely affected. In August 2000, the Navy Inspector General identified shortfalls in ATE as having a negative impact on naval aviation and, in particular, on the availability of repaired components. During the same time frame, a Navy operational advisory group, recognizing the importance of ATE in maintaining readiness, ranked support equipment, including ATE, as one of its top 20 readiness issues. We have issued several reports in the recent past addressing the shortage of spare parts—a potential result of ATE problems. In addition, according to DOD readiness reports, only 28 percent of Air Force, Navy, and Marine Corps key aircraft models met their readiness goals in fiscal year 2002. Although difficulties in meeting these goals are caused by a complex combination of interrelated logistical and operational factors, the shortage of spare parts was a major cause. ATE plays a significant role in the supply of available spares, since this equipment affects both how many parts are taken out of service for repair and how quickly they are repaired and Page 6 GAO-03-451 DOD Automated Test Equipment Modernization returned. We reported that maintenance and repair facilities routinely work around spare parts shortages by removing a working part from one aircraft to replace a nonworking part in another aircraft, a practice called “cannibalization.”6 And, although the services do not record increases in cannibalizations that are caused by ATE problems, the services use cannibalization as a routine maintenance practice when testers are not available or not working properly. In July 2001, we reported that as a result of ATE not working properly, unfilled requisitions were adversely affecting the mission capability of F-14 aircraft.7 In another case, more than 1,200 Air Force B-1 bomber components were backlogged and could not be repaired because of the same reason. Although we were unable to measure specific reductions in the readiness of F-14 and B-1 aircraft as a result of this problem, mission capable rates for the B-1 in fiscal years 1998-2002 averaged approximately 55 percent, compared with the goal of 67 percent, while mission capable rates for the F-14D, during the same period, averaged 67 percent, compared with a goal of 71 percent. Additionally, the Air Force’s 2002 B-52 study concluded that six of the seven major testers used to test B-52 components need to be modified or replaced or the availability of the aircraft will be adversely affected as early as 2006. Air Force officials believe that similar problems will continue unless the service undertakes a major ATE modernization or replacement program. Since the early 1990s, DOD policies have addressed the need for DOD Has Had Limited commonality in ATE acquisition and modernization. Although the services Success in Fostering have been making some progress, efforts to comply with these policies have been slow. For example, although the Navy has developed a single Commonality family of testers to work on many of its aircraft components, after 11 years, the replacement of its obsolete testers aboard aircraft carriers and shore maintenance facilities has not been completed. In addition, strategic planning for the modernization of automatic test equipment at Navy depots has only recently been initiated. 6 See U.S. General Accounting Office, Military Aircraft: Services Need Strategies to Reduce Cannibalizations, GAO-02-86 (Washington, D.C.: Nov. 21, 2001). 7 See U.S. General Accounting Office, Navy Inventory: Parts Shortages Are Impacting Operations and Maintenance Effectiveness, GAO-01-771 (Washington, D.C.: July 31, 2001). Page 7 GAO-03-451 DOD Automated Test Equipment Modernization Historically, the Air Force has not had a service-level ATE standardization policy and has essentially pursued unique ATE solutions for each weapon system. Since individual aircraft program offices have been doing their own planning for modernization, the Air Force has given little consideration to having common ATE or testers that are interoperable with those of other services. Planning for the Air Force’s latest aircraft acquisition, the F/A-22, calls for the development of automatic test equipment that will be unique to that aircraft. In August 2002, the Air Force initiated a planning effort to determine its long-term servicewide ATE modernization needs. The Navy Has Been According to Navy reports, obsolete ATE results in higher backlogs and Slow in Fielding Its increased flying hour costs, and adversely affects aircraft readiness. The Common Tester Navy recognized years ago, and prior to the establishment of DOD’s 1994 ATE standardization policy, that its ATE was becoming obsolete. In the 1980s the Navy embarked upon an ATE standardization program to replace 25 of its testers with one standard ATE family, the Consolidated Automated Support System (CASS), to minimize unique types of testers. The Navy designed CASS to be used at maintenance activities both ashore and afloat. In 1991, the Navy began to produce CASS for the general purpose testing of equipment such as radios, radars, and electro-optics. (See fig. 2.) Page 8 GAO-03-451 DOD Automated Test Equipment Modernization Figure 2: CASS Station CASS’s replacement of 25 types of obsolete testers, in support of 2,458 weapon system components, was scheduled for completion by fiscal year 2000. However, according to Navy officials, because of budget cuts that caused delays in developing the test program sets, only 4 of the 25 have been completely replaced by CASS, and 8 test sets have been partially replaced. Navy officials told us that the completion schedule has slipped to fiscal year 2008 for aircraft carriers and shore maintenance facilities and could be much longer for aviation depots. The Navy reports that the replacement of these testers with CASS stations, when complete, will reduce the number of test-related enlisted occupational specialties from 32 to 4, thus reducing training requirements. In addition, CASS will reduce the requirement for test equipment operators aboard each aircraft carrier from 105 to 54, and at the same Page 9 GAO-03-451 DOD Automated Test Equipment Modernization time reduce space requirements for testers from 2,700 to 1,900 square feet. Spare parts needed to repair testers will be reduced from 30,000 to 3,800. According to Navy officials, however, the revised completion schedule will not allow for the timely replacement of aging ATE, and these delays will adversely affect aircraft readiness. In addition to schedule slippage, the original CASS equipment was fielded about 10 years ago, uses 15-year-old technology and, according to Navy ATE program managers, is in need of an upgrade. Accordingly, by 2006, the first production units will have reached the point where wear and obsolete components will drive supporting costs to unacceptable levels and create a need for replacement and modernization. The Navy has begun modernization planning for CASS, including upgrades through fiscal year 2014. Integrating CASS into Navy depots may further delay ATE commonality within the service. For example, a 2001 Navy report, addressing total ATE ownership costs, noted that the depots have not maximized the use of CASS because of the limited availability of capital investment funds. In addition, at one depot we found some reluctance to use CASS. This depot had four CASS stations that had never been used—two were delivered in 1999 and installed in December 2000 and February 2001, while two others delivered in 2000 were still in crates. Depot officials said that they had elected not to put the equipment on-line, as they wanted to avoid paying for overhead and maintenance, especially without the workload to justify their use. They also noted that the development of the test program sets needed to use the CASS has been slow, thereby slowing the fielding of the equipment. The Navy has only recently begun a servicewide planning effort to modernize its depot-level testers and determine how best to integrate CASS into its depot maintenance strategy. Air Force’s Approach Unlike the Navy, the Air Force has not made commonality a priority but Has Resulted in Limited has pursued unique ATE solutions for each weapon system. In addition, Commonality it has only recently initiated efforts to collect information on ATE in its inventory, including the equipment’s condition and its need for modernization or replacement. Because the Air Force has not made concerted efforts to use one system to service multiple aircraft platforms, it has not taken advantage of efficiencies and potential savings such as those expected by the Navy as a result of CASS. Although the Air Force is developing plans to modernize its ATE, and although its policy is to consider developing common testers, it does not Page 10 GAO-03-451 DOD Automated Test Equipment Modernization yet have an overall plan to guide its modernization efforts and has made limited progress in this area. Furthermore, it does not have a process in place to ensure that commonality is given adequate consideration in its ATE acquisition and modernization. The Air Force has been primarily upgrading—rather than replacing—aging ATE; leaving ATE management up to individual program managers. In most cases, it relies on contractors to provide support for ATE, leaving it vulnerable to contractors who may decide to stop supporting testers when maintaining them is no longer profitable. In early 2001, the Air Force organized the Warner Robins Air Logistics Center Automatic Test System Division to work with program offices on ATE issues. The Division has recently initiated efforts to establish a database of all contractors that are capable of supporting existing ATE to help identify emerging supportability issues. Although the office is responsible for fostering the adoption and use of common families of testers, it has no final decision-making authority regarding ATE modernizations and no control over funding decisions on these matters. Division officials told us that they work with individual project offices to encourage them to use common ATE, but individual project offices make the final decisions. In our opinion, leaving these ATE decisions to the individual Air Force project offices has led to some questionable and unnecessary expenditures. For example: • The Air Force will spend approximately $325 million to replace a tester for the F-15 with one that has been under development for almost 10 years and is already obsolete. The new tester, called the Electronic System Test Set, is not expected to be fielded until 2004. However, this electronic tester already needs an upgrade that will cost more than $24 million. Because the new tester will not be able to perform all the required tests, the Air Force will have to keep the old tester too. • The Air Force is spending over $15 million for an interim modernization of its intermediate automatic test equipment for its B-1 aircraft while, at the same time, a new tester is being developed. If the Air Force had taken the necessary steps to replace this obsolete tester in a timely manner, these duplicative costs could likely have been avoided, and overall ATE modernization costs reduced. According to an Air Force official, the program office should have begun the acquisition of a replacement tester several years ago, but funding was not available. The service is now considering acquiring a replacement tester estimated to cost $190 million. Page 11 GAO-03-451 DOD Automated Test Equipment Modernization The Air Force’s Warner Robins Air Logistics Center Automatic Test System Division is developing a strategic plan that is expected to serve as a management plan for meeting long-term ATE needs. The Division plans to develop a baseline of its current tester capabilities, address supportability and sustainability issues, and determine whether tester failures adversely affect the availability of aircraft weapon systems. In addition, it will evaluate replacement and modernization alternatives, taking into account life-cycle costs and the potential for developing common testers. The plan’s implementation is expected to take years to complete. Services’ Approaches in While most of our work focused on ATE for the current aircraft inventory, Developing Testers for we also wanted to see how the services were approaching development of Two New Aircraft Differ testers for two new aircraft, the Joint Strike Fighter and the F/A-22. We found that very different approaches are being taken in the development of ATE for these two aircraft. The JSF, for example, will have a single tester, made up almost entirely of commercial components, which will test all components on the aircraft. The F/A-22 project office has no assurance that commonality is being considered in its tester development or that DOD’s policy to minimize unique ATE development is being followed. The JSF originated in the early 1990s through the restructuring and integration of several tactical aircraft and technology initiatives already under way. The goal was to use the latest technology in a common family of aircraft to meet the future strike requirements of the services and U.S. allies. The JSF support strategy is built upon a single tester to be used by the Air Force, Navy, and Marine Corps, as well as by foreign partners, to test all avionics and weapon systems on the aircraft. The JSF tester, referred to as the LM-STAR, is made up almost entirely of commercially available components, contributing to readily available spares and less complicated upgrades. It will be used during development and after the aircraft is fielded. Vendors participating in the development of avionics and weapon system components for the aircraft are required to produce these components so that their testing can be done by the LM-STAR. A total of $99 million has been allocated for the purchase and support of 88 of these testers during the development phase. While a final decision has not been made on whether maintenance support for the aircraft will be provided by the contractor or at a military facility, the system project office is taking steps to ensure that this tester can be used regardless of where maintenance is accomplished. Page 12 GAO-03-451 DOD Automated Test Equipment Modernization By contrast, Air Force F/A-22 program officials told us that they have not made a decision as to what testers will be used to support this new aircraft, which began development in 1991. The project office has not ensured that all components for the F/A-22 can be tested with a single tester. Project officials told us that the F/A-22 is a very complex aircraft and that opportunities to take advantage of common equipment will be limited. Yet, the same contractor that is developing the F/A-22 is also involved in the JSF, which is also very advanced and complex and which uses a common family of testers. While current projections of ATE costs are not available, estimates made early in the F/A-22 development phase exceeded $1.5 billion.8 DOD Oversight Needs In 1993, the House Appropriations Committee recommended that a Strengthening DOD-wide policy be adopted requiring that the introduction of unique ATE be minimized and that DOD establish an oversight system with sufficient authority, staffing, and funding to ensure compliance. DOD established a policy requiring the services to minimize unique types of testers to reduce redundant investments and lessen long-term costs, leveraging its investments in testers across the entire DOD establishment. In 1994, DOD appointed the Navy as its Executive Agent for ATE to oversee the implementation of this policy. As part of the tasking, the Executive Agent for ATE was directed to establish a process so that programs proposing not to use the DOD-designated standard of ATE families would have to request a waiver. In accordance with the direction provided by DOD, the Executive Agent established a waiver process. According to data provided by the Executive Agent, since its inception, 30 requests for waivers were submitted for their review. Our analysis indicated that 15 of these requests resulted in waivers or concurrence. The remaining requests were never finalized, were returned to the originating office for further action, or were 8 In providing technical comments on our draft report, the F/A-22 project office reiterated that because of the complexity of this aircraft, opportunities to use common test equipment were limited. The project office indicated that designing one set of ATE to test all components could make that tester overly complex and expensive. In addition, the project office indicated that it had taken advantage of commercial testers and incorporated diagnostics into the avionics themselves. Finally, the project office indicated that the estimate for ATE of more than $1.5 billion made early in the development phase was correct but misleading since the support philosophy had changed. We continue to believe that the F/A-22 project office has not ensured that tester commonality is being considered. The project office was not able to provide information concerning the ATE used or planned for the F/A-22 or estimates of ATE costs. Furthermore, there was no evidence of Executive Agent involvement in the F/A-22 program since November 1994, and Executive Agent officials do not know whether common testers are being considered. Page 13 GAO-03-451 DOD Automated Test Equipment Modernization determined not to require waivers. According to Executive Agent officials, the Executive Agent makes recommendations concerning the waiver requests, but it does not have the authority to disapprove them. Executive Agent officials told us however, that they have no assurance that all tester acquisitions and modifications are identified or that all required waivers are requested. As a result, they may not be aware of all ATE modifications or acquisitions or they may not be made aware of such until the process is already under way and it is too late to affect any change. For example, the Air Force did not request a waiver for a $77 million modification to ATE supporting the low altitude navigation and targeting infrared for night (LANTIRN). LANTIRN is a pod system that supports the F-15, F-16, and F-14 aircraft in low-level navigation and lazing targets. In its technical comments on our draft report, however, Air Force officials indicated that owing to the nature of the LANTIRN modification, a DOD waiver was not required. We continue to believe, however, that the Executive Agent should be notified of tester modifications of this magnitude. In addition to having no assurance that all tester acquisitions and modifications are identified, Executive Agent officials told us they do not have the necessary enforcement authority or resources to effectively implement the waiver process even when they know of the planned acquisition or modification. For example, Executive Agent officials held several discussions with F/A-22 program officials, early in the development phase, concerning the use of common testers; however, there was no evidence of the Executive Agent’s involvement in F/A-22 ATE development since November 1994. Executive Agent officials do not know whether common testers are being considered. As DOD’s Executive Agent for ATE, the Navy has achieved some success in encouraging the development of common testers and in dealing with technical issues affecting all services. In September 1998, the Executive Agent for ATE reported that DOD had avoided $284 million in costs by implementing DOD’s policy and cited one example in which the Army and the Navy achieved savings of $80 million by jointly developing an electro-optics test capability. Navy officials also told us that they believe ATE planning for the Joint Strike Fighter, which calls for vendors to use standardized test equipment or equipment having commercially available components, can also be considered an accomplishment. In addition, the Executive Agent established integrated process teams to research technical issues dealing with tester commonality, such as efforts to develop open systems architecture. In this regard, DOD provided funds Page 14 GAO-03-451 DOD Automated Test Equipment Modernization to the Executive Agent during fiscal years 1995 to 1998 for its research and development efforts. Currently, the Navy is leading a joint service technology project aimed at demonstrating that the most advance technologies can be combined into a single tester. The Executive Agent also implemented a process whereby ATE modernization and acquisitions would be reviewed for compliance with DOD policy, and developed the ATE Selection Process Guide and the ATE Master Plan to aid the services in complying with DOD’s ATE policies. ATE officials, responsible for oversight of ATE, noted that their role is essential; however, its current placement in one service (the Navy) makes it difficult to ensure other services comply with DOD guidance. A report recently prepared by a joint service working group9 noted continuing problems in the implementation of DOD policy, including ATE obsolescence, delays in modernization efforts, a lack of ATE interoperability among the services, upgrading difficulties, rising support costs, proliferation of equipment that is difficult to support, and systems that are not easily deployed. The services have made limited progress in achieving DOD’s commonality Conclusions goals for ATE, as established in the early 1990s. The department does not have a joint service forum or body that can oversee the total scope of ATE acquisition and modernization and better promote ATE commonality and the sharing of information and technology across platforms and services. DOD does not have sufficient information concerning the magnitude of the services’ modernization efforts or a departmentwide approach to accomplish ATE modernization in the most cost-effective manner. Without such an approach, the department faces a very expensive and time-consuming ATE modernization effort, with the continued proliferation of unique testers. It will also have no assurance that resources are allocated in the most effective manner to exploit commonality and commercially available technology and products. A single entity within DOD—rather than in one service—may be in the best position to provide overarching oversight and coordination between the services in planning for the modernization of ATE. We believe that high-level management commitment within DOD and all the services will 9 The DOD Executive Agent for ATE established this working group to develop a jointly funded demonstration project whereby the services would develop and share ATE innovative technologies for inclusion in future ATE acquisitions and modernizations. Page 15 GAO-03-451 DOD Automated Test Equipment Modernization be needed to achieve a cultural change that fosters the development of common ATE. We recommend that the Secretary of Defense reemphasize the policy that Recommendations for common ATE be developed to the maximum extent possible. We also Executive Action recommend that the Secretary reconsider whether placing its Executive Agent for ATE in the Navy—or any single service—is the most effective way to implement the policy. Wherever the Executive Agent is placed organizationally, we recommend that the Secretary give it authority and resources to • include representatives from all of the services, with a scope to include the oversight of ATE acquisition and modifications for all weapon systems; • establish a mechanism to ensure that all ATE acquisitions and modernizations are identified in an early enough stage to be able to provide a comprehensive look at commonality and interoperability and to ensure a coordinated effort between service entities; • direct the services to draw up modernization plans for its review so it can identify opportunities to maximize commonality and technology sharing between and within the services; and • continue efforts to research technical issues dealing with tester commonality such as the development of open system architecture and other joint service applications. The Department of Defense provided written comments on a draft of this Agency Comments report, which are reprinted in their entirety in appendix II. The department and Our Evaluation also provided technical comments which we have incorporated, as appropriate, into the report. DOD concurred with our recommendations and agreed that it should reemphasize the policy that common automatic test equipment be developed to the maximum extent possible. DOD indicated that it would propose that an ATE acquisition policy statement be included in the next issuance of DOD Instruction 5000.2, “Operation of the Defense Acquisition System,” April 5, 2002. DOD also agreed to reconsider whether the placement of its Executive Agent in the Navy—or any single service—is the most effective way to implement its ATE policy. The department further concurred that an Executive Agent for ATE should have the authority and resources to direct the services to draw up modernization plans for its review to maximize commonality, interoperability, and technology sharing between the services. In this regard, DOD agreed that there should be a mechanism to ensure all automatic test equipment acquisitions and modernizations are identified in Page 16 GAO-03-451 DOD Automated Test Equipment Modernization an early enough stage in order to have a coordinated effort among service entities. Finally, DOD agreed that the Executive Agent for ATE should include representatives from all services. DOD intends to use its authority recently published in DOD Directive 5100.88, “DOD Executive Agent,” September 3, 2002, to reconsider the placement of the Executive Agent and to provide it with sufficient authority, resources, and mechanisms to carry out its responsibilities. In addition, DOD intends to include the funding for the Executive Agent as part of the Planning, Programming, Budgeting and Execution process and to identify such funding separately so that it is visible within the DOD budget. As agreed with your office, unless you publicly announce its contents earlier, we plan no further distribution of this report until 30 days from the date of this report. At that time, we will send copies of this report to interested congressional committees; the Secretaries of Defense, the Navy, the Air Force, and the Army; the Commandant, U.S. Marine Corps; and the Director, Office of Management and Budget. We will also make copies available to other interested parties on request. In addition, the report will be available at no charge on the GAO Web site at http://www.gao.gov/. If you or your staff have any questions about the report, please contact me at (757) 552-8100. Key contributors to this assignment were Ken Knouse, William Meredith, Harry Taylor, Hugh Brady, and Stefano Petrucci. Sincerely yours, Neal P. Curtin Director, Defense Capabilities and Management Page 17 GAO-03-451 DOD Automated Test Equipment Modernization Appendix I: Scope and Methodology Appendix I: Scope and Methodology We reviewed and analyzed available reports, briefings, documents, and records and interviewed officials at the Office of the Secretary of Defense and at Air Force and Navy headquarters organizations, Washington, D.C.; the Naval Air Systems Command located at Patuxent River, Maryland; Air Force Material Command and system program offices located at Wright-Patterson Air Force Base, Ohio; Warner Robins Air Logistics Center, Georgia; the North Island Naval Aviation Depot, California; the Navy’s Aviation Intermediate Maintenance Department, Oceana Master Jet Base, Virginia; and the intermediate maintenance department aboard an aircraft carrier based in San Diego, California. The Army was not included in the scope of this study because our focus was primarily on fixed-wing aircraft and because of the Army’s efforts to standardize its automatic test equipment (ATE) around a single family of testers, a situation similar to that of the Navy’s. To identify the problems that Air Force and Navy aviation (including Marine Corps) is facing with regard to ATE, we interviewed personnel responsible for policies and oversight, obtained applicable regulations and other guidance, and analyzed data provided by the services on various testers. We provided a proforma for the Air Force’s and Navy’s use in documenting their inventory of ATE, identifying obsolete testers, and providing estimates of modernization and replacement time frames and cost. The Navy’s data on ATE were provided by the central office that manages common test equipment—PMA-260, within the Naval Air Systems Command, and the Air Force’s Automatic Test System Division at Warner Robins Air Logistics Center. We also discussed obsolescence issues and ATE problems with the managers of shore-based, aircraft carrier, and depot maintenance activities. We reviewed and analyzed our prior reports and ongoing efforts, and reports of other organizations to provide a historical and contextual framework for evaluating ATE policies and issues, for documenting readiness rates of selected aircraft, and documenting the processes put in place by the Department of Defense (DOD) to oversee the services’ efforts to acquire and modernize ATE. To determine how successful DOD and the services have been in addressing the proliferation of unique testers, we held discussions with the responsible offices within each service and DOD, analyzed regulations and guidance, and reviewed studies and other documentation. We focused our work concerning this objective at the Navy office designated as DOD’s Executive Agent for Automatic Test Equipment—PMA-260 within the Naval Air Systems Command—and the Air Force’s Automatic Test System Division at Warner Robins Air Logistics Center. At these offices, which have responsibility for ATE acquisition or sustainment, modernization, Page 18 GAO-03-451 DOD Automated Test Equipment Modernization Appendix I: Scope and Methodology and oversight, we held discussions with responsible officials, obtained documentation regarding responsibilities and decisions, and reviewed files for specific ATE acquisition and modernization programs. We also obtained information from individual system program offices, for selected aircraft, located at Wright-Patterson Air Force Base and selected Navy and Air Force depots and intermediate maintenance facilities. Because we found that Air Force testers are generally unique to specific aircraft, we selected the F-15, B-1B, and B-2 for more detailed analysis, as these are considered to be front-line aircraft depended upon heavily by the Air Force to accomplish its mission. We also obtained information on ATE acquisition for two fighter aircraft currently under development: the Joint Strike Fighter and the F/A-22. We performed our review from January 2002 through March 2003 in accordance with generally accepted government auditing standards. Page 19 GAO-03-451 DOD Automated Test Equipment Modernization Appendix II: Comments from the Department Appendix II: Comments from the Department of Defense of Defense Page 20 GAO-03-451 DOD Automated Test Equipment Modernization Appendix II: Comments from the Department of Defense Page 21 GAO-03-451 DOD Automated Test Equipment Modernization Appendix II: Comments from the Department of Defense Page 22 GAO-03-451 DOD Automated Test Equipment Modernization Related GAO Products Related GAO Products Defense Inventory: Better Reporting on Spare Parts Spending Will Enhance Congressional Oversight. GAO-03-18. Washington, D.C.: October 24, 2002. Defense Inventory: Improved Industrial Base Assessments for Army War Reserve Spares Could Save Money. GAO-02-650. Washington, D.C.: July 12, 2002. Defense Inventory: Trends in Services’ Spare Parts Purchased from the Defense Logistics Agency. GAO-02-452. Washington, D.C.: April 30, 2002. Defense Logistics: Opportunities to Improve the Army’s and Navy’s Decision-Making Process for Weapons Systems Support. GAO-02-306. Washington, D.C.: February 28, 2002. Military Aircraft: Services Need Strategies to Reduce Cannibalizations. GAO-02-86. Washington, D.C.: November 21, 2001. Defense Logistics: Actions Needed to Overcome Capability Gaps in the Public Depot System. GAO-02-105. Washington, D.C.: October 12, 2001. Defense Logistics: Air Force Lacks Data to Assess Contractor Logistics Support Approaches. GAO-01-618. Washington, D.C.: September 7, 2001. Defense Inventory: Navy Spare Parts Quality Deficiency Reporting Program Needs Improvement. GAO-01-923. Washington, D.C.: August 16, 2001. Army Inventory: Parts Shortages Are Impacting Operations and Maintenance Effectiveness. GAO-01-772. Washington, D.C.: July 31, 2001. Navy Inventory: Parts Shortages Are Impacting Operations and Maintenance Effectiveness. GAO-01-771. Washington, D.C.: July 31, 2001. Air Force Inventory: Parts Shortages Are Impacting Operations and Maintenance Effectiveness. GAO-01-587. Washington, D.C.: June 27, 2001. Defense Inventory: Information on the Use of Spare Parts Funding Is Lacking. GAO-01-472. Washington, D.C.: June 11, 2001. Defense Inventory: Approach for Deciding Whether to Retain or Dispose of Items Needs Improvement. GAO-01-475. Washington, D.C.: May 25, 2001. Page 23 GAO-03-451 DOD Automated Test Equipment Modernization Related GAO Products Military Aircraft: Cannibalizations Adversely Affect Personnel and Maintenance. GAO-01-93T. Washington, D.C.: May 22, 2001. Defense Inventory: Army War Reserve Spare Parts Requirements Are Uncertain. GAO-01-425. Washington, D.C.: May 10, 2001. (350137) Page 24 GAO-03-451 DOD Automated Test Equipment Modernization The General Accounting Office, the audit, evaluation and investigative arm of GAO’s Mission Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO’s commitment to good government is reflected in its core values of accountability, integrity, and reliability. The fastest and easiest way to obtain copies of GAO documents at no cost is Obtaining Copies of through the Internet. GAO’s Web site (www.gao.gov) contains abstracts and full- GAO Reports and text files of current reports and testimony and an expanding archive of older products. The Web site features a search engine to help you locate documents Testimony using key words and phrases. You can print these documents in their entirety, including charts and other graphics. Each day, GAO issues a list of newly released reports, testimony, and correspondence. GAO posts this list, known as “Today’s Reports,” on its Web site daily. The list contains links to the full-text document files. To have GAO e-mail this list to you every afternoon, go to www.gao.gov and select “Subscribe to daily E-mail alert for newly released products” under the GAO Reports heading. Order by Mail or Phone The first copy of each printed report is free. Additional copies are $2 each. A check or money order should be made out to the Superintendent of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a single address are discounted 25 percent. Orders should be sent to: U.S. General Accounting Office 441 G Street NW, Room LM Washington, D.C. 20548 To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202) 512-6061 Contact: To Report Fraud, Web site: www.gao.gov/fraudnet/fraudnet.htm Waste, and Abuse in E-mail: firstname.lastname@example.org Federal Programs Automated answering system: (800) 424-5454 or (202) 512-7470 Jeff Nelligan, managing director, NelliganJ@gao.gov (202) 512-4800 Public Affairs U.S. General Accounting Office, 441 G Street NW, Room 7149 Washington, D.C. 20548
Military Readiness: DOD Needs to Better Manage Automatic Test Equipment Modernization
Published by the Government Accountability Office on 2003-03-31.
Below is a raw (and likely hideous) rendition of the original report. (PDF)