United States General Accounting Office GAO Report to the Chairman, Subcommittee on Transportation and Related Agencies, Committee on Appropriations, House of Representatives March 1997 AIR TRAFFIC CONTROL Status of FAA’s Standard Terminal Automation Replacement System Project GAO/RCED-97-51 United States GAO General Accounting Office Washington, D.C. 20548 Resources, Community, and Economic Development Division B-275244 March 5, 1997 The Honorable Frank R. Wolf Chairman, Subcommittee on Transportation and Related Agencies Committee on Appropriations House of Representatives Dear Mr. Chairman: The Federal Aviation Administration (FAA) oversees the largest, busiest, and most complex air traffic control system in the world. However, components of the system are aging and are difficult and costly to maintain. FAA projects that it cannot meet projected traffic increases and make required safety and efficiency enhancements without replacing equipment. Since the early 1980s, FAA’s modernization efforts have experienced lengthy schedule delays and substantial cost overruns. Because of such problems, in 1994, FAA restructured its acquisition of the Advanced Automation System—the long-time centerpiece of its air traffic control modernization program—into more manageable segments. One of these segments is called the Standard Terminal Automation Replacement System (STARS) project. This project is expected to replace 15- to 25-year-old computers and related equipment used at FAA facilities that track aircraft in the airspace surrounding airports.1 In September 1996, FAA contracted with Raytheon Corporation to develop, produce, and install STARS. Given FAA’s past schedule and cost problems and the significance of the STARS project, you asked us to examine FAA’s acquisition planning to date. Specifically, you asked us to determine to what extent (1) the schedule estimate for STARS is attainable and (2) cost estimates to make STARS operational are likely to change. The STARS schedule, which calls for implementation at 171 air traffic Results in Brief control facilities between December 1998 and February 2005, is attainable only if FAA is successful in its efforts to mitigate certain risks. Specifically, FAA will need to (1) obtain commitment by key stakeholders to the STARS 1 STARS will also replace equipment at Department of Defense facilities across the country. Schedule and cost information relating to Defense facilities is not included in this report. Page 1 GAO/RCED-97-51 Air Traffic Control B-275244 schedule, (2) resolve schedule conflicts between STARS and other modernization efforts, and (3) overcome difficulties in developing system software that could delay implementing STARS. FAA is aware that these issues pose a risk for STARS and has begun several risk mitigation initiatives. While such actions are encouraging, it is too early to tell how effective they will be. FAA’s cost estimate for STARS has the potential to increase. The agency’s total cost estimate for STARS is $2.23 billion.2 FAA approved this estimate in January 1996. However, a September 1996 analysis by agency officials pointed to potential cost increases that could drive the total cost estimate to as much as $2.76 billion. This possible increase is attributable to expected higher costs for operating and maintaining STARS equipment. FAA officials are continuing to revise the STARS cost estimate and now believe that cost increases may be significantly lower. At this time, however, the agency could not provide us with an updated estimate. STARS is designed to replace FAA’s automated radar terminal system, which Background is composed of 15-to 25-year-old controller workstations and supporting computer systems. According to FAA, this system is prone to failures, is maintenance intensive, and requires long repair times. The system also has capacity constraints that restrict the agency from making required safety and efficiency enhancements. Automated radar terminal systems are located at 180 Terminal Radar Approach Control facilities (TRACON) and allow FAA controllers to separate and sequence aircraft near airports.3 STARS equipment (see fig. 1) is also expected to provide the platform needed to make system enhancements that would increase the level of air traffic control automation and improve weather display, surveillance, and communications. In addition, STARS is expected to permit FAA to consolidate some TRACONs and replace all Digital Bright Radar Indicator Tower Equipment systems.4 2 The total cost includes facilities and equipment and operations and maintenance costs. For this report, all dollars are expressed in current-year dollars, unless otherwise noted, because they are a better indication of the dollar amount that the Congress may have to appropriate. 3 Because FAA plans to consolidate some of its 180 TRACONs, it plans to buy 171 STARSs. 4 Digital Bright Radar Indicator Tower Equipment systems display aircraft position data to controllers in towers. These systems enable controllers to monitor traffic in bright sunlight. Page 2 GAO/RCED-97-51 Air Traffic Control B-275244 Figure 1: STARS Workstation Source: FAA. In September 1996, FAA signed a contract with Raytheon Corporation and, as mentioned, currently plans to acquire as many as 171 STARSs. In producing STARS, Raytheon intends to rely fully on commercially available hardware and, to a large extent, on commercially available software. Some original software development will still be required. In August 1996, the contractor projected that 124,000 new lines of software code will need development to meet FAA’s requirements. This estimate was revised in December 1996 to 140,000 new lines of code. STARS is an outgrowth of the troubled Advanced Automation System acquisition. As originally designed, the terminal segment of this system, Page 3 GAO/RCED-97-51 Air Traffic Control B-275244 known as the Terminal Advanced Automation System, would provide controllers in TRACONs with new workstations and supporting computer systems. However, in June 1994, the FAA Administrator ordered a major restructuring of the acquisition to solve long-standing schedule and cost problems. These schedule delays were up to 8 years behind the original schedule, and estimated costs had increased to $7.6 billion from the original $2.5 billion estimated in 1983. Specifically, regarding terminal modernization, the Administrator canceled the Terminal Advanced Automation System and expanded the STARS project to include all terminal facilities. In April 1996, FAA established a new acquisition management system, as directed by the Congress. Included in this system is the concept of life-cycle management, which is intended to be a more comprehensive, disciplined full-cost approach to managing the acquisition cycle, from analysis of mission needs and alternative investments through system development, implementation, operation, and, ultimately, disposal. Under this new system, decisions related to resource allocation (mission and investment) are made by FAA’s Joint Resources Council, which is composed of associate administrators for operations and acquisition and other key executives. Decisions associated with program planning and implementation are made within Integrated Product Teams (IPT). IPTs are responsible for bringing together all essential elements of program implementation, including scheduling, allocation of funding, and the roles and responsibilities of stakeholders. To ensure successful program implementation, the acquisition management system dictates that these issues be resolved before contracts are awarded. IPTs also generate schedule and cost baselines, which the Joint Resources Council authorizes the teams to operate under. Team members include representatives from FAA units responsible for operating and maintaining air traffic control equipment and other stakeholders in the acquisition process. To achieve the implementation schedule approved by the Joint Resources STARS Schedule Is Council in January 1996,5 FAA will have to obtain commitment from key Attainable Only If FAA stakeholders, resolve scheduling conflicts between STARS and other Is Successful in Its terminal modernization efforts, and overcome difficulties in developing the system. FAA is aware that these issues pose a risk for STARS and has Efforts to Mitigate begun several risk mitigation initiatives. While such actions are Risks encouraging, it is too early to tell how effective they will be. 5 The schedule was based on a December 1994 FAA study, which revalidated terminal requirements. Page 4 GAO/RCED-97-51 Air Traffic Control B-275244 Milestones Call for Initial FAA’sschedule for developing and implementing STARS by its January 1996 Implementation by approved baseline is shown in table 1. December 1998 Table 1: FAA’s Schedule for Installing STARS Date Activity November 1997 Complete software development for initial STARS.a April-September 1998 Test initial STARS software. September 1998 Complete software development for full STARS.b December 1998 Operate initial STARS at first site—Boston, Mass. April-July 1999 Test full STARS software. January 2000 Have full STARS ready for installation. February 2005 Operate full STARS at last site— Columbus, Ga. a FAA expects that the initial STARS software will provide the same functions as the current automated radar terminal systems. b FAA expects that the full STARS software will enhance air traffic controllers’ abilities to move aircraft more safely and efficiently. For example, this software is expected to allow controllers to space aircraft more precisely during landings and departures on converging runways. Figure 2 shows FAA’s plans for ordering, delivering, and operating STARS. FAA intends to begin operating STARS at only three TRACONs before fiscal year 2000. Operation increases after this time, with FAA expecting to operate 55 additional systems in fiscal year 2002. Page 5 GAO/RCED-97-51 Air Traffic Control B-275244 Figure 2: Number of Systems Ordered, Delivered, and Operating, 1997 Through 2005 STARS 60 58 57 55 55 50 45 44 43 41 40 40 39 38 35 30 25 20 16 15 15 13 11 10 7 7 6 6 5 5 4 3 3 2 0 0 0 0 0 0 red d g ere tin de era liv Or De Op 1997 1998 1999 2000 2001 2002 2003 2004 2005 Fiscal Year Ordered Delivered Operating Note: Three of the systems operating through fiscal year 1999 are for training and computer support at FAA’s Academy and Technical Center. Source: FAA. Lack of Commitment From FAA has yet to obtain commitment from all key stakeholders responsible Key Stakeholders for ensuring that STARS equipment is properly installed. FAA’s new acquisition management system stresses that IPTs need to reach agreement before contracts are awarded. Such agreement is necessary to ensure that all stakeholders’ roles are defined and agreed upon, facilities are ready to receive STARS, and all other equipment necessary for the operation of STARS Page 6 GAO/RCED-97-51 Air Traffic Control B-275244 is in place. In the past, poor coordination among key stakeholders has caused schedule delays in other modernization projects at FAA.6 The IPT for STARS has yet to obtain commitment to the STARS schedule from the entire Airway Facilities Service—a key stakeholder. Located in headquarters and regions, maintenance technicians who work for the Airway Facilities Service are responsible for installing and maintaining air traffic control equipment. FAA’s current schedule anticipates that STARS will be installed at most sites using a turnkey concept whereby the contractor, not FAA employees, will install the equipment. This concept presumes that a significant level of regional resources will still be required to support and oversee contractor installation. IPT officials told us that while Airway Facilities Service officials at headquarters have committed to the turnkey concept, regions’ commitment is incomplete. IPT and Airway Facilities Service officials told us that a process has been established to ensure regions’ understanding and obtain their commitment. As part of this process, the IPT has begun regional briefings and has formed implementation teams to gain regions’ commitment on turnkey issues. In addition, the IPT has yet to obtain commitment to the STARS schedule from the Professional Airways Systems Specialists—the technicians’ union. Top union officials told us that, as of late February 1997, they have not been briefed on the STARS turnkey concept and have not agreed as to how it will be implemented. The union is concerned that the turnkey installation may jeopardize the job security of its members. IPT officials said that while union representatives have been involved in reviewing vendors’ proposals for STARS, the union has not been briefed on the specifics of STARS deployment. Although FAA’s Acquisition Management System stresses that all key program implementation issues be resolved before contracts are awarded, the IPT believed that it could obtain the union’s commitment at a later date. As required by the union’s collective bargaining agreement, in January 1997, the IPT initiated actions to brief the union and obtain its commitment. Potential Scheduling FAA’s schedule for STARS can be jeopardized by scheduling conflicts with Conflicts Between STARS other modernization efforts. For example, each year, various TRACONs are and Other Modernization scheduled to be renovated or replaced. If STARS equipment is delivered during this time, installation could be delayed. Currently, the IPT is unsure Efforts of the number of these potential conflicts. In September 1996, the IPT 6 Aviation Acquisition: A Comprehensive Strategy Is Needed for Cultural Change at FAA (GAO/RCED-96-159, Aug. 22, 1996). Page 7 GAO/RCED-97-51 Air Traffic Control B-275244 identified 12 potential scheduling conflicts at the first 45 STARS sites. One month later, the number of conflicts was reduced to four, but the team did not provide us with an explanation for this decrease. We believe that the number of potential conflicts will not be known until the IPT ascertains the readiness of each facility to receive and install STARS equipment. The IPT plans to start conducting site reviews in 1997. Another potential scheduling conflict involves terminal surveillance radars, which track aircraft position and use analog or digital processing and communications to transmit the information to TRACONs. Many existing surveillance radars are not digital, but STARS requires digital processing and communications. FAA plans to replace nondigital Airport Surveillance Radar-7s (ASR-7) with new digital ASR-11s. The agency has not decided yet whether to replace other nondigital radar, ASR-8s, or to digitize them. In January 1997, FAA was concerned that 47 of 98 ASR-7s and –8s might not be upgraded in time to meet the STARS schedule. FAA officials told us that, as of late February, they had reduced the number of potential conflicts from 47 to 10 through efforts to coordinate the STARS and digital radar schedules. According to an IPT official, if digital radar does not provide coverage for a TRACON’s entire airspace, FAA may have to delay STARS or reorder the sequence of TRACONs receiving STARS. FAA officials told us that they are taking actions to identify and resolve potential scheduling conflicts. The IPT has developed project guides for the FAA regions receiving STARS. These guides identify possible scheduling conflicts with other modernization efforts. Also, Airway Facilities Service officials told us that as a result of a recent reassessment in December 1996 of the schedule for the first 39 STARSs, FAA was able to avoid potential conflicts by repositioning the order in which TRACONs received STARS. Finally, the Airway Facilities Service is developing a database to assist the IPT in maintaining current planning information. Potential Difficulties in Although STARS depends on the use of commercial off-the-shelf computer Developing STARS hardware and a significant amount of commercially available software, FAA Software and Raytheon have numerous tasks to accomplish before system development is completed. However, the nature and extent of these tasks are not completely known, and such development inevitably poses continual managerial and technical challenges. As noted in table 1, FAA’s schedule calls for software development to proceed in two phases. For the initial phase, the agency expects to complete software testing in September 1998, about 2 years from the time when the contract was Page 8 GAO/RCED-97-51 Air Traffic Control B-275244 awarded. For the second phase, the agency expects to complete testing of the full STARS software in July 1999. As an example of the challenge that software development poses for FAA, as recently as December 1996, FAA and Raytheon were discussing (1) how the system would provide specific functions and (2) whether certain functions would be needed, and if so, whether the functions would be included in the equipment with initial- or full-system capability. According to Raytheon officials, these discussions ended with FAA and Raytheon coming to closure on all of the 28 issues needing resolution. As a result, some 16,000 lines of additional software code—beyond the planned 124,000 lines of new code—must be written. Of the 140,000 lines of code, about 138,000 are for flight data processing, training, and maintenance functions, and 2,000 are to fulfill safety requirements, such as warning controllers when aircraft are not maintaining proper separation or minimum safe altitudes. Raytheon officials believe the additional code development will not affect their ability to meet the original milestones. All new code will have to be tested in conjunction with the nearly 840,000 lines of existing STARS software code. If potential difficulties in developing and testing the system are realized, initial implementation of STARS—particularly at the three TRACONs targeted for operation before fiscal year 2000—will likely be delayed. FAA’s life-cycle cost baseline has the potential to increase—from Cost Estimates for $2.23 billion, the level approved by the Joint Resources Council in STARS Have the January 1996,7 to as much as $2.76 billion.8 This possible increase is Potential to Increase attributable to expected higher costs for operating and maintaining STARS equipment. FAA expects the estimate for facilities and equipment costs to remain stable for the immediate future.9 FAA’s January 1996 facilities and equipment cost baseline is $940 million. During 1996, this baseline was reviewed by the IPT. Through September 1996, the IPT was estimating that the baseline could increase to $1.18 billion. At that time, the IPT (1) estimated higher expected costs for 7 This estimate was based on the December 1994 FAA study that revalidated associated costs. The reliability in estimating costs is discussed in our report Air Traffic Control: Improved Cost Information Needed to Make Billion-Dollar Modernization Investment Decisions (GAO/AIMD-97-20, Jan. 22, 1997). 8 The life-cycle cost estimates do not include the costs of “technical refreshment”—planned periodic updating of a system’s technological capabilities. We excluded these costs because of a lack of comparable data between the January 1996 baseline and the latest analysis dated September 1996. 9 The facilities and equipment appropriation account funds FAA’s efforts to acquire new equipment. FAA’s operations appropriation account funds FAA’s efforts to maintain and support equipment. Page 9 GAO/RCED-97-51 Air Traffic Control B-275244 software development; (2) estimated higher expected implementation, technical support, and maintenance costs because of the addition of necessary equipment; and (3) included costs for communications because the baseline estimate overlooked them. In December 1996, the IPT assessed the STARS costs on the basis of the signed contract with Raytheon. As a result, the IPT determined, that while some cost elements will increase, other elements will decrease. Specifically, significantly lower costs for hardware—key components were $40,000 less per unit than what FAA had estimated—will enable the STARS project for the present time to stay within the original baseline. Table 2 shows the differences in cost elements between the original cost baseline and the IPT’s December 1996 assessment. Page 10 GAO/RCED-97-51 Air Traffic Control B-275244 Table 2: STARS Facilities and Equipment Baseline and Potential Dollars in millions Changes January December Reasons for Facilities and equipment 1996 1996 differences in cost element baseline assessment cost estimates Development $25.4 $80.0 FAA determined additional lines of software code needed development. The cost of development includes program management, testing, and travel. Hardware 506.0 314.0 FAA added equipment but reduced its unit costs because of the contractor’s choice of less expensive equipment and quantity discounts. Implementation 164.8 219.5 FAA expects to install more equipment than originally planned and better identified sites’ needs. Technical support 129.1 132.4 FAA expects to install more equipment than originally planned and better identified program office’s needs. Planned product 87.7 110.3 FAA is currently refining improvements requirements. The cost is the best estimate to date. Maintenance—first yeara 27.2 30.0 FAA expects to install more equipment than originally planned. Communications 0 54.0 FAA omitted these costs from the original estimate. Total $940.2 $940.2 a FAA policy states that the first year of maintenance for equipment is paid for from the facilities and equipment account. Thereafter, all maintenance funds are paid from the operations account. Source: FAA. FAA’s January 1996 operations cost baseline is $1.29 billion. However, based on a September 1996 analysis,10 FAA staff identified a potential $529 million increase that could revise the baseline to $1.82 billion. FAA officials told us that this increase occurred, in part, because the agency overlooked maintenance costs in the initial estimates. Also, the officials 10 This analysis was prepared by FAA’s Program Analysis and Operations Research staff. Page 11 GAO/RCED-97-51 Air Traffic Control B-275244 attributed the increase to FAA’s deploying more STARS equipment than originally planned. IPTofficials told us that on the basis of more current information from the contractor, operations and maintenance costs are expected to be significantly closer to the $1.29 billion baseline estimate than the $1.82 billion figure. The officials could not, however, provide us with an updated cost estimate or detailed support for their views. The IPT officials told us that they are reviewing the latest cost estimates and expect to brief the Joint Resources Council on any potential changes to the baseline in March 1997. Separate and distinct from STARS life-cycle costs are two additional costs that FAA will incur to make STARS operational. First, FAA will have to prepare the TRACONs for the delivery of STARS equipment. FAA officials estimate that the agency will incur at least $18 million in costs to get the first 46 TRACONs and related facilities ready to accept the STARS equipment. Roughly half of this amount is for asbestos removal; the balance is for power upgrades and building improvements. FAA has yet to develop estimates for readying the remaining sites. Second, FAA will incur costs for upgrading radars. FAA plans to modernize the existing analog ASR-8 radars that provide data to its TRACONs. Because the implementation of STARS is approaching, FAA is faced with an immediate decision between digitizing these existing analog radars or replacing them with new digital radars. FAA officials estimate that the 20-year life-cycle costs for modifying and digitizing all the ASR-8s will be $459 million and for replacing them will be $474 million. According to FAA officials, the estimated cost difference between digitizing existing radars and buying new radars is minimal because of the higher costs of maintaining older analog equipment. The agency is continuing to refine these cost estimates, and it expects to decide later this year on which option to select. We provided the Department of Transportation with a draft of this report Agency Comments for its review and comment. We met with FAA officials, including the IPT leader for Terminal Air Traffic Systems Development; the Program Director for National Airspace System Transition and Implementation; and representatives of FAA’s Air Traffic and Airway Facilities Services. FAA was concerned about our use of the $1.82 billion estimate for operations and maintenance costs. The estimate came from a September 1996 study done by FAA’s Program Analysis and Operations Research staff. FAA told us that this estimate was preliminary and should not be reported as a basis for Page 12 GAO/RCED-97-51 Air Traffic Control B-275244 evaluating the STARS project. While FAA acknowledged that there may be some cost growth in the STARS project, it did not anticipate growth as large as we reported. We continue to include the September 1996 estimate in this report. This estimate was developed by experienced cost analysts, including a member of the STARS IPT, and was the only documented estimate available since the official baseline was approved in January 1996. Furthermore, FAA could not provide us with a more current estimate or detailed support for its views on why the September 1996 analysis may have overstated the cost estimate for operations and maintenance. FAA also expressed concern about the way the draft report characterized the extent to which key stakeholders were committed to the implementation schedule, which relies heavily on the use of the turnkey concept. We revised the report to recognize that (1) while regions’ commitment is incomplete, Airway Facilities Service officials at headquarters have committed to the turnkey concept and (2) FAA has established a process, including the formation of implementation teams to ensure regions’ understanding and obtain their commitment on turnkey issues. However, because the turnkey concept will affect regional resources and employees’ responsibilities, FAA agreed that the potential lack of regions’ commitment is a risk that must be mitigated throughout the implementation of STARS. To obtain information for this report, we interviewed officials at FAA Scope and headquarters, its New England Regional Office in Burlington, Methodology Massachusetts, its New York Regional Office in Jamaica, New York, and its William J. Hughes Technical Center in Pomona, New Jersey. We reviewed agency documentation on current schedule and life-cycle costs for STARS. We reviewed guidelines pertaining to system acquisition, compared FAA’s actions to the guidance, and identified key issues that could affect the success of the STARS project. To identify any labor issues that could affect the scheduled deployment, we interviewed union officials with the Professional Airways Systems Specialists. We conducted our review from July 1996 through January 1997 in accordance with generally accepted government auditing standards. However, we did not assess the reliability of the process used to generate cost information. We are sending copies of this report to the Secretary of Transportation, the Administrator of FAA, and other interested parties. We will also make Page 13 GAO/RCED-97-51 Air Traffic Control B-275244 copies available to others on request. Please call me at (202) 512-3650 if you or your staff have any questions about this report. Major contributors to this report are listed in appendix I. Sincerely yours, Gerald L. Dillingham Associate Director, Transportation Issues Page 14 GAO/RCED-97-51 Air Traffic Control Page 15 GAO/RCED-97-51 Air Traffic Control Appendix I Major Contributors to This Report John H. Anderson, Jr. Gregory P. Carroll Robert E. Levin Peter G. Maristch John T. Noto (341497) Page 16 GAO/RCED-97-51 Air Traffic Control Ordering Information The first copy of each GAO report and testimony is free. Additional copies are $2 each. Orders should be sent to the following address, accompanied by a check or money order made out to the Superintendent of Documents, when necessary. VISA and MasterCard credit cards are accepted, also. Orders for 100 or more copies to be mailed to a single address are discounted 25 percent. Orders by mail: U.S. General Accounting Office P.O. Box 6015 Gaithersburg, MD 20884-6015 or visit: Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders may also be placed by calling (202) 512-6000 or by using fax number (301) 258-4066, or TDD (301) 413-0006. Each day, GAO issues a list of newly available reports and testimony. To receive facsimile copies of the daily list or any list from the past 30 days, please call (202) 512-6000 using a touchtone phone. A recorded menu will provide information on how to obtain these lists. For information on how to access GAO reports on the INTERNET, send an e-mail message with "info" in the body to: email@example.com or visit GAO’s World Wide Web Home Page at: http://www.gao.gov PRINTED ON RECYCLED PAPER United States Bulk Rate General Accounting Office Postage & Fees Paid Washington, D.C. 20548-0001 GAO Permit No. G100 Official Business Penalty for Private Use $300 Address Correction Requested
Air Traffic Control: Status of FAA's Standard Terminal Automation Replacement System Project
Published by the Government Accountability Office on 1997-03-05.
Below is a raw (and likely hideous) rendition of the original report. (PDF)