United States General Accounting Office Report to Congressional Requesters GAO .i; June 1990 ‘r AIR TRAFFIC CONTROL ,:.. Smaller Terminal Systems. Lapamy Requirements Need to: l3e Defined ?: i.. ,: .. I, ( United States GAO General Accounting Office Washington, D.C. 20548 Information Management and Technology Division B-239204 June 25,199O The Honorable Frank R. Lautenberg Chairman, Subcommittee on Transportation and Related Agencies Committee on Appropriations United States Senate The Honorable William Lehman Chairman, Subcommittee on Transportation and Related Agencies Committee on Appropriations House of Representatives In response to your December 15, 1989, request, we reviewed the Fed- eral Aviation Administration’s (FAA) efforts to provide automated safety enhancements for air traffic controllers at smaller Terminal Radar Approach Control (TRACOK)facilities. These TRACONS control aircraft arriving at or departing from airports such as Pensacola, Florida, and Binghamton, New York. To implement the enhancements and upgrade existing automation systems at these TRACONs, FAA is deploying the Auto- mated Radar Terminal System (ARTS) IIA. As agreed with your offices, our objectives were to (1) determine the extent of and reasons for delays in developing, testing, and deploying the ARTS IIA hardware and software, and (2) determine if FAA has adequately identified current and future computer capacity requirements for smaller TRACONS. A detailed explanation of our objectives, scope, and methodology is contained in appendix I. Installation of safety enhancements at smaller TRACONS has been delayed Results in Brief about 3 years because FAA did not properly manage the program. FAA awarded a production contract before the system was developed and continued to change system requirements after the contract was awarded. Also, E’AAdid not require the contractor to perform integrated tests to determine if various portions of the system worked together cor- rectly. As a result, the system design had to be modified during produc- tion, which led to delays in deployment. FAA also does not have a computer capacity and performance manage- ment program for its smaller TKACONS. Such a program measures current computer utilization and is used to predict future capacity requirements. As we reported last year, the lack of a capacity management program at Page 1 GA0/IMTEC90-50AirTrafficC~ntm1 a-239204 tracking, the safety features of conflict alert and minimum safe altitude warning, and add a controller training function. Aircraft tracking would perform position predictions and display aircraft ground speed to con- trollers. Conflict alert would provide visual and sound alerts to the con- troller when two controlled2 aircraft are in or are projected to come into close proximity to each other within the next 40 seconds. Minimum safe altitude warning would provide visual and sound alerts to controllers when an aircraft is flying or is projected to fly too low. The controller training function was to simulate live traffic situations without affecting air traffic control operations at other workstations. FAA imple- mented these enhancements at its larger TRACONS in several phases between 1977 and 1989. In July 1985 FAA awarded the ARTS IIA production contract to Burroughs Corporation, now UNISYS, at an estimated cost of $35.7 million. This contract, now valued at $45.4 million, is to provide the above enhance- ments to all 119 ARTS IIA TFNZONS through new hardware and software by August 1991. In addition to these enhancements, FAA plans to award a contract late in fiscal year 1990 to improve ARTS IIA reliability and increase system capacity. This planned contract is designed to provide additional con- troller workstations, replace obsolete tape drives with disk drives, and provide uninterruptable power systems for several ARTS IIA locations. In addition, the planned contract contains an option to increase ARTS IIA capacity and provide another safety enhancement-the Mode C Intruder.” This option is designed to provide 145 new computers to sig- nificantly increase processing and memory capacity. This contract is part of FAA’s Interim Support Plan designed to sustain the air traffic control system until it is replaced by the Advanced Automation System in the mid-to-late 1990s. ‘Controlled aircraft are followed from takeoff to landing by controllers, must carry electromc equip ment mcludrng radios to communicate with controllers and a transponder to provide altitude and identity informatmn, and must file flight plans that detail their proposed journey. IJncontrolled air- craft, although sometnnes monitored by the system, do not need to carry electronic equipment unless they enter contwlled airspace. and arc not required to file fhght plans; however, they must follow well-established FAA rules regarding where they can fly, 3The Mode C Intruder is a wammg to controllers that indicates that the distance between a controlled and uncontrolled aircraft wll berome hazardous within the next 40 seconds.This warning differs from conflict alert in that It will monitor one controlled and one uncontrolled Mode C-equipped au- craft, rather than two controlled ainxiift Page 3 GAO/IMTEG90-SO Air Traffic Control -” a-239204 requirements specifying that the ARTS IIA process 102 tracks, support up to 11 displays, and use the current operational software version, as well as provide tracking, the safety enhancements, and the training capability. FL awarded the production contract without requiring the contractor to demonstrate that it could meet the new system requirements. In fact, the contractor’s proposal questioned whether the system could provide suf- ficient processing capacity to support an 1 l-display system. According to the program manager, FAA recognized at the time the production con- tract was awarded that additional modifications might be necessary to meet all AR% IIA operational requirements. The Office of Management and Budget’s procurement policy, contained in its Circular A-109, requires production decisions to be based on evi- dence that the system meets requirements before committing to produc- tion. In this instance, FAA'S decision to award the production contract before development was complete contributed to ARTS IIA implementa- tion delays. Software Baseline Not While new software was being developed under the ARTS IIA develop- Stabilized for Production ment and production contracts, FAA continued to modify the existing ARTS II software operating at smaller TRACONS. FAA did this to correct Contract inefficiencies and provide additional features, some of which were already operational at a few sites. As these improvements were com- pleted, FAA released the new software to sites. Although it is important to adequately specify the functions software should perform before committing to a production decision, FAA continued to modify the software requirements throughout the life of the production contract. This contributed to implementation delays. For example, FAA released new software to ARTS II sites 4 months after the ARTS IIA contract award and then changed the contract to reflect these changes. Then, in 1986, FAA required that seven additional software features be included in the ARTS IIA contract. By late 1987, these seven additional features had been incorporated and UNISE shipped the ARTS IIA to the FAA Technical Center for testing. The results of this test identified over 200 problems, 65 of which FAA classi- fied as critical. The most significant problem was the system’s inability to meet the 102- track requirement. Contractor personnel told us that the system failed to meet this requirement because the features added to the system baseline used significant amounts of computer capacity. As a peg.25 GAO/IMTECSO-50AirTraffcControl 8239204 planning, acquiring, and using computer resources. Last year we reported that FAA lacked a computer capacity and performance manage- ment program for its large TRACONS.~ Consequently, FAA did not recognize capacity shortfalls until controllers’ ability to maintain safe separation of aircraft was impaired. Further, adequately defining capacity requirements and establishing a computer capacity and performance management program are neces- sary to ensure maximum use of existing resources and adequate capacity for growth. Capacity expansion should be based on an analysis of current system performance, new requirements, and projections for future growth. ARTS IIA May Not Meet Although we reported in 1983” that FAA should implement a computer Small TRACON performance management function to measure terminal systems’ effi- ciency and effectiveness, FAA lacks such a program for ARTS IIA systems. Requirements PAAofficials stated that a program has not been established because AHTS II systems did not encounter capacity shortfalls. The lack of such a program constitutes a material weakness under the Federal Managers’ Financial Integrity Act (31 USC. 3512 (b) and (c)). This weakness is regarded as material given the fact that a capacity management pro- gram is an integral part of effective computer operations, which is cen- tral to FAA's fulfilling its air traffic control mission. Although FAA increased the track requirement for the ARTS IIA system to 102 tracks prior to awarding the production contract, it did not analyze current and future work loads to determine if this requirement would meet actual needs. In fact, FAA did not perform any meaningful analysis to derive this stated track requirement. As a result, FAA has no assur- ance that the work loads at ARTS IIA facilities will not exceed 102 tracks. Additionally, ARTS IIA tests conducted in 1989 indicate that the system uses 76 to 93 percent of total processing capacity when operating under maximum specified work loads. This indicates that ARTS IIA systems may approach computer capacity limitations under heavier work loads. However, because FAA does not collect data on current or projected instantaneous traffic levels, it does not know whether ARTS IIA sites are ‘AK Traffic Control: Computer Capacity Shortfalls May lmpaw Flight Safety (GAO/IMTEC-89-63. .luly ti, 1989). “FAA‘s Plans to Improve The Air TrafRc, Control System: A Step In The Right Direction But Improve- ments And lkttcr Cuordinatlon Are Kwdcd (GAO/AFMD-a-34, Feb. 16, 1983). computer capacity on the basis of inadequate analysis of current usage, future traffic levels, and future work loads, including Mode C Intruder requirements. Basing an acquisition on inadequately defined require- ments and incomplete performance data increases the risk that the expanded systems will be inadequate to meet operational requirements. Experience gained in implementing the ARTSIIA project can be used to Recommendations prevent similar problems from occurring on future projects. We recom- mend that the Secretary of Transportation direct the Administrator, FAA, to improve management of future TRACON automation projects by awarding production contracts only after development is complete, con- trolling changes to operational software during system development and production, and requiring contractors to perform integrated testing. In addition, we recommend that the Secretary of Transportation direct the Administrator to implement a computer capacity and performance management program for ARTS IIA systems. This program should include an analysis of current system performance and future work loads, including predicted traffic levels and additional software functions to determine system requirements, as well as the potential performance of the enhanced system. We also recommend that the Secretary direct the Administrator to report the lack of a computer capacity and performance management program as a material control weakness under the Federal Managers’ Financial Integrity Act until a program has been implemented. Finally, we recommend that the Secretary direct the Administrator to delay exercising the expansion contract option to procure additional computers until a computer capacity and performance management pro- gram is implemented and future capacity requirements are adequately defined. We obtained official oral comments from the Department of Transporta- Agency - - Comments _ and tion and FAA officials on a draft of this report. They stated that FAA Our Evaluation awarded the production contract in July 1985 after considering the tech- nical, schedule, and cost risks and found these risks to be acceptable based on the information available at that time. Department and FAA officials agreed that software requirements were modified after contract award and that this delayed the implementation of the safety enhance- ments. However, they commented that they took steps to control the Page 9 GAO/IMTEC90-50 Air Traffic Control Page 11 GA0/1MTEC9060 Air Traffic Control Appendix I Objectives, Scope, and Methodology Our review was conducted from May 1989 to April 1990. The views of agency officials were sought during the course of our work and their comments have been incorporated where appropriate. In addition, at the completion of our review, we discussed the report’s key facts, conclu- sions, and recommendations with FAA officials. Finally, we obtained formal oral comments from Department of Transportation and FAA offi- cials on a draft of this report. These comments and our analysis are also included in this report. We conducted our review in accordance with generally accepted government auditing standards. Page 13 GA0/1MTECSO-50 Air Traffic Control Requests for copies of GAO reports should be sent to: U.S. General Accounting Office Post Office Box 6015 Gaithersburg, Maryland 20877 Telephone 202-275-6241 The fast five copies of each report are free. Additional copies are $2.00 each. There is a 25% discount on orders for 100 or more copies mailed to a singIe address. Appendix II Major Contributors to This Report Joel C. Willemssen, Assistant Director Information Andrea M. Leopold, Assignment Manager Management and Michael S. Dombrowski, Evaluator-in-Charge Technology Division, Kelly Wolslayer, Computer Scientist Washington, D.C. Karlton P. Davis, Senior Evaluator New York Regional Office Peter Maristch, Senior Evaluator Philadelphia Regional Office Page 14 GAO/IMTEG90-50 Air Tmf’flc Control Objectives, Scope,and Methodology At the request of the House and Senate Appropriations Committees, Subcommittees on Transportation and Related Agencies, we reviewed ~;2tz’sefforts to upgrade automation systems at smaller TRACONS. Our objectives were to (1) determine the extent of and reasons for delays in developing, testing, and deploying the ARTS IIA hardware and software, and (2) determine if FAA has adequately identified current and future computer capacity requirements for ARTS IIA. To evaluate why problems occurred during the development and testing of ARTS IIA, we reviewed system specification and requirements state- ments, the development and production contracts, and test plans and schedules. We also reviewed FAA and contractor correspondence, tech- nical integration meeting minutes, and software integration group meeting minutes. We examined the results of hardware and software tests conducted at the contractor’s facility; at the FAA Technical Center in Pomona, New Jersey; and the Atlantic City, New Jersey, TRACON. We also reviewed monthly contractor progress reports prepared by UNImS and those prepared by support contractors. In addition, we interviewed agency officials, both at v.4~ headquarters in Washington, DC., and at the FAA Technical Center, and UNISkS program and contract officials, to obtain their views on why the ARTS IIA software had been delayed. To evaluate whether F’AAhas adequately identified current and future capacity requirements for ARTS IIA, we examined appropriate contracts and specifications. In addition, we reviewed the request for proposals and other documentation for the planned ARTS IIA expansion project. We interviewed agency and contractor officials to obtain their views on operational capacity requirements for ARTS IIA and its expanded system. We also discussed acquisition plans with Department of Transportation officials. We met with IX\ and contractor officials to discuss a computer capacity and performance management program for ARTS IIAs and capacity requirements for implementing the Mode C Intruder safety enhancement. We performed our work at FAA'S Washington, D.C., headquarters, its Pomona, New Jersey, Technical Center, and its regional office in Jamaica, New York. We also performed work at TRACON facilities in Pen- sacola, Florida; Binghamton, New York; Allentown, Erie, and Harris- burg, Pennsylvania; Richmond, Virginia; the Department of Transportation in Washington, D.C.; Martin Marietta Corporation in Washington, D.C.; and IJNISYSCorporation in Paoli, Pennsylvania. Page12 GAODMTEGSO-60AirTmfflcControl 5239204 changes. Officials agreed that integration testing is important and recog- nized that there is room for improvement. Finally, Department and FAA officials agreed that they do not have a computer capacity and perform- ance management program in place, but they plan to begin measuring computer performance. Although we acknowledge that risks exist in making production deci- sions, we believe that such a decision should be based on a demonstra- tion that the system meets requirements, including computer capacity requirements, which are critical to successful deployment. We are encouraged by agency commitments to improve integration testing and to measure computer capacity and performance. As arranged with your offices, unless you publicly announce the con- tents of this report earlier, we plan no further distribution of it until 30 days after the date of this letter. We will then send copies to the Secre- tary of Transportation; the Administrator, FAA; and to other interested parties. We will also make copies available to others upon request. This report was prepared under the direction of JayEtta Z. Hecker, Director, Resources, Community, and Economic Development Information Sys- tems, who can be reached at (202) 275-9675. Other major contributors are listed in appendix II. Ralph V. Carlone Assistant Comptroller General Page 10 GAO/IMTEC9@-60 Air Traftic Control - 5239204 experiencing increasing traffic levels, which could increase work loads and lead to computer capacity shortfalls. Inadequate Analysis to E’AAplans to award a $63-million ARTS IIA expansion contract in late fiscal year 1990 that includes an option to increase computer processing Support Expansion capacity at all ARTS IIA facilities. The option calls for obtaining 145 new Contract computers with sufficient processing and memory capacity to perform the majority of AKTS IIA functions. This option would provide additional processing capacity in order to provide the Mode C Intruder safety enhancement and to increase track requirements from 102 to 256. However, FAA once again did not perform any meaningful analysis to determine this track requirement. According to the program manager, this requirement represents the maximum number of targets that the ARTS IIA could track when an additional computer is implemented at each site. Because there is no computer capacity and performance man- agement program, FAA cannot adequately determine requirements for the expansion contract. Although the Federal Information Resources Management Regulation Part 201-30 states that requirements analyses should include a performance evaluation of the currently installed system, FAA evaluated neither the ARTS II nor ARTS IIA systems. Instead, FAA estimated track requirements without analyzing system perform- ance or future work loads. As a result, it is uncertain how well the ARTS IIA expansion will meet FAAneeds. Safety enhancements identified over 8 years ago are just now becoming Conclusions operational because of inadequate FAA program management. FAA did not ensure that development of the system was complete prior to award of the production contract, that software requirements were not modi- fied after award of the production contract, and that integrated tests were performed by the contractor. As a result, controllers are just now getting key automation aids to assist in separating aircraft. In addition, FAA does not have a computer capacity and performance management program at its small TRACONS. The lack of such a program constitutes a material weakness under the Federal Managers’ Financial Integrity Act because such a program is an integral part of computer operations, which are essential to FAA’Scritical air traffic control mis- sion. Without a capacity management program, FAA did not adequately determine capacity requirements and does not have assurance that the ARTS IIA system will meet small TRACON needs. Further, FAA plans to add Page8 GAO/IMTEC&O-60 Air Traffic Control B-239204 result, the contractor had to modify the system design, which further delayed system deployment. Contractor Was Not The contractor’s test plan did not require integration testing prior to Required to Perform system acceptance at the ISA Technical Center. Integration testing deter- mines if multiple pieces of a system work together correctly. In general, Integrated Tests detecting and solving software problems earlier is less time-consuming than solving them later in the development and production process. Con- tractor testing was limited to a series of stand-alone tests that verified how well individual requirements were met, but did not demonstrate full system performance. As a result, the system failed FAA integration tests several times; attempts to correct deficiencies caused additional problems. For example, a demonstration of 12 software solutions to pre- viously noted problems showed that while the solutions worked, 11 new problems were discovered. As a result, schedule delays occurred while the contractor spent additional time developing solutions to the new problems. According to federal regulations, agencies are required to perform com- FAA Lacks a puter capacity and performance management activities in planning, Computer Capacity acquiring, and using computer resources. However, FAA has not estab- and Performance lished a capacity management program because small TRACONS have not encountered capacity shortfalls. Therefore, current processor utilization Management Program has not been measured, current and future capacity requirements have not been adequately defined, and it is uncertain whether ARTS IIA will meet FAA'S needs. Additionally, FAA cannot be certain that additional capacity provided through the proposed expansion project will meet TRACOK needs. Capacity Planning and An effective capacity management and performance monitoring pro- Performance Management gram addresses both performance management and capacity planning. Performance management involves analyzing the current work load and Are Important the performance of computer systems to determine how resources are currently used and how utilization can be improved. Capacity planning assists in forecasting future work loads and other computer resource requirements to ensure that enough capacity will be available when needed. Moreover, Federal Information Resources Management Regulation Part 201-30 requires agencies to perform capacity management activities in Page6 GAO/IMTEGSO-SO Air Traffic Control Implementation of ARTS IIA has been delayed 3 years because of inade- Inadequate Program quate program management. When FAA awarded the production contract Management Has in 1985, it planned to begin deployment of the ARTS HA enhancements in Delayed the spring of 1987. However, FAA awarded a production contract prior to completion of system development and required changes to the produc- Implementation of tion software several times after the contract was awarded. Addition- ARTS IIA ally, FAA did not require the contractor to perform integration testing prior to FAA’S own testing. As a result, the system failed to meet require- ments when FAA tested it, contributing to delays in deploying ARTS HA until March 1990. ARTS IIA Has Been The original production contract schedule required UNISE to deliver both the ARTS IIA hardware and software to the FM Technical Center in Delayed 3 Years Pomona, New Jersey, for testing in November 1986. UNISYS delivered the system in December 1986, but testing revealed numerous software problems. Additionally, this version of software did not include modifi- cations already made to the software operating at smaller TFWONS. Therefore, UNIStS was tasked to include these modifications in ARTS HA software. UNISYS returned the system to the technical center for testing in December 1987. Three months later, test results indicated that the ARTS IIA did not have sufficient processing capacity to meet required work loads while maintaining all air traffic safety functions. UNISYS then modified the system design to provide additional capacity and returned it to the technical center for testing in June 1989. In the fall of 1989 FAA decided the system met requirements and sent the system to the Atlantic City, New Jersey, TRACON for testing in an operational environment. In March 1990, testing was deemed successful and the Atlantic City ARTS IIA became fully operational. Over the next year and a half, FAA plans to deploy the ARTS HA to all remaining smaller TRACONS. Development Not The 1982 development contract demonstrated, on the basis of testing at Complete Prior to the technical center, that the ARTS IIA system could provide tracking, the safety enhancements, and the controller training function. The Production Contract system also met FAA’S original requirements to process 48 tracks,4 sup- Award port six controller displays, and use a 1982 software version. However, prior to awarding the production contract, FAA decided to change the “A track occupies a prmmn of memory in the ARTS HA computer and cmtains the pertinent dataon a controlled aircraft. Page 4 GAO/IMTEG9O-6OAirTrafficContml R239204 large TRACONS led to computer shortfalls and resulting system degrada- tion. FAA has not established a capacity management program because smaller TFWONS have not encountered capacity shortfalls. As a result, FAA does not have a good understanding of current computer utilization or future computer capacity requirements at smaller TRACONS. Therefore, it is uncertain whether ARTS IIA will meet FAA'S needs at smaller TRACONS. FAA needs to improve management of future TRACOK automation acquisi- tions and upgrades by making production decisions only after develop- ment is complete, controlling system requirements changes after contract award, and requiring contractors to perform integrated testing. Further, FAA needs to implement a computer capacity and performance management program for smaller TRACON automation systems, and define future capacity requirements prior to expanding ARTSIIA. FAA'S air traffic control mission is to promote the safe, orderly, and Background expeditious flow of aircraft. Maintaining the required horizontal and vertical separation between aircraft is critical to safety. Air traffic con- trollers maintain the required separation by using information that is assembled and processed by FAA'S air traffic control computer systems and displayed on video screens at controllers’ workstations. The position information displayed on controllers’ screens is accompanied by aircraft identity, altitude, and direction data. Additional information such as a flight’s route, destination, and expected arrival time is provided to con- trollers on paper. Automation Systems at Air traffic controllers at FAA'S TRACON facilities sequence and separate Smaller TRACONs aircraft arriving at or departing from airports under their control. Con- trollers at each of the 119 smaller TRACONS are supported by a computer system known as an ARTS II.’ This computer system receives input from radar, identifies aircraft, associates the aircraft with flight plans, and displays aircraft identification and position location to controllers. In March 1982 FAA awarded a $4.1-million development contract to Bur- roughs Corporation to determine if ARTS11scould be enhanced to an upgraded system--AkTs IIA. The enhancements were to provide aircraft ‘FAA also maintains 22 air route traffic control centers, which control an traffic that is en route between airpats, and 63 larger TRACONs, which control traffic at busier airports. Each of these facilities is supported by an automated system to help controllers maintain aircraft separation. Page2 GAO/IMTECSO-50 Air Traffic Control
Air Traffic Control: Smaller Terminal Systems' Capacity Requirements Need to Be Defined
Published by the Government Accountability Office on 1990-06-25.
Below is a raw (and likely hideous) rendition of the original report. (PDF)