Air Traffic Control: Smaller Terminal Systems' Capacity Requirements Need to Be Defined

Published by the Government Accountability Office on 1990-06-25.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                    United   States   General   Accounting   Office
                    Report to Congressional Requesters

.i; June 1990

                    AIR TRAFFIC

                    Smaller Terminal
                    Systems. Lapamy
                    Requirements Need to:
                    l3e Defined



                   United States
GAO                General Accounting Office
                   Washington, D.C. 20548

                   Information   Management    and
                   Technology    Division


                   June 25,199O

                   The Honorable Frank R. Lautenberg
                   Chairman, Subcommittee on Transportation
                     and Related Agencies
                   Committee on Appropriations
                   United States Senate

                   The Honorable William Lehman
                   Chairman, Subcommittee on Transportation
                     and Related Agencies
                   Committee on Appropriations
                   House of Representatives

                   In response to your December 15, 1989, request, we reviewed the Fed-
                   eral Aviation Administration’s (FAA) efforts to provide automated safety
                   enhancements for air traffic controllers at smaller Terminal Radar
                   Approach Control (TRACOK)facilities. These TRACONS control aircraft
                   arriving at or departing from airports such as Pensacola, Florida, and
                   Binghamton, New York. To implement the enhancements and upgrade
                   existing automation systems at these TRACONs, FAA is deploying the Auto-
                   mated Radar Terminal System (ARTS) IIA. As agreed with your offices,
                   our objectives were to (1) determine the extent of and reasons for delays
                   in developing, testing, and deploying the ARTS IIA hardware and
                   software, and (2) determine if FAA has adequately identified current and
                   future computer capacity requirements for smaller TRACONS. A detailed
                   explanation of our objectives, scope, and methodology is contained in
                   appendix I.

                   Installation of safety enhancements at smaller TRACONS has been delayed
Results in Brief   about 3 years because FAA did not properly manage the program. FAA
                   awarded a production contract before the system was developed and
                   continued to change system requirements after the contract was
                   awarded. Also, E’AAdid not require the contractor to perform integrated
                   tests to determine if various portions of the system worked together cor-
                   rectly. As a result, the system design had to be modified during produc-
                   tion, which led to delays in deployment.

                   FAA also does not have a computer capacity and performance manage-
                   ment program for its smaller TKACONS. Such a program measures current
                   computer utilization and is used to predict future capacity requirements.
                   As we reported last year, the lack of a capacity management program at

                   Page 1                                      GA0/IMTEC90-50AirTrafficC~ntm1

tracking, the safety features of conflict alert and minimum safe altitude
warning, and add a controller training function. Aircraft tracking would
perform position predictions and display aircraft ground speed to con-
trollers. Conflict alert would provide visual and sound alerts to the con-
troller when two controlled2 aircraft are in or are projected to come into
close proximity to each other within the next 40 seconds. Minimum safe
altitude warning would provide visual and sound alerts to controllers
when an aircraft is flying or is projected to fly too low. The controller
training function was to simulate live traffic situations without
affecting air traffic control operations at other workstations. FAA imple-
mented these enhancements at its larger TRACONS in several phases
between 1977 and 1989.

In July 1985 FAA awarded the ARTS IIA production contract to Burroughs
Corporation, now UNISYS, at an estimated cost of $35.7 million. This
contract, now valued at $45.4 million, is to provide the above enhance-
ments to all 119 ARTS IIA TFNZONS through new hardware and software
by August 1991.

In addition to these enhancements, FAA plans to award a contract late in
fiscal year 1990 to improve ARTS IIA reliability and increase system
capacity. This planned contract is designed to provide additional con-
troller workstations, replace obsolete tape drives with disk drives, and
provide uninterruptable power systems for several ARTS IIA locations. In
addition, the planned contract contains an option to increase ARTS IIA
capacity and provide another safety enhancement-the Mode C
Intruder.” This option is designed to provide 145 new computers to sig-
nificantly increase processing and memory capacity. This contract is
part of FAA’s Interim Support Plan designed to sustain the air traffic
control system until it is replaced by the Advanced Automation System
in the mid-to-late 1990s.

‘Controlled aircraft are followed from takeoff to landing by controllers, must carry electromc equip
ment mcludrng radios to communicate with controllers and a transponder to provide altitude and
identity informatmn, and must file flight plans that detail their proposed journey. IJncontrolled air-
craft, although sometnnes monitored by the system, do not need to carry electronic equipment unless
they enter contwlled airspace. and arc not required to file fhght plans; however, they must follow
well-established FAA rules regarding where they can fly,
3The Mode C Intruder is a wammg to controllers that indicates that the distance between a controlled
and uncontrolled aircraft wll berome hazardous within the next 40 seconds.This warning differs
from conflict alert in that It will monitor one controlled and one uncontrolled Mode C-equipped au-
craft, rather than two controlled ainxiift

Page 3                                                       GAO/IMTEG90-SO Air Traffic Control


                            requirements specifying that the ARTS IIA process 102 tracks, support
                            up to 11 displays, and use the current operational software version, as
                            well as provide tracking, the safety enhancements, and the training

                            FL awarded the production contract without requiring the contractor to
                            demonstrate that it could meet the new system requirements. In fact, the
                            contractor’s proposal questioned whether the system could provide suf-
                            ficient processing capacity to support an 1 l-display system. According
                            to the program manager, FAA recognized at the time the production con-
                            tract was awarded that additional modifications might be necessary to
                            meet all AR% IIA operational requirements.

                            The Office of Management and Budget’s procurement policy, contained
                            in its Circular A-109, requires production decisions to be based on evi-
                            dence that the system meets requirements before committing to produc-
                            tion. In this instance, FAA'S decision to award the production contract
                            before development was complete contributed to ARTS IIA implementa-
                            tion delays.

Software Baseline Not       While new software was being developed under the ARTS IIA develop-
Stabilized for Production   ment and production contracts, FAA continued to modify the existing
                            ARTS II software operating at smaller TRACONS. FAA did this to correct
Contract                    inefficiencies and provide additional features, some of which were
                            already operational at a few sites. As these improvements were com-
                            pleted, FAA released the new software to sites. Although it is important
                            to adequately specify the functions software should perform before
                            committing to a production decision, FAA continued to modify the
                            software requirements throughout the life of the production contract.
                            This contributed to implementation delays. For example, FAA released
                            new software to ARTS II sites 4 months after the ARTS IIA contract award
                            and then changed the contract to reflect these changes. Then, in 1986,
                            FAA required that seven additional software features be included in the
                            ARTS IIA contract.

                            By late 1987, these seven additional features had been incorporated and
                            UNISE shipped the ARTS IIA to the FAA Technical Center for testing. The
                            results of this test identified over 200 problems, 65 of which FAA classi-
                            fied as critical. The most significant problem was the system’s inability
                            to meet the 102- track requirement. Contractor personnel told us that
                            the system failed to meet this requirement because the features added to
                            the system baseline used significant amounts of computer capacity. As a

                            peg.25                                      GAO/IMTECSO-50AirTraffcControl

                        planning, acquiring, and using computer resources. Last year we
                        reported that FAA lacked a computer capacity and performance manage-
                        ment program for its large TRACONS.~ Consequently, FAA did not recognize
                        capacity shortfalls until controllers’ ability to maintain safe separation
                        of aircraft was impaired.

                        Further, adequately defining capacity requirements and establishing a
                        computer capacity and performance management program are neces-
                        sary to ensure maximum use of existing resources and adequate
                        capacity for growth. Capacity expansion should be based on an analysis
                        of current system performance, new requirements, and projections for
                        future growth.

ARTS IIA May Not Meet   Although we reported in 1983” that FAA should implement a computer
Small TRACON            performance management function to measure terminal systems’ effi-
                        ciency and effectiveness, FAA lacks such a program for ARTS IIA systems.
Requirements            PAAofficials stated that a program has not been established because
                        AHTS II systems did not encounter capacity shortfalls. The lack of such a
                        program constitutes a material weakness under the Federal Managers’
                        Financial Integrity Act (31 USC. 3512 (b) and (c)). This weakness is
                        regarded as material given the fact that a capacity management pro-
                        gram is an integral part of effective computer operations, which is cen-
                        tral to FAA's fulfilling its air traffic control mission.

                        Although FAA increased the track requirement for the ARTS IIA system to
                         102 tracks prior to awarding the production contract, it did not analyze
                        current and future work loads to determine if this requirement would
                        meet actual needs. In fact, FAA did not perform any meaningful analysis
                        to derive this stated track requirement. As a result, FAA has no assur-
                        ance that the work loads at ARTS IIA facilities will not exceed 102 tracks.

                        Additionally, ARTS IIA tests conducted in 1989 indicate that the system
                        uses 76 to 93 percent of total processing capacity when operating under
                        maximum specified work loads. This indicates that ARTS IIA systems
                        may approach computer capacity limitations under heavier work loads.
                        However, because FAA does not collect data on current or projected
                        instantaneous traffic levels, it does not know whether ARTS IIA sites are

                        ‘AK Traffic Control: Computer Capacity Shortfalls May lmpaw Flight Safety (GAO/IMTEC-89-63.
                        .luly ti, 1989).

                        “FAA‘s Plans to Improve The Air TrafRc, Control System: A Step In The Right Direction But Improve-
                        ments And lkttcr Cuordinatlon Are Kwdcd (GAO/AFMD-a-34, Feb. 16, 1983).
                     computer capacity on the basis of inadequate analysis of current usage,
                     future traffic levels, and future work loads, including Mode C Intruder
                     requirements. Basing an acquisition on inadequately defined require-
                     ments and incomplete performance data increases the risk that the
                     expanded systems will be inadequate to meet operational requirements.

                     Experience gained in implementing the ARTSIIA project can be used to
Recommendations      prevent similar problems from occurring on future projects. We recom-
                     mend that the Secretary of Transportation direct the Administrator,
                     FAA, to improve management of future TRACON automation projects by
                     awarding production contracts only after development is complete, con-
                     trolling changes to operational software during system development and
                     production, and requiring contractors to perform integrated testing.

                     In addition, we recommend that the Secretary of Transportation direct
                     the Administrator to implement a computer capacity and performance
                     management program for ARTS IIA systems. This program should include
                     an analysis of current system performance and future work loads,
                     including predicted traffic levels and additional software functions to
                     determine system requirements, as well as the potential performance of
                     the enhanced system.

                     We also recommend that the Secretary direct the Administrator to
                     report the lack of a computer capacity and performance management
                     program as a material control weakness under the Federal Managers’
                     Financial Integrity Act until a program has been implemented.

                     Finally, we recommend that the Secretary direct the Administrator to
                     delay exercising the expansion contract option to procure additional
                     computers until a computer capacity and performance management pro-
                     gram is implemented and future capacity requirements are adequately

                     We obtained official oral comments from the Department of Transporta-
- - Comments
        _      and   tion and FAA officials on a draft of this report. They stated that FAA
Our Evaluation       awarded the production contract in July 1985 after considering the tech-
                     nical, schedule, and cost risks and found these risks to be acceptable
                     based on the information available at that time. Department and FAA
                     officials agreed that software requirements were modified after contract
                     award and that this delayed the implementation of the safety enhance-
                     ments. However, they commented that they took steps to control the

                     Page 9                                     GAO/IMTEC90-50 Air Traffic Control
Page 11   GA0/1MTEC9060   Air Traffic Control
Appendix I
Objectives, Scope, and Methodology

Our review was conducted from May 1989 to April 1990. The views of
agency officials were sought during the course of our work and their
comments have been incorporated where appropriate. In addition, at the
completion of our review, we discussed the report’s key facts, conclu-
sions, and recommendations with FAA officials. Finally, we obtained
formal oral comments from Department of Transportation and FAA offi-
cials on a draft of this report. These comments and our analysis are also
included in this report. We conducted our review in accordance with
generally accepted government auditing standards.

Page 13                                     GA0/1MTECSO-50 Air Traffic Control
Requests for copies of GAO reports    should be sent to:

U.S. General Accounting    Office
Post Office Box 6015
Gaithersburg,  Maryland    20877

Telephone   202-275-6241

The fast five copies of each report   are free. Additional   copies are
$2.00 each.

There is a 25% discount    on orders for 100 or more copies mailed to a
singIe address.
Appendix II

Major Contributors to This Report

                        Joel C. Willemssen, Assistant Director
Information             Andrea M. Leopold, Assignment Manager
Management and          Michael S. Dombrowski, Evaluator-in-Charge
Technology Division,    Kelly Wolslayer, Computer Scientist
Washington, D.C.

                        Karlton P. Davis, Senior Evaluator
New York Regional

                        Peter Maristch, Senior Evaluator
Philadelphia Regional

                        Page 14                                      GAO/IMTEG90-50 Air Tmf’flc Control
Objectives, Scope,and Methodology

               At the request of the House and Senate Appropriations Committees,
               Subcommittees on Transportation and Related Agencies, we reviewed
               ~;2tz’sefforts to upgrade automation systems at smaller TRACONS. Our
               objectives were to (1) determine the extent of and reasons for delays in
               developing, testing, and deploying the ARTS IIA hardware and software,
               and (2) determine if FAA has adequately identified current and future
               computer capacity requirements for ARTS IIA.

               To evaluate why problems occurred during the development and testing
               of ARTS IIA, we reviewed system specification and requirements state-
               ments, the development and production contracts, and test plans and
               schedules. We also reviewed FAA and contractor correspondence, tech-
               nical integration meeting minutes, and software integration group
               meeting minutes. We examined the results of hardware and software
               tests conducted at the contractor’s facility; at the FAA Technical Center
               in Pomona, New Jersey; and the Atlantic City, New Jersey, TRACON. We
               also reviewed monthly contractor progress reports prepared by UNImS
               and those prepared by support contractors. In addition, we interviewed
               agency officials, both at v.4~ headquarters in Washington, DC., and at
               the FAA Technical Center, and UNISkS program and contract officials, to
               obtain their views on why the ARTS IIA software had been delayed.

               To evaluate whether F’AAhas adequately identified current and future
               capacity requirements for ARTS IIA, we examined appropriate contracts
               and specifications. In addition, we reviewed the request for proposals
               and other documentation for the planned ARTS IIA expansion project. We
               interviewed agency and contractor officials to obtain their views on
               operational capacity requirements for ARTS IIA and its expanded system.
               We also discussed acquisition plans with Department of Transportation
               officials. We met with IX\ and contractor officials to discuss a computer
               capacity and performance management program for ARTS IIAs and
               capacity requirements for implementing the Mode C Intruder safety

               We performed our work at FAA'S Washington, D.C., headquarters, its
               Pomona, New Jersey, Technical Center, and its regional office in
               Jamaica, New York. We also performed work at TRACON facilities in Pen-
               sacola, Florida; Binghamton, New York; Allentown, Erie, and Harris-
               burg, Pennsylvania; Richmond, Virginia; the Department of
               Transportation in Washington, D.C.; Martin Marietta Corporation in
               Washington, D.C.; and IJNISYSCorporation in Paoli, Pennsylvania.

               Page12                                      GAODMTEGSO-60AirTmfflcControl

changes. Officials agreed that integration testing is important and recog-
nized that there is room for improvement. Finally, Department and FAA
officials agreed that they do not have a computer capacity and perform-
ance management program in place, but they plan to begin measuring
computer performance.

Although we acknowledge that risks exist in making production deci-
sions, we believe that such a decision should be based on a demonstra-
tion that the system meets requirements, including computer capacity
requirements, which are critical to successful deployment. We are
encouraged by agency commitments to improve integration testing and
to measure computer capacity and performance.

As arranged with your offices, unless you publicly announce the con-
tents of this report earlier, we plan no further distribution of it until 30
days after the date of this letter. We will then send copies to the Secre-
tary of Transportation; the Administrator, FAA; and to other interested
parties. We will also make copies available to others upon request. This
report was prepared under the direction of JayEtta Z. Hecker, Director,
Resources, Community, and Economic Development Information Sys-
tems, who can be reached at (202) 275-9675. Other major contributors
are listed in appendix II.

Ralph V. Carlone
Assistant Comptroller    General

Page 10                                       GAO/IMTEC9@-60 Air Traftic Control

                             experiencing increasing traffic levels, which could increase work loads
                             and lead to computer capacity shortfalls.

Inadequate Analysis to       E’AAplans to award a $63-million ARTS IIA expansion contract in late
                             fiscal year 1990 that includes an option to increase computer processing
Support Expansion            capacity at all ARTS IIA facilities. The option calls for obtaining 145 new
Contract                     computers with sufficient processing and memory capacity to perform
                             the majority of AKTS IIA functions. This option would provide additional
                             processing capacity in order to provide the Mode C Intruder safety
                             enhancement and to increase track requirements from 102 to 256.

                             However, FAA once again did not perform any meaningful analysis to
                             determine this track requirement. According to the program manager,
                             this requirement represents the maximum number of targets that the
                             ARTS IIA could track when an additional computer is implemented at
                             each site. Because there is no computer capacity and performance man-
                             agement program, FAA cannot adequately determine requirements for
                             the expansion contract. Although the Federal Information Resources
                             Management Regulation Part 201-30 states that requirements analyses
                             should include a performance evaluation of the currently installed
                             system, FAA evaluated neither the ARTS II nor ARTS IIA systems. Instead,
                             FAA estimated track requirements without analyzing system perform-
                             ance or future work loads. As a result, it is uncertain how well the ARTS
                             IIA expansion will meet FAAneeds.

                             Safety enhancements identified over 8 years ago are just now becoming
Conclusions                  operational because of inadequate FAA program management. FAA did
                             not ensure that development of the system was complete prior to award
                             of the production contract, that software requirements were not modi-
                             fied after award of the production contract, and that integrated tests
                             were performed by the contractor. As a result, controllers are just now
                             getting key automation aids to assist in separating aircraft.

                             In addition, FAA does not have a computer capacity and performance
                             management program at its small TRACONS. The lack of such a program
                             constitutes a material weakness under the Federal Managers’ Financial
                             Integrity Act because such a program is an integral part of computer
                             operations, which are essential to FAA’Scritical air traffic control mis-
                             sion. Without a capacity management program, FAA did not adequately
                             determine capacity requirements and does not have assurance that the
                             ARTS IIA system will meet small TRACON needs. Further, FAA plans to add

                             Page8                                        GAO/IMTEC&O-60 Air Traffic Control

                         result, the contractor had to modify the system design, which further
                         delayed system deployment.

Contractor Was Not       The contractor’s test plan did not require integration testing prior to
Required to Perform      system acceptance at the ISA Technical Center. Integration testing deter-
                         mines if multiple pieces of a system work together correctly. In general,
Integrated Tests         detecting and solving software problems earlier is less time-consuming
                         than solving them later in the development and production process. Con-
                         tractor testing was limited to a series of stand-alone tests that verified
                         how well individual requirements were met, but did not demonstrate full
                         system performance. As a result, the system failed FAA integration tests
                         several times; attempts to correct deficiencies caused additional
                         problems. For example, a demonstration of 12 software solutions to pre-
                         viously noted problems showed that while the solutions worked, 11 new
                         problems were discovered. As a result, schedule delays occurred while
                         the contractor spent additional time developing solutions to the new

                         According to federal regulations, agencies are required to perform com-
FAA Lacks a              puter capacity and performance management activities in planning,
Computer Capacity        acquiring, and using computer resources. However, FAA has not estab-
and Performance          lished a capacity management program because small TRACONS have not
                         encountered capacity shortfalls. Therefore, current processor utilization
Management Program       has not been measured, current and future capacity requirements have
                         not been adequately defined, and it is uncertain whether ARTS IIA will
                         meet FAA'S needs. Additionally, FAA cannot be certain that additional
                         capacity provided through the proposed expansion project will meet
                         TRACOK needs.

Capacity Planning and    An effective capacity management and performance monitoring pro-
Performance Management   gram addresses both performance management and capacity planning.
                         Performance management involves analyzing the current work load and
Are Important            the performance of computer systems to determine how resources are
                         currently used and how utilization can be improved. Capacity planning
                         assists in forecasting future work loads and other computer resource
                         requirements to ensure that enough capacity will be available when

                         Moreover, Federal Information Resources Management Regulation Part
                         201-30 requires agencies to perform capacity management activities in

                         Page6                                       GAO/IMTEGSO-SO Air Traffic Control
                      Implementation of ARTS IIA has been delayed 3 years because of inade-
Inadequate Program    quate program management. When FAA awarded the production contract
Management Has        in 1985, it planned to begin deployment of the ARTS HA enhancements in
Delayed               the spring of 1987. However, FAA awarded a production contract prior to
                      completion of system development and required changes to the produc-
Implementation of     tion software several times after the contract was awarded. Addition-
ARTS IIA              ally, FAA did not require the contractor to perform integration testing
                      prior to FAA’S own testing. As a result, the system failed to meet require-
                      ments when FAA tested it, contributing to delays in deploying ARTS HA
                      until March 1990.

ARTS IIA Has Been     The original production contract schedule required UNISE to deliver
                      both the ARTS IIA hardware and software to the FM Technical Center in
Delayed 3 Years       Pomona, New Jersey, for testing in November 1986. UNISYS delivered
                      the system in December 1986, but testing revealed numerous software
                      problems. Additionally, this version of software did not include modifi-
                      cations already made to the software operating at smaller TFWONS.
                      Therefore, UNIStS was tasked to include these modifications in ARTS HA

                      UNISYS returned the system to the technical center for testing in
                      December 1987. Three months later, test results indicated that the ARTS
                      IIA did not have sufficient processing capacity to meet required work
                      loads while maintaining all air traffic safety functions. UNISYS then
                      modified the system design to provide additional capacity and returned
                      it to the technical center for testing in June 1989. In the fall of 1989 FAA
                      decided the system met requirements and sent the system to the Atlantic
                      City, New Jersey, TRACON for testing in an operational environment. In
                      March 1990, testing was deemed successful and the Atlantic City ARTS
                      IIA became fully operational. Over the next year and a half, FAA plans to
                      deploy the ARTS HA to all remaining smaller TRACONS.

Development Not       The 1982 development contract demonstrated, on the basis of testing at
Complete Prior to     the technical center, that the ARTS IIA system could provide tracking,
                      the safety enhancements, and the controller training function. The
Production Contract   system also met FAA’S original requirements to process 48 tracks,4 sup-
Award                 port six controller displays, and use a 1982 software version. However,
                      prior to awarding the production contract, FAA decided to change the

                      “A track occupies a prmmn of memory in the ARTS HA computer and cmtains the pertinent dataon
                      a controlled aircraft.

                      Page 4                                                  GAO/IMTEG9O-6OAirTrafficContml

                        large TRACONS led to computer shortfalls and resulting system degrada-
                        tion. FAA has not established a capacity management program because
                        smaller TFWONS have not encountered capacity shortfalls. As a result,
                        FAA does not have a good understanding of current computer utilization
                        or future computer capacity requirements at smaller TRACONS. Therefore,
                        it is uncertain whether ARTS IIA will meet FAA'S needs at smaller

                        FAA  needs to improve management of future TRACOK automation acquisi-
                        tions and upgrades by making production decisions only after develop-
                        ment is complete, controlling system requirements changes after
                        contract award, and requiring contractors to perform integrated testing.
                        Further, FAA needs to implement a computer capacity and performance
                        management program for smaller TRACON automation systems, and
                        define future capacity requirements prior to expanding ARTSIIA.

                        FAA'S air traffic control mission is to promote the safe, orderly, and
Background              expeditious flow of aircraft. Maintaining the required horizontal and
                        vertical separation between aircraft is critical to safety. Air traffic con-
                        trollers maintain the required separation by using information that is
                        assembled and processed by FAA'S air traffic control computer systems
                        and displayed on video screens at controllers’ workstations. The position
                        information displayed on controllers’ screens is accompanied by aircraft
                        identity, altitude, and direction data. Additional information such as a
                        flight’s route, destination, and expected arrival time is provided to con-
                        trollers on paper.

Automation Systems at   Air traffic controllers at FAA'S TRACON facilities sequence and separate
Smaller TRACONs         aircraft arriving at or departing from airports under their control. Con-
                        trollers at each of the 119 smaller TRACONS are supported by a computer
                        system known as an ARTS II.’ This computer system receives input from
                        radar, identifies aircraft, associates the aircraft with flight plans, and
                        displays aircraft identification and position location to controllers.

                        In March 1982 FAA awarded a $4.1-million development contract to Bur-
                        roughs Corporation to determine if ARTS11scould be enhanced to an
                        upgraded system--AkTs IIA. The enhancements were to provide aircraft

                        ‘FAA also maintains 22 air route traffic control centers, which control an traffic that is en route
                        between airpats, and 63 larger TRACONs, which control traffic at busier airports. Each of these
                        facilities is supported by an automated system to help controllers maintain aircraft separation.

                        Page2                                                         GAO/IMTECSO-50 Air Traffic Control