oversight

Aviation Safety: Efforts to Implement Flight Operational Quality Assurance Programs

Published by the Government Accountability Office on 1997-12-02.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                 United States General Accounting Office

GAO              Report to Congressional Requesters




December 1997
                 AVIATION SAFETY
                 Efforts to Implement
                 Flight Operational
                 Quality Assurance
                 Programs




GAO/RCED-98-10
                   United States
GAO                General Accounting Office
                   Washington, D.C. 20548

                   Resources, Community, and
                   Economic Development Division

                   B-275990

                   December 2, 1997

                   The Honorable Wendell H. Ford
                   The Honorable Ron Wyden
                   United States Senate

                   The analysis of aircraft data recorded during flight has played a crucial
                   role in determining the causes of crashes. Recently, however, some U.S.
                   airlines have begun to analyze flight data from uneventful airline flights to
                   identify potential problems and correct them before they lead to accidents.
                   In your letter of December 2, 1996, you asked us to examine efforts by the
                   Federal Aviation Administration (FAA) and U.S. airlines to implement
                   Flight Operational Quality Assurance (FOQA) programs. The objective of a
                   FOQA program is to use flight data to detect technical flaws, unsafe
                   practices, or conditions outside of desired operating procedures early
                   enough to allow timely intervention to avert accidents or incidents. These
                   programs are voluntary efforts by airlines that involve equipping aircraft
                   with specialized devices to continuously record up to hundreds of
                   different flight data parameters from aircraft systems and sensors,
                   analyzing the data, identifying trends, and taking action to correct
                   potential problems. The analysis of flight data allows airlines to
                   reconstruct entire flights on the basis of the values over time of flight data
                   parameters, such as heading, altitude, throttle settings, ground speed, and
                   many others. Currently, about 33 foreign airlines and 4 U.S.
                   airlines—United, US Airways, Continental, and Alaska—have implemented
                   FOQA or FOQA-type programs.


                   You requested that we determine (1) how FOQA programs will enhance
                   aviation safety, (2) the costs and benefits of such programs, and (3) the
                   factors that could impede their full implementation and actions that could
                   be taken to overcome any impediments.


                   The early experience of domestic airlines with established Flight
Results in Brief   Operational Quality Assurance programs, as well as the testimony of
                   foreign airlines with extensive experience in this area, attests to the
                   potential of such programs to enhance aviation safety by identifying
                   possible safety problems that could lead to accidents. Airlines have used
                   Flight Operational Quality Assurance programs to identify potential
                   problems that were previously unknown or only suspected. Where
                   potential problems were already known, airlines have used these
                   programs to confirm and quantify the extent of the problems. And most




                   Page 1                 GAO/RCED-98-10 Flight Operational Quality Assurance Programs
             B-275990




             important, on the basis of analyses of flight data, airlines have taken
             actions to correct problems and enhance aviation safety.

             The costs associated with implementing a Flight Operational Quality
             Assurance program depend upon a large number of factors, including the
             technology used to capture flight data, the number and types of aircraft to
             be equipped with this technology, and personnel costs. Although the
             program is primarily viewed as a safety program, U.S. and foreign airlines
             have reported financial benefits as well. With additional data on aircraft
             systems and engine conditions, airlines are better able to achieve optimum
             fuel consumption and avoid unneeded engine maintenance. Although more
             difficult to quantify, enhanced safety should result in lower costs over time
             as a result of accidents avoided and lower insurance premiums. FAA’s
             preliminary estimates place the annual cost of a program with 50 aircraft
             at approximately $760,000. Savings from reduced expenditures for fuel,
             engine maintenance, and accident costs for a 50-aircraft program are
             estimated at $1.65 million per year. FAA’s estimates suggest a net savings
             from 50 aircraft of $892,000 per year.

             The primary factor impeding the implementation of Flight Operational
             Quality Assurance programs among the major domestic carriers is the
             resolution of data protection issues. Airline managers and pilots raise
             three significant data protection concerns: (1) use of the data for
             enforcement/disciplinary purposes; (2) disclosure to the media and the
             public under the provisions of the Freedom of Information Act; and
             (3) disclosure through the civil litigation discovery process. FAA has taken
             a number of actions that may resolve these issues, although it is not clear
             whether the aviation community will be satisfied with FAA’s actions. First,
             FAA has begun work on a rulemaking procedure to establish what
             protections from enforcement actions, if any, will apply to information
             submitted to FAA under a Flight Operational Quality Assurance program.
             Second, on October 9, 1996, the Congress enacted legislation and FAA has
             begun work on a rulemaking procedure that would prohibit the
             Administrator from disclosing voluntarily submitted safety information
             under certain circumstances. These actions may ameliorate concerns
             about the Freedom of Information Act. And third, airlines currently seek to
             protect voluntarily collected safety information from disclosure in civil
             litigation on a case-by-case basis.


             Modern commercial aircraft contain sophisticated electronic systems that
Background   gather, process, and manage digital data on many aspects of flight. These



             Page 2                 GAO/RCED-98-10 Flight Operational Quality Assurance Programs
B-275990




data originate from various systems and sensors throughout the aircraft.
The data range from pilot operations to the outputs of sensors and
systems. Some of these data are continuously recorded by the aircraft’s
digital flight data recorder to help investigators understand what happened
if the aircraft is involved in an accident or a serious incident.1 Designed to
survive crashes, flight data recorders typically retain the data recorded
during the last 25 hours of flight.

Rather than analyzing flight data only after an incident or accident, some
airlines routinely analyze the flight data from regular flights. Their aim is
to identify problems that occur in routine operations and to correct these
problems before they become accidents or incidents. In its 1992 study for
FAA,2 the Flight Safety Foundation coined the term “Flight Operational
Quality Assurance” to describe this function. The Foundation defined FOQA
as “a program for obtaining and analyzing data recorded in flight to
improve flight crew performance, air carrier training programs and
operating procedures, air traffic control procedures, airport maintenance
and design, and aircraft operations and design.”

FOQA  has its origin in the use of flight data recorders3 as mandated by the
Civil Aeronautics Administration in 1958. Although the first flight data
recorders captured only six parameters,4 they were a valuable tool for
reconstructing what had occurred preceding a crash. In addition to
recording data to assist in crash investigations, some airlines began to
monitor data recorded on routine flights. Initially, the monitoring systems
captured airworthiness data, but over time they have expanded to include
operational data. FOQA programs were first established in Europe and Asia,
and only within the past few years have some U.S. airlines begun adopting
such a system on a trial basis. At present, about 33 foreign airlines and 4
U.S. airlines—United, US Airways, Continental, and Alaska—have
implemented FOQA or FOQA-type programs. (See app. I for more detailed
background information on FOQA and U.S. airlines’ experience with these
programs; see app. II for a list of airlines worldwide that have
implemented FOQA programs.)

1
 The National Transportation Safety Board, the official source of information on airline accidents,
defines accidents as events in which individuals are killed or suffer serious injury, or the aircraft is
substantially damaged. Incidents are defined as occurrences other than accidents associated with the
operation of an aircraft that affect or could affect the safety of operations. 49 C.F.R. 830.2.
2
 Flight Safety Foundation, Air Carrier Voluntary Flight Operational Quality Assurance Program (1992).
3
 The flight data recorder is commonly referred to as the “black box.”
4
 The six required parameters were time, airspeed, heading, altitude, vertical acceleration, and time of
radio transmission.



Page 3                         GAO/RCED-98-10 Flight Operational Quality Assurance Programs
B-275990




As part of FAA’s strategy to achieve significant reductions in aviation
accident rates despite the rapid increase in air travel anticipated over the
next decade, in 1995 the agency initiated a FOQA demonstration project to
promote the voluntary implementation of FOQA programs by U.S. airlines.
The objective of such a program is to use flight data to detect technical
flaws, unsafe practices, or conditions outside of desired operating
procedures early enough to allow timely intervention to avert accidents or
incidents. For example, identifying repeated instances of unstabilized
approaches to a particular airport could help to define a new approach
pattern less likely to lead to an accident under adverse conditions, or to
improved pilot training. Such a system has potentially broad application to
flight crews’ performance and training, aircraft operating procedures, air
traffic control procedures, aircraft maintenance, and airport design and
maintenance. Major airlines in Europe and Asia, as well as the U.S. airlines
that have FOQA programs, are uniform in their support of the program.

How FOQA Works. FOQA involves (1) capturing and analyzing flight data to
determine if the pilot, the aircraft’s systems, or the aircraft itself deviated
from typical operating norms; (2) identifying trends; and (3) taking action
to correct potential problems. Airlines with FOQA programs typically use a
device called a quick access recorder to capture flight data onto a
removable optical disk that facilitates the data’s frequent removal from the
aircraft.5 Periodically, the optical disks are removed from the aircraft, and
the flight data are analyzed by the ground analysis system at a centralized
location. The data are analyzed by a computer system that evaluates about
40 to 80 predefined events for deviations from the airline’s specified
tolerance thresholds. For example, an event might be the descent rate
during approach. Deviations of more than certain predetermined values,
called “exceedances,” are flagged and evaluated by a monitoring team.
After investigating these exceedances to determine their validity and
analyzing them to understand possible causes, the monitoring team will
propose and evaluate corrective actions. Periodically, airlines aggregate
exceedances over time to determine and monitor trends. (For a more
complete discussion of FOQA operations, see app. I.)

The FOQA Demonstration Project. In July 1995, FAA initiated a 3-year,
$5.5 million demonstration project to facilitate the start-up of voluntary
airline FOQA programs and to assess the costs, benefits, and safety
enhancement associated with such programs. FAA provided hardware and
software to each of the three airlines—United, US Airways, and

5
 These data typically include the parameters required to be collected on the aircraft’s flight data
recorder plus many more parameters. See app. I for more information on quick access recorders and
flight data recorders.



Page 4                        GAO/RCED-98-10 Flight Operational Quality Assurance Programs
B-275990




Continental—that have implemented FOQA programs according to the
demonstration project’s requirements. FAA purchased quick access
recorders to equip 15 Boeing 737 aircraft at each of the three airlines. FAA
also purchased a ground analysis system—the computer hardware and
software for analyzing and visualizing FOQA data—for US Airways and
Continental. Because United already had purchased a ground analysis
system that analyzes these data for other types of aircraft, FAA purchased
for the airline the additional software needed to analyze FOQA data from
737s. For their part, these airlines funded the cost of obtaining
supplemental type certification6 of the airborne equipment, the costs of
installation and maintenance, and the cost of personnel to run and monitor
the program. Alaska Airlines is the fourth U.S. airline to have begun a FOQA
program, but it has only recently met the demonstration project’s
requirement for an agreement on FOQA by the airline’s pilot union.
Consequently, the project has not yet provided any equipment to the
airline. Alaska Airlines, however, received quick access recorders and a
ground analysis system from the FAA Structural Loads Program and uses
this equipment to operate its FOQA program. (See app. III for more
information on the Structural Loads Program.) Other airlines that are
participating in the demonstration project and are considering the
implementation of a FOQA program are America West, Delta, Northwest,
Trans World, Southwest, Continental Express, and United Parcel Service.7
(See app. I for a detailed description of the FAA demonstration project.) As
a research and development effort of the FOQA initiative, FAA is developing
the Aviation Performance Measuring System, an advanced system for
conducting automated analysis and research on FOQA data. (See app. III for
a description of this system and FAA’s other related technical programs.)

Rather than requiring airlines to implement FOQA, FAA has chosen to
promote the initiative through a cooperative demonstration project in
partnership with the industry. According to the demonstration project’s
program manager, it would be premature for FAA to mandate FOQA at this
time because U.S. aviation is still in the early stages of developing FOQA
and is primarily in a learning mode. The program manager contends that a
mandated program would stifle innovation, encounter substantial
resistance from airlines and pilots, and most likely result in minimal
compliance. Thus, at present, FAA is working with the industry to raise
interest in the concept, facilitate the design and implementation of

6
 An FAA type certificate is issued when an aircraft, aircraft engine, propeller, or appliance is properly
designed and manufactured, performs properly, and meets the regulations and minimum standards
prescribed by the Administrator. An FAA supplemental type certificate is required when there is a
change to an aircraft, aircraft engine, propeller, or appliance. 49 U.S.C. 44704.
7
 Although not a participant in the demonstration project, American Airlines is considering the
implementation of an internal FOQA-type program.


Page 5                         GAO/RCED-98-10 Flight Operational Quality Assurance Programs
                   B-275990




                   voluntary FOQA programs, provide financial and technical assistance, and
                   foster innovation.


                   The primary characteristic that distinguishes FOQA from other safety
FOQA Identifies    reporting programs, such as the Aviation Safety Reporting Program or
Potential Safety   Aviation Safety Action Programs,8 is that FOQA provides objective,
Problems           quantitative data on what occurs during flights rather than what is
                   subjectively reported by individuals. Instead of needing to rely on
                   perceived problems or risks, FOQA yields precise information on many
                   aspects of flight operations, and this information can be used to help
                   objectively evaluate a wide range of safety-related issues.

                   U.S. and foreign airlines have reported on previously unknown or
                   suspected problems for which FOQA has provided objective information
                   that resulted in corrective actions. One airline found through its FOQA
                   program that more exceedances occurred during visual flying than during
                   instrument flying. This finding prompted the airline’s flight-training
                   managers to rethink the relative emphasis given visual and instrument
                   flying in the airline’s training programs. Another airline’s FOQA analysis
                   confirmed that the incidence of descent rate exceedances during
                   approaches was significantly higher at a particular runway at a U.S. airport
                   than at other runways. After investigating the problem, the airline
                   concluded that the air traffic control approach had been set too high,
                   requiring pilots to descend more steeply than usual during their final
                   approach. When the airline shared its findings with FAA management, the
                   approach was modified to correct this potential problem.

                   For landings, some airports’ air traffic control procedures require pilots to
                   approach high and fast and then descend steeply. These approaches can
                   result from a number of factors, including noise abatement rules, traffic
                   volume, terrain, or weather conditions. Although airline managers know
                   about the situations from pilots’ reports, FOQA gives them the quantitative
                   information to demonstrate the extent of this problem at the various
                   airports. With these data in hand, managers can be more effective in
                   addressing the problem and taking action to mitigate or eliminate risks.

                   FOQA can also help airlines determine the frequency of certain occurrences
                   rather than having to rely on human judgment, particularly for the level of
                   maintenance required. Two examples of these types of occurrences are
                   hard landings and exceedances in engine temperatures. Prior to FOQA,

                   8
                    See app. IV for a description of these programs.



                   Page 6                         GAO/RCED-98-10 Flight Operational Quality Assurance Programs
                     B-275990




                     airlines generally relied on pilots’ judgment of the necessity for corrective
                     action if a hard landing occurred or an engine overheated. FOQA, however,
                     can provide better information on the amount of force the aircraft
                     experienced during a hard landing. Similarly, FOQA gives more data on the
                     engines’ temperatures and the duration of overheating in some aircraft
                     than were previously available without FOQA. With these data, managers
                     can make more informed decisions about whether the aircraft needs to be
                     inspected to check for structural damage or whether an engine needs to be
                     overhauled.

                     U.S. and foreign airlines have reported that they have used FOQA analysis to
                     identify a variety of potential safety problems and take corrective action to
                     resolve or mitigate them. These have included steep takeoffs, which can
                     damage the aircraft’s tail; approaches that are outside the prescribed
                     procedures for a “stabilized” approach; descent rates or bank angles that
                     are considered excessive; high taxi speeds; hard landings; wind shear
                     occurrences; ground proximity warnings; and engine malfunctions.
                     Corrective action can include notifying pilots of a change in standard
                     operating procedures or restating and emphasizing them, correcting an
                     equipment problem, or providing additional training. The continued
                     monitoring of trends will tell the airline if the corrective action has been
                     effective or whether additional measures are needed.

                     A number of airlines plan to complement the use of FOQA data with
                     information from safety reporting systems, such as Aviation Safety Action
                     Programs or internal pilot reporting systems. FOQA data, originating from
                     aircraft sensors and systems, tell “what” happened to the aircraft. Internal
                     safety reporting systems, based on reports of pilots, flight crews, and other
                     persons, are more likely to tell “why” something happened. Together,
                     information from FOQA and internal reporting systems can provide valuable
                     insight into current and emerging problems.


                     Based on preliminary estimates from an ongoing cost-benefit study by
FOQA’s Potential     Universal Technical Resource Services, Inc. (UTRS), an FAA contractor,
Costs and Benefits   table 1 summarizes the estimated annual costs for airlines to equip 15, 50,
                     and 100 aircraft with quick access recorders, purchase a ground analysis
                     system, and pay FOQA-related salaries.9



                     9
                      Because FAA’s cost-benefit study is in progress, we were not able to verify FAA’s estimates of FOQA
                     costs and savings. The cost and savings figures are preliminary and may change as more data are
                     gathered.



                     Page 7                        GAO/RCED-98-10 Flight Operational Quality Assurance Programs
                                         B-275990




Table 1: Estimated Total Annual Costs,
by Fleet Size                                                                    15 aircraft           50 aircraft           100 aircraft
                                         Equipment costs                           $98,500               $259,000              $492,000
                                         Personnel costs                           385,000                500,000                775,000
                                         Total annual costs                       $483,500               $759,000            $1,267,000
                                         Note: Equipment costs are based on the invoice price paid to vendors in the FOQA
                                         demonstration project. To annualize the figures, the equipment purchase costs have been spread
                                         over a 5-year period. Personnel costs are based primarily on estimates of FOQA management,
                                         analysis, monitoring, and engineering costs from an airline participating in the demonstration
                                         project.

                                         Source: UTRS.



                                         The cost-benefit study estimates that airlines will reduce their
                                         expenditures for fuel and maintenance as well as reduce the number of
                                         accidents and incidents over time, avoiding their associated costs.
                                         Because FOQA programs analyze additional data on aircraft systems and
                                         engine conditions, airlines are better able to achieve optimum fuel
                                         consumption and avoid unneeded engine maintenance. Although more
                                         difficult to quantify and directly relate to a FOQA program, enhanced safety
                                         should result in lower costs over time as a result of accidents avoided and
                                         lower insurance premiums. Table 2 summarizes the estimated annual
                                         savings for fleet sizes of 15, 50, and 100 aircraft. Fuel savings and engine
                                         savings figures are based on estimates of a 0.5-percent reduction in fuel
                                         consumption and a 1-percent reduction in engine maintenance costs. The
                                         safety savings figure is based on a hypothetical 1-percent reduction in the
                                         annual costs incurred from accidents. FAA’s contractor based its safety
                                         savings calculation on a current loss rate of 2 aircraft per million
                                         departures at a cost of $150 million for each loss.




                                         Page 8                      GAO/RCED-98-10 Flight Operational Quality Assurance Programs
                                  B-275990




Table 2: Estimated Total Annual
Savings, by Fleet Size                                                    15 aircraft           50 aircraft           100 aircraft
                                  Fuel savings                             $145,800               $486,000               $972,000
                                  Engine savings                             300,000             1,000,000              2,000,000
                                  Safety Savings                              49,500               165,000                330,000
                                  Total annual savings                     $495,300             $1,651,000             $3,302,000
                                  Note: Fuel and engine savings were estimated on the basis of discussions with an airline
                                  participating in the FOQA demonstration project. Safety savings were estimated on the basis of
                                  information from a European airline with a long-term FOQA program. Savings estimates were also
                                  based on an assumption of 3,000 flight hours per aircraft per year.

                                  Source: UTRS.



                                  According to these annual cost and savings estimates, FOQA would result in
                                  net annual savings of $11,800 for 15 aircraft, $892,000 for 50 aircraft, and
                                  $2,035,000 for 100 aircraft. See table 3.

Table 3: Estimated Net Annual
Savings, by Fleet Size                                                    15 aircraft           50 aircraft           100 aircraft
                                  Total annual costs                       $483,500               $759,000             $1,267,000
                                  Total annual savings                       495,300             1,651,000              3,302,000
                                  Net annual savings                         $11,800              $892,000             $2,035,000
                                  Source: UTRS.




                                  Although airline officials, pilot organizations, and FAA officials recognize
Factors Impeding                  the potential for improving safety and operations through FOQA programs,
Implementation and                airline officials and representatives of the pilot organizations were
Actions to Overcome               unanimous in their view that data protection issues need to be resolved.
                                  Both airline officials and pilots’ representatives stated that the lack of
Impediments                       protections for FOQA data has been a major contributor to pilot unions’
                                  reluctance to sign FOQA agreements with airlines and airlines’ reluctance to
                                  implement FOQA programs.

                                  According to the Flight Safety Foundation’s 1992 report, the greatest
                                  impediment to the implementation of FOQA in the United States is
                                  associated with the “protection of data from use for other than safety and
                                  operational improvement purposes.” Basically, airline managers and pilots
                                  have three concerns: (1) that the information may be used in
                                  enforcement/discipline actions, (2) that such data in the possession of the
                                  federal government may be obtained by the public and the media through




                                  Page 9                      GAO/RCED-98-10 Flight Operational Quality Assurance Programs
              B-275990




              the provisions of the Freedom of Information Act (FOIA), and (3) that the
              information may be obtained in civil litigation through the discovery
              process. Similar concerns have been expressed in connection with other
              programs under which information is submitted voluntarily to FAA.


Enforcement   Representatives from each of the major airlines as well as the unions that
              represent pilots from the major airlines—the Air Line Pilots Association,
              the Allied Pilots Association, the Independent Association of Continental
              Pilots, and the Southwest Airlines Pilot Association—told us that the
              airlines and pilots fear the possibility that FOQA data might be used against
              them in FAA enforcement proceedings. In addition to these concerns,
              pilots’ representatives were concerned that airline managers could use
              FOQA data to punish or discipline pilots.


              FAA  Enforcement. Many U.S. airlines and their pilots appear frustrated with
              FAA’s delay in issuing a regulation implementing the nonenforcement
              policy articulated in a February 1995 policy letter from the Administrator
              to the Air Line Pilots Association and the Air Transport Association. FAA’s
              letter said that no enforcement action will be taken on the basis of the
              information gained through FOQA. Specifically the letter stated:

              “The FAA commits that it will not use information collected by a carrier in an FOQA program
              to undertake any certificate or other enforcement action against an air carrier participating
              in such a program or one of its individual employees. Notwithstanding, the FAA reserves its
              rights to use, for any other purpose, information obtained from sources other than FOQA,
              including flight-recorder parameters specifically required by the Federal Aviation
              Regulations. The limitation on the use of information applies only to information collected
              specifically in an FOQA program.”


              In an April 1997 letter to the Air Transport Association’s FOQA Steering
              Committee, the Director of FAA’s Flight Standards Service said that the
              1995 policy letter will remain in effect until the regulation on enforcement
              is issued. The letter stated that a proposed rulemaking setting forth FAA’s
              enforcement protection policy should be ready by the end of 1997.

              According to airline officials and a pilot union’s representative, FAA’s delay
              in promulgating an enforcement regulation has hampered efforts to reach
              agreement with some pilot unions and threatens the continuance of
              agreements already reached. One of the issues facing FAA is how broad the
              enforcement protection should be. FAA attorneys have concluded that it is
              beyond the scope of FAA’s authority and in violation of its statutory duties




              Page 10                    GAO/RCED-98-10 Flight Operational Quality Assurance Programs
B-275990




to issue a regulation that precludes the agency from taking action if FOQA
data reveal that an airplane was not in a condition for safe flight or that a
pilot lacked qualifications. Pilots’ representatives, however, have cited the
precedent of FAA’s cockpit voice recorder regulation that prohibits the
agency from using the record in enforcement actions without exceptions.10


FAA officials told us that the agency is trying to find the proper balance
between carrying out its enforcement responsibilities and providing
incentives for implementing safety programs and sharing information with
FAA. In similar programs, such as the Aviation Safety Reporting Program,
Air Carrier Voluntary Disclosure Reporting Procedures, and Aviation
Safety Action Programs,11 under which safety information is voluntarily
submitted, the agency has a policy of addressing alleged violations through
administrative actions or forgoing and/or waiving the imposition of any
legal enforcement if certain qualifying criteria are met. These programs are
intended to encourage prompt reporting of violations, sharing of important
safety information, and pilot training to enhance future compliance. While
the qualifying criteria differ for each program, these programs exclude
actions that are deliberate or demonstrate or raise questions of
qualifications. Generally, the parameters of the programs, including the
qualifying criteria, are spelled out in the governing advisory circular. It is
FAA’s belief that by offering incentives, such as forgoing legal enforcement
actions under certain conditions, more problems may be reported and
ultimately corrected than could be discovered through other means, such
as inspections.

Airline Enforcement. Airline managers are working with their respective
pilot unions to enter into data-use agreements that include individual
protection provisions. According to the Flight Safety Foundation study,
data-use agreements with pilot associations have existed since flight data
recorders were first required in the late 1950s. Having such an agreement
is a precursor to becoming a full partner in the FOQA demonstration
project. Generally, these agreements provide, among other things, the
company’s assurance not to use the recorded flight data for punitive or

10
  The cockpit voice recorder regulation provides that: “The Administrator does not use the cockpit
voice recorder record in any civil penalty or certificate action.” 14 C.F.R. 91.609(g). FAA’s regulations
also provide enforcement protection with some qualifications to information collected under the
Aviation Safety Reporting Program. Specifically, the regulation provides that “The Administrator of the
FAA will not use reports submitted to the National Aeronautics and Space Administration under the
Aviation Safety Reporting Program (or information derived therefrom) in any enforcement action
except information concerning accidents or criminal offenses which are wholly excluded from the
Program.” 14 C.F.R. 91.25.
11
  See app. IV for a description of these programs.



Page 11                        GAO/RCED-98-10 Flight Operational Quality Assurance Programs
                         B-275990




                         disciplinary action against a crew member, or as evidence in any
                         proceeding. Also, to ensure the protection of the company’s employees,
                         the data-use agreements generally provide for the de-identification of the
                         information as soon as possible, usually within 7 days. This practice
                         ensures the confidentiality and anonymity of the flight crew members
                         participating in the program.


Freedom of Information   Both airlines and pilots are concerned that FOQA data could become public
Act Requests             and available to the media through the federal FOIA, if such data are
                         provided directly to FAA.12 The federal FOIA sets forth a policy of broad
                         disclosure of government documents to ensure “an informed citizenry,
                         vital to the functioning of a democratic society.” NLRB v. Robbins Tire &
                         Rubber Co., 437 U.S. 214, 242 (1978). The Congress understood, however,
                         that “legitimate governmental and private interest could be harmed by
                         release of certain types of information.” FBI v. Abramson, 456 U.S. 615, 621
                         (1982). Accordingly, the act provides for nine categorical exemptions.

                         In the past, safety information voluntarily submitted to FAA, for example
                         under Air Carrier Voluntary Disclosure Reporting Procedures, has been
                         protected under exemption 4 of FOIA. Exemption 4 protects trade secrets
                         and commercial or financial information obtained from a person that is
                         privileged or confidential. Airline officials and pilots’ representatives
                         expressed concern that FOQA data may not be protectable under this
                         exemption.

                         Recently, the Congress enacted the Federal Aviation Reauthorization Act
                         of 1996, which contains a provision that protects voluntarily submitted
                         information under certain circumstances. Specifically, under the
                         provision, notwithstanding any other provision of law, the Administrator is
                         barred from disclosing voluntarily provided safety- or security-related
                         information if the Administrator finds that

                         “(1) the disclosure of the information would inhibit the voluntary provision of that type of
                         information and that the receipt of that type of information aids in fulfilling the
                         Administrator’s safety and security responsibilities; and (2) withholding such information
                         from disclosure would be consistent with the Administrator’s safety and security
                         responsibilities.” 49 U.S.C. 40123.13



                         12
                          Currently, airlines provide no FOQA data to FAA. Rather, FAA reviews aggregated trend information
                         on the airlines’ premises.
                         13
                          A similar provision was included in the National Transportation Safety Board Amendments of 1996 to
                         protect information that is voluntarily submitted to the Board.



                         Page 12                      GAO/RCED-98-10 Flight Operational Quality Assurance Programs
                             B-275990




                             The provision also requires the Administrator to issue regulations to
                             implement the section.

                             The House report accompanying this legislation noted with approval the
                             data-sharing programs such as FOQA and the Committee’s intent to
                             encourage and promote these sorts of innovative safety programs. The
                             report provides that information submitted under these programs would
                             arguably be protected from release under exemption 4 of FOIA; however,
                             the report notes that such a decision to withhold the information would be
                             discretionary with the agency. The report states that to provide assurance
                             that such information is not publicly released, the legislation would
                             prohibit FAA from disclosing voluntarily submitted safety information.
                             According to the report, this protection should “alleviate the aviation
                             community’s concerns and allow the data-sharing safety programs to move
                             forward.” Moreover, the report noted that the provision would not reduce
                             the information available to the public, since the public does not receive
                             the data. Rather, the report states that public safety will be enhanced by
                             the increase in FAA’s understanding of ongoing trends in operations and
                             technologies.14

                             FAA is currently working on a rulemaking procedure that will prohibit the
                             release of voluntarily submitted safety data through FOIA.15 It is expected
                             that the rulemaking will provide the procedures that the agency will use in
                             making the required determinations. It is also expected that FOQA data will
                             be proposed as qualifying for the protection. According to an FAA attorney,
                             the determinations for the FOQA program may be included in the notice of
                             proposed rulemaking on the FOQA nonenforcement policy. The anticipated
                             FOIA rulemaking and the subsequent findings to include the FOQA program
                             within the protection should help mitigate or resolve the industry’s fears
                             about the possible disclosure of FOQA data through FOIA requests if FOQA
                             data are provided directly to FAA.


Discovery Process in Civil   Some airline officials have told us that although they want to improve
Litigation                   aviation safety by implementing a FOQA program, the voluntary collection

                             14
                               H.R. Rep. No. 104-714 pt. 1 at pp. 40-41 (1996).
                             15
                               In the Final Report of the White House Commission on Aviation Safety and Security, dated Feb. 12,
                             1997, a recommendation was made that FAA should work with the aviation community to develop and
                             protect the integrity of standard safety databases that can be shared in accident prevention programs.
                             The report stated that FAA needed to expeditiously complete rulemaking to implement the voluntary
                             disclosure protection provision and that the agency should assess the adequacy of the new legislative
                             authority and implementing regulation 1 year after the regulations take effect. The report further
                             stated that any necessary regulatory or legislative modifications identified at that time should be
                             promptly addressed.



                             Page 13                         GAO/RCED-98-10 Flight Operational Quality Assurance Programs
B-275990




of data may potentially expose airlines to greater liability in civil litigation.
FOQA data may indicate conditions outside of desired operating
procedures. Airline officials and pilot representatives told us that they are
concerned that through broad discovery rules, FOQA data could be
inappropriately used or disclosed to the public. The general purpose of
discovery is to remove surprise from trial preparation so that parties may
obtain the evidence necessary to evaluate and resolve their dispute.16
Since FOQA data are retained at the airlines and are not currently provided
directly to FAA, the focus has been on the airlines’ ability to protect the
information.

Under federal rules, parties in litigation in federal court are authorized to
obtain discovery of any matter, not privileged, which is relevant to the
subject matter involved in the pending action, whether it relates to the
claim or defense of the party seeking discovery or to the claim or defense
of any other party. Generally, privileges are narrowly construed and in
some cases are qualified. However, even in the absence of a privilege, a
district court has broad discretion under the federal rules to issue an order
to protect a person from annoyance, embarrassment, oppression, or undue
burden or expense if there is a good cause for issuance of the order.
Courts generally invoke a balancing test to decide when a protective order
is appropriate and how it is to be applied.17

In two recent cases, the airlines have tried to convince federal courts that
voluntarily collected safety data similar to FOQA data should be protected
from discovery or, at the very least, covered under a protective order.18 In
both cases, the courts sought to achieve a balance between the airlines’
desire to protect the information and the plaintiffs’ right to a fair trial. In
the first case, the court rejected a claim that the information should be
protected under the self-critical evaluation privilege but limited the
possible uses of the documents it ordered to be produced.19 This
determination was effected through a protective order. In the other case,

16
  6 Moore’s Federal Practice, section 26.02 (Matthew-Bender 3d ed.).
17
  6 Moore’s Federal Practice, ch. 26 (Matthew-Bender 3d ed.).
18
 Court Order of Oct. 26, 1995, In re: Air Crash at Charlotte, North Carolina, on July 2, 1994, MDL
Docket No. 1041 (D.S.C. 1995) (the court rejected the claim of self-critical evaluation privilege); but
see, Court Order of Nov. 14, 1995, In re: Air Crash at Charlotte, North Carolina, on July 2 1994, MDL
Docket No. 1041 (D.S.C. 1995) (the court issued a protective order); and In re: Air Crash Near Cali,
Columbia, on Dec. 20, 1995, 959 F. Supp. 1529 (S.D. Fla. 1997) (the court rejected the claim of
self-critical evaluation privilege but recognized a new qualified privilege for the American Airlines
Safety Action Program). For a more detailed discussion of these court cases, see app. V.
19
 The self-critical evaluation privilege, when recognized, protects documents that reflect an internal
self-analysis.



Page 14                        GAO/RCED-98-10 Flight Operational Quality Assurance Programs
                  B-275990




                  the court also rejected the claim of self-critical evaluation privilege but at
                  the same time recognized a new qualified privilege for information
                  collected under a partnership program with FAA, the American Airlines
                  Safety Action Program.20

                  Although airlines are generally pleased with the court’s decision to grant a
                  qualified privilege to Aviation Safety Action Program materials, it is not
                  clear whether other courts will recognize this new privilege or extend it to
                  other safety and security information that has been voluntarily collected.
                  Nor is there a guarantee that FOQA data or other similar information, if
                  found not to be privileged, would be covered under a protective order.
                  However, we found no instances to date in which FOQA data have been
                  subject to a discovery request. This situation may result from the fact that
                  airlines are just beginning to institute FOQA programs. However, some of
                  the pilot union officials we spoke with noted that discovery is a concern
                  because of the potentially large amounts of data that will be collected.
                  While some in the aviation community believe that one way to ensure
                  protection would be through legislation, there does not appear to be a
                  consensus to seek legislation at this time.21 Concern has been expressed
                  that the failure of a legislative effort may adversely affect how courts treat
                  voluntarily collected safety information.

                  In the event that FAA does receive FOQA data directly, according to FAA
                  attorneys, it has provisions in place for dealing with requests from private
                  litigants for documents in the agency’s possession. FAA attorneys noted
                  that a request for records from a private litigant, when the agency is not a
                  party to the action, will generally be treated as a FOIA request (see 49
                  C.F.R. 9.13). If the agency is a party to the litigation, FAA will seek to
                  protect the information, if appropriate, under a claim of government
                  privilege and, if that fails, to release the information under a protective
                  order.


                  We provided copies of a draft of this report to the Department of
Agency Comments   Transportation and FAA for their review and comment. We met with
                  officials, including FAA’s Deputy Associate Administrator for Regulation
                  and Certification and the demonstration project’s program manager. They


                  20
                   Thus, the court provided that the plaintiff could come forward with a persuasive showing of need and
                  hardship. In such case, the court would review the voluntarily collected information in camera and
                  evaluate whether the plaintiff’s interests overcome the powerful interest that weighs in favor of
                  preserving the confidentiality of the information. No such showing was made in this case.
                  21
                    Limited legislative protection has been provided for cockpit voice recorders, 49 U.S.C. 1154.



                  Page 15                        GAO/RCED-98-10 Flight Operational Quality Assurance Programs
              B-275990




              agreed with the report and provided several technical corrections, which
              were incorporated into the report.


              To obtain the information in this report, we reviewed FAA’s FOQA
Scope and     demonstration project’s requirements, policies, and plans to assist airlines
Methodology   in implementing FOQA programs. We discussed specific details of the
              project with FAA’s Deputy Associate Administrator for Regulation and
              Certification as well as the demonstration project’s program manager and
              contractor. We conducted interviews with FAA and National Aeronautics
              and Space Administration officials responsible for developing the Aviation
              Performance Measuring System. We discussed FOQA issues with the
              National Transportation Safety Board. We interviewed representatives of
              each of the 10 largest passenger airlines: Alaska, America West, American,
              Continental, Delta, Northwest, Southwest, Trans World, United, and US
              Airways; representatives of each of the four unions—the Air Line Pilots
              Association, the Allied Pilots Association, the Independent Association of
              Continental Pilots, and Southwest Airlines Pilot Association—representing
              the pilots of these airlines; and United Parcel Service. We also conducted
              interviews with the Air Transport Association, the Flight Safety
              Foundation, and the vendors providing hardware and software for the
              demonstration project. Last, we interviewed and collected information
              from foreign airlines and Britain’s Civil Aviation Authority on their
              respective FOQA efforts.

              We performed our work primarily at FAA headquarters in Washington, D.C.,
              and conducted our evaluation from January through October 1997 in
              accordance with generally accepted government auditing standards.


              As requested, we plan no further distribution of this report until 30 days
              after the date of this letter unless you publicly announce the report’s
              contents earlier. At that time, we will send copies to the appropriate
              congressional committees and the Department of Transportation and FAA.




              Page 16               GAO/RCED-98-10 Flight Operational Quality Assurance Programs
B-275990




We will also make copies available to others upon request. Please call me
at (202) 512-2834 if you have any questions about this report. Major
contributors to this report are listed in appendix VI.




John H. Anderson, Jr.
Director, Transportation Issues




Page 17               GAO/RCED-98-10 Flight Operational Quality Assurance Programs
Contents



Letter                                                                                                  1


Appendix I                                                                                             20

The FOQA Concept
and Its
Implementation in the
United States
Appendix II                                                                                            30

U.S. and Foreign
Airlines With FOQA or
FOQA-TYPE
Programs
Appendix III                                                                                           31

FAA’s Related
Technical Programs
Appendix IV                                                                                            33

FAA’s Voluntary Safety
Reporting: Selected
Programs
Appendix V                                                                                             35

Discovery-Related
Court Actions
Appendix VI                                                                                            38

Major Contributors to
This Report
Tables                   Table 1: Estimated Total Annual Costs, by Fleet Size                           8
                         Table 2: Estimated Total Annual Savings, by Fleet Size                         9




                         Page 18              GAO/RCED-98-10 Flight Operational Quality Assurance Programs
Contents




Table 3: Estimated Net Annual Savings, by Fleet Size                           9




Abbreviations

APMS          Aviation Performance Measuring System
AQP           Advanced Qualification Program
ASAP          Aviation Safety Action Programs
ASRP          Aviation Safety Reporting Program
ASRS          Aviation Safety Reporting System
BA            British Airways
DEMOPROJ      FOQA demonstration project
DFDAU         digital flight data acquisition unit
FAA           Federal Aviation Administration
FAR           Federal Aviation Regulation
FOIA          Freedom of Information Act
FOQA          Flight Operational Quality Assurance
FSF           Flight Safety Foundation
GAIN          Global Analysis and Information Network
KLM           Royal Dutch Airlines
NASA          National Aeronautics and Space Administration
PCMCIA        Personal Computer Memory Card International
                    Association
QAR           quick access recorder
UTRS          Universal Technical Resource Services, Inc.


Page 19              GAO/RCED-98-10 Flight Operational Quality Assurance Programs
Appendix I

The FOQA Concept and Its Implementation
in the United States

FOQA’s Background   Flight Operational Quality Assurance (FOQA) had its origin in the use of
                    flight data recorders as mandated by the Civil Aeronautics Administration
                    in 1958. Although the first flight data recorders captured only six
                    parameters—time, airspeed, heading, altitude, vertical acceleration, and
                    time of radio transmission—they were a valuable tool for reconstructing
                    what had occurred before and during accidents. By the 1960s, airlines had
                    begun to monitor data on routine flights. Initially, the monitoring systems
                    captured airworthiness data, but over time they have expanded to include
                    operational data. In the late 1960s, Trans World Airlines began a program
                    to monitor a limited number of parameters related to approaches and
                    landings as flight data recorders received periodic maintenance.

                    At least eight foreign airlines have had FOQA-type programs in operation for
                    over 25 years. A program using data from flight data recorders was begun
                    by British Airways (BA) in 1962 to validate airworthiness criteria. Although
                    limited by today’s standards, BA’s program contained the seeds of a
                    modern, safety-oriented FOQA program. Currently, BA analyzes the flight
                    data from all of the aircraft in its fleet through its Special Events Search
                    and Master Analysis program. Over the years, the number of foreign
                    airlines that have implemented a FOQA-type program has steadily risen.
                    Japan Airlines’ FOQA program of over 15 years includes a printer in the
                    cockpit so that pilots can monitor their own performance. All Nippon
                    Airways began a program to analyze flight data in 1974. Other foreign
                    airlines with established FOQA programs include Scandinavian Airlines
                    System, Royal Dutch Airlines (KLM), and Lufthansa. Many of these airlines
                    are convinced that FOQA is a critical component in their safety efforts and
                    that the program has paid valuable safety dividends over the years.
                    Currently, about 33 foreign airlines and 4 U.S. airlines—United, US
                    Airways, Continental, and Alaska—have implemented FOQA programs (see
                    app. II for the complete list).

                    Recognizing the value of operational flight data and the critical nature of
                    flight crews’ performance in accidents, the Flight Safety Foundation (FSF)
                    proposed and was selected by the Federal Aviation Administration (FAA) in
                    1991 to study FOQA. In its 1992 report on FOQA, FSF said that

                    “The proposal was based on FSF’s conviction, formed by the positive experiences of its
                    international member airlines using FOQA, that the appropriate use of FOQA data by airlines,
                    pilot associations and aircraft and equipment manufacturers would result in a significant
                    improvement of flight safety by identifying operational irregularities that can foreshadow
                    accidents and incidents.”




                    Page 20                    GAO/RCED-98-10 Flight Operational Quality Assurance Programs
                 Appendix I
                 The FOQA Concept and Its Implementation
                 in the United States




                 The FSF study concluded that FOQA must proceed in the United States and
                 that the implementation of FOQA by U.S. airlines would have a more
                 positive impact on Part 121 operational safety than any other human
                 factors program included in FAA’s research and development plans. FSF
                 recommended that FAA promote voluntary FOQA programs by instituting a
                 demonstration program in partnership with industry. In 1992, FAA’s Flight
                 Standards Service proposed funding for a demonstration program. On
                 February 9, 1995, FAA announced its plans for an FAA-industry
                 demonstration project, and the Administrator sent a policy letter to the Air
                 Transport Association and the Air Line Pilots Association stating that FAA
                 would not use FOQA data for enforcement purposes, provided that the
                 airlines met certain requirements.


How FOQA Works   At a minimum, FOQA involves the analysis of flight data on a routine basis
                 to reveal situations requiring corrective actions before problems occur. To
                 institute such a program, airlines need methods to capture flight data,
                 transform the data into the appropriate format for analysis, and generate
                 reports and visualizations to assist personnel in analyzing the data.
                 Although different methods are available, the following describes how a
                 representative FOQA program operates; the descriptions are based on the
                 experience of the four U.S. airlines that have implemented FOQA.

                 Management. A typical program is managed and operated by a FOQA
                 manager, one or more analysts, and a FOQA monitoring team (sometimes
                 referred to as the exceedance guidance team) made up of airline pilots
                 who work on FOQA on a part-time basis. Generally, the majority of the
                 monitoring team’s pilots are also representatives of the pilot union. These
                 individuals manage the FOQA program in strict adherence to the
                 agreements made with the pilot union, most notably on ensuring the
                 confidentiality of pilots’ identities. This group is responsible for defining
                 and refining exceedances and parameters, reviewing and analyzing data,
                 and determining and monitoring corrective actions.

                 Data capture. The first step is the capture of data over the duration of the
                 flight. Flight data comprise snapshots of values or measurements from
                 various aircraft systems. Each data item represents information from a
                 discrete source, such as an instrument or sensor. Generally, these data
                 items are referred to as “parameters.” Examples of parameters are
                 “altitude” or “landing gear position.” Recording rates vary, depending on
                 the parameter, ranging from many times per second to about once per
                 minute.



                 Page 21                 GAO/RCED-98-10 Flight Operational Quality Assurance Programs
Appendix I
The FOQA Concept and Its Implementation
in the United States




Although flight data recorders continuously record, at a minimum,
FAA-mandated parameters during every flight, they typically are not
designed to provide frequent access to their data but rather to survive the
extreme conditions during and after crashes to preserve flight data for
accident investigations. These devices are housed in crash-resistant,
sealed containers designed to withstand high “g” forces, submersion in
water, and fire. Obtaining frequent access to flight data recorders for FOQA
purposes, however, would produce increased wear on internal
mechanisms and result in shortened mechanical life and increased
expense for a very specialized device.22 Also, flight data recorders may not
capture a sufficient number of parameters to be useful for FOQA purposes.
Currently, FAA requires from 16 to 29 parameters to be recorded on flight
data recorders in transport aircraft;23 a FOQA program, however, would
likely capture many more parameters. Typically, the 200-500 parameters
available on modern digital aircraft allow a more comprehensive set of
conditions to be monitored. Finally, flight data recorders hold about 25
hours of flight data, a relatively short time period. Instead, some U.S.
airlines use a device called a quick access recorder (QAR) to record FOQA
data to a removable optical disk or Personal Computer Memory Card
International Association (PCMCIA) card.24 QARs record flight data that are
output from the aircraft’s digital flight data acquisition unit (DFDAU), the
same device that feeds parameters to the flight data recorder. On average,
QARs hold from 100 to 200 hours of flight data.


Data transfer. As aircraft receive periodic servicing, the medium (optical
disk or PCMCIA card) containing flight data is removed from the QAR and
sent to a central location for analysis. A new disk or card is inserted into
the QAR for the next round of flights. Airlines retrieve the data on
schedules ranging from 3 to 20 days.

An alternative to physical recording media is the use of datalink systems to
transmit information directly to the ground-based system, eliminating the
need to retrieve data from the aircraft. Two participating airlines are
investigating the use of automatic wireless data transfer upon landing at
specially equipped airports. Data would be transmitted on a radio

22
 The newer solid-state flight data recorders, however, have no moving parts and would not experience
wear problems. Transferring data from these devices takes several minutes to perform.
23
  Under a recently issued rule, FAA requires the recording of 16 to 29 parameters by the flight data
recorder on all existing transport aircraft, depending on the aircraft model, its internal systems, and its
date of manufacture. Aircraft manufactured after the rule, however, will be required to record 88
parameters within 5 years. 62 Fed. Reg. 38362 (July 17, 1997).
24
 Other airborne data collection systems in use around the world include QARs using tape cartridges
and solid-state devices.



Page 22                        GAO/RCED-98-10 Flight Operational Quality Assurance Programs
Appendix I
The FOQA Concept and Its Implementation
in the United States




frequency link from the aircraft to a receiving station after the aircraft
lands. In turn, a local area network would transfer the data to the ground
analysis station. Data encryption and other methods would be used to
ensure the security of the transmitted FOQA data.

Data processing and analysis. Each airline has a ground analysis system
where airborne collected data are processed and analyzed. The ground
analysis system transforms the raw digital flight records into usable form
for processing, analyzes the flight information, and generates information
on any detected exceedances that represent deviations from normal
operating practices or exceptional conditions.

The flight data analysis component of the ground analysis system
categorizes operational events to be flagged by defining a set of
parameters that indicate normal operating envelopes. The associated
thresholds for these parameters vary by the type of aircraft and associated
operating limits, accepted practices for safe operations, the phase of flight,
and the duration of any irregularity. For example, the threshold of selected
parameters may be defined for various altitudes, e.g., 1,000, 500, 250, and
100 feet, during landing mode events. Typically, 40 to 80 events are defined
and analyzed for a particular aircraft. For example, events might be the
ground speed during taxi or the descent rate during approach. The
analysis software will track the descent over time to calculate a rate in
terms of feet per minute. Depending on the aircraft’s altitude, a descent
rate in excess of specified thresholds will trigger an exceedance. Various
categorization schemes are used to classify the seriousness of the
exceedance. U.S. airlines use two or three categories to describe the
seriousness of exceedances, ranging from minor deviations to major
deviations. Exceedances are typically specified on the basis of a strategy
for identifying those that have the greatest potential for safety and
performance considerations. Once the initial exceedance categories and
associated parameters have been defined and utilized, they are subject to
an ongoing evaluation and refinement process.

The ground analysis software also validates the quality and integrity of the
collected data and filters out any marginal or transitory irregularities.
Ground analysis systems also include protective mechanisms, such as the
de-identification of pilot and specific flight information and user access
privileges based on assigned passwords. As the data are processed, the
flight number and day of the month are removed and saved into a separate
controlled file. This step “de-identifies” the FOQA data.




Page 23                 GAO/RCED-98-10 Flight Operational Quality Assurance Programs
Appendix I
The FOQA Concept and Its Implementation
in the United States




The FOQA monitoring team investigates each exceedance to determine
what occurred and the severity of the exceedance. An analyst will review
the parameter values surrounding the event and other information to
determine if the exceedance was valid or if the exceedance was based on
bad data, a faulty sensor, or some other invalidating factor. For example,
one flight had excessive rudder input on landing that correctly registered
as an exceedance. On closer examination, it was determined that because
the aircraft was making a cross-wind landing, the use of large rudder input
was justified. In this case, the exceedance was deemed invalid and was
removed from the exceedance database.

Depending on the particular circumstances of the exceedance, the pilot
association’s representative may contact the flight crew to gather more
information. After reviewing the situation to determine the exceedance’s
cause, the FOQA monitoring team and pilot association’s representative will
determine any necessary corrective action. Corrective action can range
from additional flight crew training, to revisions of the operating
procedures, to redesigns of equipment.

Trend analysis. On a periodic basis, airlines aggregate and analyze
exceedances over time—for example, the number of unstabilized
approaches at a particular airport per month, over the last 12 months. This
type of analysis provides valuable information to the airline, especially in
terms of whether the airline’s performance is improving, holding steady, or
deteriorating. This look at aggregate exceedances over time provides
airline managers with a new perspective on potential problems that would
not be visible otherwise. On the basis of the trend analysis, airline
managers can take corrective action to reduce or eliminate these
exceedances by focusing on the root causes and making or recommending
changes.

Data retention. Detailed FOQA data, including exceedances, are destroyed
in 30 days or less by three of the four U.S. airlines with FOQA programs.
Trend data, however, are kept indefinitely.

Aircraft equipping decisions. The U.S. airlines with active FOQA programs
have each equipped a portion of their available fleets with QARs. They
began their programs by equipping their more modern, technically
advanced aircraft with QARs—late-generation aircraft already contain the
sensors and advanced digital systems that acquire and control many more
flight data parameters than earlier-generation aircraft. Generally, these
airlines do not plan to equip any of their older, analog-based aircraft, such



Page 24                 GAO/RCED-98-10 Flight Operational Quality Assurance Programs
                         Appendix I
                         The FOQA Concept and Its Implementation
                         in the United States




                         as Lockheed L-1011, McDonnell Douglas DC-9 and DC-10, and Boeing 727,
                         737-100, and 737-200, with QARs to record flight data because these aircraft
                         would be expensive to retrofit and because the airlines plan to retire many
                         of them in the near future.

                         Several U.S. airlines plan to equip all new aircraft with QARs or other
                         technology to capture FOQA data. Some new aircraft, for example, are
                         delivered with QARs as standard equipment. Airlines cited several
                         advantages in having new aircraft delivered with factory-equipped QARs.
                         One advantage is that aircraft are not taken out of service to be retrofitted
                         with equipment. Another advantage is that the additional cost of a QAR can
                         be spread over the finance period of the new aircraft.

                         Depending on the specific goals of a FOQA program, an airline may wish to
                         equip some or all of its fleet to collect flight data. If a program’s goal is to
                         identify broad trends in flight operations and safety, the airline may
                         choose to equip only a portion of its fleet. If a program’s goal, however, is
                         to more closely monitor the flight operations and performance of
                         individual aircraft, the airline may want to equip more or all of its fleet.
                         For an airline that begins by equipping only a portion of its fleet, more
                         aircraft will likely be added to the program so that these data can be
                         monitored as its FOQA program matures and efficiency and maintenance
                         functions are added to the program. Some U.S. airlines, for example, are
                         planning to use FOQA data to cut aircraft maintenance costs by more
                         closely monitoring engine conditions and fuel consumption.


The FOQA Demonstration   On July 11, 1995, FAA awarded a 2-year contract25 to execute a FOQA
Project                  demonstration project, referred to as DEMOPROJ by FAA, to Universal
                         Technical Resource Services, Inc. (UTRS), of Cherry Hill, New Jersey. The
                         contract stated that

                         “The goal of DEMOPROJ is to facilitate the start-up of the FOQA initiative and to
                         comprehensively assess the cost-benefits and safety enhancement effectiveness of an
                         implemented FOQA program in which airlines voluntarily employ in-flight recorded data to
                         routinely monitor their flight operations.”


                         UTRS facilitated the establishment of collaborative partnerships between
                         FAA, UTRS, and interested airlines. Airlines may participate in DEMOPROJ at
                         one of three levels within the project, ranging from attending meetings and
                         expressing interest to a full partnership with FAA. Level 3 participation

                         25
                           The term of the contract was later revised to 3 years.



                         Page 25                        GAO/RCED-98-10 Flight Operational Quality Assurance Programs
Appendix I
The FOQA Concept and Its Implementation
in the United States




refers to the airlines that have not yet established an official FOQA program
but attend meetings to learn about FOQA. At Level 2, the airlines already
have their own equipment or will acquire equipment using airline funding,
but they allow UTRS to monitor and document their program. Level 1
describes a full partnership in which equipment and software are provided
through DEMOPROJ. Currently, 11 airlines are participating in DEMOPROJ. The
airlines participating at Level 1 are United, US Airways, and Continental.
All other participating airlines in DEMOPROJ are at Level 3: Alaska, America
West, Delta, Northwest, Trans World, Southwest, Continental Express, and
United Parcel Service.

The airline participants were selected on the basis of a number of
characteristics, including financial stability, management commitment,
resource commitment, fleet characteristics, fleet size, aircraft availability,
and an approved implementation and operation plan. Additionally, airlines
are required to sign nondisclosure and cooperation agreements that define
the treatment of confidential and proprietary information, enumerate data
access control and security provisions, and specify the responsibilities and
contributions of each party. Participating airlines also had to secure
agreements with their pilot associations for the collection and analysis of
flight data. These airlines made the commitment to record and process
FOQA data on all scheduled flights that are equipped with FAA-supplied
equipment, participate in periodic project reviews, and allow UTRS to
interview airline personnel during the project to document procedures,
problems, issues, and solutions.

UTRS  assisted airlines in determining the equipment best suited to their
needs, acquiring the equipment, and delivering it for installation by the
airlines. Hardware and software were selected from commercially
available, off-the-shelf sources. As part of this effort, the contractor
developed an Equipment Overview to facilitate the airlines’ analysis and
selection of available equipment.

UTRS also monitors and documents the airlines’ FOQA demonstration
programs’ policies, procedures, usage, and effectiveness. The contractor is
collecting and analyzing information on how each airline is implementing
FOQA, including data processing and analysis; the retention of detail and
trend data; the selection of flight data parameters; and the adjustment of
threshold values, system effectiveness, technical problems, and resource
information for establishing and maintaining a FOQA program. These
findings are integrated and disseminated among participants throughout
the study. UTRS is also collecting information about the projects’ costs and



Page 26                 GAO/RCED-98-10 Flight Operational Quality Assurance Programs
                            Appendix I
                            The FOQA Concept and Its Implementation
                            in the United States




                            anticipated benefits. The contractor is determining how each airline
                            transforms FOQA data into information and how this information is used in
                            the airline’s decision-making. UTRS holds periodic meetings for all partners
                            to promote the sharing of information and lessons learned.

                            UTRS, with airlines’ and pilot associations’ involvement, is developing a
                            FOQA advisory circular to provide information and guidance to airlines on
                            how to design, implement, and maintain a FOQA program. This document is
                            scheduled to be issued approximately 90 days after FAA issues its proposed
                            rulemaking on enforcement policy in connection with FOQA. UTRS is also
                            developing a cost-benefit analysis that will provide estimates of (1) the
                            costs that an airline would incur when starting and maintaining a FOQA
                            program and (2) potential savings. The cost-benefit study is scheduled to
                            be completed in January 1998.

                            UTRS  will issue a technical report and a set of FOQA guidelines in June 1998.
                            The technical report will be an overall description of the technical effort to
                            implement FOQA, summarizing the airlines’ experiences with commercially
                            available equipment and systems. The FOQA guidelines will synthesize the
                            airlines’ experiences in implementing FOQA with a view toward helping
                            new airlines learn from the airlines that have implemented a FOQA
                            program. The guidelines will include information on (1) designing a FOQA
                            program; (2) the start-up and initial operation of a system; (3) the use of
                            FOQA for trend analysis, knowledge building, and decision-making; and
                            (4) critical success factors for implementing a FOQA program.

                            In fiscal years 1995 through 1997, according to the FAA FOQA program
                            manager, FAA allocated $5.5 million for DEMOPROJ. The manager stated that,
                            as of September 26, 1997, DEMOPROJ had expended $2.1 million, including
                            $1.1 million for the purchase of hardware and software for the three Level
                            1 airline participants. FAA plans to pursue follow-on development focused
                            on the acquisition and use of FOQA information by FAA for safety monitoring
                            purposes.


U.S. Airlines With Active   Currently, four U.S. airlines have active FOQA programs: United Airlines,
FOQA Programs               US Airways, Continental Airlines, and Alaska Airlines. These airlines have
                            equipped a number of their aircraft with QARs, from 7 aircraft at Alaska
                            Airlines to 52 aircraft at United Airlines. The number of parameters
                            continuously recorded on the QARs range from about 38 to over 300,
                            depending on the airline and the type of aircraft.




                            Page 27                 GAO/RCED-98-10 Flight Operational Quality Assurance Programs
Appendix I
The FOQA Concept and Its Implementation
in the United States




United Airlines. United Airlines has the largest and longest-running FOQA
program of any U.S. airline, begun in 1995. As of August 1997, United had
52 aircraft equipped with QARs and had collected FOQA data on over 25,000
flights. The aircraft currently equipped include Boeing 737-500s and 777s
and Airbus 319s and 320s. United plans to equip over 120 aircraft by 1999,
including all new aircraft currently on order. DEMOPROJ has funded the
purchase of QARs to equip 15 Boeing 737-500s and additional data analysis
packages and computer equipment to run on systems that United had
already established. The remainder of the hardware and software was
purchased by United, which has been tracking and correcting exceedance
events for more than a year. United has identified and taken corrective
action to reduce the incidence of a number of safety- and
maintenance-related exceedances.

US Airways. US Airways has 23 QAR-equipped aircraft. Its program, begun
in September 1996, has collected FOQA data on over 18,000 flights to date.
Aircraft equipped include Boeing 737-400s and 767s. US Airways, however,
characterizes its program as being in the data collection and
trouble-shooting phase and just beginning the data analysis and trending
phase. DEMOPROJ has funded the purchase of QARs to equip 15 Boeing
737-400s and a ground analysis system. Six additional 737-400s have been
equipped with QARs paid for by a separate FAA program, the Structural
Loads Program (see app. III). In addition to these aircraft, US Airways is in
the process of purchasing QARs and equipping 12 Boeing 767s. Data from
all QARs are being accessed by both programs. DEMOPROJ has also funded a
trial program of a wireless ground datalink system with five specially
equipped Boeing 757s.

Continental Airlines. Continental has equipped 15 Boeing 737-500s with
QARs. In addition, Continental plans to equip with QARs all new aircraft on
order. These include Boeing 737-500s, –600s, –700s, and –800s as well as a
number of 757s. Begun in December 1996, Continental’s program has
analyzed the flight data from over 11,000 flights to date. According to the
program manager, this program is in the data collection phase and will
soon be making the transition to the data analysis and trending phase.

Alaska Airlines. Alaska Airlines has equipped six McDonnell Douglas
MD-80s and one Boeing 737-400 with QARs. In addition, Alaska has
equipped a flight simulator with equipment to record hundreds of flight
parameters. Begun in July 1996, the program has analyzed the flight data
from over 5,000 flights to date. Still in the early stages of the program,




Page 28                 GAO/RCED-98-10 Flight Operational Quality Assurance Programs
Appendix I
The FOQA Concept and Its Implementation
in the United States




Alaska plans to “go slow” and refine its program. Alaska’s FOQA manager
said that the airline may eventually equip every aircraft in its fleet.

Unlike United, US Airways, and Continental, which are Level 1
participants in DEMOPROJ, Alaska is a not yet a full partner in DEMOPROJ
because it has only recently secured the required agreement with its pilot
union on FOQA. The airline, however, has received six QARs and a ground
analysis system from FAA’s Structural Loads Program (see app. III). Alaska
uses the equipment and analysis system for both the Structural Loads
Program and FOQA.




Page 29                 GAO/RCED-98-10 Flight Operational Quality Assurance Programs
Appendix II

U.S. and Foreign Airlines With FOQA or
FOQA-TYPE Programs

               Adria Airways
               Aeroflot
               Air Afrique
               Air France
               Air Inter
               Air Liberte
               Alaska Airlines
               All Nippon Airways
               Asiana Airlines
               Balkan Airlines
               Britannia
               British Airways
               British Midland
               Cathay Pacific Airways
               China Airlines
               China Southern Airlines
               China South West Airlines
               Continental Airlines
               Emirates
               Ethiopian Airlines
               EVA Airways
               Garuda Indonesia
               GB Airways
               Gulf Air
               Japan Airlines
               Royal Dutch Airlines (KLM)
               Kuwait Airways
               Lufthansa
               Qantas Airways
               Saudia Arabian Airlines
               Scandanavian Airline System (SAS)
               Singapore Airlines
               TAP Air Portugal
               Thai Airways International
               United Airlines
               US Airways
               Wideroe26

               Source: GAO and The Flight Data Company, Ltd.




               26
                 The list may not be comprehensive.



               Page 30                      GAO/RCED-98-10 Flight Operational Quality Assurance Programs
Appendix III

FAA’s Related Technical Programs


               Aviation Performance Measuring System. In 1993, FAA contracted with the
               National Aeronautics and Space Administration (NASA) to establish and
               demonstrate the feasibility of developing an Aviation Performance
               Measuring System (APMS). The objective of the APMS effort is to develop
               tools and methodologies to allow large quantities of flight data to be
               processed in a highly automated manner to address questions relating to
               operational performance and safety. APMS is concerned with converting
               flight data into useful safety information in support of the national air
               transport system, airlines, and air crews. Although concerned with all
               aspects of flight operations, APMS primarily will develop an objective
               method for continuously evaluating air crews’ technical performance in
               support of FOQA and the Advanced Qualification Program (discussed
               below).

               Current FOQA programs focus on exceedances; APMS, however, will expand
               FOQA’s scope by utilizing all flight data. The tools will facilitate multiple
               functions, including the acquisition of flight data, their storage in a
               database management system, the study of statistical characteristics and
               trends, the development of “data mining” techniques, and better methods
               of visualizing flight data. APMS will also investigate flight animation
               capabilities to assist flight crews in replaying and understanding
               exceedances. Finally, APMS will facilitate the sharing of data among
               databases, products, and interested parties. According to NASA officials,
               one of the most important components to be developed by APMS is a risk
               assessment tool to measure how much risk is associated with certain
               activities, for example, the riskiness of flights to/from certain airports.

               After APMS began in 1993, the project documented the status of the
               technologies, systems, and software used by foreign airlines with FOQA
               programs. According to the NASA project manager, the project has
               conducted user needs studies at Alaska Airlines, United Airlines, and US
               Airways and has commitments to conduct user needs studies at America
               West, Trans World Airlines, Comair, and United Parcel Service. The APMS
               team is also building prototype systems at several airlines. Alaska Airlines
               is now in its third prototype APMS system. The project was scheduled to
               begin building the initial prototype system at United Airlines on
               November 1, 1997. Eventually the developed technology will be
               transferred to industry so that a relatively low-cost system will be
               commercially available. APMS management hopes to initiate the transfer of
               this technology to commercial vendors in 12 to 18 months.




               Page 31                GAO/RCED-98-10 Flight Operational Quality Assurance Programs
Appendix III
FAA’s Related Technical Programs




To date, NASA has received $2.9 million in funding from FAA for the
development of APMS. NASA contributed $300,000 to the project in fiscal
year 1997. The extent of future NASA and FAA funding for further
development and implementation of APMS has not yet been determined.

Structural Loads Program. As part of FAA’s Aging Aircraft Research and
Development Program, the Structural Loads Program is a cooperative FAA
and NASA effort to collect information about the external loads to which
airframe components are subjected during flight. The collected data will
be used to develop and maintain an extensive database of transport
aircraft usage to continuously validate and update flight and landing load
airworthiness certification standards on the basis of actual measured
usage. To date, the Structural Loads Program has equipped with QARs six
MD-80s at Alaska Airlines and six Boeing 737-400s at US Airways. Data
collected from these QARs are also being made available for FOQA analysis.

Advanced Qualification Program. FAA’s Advanced Qualification Program
(AQP) is an alternate method of qualifying, training, certifying, and ensuring
the competency of flight crew members and other operations personnel
subject to the training and evaluation requirements of Federal Aviation
Regulation (FAR) parts 121 and 135. AQP’s intent is to achieve the highest
possible standards of individual and crew performance without undue
increases in training costs. FOQA and APMS will be used to continuously
evaluate air crews’ technical skills and airlines’ procedures and training in
support of AQP. For example, FOQA data could be used to identify problems
occurring during recurrent flight simulator training and to highlight
training areas for increased emphasis.

Global Analysis and Information Network. The Global Analysis and
Information Network (GAIN) is a concept being actively explored by the
aviation community to facilitate the analysis, sharing, and dissemination of
aviation safety information with a goal of achieving zero accidents. GAIN
would have many information sources—FOQA information would be one of
the most important. Proposed by FAA in May 1996, GAIN will function as a
“significantly improved operational early warning capability that is
sensitive enough to detect and alert the aviation community to existing
and emerging problems.” Information will be shared among airlines and
manufacturers and at the different functional levels within organizations.
Although GAIN is still in the conceptual phase, the aviation community and
FAA are working to address the needs and concerns of prospective
members as well as explore potential designs for a prototype system.




Page 32                  GAO/RCED-98-10 Flight Operational Quality Assurance Programs
Appendix IV

FAA’s Voluntary Safety Reporting: Selected
Programs

               FAA has implemented a number of voluntary programs involving the
               self-reporting of safety-related information to enhance aviation safety.
               Although these programs involve the reporting of information by people
               instead of by automated systems, they are similar to FOQA in that they
               involve voluntary efforts to identify and correct potential safety problems.
               We have highlighted three such programs.

               Aviation Safety Reporting Program. Established by FAA in 1975 and
               administered by NASA, the Aviation Safety Reporting Program (ASRP)
               promotes the voluntary reporting of problems into the Aviation Safety
               Reporting System (ASRS). Under 14 C.F.R. 91.25, FAA will not use reports
               submitted under the program in any enforcement action (except accidents
               or criminal offenses). Under FAA’s policy, although a finding of violation
               may be made, no sanction will be imposed if (1) the violation was
               inadvertent and not deliberate, (2) the violation did not involve a criminal
               offense or an accident or an action that discloses a lack of qualification or
               competency, (3) the person filing the report has not been found in any
               prior FAA enforcement action to have committed a violation of federal
               aviation regulations or law within a period of 5 years prior to the
               occurrence, and (4) the report was filed within 10 days after the violation.
               AC 00-46D (Feb. 26, 1997), “Aviation Safety Reporting Program.”27

               Air Carrier Voluntary Disclosure Reporting Procedures. Initiated by FAA in
               1990 for air carriers,28 the Voluntary Disclosure Procedures encourage
               airlines to promptly disclose to FAA any instances of noncompliance with
               the requirements for maintenance, flight operations, and security. FAA
               initiated a policy of forgoing civil penalty actions if five conditions are met:
               (1) the certificate holder immediately notifies FAA of the apparent violation
               after detecting it and before the agency learns of it; (2) the apparent
               violation is inadvertent; (3) the apparent violation does not indicate a lack,
               or reasonable question, of the basic qualification of the certificate holder;
               (4) immediate action must have been taken, or begun, upon discovery to
               terminate the conduct that resulted in the apparent violation; and (5) the
               certificate holder must develop and implement a comprehensive solution
               satisfactory to the FAA. AC 120-56 (Jan. 23, 1992), “Air Carrier Voluntary
               Disclosure Reporting Procedures.”




               27
                 Provisions concerning air traffic controllers involved in incidents reported under ASRS are addressed
               in FAA Order 7210.3.G, Facility Operations and Administration.
               28
                According to an assistant chief counsel at FAA, the procedures have since been expanded to include
               production approval holders and repair stations.



               Page 33                       GAO/RCED-98-10 Flight Operational Quality Assurance Programs
Appendix IV
FAA’s Voluntary Safety Reporting: Selected
Programs




Aviation Safety Action Programs. FAA has established several
demonstration Aviation Safety Action Programs (ASAP), including the
USAir Altitude Awareness Program, the Alaska Airlines Altitude
Awareness Program, and the American Airlines Safety Action Program.29
These programs established incentives to encourage the employees of the
air carriers that are participating in the programs to disclose information
and identify possible violations of the Federal Aviation Regulations
without fear of punitive legal enforcement sanctions. FAA has recently
expanded the use of ASAP through the implementation of a 2-year
demonstration program. Under this program, apparent violations will
normally be addressed with administrative action if the apparent
violations do not involve (1) deliberate misconduct, (2) a substantial
disregard for safety and security, (3) criminal conduct, or (4) conduct that
demonstrates or raises a lack of qualifications. For apparent violations not
excluded under an ASAP, neither administrative action nor punitive legal
enforcement actions will be taken against an individual unless there is
sufficient evidence of the violation other than the individual’s
safety-related report. AC 120-66 (Jan. 8, 1997), “Aviation Safety Action
Programs (ASAP).”




29
  The Altitude Awareness programs at USAir and Alaska Airlines were joint programs with the Air Line
Pilots Association and FAA to eliminate altitude deviations. USAir’s program, in operation from
October 1990 through February 1992, and Alaska Airlines’ program, in operation from August 1994
through February 1995, encouraged flight crews to report altitude problems so that corrective action
could be taken. The American Airlines Safety Action Program is a joint program with the Allied Pilots
Association (American’s pilot union) and FAA. Begun in June 1994, the program encourages pilots to
report all types of potential safety problems.



Page 34                       GAO/RCED-98-10 Flight Operational Quality Assurance Programs
Appendix V

Discovery-Related Court Actions


              Airline officials and pilot unions’ representatives are concerned about the
              use of discovery in civil litigation to reveal voluntarily collected safety
              information. In two recent cases, the courts have sought to find a balance
              between the airlines’ desire to protect such information and the plaintiffs’
              right to a fair trial. In one case, the documents were required to be
              produced, but under a protective order. In the other case, the court
              recognized a new qualified privilege.

              In 1995, the United States District Court, District of South Carolina,
              Columbia Division, rejected USAir, Inc.’s30 argument that certain safety
              data were protected under the self-critical evaluation privilege. Court
              Order of Oct. 26, 1995, In re Air Crash at Charlotte, North Carolina, on
              July 2, 1994, MDL Docket No. 1041 (D.S.C. 1995). This privilege, when
              recognized, protects documents that reflect self-analysis.

              The district court noted that the self-critical evaluation privilege is a
              privilege of recent origin and one that is narrowly applied even in those
              jurisdictions where it is recognized. The court described the privilege by
              citing to Dowling v. American Hawaii Cruises, Inc., 971 F.2d 423, 425-426
              (9th Cir. 1992), which explained:

              “[O]ther courts have generally required that the party asserting the privilege demonstrate
              that the material to be protected satisfies at least three criteria: first, the information must
              result from a critical self-analysis undertaken by the party seeking protection; second, the
              public must have a strong interest in preserving the free flow of the type of information
              sought; finally, the information must be of the type whose flow would be curtailed if
              discovery were allowed. . . .To these requirements should be added the general proviso that
              no document will be accorded a privilege unless it was prepared with the expectation that
              it would be kept confidential, and has in fact been kept confidential.”


              The court found that the safety documents did not meet the criteria for the
              privilege. According to the court, the most significant stumbling block for
              the airline was meeting the third criterion—that the flow of the
              information would be curtailed if discovery was allowed. Specifically, the
              court found that the airline industry is highly competitive and tightly
              regulated and that airlines have a keen interest in advancing and
              promoting safety as well as services. Thus, the court reasoned that the
              airlines were likely to conduct internal audits. The court reasoned that
              while the disclosure of such audits to competitors would deter their use in
              the future, disclosure for limited use in litigation is unlikely to have such
              an impact.


              30
                USAir changed its name to US Airways on Feb. 27, 1997.



              Page 35                       GAO/RCED-98-10 Flight Operational Quality Assurance Programs
Appendix V
Discovery-Related Court Actions




Subsequently, the court limited the possible uses of the documents it
ordered to be produced. Specifically, the court ordered:

“. . . plaintiff and their counsel shall be prohibited from disclosing, disseminating or
communicating in any manner to any person or entity not involved in this litigation any
portion of the information contained in those documents. . . . Plaintiff and their counsel
shall be further precluded from utilizing these documents or the information contained in
them for any purpose other than for this multidistrict litigation.


“In furtherance of this order, plaintiffs’ counsel shall insure that each person who is to be
given access to the referenced documents, including plaintiff and their attorneys, shall first
sign a document acknowledging that they are aware of and will comply with this order.
Plaintiffs’ counsel shall maintain a list of those persons which shall be provided to USAir’s
attorney upon request, subject to protection upon application to this court for good cause
shown.” Court Order of Nov. 14, 1995, In Re: Air Crash at Charlotte, North Carolina, on
July 2, 1994, MDL Docket No. 1041 (D.S.C. 1995).


In October 1996, the Supreme Court let stand the district court order
rejecting the airline’s assertion of a self-critical evaluation privilege. 65
U.S.L.W. 3221 (Oct. 8, 1996).

Recently, in another case involving documents prepared by American
Airlines’ employees collected under the American Airlines Safety Action
Program, the United States District Court, Southern District of Florida, on
a motion for reconsideration, also rejected the airline’s self-critical
analysis privilege claim. However, in this case the court recognized a new
qualified privilege to protect these documents. In re Air Crash Near Cali,
Colombia, on Dec. 20, 1995, 959 F. Supp. 1529 (S.D. Fla. 1997).

With respect to the self-critical analysis privilege, the court stated that “the
touchstone of a self-critical analysis is that it is an ’in house’ review
undertaken primarily, if not exclusively, for the purpose of internal quality
control.” In this case, the court rejected the application of the privilege,
finding the following:

“Even assuming that the materials prepared by American’s pilots in conjunction with the
ASAP program may be of a type whose creation might be curtailed if discovery is allowed,
these materials were prepared for dissemination to representatives of entities unaffiliated
with American (a federal regulatory agency and a union).”


The court, however, recognized a new, qualified common law privilege for
the ASAP materials. In recognizing a new privilege, the court considered the




Page 36                     GAO/RCED-98-10 Flight Operational Quality Assurance Programs
Appendix V
Discovery-Related Court Actions




principles for evaluating claims of federal common law privileges recently
articulated in the Supreme Court case, Jaffee v. Redmond, __ U.S. __, 116
S. Ct. 1923 (1996): (1) the “private interest” involved—in other words
whether the dissemination of the information would chill the frank and
complete disclosure of fact; (2) the “public interests” furthered by the
proposed privilege; (3) the “likely evidentiary benefit that would result
from the denial of the privilege;” and (4) the extent to which the privilege
has been recognized by state courts and legislatures.31

The court found that American had met its burden of proving that a
qualified “ASAP privilege” is appropriate. Specifically, the court stated as
follows:

“The ASAP materials in dispute . . . were prepared voluntarily, in confidence and for use in a
discrete, limited context in cooperation with the FAA and the pilot’s union. There is a
genuine risk of meaningful and irreparable chill from the compelled disclosure of ASAP
materials in connection with the pending litigation.”


The court specified that the privilege should be qualified. Accordingly, the
plaintiffs could overcome the privilege with a persuasive showing of need
and hardship. The plaintiffs did not make such a showing in the case.




31
  The court recognized a psychotherapist-patient privilege in this case.



Page 37                        GAO/RCED-98-10 Flight Operational Quality Assurance Programs
Appendix VI

Major Contributors to This Report


                        David E. Bryant, Jr.
Resources,              Thomas F. Noone
Community, and          Robert E. White
Economic
Development Division
                        Mindi Weisenbloom
Office of the General
Counsel




(341514)                Page 38                GAO/RCED-98-10 Flight Operational Quality Assurance Programs
Ordering Information

The first copy of each GAO report and testimony is free.
Additional copies are $2 each. Orders should be sent to the
following address, accompanied by a check or money order
made out to the Superintendent of Documents, when
necessary. VISA and MasterCard credit cards are accepted, also.
Orders for 100 or more copies to be mailed to a single address
are discounted 25 percent.

Orders by mail:

U.S. General Accounting Office
P.O. Box 37050
Washington, DC 20013

or visit:

Room 1100
700 4th St. NW (corner of 4th and G Sts. NW)
U.S. General Accounting Office
Washington, DC

Orders may also be placed by calling (202) 512-6000
or by using fax number (202) 512-6061, or TDD (202) 512-2537.

Each day, GAO issues a list of newly available reports and
testimony. To receive facsimile copies of the daily list or any
list from the past 30 days, please call (202) 512-6000 using a
touchtone phone. A recorded menu will provide information on
how to obtain these lists.

For information on how to access GAO reports on the INTERNET,
send an e-mail message with "info" in the body to:

info@www.gao.gov

or visit GAO’s World Wide Web Home Page at:

http://www.gao.gov




PRINTED ON    RECYCLED PAPER
United States                       Bulk Rate
General Accounting Office      Postage & Fees Paid
Washington, D.C. 20548-0001           GAO
                                 Permit No. G100
Official Business
Penalty for Private Use $300

Address Correction Requested