oversight

Aviation Safety: FAA's New Inspection System Offers Promise, but Problems Need to Be Addressed

Published by the Government Accountability Office on 1999-06-28.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                  United States General Accounting Office

GAO               Report to the Subcommittee on Aviation,
                  Committee on Transportation and
                  Infrastructure, House of Representatives


June 1999
                  AVIATION SAFETY
                  FAA’s New Inspection
                  System Offers Promise,
                  but Problems Need to
                  Be Addressed




GAO/RCED-99-183
          United States
GAO       General Accounting Office
          Washington, D.C. 20548

          Resources, Community, and
          Economic Development Division

          B-281502

          June 28, 1999

          The Honorable John J. Duncan
          Chairman
          The Honorable William O. Lipinski
          Ranking Democratic Member
          Subcommittee on Aviation
          Committee on Transportation and Infrastructure
          House of Representatives

          The aviation industry has forecast a potential 66-percent increase in
          passenger travel from 1999 to 2008. The U.S. aviation accident rate, which
          has remained relatively constant over the past two decades,1 must be
          substantially lowered to avoid escalating numbers of aviation deaths as air
          traffic increases. A key to reducing the aviation accident rate is for the
          Federal Aviation Administration (FAA) to have an effective process for
          inspecting the nation’s airline operations. In the past, we and others have
          expressed concerns about the adequacy of FAA’s inspection process to
          meet that challenge. Concerns about the inspection process focused on
          unstructured, nonsystematic inspections that produced few reports of
          safety problems and on the adequacy of inspectors’ technical training.
          These concerns also raised questions about the quality and consistency of
          the resulting inspection data and their usefulness for conducting analyses
          and targeting FAA’s resources to the greatest safety risks.

          FAA has responded to these concerns by redesigning the safety inspection
          system that it uses to oversee the nation’s airlines. FAA began using the
          revised approach, called the Air Transportation Oversight System (ATOS),
          for a limited number of airlines during the system’s initial implementation
          on October 1, 1998. Currently, the nation’s 10 largest passenger airlines are
          under ATOS.2 At your request, we reviewed FAA’s implementation of the new
          system. This report summarizes our work by addressing the following
          questions:

      •   To what extent does ATOS address past concerns about FAA’s aviation
          safety inspections?
      •   What factors, if any, surfaced during the implementation of ATOS that could
          impede its success?

          1
           The National Transportation Safety Board’s statistics show an accident rate of 5 fatal accidents for
          each 10 million flights on scheduled and nonscheduled service by U.S. airlines operating under part
          121 of the Federal Aviation Regulations from 1982 through 1998.
          2
          These airlines are Alaska, America West, American, Continental, Delta, Northwest, Southwest, Trans
          World, United, and US Airways.



          Page 1                                 GAO/RCED-99-183 FAA’s New Aviation Inspection System
                       B-281502




                   •   What is FAA doing to address any factors that could impede the success of
                       ATOS?



                       The Air Transportation Oversight System is largely responsive to past
Results in Brief       concerns raised about key aspects of FAA’s aviation safety inspections and
                       the usefulness of inspection data. These concerns centered on FAA’s
                       unstructured inspection process, the adequacy of technical training for
                       inspectors, the quality and consistency of inspection data, and the
                       usefulness of those data for identifying safety problems and targeting the
                       agency’s resources to the greatest risks. Addressing these concerns
                       involved a fundamental redesign of the way FAA inspects the nation’s
                       airlines. To improve inspection quality, the new program emphasizes a
                       system safety approach that goes beyond spot-checking airlines for
                       compliance with Federal Aviation Regulations. Using safety principles
                       originally created for the nuclear industry, it calls for a systematic review
                       of airlines’ policies and procedures to ensure that they incorporate basic
                       safety principles, such as clear lines of responsibility and written
                       documentation. It fosters more consistent, structured inspections by
                       standardizing inspection tasks, linking inspectors’ training more closely to
                       their assigned responsibilities, and using teams rather than individual
                       inspectors to perform many inspections. The program also calls for a
                       number of enhancements to improve the usefulness of inspection data for
                       analysis and targeting. They include a standardized database for reporting
                       inspection results and the addition of data quality assurance managers and
                       analysts. The goal of this redesign is to target inspection resources to
                       those areas that present the greatest safety risks.

                       ATOS  offers promise for significantly strengthening FAA’s inspection
                       process, but FAA must also address the problems identified in this report to
                       ensure that the new system fulfills its promise. FAA’s ability to conduct
                       effective inspections remains limited by a lack of clear guidance, staff
                       turnover, and continued difficulties with the adequacy of inspectors’
                       technical training and experience. The anticipated enhancements to make
                       inspection data more useful have not been achieved because of problems
                       with reporting requirements and the incompatibility of the program’s
                       database with FAA’s primary inspection analysis system. In addition, FAA
                       planned to hire an analyst for each of its new inspection teams to analyze
                       inspection data for safety trends and to guide inspection planning, but has
                       not yet done so because of higher priorities, such as increasing salaries for
                       air traffic controllers. These problems resulted largely from FAA’s decision
                       to implement the new inspection system on an overly ambitious schedule.



                       Page 2                       GAO/RCED-99-183 FAA’s New Aviation Inspection System
                 B-281502




                 Meeting FAA’s target date for implementation meant that complex, critical
                 steps—such as developing guidance, training inspectors, creating
                 databases, and consulting with the affected airlines—had to be
                 compressed into a very short time.

                 FAA has begun to address some of the problems with the program. FAA
                 management officials acknowledged that ATOS faces significant challenges.
                 They agreed with our conclusion that the program should not be expanded
                 beyond the nation’s 10 major airlines until the problems that emerged
                 during the program’s initial implementation are resolved. However, some
                 of these problems have not yet been fully addressed. Consequently, we
                 recommend several specific actions to clarify the program guidance and
                 improve the usefulness of FAA’s database for targeting inspection
                 resources to the areas of greatest potential safety risk.


                 Federal law establishes that the safety of U.S. air passengers is a joint
Background       responsibility of the airlines and FAA. The airlines are responsible for
                 operating their aircraft safely. FAA is responsible for, among other things,
                 examining an airline’s operations when the airline seeks a certificate to
                 operate and for conducting periodic inspections to ensure continued
                 compliance with safety regulations. Within FAA, the Office of Flight
                 Standards Service develops the Federal Aviation Regulations that airlines
                 must follow and prepares guidance on how FAA’s safety inspectors should
                 perform inspections.

                 FAA has nearly 3,300 safety inspectors located in 101 district offices
                 throughout the United States. One of the inspectors’ primary functions is
                 conducting what FAA calls “routine surveillance”—a process of continuous
                 periodic safety inspections of airlines and aviation-related activities.3
                 These inspections include having an inspector visually spot-check an
                 airplane at the gate, monitor procedures on a scheduled flight, or observe
                 maintenance being performed on an aircraft or its component parts. The
                 inspections cover four main areas:

             •   Operations inspections focus on such items as pilots’ certification and
                 performance, flight crews’ training, and in-flight record keeping.
             •   Maintenance inspections examine an airline’s overall maintenance
                 program, including the training of aviation mechanics, the development of



                 3
                  Other primary functions include certifying airlines’ operations, investigating accidents and incidents,
                 and taking other steps to promote safety.



                 Page 3                                 GAO/RCED-99-183 FAA’s New Aviation Inspection System
                             B-281502




                             maintenance manuals, and procedures for repairing aircraft and their
                             components.
                         •   Avionics inspections focus on electronic components of the aircraft.
                         •   Cabin safety inspections concentrate on cabin procedures, passenger
                             safety, and carry-on baggage.

                             In recent years, we and others have reported on problems with FAA’s
                             inspections.4 Reacting to these reports and to a series of fatal aviation
                             accidents in the mid-1990s, FAA conducted a number of studies of its own,
                             including the report entitled FAA 90 Day Safety Review, issued in
                             September 1996.5 This review recommended that inspections be made
                             more systematic and that they be targeted to deal with identified risks,
                             such as airlines’ financial instability and significant contracting out of
                             maintenance. ATOS, developed by the Office of Flight Standards Service
                             and adapted in part from safety principles originally created for the
                             nuclear industry, resulted from these recommendations.


                             FAA incorporated features in ATOS to address past concerns about
The ATOS Concept Is          inspection quality and the usefulness of inspection data for identifying
Responsive to Many           potential safety threats and for targeting resources to areas that pose the
Past Concerns About          greatest risk. Before developing its new aviation safety inspection system,
                             FAA analyzed past concerns about its inspections. The resulting ATOS
FAA’s Safety                 inspection concept focuses on ensuring that an airline has operating
Inspections                  systems in place to control the potential hazards and risks of flying and to
                             prevent accidents. FAA structured ATOS to evaluate both an airline’s
                             operating systems and its adherence to those systems in day-to-day
                             operations. In addition, FAA incorporated specific features into ATOS to
                             make inspections more consistent, structured, and thorough and to
                             improve the collection and analysis of inspection data.


ATOS Focuses on System       FAA emphasizes a system safety approach in ATOS that replaces routine
Safety and Accident          surveillance and goes beyond spot-checking airlines for compliance with
Prevention                   aviation regulations. System safety involves the application of technical
                             and managerial skills to identify, analyze, assess, and control hazards and
                             risks. It covers every aspect of an airline’s operations, from the design of


                             4
                              For a list of relevant reports, see the bibliography and the section citing related GAO products at the
                             end of this report.
                             5
                              Challenge 2000: Recommendations for Future Aviation Safety Regulation, prepared for the Federal
                             Aviation Administration by Booz-Allen & Hamilton Inc. (Apr. 1996); and FAA 90 Day Safety Review,
                             Federal Aviation Administration (Sept. 16, 1996).



                             Page 4                                 GAO/RCED-99-183 FAA’s New Aviation Inspection System
B-281502




the hardware to the culture and attitudes of the airline’s personnel. ATOS
calls for a systematic review of an airline’s policies and procedures to
ensure that they incorporate such basic safety principles as clear lines of
responsibility and written documentation. FAA intended ATOS to ensure that
an airline has and follows policies and procedures that build in system
safety. FAA implemented ATOS on October 1, 1998, to cover the nation’s 10
major passenger airlines.

Under ATOS, FAA assigns a team of inspectors to oversee each airline. Three
principal inspectors lead the team, one for each major area of inspections
(operations, maintenance, and avionics). Additional team members can be
based in one of two ways. Inspectors based at the FAA office that holds the
airline’s operating certificate work full time on the ATOS team. In contrast,
field inspectors, who work in other locations to which the airline flies,
work part time on the ATOS team and complete additional duties, such as
accident investigations and other inspections, for the local FAA office to
which they are assigned. About 540 inspectors are assigned to the 10 ATOS
teams. Each ATOS team also includes one cabin safety specialist, whose
inspections focus on such areas as flight attendants’ training, carry-on
baggage, and emergency evacuation procedures.

FAA included two kinds of guidance in ATOS to help a team plan and carry
out inspections of the airline it oversees. First, automated ATOS planning
guidance is used to develop the comprehensive surveillance plan for each
airline. The planning guidance calls for using existing safety data, risk
indicators, and the inspectors’ knowledge of an airline’s operations to
determine the priority and frequency of inspection activities. The resulting
comprehensive surveillance plan includes a series of inspection tasks to
determine whether the airline has systems in place to ensure safety and a
second series of inspections to verify that the airline is actually using those
systems. FAA also developed ATOS guidance for conducting inspections that
is intended to describe the tasks to be performed for each type of
inspection. For a more detailed description of the ATOS guidance and the
development of the comprehensive surveillance plan, see appendix I.

FAA designed ATOS to be improved on an ongoing basis. FAA has established
an ATOS Program Office to formulate and implement changes to ATOS and to
support FAA inspection teams through a hotline, help desk, and Web site.
FAA has also established an internal audit team of aviation safety
inspectors to evaluate the program, the System Process Audit Group. This
internal audit team is an independent FAA organization that reports directly
to the Director of Flight Standards.



Page 5                       GAO/RCED-99-183 FAA’s New Aviation Inspection System
                      B-281502




ATOS Addresses Many   In the past, Flight Standards typically allocated a large portion of its
Past Concerns About   inspection resources to thousands of unstructured inspections that
Inspection Quality    produced few reports of problems. We reported last year, for example,
                      that inspectors reported no problems for 96 percent of the inspections
                      they conducted in fiscal years 1990 through 1996.6 Reviews of FAA’s
                      inspection program suggest that FAA detects more problems through
                      rigorous structured inspections than through unstructured inspections.
                      Our recent review of FAA’s oversight of the facilities airlines use to repair
                      aircraft confirmed that standardizing inspection tasks through the use of
                      checklists promotes more comprehensive inspections.7 Past concerns also
                      included problems with inspectors’ training. Specifically, inspectors have
                      performed inspections for which they did not have appropriate or current
                      credentials, in part because of limited funding for training. Providing
                      adequate technical training for FAA’s inspector workforce has proven
                      difficult because of the rapid change in aviation technology. In addition,
                      airlines can meet regulatory requirements in a variety of ways, making it
                      difficult for FAA’s inspectors who inspect many different airlines to be
                      familiar with the FAA-approved procedures of each airline. Both principal
                      inspectors and airline officials we interviewed said that this lack of
                      familiarity sometimes resulted in airlines’ being unfairly cited for
                      noncompliance. Finally, our review of aircraft repair facilities noted that
                      individual inspectors generally identify far fewer deficiencies than teams
                      do. The unstructured inspection activities and the underreporting of
                      violations by inspectors resulted in inaccurate, incomplete, and
                      inconsistent information that was not very useful for analyzing safety risks
                      or targeting the agency’s resources to the problems that pose the greatest
                      risks.

                      FAA  included features in ATOS to move toward more consistent, structured
                      inspections by using a system safety approach and by providing new,
                      standardized inspection tasks. FAA developed automated ATOS planning
                      guidance to ensure that inspectors use the same criteria to determine the
                      annual inspection activities for each of the major airlines. An ATOS team
                      uses the planning guidance to identify potential problem areas at each
                      airline that should be inspected more frequently. Similarly, to standardize
                      inspection activities across airlines, the guidance for conducting
                      inspections lists tasks for each inspection. The inspection guidance is
                      designed to ensure that each inspector looks at an airline’s systems and

                      6
                       Aviation Safety: Weaknesses in Inspection and Enforcement Limit FAA in Identifying and Responding
                      to Risks (GAO/RCED-98-6, Feb. 27, 1998).
                      7
                        Aviation Safety: FAA Oversight of Repair Stations Needs Improvement (GAO/RCED-98-21, Oct. 24,
                      1997).



                      Page 6                               GAO/RCED-99-183 FAA’s New Aviation Inspection System
B-281502




elements in the same way. It is also supposed to serve as a checklist that
inspectors can use to ensure that their inspections are thorough. While the
old inspection system listed multiple tasks that could be performed,
inspectors were not required to complete any specific tasks or to
document which tasks they performed. Thus, the scope of the inspection
work actually completed could not be determined. In contrast, ATOS
requires inspectors to document whether the airline being inspected
complied with each item on the inspection checklist. The checklist also
serves as a template for reporting inspection results in the ATOS database.

FAA  also incorporated team inspections in the ATOS approach. We have
reported in the past that teams have been more effective than individuals
in identifying areas where airlines were not in compliance with FAA
regulations. In many cases, the deficiencies identified by teams are
systemic and long-standing. Under ATOS, teams will identify deficiencies
and plan inspections. Many inspections will be performed by teams rather
than by individual inspectors, as has been done in the past. Individual
inspectors will continue to perform some of the inspection work identified
in the plan.

FAA also included several features in ATOS to address past concerns about
inspectors’ training by more effectively linking inspectors’ technical
training and qualifications to their job responsibilities. First, FAA designed
ATOS to link inspection assignments to the technical background of each
inspector and to identify any additional technical training needed to
accomplish the work plan. Inspectors cannot adequately inspect aircraft
or systems unless they have had the appropriate technical training.
Second, inspectors assigned to an airline, including field inspectors, must
complete training on both ATOS and the airline’s specific policies and
procedures before they can conduct inspection activities. The training on
ATOS provides an overview of the system safety concept and how it differs
from FAA’s past inspection approach. The training on the airline’s policies
and procedures familiarizes inspectors with the approved operating
procedures of the airline they oversee.




Page 7                       GAO/RCED-99-183 FAA’s New Aviation Inspection System
                           B-281502




ATOS Addresses Many        FAA needs complete, accurate inspection data to target its limited
Past Problems With the     inspection resources to the areas that pose the greatest potential safety
Usefulness of Inspection   risks. We reported in 1989 and again in 1991 on inaccurate and incomplete
                           data in FAA’s inspection database and recommended in 1995 that FAA
Data                       develop a comprehensive and coordinated strategy for improving its data.8
                            More recently, we reported last year that 70 percent of Flight Standards’
                           inspectors did not enter all of the violations they found into their
                           inspection tracking system in fiscal year 1996.9 Some inspectors said they
                           did not report violations when compliance could be achieved informally
                           by bringing problems to the attention of the airlines. Others said they
                           handled less serious violations informally because the paperwork involved
                           in reporting violations was too burdensome. FAA is implementing a
                           streamlined procedure for documenting and processing minor
                           administrative violations, which should better enable the agency to target
                           its limited inspection resources to the areas that pose the greatest risks.
                           The streamlined procedure will reduce paperwork for some types of
                           enforcement cases, but other efforts will be needed to ensure the
                           complete, accurate inspection data needed for improved targeting of
                           inspection resources.

                           FAA  included several features in ATOS to address past concerns about the
                           usefulness of inspection data for analysis and targeting. First, the
                           standardization of inspections and the development of guidance for
                           planning and conducting inspections are steps intended to improve the
                           quality of FAA’s data by making inspections more systematic and thorough.
                           When inspections are more standardized across airlines, data quality is
                           improved. Second, FAA created a new position within the ATOS team
                           overseeing each airline: a data evaluation program manager, whose job
                           will be to review data for validity, accuracy, and completeness before they
                           are finalized in the ATOS database for analysis. ATOS also added a new
                           position for an analyst on each team. The analyst is responsible for
                           collecting and analyzing data to support inspection planning and
                           retargeting. Finally, FAA included features in ATOS to improve the targeting
                           of inspection resources. FAA designed ATOS to allow the targeting of
                           inspections based on an airline’s size, operations, past history, and known
                           problem areas. The automated planning guidance can be used to indicate
                           the risk factors applicable to the airline, such as whether an airline is a


                           8
                            Aviation Safety: FAA’s Inspection Management System Lacks Adequate Oversight (GAO/RCED-90-36,
                           Nov. 13, 1989); Aviation Safety: Problems Persist in FAA’s Inspection Program (GAO/RCED-92-14, Nov.
                           20, 1991); and Aviation Safety: Data Problems Threaten FAA Strides on Safety Analysis System
                           (GAO/AIMD-95-27, Feb. 8, 1995).
                           9
                            GAO/RCED-98-6, Feb. 27, 1998.



                           Page 8                               GAO/RCED-99-183 FAA’s New Aviation Inspection System
                              B-281502




                              new entrant or contracts with other companies for its maintenance,
                              training, or ground handling services. Based on the risk factors, the
                              comprehensive surveillance plan targets particular areas. FAA also
                              recognized that needed inspection work must drive the assigned
                              inspection resources for each airline. Finally, ATOS gives inspectors the
                              flexibility to retarget resources at any point during the year based on
                              inspection results. This flexibility allows FAA to focus on new problems as
                              they surface, rather than waiting until the next year’s work.

                              Because ATOS currently focuses on major airlines, it does not address
                              concerns about the need to provide additional oversight of new entrant
                              airlines (that is, airlines in their first 5 years of operation). A separate FAA
                              initiative, the Certification Standardization and Evaluation Team, has
                              standardized and automated the process for granting operating certificates
                              to new airlines. ATOS system safety concepts have been integrated into the
                              certification process for new airlines. The new certification concept
                              includes a national team to assist local district offices in reviewing the
                              applications of new airlines and monitoring these airlines for their first 5
                              years of operation. As new airlines receive certification, FAA plans to
                              oversee them using the ATOS program. FAA is not, however, currently
                              providing any additional oversight of new entrant airlines that were
                              already in operation prior to the new certification process.


                              Problems that emerged during design and implementation limit the
ATOS’ Design and              potential of the ATOS concept to bring about needed improvements in FAA’s
Implementation                aviation safety inspections. Problems with the ATOS inspection guidance,
Problems Limit FAA’s          the links between inspectors’ qualifications and their work assignments,
                              and assembling effective teams affect the improvements envisioned for
Efforts to Improve            inspection quality and the usefulness of inspection data for analysis and
Safety Inspections            targeting. In addition, FAA did not take advantage of the expertise of airline
                              or industry representatives in developing ATOS.


Design and                    Although ATOS calls for (1) more systematic, structured inspections,
Implementation Problems       (2) closer links between inspectors’ training and their assigned work
Limit Improvements to         responsibilities, and (3) greater use of team inspections to improve
                              inspection quality, its success in the first 6 months has been limited:
Inspection Quality
                          •   Inspection guidance is not complete and is not sufficiently clear and
                              detailed to accomplish the systematic, structured inspections promised by
                              the ATOS concept.



                              Page 9                        GAO/RCED-99-183 FAA’s New Aviation Inspection System
                                    B-281502




                                •   ATOS has not resolved the long-standing problems with matching
                                    inspectors’ qualifications to their work assignments.
                                •   Team inspections are hampered by problems with assigning inspectors,
                                    including turnover, work locations that do not match inspection needs, an
                                    unwillingness of inspectors to travel, and FAA’s inability to estimate the
                                    resources needed to complete ATOS inspections.

ATOS Inspection Guidance            The new inspection guidance is not adequate to ensure the
Needs Significant Improvement       comprehensive, standardized inspections envisioned by the ATOS concept.
                                    We found several problems with the ATOS inspection guidance as
                                    implemented.

                                    Guidance does not cover all applicable regulations. One basic purpose of
                                    inspections is to ensure that an airline complies with Federal Aviation
                                    Regulations. Although ATOS aims to go beyond ensuring compliance to see
                                    whether safety is built into an airline’s operations, compliance should also
                                    be confirmed. Principal inspectors we interviewed expressed several
                                    concerns about the link between the ATOS inspection guidance and aviation
                                    regulations. One group of inspectors analyzed the ATOS inspection
                                    guidance to determine whether it covered all applicable Federal Aviation
                                    Regulations. They identified 296 specific regulatory requirements that the
                                    ATOS inspection guidance did not address out of approximately 2,300
                                    applicable requirements. While we have not verified each of the 296
                                    requirements said to be missing, our sampling of the inspectors’ results
                                    confirmed that ATOS overlooked some key regulatory requirements. For
                                    example, the inspection guidance does not cover regulations requiring
                                    airline employees to be trained to handle hazardous materials. Because the
                                    ATOS guidance does not include all applicable regulatory requirements,
                                    inspections may not be thorough enough to ensure compliance.

                                    Some guidance is not applicable to ATOS airlines. In addition to the
                                    regulatory requirements not covered by ATOS, principal inspectors we
                                    interviewed identified a number of regulations referenced in ATOS that are
                                    not applicable to the major airlines currently under ATOS. For example, FAA
                                    requires that major airlines maintain an aircraft’s weight and balance to
                                    ensure that it remains within approved limits. However, the ATOS
                                    inspection guidance for the weight and balance program is based on the
                                    FAA regulations governing commuter airlines.10 Because ATOS currently
                                    covers only the largest airlines, the inspection guidance should exclude



                                    10
                                     Commuter airlines are those that conduct scheduled passenger-carrying operations in aircraft that
                                    have 10 or fewer seats and operate under part 135 of the Federal Aviation Regulations.



                                    Page 10                               GAO/RCED-99-183 FAA’s New Aviation Inspection System
B-281502




those regulations that are applicable to commuter airlines or other types
of operators, such as general aviation aircraft.

Principal inspectors we interviewed also questioned the appropriateness
of some parts of the ATOS guidance that have no basis in regulatory
requirements. For example, the ATOS guidance calls for reviewing the
résumés of some airline officials, although regulations do not specify
qualifications and experience for their positions. Principal inspectors also
questioned the basis for some of the ATOS determinations that rely on very
subjective judgments, such as whether the airline has a “safety focus.”
They noted that the ATOS guidance does not distinguish inspection tasks
and findings based on regulations, which are legally enforceable, from
those based on such other sources as inspector handbook guidance, which
is advisory. Safety officials at most of the major airlines echoed the
inspectors’ concerns.

Guidance is not sufficiently clear and detailed. The fact that ATOS provides
guidance to inspectors on how to plan and perform their inspections
represents a major step toward the standardization of inspection tasks.
However, the guidance it offers is not yet thorough or detailed enough to
achieve that goal. Principal inspectors we interviewed questioned its
usefulness, saying that it was not clear or detailed enough. They reported
that they found the language of the guidance for planning inspections
difficult to use because it does not detail the tasks to be performed well
enough. Staff at Sandia National Laboratories, who were asked by FAA to
comment on the ATOS program because of their expertise in system safety
in the nuclear industry, had pointed to similar concerns before ATOS was
implemented. In a report on ATOS, the Sandia staff noted that the
inspection guidance was not based on analyses of specific ATOS inspection
tasks and the recording of the results.11 The Sandia report noted that a
proper task analysis describes the steps to take and the standards for
determining that the results are correct and complete. The inspection
guidance does not provide this level of detail.

FAA’s guidance material was not thoroughly tested. The lack of clarity and
detail in the ATOS guidance, both for (1) planning and (2) conducting
inspections, reflects the fact that FAA did not thoroughly test and validate it
before implementation. When FAA first tested the planning guidance in
April 1998, the inspectors involved said they had trouble using it because
the questions were too vague and broad. When FAA later tested a revised

11
 “Analysis and Data Issues for the Implementation of a Systems Safety Focus in Air Transportation
Oversight,” Oct. 6, 1998.



Page 11                              GAO/RCED-99-183 FAA’s New Aviation Inspection System
B-281502




automated version of the planning guidance, most test participants were
members of the ATOS work group. Work group members filled in the
answers without actually performing the relevant analysis or inspection
tasks. While FAA tested the planning guidance, it never tested the guidance
for conducting inspection tasks.

Flight standards did not allow sufficient time for ATOS’ implementation.
While FAA spent several years developing the ATOS concept, the target
implementation date left only 7 months to take the agency’s new
inspection program from concept to reality. The target date of October 1,
1998, did not allow time for the adequate development, testing, and
validation of new inspection procedures and tracking systems or for
training and preparing the inspector workforce. The ATOS work group
established in February 1998 faced multiple challenges in meeting the
implementation deadline in the allotted 7 months. These challenges
included (1) implementing a completely reengineered process for
conducting inspections, (2) developing new automated systems for
planning ATOS inspections and tracking their results, and (3) preparing FAA
for the cultural changes involved in having inspectors perform their work
differently.

FAA did not adequately train inspectors to use ATOS. The full impact of the
shortcuts taken to meet the implementation deadline became apparent
during the inspectors’ training and, subsequently, as the teams of
inspectors used the new ATOS planning guidance to develop a
comprehensive surveillance plan for each major airline and to initiate
inspections. FAA trained more than 800 inspectors, managers, and other FAA
staff on the ATOS concept and planning guidance from September through
November 1998.

Because of the speed with which the ATOS guidance and tracking systems
were developed, FAA had little time to develop training for its inspectors.
For example, very few of the 88 ATOS lists of inspection tasks were
available in time for the training sessions, and the database for reporting
inspection results was not yet completed. Participants in the training
sessions repeatedly noted that the implementation of ATOS had been
rushed and that there were too many unknowns to allow for this initiative
to go smoothly. In course evaluations and in our interviews, inspectors
said that they were not adequately trained on (1) how to perform the new
inspection tasks, (2) how to record the results of their inspections in the
ATOS database, and (3) how to use the data on inspections to retarget
resources. The ATOS internal audit team concluded, “The ATOS training . . .



Page 12                     GAO/RCED-99-183 FAA’s New Aviation Inspection System
                                 B-281502




                                 dedicated an insufficient amount of time to training on the [inspection
                                 guidance].” Standardized inspections and reporting will not occur without
                                 clear, understandable guidance, detailed information on tasks to be
                                 completed, and training in both conducting inspections and reporting
                                 findings.

Links Between Inspectors’        The ATOS model for linking inspectors’ qualifications and training to
Qualifications and Assignments   assignments is not yet a reality. We found that some inspectors still lack
Are Not Fully Established        experience with FAA or major airlines or lack the specific training needed
                                 to perform their jobs effectively.

                                 Lack of experience with FAA or major airlines. In selecting staff to fill
                                 positions on ATOS teams, FAA did not effectively match the qualifications of
                                 field inspectors to their new positions. The selection of field inspectors to
                                 fill ATOS assignments was not based on any nationwide criteria or
                                 guidance. Principal inspectors told us that they had no say in the selection
                                 of the field inspectors assigned to their teams. As a result, the
                                 qualifications of some field inspectors did not match the teams’ needs.
                                 Several principal inspectors and managers said that field inspectors
                                 assigned to them included newly hired staff who were unfamiliar with FAA
                                 or inspectors—including experienced FAA staff—who lacked background
                                 with major airlines. For example, one manager said that some of the field
                                 inspectors assigned to his team had no experience in large aircraft and
                                 lacked appropriate qualifications. FAA principal inspectors have told us
                                 that it takes several years to develop familiarity with the agency’s
                                 regulations and procedures or with an airline’s procedures. Consequently,
                                 newly hired personnel may need several years of experience before they
                                 can work independently and be fully productive. Similarly, inspectors
                                 trained on small aircraft may need extensive on-the-job training to
                                 understand the workings of a major airline. Principal inspectors
                                 questioned whether some team members had the appropriate skills to
                                 oversee a major airline.

                                 Lack of specific training. Even inspectors who have experience with major
                                 airlines may not have the specific technical qualifications to perform the
                                 ATOS inspection tasks planned for the airline to which they are assigned.
                                 Principal inspectors we interviewed said that the technical qualifications
                                 of a number of the inspectors assigned to them did not match those
                                 needed to oversee the airline in question. For example, one operations
                                 inspector said that three of his field inspectors did not have the
                                 appropriate license required to fly the aircraft used by the airline or to
                                 conduct flight checks to observe its pilots. Similarly, a principal avionics



                                 Page 13                      GAO/RCED-99-183 FAA’s New Aviation Inspection System
                               B-281502




                               inspector said that neither he nor his assistant had the appropriate
                               technical training in the Airbus aircraft that the airline flies. Furthermore,
                               the rapid changes in the aviation industry make it difficult to provide
                               sufficient technical training to keep experienced inspectors current. These
                               difficulties in keeping pace with technological advances reflect FAA’s
                               persistent problems with providing technical training for inspectors and
                               matching inspectors’ qualifications to their job assignments.

Assignment Issues Affect the   Staff turnover, travel requirements, and conflicting demands on inspectors’
Quality of Team Inspections    time undermine FAA’s ability to assemble effective ATOS teams. Many of
                               these assignment issues cannot be resolved within the context of the ATOS
                               program alone because they involve broader decisions about inspectors’
                               responsibilities and Flight Standards’ staffing.

                               Staff turnover. The turnover among field inspectors assigned to the
                               airlines covered by ATOS has made it difficult to plan inspection work or to
                               meet the ATOS training requirements. Principal inspectors said that they
                               had lost field inspectors originally assigned to their teams, primarily
                               because field inspectors went elsewhere in FAA to accept promotions that
                               are not available to them under Flight Standards’ current staffing
                               structure. For example, of the 28 inspectors assigned to one ATOS team, 11
                               had been reassigned, leaving only 17 available. Because some
                               replacements lacked ATOS training, training on the airline’s policies and
                               procedures, or both, they were ineligible to conduct inspections under
                               ATOS. Principal inspectors were concerned about how, if turnover
                               continues, to meet the requirement that inspectors be trained on the
                               policies and procedures of the airline they oversee. In many cases, the
                               airlines provided this training in cooperation with FAA. Several principal
                               inspectors said that repeated requests for training would be burdensome
                               to the airlines. Principal inspectors noted that the turnover of field
                               inspectors would probably continue unless changes are made in Flight
                               Standards’ grade and pay levels to permit field inspectors to be promoted.

                               Inspectors’ work locations do not always match inspection needs. Because
                               the work locations of some inspectors assigned to ATOS teams do not
                               match inspection needs, the inspectors will have to travel to complete
                               their work. This has made it difficult to assemble effective teams for
                               several reasons. First, many of the principal inspectors we interviewed
                               told us that field inspectors were assigned to locations where they are not
                               needed, while parts of the country where the airlines have substantial
                               activity have no field presence. For example, one principal inspector told
                               us that he had a field inspector assigned to his team who was located in



                               Page 14                      GAO/RCED-99-183 FAA’s New Aviation Inspection System
B-281502




Boise, Idaho, where the airline he oversees does not fly, but that he had no
one in Kansas City, where he needs an inspector to oversee maintenance
operations. Because of these problems, a number of principal inspectors
suggested that they be given a role in identifying the needed qualifications
and work locations for field inspectors assigned to the ATOS teams.
Furthermore, in the past, teams overseeing an airline drew on field
inspectors in FAA’s international offices to inspect foreign maintenance
facilities and other overseas operations of airlines with international
routes. Because FAA has not trained inspectors in its international offices
to perform ATOS inspections, ATOS team members will have to travel
overseas to complete planned inspections, despite FAA’s already limited
travel funds.

ATOS does not resolve resource constraints. FAA designed ATOS to address
the inspection portion of its inspectors’ workload and did not address the
inspectors’ other responsibilities. In addition to inspections, the oversight
of an airline includes ongoing activities referred to as “demand” work.
Demand work includes certification and approvals for initiatives taken by
the airline, such as adding new aircraft types to the fleet, adding new
destinations, implementing computerized record keeping, and
restructuring by management. The ATOS internal audit staff confirmed that
a conflict exists between demand work and ATOS inspection work.
Inspectors based near the offices of the airlines they oversee are especially
prone to this conflict because they perform both demand work and ATOS
inspections. Many principal inspectors said that these inspectors do not
have sufficient time to complete both their demand work and ATOS
inspection work. Field inspectors, who are assigned only part time to ATOS,
are subject to different pressures. ATOS field inspectors must also
investigate accidents and perform other inspection work for the local
offices to which they are assigned. Principal inspectors expressed concern
that field inspectors might not be available for ATOS work when needed
because of demands from their local offices. In addition, several principal
inspectors and managers questioned whether the emphasis on ATOS has
shifted resources away from other areas of concern, such as repair
stations, troubled smaller carriers, and general aviation safety. Managers
contended that such conflicts between demand work and inspection
activities will persist as long as ATOS focuses solely on inspections, rather
than having a broader view of the work that inspectors perform.

These conflicting demands on inspectors’ time and on resources such as
travel funds are exacerbated because managers do not know what
resources will be needed to perform ATOS inspections. The ATOS guidance



Page 15                     GAO/RCED-99-183 FAA’s New Aviation Inspection System
                                B-281502




                                does not include estimates of how long various inspection tasks should
                                take. Without such information, it is difficult to estimate how many
                                inspectors will be needed to perform the tasks or what travel funds they
                                may need to accomplish the planned work. Both inspectors and their
                                managers told us that they are unable to estimate how much time many of
                                the ATOS inspection tasks will require. Two principal inspectors told us that
                                managers of some local offices will not approve ATOS work plans for field
                                inspectors unless they know how much time will be needed to complete
                                the work. Testing and validation of the ATOS guidance would have provided
                                preliminary estimates for the time needed to complete various ATOS
                                inspection tasks.


The ATOS Data Collection        Although ATOS includes initiatives to address past problems with the
Process Limits the              usefulness of FAA’s inspection data for analysis and targeting of resources
Usefulness of Data for          to the greatest safety risks, this goal may not be fully realized for several
                                reasons. First, because of the problems with the ATOS guidance that we
Analysis and Targeting          have already discussed, the data collected from ATOS inspections are not
                                likely to be reliable enough to support meaningful analyses. Furthermore,
                                in translating ATOS from concept to design and eventual implementation,
                                FAA did not adequately determine its data analysis needs. The volume of
                                inspection data available for analysis has also dropped dramatically under
                                ATOS because few inspections have been completed. Finally, principal
                                inspectors responsible for overseeing airlines did not have timely access
                                even to the limited data available until FAA granted them access in
                                May 1999.

FAA Did Not Adequately          While ATOS calls for structured inspections intended to result in more
Determine Data Analysis Needs   thorough and consistent data, the way ATOS collects data limits the
                                potential of its database as a tool for analysis and targeting. The data
                                limitations reflect the fact that FAA did not sufficiently analyze its data
                                needs before developing ATOS inspection guidance and its automated
                                database. The development of an effective automated system begins with a
                                thorough analysis of the data required to meet the needs of those using the
                                database. In the case of ATOS, a thorough analysis would describe in detail
                                the questions that need to be asked to improve aviation safety, determine
                                precisely what data are needed to answer those questions, and plan the
                                appropriate analyses to be conducted on those data to answer the
                                questions. After the preliminary analysis of the users’ data needs is
                                completed, the prototype system must be tested, validated, and revised in
                                an iterative process between data users and automation developers. Staff
                                from Sandia National Laboratories reviewed ATOS and concluded that its



                                Page 16                      GAO/RCED-99-183 FAA’s New Aviation Inspection System
B-281502




developers did not go beyond abstract, high-level statements about users’
needs to the level of detail essential to ensure data quality. They added
that without more detailed information it will be difficult to identify the
data and information needed to answer questions about aviation safety
and impossible to support the detailed automated analysis of an airline.

Because FAA did not sufficiently analyze or list the data needed by
inspectors to make determinations about aviation safety and did not
adequately test or validate ATOS, features that would maximize the
usefulness of data for analysis and targeting were not built into the system.
We found four specific limitations with the ATOS database.

Key information is not required. ATOS does not currently require inspectors
to record inspection data that are essential for effective analyses. To
perform effective analyses of safety data, basic information is needed,
such as the airline’s name; the make, model, and series of the aircraft; the
aircraft and pilot identification numbers; and where the inspection was
performed. However, an ATOS inspection record can be closed without any
of this basic information having been entered because the system requires
only that inspectors indicate whether an airline is in compliance and
explain any violations. For example, our review of completed ATOS
inspections found 18 completed inspection activities related to de-icing
operations for which the location had been entered, not in the appropriate
field, but in a comment field. Data contained in comment fields cannot be
used for automated analyses to determine, for example, which airports are
experiencing problems with de-icing.

In addition, the ATOS inspection guidance does not provide inspectors with
an index or clear instructions on where in the ATOS database to report
findings. If an inspector cannot easily figure out where to report a finding,
it may go unreported. For example, an inspector who found incorrect
safety placards on an aircraft said he had to search ATOS for nearly 4 hours
to record this violation. Inspectors’ underreporting of violations that they
observed has jeopardized the completeness and quality of inspection data
in the past. ATOS does not resolve the problems that contributed to
underreporting, and the difficulty in finding where to report violations in
ATOS may exacerbate this problem.


Response options preclude meaningful analyses. Many ATOS inspection
activities are to be summed up in a report with a single “yes” or “no”
response to the items on the inspection checklist for a given area. If an
inspector finds problems during a single inspection activity, it results in a



Page 17                      GAO/RCED-99-183 FAA’s New Aviation Inspection System
B-281502




“no” for the entire question. Regardless of whether an inspector finds one
problem in 10 inspections or one in 100, a “no” results. Without knowing
the proportion of “no” responses for a given item, FAA cannot use the
inspection results to conduct quantitative analyses on ATOS data, for
example, a trend analysis to determine whether the airline’s compliance in
a given area has improved.

Minimum number of inspection activities needed is not specified. ATOS
currently does not specify the minimum number of activities needed to
complete each inspection. Instead, individual inspectors decide
independently how many times inspection observations should be
performed to determine whether the airline follows its procedures and
complies with regulations. This determination is subjective. In our review
of the ATOS data available as of March 31, 1999, we found that four
inspectors conducted anywhere from 2 to 12 observations to complete the
record for the same type of inspection. Because these reports result in
“yes” or “no” answers on the completed inspection report, rather than in
a quantifiable report of the proportion of “no” answers, the data
submitted by these inspectors cannot be compared or analyzed. Because
inspectors will never have enough time to observe every safety-related
component of an airline’s system, it is important to define the minimum
number of inspections to be conducted and to report accurately the
proportion of instances of noncompliance.

ATOS does not link to Flight Standards’ existing data analysis system. The
information in the ATOS database cannot be analyzed by Flight Standards’
existing aviation risk analysis system, the Safety Performance Analysis
System (SPAS). FAA has spent $95 million developing SPAS to analyze key
aviation safety data, identify trends and potential safety concerns, and
target inspection resources accordingly. In January 1998, the ATOS
development team noted that ATOS inspection data could not be analyzed
by SPAS. The development team recommended that ATOS not be
implemented until FAA could develop appropriate links between ATOS and
SPAS. However, FAA, in its efforts to meet the October 1, 1998,
implementation date, went forward without addressing this
recommendation. SPAS program officials told us that significant work
remains to link the two systems. An ongoing FAA work group studying
ways to incorporate system safety into Flight Standards’ programs has
been given responsibility for linking ATOS and SPAS. This work group has
not yet established a timetable for how or when the link will be completed.




Page 18                     GAO/RCED-99-183 FAA’s New Aviation Inspection System
                            B-281502




Availability of ATOS Data   The usefulness of ATOS data for analysis and targeting has been limited by
Limits Analysis             the overall lack of information in the ATOS database as well as by initial
                            problems with accessing what information is available. FAA devoted much
                            of the early part of fiscal year 1999 to planning ATOS inspections and
                            training inspectors in the new system. As a result, very few data on the
                            performance of the airlines covered by ATOS have been available for
                            analysis. By the end of the second quarter, fewer than 60 of approximately
                            5,400 planned inspections had been completed, entered into the ATOS
                            database, and made available to principal inspectors. These completed
                            inspections comprised fewer than 1,200 individual inspection activities,
                            substantially fewer than those recorded in previous years.12 The flow of
                            inspection activity results into the ATOS database has since accelerated,
                            with an additional 267 inspections and 1,808 individual inspection
                            activities recorded over the first 6 weeks of the third quarter.
                            Nevertheless, only 326 inspections, consisting of 3,079 inspection
                            activities, had been completed by May 11, 1999, and for one airline no
                            inspections had been completed. In contrast, during the first two quarters
                            of fiscal year 1998, over 37,000 inspection activities had been recorded in
                            the Program Reporting and Tracking System for the 10 airlines now
                            covered by ATOS.13

                            We also found that access to the information available in the ATOS database
                            was limited until inspections were completed. FAA designed ATOS so that
                            neither principal inspectors nor analysts could access data until inspectors
                            completed all inspection activities and the team’s data evaluation program
                            manager reviewed and approved the data. The lack of access to key safety
                            data created problems for principal inspectors, who are responsible for
                            overseeing operations, maintenance, and avionics inspections at each
                            airline. They could not view inspection results until the data evaluation
                            program manager reviewed the data for clarity and consistency. Several
                            principal inspectors expressed concern about not having timely access to
                            key safety data, which they use as an early warning of potential safety
                            risks. In February 1999, we briefed FAA on the concerns expressed about
                            delayed access to inspection results in the ATOS database. In March 1999,
                            FAA directed its automation contractor to take the steps necessary to grant
                            principal inspectors immediate read-only access to inspection findings.
                            Until this access was granted, principal inspectors remained dependent on


                            12
                             Under ATOS, each inspection recorded in the database summarizes the results of multiple
                            observations called inspection activities.
                            13
                             While the 3,079 ATOS inspection activities may not be directly comparable to the 37,000 inspections
                            conducted under the previous inspection system, a substantial drop-off in the inspection information
                            available to managers and analysts has clearly occurred over the first half of fiscal year 1999.



                            Page 19                               GAO/RCED-99-183 FAA’s New Aviation Inspection System
                              B-281502




                              such informal means as e-mail and telephone calls to learn of any
                              problems or violations found during inspections. Principal inspectors now
                              have access to all final inspection activity reports related to their specialty
                              for the airline that they oversee. This change made data on 2,724 ATOS
                              inspection activities available to principal inspectors as of May 11, 1999.


ATOS’ Targeting               The previously mentioned inadequacies in the ATOS guidance and problems
Capabilities Are Limited by   with the usefulness of the resulting data for analysis limit FAA’s ability to
Inadequate Inspection         use ATOS for targeting inspection resources to the problems that pose the
                              greatest safety risks. Prior to the implementation of ATOS, the primary
Guidance and Problems         purpose of inspections was to identify individual safety problems and
With the Usefulness of        ensure their correction. Under ATOS, the primary objective is to provide
Data                          reliable data to enable FAA to identify the highest-priority safety concerns
                              and target the agency’s resources to reduce these concerns or risks as well
                              as to ensure that individual safety problems are corrected. As
                              implemented, ATOS falls short of this goal. Significant revisions will be
                              needed to the ATOS guidance and database before the potential of this new
                              inspection system can be realized.


FAA Did Not Take              Although FAA has many ongoing initiatives with the aviation industry,
Advantage of Industry         coordination with industry was lacking in the design of ATOS. Although
Expertise in Developing       several aspects of ATOS involve areas in which industry safety experts have
                              experience and similar goals, the design process did not include airline or
ATOS                          industry representatives, who were briefed on ATOS after the concept was
                              developed. The industry’s input is missing, for instance, in the risk weights
                              ATOS uses in its planning guidance to help determine the number of
                              inspections a team is to conduct, thus directing resources to areas that
                              require additional oversight. For example, the weight ATOS gives to an
                              airline’s screening, boarding, and briefing procedures for passengers is
                              greater than that assigned to cockpit procedures. FAA neither validated
                              these weights nor discussed them with airline safety officials.


                              We briefed FAA in December 1998 and again in February 1999 on the
FAA Has Initiated             problems we had found with the implementation of ATOS. Because of the
Improvements to               magnitude and seriousness of the problems associated with the current
ATOS                          ATOS guidance and database, we suggested that FAA not expand ATOS to
                              additional airlines, repair stations, or other aviation operations until these
                              problems are resolved. Acknowledging that there were significant




                              Page 20                       GAO/RCED-99-183 FAA’s New Aviation Inspection System
              B-281502




              challenges to further implementing the program, FAA agreed that
              expansion of the program needs to be delayed.14

              In March 1999, the Director of Flight Standards and key ATOS program
              officials met with the principal inspectors who lead the 10 ATOS teams to
              brief them on our findings and to obtain their views on the system’s
              implementation. This meeting resulted in a list of immediate and future
              actions to be undertaken to address the concerns raised. First, FAA has
              taken steps to provide principal inspectors with immediate access to
              completed inspection reports. FAA also provided partial funding for
              planned overseas inspections, which will allow inspectors to complete
              some of the more critical overseas inspections. Finally, FAA will
              incorporate additional information on the inspection guidance task lists
              and database tracking system into ATOS training beginning in July. These
              initiatives will help alleviate several of the more immediate problems with
              ATOS by (1) making key safety information available to principal inspectors
              as soon as inspections are completed, (2) allowing at least some of the
              planned overseas inspections to take place, and (3) providing better
              training for inspectors on how to conduct inspections and record their
              results. In addition, both the ATOS Program Office and the ATOS internal
              audit group will continue to monitor the implementation closely to ensure
              that these problems and others that may arise are addressed. Funding has
              not yet been allocated to support the needed improvements to ATOS or to
              link ATOS with FAA’s existing data targeting and analysis system, SPAS.


              The ATOS concept offers significant promise for helping FAA overcome
Conclusions   deficiencies in its past approach to aviation safety inspections. However,
              its potential will not be fully realized until FAA resolves the problems
              resulting from the ambitious schedule it followed in implementing ATOS.
              These problems limit both FAA’s ability to conduct more systematic,
              structured inspections and analyze the resulting data to identify safety
              trends and its ability to target its resources to the greatest risks. The ATOS
              guidance is not clear and detailed enough to ensure more systematic,
              structured inspections that will result in more usable data. In addition, FAA
              has not adequately analyzed the data needs of ATOS users to ensure that the
              system collects the information that will enable the agency to perform
              critical trend and safety analyses. Such analyses are also limited because
              ATOS does not link to FAA’s other major database for safety analyses (SPAS).
              FAA has recognized the need for significant improvements before ATOS will


              14
                FAA already had plans to bring two additional airlines into ATOS in the near future. These airlines
              have recently completed certification activities and will begin ATOS inspections upon completion of a
              transition phase.



              Page 21                               GAO/RCED-99-183 FAA’s New Aviation Inspection System
                      B-281502




                      achieve its full potential as a system for overseeing commercial airline
                      safety, and the agency has taken actions to correct some of the problems
                      we found. Because of the challenges involved in making the needed
                      improvements to ATOS, FAA officials have postponed plans to expand ATOS
                      to other airlines until these problems are corrected. In addition, FAA is
                      aware that the resolution of some problems with the implementation of
                      ATOS involve broader issues that concern staffing decisions and workload
                      issues that can affect all of Flight Standards’ inspectors.


                      To strengthen the efforts to improve FAA’s aviation safety inspections and
Recommendations       the usefulness of the data that result from these inspections for analysis
                      and for targeting the agency’s resources to the greatest potential safety
                      threats, we recommend that the Secretary of Transportation direct the FAA
                      Administrator to take the following actions:

                  •   Develop a structured process and timeline for working with inspectors to
                      revise the Air Transportation Oversight System’s planning and inspection
                      guidance. The process should involve the inspectors now using this
                      guidance to (1) identify problems with the clarity of the guidance,
                      (2) revise the inspection guidance to include tasks related to all applicable
                      Federal Aviation Regulations, and (3) define the tasks to be completed
                      during inspections.

                  •   Revise the inspection guidance to (1) include guidelines on the minimum
                      number of times to perform various inspection tasks and (2) distinguish
                      between tasks based on regulatory requirements and those based on
                      handbook or other guidance.

                  •   Develop a plan that involves both inspectors and experts in risk
                      assessment and database development in revising and refining the analysis
                      of the data needs of users of the new inspection program. The
                      requirements analysis should describe in detail the questions that need to
                      be asked to improve safety, determine precisely what data are needed to
                      answer those questions, and plan the appropriate analyses to be
                      conducted on those data to answer the questions.

                  •   Restructure the inspection database to (1) require that essential data fields
                      be completed before inspection reports can be closed out and (2) clearly
                      indicate the proportion of inspection observations in which an airline
                      complies with regulations.




                      Page 22                      GAO/RCED-99-183 FAA’s New Aviation Inspection System
                         B-281502




                     •   Determine what revisions will be needed to the Air Transportation
                         Oversight System database and the agency’s existing Safety Performance
                         Analysis System database to maximize the potential of these two systems
                         by coordinating their trend analyses to identify potential safety risks.

                     •   Test and validate the revised guidance and database for the new
                         inspection program.


                         We provided FAA with a draft of this report for review and comment. We
Agency Comments          met with the Deputy Associate Administrator for Regulation and
and Our Evaluation       Certification, the Director of the Office of Flight Standards Service, the
                         Acting Manager of the Air Transportation Oversight System Program
                         Office, the Manager of the System Process Audit Group, and other FAA
                         officials. The agency agreed with the substance of the report but
                         commented that the tone was unnecessarily negative and could leave the
                         impression that we believe that the program should be abandoned. Agency
                         officials also said that they made a conscious decision to implement the
                         new system aggressively, rather than in stages, and recognized that this
                         approach would result in some implementation problems. However, they
                         believe that ultimately the new system will be fully operational sooner
                         than if they followed a more conservative implementation approach. The
                         agency also commented that our review of the program was premature
                         and suggested that an evaluation of the program in another year would
                         find that most of the problems we reported had been resolved.

                         We do not believe the program should be abandoned. We believe that our
                         report clearly supports the Air Transportation Oversight System and
                         acknowledges its potential for significantly strengthening FAA’s inspection
                         process. However, we continue to believe that serious challenges need to
                         be overcome before this program can achieve its potential. Because our
                         review coincided with the program’s implementation, we were able to
                         identify serious problems early and to promote constructive action by FAA
                         to begin resolving them. As we reported, FAA has begun to address some of
                         these problems. In commenting on this report, FAA also provided some
                         updated information on its inspection activities and suggested wording
                         revisions that we incorporated as appropriate.


                         We conducted our work from September 1998 through June 1999 in
                         accordance with generally accepted government auditing standards.
                         Appendix II contains details of the scope and methodology of our review.



                         Page 23                     GAO/RCED-99-183 FAA’s New Aviation Inspection System
B-281502




As you requested, unless you publicly announce its contents earlier, we
plan no further distribution of this report until 10 days from the date of
this letter. We will then send copies to the appropriate congressional
committees; Rodney E. Slater, the Secretary of Transportation; Jane F.
Garvey, the Administrator, FAA; Jacob J. Lew, the Director, Office of
Management and Budget; and other interested parties. We will also make
copies available to others upon request.

If you have any questions about this report or need additional information,
please call me at (202) 512-2834. Major contributors to this report are
listed in appendix III.




John H. Anderson, Jr.
Director, Transportation Issues




Page 24                     GAO/RCED-99-183 FAA’s New Aviation Inspection System
Page 25   GAO/RCED-99-183 FAA’s New Aviation Inspection System
Contents



Letter                                                                                              1


Appendix I                                                                                         28
                       The System Safety Assessment Tool                                           30
ATOS Guidance and      The Air Carrier Assessment Tool                                             32
the Comprehensive      The Comprehensive Surveillance Plan                                         33
                       Safety Attribute Inspections                                                33
Surveillance Plan      Element Performance Inspections                                             34

Appendix II                                                                                        35

Objectives, Scope,
and Methodology
Appendix III                                                                                       37

GAO Contacts and
Staff
Acknowledgments
Bibliography                                                                                       38


Related GAO Products                                                                               39


Tables                 Table I.1: The Seven Airline Systems Defined in ATOS                        30
                       Table I.2: Categories of System Safety Used With SSAT                       31
                       Table I.3: ACAT Risk Indicators                                             32

Figure                 Figure I.1: How FAA Designs Comprehensive Surveillance Plans                29
                         Through ATOS


                       Abbreviations

                       ACAT      air carrier assessment tool
                       ATOS      Air Transportation Oversight System
                       EPI       element performance inspection
                       FAA       Federal Aviation Administration
                       GAO       General Accounting Office
                       SAI       safety attribute inspection
                       SPAS      Safety Performance Analysis System
                       SSAT      system safety analysis tool


                       Page 26                    GAO/RCED-99-183 FAA’s New Aviation Inspection System
Page 27   GAO/RCED-99-183 FAA’s New Aviation Inspection System
Appendix I

ATOS Guidance and the Comprehensive
Surveillance Plan

             The Air Transportation Oversight System (ATOS) is a process designed to
             improve the Federal Aviation Administration’s (FAA) oversight of airlines.
             FAA’s Office of Flight Standards Service developed ATOS with the support of
             Sandia National Laboratories. ATOS uses system safety principles created
             for the nuclear industry and risk management to ensure that airlines have
             safety built into their operating systems. A systems safety approach means
             that FAA’s inspection efforts will cover all aspects of an airline’s
             performance that can affect safety and will focus on preventing accidents.

             A certificate management team oversees each of the 10 airlines under
             ATOS. Each team is led by three principal inspectors, one for each major
             area of inspections (operations, maintenance, and avionics). Additional
             team members include those based at the FAA office that holds the airline’s
             operating certificate and field inspectors in other FAA offices at locations
             to which the airline flies.

             The team uses automated planning guidance to develop a comprehensive
             surveillance plan for the airline. The planning guidance consists of two
             automated tools—the system safety analysis tool (SSAT) and the air carrier
             assessment tool (ACAT). The principal inspectors complete the SSAT and
             ACAT prior to an annual planning meeting. During the meeting, team
             members discuss the SSAT and ACAT, and their feedback is included in the
             final version. The results of the ACAT help define inspection activities that
             the team will include in the airline’s comprehensive surveillance plan. The
             SSAT, ACAT, and comprehensive surveillance plan are described in more
             detail below. Figure I.1 shows how these ATOS components relate to one
             another.




             Page 28                      GAO/RCED-99-183 FAA’s New Aviation Inspection System
                                       Appendix I
                                       ATOS Guidance and the Comprehensive
                                       Surveillance Plan




Figure I.1: How FAA Designs Comprehensive Surveillance Plans Through ATOS




                                       Source: Federal Aviation Administration.




                                       Page 29                              GAO/RCED-99-183 FAA’s New Aviation Inspection System
                                       Appendix I
                                       ATOS Guidance and the Comprehensive
                                       Surveillance Plan




                                       Using the automated planning guidance, the team analyzes the airline’s
                                       operations, which are divided into seven systems (see table I.1). These
                                       systems are made up of 14 subsystems and of 79 elements. For example,
                                       the flight operations system is made up of two subsystems, which
                                       encompass elements such as aircraft dispatch and flight deck procedures.
                                       FAA designed the SSAT to help the team assess the systems in place at an
                                       airline to ensure safe operations. The ACAT applies a set of risk indicators
                                       to the airline’s subsystems and elements to generate a comprehensive
                                       surveillance plan. These risk indicators for the ACAT are based on safety
                                       and performance information that reflects areas of potential risk for an
                                       airline’s operations. Hence, the comprehensive surveillance plan will
                                       target those areas most likely to have safety problems.

Table I.1: The Seven Airline Systems
Defined in ATOS                        System                                           System’s purpose
                                       Aircraft configuration and control               Maintains the physical condition of the
                                                                                        aircraft and associated components.
                                       Manuals                                          Controls the information and instructions
                                                                                        that define and govern an airline’s
                                                                                        activities.
                                       Flight operations                                Governs aircraft movement.
                                       Personnel training and qualifications            Ensures that an airline’s personnel are
                                                                                        trained and qualified.
                                       Route structures                                 Maintains an airline’s facilities on approved
                                                                                        routes.
                                       Airman/crew member flight, rest, and duty        Prescribes time limitations for airline
                                       time                                             employees.
                                       Technical administration                         Addresses all other aspects of an airline’s
                                                                                        certification and operations.
                                       Source: FAA Order 8400.10, Air Transportation Operations Inspector’s Handbook, appendix 6.




                                       The SSAT is a computerized tool designed to focus the inspection team’s
The System Safety                      attention on the systems that an airline has in place. It poses questions to
Assessment Tool                        the team covering six categories: safety attributes, safety culture,
                                       communications, accountability, training programs, and potential problem
                                       areas. (See table I.2.) The principal inspectors complete the SSAT prior to a
                                       yearly meeting to plan inspections to oversee the airline’s operations. To
                                       complete the SSAT, the principal inspectors rely on their knowledge of the
                                       airline and on the data available through FAA’s Safety Performance
                                       Analysis System (SPAS), the Flight Standards Automated System, or other
                                       sources. Before the annual meeting, the team members review the SSAT




                                       Page 30                              GAO/RCED-99-183 FAA’s New Aviation Inspection System
                                         Appendix I
                                         ATOS Guidance and the Comprehensive
                                         Surveillance Plan




                                         completed by the principal inspectors and provide feedback. The SSAT is
                                         finalized at the annual meeting but may be revised during the year to
                                         retarget inspection resources.

Table I.2: Categories of System Safety
Used With SSAT                           Category                                         Definition
                                         Safety attributes                                ATOS identifies six safety attributes:
                                                                                          Responsibility: The unit or person in the
                                                                                          airline that determines the course of action
                                                                                          for a process.
                                                                                          Authority: The unit or person in the airline
                                                                                          that has the authority to establish or modify
                                                                                          a process.
                                                                                          Procedures: A documented method of
                                                                                          accomplishing a process.
                                                                                          Controls: A check or restraint that is
                                                                                          designed into a process to ensure a
                                                                                          desired result.
                                                                                          Process measurements: The unit or person
                                                                                          in the airline that measures and assesses
                                                                                          information to identify, detect, analyze, and
                                                                                          document problems or potential problems.
                                                                                          Interfaces: Points at which independent
                                                                                          processes interact.
                                         Safety culture                                   The priority given to safety by the airline’s
                                                                                          systems, including the airline’s
                                                                                          identification and response to safety risks,
                                                                                          and the effectiveness of internal evaluation
                                                                                          systems.
                                         Communications                                   The communication and feedback
                                                                                          channels within the airline to report and
                                                                                          respond to safety risks as well as open
                                                                                          and timely communication with FAA and
                                                                                          equipment manufacturers.
                                         Accountability                                   The extent to which the airline holds its
                                                                                          management and employees accountable
                                                                                          for their assigned responsibility and
                                                                                          authority.
                                         Training programs                                The priority an airline places on training as
                                                                                          well as the effectiveness of initial and
                                                                                          recurrent training programs.
                                         Potential problem areas                          The existence of concerns based on
                                                                                          previous accidents or incidents, hotline
                                                                                          complaints, or trends revealed in safety
                                                                                          data.
                                         Source: FAA Order 8400.10, Air Transportation Operations Inspector’s Handbook, appendix 6.




                                         Page 31                             GAO/RCED-99-183 FAA’s New Aviation Inspection System
                                  Appendix I
                                  ATOS Guidance and the Comprehensive
                                  Surveillance Plan




                                  By completing the SSAT, the ATOS inspection team assesses how well an
                                  airline addresses system safety issues. Using this information, the team
                                  determines whether to inspect any of the systems more or less frequently
                                  than suggested by the ATOS guidance and incorporates the inspections in
                                  the comprehensive surveillance plan for the airline.


                                  In completing the ACAT, principal inspectors use the results of the SSAT,
The Air Carrier                   their knowledge of the airline their team oversees, and any other available
Assessment Tool                   data to indicate concerns about any real or potential problem that could
                                  contribute to the failure of one of the airline’s elements, subsystems, or
                                  systems. The ACAT applies risk indicators to each of the airline’s systems.
                                  Table I.3 shows the types of risk indicators that are assessed when
                                  inspectors complete an ACAT.

Table I.3: ACAT Risk Indicators
                                  Type of risk indicator                           Definition and examples
                                  Operational stability                            Those aspects of an airline’s organization
                                                                                   and environment over which it has no
                                                                                   direct control and that, when managed
                                                                                   effectively, could enhance system safety
                                                                                   and stability (e.g., turnover in personnel, or
                                                                                   a merger or takeover).
                                  Airline dynamics                                 Aspects of an airline’s environment that it
                                                                                   directly controls and that could be used to
                                                                                   enhance system safety and stability (e.g.,
                                                                                   an internal evaluation program, and risk
                                                                                   management).
                                  Performance history                              The results of an airline’s operations over
                                                                                   time (e.g., enforcement actions,
                                                                                   self-disclosure reports to FAA).
                                  Environmental criticality                        Those aspects of an airline’s surroundings
                                                                                   that could lead to or trigger a failure of one
                                                                                   of its systems, subsystems, or elements
                                                                                   and potentially create an unsafe condition
                                                                                   (e.g., age of the fleet, outsourcing of
                                                                                   maintenance).
                                  Source: FAA Order 8400.10, Air Transportation Operations Inspector’s Handbook, appendix 6.



                                  The principal inspectors complete the ACAT prior to the annual planning
                                  meeting. During the annual planning meeting, the team members provide
                                  feedback on the ACAT that is included in the final version.




                                  Page 32                             GAO/RCED-99-183 FAA’s New Aviation Inspection System
                    Appendix I
                    ATOS Guidance and the Comprehensive
                    Surveillance Plan




                    The comprehensive surveillance plan is automatically generated based on
The Comprehensive   the information the team enters into ACAT. This provides a baseline
Surveillance Plan   surveillance plan that is tailored to the airline, reflecting concerns
                    indicated by the principal inspectors. Each comprehensive surveillance
                    plan incorporates two types of inspections, safety attribute inspections
                    (SAI) and element performance inspections (EPI). SAIs appraise the quality
                    of an airline’s safety attributes (see table I.2) for each system, its
                    subsystems, and its elements. A team of inspectors conducts these system
                    inspections. EPIs determine whether an airline adheres to its written
                    procedures and controls for each system element and whether the
                    established performance measures for each element are met. Individual
                    inspectors conduct these inspections.

                    ATOS  allows the principal inspector to increase or, in some cases, decrease
                    the level of inspection generated by the surveillance plan. This allows
                    principal inspectors to use their expertise and personal knowledge of the
                    airline to target resources toward the greatest safety risks.

                    Although the comprehensive surveillance plan is automatically generated
                    based on the results of the SSAT and ACAT, the plan is not finalized until the
                    annual inspection planning meeting, which is attended by all members of
                    the team. This allows the principal inspectors to discuss the completed
                    SSAT and ACAT and to make changes based on other inspectors’ feedback.
                    In addition, work assignments are discussed and made for each of the SAIs
                    and EPIs that are planned. Principal inspectors complete and approve the
                    final plan.


                    The ACAT provides information directly to the SAI planning system in ATOS,
Safety Attribute    which indicates an inspection priority for each of the airline’s subsystems.
Inspections         Considering the SAI priority, a principal inspector enters the number of SAIs
                    to be completed for each of the airline’s elements during the year.
                    Automation of the SAI also allows a principal inspector to assign teams for
                    each of the SAI activities. The principal inspector can also provide specific
                    instructions to the team regarding the inspections.

                    An SAI is an in-depth look at an airline’s policies and procedures for a
                    system element. This inspection is structured to look at the safety
                    attributes shown in table I.2. An SAI is completed by a team of inspectors,
                    led by a team coordinator. This team assesses the accuracy and
                    completeness of written policies and procedures governing each safety
                    attribute associated with one of the airline system elements. For example,



                    Page 33                       GAO/RCED-99-183 FAA’s New Aviation Inspection System
                      Appendix I
                      ATOS Guidance and the Comprehensive
                      Surveillance Plan




                      one element of the route structure system is the line stations for servicing
                      and maintaining the airline’s aircraft at each city it serves. Teams
                      performing line station SAIs determine if there are qualified persons
                      accountable for the line stations and if those persons have the authority to
                      change the processes governing those facilities. The teams also review the
                      procedures governing line stations to determine if proper controls are in
                      place—such as standards for the maintenance conducted at the line
                      station or for the training of line station employees. Finally, the teams
                      determine if processes are in place to identify and correct problems as
                      well as to ensure that other processes, such as de-icing and refueling, are
                      coordinated.


                      FAA established a baseline for how frequently each of the 79 ATOS elements
Element Performance   should be inspected. This frequency baseline identifies whether a system
Inspections           element should be inspected on an annual, semiannual, or quarterly basis
                      within the planning cycle. The ACAT calculates an assessment value that,
                      when applied to the frequency baseline, increases or decreases the
                      number of inspections based on the concerns the principal inspectors
                      have identified. Elements must be inspected at least once a year. Once the
                      principal inspectors have determined the number of EPIs that will be
                      conducted, the work is assigned to other inspectors on the team using the
                      automated system.

                      An EPI shows whether the airline follows the airline’s procedures and
                      controls. Individual inspectors conduct EPIs, which most resemble the
                      routine inspections FAA conducted in the past. For example, the line
                      station EPI requires an inspector to visit a line station to determine if
                      procedures and controls in place are being followed at that location. A line
                      station inspection under ATOS may include multiple visits to one location
                      or to a variety of other locations as well. All of these visits may be included
                      in a single EPI report.




                      Page 34                       GAO/RCED-99-183 FAA’s New Aviation Inspection System
Appendix II

Objectives, Scope, and Methodology


                  In September 1998, the Chairman and Ranking Democratic Member of the
                  Subcommittee on Aviation, House Committee on Transportation and
                  Infrastructure, asked us to address questions related to FAA’s new Air
                  Transportation Oversight System:

              •   To what extent does ATOS address past concerns about FAA’s aviation
                  safety inspections?

              •   What factors, if any, surfaced during the implementation of ATOS that could
                  impede its success?

              •   What is FAA doing to address any factors that could impede the success of
                  ATOS?


                  To determine to what extent ATOS addresses problems identified in the
                  past with FAA’s inspection program, we reviewed previous reports by GAO,
                  the Department of Transportation’s Inspector General, and internal FAA
                  reports, such as FAA 90 Day Safety Review. In addition, we attended ATOS
                  training provided to FAA inspectors and the annual inspection planning
                  meeting held by 1 of FAA’s 10 certificate management teams. We
                  interviewed members of the ATOS work group and program office to
                  discuss how the new program was developed and the agency’s intentions
                  for its implementation. In total, we interviewed 68 FAA employees assigned
                  to the airlines ATOS covers, including 64 of 540 ATOS inspectors and 4 of the
                  10 unit supervisors. The 64 inspectors included 28 of the 30 principal
                  inspectors who oversee ATOS airlines. In the two cases, we interviewed the
                  assistant principal inspectors because the principal inspectors were not
                  available. We discussed the ATOS concept, training, and implementation
                  with each inspector. In addition to the 68 FAA employees assigned to
                  oversee airlines under ATOS, we interviewed five Flight Standards district
                  office managers and supervisors who oversee ATOS field inspectors to gain
                  a broader perspective on inspector workload issues beyond those
                  involving ATOS inspections. We also reviewed all 10 comprehensive
                  surveillance plans developed for the 10 airlines covered by ATOS, as well as
                  inspection findings reported through May 11, 1999.

                  To determine what factors, if any, surfaced during the implementation of
                  ATOS that could impede its success, we interviewed FAA’s principal
                  inspectors for all 10 airlines covered by ATOS and also inspectors
                  (including field inspectors) assigned to the certificate management teams.
                  We interviewed staff from Sandia National Laboratories who served as
                  consultants on the ATOS project and reviewed Sandia’s reports on the



                  Page 35                      GAO/RCED-99-183 FAA’s New Aviation Inspection System
Appendix II
Objectives, Scope, and Methodology




system’s implementation. At FAA headquarters, we interviewed data
management and system officials as well as staff assigned to the System
Process Audit Group. We also reviewed reports prepared by these FAA
units on the implementation of ATOS. In addition, we discussed ATOS
specifics with key safety officials at the 10 airlines covered by the system.

To determine what FAA is doing to address any factors that could impede
the success of ATOS, we reviewed internal reports on ATOS and associated
recommendations. We discussed these recommendations and what impact
FAA’s budget shortfall will have on ATOS inspection efforts with principal
inspectors on the ATOS teams and with Flight Standards headquarters
officials.

We conducted our work from September 1998 through June 1999 in
accordance with generally accepted government auditing standards.




Page 36                          GAO/RCED-99-183 FAA’s New Aviation Inspection System
Appendix III

GAO Contacts and Staff Acknowledgments


                  John H. Anderson, Jr., (202) 512-2834
GAO Contacts      Robert White, (202) 512-5463


                  In addition to those named above, Leslie Albin, Bonnie A. Beckett, David
Acknowledgments   K. Hooper, Christopher M. Jones, Fran Featherston, Debra Prescott, and
                  Stan Stenerson made key contributions to this report.




                  Page 37                     GAO/RCED-99-183 FAA’s New Aviation Inspection System
Bibliography


               Booz-Allen & Hamilton Inc. Challenge 2000: Recommendations for Future
               Aviation Safety Regulation. Prepared for the Federal Aviation
               Administration. Apr. 1996.

               Federal Aviation Administration. FAA 90 Day Safety Review. Sept. 16, 1996.

               Sandia National Laboratories. “Analysis and Data Issues for the
               Implementation of a Systems Safety Focus in Air Transportation
               Oversight.” Oct. 6, 1998.

               U.S. Department of Transportation, Office of Inspector General. Audit of
               Aviation Inspection Program: Federal Aviation Administration
               (R6-FA-2-084). Washington, D.C.: May 29, 1992.

               U.S. Department of Transportation, Office of Inspector General. Final
               Report on Audit of Flight Standards Inspection Program in Southwest
               Region (R6-FA-9-078). Washington, D.C.: Mar. 17, 1989.

               U.S. Department of Transportation, Office of Inspector General. Pilot
               Examiner Program: Federal Aviation Administration (R2-FA-7-001).
               Washington, D.C.: Oct. 22, 1996.

               U.S. Department of Transportation, Office of Inspector General. Report on
               Audit of the Certification and Surveillance of Domestic and Foreign Repair
               Stations: Federal Aviation Administration (R4-FA-4-009). Washington,
               D.C.: Mar. 7, 1994.

               U.S. Department of Transportation, Office of Inspector General. Statement
               of Raymond J. DeCarli, Associate Deputy Inspector General, U.S.
               Department of Transportation. Washington, D.C.: May 21, 1997.

               White House Commission on Aviation Safety and Security. Final Report of
               the White House Commission on Aviation Safety and Security. Feb. 12,
               1997.




               Page 38                     GAO/RCED-99-183 FAA’s New Aviation Inspection System
Related GAO Products


              Aviation Safety: Weaknesses in Inspection and Enforcement Limit FAA in
              Identifying and Responding to Risks (GAO/RCED-98-6, Feb. 27, 1998).

              Aviation Safety: FAA Oversight of Repair Stations Needs Improvement
              (GAO/RCED-98-21, Oct. 24, 1997).

              Aviation Safety and Security: Challenges to Implementing the
              Recommendations of the White House Commission on Aviation Safety and
              Security (GAO/T-RCED-97-90, Mar. 5, 1997).

              Aviation Safety: New Airlines Illustrate Long-Standing Problems in FAA’s
              Inspection Program (GAO/RCED-97-2, Oct. 17, 1996).

              Aviation Safety: Targeting and Training of FAA’s Safety Inspector
              Workforce (GAO/T-RCED-96-26, Apr. 30, 1996).

              Aviation Safety: Data Problems Threaten FAA Strides on Safety Analysis
              System (GAO/AIMD-95-27, Feb. 8, 1995).

              FAA   Technical Training (GAO/RCED-94-296R, Sept. 26, 1994).

              Aviation Safety: FAA and the State Department Can Better Manage Foreign
              Enforcement Cases (GAO/RCED-94-87, Mar. 17, 1994).

              Aviation Safety: Progress on FAA Safety Indicators Program Slow and
              Challenges Remain (GAO/IMTEC-92-57, Aug. 31, 1992).

              Aviation Safety: FAA Needs to More Aggressively Manage Its Inspection
              Program (GAO/T-RCED-92-25, Feb. 6, 1992).

              Aviation Safety: Problems Persist in FAA’s Inspection Program
              (GAO/RCED-92-14, Nov. 20, 1991).

              Aviation Safety: Emergency Revocation Orders of Air Carrier Certificates
              (GAO/RCED-92-10, Oct. 17, 1991).

              Aviation Safety: FAA’s Safety Inspection Management System Lacks
              Adequate Oversight (GAO/RCED-90-36, Nov. 13, 1989).

              Aviation Training: FAA Aviation Safety Inspectors Are Not Receiving
              Needed Training (GAO/RCED-89-168, Sept. 14, 1989).




              Page 39                        GAO/RCED-99-183 FAA’s New Aviation Inspection System
           Related GAO Products




           Aviation Safety: Needed Improvements in FAA’s Airline Inspection Program
           Are Underway (GAO/RCED-87-62, May 19, 1987).




(348129)   Page 40                    GAO/RCED-99-183 FAA’s New Aviation Inspection System
Ordering Information

The first copy of each GAO report and testimony is free.
Additional copies are $2 each. Orders should be sent to the
following address, accompanied by a check or money order
made out to the Superintendent of Documents, when
necessary. VISA and MasterCard credit cards are accepted, also.
Orders for 100 or more copies to be mailed to a single address
are discounted 25 percent.

Orders by mail:

U.S. General Accounting Office
P.O. Box 37050
Washington, DC 20013

or visit:

Room 1100
700 4th St. NW (corner of 4th and G Sts. NW)
U.S. General Accounting Office
Washington, DC

Orders may also be placed by calling (202) 512-6000
or by using fax number (202) 512-6061, or TDD (202) 512-2537.

Each day, GAO issues a list of newly available reports and
testimony. To receive facsimile copies of the daily list or any
list from the past 30 days, please call (202) 512-6000 using a
touchtone phone. A recorded menu will provide information on
how to obtain these lists.

For information on how to access GAO reports on the INTERNET,
send an e-mail message with "info" in the body to:

info@www.gao.gov

or visit GAO’s World Wide Web Home Page at:

http://www.gao.gov




PRINTED ON    RECYCLED PAPER
United States                       Bulk Rate
General Accounting Office      Postage & Fees Paid
Washington, D.C. 20548-0001           GAO
                                 Permit No. G100
Official Business
Penalty for Private Use $300

Address Correction Requested