oversight

Aviation Safety: FAA Oversight of Repair Stations Needs Improvement

Published by the Government Accountability Office on 1997-10-24.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                 United States General Accounting Office

GAO              Report to Congressional Requesters




October 1997
                 AVIATION SAFETY
                 FAA Oversight of
                 Repair Stations Needs
                 Improvement




GAO/RCED-98-21
      United States
GAO   General Accounting Office
      Washington, D.C. 20548

      Resources, Community, and
      Economic Development Division

      B-274165

      October 24, 1997

      The Honorable Wendell H. Ford
      Ranking Minority Member
      Subcommittee on Aviation
      Committee on Commerce, Science, and Transportation
      United States Senate

      The Honorable Ron Wyden
      United States Senate

      As you requested, this report examines the Federal Aviation Administration’s (FAA) oversight of
      the aviation repair station industry. Specifically, this report addresses the following questions:
      (1) What is the nature and scope of the oversight of repair stations conducted by FAA personnel?
      (2) How well does FAA follow up on inspections to ensure that the deficiencies in repair station
      operations are corrected once they have been identified? (3) What steps has FAA taken to
      improve the oversight of repair stations? This report contains recommendations to the
      Secretary of Transportation for improving FAA’s oversight of repair stations.

      As you requested, unless you publicly announce its contents earlier, we plan no further
      distribution of this report until 30 days after the date of this letter. We will then send copies to
      the Secretary of Transportation; the Director, Office of Management and Budget; and other
      interested parties. We will make copies available to others upon request.

      If you or your staff have any questions, please call me at (202) 512-3650. Major contributors to
      this report are listed in appendix IV.




      Gerald L. Dillingham
      Associate Director,
        Transportation Issues
Executive Summary


                U.S. airlines and air cargo companies now operate more than 6,700
Purpose         aircraft, nearly 1,000 more than in 1990. Maintaining, repairing, and
                renovating this fleet costs about $6.5 billion a year. Nearly half of this
                work is now done by about 2,800 independent repair stations rather than
                by the air carriers themselves. Located worldwide, these repair stations
                vary greatly in size and scope. Some employ only a few people and fix a
                limited range of components, such as radios or instruments. Others have
                thousands of workers doing everything from conducting routine engine
                maintenance to rebuilding entire airframes. Although repair stations have
                been part of the aviation industry for decades, their use has grown
                substantially in recent years, particularly by airlines and cargo companies
                just entering the market. These new carriers have found it more
                economical to contract out much of their maintenance work rather than
                hiring their own staffs and building extensive facilities.

                Because repair stations deal with virtually all aircraft components,
                ensuring that their work is competently done is an important part of
                enhancing aviation safety. The Federal Aviation Administration (FAA) is the
                federal agency responsible for doing so. FAA conducts this oversight in two
                main ways: by sending its own inspectors to review repair station
                operations and by making airlines and air cargo companies responsible for
                ensuring that repair stations are following proper procedures. In recent
                years, FAA’s oversight of repair stations has become a matter of concern, in
                part because work performed by repair stations has been identified as a
                factor in several aircraft accidents. For example, the National
                Transportation Safety Board determined that the probable cause of an
                engine fire that destroyed a ValuJet DC-9 on an Atlanta runway in
                June 1995 was the inadequate procedures used by repair station personnel.

                The Ranking Minority Member of the Aviation Subcommittee of the Senate
                Committee on Commerce, Science, and Transportation, and Senator Ron
                Wyden asked GAO to examine FAA’s oversight of repair stations. GAO’s
                review focused on FAA’s own inspection activities at repair stations. At a
                later date, GAO plans to conduct a more detailed examination of FAA’s role
                in requiring airlines and air cargo companies to ensure that repair stations
                are following proper procedures. This current review examined the
                following questions:

            •   What is the nature and scope of the oversight of repair stations conducted
                by FAA personnel?
            •   How well does FAA follow up on inspections to ensure that the deficiencies
                in repair stations’ operations are corrected once they have been identified?



                Page 2                                           GAO/RCED-98-21 Aviation Safety
                       Executive Summary




                   •   What steps has FAA taken to improve the oversight of repair stations?


                       As the size of the airline and air cargo industries has grown, so has the
Background             reliance on repair stations. In 1990, repair stations performed 37 percent of
                       air carriers’ maintenance; by 1996, the figure was 46 percent. More than
                       2,500 domestic and 270 foreign repair stations do work for air carriers.
                       FAA’s certification process establishes what the repair stations are qualified
                       to do. While many repair stations have fewer than 15 employees and a
                       limited range of activities that FAA has certified, some employ thousands of
                       workers who completely overhaul engines and renovate aging airframes
                       for additional years of service. FAA’s inspection activities are aimed at
                       ensuring that repair stations are still meeting the certification
                       requirements.

                       FAA had about 3,000 inspectors in fiscal year 1997. About 600 of them were
                       involved in repair station inspections. FAA’s guidelines require that each
                       repair station be inspected at least once a year. These inspections involve
                       checking such matters as whether repair station staff have the appropriate
                       qualifications to do the work and whether repair procedures meet FAA
                       regulations. About 550 inspectors oversee repair stations located in the
                       United States, usually through inspections conducted by individual
                       inspectors. For larger facilities, these inspections may take place over
                       several visits. Most of the inspectors are responsible for several repair
                       stations as well as other types of operations, such as helicopter operators
                       and training schools for pilots and mechanics. The remaining 50 inspectors
                       inspect foreign repair stations that work on aircraft registered in the
                       United States. These inspectors have fewer additional duties because FAA
                       generally has no regulatory authority over foreign operations that do not
                       directly affect aviation in the United States. Unlike their counterparts on
                       the domestic side, these inspectors conduct most of their repair station
                       inspections in teams.


                       FAA’srecords indicate that the agency is meeting its goal of inspecting
Results in Brief       every repair station at least once a year. GAO examined FAA’s 1996
                       inspection records on about one-fourth of the 2,800 repair stations doing
                       work for air carriers and confirmed that minimum inspection requirements
                       had been met. In addition, 84 percent of the inspectors GAO surveyed
                       stated that they believed the overall compliance of repair stations was
                       good or excellent. However, more than half of the inspectors stated that
                       there were areas of compliance that repair stations could improve. FAA



                       Page 3                                            GAO/RCED-98-21 Aviation Safety
Executive Summary




relies primarily on reviews by individual inspectors of most domestic
repair stations. In a few cases, FAA also uses teams to assess compliance at
large, complex facilities. At such facilities, a team approach has been
shown to be more effective at identifying problems than visits by
individual inspectors, uncovering more systemic and long-standing
deficiencies. A few of FAA’s offices have recognized that the traditional
approach of relying on one inspector may be inadequate in such situations
and have begun to use teams to inspect large repair stations. FAA officials
acknowledge and support these initiatives. They said they believe these
initiatives need to be evaluated and, if appropriate, used at other offices.

GAO  could not find sufficient documentation to determine how well FAA
followed up to ensure that the deficiencies found during the inspections of
repair stations were corrected. Thus, it was impossible to assess how
completely or quickly repair stations were bringing themselves into
compliance. FAA does not tell its inspectors what documentation to keep,
and the resulting information gaps lessen the agency’s ability to determine
how well its inspection activities are working or to identify and react to
trends. These gaps in documentation are particularly important because
FAA is spending more than $30 million to develop a reporting system that,
among other things, is designed to use the documentation to make
inspection decisions, such as where to apply the agency’s inspection
resources to address those areas that pose the greatest risk to aviation
safety.

Following the May 1996 crash of a ValuJet DC-9 in the Florida Everglades,
FAA announced new initiatives to upgrade the oversight of repair stations.
These initiatives were directed at clarifying and augmenting air carriers’
oversight of repair stations, not at ways in which FAA’s own inspection
resources could be better utilized. However, FAA does have three other
efforts under way that would have a more direct bearing on its own
inspection activities at repair stations. One effort would revise the
regulations governing repair station operations, and another would revise
the regulations governing the qualifications of repair station personnel.
The revision of the repair station regulations began in 1989 and has been
repeatedly delayed. The third effort is the addition of more FAA inspectors,
which should mean that more resources can be devoted to inspecting
repair stations.




Page 4                                           GAO/RCED-98-21 Aviation Safety
                        Executive Summary




Principal Findings

Current Inspection      Most of FAA’s offices use the approach of assigning an individual inspector
Approach Limits FAA’s   to a repair station, even one that is large and complex, rather than
Ability to Ensure       assigning a team of inspectors. Although this one-inspector approach
                        constitutes FAA’s primary frontline surveillance of repair stations, each
Compliance at Large     year regional and national decisions are made to use teams for more
Repair Stations         comprehensive reviews of a few repair stations. When direct comparisons
                        could be made, teams were shown to be more effective than individual
                        inspectors in identifying those areas in which repair stations were not in
                        compliance with FAA’s rules and regulations, even if one inspector visited
                        the facility several times and the team visited it just once. GAO reviewed 19
                        instances in which large repair stations inspected by one person had also
                        been inspected by a special team during the same year. These special
                        inspections are conducted at selected facilities that FAA regards as needing
                        additional attention. The teams found a total of 347 deficiencies, only 15 of
                        which had been identified in all of the visits made by individual inspectors
                        in the year or more leading up to the special inspections. Deficiencies the
                        teams identified included many that were systemic and apparently
                        long-standing, such as inadequate training programs or poor manuals for
                        quality control. Such deficiencies were likely to have been present when
                        the repair stations were inspected earlier by individual inspectors.

                        There are several reasons why team inspections identify a higher
                        proportion of the deficiencies that may exist in the operation of large
                        repair stations. Teams are better than individuals at ensuring that the
                        inspection covers all areas of operations and that inspectors stay focused
                        on the task at hand. Many FAA inspectors responsible for conducting
                        inspections on their own said that because they have many competing
                        demands on their time, their inspections of repair stations may not be as
                        thorough as they would like. Another reason is that team inspections make
                        greater use of checklists or other job aids for ensuring that all points are
                        covered. FAA’s guidance requires inspectors to address all aspects of repair
                        stations’ operations but does not prescribe any checklist or other means
                        for specifying the items to be covered. The lack of a standardized
                        approach increases the possibility that items will not be covered. Finally,
                        inspectors believe team inspections help ensure that their judgments are
                        independent because most team members have no ongoing relationship
                        with the repair station. By contrast, individual-inspector reviews are




                        Page 5                                           GAO/RCED-98-21 Aviation Safety
                     Executive Summary




                     conducted by personnel who have continuing regulatory responsibility for
                     the facilities.

                     A few of FAA’s offices have recognized that the traditional approach of
                     relying on one inspector may be inadequate for overseeing the operations
                     of large repair stations and have reconfigured their inspection resources to
                     do more team inspections without adversely affecting other duties. They
                     have done so mainly by redirecting the time formerly spent on reviews by
                     individual inspectors into more systematic inspections done by a team of
                     local, in-house staff. GAO identified FAA offices in Scottsdale, Arizona;
                     Miami, Florida; and Seattle, Washington, as having initiated such changes
                     on their own. FAA headquarters officials acknowledge and support these
                     offices’ initiatives. They said they believe these initiatives need to be
                     evaluated and, if appropriate, used at other offices.


Follow-Up and        FAA’s guidance is limited in specifying for inspectors what documents
Documentation Need   pertaining to inspections and follow-up need to be maintained in repair
Attention            station files. The closest thing to a requirement is a statement in the
                     Airworthiness Inspector’s Handbook that the deficiency letter FAA sends to
                     the repair station describing all deficiencies should be included in the
                     repair station case file. GAO examined records of 172 instances in which
                     FAA sent deficiency letters to domestic repair stations. The responses from
                     the repair stations were not on file in about one-fourth of these instances,
                     and FAA’s assessments of the adequacy of the corrective actions taken by
                     the repair stations were not on file in about three-fourths of the instances.
                     GAO also examined computer-based reports summarizing inspection
                     information for FAA managers and found these reports were even less
                     complete. Without complete documentation, it was impossible to assess
                     how completely or quickly repair stations were bringing themselves into
                     compliance.

                     Better documentation is needed not only to allow FAA to demonstrate how
                     quickly and thoroughly repair stations are complying with regulations, but
                     also because it can affect FAA’s ability to identify performance trends
                     involving the inspection of repair stations and to make informed decisions
                     about them. FAA is spending more than $30 million to develop a reporting
                     system that, among other things, is designed to use this documentation to
                     make decisions on applying inspection resources to those areas posing the
                     greatest risk to aviation safety. Such a system will be of limited use if the
                     documentation on which it is based is inaccurate, incomplete, or outdated.
                     FAA must have data to show where safety problems and deficiencies exist




                     Page 6                                           GAO/RCED-98-21 Aviation Safety
                            Executive Summary




                            and, thus, where to better target its limited inspection resources. In 1995,
                            as part of a prior study examining FAA’s information management systems,
                            GAO recommended that FAA develop a comprehensive strategy for making
                            data-related improvements. FAA agreed, but it fell behind in its schedule for
                            making improvements. Continued monitoring will be needed to ensure
                            that the actions taken are sufficient to resolve the problems by
                            December 1999, when the new reporting system is scheduled for
                            completion.

                            Documentation of inspections and follow-up was better in FAA’s files for
                            foreign repair stations, perhaps in part because under FAA regulations,
                            foreign repair stations must renew their certification every 2 years. By
                            comparison, domestic repair stations retain their certification indefinitely
                            unless they surrender it or FAA suspends or revokes it. Foreign repair
                            stations appear to be correcting their deficiencies quickly so that they
                            qualify for certificate renewal. The 34 FAA inspectors GAO interviewed who
                            had conducted inspections of both foreign and domestic repair stations
                            were unanimous in concluding that compliance occurred more quickly at
                            foreign facilities. They attributed the quicker compliance to the renewal
                            requirement and said that it allowed them to spend less time on follow-up,
                            freeing them for other surveillance work. However, because of the poor
                            documentation in domestic repair station files, GAO was unable to confirm
                            whether foreign repair stations achieve compliance more quickly than
                            domestic repair stations do.


Actions Under Way           The six repair station initiatives announced in June 1996 by the previous
Directed Primarily at Air   FAA Administrator following the ValuJet crash are directed at clarifying

Carriers’ Oversight of      and augmenting air carriers’ responsibilities for overseeing repair stations.
                            For example, one initiative requires that before an air carrier can add a
Repair Stations             repair station to the list of repair stations doing substantial maintenance
                            on its aircraft, the carrier must conduct an audit to verify that the repair
                            station is capable of doing the work in accordance with the carrier’s
                            approved programs. GAO did not directly assess the initiatives in this
                            review because the initiatives are not focused on strengthening FAA’s own
                            inspection and follow-up efforts. FAA inspectors assigned to oversee repair
                            stations told GAO that the initiatives would have no effect on their direct
                            inspections of repair stations.

                            Several other efforts unrelated to the June 1996 initiatives may hold
                            potential for improving FAA’s own inspections of repair stations. Two
                            involve initiatives to change the regulations covering repair station



                            Page 7                                           GAO/RCED-98-21 Aviation Safety
                      Executive Summary




                      operations and the certification requirements for mechanics and
                      repairmen. FAA acknowledges that the existing regulations do not reflect
                      many of the technological changes that have occurred in the aviation
                      industry in recent years. The FAA inspectors surveyed by GAO strongly
                      supported a comprehensive update of repair station regulations as a way
                      to improve repair stations’ compliance. This update began in 1989, has
                      been repeatedly delayed, and still remains in process. The most recent
                      target—to have draft regulations for comment published in the Federal
                      Register during summer 1997—was not met. Similarly, the update of the
                      certification requirements for maintenance personnel has been suspended
                      since 1994. Because of these long-standing delays, completion of both
                      updates may require additional attention on management’s part to help
                      keep both efforts on track. The third effort involves increasing FAA’s
                      inspection resources: Since fiscal year 1995, FAA has been in the process of
                      adding more than 700 inspectors to its workforce who will, in part,
                      oversee repair stations. Survey responses from current inspectors
                      indicated that the success of this effort will depend partly on the
                      qualifications of the new inspectors and on the training available to all
                      those in the inspector ranks.


                      To improve FAA’s oversight of repair stations, GAO recommends that the
Recommendations       Secretary of Transportation instruct the Administrator of FAA to take the
                      following actions:

                  •   Expand the use of locally based teams for repair station inspections,
                      particularly for repair stations that are large, complex, have higher rates of
                      noncompliance, or meet predetermined risk indicators; and develop and
                      use checklists or job aids for inspectors as a way of bringing about more
                      comprehensiveness and standardization.
                  •   Specify what documentation should be kept in repair station files to
                      record complete inspection results and follow-up actions.
                  •   Monitor the implementation of the strategy for improving the quality of the
                      data to be used in FAA’s new management information system.
                  •   Expedite the efforts to update regulations pertaining to the oversight of
                      repair stations, and establish and meet schedules for completing the
                      updates.


                      GAO provided the Department of Transportation and FAA with a draft of this
Agency Comments       report for their review and comment. GAO met with FAA officials, including
                      the Deputy Associate Administrator for Regulation and Certification



                      Page 8                                            GAO/RCED-98-21 Aviation Safety
Executive Summary




(acting on behalf of the FAA Administrator) to obtain FAA’s comments. FAA
agreed with the draft’s overall message and recommendations.

FAA said it will build on its already successful repair station inspection
program to enhance the oversight of this sector of the aviation industry.
FAA cited several agency initiatives that it said are under way to do that.
FAA’s 90-day safety review conducted last year recommended the creation
of an analytic unit that could provide safety trend data to inspectors. FAA
said an office within the Flight Standards Service was created on May 20,
1997, to provide data that will help focus inspection and other resources.
The review also recommended that field and division managers be given
the flexibility to determine the skills needed in a particular field office to
ensure the appropriate mix of technical, paratechnical, support, and
clerical expertise. FAA said that this flexibility will be supported through
the establishment of new staffing standards—a long-term project that is
already under way.

Additionally, FAA provided some technical comments and corrections on
the draft report. GAO revised the report as appropriate to reflect FAA’s
technical comments and corrections.




Page 9                                             GAO/RCED-98-21 Aviation Safety
Contents



Executive Summary                                                                                  2


Chapter 1                                                                                         14
                         Growing Air Carrier Industry Creates Additional Need for                 14
Background                 Maintenance and Repair Services
                         What Are Repair Stations?                                                15
                         How Does FAA Oversee Repair Stations?                                    19
                         Who Else Has Oversight Responsibility for Repair Stations?               21
                         Why Has FAA’s Oversight of Repair Stations Been a Matter of              22
                           Concern?
                         Objectives, Scope, and Methodology                                       25

Chapter 2                                                                                         27
                         Number of Inspections Conducted Meets the Required Minimum               27
Current Inspection       Team Inspections Provide More Detailed Review Than Those                 30
Approach Limits            Conducted by Individual Inspectors
                         Different Approaches to Repair Station Surveillance Show                 39
FAA’s Ability to           Promise
Ensure Compliance        Conclusions                                                              47
                         Recommendations                                                          47
                         Agency Comments                                                          48

Chapter 3                                                                                         49
                         Information on Inspection Results Is Important but Incomplete            49
Incomplete               Compliance May Come More Quickly at Foreign Repair Stations              56
Documentation of         Conclusions                                                              58
                         Recommendations                                                          59
Inspections Makes        Agency Comments                                                          59
Extent of Follow-Up
Difficult to Determine
Chapter 4                                                                                         60
                         Existing Initiatives Are Targeted Primarily at Air Carriers              60
Actions Currently        Planned Update of Repair Station Regulations Is Slow in Coming           65
Under Way to             Adding to the Inspector Corps and Improving Training                     68
                         Upgrading Certification and Training of Aviation Maintenance             70
Augment Oversight of       Personnel
Repair Stations          Conclusions                                                              71
                         Recommendation                                                           71
                         Agency Comments                                                          71




                         Page 10                                       GAO/RCED-98-21 Aviation Safety
             Contents




Appendixes   Appendix I: Objectives, Scope, and Methodology                            72
             Appendix II: Survey Methodology                                           77
             Appendix III: Survey Questions and Responses                              81
             Appendix IV: Major Contributors to This Report                            91

Tables       Table 1.1: Repair Station Rating Categories                               15
             Table 1.2: Comparison of FAA’s Oversight of Domestic and                  20
               Foreign Repair Stations
             Table 1.3: Recent Accidents Involving Aircraft Maintained by              24
               Contract Repair Stations
             Table 2.1: Characteristics of a Quality Inspection                        32
             Table I.1: Airlines Included in GAO’s Detailed Review                     73
             Table I.2: Repair Stations Included in Detailed Review                    74
             Table I.3: Distribution of Repair Stations Overseen by FAA                75
               Inspectors, by Size
             Table II.1: Sampling Errors for Selected Percentages                      78
             Table II.2: Sampling Errors for Repair Stations by Number of              78
               Employees
             Table II.3: Summary of Returns to Mail Surveys                            80

Figures      Figure 1.1: Types of Maintenance Performed by Repair Stations             16
             Figure 1.2: Maintenance Being Done on a Boeing 727 at Tramco,             18
               Inc., Everett, Washington
             Figure 1.3: Overview of the Oversight Given Repair Stations               22
             Figure 2.1: Types of Deficiencies FAA Teams Identified During             31
               Special, In-Depth Inspections
             Figure 2.2: FAA Inspectors’ Responses on Barriers to                      34
               Comprehensive Inspections
             Figure 2.3: FAA Inspectors’ Responses on Barriers to Focusing             35
               During Inspections
             Figure 2.4: FAA Inspectors’ Views on the Current Overall                  37
               Compliance of Repair Stations
             Figure 2.5: FAA Inspectors’ Responses on Amount of                        39
               Improvement Needed for Repair Station Compliance
             Figure 2.6: FAA Inspectors’ Responses on Using Teams to                   45
               Improve Compliance
             Figure 3.1: Extent of Follow-Up Documentation for Inspections             52
               of Selected Domestic Repair Stations, Fiscal Years 1993 Through
               1996




             Page 11                                        GAO/RCED-98-21 Aviation Safety
Contents




Figure 3.2: Extent of Follow-Up Documentation for Selected                53
  Foreign Repair Station Inspections, in Fiscal Years 1993 Through
  1996
Figure 4.1: June 1996 Initiatives Designed to Improve Air Carrier         62
  Oversight of Repair Stations
Figure 4.2: FAA Inspectors’ Responses to Ideas for Improving              67
  Compliance Through Changes Related to Existing Regulations
Figure 4.3: FAA Inspectors’ Responses to Ideas for Improving              69
  Compliance Through Inspector Qualifications




Abbreviations

FAA        Federal Aviation Administration
JAA        Joint Aviation Authorities
NASIP      National Aviation Safety Inspection Program
PTRS       Program Tracking and Reporting Subsystem
RASIP      Regional Aviation Inspection Program
SPAS       Safety Performance Analysis System


Page 12                                        GAO/RCED-98-21 Aviation Safety
Page 13   GAO/RCED-98-21 Aviation Safety
Chapter 1

Background


                      Passenger airlines, air freight companies, and other air carriers in the
                      United States spend almost $6.5 billion every year maintaining and
                      repairing their aircraft, according to industry estimates. While these
                      carriers have traditionally performed much of this maintenance and repair
                      work themselves, many are now contracting an increasing portion of the
                      work to about 2,800 repair stations in the United States and other
                      countries.1 As the agency responsible for overseeing the aviation industry,
                      the Federal Aviation Administration (FAA) has the primary responsibility
                      for ensuring that repair stations are operating in accordance with laws and
                      regulations.


                      Commercial air carriers certified in the United States now operate more
Growing Air Carrier   than 6,700 aircraft, nearly 1,000 more than in 1990. Operators include more
Industry Creates      than 150 airlines, freight carriers, charter firms, and other companies
Additional Need for   certified by FAA and operating under part 121 of the Federal Aviation
                      Regulations.2 The aircraft they operate range from planes such as a
Maintenance and       Fairchild Metroliner III, which typically carries a maximum of 19
Repair Services       passengers or about 5,000 pounds of cargo, to planes such as a Boeing
                      747-400, which is capable of carrying more than 400 passengers or 122 tons
                      of cargo. Some of the largest companies, like United Airlines or American
                      Airlines, may have 500 or more aircraft.

                      With more aircraft flying, the need for maintenance and repair services has
                      grown. Air carriers spent almost $6.5 billion for maintenance and repair on
                      their aircraft in 1996, according to an industry estimate.3 This amount is an
                      increase of $1.2 billion, or 23 percent, over the estimate of $5.3 billion in
                      1990. The term “maintenance and repair” encompasses a wide variety of
                      activities. Some activities involve frequent servicing, such as overhauling
                      tires, wheels, and brakes. Others include more extensive renovation, such
                      as airframe maintenance, that must be done as aircraft get older. FAA
                      classifies maintenance and repair activities into six rating categories (see


                      1
                       Although nearly 5,000 repair stations are certified by FAA, data provided in FAA’s Vital Information
                      Subsystem specifically identified about 2,800 of those as performing work on aircraft with 10 or more
                      seats. Of these facilities, more than 2,500 are in the United States, and 273 are in foreign countries. This
                      report addresses the oversight of these 2,800 repair stations.
                      2
                       Air carriers now operating under part 121 use aircraft configured for 10 or more passengers. New
                      rules adopted by the FAA in 1995 require certain commuter operators conducting scheduled
                      operations under part 135 to conduct those operations under part 121 beginning in March 1997.
                      Included were those air carriers conducting scheduled operations carrying passengers with aircraft
                      configured for 10 to 30 seats. This report uses the term “air carriers” to refer to companies operating
                      under part 121, including those that formerly operated under part 135.
                      3
                       This figure includes air carriers with revenues exceeding $100 million annually.



                      Page 14                                                              GAO/RCED-98-21 Aviation Safety
                                   Chapter 1
                                   Background




                                   table 1.1), which it uses to designate the type of maintenance or activity it
                                   has certified a repair station to perform.

Table 1.1: Repair Station Rating
Categories                         Ratinga                                              Examples of service performed
                                   Accessory                                            Functional check and calibration of fuel
                                                                                        control unit
                                   Airframe                                             Inspection and repair for corrosion and
                                                                                        fatigue damage
                                   Instrument                                           Calibration of air speed indicator
                                   Power plant                                          Borescope inspection of internal engine
                                                                                        components
                                   Propeller                                            Examinations and repair of cracks, nicks,
                                                                                        and deformations
                                   Radio                                                Measurement of frequency and power of
                                                                                        transmitting unit
                                   a
                                    FAA also issues limited ratings for items such as nondestructive testing, maintenance on
                                   emergency equipment or landing gear, or other specialized services not included in the aircraft
                                   rating categories.



                                   Some major air carriers, such as American Airlines and United Airlines,
                                   have substantial maintenance facilities of their own. However, many air
                                   carriers, including smaller air carriers, have used third-party repair
                                   stations rather than invest in the additional staff and hardware needed to
                                   do the work in-house. Some new air carriers entering the passenger or air
                                   freight markets have chosen to rely heavily—and in some cases, almost
                                   exclusively—on repair stations.


                                   The term “repair station” spans a wide variety of operations. In 1996, there
What Are Repair                    were almost 5,000 repair stations certified by FAA, about 2,800 of which
Stations?                          performed maintenance work on aircraft used by air carriers.4 A repair
                                   station’s certificate specifies the types of maintenance it can perform.
                                   Some repair stations specialize in one particular maintenance and repair
                                   category, while others may conduct work in several categories. As figure
                                   1.1 shows, the types of maintenance most often included in the certificates
                                   of these 2,800 repair stations were for accessories and airframes.




                                   4
                                    The other 2,200 certified repair stations worked on general aviation aircraft which are regulated under
                                   part 91 of the Federal Aviation Regulations. General aviation comprises all civil aircraft operations
                                   except those involving such commercial activities as the transport of revenue-paying passengers. Over
                                   90 percent of the aircraft registered in the United States are general aviation aircraft.



                                   Page 15                                                           GAO/RCED-98-21 Aviation Safety
                                          Chapter 1
                                          Background




Figure 1.1: Types of Maintenance Performed by Repair Stations

                                      FAA-certified repair
                                      stations performing
                                      work for air carriers:                     Type of maintenance
                                      2,800                                      covered in certification

                                                                  Accessories

                                                                     Airframes
              56%
                                                                 Power plants

                                      All other                    Instruments
                                      FAA-certified
                44%                   repair stations:                   Radio
                                      2,200
                                                                    Propellers

                                                                                 0     200      400     600      800    1,000 1,200
                                                                                 Number of repair stations


                                          Note: Some repair stations are certified for more than one type of maintenance.

                                          Source: FAA data.




                                          In addition to specifying the types of maintenance a repair station can
                                          perform, FAA may limit the scope of a repair station’s activities. For
                                          example, whenever appropriate, FAA may issue a rating that limits a repair
                                          station’s work to maintaining or altering only certain types of airframes,
                                          power plants, propellers, radios, instruments, or accessories. Such a rating
                                          may be limited to a specific model of aircraft, engine, or constituent part
                                          or to any number of parts made by a particular manufacturer. FAA also
                                          issues limited ratings for specialized work, such as nondestructive testing,
                                          maintenance on landing gear or emergency equipment, or other specific
                                          areas not included in any of the six standard rating categories.5



                                          5
                                           As of Nov. 15, 1996, 251 repair stations performing work for air carriers held one or more limited
                                          ratings for items such as nondestructive testing, maintenance on emergency equipment or landing
                                          gear, or other specialized service or maintenance not included in the aircraft rating categories.



                                          Page 16                                                             GAO/RCED-98-21 Aviation Safety
Chapter 1
Background




Repair stations vary considerably in size and scope of operations. For
example, Tramco, Inc., located in Everett, Washington, is one of the
largest repair stations in the United States, with hangar facilities of 450,200
square feet and a workforce of more than 2,000. At one time, this facility
can accommodate five wide-body aircraft, such as Boeing 747s, and five
narrow-body aircraft, such as Boeing 737s. The repair station primarily
conducts regularly scheduled maintenance and modifications, and it also
modifies new aircraft when specifications are changed after
manufacturing is completed. Figure 1.2 shows maintenance being done on
a Boeing 727 at this facility. By contrast, Precision Avionics & Instruments
in Atlanta, Georgia, is a much smaller repair station. It employs 35 workers
and has a facility of 24,000 square feet where it primarily services
instruments, electrical and electronic components and accessories.




Page 17                                           GAO/RCED-98-21 Aviation Safety
                                          Chapter 1
                                          Background




Figure 1.2: Maintenance Being Done on a Boeing 727 at Tramco, Inc., Everett, Washington




                                          While most domestic repair stations are operated independently of
                                          commercial airlines, a few are in-house maintenance operations that
                                          conduct work for other airlines on a contractual basis. For example, at its
                                          own maintenance facilities, Delta Airlines performs power plant
                                          maintenance for such carriers as American Airlines, Air Jamaica, Trade
                                          Winds, and Aeroflot Russian International Airlines.

                                          Repair stations that work on the aircraft of U.S. carriers are found
                                          throughout the rest of the world, though not in as great a number as repair



                                          Page 18                                         GAO/RCED-98-21 Aviation Safety
                 Chapter 1
                 Background




                 stations in the United States. In all, about 270 FAA-certified foreign repair
                 stations perform repair work for U.S. air carriers. For example, Sabena
                 Technic, the maintenance arm of Sabena Belgian World Airlines, does
                 engine repair work for Federal Express and other carriers at its facility in
                 Brussels. Sabena has FAA’s approval for work on airframes, power plants,
                 radios, instruments, and accessories.


                 FAA’s oversight of repair stations is divided into two phases—certification
How Does FAA     and surveillance. Certification initially involves a repair station’s applying
Oversee Repair   to FAA for authority to perform certain types of maintenance on certain
Stations?        types of aircraft. FAA inspects the repair station to ensure that the
                 applicant’s proposed procedures are effective and that the equipment
                 meets regulatory requirements. In addition, FAA also inspects facilities,
                 personnel, and material as well as the repair station’s inspection system. If
                 FAA finds these things to be in order, it issues a certificate with a set of
                 “operation specifications” that cover what maintenance activities the
                 repair station is authorized to perform. Certification is handled in one of
                 two ways, depending on whether the repair station is in the United States
                 or abroad. FAA requires foreign repair stations to renew their certification
                 at least every 2 years, but for domestic repair stations, certification is
                 permanent unless it is surrendered by the applicant or suspended or
                 revoked by FAA.6

                 Surveillance, usually in the form of inspections, follows certification. FAA’s
                 guidelines require its safety inspectors to perform a facility inspection of
                 each domestic and foreign repair station at least once every year. For
                 many of the larger domestic repair stations, this inspection is broken into
                 multiple visits. For example, FAA inspectors visited Evergreen Air Center,
                 one of the larger repair stations we reviewed in depth, more than 20 times
                 during fiscal year 1996. Located in Marana, Arizona, Evergreen employs
                 about 590 workers who conduct all types of maintenance on most types of
                 large transport aircraft. FAA divides repair station inspections into two
                 categories, avionics and maintenance. Avionics inspections focus on a
                 repair station’s overall program for aircraft electronic components,
                 including personnel training, policies, and procedures. Maintenance
                 inspections cover a repair station’s overall maintenance program,
                 including personnel training, policies, and procedures.


                 6
                  Repair stations are regulated under 14 C.F.R. part 145. Specifically, under FAA regulations, a foreign
                 repair station’s certificate, or rating, expires 12 months after the date on which it was issued, unless it
                 is surrendered, suspended, or revoked before that time. FAA can make subsequent renewals for
                 periods of up to 24 months.



                 Page 19                                                              GAO/RCED-98-21 Aviation Safety
                                    Chapter 1
                                    Background




                                    FAA’s certification and inspection activities are carried out by inspectors
                                    based in the United States and abroad (see table 1.2).7 On the domestic
                                    side, certification and inspection activities are carried out by more than
                                    550 FAA inspectors, most of whom have many other responsibilities as
                                    well. Unless they are assigned to one of the largest operations, inspectors
                                    usually are responsible for more than one repair station. We examined the
                                    workloads of 98 inspectors at the FAA offices we visited and found that the
                                    number of repair stations these inspectors were responsible for ranged
                                    from 1 to 42, with a median workload of 12 repair stations. These repair
                                    stations varied in size and complexity. Most of the inspectors had many
                                    other surveillance responsibilities as well, such as overseeing training
                                    schools for pilots and mechanics, helicopter operators, agricultural
                                    operators, and air taxis. On the foreign side, about 50 FAA inspectors
                                    handle the oversight of repair stations, again with responsibility for
                                    multiple repair stations. Unlike their counterparts in the United States,
                                    however, inspectors in these offices generally have the oversight of repair
                                    stations as their primary responsibility.

Table 1.2: Comparison of FAA’s
Oversight of Domestic and Foreign                                        Domestic                           Foreign
Repair Stations                     Number of FAA principal              552                                49
                                    inspectors assigned to
                                    oversee repair stations as of
                                    Nov. 15, 1996
                                    Location of FAA principal            86 FAA offices throughout          7 FAA offices—Brussels,
                                    inspectors assigned to               the United States                  Dallas/Fort Worth, Frankfurt,
                                    oversee repair stations                                                 London, Miami, San
                                                                                                            Francisco, and Singapore
                                    Number of repair stations      2,504a                                   273a
                                    inspectors are responsible for
                                    Responsibility for oversight of Varies with other oversight             Primary responsibility
                                    repair stations                 duties
                                    a
                                     These numbers only include those repair stations identified in FAA’s Vital Information Subsystem
                                    as performing work on aircraft with 10 or more seats.




                                    7
                                     In addition to inspectors with direct oversight, other FAA inspectors may also visit repair stations. An
                                    inspector responsible for an air carrier that contracts with a repair station may also review the repair
                                    station’s operations, but only insofar as they pertain to the work being done for the air carrier. If an
                                    inspector is not based near a repair station, he or she may request that another inspector—called a
                                    geographic inspector—close to the facility make the visit.



                                    Page 20                                                            GAO/RCED-98-21 Aviation Safety
                         Chapter 1
                         Background




                         Under Federal Aviation Regulations, air carriers must ensure that repair
Who Else Has             stations are conducting work that conforms with the air carriers’ manuals
Oversight                and the applicable FAA regulations.8 As part of meeting this requirement,
Responsibility for       air carriers may use one or both of the following means:

Repair Stations?     •   They may conduct their own audits of repair stations—generally every 2
                         years—to ensure that the facilities have the capability to perform the work
                         in accordance with the air carriers’ maintenance policies, procedures, and
                         requirements.
                     •   They may rely on audits conducted by the Coordinating Agency for
                         Supplier Evaluation, an international industry organization of major
                         airlines and aerospace and marine contractors. These audits are
                         conducted—again, generally every 2 years—by staff from member airlines
                         who use a standardized approach that includes Federal Aviation
                         Regulation requirements. Because many airlines use the same repair
                         stations, these audits eliminate the expense of redundant evaluations of
                         repair stations.

                         Repair stations, both foreign and domestic, are also potentially subject to
                         review by the regulatory agencies of other countries. Many of the national
                         aviation authorities in countries where repair stations are located have
                         developed their own extensive inspection, surveillance, evaluation, and
                         certification programs for repair facilities. Like FAA, many of these
                         agencies review repair stations in other countries as well (including the
                         United States). Twenty-seven European nations have banded together to
                         coordinate their efforts through an organization called the Joint Aviation
                         Authorities (JAA), but many nations such as China conduct reviews on
                         their own.9 Like FAA, these other regulators have set up their programs to
                         help ensure compliance with their own national standards.

                         Figure 1.3 summarizes the relationship of the various parties involved in
                         the oversight of repair stations.




                         8
                          See subpart L of part 121 and subpart J of part 135.
                         9
                          Under the provisions of the Bilateral Aviation Safety Agreement, FAA is negotiating maintenance
                         implementation procedures with European countries that are members of JAA to provide reciprocal
                         acceptance of surveillance information on FAA-certified repair stations in Europe, and JAA-approved
                         maintenance organizations in the United States. We did not address these negotiations in our review
                         because discussions are still largely in the formative stage and little or no near-term effect is
                         anticipated.



                         Page 21                                                          GAO/RCED-98-21 Aviation Safety
                                            Chapter 1
                                            Background




Figure 1.3: Overview of the Oversight Given Repair Stations




                    Federal Aviation                             Air Carriers                 Other Countries
                     Administration                                                              (if applicable)




      Certification                  Surveillance                     Audits                       National
                                                                                                 requirements

 After receiving repair         All repair stations are       Air carriers must ensure        U.S. repair stations may
 station's application, FAA     inspected by FAA at least     that repair stations can        receive oversight from
 reviews equipment,             annually                      conform with the air            other countries if parts
 procedures, and personnel                                    carriers' policies,             or products they repair
 to ensure regulatory                                         procedures, and                 are operated in that
 requirements are met           FAA inspectors
                                overseeing air carriers       requirements                    country
                                that contract with the
 FAA issues "operations         repair station may also       Audits can be conducted         Foreign repair stations
 specification" listing tasks   review repair station         by individual air carriers or   may be reviewed by
 and aircraft for which         operations                    by an industry association      their own country's
 repair station is approved                                                                   regulatory agency
                                When FAA inspectors
 Foreign repair stations        find deficiencies, repair
 must be recertified at least   stations are responsible
 every 2 years; domestic        for bringing operations
 repair stations face no        into compliance
 recertification requirement




                                            Concern has arisen about FAA’s oversight of repair stations for three
Why Has FAA’s                               reasons: Air carriers are relying on repair stations much more than in the
Oversight of Repair                         past. Several recent accidents have involved aircraft maintained by repair
Stations Been a                             stations. And FAA’s oversight of repair stations is comparatively limited.

Matter of Concern?                          Steady growth in air carriers’ use of repair stations is one development
                                            that has focused additional attention on how FAA is carrying out its
                                            responsibility to oversee repair stations. Reliance on repair stations among
                                            air carriers has grown from an estimated 37 percent of total maintenance
                                            in 1990 to an estimated 46 percent in 1996.




                                            Page 22                                            GAO/RCED-98-21 Aviation Safety
Chapter 1
Background




Reliance on repair stations has been particularly heavy among newer
carriers such as ValuJet, Western Pacific, Reno Air, and Frontier Airlines.
According to FAA officials with whom we spoke, newer carriers use repair
stations extensively because they do not have enough repair work to make
performing it themselves economical or because they want to ensure that
they get an experienced cadre of mechanics with sound practices and
procedures. For example, Reno Air uses AAR Oklahoma, Inc., to perform
heavy airframe maintenance and major alterations of its MD-80s and
MD-90s. Operating only 30 of these aircraft does not warrant Reno Air’s
investing in the in-house repair capabilities for this type of maintenance.
And even though established air carriers tend to use repair stations less
extensively than smaller, newer air carriers, the amount of maintenance
they conduct is so great that if only a small percentage of their
maintenance is performed at repair stations, it still represents a substantial
amount of work. For example, a United Airlines official estimated that
while the company contracts out only about 7 percent of its maintenance
budget to repair stations, this amounted to about $126 million worth of
work in 1996.

A second, and significant, reason for concern about FAA’s oversight of
repair stations stemmed from domestic aviation accidents in 1995 and
1996. Table 1.3 describes four aviation accidents for which the National
Transportation Safety Board found contributing factors that involved
inadequate inspection or maintenance or improper handling of hazardous
cargo by repair stations.




Page 23                                           GAO/RCED-98-21 Aviation Safety
                                         Chapter 1
                                         Background




Table 1.3: Recent Accidents Involving
Aircraft Maintained by Contract Repair                                                                     Link to repair station
Stations                                 Airline, aircraft, and date       Nature of accident              activity
                                         ValuJet DC-9, June 8, 1995        During a takeoff at Atlanta,    The National Transportation
                                                                           an uncontained engine           Safety Board (the Board)
                                                                           failure caused a fire that      determined that the
                                                                           destroyed the aircraft’s        probable cause of the fire
                                                                           fuselage. Several people        was the failure of repair
                                                                           were injured during the         station personnel to
                                                                           evacuation.                     conduct a proper
                                                                                                           inspection of the engine
                                                                                                           assembly.
                                         Atlantic Southeast Airlines       The aircraft lost a propeller   The Board determined that
                                         Embraer-120, August 21,           while climbing above            the probable cause of the
                                         1995                              18,100 feet. It crashed         loss of the propeller was a
                                                                           during an emergency             fatigue fracture from
                                                                           landing, killing 8 and          corrosion pits that were not
                                                                           injuring 21 others on board.    discovered or properly
                                                                                                           repaired by the
                                                                                                           manufacturer’s repair
                                                                                                           station.
                                         ValuJet DC-9, May 11, 1996        A fire broke out in a cargo     In an abstract of the final
                                                                           compartment of the aircraft,    report, the Board said a
                                                                           which crashed in the Florida    probable cause of the
                                                                           Everglades, killing all 110     accident was the failure of a
                                                                           on board.                       repair station to properly
                                                                                                           prepare, package, identify,
                                                                                                           and track unexpended
                                                                                                           oxygen generators, a
                                                                                                           hazardous material.
                                         Tower Air                         The aircraft sustained minor    The Board found problems
                                         Boeing 747-136, June 17,          damage when an engine           associated with a repair
                                         1996                              accessory gearbox caught        station’s overhaul and
                                                                           fire during the descent for     assembly of a drive unit.
                                                                           landing. No one was injured.
                                         Source: National Transportation Safety Board.



                                         A third reason for concern is the relatively limited amount of oversight
                                         that FAA gives repair stations compared with the oversight it gives air
                                         carriers. FAA is responsible for ensuring that repair stations comply with
                                         regulations, and the agency’s annual guidance for surveillance sets forth
                                         minimum inspection requirements for all certificate holders. For fiscal
                                         year 1997, each repair station was to have a minimum of one facility
                                         inspection, while each air carrier was required to have many more
                                         inspections. An air carrier such as Alaska Airlines, for instance, had to
                                         have a minimum of 62 inspections.




                                         Page 24                                                      GAO/RCED-98-21 Aviation Safety
                         Chapter 1
                         Background




                         The Ranking Minority Member of the Aviation Subcommittee of the Senate
Objectives, Scope,       Committee on Commerce, Science, and Transportation, and Senator Ron
and Methodology          Wyden asked us to examine FAA’s oversight of repair stations. Specifically,
                         we were asked to address the following questions:

                     •   What is the nature and scope of the oversight of repair stations conducted
                         by FAA personnel?
                     •   How well does FAA follow up on inspections to ensure that deficiencies in
                         repair stations’ operations are corrected once they have been identified?
                     •   What steps has FAA taken to improve the oversight of repair stations?

                         Our analysis was based in part on agencywide data FAA provided and in
                         part on a detailed review of a cross-section of airlines, repair stations, FAA
                         offices, and FAA inspectors. In general, we did the following:

                     •   We reviewed the use of repair stations by eight air carriers, chosen
                         because, like the industry as a whole, they varied greatly in the extent to
                         which they used repair stations.10 The number of aircraft operated by
                         these carriers ranged from 3 to 659.
                     •   We reviewed operations at 10 repair stations, chosen because they
                         represented a wide variety of locations (both domestic and foreign), types
                         of repair activities, and size of operations.11 The repair stations ranged
                         from a wheel and brake specialist with about 20 employees to facilities
                         conducting many types of maintenance and employing more than 3,000
                         workers.
                     •   We examined oversight activities and discussed the oversight of repair
                         stations at FAA headquarters, 4 of FAA’s 9 regional offices, 8 of FAA’s 86
                         Flight Standards district offices, and 6 of FAA’s 7 offices with international
                         responsibilities.12 Our work at these offices included reviewing inspection
                         files for nearly 500 repair stations.
                     •   We conducted a survey of 275 FAA principal inspectors on their views
                         about ways to improve the oversight of repair stations. Our survey had a


                         10
                          The air carriers were Alaska, American, America West, Delta, Sierra Pacific, Simmons, Southwest,
                         and United.
                         11
                          The repair stations were Advanced Material Technologies, Inc., Tempe, Arizona; AeroControls, Inc.,
                         Auburn, Washington; B.F. Goodrich Component Services Division, Tempe, Arizona; Chromalloy Los
                         Angeles, Gardena, California; Evergreen Air Center, Marana, Arizona; Greenwich Air Services, Inc.,
                         Miami, Florida; Lufthansa Technik AG, Frankfurt, Germany; Precision Avionics and Instruments, Inc.,
                         Atlanta, Georgia; Sabena Technic, Brussels, Belgium; and Tramco, Inc., Everett, Washington.
                         12
                          Regional offices reviewed were the Northwest Mountain, Southern, Southwest, and Western-Pacific;
                         Flight Standards district offices were Atlanta, Dallas, Dallas/Fort Worth, Fort Worth, Los Angeles,
                         Miami, Scottsdale, and Seattle; offices with international responsibility were Brussels, Dallas/Fort
                         Worth, Frankfurt, London, Miami, and San Francisco.



                         Page 25                                                          GAO/RCED-98-21 Aviation Safety
Chapter 1
Background




response rate of 90 percent, and its results can be generalized to all FAA
inspectors with responsibility for repair stations.

We conducted our review from August 1996 through October 1997 in
accordance with generally accepted government auditing standards. In
September 1997, we provided the Department of Transportation and FAA
with a draft of this report for their review and comment. We met with FAA
officials, including the Deputy Associate Administrator for Regulation and
Certification (acting on behalf of the FAA Administrator) to obtain FAA’s
comments. Those comments and our responses are included in the
executive summary and chapters 2, 3, and 4. For a more detailed
discussion of our scope and methodology, see appendix I.




Page 26                                           GAO/RCED-98-21 Aviation Safety
Chapter 2

Current Inspection Approach Limits FAA’s
Ability to Ensure Compliance

                      Although FAA is meeting its oversight goal to inspect every domestic and
                      foreign repair station at least once a year, the use of one-person
                      inspections at large, complex facilities restricts the agency’s ability to
                      identify deficiencies and ensure compliance with regulations. We reviewed
                      19 instances in which FAA conducted a special team inspection of a facility
                      that had received a one-person inspection within the previous year. These
                      special team inspections identified far more deficiencies than inspections
                      done by individual inspectors. Team inspections tend to be more
                      comprehensive and focused, and team members are more organizationally
                      independent of the repair station and have a more standardized approach
                      to ensuring that all aspects of compliance with rules and regulations are
                      checked. Many inspectors acknowledged the advantages of using a team
                      rather than an individual inspector to review such facilities, stating that
                      the pressure of other duties keeps them from conducting inspections on
                      their own that thoroughly identify deficiencies and, thus, ensure
                      compliance. Some FAA offices we visited have developed ways to conduct
                      inspections using teams rather than individual inspectors and to do so
                      without adversely affecting other demands on inspectors’ time. Their
                      actions hold promise as a “best practice” that FAA could examine for
                      broader application.


                      Surveillance is one of the most important functions FAA inspectors perform
Number of             to ensure safety and regulatory compliance in the aviation system. Each
Inspections           year, FAA identifies specific surveillance activities that must be conducted
Conducted Meets the   during the year, including an inspection of each repair station. This
                      inspection is conducted by the FAA Flight Standards district office that
Required Minimum      maintains a repair station’s certificate. According to FAA’s guidance, the
                      inspection is to cover all aspects of a repair station’s operations, including
                      the currency of technical data, facilities, calibration of special tooling and
                      equipment, and inspection procedures. The inspection is also to ensure
                      that the repair station is performing only work that it has approval to do.
                      While FAA’s guidance does not prescribe precisely how each inspection
                      must be conducted, it provides some direction on how to perform a repair
                      station inspection. It does not require inspectors to follow checklists or
                      other prescribed approaches to conduct the inspection.

                      FAA’s guidance requires, at minimum, one maintenance or avionics facility
                      inspection of each repair station per year. Those repair stations with both
                      maintenance and avionics ratings receive at least two facility inspections,
                      one examining maintenance capabilities and the other, avionics functions.




                      Page 27                                           GAO/RCED-98-21 Aviation Safety
                            Chapter 2
                            Current Inspection Approach Limits FAA’s
                            Ability to Ensure Compliance




                            The standard of one inspection per year has not changed in recent years as
                            air carriers have increased their reliance on repair stations.1

                            All 2,800 repair stations in the United States and around the world doing
                            work on aircraft flown by FAA-certified air carriers received the
                            inspections FAA’s guidance required in fiscal year 1996, according to
                            officials at FAA headquarters. As partial verification of the FAA officials’
                            statement, we reviewed FAA’s Program Tracking and Reporting Subsystem
                            (PTRS) data from the 13 FAA offices we visited to determine if the offices
                            had made the facility inspections of the repair stations assigned to them.
                            In all, these 13 offices were responsible for more than 950 inspections at
                            over 750 repair stations working for FAA-certified air carriers. Our analysis
                            of the data confirmed that these minimum inspection requirements were
                            met.


Type of Inspection Varies   How repair stations are inspected varies based on decisions made by both
                            FAA managers and the inspectors themselves. The approach also varies
                            depending on whether the repair station is in the United States or abroad.
                            Moreover, review of some repair stations’ activities is not limited to the
                            annual facility inspection. Each year, FAA selects a few facilities for
                            special, in-depth inspections, which FAA officials stated complement the
                            surveillance conducted by individual inspectors. In the past 4 years, an
                            average of only 23 of these inspections have been conducted at repair
                            stations per year (less than 1 percent of the repair stations performing
                            work for air carriers).

                            In practice, most facility inspections of domestic repair stations are
                            conducted by the individual inspectors who have been assigned the
                            oversight responsibility for the repair stations. This approach is FAA’s front
                            line of surveillance of repair stations. The inspectors assigned
                            responsibility for repair stations are also assigned oversight of other
                            aviation activities such as air taxis, agricultural operators, helicopter
                            operators, and training schools for pilots and mechanics. In addition, the
                            inspectors have other duties such as certifying new operators and
                            investigating accidents and incidents.

                            In performing routine surveillance, an inspector may make repeated visits
                            to a single facility to complete the inspection because there is too much to
                            accomplish in just one visit. This is particularly true at larger, more


                            1
                             Under fiscal year 1997 guidelines, FAA required inspectors conducting facility inspections to also
                            inspect the repair stations’ procedures for the detection of suspected unapproved parts.



                            Page 28                                                           GAO/RCED-98-21 Aviation Safety
Chapter 2
Current Inspection Approach Limits FAA’s
Ability to Ensure Compliance




complex repair stations. Inspectors responsible for such repair stations
told us that they often make multiple visits to complete a single inspection.
FAA’s guidance to inspectors also recognizes that because the size of repair
stations can vary from a one-person operation to a large overhaul facility,
the size and complexity of the facility may warrant the inspection being
conducted by a team, rather than by an individual inspector. Some FAA
offices do, in fact, assign teams to inspect some facilities.

Like domestic repair stations, foreign repair stations are inspected every
year. Unlike domestic repair stations, however, foreign repair stations
must renew their certification with FAA at least every 2 years. The renewal
inspection assesses whether the foreign repair station continues to meet
Federal Aviation Regulations and fulfills FAA’s requirement for an annual
facility inspection. The renewal inspection and the facility inspection
cover the same aspects of repair station operations, according to FAA
officials and inspectors with both domestic and foreign oversight
experience. Like the facility inspection, the renewal inspection can be
performed by an individual or by a team of inspectors. In the six offices we
visited with responsibility for the oversight of foreign repair stations, both
types of inspections were generally conducted by teams, particularly at
larger repair stations.

Each year, FAA does special, in-depth inspections at a small portion of the
repair stations in the United States and abroad through its National
Aviation Safety Inspection Program (NASIP) or its Regional Aviation
Inspection Program (RASIP). FAA determines which facilities should receive
additional oversight through these comprehensive reviews, selecting them
on the basis of submissions from district and regional offices. In general,
inspectors recommend, through their offices, facilities for special
inspections based on inspection results or other reasons such as the size
and complexity of operations. Although FAA’s emphasis has been on
in-depth inspections of air carriers, repair stations have been part of the
special inspection effort. In fiscal years 1993 through 1996, FAA conducted
428 special, in-depth inspections, 92 (or 21 percent) of which were of
repair stations. Unlike the facility or renewal inspections, special
inspections are performed by teams of inspectors that are independent of
the district offices that have oversight responsibility for the carriers or
facilities being inspected.




Page 29                                           GAO/RCED-98-21 Aviation Safety
                        Chapter 2
                        Current Inspection Approach Limits FAA’s
                        Ability to Ensure Compliance




                        Individual inspectors generally identify far fewer deficiencies than teams
Team Inspections        do. Although most repair stations are not inspected by both individuals
Provide More Detailed   and teams, at the FAA offices we reviewed, 16 repair stations routinely
Review Than Those       inspected by individuals were also inspected by one or more special teams
                        during fiscal years 1993 through 1996. These teams found a total of 347
Conducted by            deficiencies, of which only 15 (or 4 percent) had been identified by the
Individual Inspectors   individual inspectors in the 12 to 18 months prior to the special facility
                        inspections.

                        Because many of the deficiencies relate to work on specific aircraft or
                        components, and because aircraft or components at a repair station vary
                        from day to day, some variation in inspection findings is to be expected.
                        However, a close look at the results suggests that individual inspectors,
                        even when they make multiple visits to repair stations, may not identify
                        many of the deficiencies that teams find. The special inspections we
                        reviewed turned up many systemic deficiencies, such as problems with
                        training or quality assurance, that appeared to be long-standing and that
                        therefore could have been detected in earlier inspections. For example, a
                        team conducting a special inspection found that a repair station’s manual
                        contained procedures for aircraft fuel servicing and fuel tank maintenance
                        that may have been counter to the policies of the air carriers for which the
                        work was done. The individual inspector, who had visited this repair
                        station many times in the previous 18 months had not reported this
                        problem.

                        Often, the deficiencies identified in the special inspections but not in the
                        regular inspections were significant. The findings of special inspections
                        are categorized as (1) violations of Federal Aviation Regulations,
                        (2) violations of the repair stations’ FAA-approved repair station manuals,
                        or (3) lack of systems to ensure continuing compliance. Of the deficiencies
                        reported in the 19 special inspections on 16 repair stations we reviewed,
                        one-third involved violations of FAA regulations (see fig. 2.1).2 For example,
                        an inspection team found that a repair station was not segregating new
                        and serviceable parts from those parts that were not serviceable. In
                        another case, a repair station on three occasions approved an aircraft for
                        return to service following a major repair that, according to the inspector’s
                        report, was not completed “based on FAA-approved technical data.”




                        2
                        FAA does not have comparable data for deficiencies identified during regular inspections.
                        Deficiencies are categorized only for special inspections.



                        Page 30                                                          GAO/RCED-98-21 Aviation Safety
                                        Chapter 2
                                        Current Inspection Approach Limits FAA’s
                                        Ability to Ensure Compliance




Figure 2.1: Types of Deficiencies FAA
Teams Identified During Special,
In-Depth Inspections                                                                     Violations of FAA regulations



                                                         36.6%




                                                                       37.5%
                                                                                        Violations of FAA-approved manuals
                                                   25.9%




                                                                                        Lack of systems to ensure continued
                                                                                        compliance


                                        Note: Percentages are based on a total of 317 reported deficiencies. Although 347 deficiencies
                                        were reported, 30 were not categorized.

                                        Source: FAA’s data.




                                        Violations of a repair station’s approved manual also accounted for about
                                        one-third of the deficiencies. For example, one team found a repair station
                                        did not inspect subcontracted work in accordance with its manual. A floor
                                        mechanic was performing these inspections, rather than the quality
                                        control inspector.

                                        We contacted 13 inspectors responsible for the repair stations covered by
                                        the 19 special inspection reports to obtain their views on why the special
                                        inspections found so many more deficiencies, including ones that
                                        appeared to be long-standing. They said the pressure of other duties kept
                                        their individual reviews from being more comprehensive. For example,
                                        one inspector was responsible for 7 other repair stations, 11 air taxi
                                        operators, 3 helicopter and agricultural operators, 11 executive aircraft,
                                        and more than 30 airmen. In addition, while they were at the repair
                                        stations, inspectors had to deal with employees’ questions or concerns
                                        about matters unrelated to the inspections. All 13 inspectors said that for
                                        reasons such as these, an individual inspector has a greater chance of not
                                        identifying deficiencies, even after repeated visits.




                                        Page 31                                                        GAO/RCED-98-21 Aviation Safety
                                          Chapter 2
                                          Current Inspection Approach Limits FAA’s
                                          Ability to Ensure Compliance




Attributes of Quality                     The quality of repair station inspections is important because surveillance
Inspections More                          is one of FAA’s primary means for ensuring that repair stations continually
Prevalent in Team                         meet Federal Aviation Regulations. FAA’s guidance to inspectors states that
                                          if surveillance is to meet its intended purpose, quality inspections are
Inspections Than in                       essential. We developed four characteristics of a quality inspection based
Individual Inspections                    on our initial discussions with FAA officials and inspectors, staff from the
                                          U.S. Department of Transportation’s Office of the Inspector General who
                                          were involved with repair station work, Department of Defense officials
                                          responsible for audits of carriers with contracts for transporting military
                                          personnel, and airline quality assurance officials. These four
                                          characteristics, explained in table 2.1, are independence,
                                          comprehensiveness, focus, and standardization. In subsequent
                                          discussions, FAA officials and inspectors agreed that a quality inspection
                                          should have these four characteristics.

Table 2.1: Characteristics of a Quality
Inspection                                Characteristic                             Definition/explanation
                                          Independence                               Inspectors need to be free of undue
                                                                                     interest or complacency regarding the
                                                                                     repair station’s operation. Inspectors who
                                                                                     must deal with a repair station on an
                                                                                     ongoing basis can lose their objectivity
                                                                                     because they may feel they already know
                                                                                     that the repair station’s operations are in
                                                                                     good order.
                                          Comprehensiveness                          Each inspection needs sufficient time to
                                                                                     cover all of the elements that are
                                                                                     supposed to be covered.
                                          Focus                                      The inspection needs to be performed
                                                                                     without distraction. This means minimizing
                                                                                     competing demands, such as dealing with
                                                                                     repair station employees’ questions or
                                                                                     concerns about other matters. Too many
                                                                                     distractions can prevent inspectors from
                                                                                     conducting a thorough inspection.
                                          Standardization                            Even though repair stations are different,
                                                                                     each one needs to be reviewed for all of
                                                                                     the applicable requirements. Use of an
                                                                                     agreed-upon checklist or job aid helps to
                                                                                     ensure that all similar inspections are
                                                                                     conducted in a similar fashion.

                                          The size and complexity of many large repair stations are such that an
                                          individual inspector may have difficulty maintaining these quality
                                          characteristics. In examining FAA’s facility inspection records, and in
                                          discussions with inspectors, we found these four characteristics were




                                          Page 32                                               GAO/RCED-98-21 Aviation Safety
                    Chapter 2
                    Current Inspection Approach Limits FAA’s
                    Ability to Ensure Compliance




                    more prevalent in facility inspections conducted by teams than in those
                    conducted by individual inspectors.

Independence        Inspectors assigned to teams have no ongoing responsibility for the repair
                    station and have no relationship with its operator. By contrast, the
                    individual inspector who conducts a facility inspection is usually the one
                    assigned to manage the repair station’s certificate for an extended period
                    of time. During this period, the primary contact the repair station has with
                    the FAA is through this inspector. Inspectors we spoke with during field
                    visits consistently stated that an inspector with a “fresh set of eyes” often
                    identifies deficiencies that the principal inspector misses.

Comprehensiveness   Teams cover all subject areas during the course of their inspections,
                    whereas individual inspectors’ other duties may limit the time they can
                    spend and the extent of work they can do during their visits to repair
                    stations. Many of the inspectors responding to our survey indicated that
                    their ability to conduct a quality inspection was affected by factors related
                    to comprehensiveness (see fig. 2.2). For example, 75 percent of the
                    respondents said having too many inspection duties affected their ability
                    to conduct comprehensive inspections to some degree, with 43 percent
                    saying it was a major reason for the problem. (For a more detailed
                    breakdown of the survey results for these and other survey topics, see app.
                    III.) In addition, inspectors we spoke with at field offices said that it was
                    very difficult to cover everything at a large or complex repair station. For
                    example, one inspector said the week he spends at a large engine repair
                    station is not enough time to complete a facility inspection. The size of the
                    repair station and the complexity of the work being done, he said, makes it
                    difficult to ensure that he is making a comprehensive inspection.




                    Page 33                                           GAO/RCED-98-21 Aviation Safety
                                        Chapter 2
                                        Current Inspection Approach Limits FAA’s
                                        Ability to Ensure Compliance




Figure 2.2: FAA Inspectors’ Responses
on Barriers to Comprehensive
                                          50      Percentage of respondents
Inspections


                                          40



                                          30



                                          20



                                          10



                                           0
                                                  Too many             Not enough          Difficult to inspect
                                                  inspection           time to visit       larger facilities
                                                  duties to be         repair stations     because of
                                                  comprehensive        as often as         size/complexity
                                                                       desired

                                                      Major reason hindering success in ensuring compliance
                                                      Minor reason
                                                      Not a reason



                                        Note: Percentages may not add to 100 because a few respondents said they had no basis to
                                        make a judgment.


Focus                                   In a team inspection, completing a portion of an inspection is the only duty
                                        of each team member. An inspector conducting an inspection alone faces
                                        work demands from other locations as well as divided responsibilities at
                                        the repair station being inspected. Inspectors’ responses to our survey also
                                        reflected concerns about their ability to focus sufficiently during
                                        inspections (see fig. 2.3). For example, 80 percent of the inspectors
                                        responding indicated that spending time on other duties had an effect on
                                        the quality of the inspections they performed. For example, during our
                                        interviews, inspectors said they needed to spend time during inspection
                                        visits answering questions or clarifying regulations for repair station
                                        employees. They said such duties were part of their job, but some noted
                                        that these conflicting demands can interfere with their ability to focus on
                                        the inspection they are trying to conduct.




                                        Page 34                                                        GAO/RCED-98-21 Aviation Safety
                                        Chapter 2
                                        Current Inspection Approach Limits FAA’s
                                        Ability to Ensure Compliance




Figure 2.3: FAA Inspectors’ Responses
on Barriers to Focusing During
Inspections                                 60      Percentage of respondents




                                            40




                                            20




                                             0

                                                     Need to spend too            Need to spend time
                                                     much time on                 providing education
                                                     duties other than            and guidance to
                                                     inspections                  operators


                                                        Major reason hindering success in ensuring compliance
                                                        Minor reason
                                                        Not a reason



                                        Note: Percentages may not add to 100 because a few respondents said they had no basis to
                                        make a judgment.


Standardization                         Inspection results can be more useful to inspectors and FAA if there is
                                        assurance that all areas have been adequately covered. If all areas are
                                        covered, inspectors have greater assurance that the repair station
                                        complies with regulations. Checklists or other similar job aids are one way
                                        to provide this assurance and to do so in a structured, consistent manner.
                                        A checklist or similar job aid for repair station inspections would include
                                        all areas that inspectors must review as part of the inspection as well as
                                        how the regulations governing repair station activities relate to these
                                        areas. At present, however, FAA does not require the use of a checklist
                                        during a repair station inspection.

                                        We found evidence from a number of sources that the use of an effective
                                        checklist plays an important role in a thorough inspection. Officials from
                                        FAA, industry, and the Department of Defense (which reviews air carriers
                                        before awarding defense contracts) told us that they would question the




                                        Page 35                                                        GAO/RCED-98-21 Aviation Safety
Chapter 2
Current Inspection Approach Limits FAA’s
Ability to Ensure Compliance




comprehensiveness of any facility inspection of a repair station that was
not done using a job aid or checklist. Air carriers and the Department of
Defense reported that their own inspectors use such aids to guide their
work. They said the scope of the inspection of many repair stations is large
enough that it is not difficult to overlook a portion of what must be
covered.

While FAA does not require the use of a checklist or job aid for routine
surveillance, teams, whether conducting routine or special inspections, are
more likely than individual inspectors to use checklists or other job aids
that help ensure that all areas are covered, based on our observations. We
found that during team inspections, team members use the same
structured approach, typically in the form of an inspection job aid or
checklist, such as the NASIP checklist. For example, one overseas office
that conducts its inspections with in-house teams has a job aid covering
each portion of the inspection. By contrast, we found that the approaches
used by inspectors conducting their own inspections varied greatly, and
individual inspectors were less likely to use checklists or other job aids to
ensure that all areas had been covered. For example, while one inspector
showed us a detailed checklist he developed combining guidance from the
Airworthiness Inspector’s Handbook with the regulations applicable to
repair stations, others said they do not use any job aid and work instead
from memory when inspecting repair stations.

We also found that when individual inspectors use a checklist, they tend to
use one that is not detailed enough to ensure that compliance with
regulations is checked. FAA’s standard, most commonly used job aid, which
is available to inspectors through the Flight Standards Automated System,
is not directly tied to the standards that repair stations must meet.
Although it lists items to review, it does not provide references to the
regulations governing repair stations. By contrast, the checklist used
during NASIP inspections provides this link, as do the checklists and job
aids the aviation industry uses to evaluate repair stations. Officials at the
regional and office levels have indicated that knowing how an inspection
finding relates to the regulations is important for pursuing enforcement
action when a violation is identified. We found one office that requires all
inspectors to use a job aid tied to the regulations. The office manager said
that by having all inspectors use a standardized approach, he has greater
assurance that effective and comprehensive inspections are being
performed and that repair stations are in compliance with regulations.




Page 36                                          GAO/RCED-98-21 Aviation Safety
                                       Chapter 2
                                       Current Inspection Approach Limits FAA’s
                                       Ability to Ensure Compliance




                                       We asked FAA headquarters officials what they thought of encouraging
                                       inspectors to make greater use of checklists and other job aids. They said
                                       that guidance and job task lists provided to inspectors encourage the
                                       development of good work processes by each inspector without removing
                                       the flexibility required for them to evaluate a repair station’s compliance.
                                       However, the greater use of checklists or other job aids to help ensure that
                                       comprehensive inspections are being performed could be instituted in a
                                       way that does not diminish the inspectors’ flexibility. At a minimum, these
                                       types of tools would serve to remind the inspectors of the elements of the
                                       inspection that are the most critical to safety.


Inspectors Describe                    Most inspectors responding to our survey responded favorably when
Overall Repair Station                 asked for their general impressions about repair stations’ overall
Compliance Favorably, but              compliance with regulations. Sixteen percent of the inspectors put
                                       compliance at that top, or “excellent” level, 68 percent rated compliance as
See a Need for                         “good,” and 12.5 percent rated it as “fair.” (See fig. 2.4.)
Improvement in Many
Areas
Figure 2.4: FAA Inspectors’ Views on
the Current Overall Compliance of
Repair Stations                                                                                     Excellent




                                                               68%                                  Good
                                                 16%


                                             12.5%                                                  Fair



                                                                                                    Poor/Very Poor
                                                                            2%



                                                                                                    Uncertain
                                                   1.5%




                                       Page 37                                         GAO/RCED-98-21 Aviation Safety
Chapter 2
Current Inspection Approach Limits FAA’s
Ability to Ensure Compliance




Inspectors acknowledged, however, that there was room for improvement.
We asked inspectors about eight areas of compliance, such as the repair
stations’ use of up-to-date manuals from manufacturers and air carriers. In
each of the eight areas, more than half of the inspectors surveyed saw the
need for at least some improvement. Specifically, in none of the eight
categories was the percentage higher than 38 percent for respondents who
thought little or no improvement was needed. By contrast, the percentage
of inspectors who saw a need for some or moderate improvement ranged
from 39 to 52 percent, and those who saw the need for great or very great
improvement ranged from 18.5 to 33 percent. (See fig. 2.5.)




Page 38                                         GAO/RCED-98-21 Aviation Safety
                                                 Chapter 2
                                                 Current Inspection Approach Limits FAA’s
                                                 Ability to Ensure Compliance




Figure 2.5: FAA Inspectors’ Responses on Amount of Improvement Needed for Repair Station Compliance

60      Percentage of respondents




50




40




30




20




10




 0
      Repair            Repair           Actual            Altered/          Require-          Air carrier's    Repair          Supervisors at
      station's         station is       procedures of     repaired          ments met for     current          station's       repair stations
      inspection        using            repair station    parts equal in    recording         manuals          inspection      receive training
      procedures        manufac-         give              condition to      mainte-           and              program         from air
      manual gives      turers'          satisfactory      original parts    nance             procedures       complies with   carriers
      satisfactory      current          control                                               are being        air carrier's
      quality control   manuals                                                                used             maintenance
                                                                                                                program


           Great/very great improvement needed
           Some/moderate improvement needed
           Little/no improvement needed


                                                 Note: Percentages may not add to 100 because a few respondents said they had no basis to
                                                 make a judgment.



                                                 How could FAA offices, already facing a diverse and extensive mix of
Different Approaches                             responsibilities, do a better job of inspecting repair stations without
to Repair Station                                adversely affecting other operations? To determine if there were workable
Surveillance Show                                answers to that question, we turned to the field offices themselves. FAA
                                                 field offices are given some flexibility by FAA headquarters in deciding how
Promise

                                                 Page 39                                                       GAO/RCED-98-21 Aviation Safety
                                   Chapter 2
                                   Current Inspection Approach Limits FAA’s
                                   Ability to Ensure Compliance




                                   to accomplish their surveillance programs. We looked to see if some of
                                   these offices had developed alternative approaches that might hold
                                   promise for other locations. We identified several offices that had adopted
                                   approaches that might prove useful on a broader scale. In general, their
                                   practices fell into two main categories: (1) placing greater emphasis on the
                                   oversight of repair stations and (2) finding ways to shift local staff
                                   resources so that they could conduct more repair station inspections with
                                   teams rather than with individual inspectors.3 FAA headquarters is also
                                   examining a revised approach to surveillance that could help improve the
                                   inspection process.


Placing Higher Priority on         Some of the FAA field offices we reviewed, departing from the standard
Inspections of Repair              approach to assigning inspectors’ responsibilities, have developed new
Stations                           approaches on their own to place greater emphasis on repair stations.
                                   Typically, inspectors working in a field office are separated into two
                                   disciplines: general aviation and air carrier. General aviation inspectors
                                   are assigned to specific repair stations and also inspect operators covered
                                   by part 135 of the Federal Aviation Regulations—that is, air taxi operators.
                                   In addition, they inspect other aspects of the industry such as agricultural
                                   aircraft operators, technical schools for pilots and mechanics, and
                                   helicopter operators. Air carrier inspectors are responsible for operations
                                   covered by part 121 of the Federal Aviation Regulations—that is, for
                                   domestic air carriers. They are assigned specific carriers to inspect, and
                                   may, in that context, inspect the carriers’ in-house repair stations or those
                                   the carriers use, assessing whether repairs being made conform with the
                                   carriers’ FAA-approved maintenance manuals. Two FAA offices we visited
                                   have found this structure does not recognize the importance of overseeing
                                   repair stations and are pursuing other approaches that place a higher
                                   priority on it. Officials at FAA headquarters said they supported these
                                   efforts but will first evaluate the results and then, if appropriate, use the
                                   approaches at other offices.

Scottsdale Flight Standards        In the Western-Pacific Region, officials have approved a new
District Office, Western-Pacific   organizational structure at the Flight Standards district office in
Region                             Scottsdale, Arizona. Under the new structure, the district office is divided
                                   into an air transport and an air commerce unit. The air transport unit
                                   oversees air carriers and large repair stations, while the air commerce unit


                                   3
                                    One criterion for our consideration of possible approaches was that the approach require no
                                   additional resources beyond what the field office believed it could commit to the oversight of repair
                                   stations at current or anticipated staffing levels. However, under staffing increases approved by the
                                   Congress, many field offices will have more inspectors. Ch. 4 discusses the staff increases in greater
                                   detail.



                                   Page 40                                                            GAO/RCED-98-21 Aviation Safety
                                    Chapter 2
                                    Current Inspection Approach Limits FAA’s
                                    Ability to Ensure Compliance




                                    oversees air taxis, technical schools for pilots and mechanics, and other
                                    operators. Within the air transport unit, one team focuses its work on the
                                    five largest repair stations in the area, all of which perform heavy airframe
                                    maintenance. According to a district office supervisor, two additional staff
                                    members with significant repair station experience have been hired in the
                                    last 3 years. The expanded staff made it possible to establish a repair
                                    station team without asking for additional resources. Large-component
                                    repair stations, such as those working on landing gear or engine parts, may
                                    be added to the team’s responsibility at a future date, an official said.

                                    According to district office staff, the office has been concerned with FAA’s
                                    lack of surveillance of larger, more complex repair stations. Inspectors
                                    stated that the reorganization will allow them to spend more time on those
                                    facilities without affecting the surveillance of smaller repair stations. In
                                    conjunction with the reorganization, office management also redistributed
                                    the repair station workload among inspectors to allow them to provide
                                    more effective surveillance of larger facilities.

Seattle Flight Standards District   The Seattle Flight Standards district office revised the position description
Office, Northwest Mountain          of several inspectors to place a greater emphasis on the oversight of repair
Region                              stations. Under these revised position descriptions, three maintenance
                                    inspectors, each with an assistant, will be responsible for the nine largest
                                    and most complex repair stations in the district. According to a district
                                    office official, FAA headquarters must approve this change because under
                                    the present set of position descriptions for inspectors, all repair stations
                                    are considered to have the same degree of complexity. In a letter to the
                                    regional office justifying the new positions, the district office manager
                                    stated that the office has not had the resources to “become proactive in
                                    the day-to-day activities of the facilities.” For example, at a repair station
                                    that works on over 400 aircraft annually with a staff of 2,000, adequate
                                    surveillance was not provided in several areas, such as compliance with
                                    customer airline procedures and regulatory requirements, according to
                                    office staff. In addition, adequate spot checks of maintenance performed
                                    by the company had not been made. The inspectors responsible for this
                                    facility were responsible for other facilities as well and did not have
                                    enough time to do a comprehensive review of the repair station.

                                    Under the new position descriptions, the principal inspectors have fewer
                                    responsibilities and so will be able to spend more time at each of the large
                                    facilities assigned to them. As in Scottsdale, the workload for the
                                    inspectors in Seattle will change dramatically. For example, one inspector
                                    will go from overseeing 16 repair stations, 8 air taxi operators, 4 executive



                                    Page 41                                           GAO/RCED-98-21 Aviation Safety
                               Chapter 2
                               Current Inspection Approach Limits FAA’s
                               Ability to Ensure Compliance




                               aircraft operators, 2 helicopter operators, 2 agricultural operators, 45
                               airmen, and a pilot school to overseeing just 7 repair stations, all of which
                               are facilities working on aircraft component parts. The remaining
                               responsibilities will be distributed among existing and projected additional
                               staff.4

European International Field   The Eastern Region has four international field offices, three in Europe
Offices, Eastern Region        and one in New York.5 Because the European offices are not faced with
                               many of the other responsibilities that domestic offices must
                               handle—such as overseeing pilot and mechanic schools, agricultural
                               aircraft, and certificate management of air carriers—the primary focus of
                               their work program is on the certification and surveillance of repair
                               stations along with limited surveillance of U.S. air carriers. (The New York
                               office handles other responsibilities such as the oversight of foreign air
                               carriers and the International Aviation Assessment Program.)
                               Consequently, inspectors at the European offices are able to spend more
                               time on the surveillance of repair stations than their U.S.-based
                               counterparts. All of the inspectors we interviewed at the European offices
                               said they spent 80 percent or more of their time on repair stations,
                               whereas inspectors at domestic offices said they spent only about
                               30 percent of their time on surveillance of all types of facilities, including
                               repair stations.


Conducting More Locally        A second and closely related development we observed was the use of
Based Team Inspections         locally based teams to conduct surveillance. The use of in-house teams in
                               these offices ranges from making them the typical surveillance approach,
                               as in the international field offices, to using them occasionally for areas in
                               which problems have been identified. Because these teams are made up of
                               local office staff, the cost is lower than for special inspections conducted
                               by NASIP or RASIP teams assembled from around the country or around the
                               region. Moreover, because local resources are used, the office can assess
                               the effect of this approach on the office’s other responsibilities. The move
                               toward team-based surveillance inspections was supported by the
                               inspectors we surveyed. Officials at FAA headquarters also said they
                               support these efforts but will assess them before asking other offices to
                               make greater use of in-house teams.




                               4
                                See ch. 4 for further discussion of FAA’s approved staff increases.
                               5
                                The three European offices are in London, Brussels, and Frankfurt.



                               Page 42                                                            GAO/RCED-98-21 Aviation Safety
                                    Chapter 2
                                    Current Inspection Approach Limits FAA’s
                                    Ability to Ensure Compliance




Scottsdale Flight Standards         Prior to the office’s restructuring, staff at the Scottsdale Flight Standards
District Office, Western-Pacific    district office routinely performed team inspections on the largest repair
Region                              stations each year. They also performed team inspections on selected
                                    smaller facilities. According to one official at the office, team inspections
                                    are a big part of the overall surveillance program because the office
                                    believes such inspections are the only way it can ensure that the repair
                                    stations are meeting all applicable regulations. Use of team inspections is
                                    expected to increase under the new office organization. Team inspections
                                    at the Scottsdale office are led by the principal inspector, the person with
                                    the most knowledge about a repair station.

Seattle Flight Standards District   The Seattle Flight Standards district office has increasingly relied on
Office, Northwest Mountain          in-depth reviews conducted by teams of inspectors as a way to strengthen
Region                              its oversight of repair stations. According to officials in the region’s Flight
                                    Standards Division, current surveillance of repair stations, as well as
                                    surveillance of other certificate holders, is not as effective as it should be.
                                    As evidence, they cite national statistics indicating that only five
                                    enforcement actions (such as a warning notice or a civil penalty) result
                                    from every 1,000 inspections FAA conducts (an enforcement rate of
                                    0.5 percent). By contrast, NASIP inspections, which are more in-depth,
                                    result in an enforcement rate of 20 percent.

                                    District office officials said that the team approach is being used so that
                                    the staff is more aware of what is happening at the facilities they oversee.
                                    Moreover, the office has found team inspections conducted to date to be
                                    very successful. For example, a recent team inspection at a large
                                    component shop repair station identified 17 deficiencies that the principal
                                    inspector said he had not identified in several prior inspections. According
                                    to the inspector, his workload and the complexity of this repair station
                                    prevented him from performing an inspection comprehensive enough to
                                    identify the kinds of deficiencies found by a team. This inspector, along
                                    with others we interviewed at the district office, agreed that team
                                    inspections are necessary for adequate surveillance in some cases,
                                    particularly for larger, more complex repair stations.

Miami Flight Standards District     The Miami Flight Standards district office has established a quality
Office, Southern Region             assurance unit that, among other activities, performs team inspections of
                                    repair stations. The teams inspect air carriers, repair stations, and other
                                    operators in response to complaints or an inspector’s request. According
                                    to a regional office official and the district office supervisor of the
                                    inspection teams, the inspections are more objective and comprehensive
                                    than the routine inspections. In addition, the inspections allow the team to



                                    Page 43                                            GAO/RCED-98-21 Aviation Safety
                                 Chapter 2
                                 Current Inspection Approach Limits FAA’s
                                 Ability to Ensure Compliance




                                 identify and correct potential problems that if left unaddressed could
                                 develop into compliance problems. From fiscal year 1993 through 1996, 32
                                 in-depth team inspections were conducted by the office, 14 of them at
                                 repair stations.

International Field Offices      At all six of the offices we visited that oversee foreign repair stations,
                                 inspections are typically conducted by teams, although smaller repair
                                 stations with very few employees or capabilities may be assigned to only
                                 one inspector. For example, the Frankfurt office specifies that annual
                                 surveillance on a repair station with more than 100 employees be
                                 performed by a team of up to five inspectors. The team approach is used
                                 because the office does not believe an individual inspector can cover an
                                 entire facility. According to the office’s manager, a team provides broader,
                                 deeper coverage and the end result is that more deficiencies are identified.

                                 As in the NASIP and RASIP special inspections, segments of the facility
                                 inspection are divided among the participating inspectors. For example,
                                 one inspector will review the landing gear and window repair shops, while
                                 another will inspect the technical library and the calibration laboratory.
                                 The principal inspector assigned to the facility acts as the team leader and
                                 prepares the findings presented to the repair station at the conclusion of
                                 the inspection. Inspectors responsible for foreign repair stations said that
                                 although they may visit some repair stations only once a year, less than
                                 when they performed domestic repair station surveillance, the surveillance
                                 of foreign repair stations is more thorough because of the team approach.

Inspectors Support Greater Use   A substantial number of the inspectors we surveyed supported the use of
of Team Inspections              more team inspections. Figure 2.6 shows survey responses on using team
                                 inspections to improve compliance. Results show that 71 percent of the
                                 inspectors responding favor using team inspections staffed from within
                                 the district office, and 50 percent favor an increase in NASIP or RASIP
                                 inspections staffed from offices nationwide or within the region.
                                 Opposition was weak both to locally based team inspections (11 percent)
                                 and to the increased use of NASIP or RASIP inspections (16 percent).




                                 Page 44                                          GAO/RCED-98-21 Aviation Safety
                                        Chapter 2
                                        Current Inspection Approach Limits FAA’s
                                        Ability to Ensure Compliance




Figure 2.6: FAA Inspectors’ Responses
on Using Teams to Improve
Compliance                               80       Percentage of respondents




                                         60




                                         40




                                         20




                                          0
                                                  Encourage team              Increase
                                                  inspections within          regional and
                                                  the individual office       national team
                                                                              inspections

                                                      In favor
                                                      Neutral
                                                       Opposed



                                        Note: Percentages may not add to 100 because a few respondents said they had no basis to
                                        make a judgment.




                                        This support was affirmed in our interviews with inspectors at the offices
                                        we visited. One inspector stated that while he worked in the airline
                                        industry, the company would never send fewer than two inspectors to a
                                        contract repair station. Still other inspectors stated that individually, they
                                        are unable to obtain reasonable assurance of compliance with regulatory
                                        requirements at larger facilities.


New Inspection Approach                 Other developments within FAA may have future implications for how
for Air Carriers May Have               repair station inspections are conducted. One recommendation from FAA’s
Applicability for Repair                90-Day Aviation Safety Review completed in September 1996 was the
                                        creation of the Certification Program Office, which would include a
Stations                                National Certification Team to assist local Flight Standards district offices
                                        in processing new air carrier certifications. In addition, the new office will



                                        Page 45                                                      GAO/RCED-98-21 Aviation Safety
Chapter 2
Current Inspection Approach Limits FAA’s
Ability to Ensure Compliance




also include a centralized safety analysis and information management
office that will assist inspectors in targeting surveillance resources and
taking necessary corrective actions to mitigate safety risks. These
approaches to improving the surveillance of air carriers can also be
applied to the surveillance of repair stations’ operations.

In a separate effort, FAA is testing a method of surveillance that emphasizes
the compliance with specific regulations rather than the completion of a
series of inspections. According to an FAA official involved with this test,
FAA is examining this new approach because it is concerned that the
current approach does not adequately link inspections to specific
regulations. The test is being done on air carriers, not repair stations, but it
could potentially be extended to repair stations, according to FAA
personnel.

FAA’s current inspection approach is based on the National Program Work
Guidelines, issued annually by FAA headquarters. These guidelines list
specific inspections that must be completed. The guidance tells inspectors
what types of inspections to perform, but it does not tell them what
regulations they are to verify compliance with.

The new approach, called “virtual recertification,” works in much the
opposite way. Instead of specifying the types of inspections to perform, it
specifies the applicable safety regulations to be checked and leaves it up
to the inspector to determine how to verify compliance. Given this
emphasis, the inspector must ensure that surveillance activities are
comprehensive enough to cover all aspects of the regulations. For
example, an inspector would verify that the repair station is meeting
requirements under 145.47(b), specifically, that the repair station ensures
that all inspection and test equipment is tested at regular intervals to
ensure correct calibration to a standard derived from the National Bureau
of Standards. This approach may prove more successful at ensuring that
important safety requirements are not omitted from surveillance.

FAA  is testing the approach in one region, where it is being applied to two
air carriers. The region would like to extend the test to one large repair
station as well, according to FAA officials. Although it is too early to judge
the effects of this test on FAA’s approach to surveillance and its potential
effect on repair stations is unknown, it may influence how FAA
headquarters and offices adjust their future oversight of repair stations to
provide the maximum benefit with the limited resources available.




Page 46                                            GAO/RCED-98-21 Aviation Safety
                  Chapter 2
                  Current Inspection Approach Limits FAA’s
                  Ability to Ensure Compliance




                  FAA appears to have the opportunity to enhance the effectiveness of its
Conclusions       repair station inspections. While just one inspector may be sufficient to
                  conduct surveillance on smaller or more specialized repair stations, this
                  approach does not appear to be nearly as effective at large, complex
                  facilities. At such facilities, team inspections have proven more effective in
                  identifying deficiencies. In addition, team inspections do a better job of
                  incorporating the four characteristics of quality inspections, in that they
                  are more independent, comprehensive, focused, and standardized than
                  inspections conducted by individual inspectors.

                  Acting on their own, several FAA offices are reconfiguring their staffs and
                  adjusting their operations to conduct more team inspections. Their
                  approaches hold promise both for making more efficient use of inspection
                  staff and for improving the quality of surveillance. FAA headquarters
                  officials support these efforts but will evaluate them before asking other
                  offices to examine such an approach. We think it is appropriate for all
                  district offices, especially those with high concentrations of repair
                  stations, to reevaluate their organization and surveillance approach to
                  determine if they can make better use of their current inspection
                  resources. For example, an office may determine that local team
                  inspections are appropriate and a good use of resources for repair stations
                  that are large, complex, or have higher rates of noncompliance.

                  An analysis of the widely varying inspection approaches also highlights the
                  importance of a standardized checklist or other effective job aid in
                  ensuring that inspections are comprehensive. Repair stations can be very
                  complex, and a checklist can help ensure that all applicable areas are
                  covered and that this coverage is consistent from facility to facility. Such
                  checklists are in widespread use by other organizations, such as air
                  carriers and the Department of Defense, that conduct similar types of
                  inspections. Where they are already in use within FAA, they appear to
                  improve both comprehensiveness and standardization. Wider use of such
                  checklists appears to be another appropriate way to increase the
                  effectiveness of FAA’s inspection effort.


                  We recommend that the Secretary of Transportation instruct the
Recommendations   Administrator of FAA to (1) expand the use of locally based teams to
                  conduct routine facility inspections, particularly for facilities that are
                  large, complex, have higher rates of noncompliance, or meet
                  predetermined risk indicators; and (2) develop and use checklists or job
                  aids for inspectors that allow a greater degree of comprehensiveness,



                  Page 47                                           GAO/RCED-98-21 Aviation Safety
                  Chapter 2
                  Current Inspection Approach Limits FAA’s
                  Ability to Ensure Compliance




                  standardization, and assurance that the repair station complies with
                  regulatory requirements.


                  FAA agreed with the recommendations. With regard to expanding the use
Agency Comments   of locally based teams to conduct inspections, FAA headquarters officials
                  said they support field office efforts currently under way but will assess
                  these efforts before asking other offices to make greater use of in-house
                  teams. While agreeing with the recommendation to provide better job aids
                  for inspectors, FAA officials did not provide specific details on how or
                  when they would implement this recommendation.

                  In addition, FAA cited several agency initiatives that it said are under way
                  to enhance its oversight of the repair station sector of the aviation
                  industry. FAA said its 90-day safety review conducted last year
                  recommended the creation of an analytical unit that could provide safety
                  trend data to inspectors. FAA said an office within the Flight Standards
                  Service was created on May 20, 1997, to provide data that will help focus
                  inspection and other resources. The review also recommended that field
                  and division managers be given flexibility to determine the skills needed in
                  a particular field office to ensure the appropriate mix of technical,
                  paratechnical, support, and clerical expertise. FAA said that this flexibility
                  will be supported through the establishment of new staffing standards—a
                  long-term project that is already under way. FAA also said that it had
                  recognized the need to evaluate the air operators safety systems, including
                  those of repair stations, and had initiated a Surveillance Improvement
                  Program. Under this program, a team of safety inspectors, technical
                  personnel, and managers, aided by Sandia National Laboratories,
                  investigated ways to improve the surveillance process. The team
                  recommended improvements in standardization and communication, as
                  well as other areas that will allow FAA to evaluate compliance more
                  effectively. Efforts are under way to implement the fundamental changes
                  to surveillance that were recommended by this team.




                  Page 48                                           GAO/RCED-98-21 Aviation Safety
Chapter 3

Incomplete Documentation of Inspections
Makes Extent of Follow-Up Difficult to
Determine
                        In many instances, we were unable to determine how well FAA was
                        following up to ensure that repair stations corrected deficiencies identified
                        during inspections. Particularly for domestic repair stations, the lack of
                        documentation made it impossible to assess how quickly or thoroughly
                        repair stations brought themselves into compliance. Documentation was
                        better for foreign repair stations, which generally appeared to be
                        correcting deficiencies quickly to qualify for renewal of their certificates.
                        Resolving problems with documentation is particularly important because
                        FAA is taking new steps to use its management information systems to
                        determine where inspection resources should be targeted. Incomplete data
                        can make such efforts less effective.


                        Much of the value of inspection activity is not in finding and listing
Information on          problems but in resolving the problems effectively, according to FAA field
Inspection Results Is   office managers and supervisors. They said that although much of the
Important but           resolution may hinge on the working relationship between the FAA
                        inspector or inspectors and the repair station’s personnel, effective
Incomplete              documentation of the actions taken is a necessary part of demonstrating
                        what problems were found, what was done to resolve them, and whether
                        all parties are in agreement that deficiencies have been corrected. FAA
                        officials acknowledged that effective documentation of inspection and
                        follow-up activity is needed.


Types of Information    FAA’s guidance is limited in specifying for inspectors what documents to
Needed for Basic        include in repair station files. FAA’s files on repair stations are the agency’s
Documentation and       official record of inspection-related activity—and therefore the backbone
                        of any system that uses management information to help spot trends,
Management Reports      identify problems, and target inspection resources, according to field
                        office managers and supervisors. The guidance points out generally that
                        the kinds of documentation of inspections and surveillance activities
                        include inspection reports and related correspondence, but the guidance
                        does not specifically require that any document be included. The closest
                        thing to a requirement is a statement in the Airworthiness Inspector’s
                        Handbook that the letter to the repair station describing all deficiencies
                        should be included in the case file.

                        After analyzing FAA’s inspection and follow-up program, we determined
                        that, at a minimum, the files need to contain the following if the extent to
                        which repair stations are correcting problems in a timely manner is to be
                        monitored:



                        Page 49                                            GAO/RCED-98-21 Aviation Safety
    Chapter 3
    Incomplete Documentation of Inspections
    Makes Extent of Follow-Up Difficult to
    Determine




•   a memo to the file or other documentation showing that an inspection was
    performed, what was inspected, and the results;
•   a deficiency letter from FAA informing the repair station of the problems
    that needed to be corrected;
•   a response from the repair station indicating what actions it was taking to
    address the deficiencies; and
•   a memo to the file or other acknowledgment that the repair station’s
    actions were an acceptable response and that the deficiencies had been
    resolved.

    FAA officials agreed that these items are important in developing complete
    supporting information about the extent to which deficiencies were being
    resolved in a timely fashion.

    We believe—and FAA officials agreed—that beyond effective
    documentation in the repair station files, FAA also needs an effective
    management information system for capturing this basic information,
    combining it with information from other activities, and synthesizing it in
    ways that allow management to plan surveillance activities, schedule
    manpower resources, evaluate accomplishments, analyze results for
    patterns or trends, and modify planned activities. FAA’s management
    information tool for its inspection activity is its Program Tracking and
    Reporting Subsystem (PTRS). To provide data for planning and oversight of
    FAA’s inspection program, inspectors record inspection results in the
    computer-based PTRS. FAA’s PTRS Procedures Manual requires that
    inspectors record comprehensive reports demonstrating that inspections
    were performed, including inspection results, whether the repair station
    took any action, and whether the inspector took any follow-up action to
    ensure that deficiencies were corrected.

    Our analysis of the system showed that the key items of information
    needed for useful management reporting are the following:

•   an indication that a repair station was inspected and the results;
•   an indication that all deficiencies were communicated to the repair station
    in a deficiency letter; and
•   an indication that the deficiency letter was “closed out” when corrective
    actions by the repair station were determined to be acceptable by the
    inspector.

    FAA officials agreed that PTRS should contain these items of information if
    the system’s reports are to be of substantial use.



    Page 50                                          GAO/RCED-98-21 Aviation Safety
                              Chapter 3
                              Incomplete Documentation of Inspections
                              Makes Extent of Follow-Up Difficult to
                              Determine




Extent of Documentation       We did not find sufficient information in FAA’s repair station files to
in Repair Station Files       assemble a clear picture of how quickly and completely deficiencies found
                              during the inspections of repair stations were being corrected. This was
                              particularly true for domestic repair stations. In all, we reviewed the files
                              on 331 domestic and 157 foreign repair stations for fiscal years 1993
                              through 1996.1 Determining the speed and completeness with which
                              deficiencies were corrected was not possible for the following reasons:

                          •   When there was evidence that problems had been found, evidence of
                              corrective action was usually absent. Of the 331 domestic files reviewed,
                              96 contained deficiency letters, indicating that the facility had been
                              notified that problems existed. Response letters from repair stations were
                              present for 73 percent of the deficiency letters. However, only 22 percent
                              of the deficiency letters were accompanied by documentation showing
                              that the repair stations’ responses about resolving deficiencies were
                              acceptable (see fig. 3.1).
                          •   When there was no evidence in a file of a problem at a facility, this alone
                              could not be taken as assurance that no problems had been found. Even
                              for the 235 files that did not contain deficiency letters, it was not possible
                              to assume that FAA inspectors had not identified deficiencies because the
                              files contained no documentation showing that an inspection had been
                              completed.2




                              1
                               Our set of 488 files was based on a judgmental sample taken at 11 of the 13 domestic and foreign FAA
                              offices we visited. The sample ranged between 44 and 100 percent of those repair stations for which
                              the office had oversight responsibility and was limited to only repair stations doing work for air
                              carriers.
                              2
                               In our review of the files at domestic and foreign offices, when FAA’s documentation did not include
                              copies of deficiency letters, we did not follow up to determine whether repair stations had actually
                              received letters.



                              Page 51                                                           GAO/RCED-98-21 Aviation Safety
                                           Chapter 3
                                           Incomplete Documentation of Inspections
                                           Makes Extent of Follow-Up Difficult to
                                           Determine




Figure 3.1: Extent of Follow-Up Documentation for Inspections of Selected Domestic Repair Stations, Fiscal Years 1993
Through 1996



    Was a deficiency letter present?                                               If a deficiency letter was present,
                                                                                  was documentation of subsequent
                                                                                           follow-up in place?


                                   Deficiency letter                  Repair station's
                                   96 repair stations                     response to
                                                                      deficiency letter

                                                                    FAA's analysis of
                                                                      repair station's
                                                                           response


                                                                                          0        25         50         75            100
                                                                                                                                   b
                                                                                          Percentage with documentation in place




          No deficiency letter
          235 repair stations a

                                           a
                                            Files did not indicate whether these repair stations received satisfactory inspections.
                                           b
                                            Percentages are based on follow-up documentation for 172 deficiency letters (some repair
                                           stations had more than one deficiency letter during the period examined). Categories are
                                           exclusive of each other and do not add to 100.


                                           Our review of files on foreign repair stations found more complete
                                           documentation that follow-up had occurred (see fig. 3.2). We reviewed 157
                                           files and found deficiency letters in 135. Response letters from repair
                                           stations were present for nearly 80 percent of the 356 deficiency letters in
                                           these 135 files. An even higher percentage—85 percent—contained some
                                           form of documentation indicating that FAA had followed up. For most
                                           repair stations, this documentation took a form not found in domestic
                                           files—a certificate renewal letter. Unlike domestic repair stations, foreign
                                           repair station certificates are subject to renewal by FAA at least every 24
                                           months. In practice, many are renewed every year, according to FAA
                                           personnel. As part of the renewal, the FAA office issues a new certificate
                                           once it is satisfied that the repair station has taken appropriate actions to
                                           resolve the deficiencies. While certificate renewal letters were the most
                                           common form of follow-up documentation, nearly one-fourth of the files




                                           Page 52                                                           GAO/RCED-98-21 Aviation Safety
                                            Chapter 3
                                            Incomplete Documentation of Inspections
                                            Makes Extent of Follow-Up Difficult to
                                            Determine




                                            with deficiency letters also contained additional evidence of FAA’s analysis
                                            of the repair stations’ responses.



Figure 3.2: Extent of Follow-Up Documentation for Selected Foreign Repair Station Inspections, in Fiscal Years 1993
Through 1996



    Was a deficiency letter present?                                                If a deficiency letter was present,
                                                                                   was documentation of subsequent
                                                                                            follow-up in place?


                                  Deficiency letter                  Repair station's
                                  135 repair stations                    response to
                                                                     deficiency letter

                                                                 Recertification letter
                                                                   and/or other FAA
                                                                    analysis of repair
                                                                  station's response


                                                                                          0         25         50         75           100
                                                                                                                                   b
                                                                                          Percentage with documentation in place

                             No deficiency letter
                                                a
                             22 repair stations

                                            a
                                             Files did not indicate whether these repair stations received satisfactory inspections.
                                            b
                                             Percentages are based on follow-up documentation for 356 deficiency letters (some repair
                                            stations had more than one deficiency letter during the period examined). Categories are
                                            exclusive of each other and do not add to 100.


                                            One office has taken a further step to tie the renewal letter more closely to
                                            the resolution of deficiencies. According to the manager of the Frankfurt
                                            field office, the office recognized in 1996 that files contained no formal
                                            documentation that a repair station had taken corrective action because
                                            the renewal letters did not specifically mention it. As a result, the office
                                            revised the renewal letter, adding a statement that reads, “We are pleased
                                            to inform you that the corrective actions and corrective action plan
                                            developed subsequent to the Frankfurt International Field Office repair
                                            station inspection, has been reviewed and accepted by the principal
                                            inspector(s).” The office began using these letters in August 1996.




                                            Page 53                                                           GAO/RCED-98-21 Aviation Safety
                          Chapter 3
                          Incomplete Documentation of Inspections
                          Makes Extent of Follow-Up Difficult to
                          Determine




Extent of Documentation   Documentation in PTRS is even less complete than documentation in the
in Database Records       individual files on repair stations. We examined PTRS records to determine
                          the degree to which they contained inspection and follow-up information
                          for the deficiency letters we found in our review of domestic and foreign
                          repair station files. PTRS generally showed that an inspection had been
                          conducted, but responses on actions to correct deficiencies were less
                          frequently recorded than in the files, as were indications that closure had
                          been reached. A great deal of inconsistency was reflected in the data
                          entered into the system. Some inspectors entered inspection results in
                          great detail, others entered only partial data, and still others entered no
                          data at all about the deficiencies found. For example, the PTRS entry for
                          one inspection said, “Discrepancies listed in letter dated 04/16/96,” and
                          provided no indication whether deficiencies had been corrected. In
                          another example, the deficiency letter showed the inspector had
                          documented nine deficiencies, but PTRS showed only three of these
                          deficiencies and did not show if any had been corrected.


Lack of Reliable          As far back as 1987, we have reported on FAA’s shortcomings in having
Documentation Is a        current and reliable information on key program elements. In 1991, we
Long-Standing Problem     reported that the data in PTRS were unreliable for providing information on
                          the performance of FAA’s inspection program and were inadequate for
                          ensuring the accomplishment of key elements of the inspection program.3
                          In 1995, we concluded that FAA may be building its future information
                          management system from a number of databases that contain incomplete,
                          inconsistent, and inaccurate data.4 Again in 1996, we reported that until
                          FAA implements a strategy to improve the quality of its data, problems with
                          data quality may limit the usefulness of the system and prevent FAA from
                          realizing its full potential for targeting limited inspection resources to
                          higher-risk activities.5 In response to our reports, FAA has developed and
                          implemented a comprehensive strategy to improve data quality.

                          The Department of Transportation’s Inspector General has made similar
                          observations. In March 1994, the Inspector General reported that FAA
                          inspectors do not routinely document items inspected at repair stations or



                          3
                           Aviation Safety: Problems Persist in FAA’s Inspection Program (GAO/RCED-92-14, Nov. 20, 1991).
                          4
                           Aviation Safety: Data Problems Threaten FAA Strides on Safety Analysis System (GAO/AIMD-95-27,
                          Feb. 8, 1995).
                          5
                           Aviation Safety: New Airlines Illustrate Long-Standing Problems in FAA’s Inspection Program
                          (GAO/RCED-97-2, Oct. 17, 1996).



                          Page 54                                                         GAO/RCED-98-21 Aviation Safety
                            Chapter 3
                            Incomplete Documentation of Inspections
                            Makes Extent of Follow-Up Difficult to
                            Determine




                            follow-up actions taken.6 In 1995, the Inspector General found that FAA
                            inspectors were not interpreting PTRS reporting procedures consistently,
                            resulting in inaccurate, inconsistent reports.7


Quality of Data Threatens   The quality of PTRS data is important because PTRS is expected to provide
Effectiveness of New        data for FAA’s new information management initiative, the Safety
Management Information      Performance Analysis System (SPAS). SPAS is a computer-based analysis
                            system designed to assist FAA in applying its limited inspection resources
Initiative                  to those entities and areas that pose the greatest risk to aviation safety.
                            This system, estimated to cost $32 million to develop and install, is also
                            expected to highlight particular types of repair stations for increased
                            surveillance or oversight because they are experiencing problems at rates
                            that exceed the averages for that group. However, if the data on which
                            SPAS is based are not complete and accurate, FAA could be limited in its
                            ability to identify trends and target inspection resources.

                            In the past, we have recommended data improvements as a preliminary
                            step to implementing SPAS. Our 1995 report, which concluded that SPAS will
                            not be effective if the quality of its data is not improved, recommended
                            that FAA develop and implement a comprehensive strategy to make such
                            improvements. FAA agreed with this recommendation. Although FAA
                            initially intended to have its approach in place by the end of 1995, it was
                            October 1996 when FAA issued a strategy that provides clear and
                            measurable objectives for data quality, accurate assessments of the quality
                            of the current data in each database (including an analysis and possible
                            redirection of FAA’s existing initiatives to improve data quality), milestones
                            for attaining the stated quality objectives, and estimates of the resources
                            required. According to headquarters officials, full deployment of SPAS will
                            be completed in December 1999, as required by legislation. Even so, until
                            FAA completes the implementation of its strategy to improve data quality,
                            problems with data quality will limit SPAS’ usefulness and prevent it from
                            realizing its full potential.




                            6
                             Audit of the Certification and Surveillance of Domestic and Foreign Repair Stations: Federal Aviation
                            Administration, Office of Inspector General, U.S. Department of Transportation, R4-FA-4-009 (Mar. 7,
                            1994).
                            7
                              Surveillance of Pilot Schools: Federal Aviation Administration, Office of Inspector General, U.S.
                            Department of Transportation, R9-FA-002 (Nov. 8, 1995).


                            Page 55                                                            GAO/RCED-98-21 Aviation Safety
                            Chapter 3
                            Incomplete Documentation of Inspections
                            Makes Extent of Follow-Up Difficult to
                            Determine




                            While the lack of good documentation precludes a precise comparison of
Compliance May              FAA’s follow-up of deficiencies at foreign and domestic repair stations,
Come More Quickly at        some inspectors said that compliance comes more quickly at foreign
Foreign Repair              repair stations. We interviewed 34 FAA inspectors who had conducted
                            inspections of both foreign and domestic repair stations, and they were
Stations                    unanimous in concluding that compliance came more quickly at foreign
                            stations. They said quicker compliance meant inspectors spent less time
                            on follow-up and had more time for other work. The inspectors attributed
                            this quicker compliance to the renewal requirement for foreign repair
                            stations. Under FAA regulations, foreign repair stations must renew their
                            certificates within 12 months of initial issuance and then at least every 24
                            months thereafter. By comparison, domestic repair stations retain their
                            certificates indefinitely unless their operations are so badly run that FAA
                            elects to take legal action to suspend or revoke their certificates.
                            According to some inspectors, because new certificates cannot be issued
                            until problems are resolved, foreign repair stations have an incentive to
                            correct problems more quickly. Because of the poor documentation of the
                            inspection results for domestic repair stations, however, we were unable
                            to validate the inspectors’ views or to verify whether foreign repair
                            stations achieve compliance in a more timely fashion than domestic repair
                            stations.


Opinions Vary on            Within FAA, the strongest support for extending the certificate renewal
Advisability of Extending   requirement to domestic repair stations comes from inspectors who have
Renewal Requirement to      inspected both foreign and domestic repair stations. They solidly
                            supported extending renewal to domestic repair stations to gain quicker
Domestic Repair Stations    resolution of noncompliance issues. Of the inspectors responding to our
                            mail survey who had experience with foreign repair stations in fiscal year
                            1996, 89 percent supported extending the requirement. Similarly, 31 of the
                            34 inspectors we interviewed who had inspected both kinds of repair
                            stations said they favored a domestic renewal requirement. In our
                            discussions with inspectors, we were told that the lack of a renewal
                            requirement makes inspectors less efficient because they must spend
                            more time following up on repair stations with deficiencies and less time
                            on much needed surveillance or other responsibilities. A specific deadline
                            for correcting deficiencies to retain certification creates an incentive or
                            sense of urgency to resolve deficiencies in a timely manner.

                            Inspectors with experience only with domestic repair stations were
                            somewhat less favorable toward certificate renewal. Our survey results
                            show that 48 percent of the respondents who had experience only with



                            Page 56                                          GAO/RCED-98-21 Aviation Safety
Chapter 3
Incomplete Documentation of Inspections
Makes Extent of Follow-Up Difficult to
Determine




domestic repair stations in fiscal year 1996 were not in favor of renewing
domestic repair station certificates on a regular basis (for example, every
24 months) as a way to improve compliance. Our follow-up discussions
with 39 inspectors who had experience only with domestic repair stations
identified their reason for opposing such a requirement was a concern that
substantially more work would be generated. However, this perception
appears to be unfounded. The 34 inspectors we interviewed who had
inspected both kinds of repair stations indicated that extending the
requirement would not place additional requirements on inspectors. These
inspectors said inspection requirements for renewal were the same as
those for the annual facility inspection.

Support for extending the renewal requirement was lowest among FAA
management. They did not agree with inspectors who said that renewal
would improve safety by obtaining quicker resolution of noncompliance
problems. The Acting Manager of the Aircraft Maintenance Division said,
for example, “No one has demonstrated that FAA would get one added
ounce of safety if it revised the rule to require recertification of domestic
repair stations.” They also raised several concerns about adopting this
approach.

Additional resources would be needed. According to the Acting Manager
of the Aircraft Maintenance Division, it would take too many resources
(staff time, etc.) to recertify every repair station. We pointed out that
according to inspectors who had done both facility and renewal
inspections, the requirements were the same. For the most part, he did not
provide us with further information about what additional resources
would be needed if the recertification requirement were extended. FAA
attorneys, however, said they would need additional resources to process
any cases stemming from the denial of recertification.

Current procedures already allow certificates to be revoked. The current
enforcement process utilizes a range of enforcement actions—from
educational and remedial to punitive legal enforcement remedies,
including fines and criminal sanctions in the most serious cases. When FAA
determines that an immediate need exists to protect public safety, it can
issue an emergency order revoking or suspending a certificate. However,
although revoking a domestic repair station’s certificate is a possibility if
the facility remains out of compliance, it is a time-consuming process that
can often take years. In a forthcoming GAO report on FAA enforcement
actions, we analyzed all 2,200 certificate actions (this figure includes
airlines, repair stations, pilots, etc.) taken during fiscal year 1996 and



Page 57                                           GAO/RCED-98-21 Aviation Safety
              Chapter 3
              Incomplete Documentation of Inspections
              Makes Extent of Follow-Up Difficult to
              Determine




              found that it took an average of 13 months to close an enforcement case.
              Our point is not that a sizeable number of repair stations have such serious
              deficiencies that FAA should undertake enforcement actions. Rather, it is
              that certificate renewal appears to operate as a sort of “gate” that helps
              ensure that repair stations quickly fix their problems, big or small, thus
              helping to bring their operations up to the quality intended by Federal
              Aviation Regulations.

              Due process could be a concern. A headquarters official said that if FAA
              allows an inspector to make the decision about whether a repair station
              keeps or loses its certificate, it could infringe on the due process
              requirements afforded domestic repair station operators. However,
              according to FAA attorneys, due process would not be a concern. The
              manager of the FAA Airworthiness Law Branch explained that as currently
              is the case with foreign repair stations, the final decision not to renew a
              certificate would rest with the Administrator. The same due process rights
              accorded to foreign repair stations would apply to domestic repair
              stations.


              Our audit work and recent work by the Department of Transportation’s
Conclusions   Inspector General have identified continuing problems with the
              documentation of inspections and the quality of data entered into the
              Program Tracking and Reporting Subsystem, FAA’s tracking system. FAA’s
              guidance is very limited in specifying what documentation offices should
              keep. As a result, the documentation contained in the files on domestic
              repair stations, which acts as the official record of dealings between FAA
              and repair stations, did not contain sufficient data for us to determine how
              quickly and completely deficiencies found during inspections were being
              corrected. While FAA’s computer-based tracking system gives the agency
              means for overseeing the inspection program, problems with the quality of
              its data could jeopardize the reliability of FAA’s new computerized system,
              the Safety Performance Analysis System, in determining when to target
              greater inspection resources to repair stations that warrant more intensive
              oversight than others. In previous reports, we have recommended that FAA
              develop and implement a comprehensive strategy to improve the quality of
              all data used in its databases. Until FAA completes the implementation of
              this strategy, the extent and the impact of the problems with the quality of
              the system’s data will remain unclear.

              Data problems notwithstanding, there is some anecdotal evidence that
              foreign repair stations may have greater incentive than domestic repair



              Page 58                                         GAO/RCED-98-21 Aviation Safety
                      Chapter 3
                      Incomplete Documentation of Inspections
                      Makes Extent of Follow-Up Difficult to
                      Determine




                      stations to quickly fix deficiencies found during inspections because
                      foreign repair stations are faced with a certificate renewal requirement
                      and domestic repair stations are not. However, we were unable to verify
                      this because of FAA’s poor documentation, particularly for domestic repair
                      stations. Quick correction of all problems, large or small, helps to ensure
                      better repair station operations, and having to do less follow-up to
                      determine whether repair stations have taken corrective action frees FAA
                      inspectors to conduct other work. Views within FAA vary widely as to
                      whether a certificate renewal requirement, which appears to be at the
                      heart of quicker compliance, should be extended to domestic repair
                      stations. Although the evidence is not complete enough to support a
                      recommendation that FAA take such a step, extending the certificate
                      renewal requirement remains a potential option for consideration.


                      To ensure that FAA inspectors are effectively documenting and resolving
Recommendations       deficiencies found during inspections, we recommend that the Secretary
                      of Transportation instruct the Administrator of FAA to take the following
                      actions:

                  •   Specify what documentation should be kept in files on repair stations to
                      record complete inspection results and follow-up actions. The
                      documentation should include inspection results, deficiency letters, repair
                      station responses to deficiencies, and FAA’s responses indicating that the
                      deficiencies were corrected.
                  •   Monitor the implementation of the strategy to improve data quality to
                      ensure it is completed as soon as possible so that the data used in SPAS are
                      reliable when the system is fully implemented in 1999.


                      FAA agreed with these recommendations but did not indicate how or when
Agency Comments       it would implement them.




                      Page 59                                          GAO/RCED-98-21 Aviation Safety
Chapter 4

Actions Currently Under Way to Augment
Oversight of Repair Stations

                       Following the May 1996 crash of a ValuJet airplane in the Florida
                       Everglades, FAA announced six initiatives to upgrade the oversight of
                       repair stations. These initiatives are not aimed at the inspection and
                       follow-up activities discussed in the previous chapters. Instead, the
                       initiatives are aimed at clarifying and augmenting the oversight role of air
                       carriers, which, under FAA regulations, share responsibility for ensuring
                       that repair stations are qualified to do the work and are performing
                       responsibly. FAA did not intend that these initiatives would provide for any
                       significant improvements in FAA’s own inspections of repair stations.
                       However, other efforts now under way, coupled with our
                       recommendations in the previous chapters, could help address problems
                       with current inspections.

                       For many years, FAA officials have acknowledged that regulations
                       governing the inspection of repair stations need to be improved. Since
                       1989, FAA has been in the process of revising these regulations. Though
                       progress has been made, FAA officials remain uncertain about when the
                       proposed regulations will be published for review and comment.
                       Inspectors responding to our survey said overwhelmingly that revised
                       regulations would help the oversight effort. FAA is also adding more than
                       700 inspectors to its ranks who will, in part, oversee repair stations.
                       Survey responses from current inspectors indicated that the success of
                       this effort will depend in part on the qualifications of new inspectors and
                       on the training available to all those in the inspector ranks. Finally, FAA is
                       revising its regulations to require that repair station mechanics receive
                       more training.


                       FAA announced the improvements to its inspection policies on June 18,
Existing Initiatives   1996. These improvements consist of six specific initiatives designed,
Are Targeted           according to FAA, to “toughen the FAA’s oversight of airlines that rely on
Primarily at Air       contract maintenance and training.” Under Federal Aviation Regulations,
                       air carriers share with FAA the responsibility of ensuring that repair
Carriers               stations are conducting work that meets safety standards. FAA is
                       responsible for ensuring that repair stations comply with regulations; air
                       carriers are responsible for ensuring that repair stations perform
                       maintenance in accordance with the air carriers’ manuals. The six
                       initiatives were issued because FAA identified problems in the way some
                       air carriers provided oversight of repair stations, according to FAA’s Deputy
                       Associate Administrator for Regulation and Certification. Accordingly,
                       FAA’s initiatives focus on strengthening the oversight role of air carriers,




                       Page 60                                            GAO/RCED-98-21 Aviation Safety
Chapter 4
Actions Currently Under Way to Augment
Oversight of Repair Stations




not on modifying FAA’s own approach to inspecting repair stations and
ensuring that corrective action is taken.

The six initiatives involve actions to be taken by the air carriers or by FAA
inspectors overseeing air carriers (see fig. 4.1). Collectively, these
initiatives require that (1) air carriers demonstrate regulatory compliance
for each of their contract facilities doing substantial heavy maintenance or
repairs; (2) FAA ensure that air carriers list all contractors performing
substantial maintenance for them; and (3) air carriers audit repair stations
they want to begin using. They also call for additional review by FAA
inspectors—mainly those inspectors who oversee air carriers.




Page 61                                          GAO/RCED-98-21 Aviation Safety
                                             Chapter 4
                                             Actions Currently Under Way to Augment
                                             Oversight of Repair Stations




Figure 4.1: June 1996 Initiatives Designed to Improve Air Carrier Oversight of Repair Stations



                                     Initiative: FAA will require airlines to demonstrate the regulatory compliance of each
     Stricter repair                 of their major contract maintenance and training programs.
     station oversight
     by air carriers



                                    Initiative: FAA will review the airline's procedures to demonstrate compliance of
     More FAA review                major contract maintenance organizations to ensure that they are included as part of
     of what air carriers           the carrier's approved maintenance program. Also, quality assurance oversight
     are doing to                   performed by the carrier will be reviewed to ensure that the work conducted by the
     oversee repair                 contractor conforms to the maintenance program.
     stations
                                    Initiative: FAA will review the "check airman" involvement and on-site oversight
                                    being provided by the carrier of contract training facilities to ensure that the
                                    contractor is providing services that comply with the regulatory requirements levied
                                    upon the carrier.

                                    Initiative: FAA will create new oversight requirements for inspectors who monitor
                                    repair stations and training centers. These inspectors will be required to check not
                                    only a repair station's compliance with the regulations but also air carriers' assurance
                                    that the maintenance and repair done by the repair station are in compliance with air
                                    carriers' maintenance programs.




                                     Initiative: FAA's principal inspectors will require that carriers list all contractors
     Listing all repair              performing substantial maintenance and training in an airline's operations
     stations performing             specifications.
     substantial
     maintenance




                                     Initiative: Before the use of new contractors is approved by the principal inspector
     Air carrier audits of           for addition to the operations specifications, the carrier must conduct an audit of the
                                     contractor. This audit must demonstrate to the principal inspector that the
     any repair stations             contractor is capable of performing the contracted work in accordance with the
     added to the list               carrier's approved programs.




                                             Page 62                                                          GAO/RCED-98-21 Aviation Safety
                         Chapter 4
                         Actions Currently Under Way to Augment
                         Oversight of Repair Stations




                         Because these initiatives were directed at air carriers, they had little or no
                         effect on FAA’s direct oversight of repair stations. In fact, all 72 repair
                         station inspectors who responded specifically to our question about the
                         impact of FAA’s initiatives said that the initiatives and guidance have had
                         no impact on the extent of their surveillance activities. They told us that
                         because the initiatives were directed at the air carriers and the FAA
                         inspectors responsible for overseeing the air carriers, rather than at the
                         inspectors overseeing repair stations, their oversight activities have not
                         changed.


FAA Has Taken Steps to   FAA implemented these initiatives through two major efforts—issuing
Implement All Six        guidelines and issuing a handbook bulletin. In July 1996, FAA headquarters
Initiatives              issued the National Flight Standards Work Program Guidelines. These
                         guidelines stated that air carriers would be required to demonstrate that
                         programs at each major repair station complied with regulations. The
                         guidelines required air carriers to list all repair stations performing
                         substantial heavy maintenance in their operations specifications.1

                         For the FAA inspectors whose duties involve monitoring the air carriers (as
                         opposed to monitoring the repair stations themselves), the guidelines
                         stressed the importance of careful oversight but did not require specific
                         actions on the inspectors’ part. FAA headquarters officials said the
                         guidelines left it to inspectors to decide whether to place more emphasis
                         on the surveillance of repair stations as part of their oversight of air
                         carriers. Matters in the guidance for inspectors to consider in making their
                         decisions included a particular FAA office’s work demands and the
                         complexities or problem areas of the repair stations involved. Inspectors
                         overseeing air carriers told us these initiatives have increased their
                         awareness of the need to oversee repair stations working for air carriers,
                         but inspectors overseeing repair stations told us the initiatives had not
                         changed their oversight activities.

                         FAA augmented the guidelines a month later with a handbook bulletin that
                         provided more specific guidance for the last two initiatives shown in figure




                         1
                          Operations specifications name the maintenance activities the repair station is authorized to perform.
                         FAA issues a set of operations specifications during certification.



                         Page 63                                                           GAO/RCED-98-21 Aviation Safety
    Chapter 4
    Actions Currently Under Way to Augment
    Oversight of Repair Stations




    4.1. The handbook bulletin defined “substantial maintenance”2 for air
    carriers, thereby clarifying which repair stations needed to be included on
    an air carrier’s operations specifications or audited by an air carrier prior
    to adding them to the list. The handbook bulletin’s more specific guidance
    on these matters was as follows:

•   To implement the initiative that all repair stations performing significant
    maintenance be listed, FAA inspectors were to list, by October 1, 1996, the
    repair stations that performed substantial maintenance, ensure that each
    repair station had had a current audit, include the repair stations as part of
    the operations specifications, and update FAA’s Vital Information
    Subsystem database with the new operation specification information.
•   To implement the initiative that air carriers conduct audits of any repair
    stations added to the list, FAA inspectors were to ensure, effective
    September 1, 1996, that air carriers audit any new maintenance repair
    station they want to add to their operations specifications. The audits’
    purpose was to ensure that the repair stations are capable of performing
    the contracted work in accordance with the carriers’ approved programs.
    Under the handbook’s procedures, FAA must also review and accept the
    audit before an air carrier can use the contractor.

    FAA officials told us about 150 air carriers needed to comply with the
    bulletin by identifying the repair stations performing substantial
    maintenance for them. The officials said all of the carriers had complied
    by late September or early October 1996. Subsequent to that, air carriers
    have added other repair stations to their operations specifications after
    having their audits of the repair stations approved by FAA inspectors. For
    example, one air carrier recently added two repair stations to its
    operations specifications after the FAA inspector reviewed and approved
    both repair station audits. In contrast, another FAA inspector responsible
    for an air carrier denied use of two new repair stations when he found
    deficiencies in the audit reports for both repair stations.

    FAA officials stated that the review of air carriers’ audits of repair stations
    is something the inspectors will be doing on an ongoing basis. Any time an
    air carrier wants to add a repair station that does substantial maintenance,
    the carrier must audit the repair station and the principal maintenance

    2
     FAA defined substantial maintenance as any activity involving a C-check (routine airframe
    maintenance) or greater maintenance; any engine maintenance requiring case separation or teardown;
    and/or major alterations or repairs performed on airframes, engines, or propellers. Under FAA’s
    definition of “substantial maintenance,” a relatively small number of repair stations are likely to be
    affected by these requirements. We reviewed the operations specifications for 10 air carriers that
    implemented FAA’s requirements. Of the more than 5,000 total vendors on these air carriers’ lists of
    approved vendors, 268 were classified as authorized to perform “substantial maintenance.”



    Page 64                                                           GAO/RCED-98-21 Aviation Safety
                         Chapter 4
                         Actions Currently Under Way to Augment
                         Oversight of Repair Stations




                         inspector for that air carrier will need to accept the audit before the repair
                         station can be used. Moreover, every time a new repair station is added,
                         the air carrier’s operations specifications must be changed.


                         FAA headquarters officials and all of the 86 repair station inspectors we
Planned Update of        interviewed told us current regulations governing the oversight of repair
Repair Station           stations are out of date in a number of respects. According to FAA, the
Regulations Is Slow in   current repair station regulations are based primarily on concepts that
                         were developed during the infancy of the aviation industry. Aircraft, power
Coming                   plants, maintenance, alteration concepts, and technology have progressed
                         substantially in the last three decades. However, very few substantive
                         changes have been made to the regulations since 1962. Portions of the
                         regulations are no longer appropriate or have become increasingly difficult
                         to administer, while some other portions no longer make a significant
                         contribution to aviation safety. As a result, FAA has had to grant
                         exemptions and create special administrative procedures to handle
                         situations not provided for adequately in the regulations.

                         In 1975, FAA and industry officials recommended revising substantial
                         requirements of the repair station regulations. According to FAA, minor
                         amendments to the regulations were subsequently adopted, but no major
                         revision was made. In 1989, in light of public meetings that were part of its
                         regulatory review, FAA decided to revise the regulations completely.
                         However, it has taken 8 years to prepare the revisions, FAA officials said.
                         They attributed the delays to the project being preempted by other
                         rulemaking and policy projects.

                         FAA hopes to begin implementing revised regulations in the coming
                         months. It has prepared revised regulations, and headquarters officials
                         told us the revisions are now being reviewed by the Department of
                         Transportation and the Office of Management and Budget. Officials did not
                         know when the revisions would be published for comment in the Federal
                         Register. FAA had established a target of summer 1997 for publishing these
                         revisions, but this target was not met. FAA will consider the comments
                         received before taking action on the proposed revisions. The Deputy
                         Associate Administrator for Regulation and Certification anticipated that
                         FAA would receive voluminous comments, necessitating considerable time
                         for review and response.

                         FAA headquarters officials outlined several things they hoped to
                         accomplish with the proposed regulations. First, they are proposing that



                         Page 65                                           GAO/RCED-98-21 Aviation Safety
Chapter 4
Actions Currently Under Way to Augment
Oversight of Repair Stations




domestic repair stations be required to have a quality control system that
is based on the Joint Aviation Authorities’ (JAA) system.3 Second, the
proposed regulations simplify the repair station rating system and make
the ratings less confusing than the existing system. Third, the proposed
regulations impose training requirements for entry-level personnel.
Existing regulations do not require that entry-level personnel be trained.
They require only that repair stations use practical tests or employment
records to determine the abilities of uncertified employees. Finally, the
proposed regulations make repair stations responsible for controlling and
evaluating their vendors. Existing regulations do not require that repair
stations evaluate their subcontractors or vendors. They require only that
repair stations have a method of inspecting incoming material to ensure
that it is free from apparent defects or malfunctions. FAA attorneys noted
that the proposed rule will remove the distinction between most domestic
and foreign repair stations.

There are indications that some in the repair station industry may oppose
many of these changes. For example, the National Air Transportation
Association, an industry association, stated that it expects that the
proposed regulations will require new training programs, additional
record-keeping requirements, and the implementation of quality assurance
systems like the air carriers’ quality assurance systems. This, the
association predicts, will increase repair station costs, causing as many as
a third of them to turn in their repair station certificates. The association
contends that the anticipated proposed regulations will “cripple the
maintenance industry,” and it plans to fight them. This opposition
indicates that completing the rulemaking process may take a significant
amount of time.

The importance of completing this project can be seen in inspectors’
responses to our survey. As figure 4.2 shows, most inspectors believe that
various changes in the regulations would help them carry out their
inspection duties. Of the inspectors we surveyed, most (88 percent)
favored updating the regulations as a way to improve repair station
regulatory compliance. In particular, most inspectors (77 percent) favored
changes to require repair stations to notify FAA of the names of air carriers
for which they do work. Most inspectors (78 percent) also favored


3
 A number of European nations have banded together to coordinate their efforts through JAA. Like
FAA, the national aviation authorities of JAA member nations have developed their own extensive
inspection, surveillance, evaluation, and certification programs for repair stations. Under the
provisions of the Bilateral Aviation Safety Agreement, FAA is negotiating agreements and procedures
with several countries that will eliminate the duplicate oversight of repair facilities by FAA and such
other entities. Discussions are still largely in the formative stage.



Page 66                                                            GAO/RCED-98-21 Aviation Safety
                                        Chapter 4
                                        Actions Currently Under Way to Augment
                                        Oversight of Repair Stations




                                        changing regulations to require air carriers to provide their manuals or
                                        procedures along with the parts to be repaired by repair stations.


Figure 4.2: FAA Inspectors’ Responses
to Ideas for Improving Compliance
                                         100      Percentage of respondents
Through Changes Related to Existing
Regulations

                                           80



                                           60



                                           40



                                           20



                                            0
                                                  Issue update            Require repair      Change
                                                  for repair              stations to         regulations to
                                                  station                 notify FAA of       require airlines to
                                                  regulations             the names of        provide
                                                                          all air carriers    maintenance
                                                                          for which they      manuals or
                                                                          do work             procedures with
                                                                                              parts to be
                                                                                              repaired
                                                      Strongly or generally in favor
                                                      Neutral
                                                      Strongly or generally opposed



                                        Note: Percentages may not add to 100 because a few respondents said they had no basis to
                                        make a judgment.




                                        Inspectors provided examples of why revised regulations are needed. One
                                        inspector said the regulations do not address new repair techniques such
                                        as nondestructive testing and repair of composite materials, which means
                                        that the inspector must evaluate a repair station’s practices using his or
                                        her own judgment. Two inspectors noted that regulations require repair




                                        Page 67                                                      GAO/RCED-98-21 Aviation Safety
                      Chapter 4
                      Actions Currently Under Way to Augment
                      Oversight of Repair Stations




                      stations to have an FAA-approved inspection procedure manual but do not
                      require repair stations to follow it. As a result, an inspector who finds that
                      a repair station failed to follow its approved manual cannot write a
                      violation.


                      The Congress has provided FAA with additional resources to hire more
Adding to the         inspectors. FAA increased the number of inspectors from 2,324 in fiscal
Inspector Corps and   year 1994 to an estimated 3,062 at the end of fiscal year 1997, a 32-percent
Improving Training    increase. FAA’s approved budget for fiscal year 1995 authorized 201
                      additional inspectors; for fiscal year 1996, 237 additional inspectors, and
                      for fiscal year 1997, 300 additional inspectors. To fill the additional
                      authorizations and to rehire for attrition, FAA hired 302 inspectors in fiscal
                      year 1995 and 361 in fiscal year 1996. About 63 percent of the inspectors
                      hired were airworthiness inspectors. According to the manager of the
                      Human Resource Programs Branch, most airworthiness inspectors have
                      the oversight of repair stations as part of their duties. FAA has requested
                      235 additional inspectors in its fiscal year 1998 budget estimate. If the
                      request is approved, this would represent a 42-percent increase in
                      inspector staffing since fiscal year 1994.

                      Responses to our survey indicate that current FAA inspectors believe the
                      impact of these new inspectors will depend mainly on how strong the
                      applicants’ qualifications are. As figure 4.3 shows, current inspectors
                      believe that aviation industry experience, particularly supervisory
                      experience, is important. For example, 81 percent of inspectors
                      responding to our survey strongly or generally favored having inspectors
                      with hands-on technical experience in industry as a way to improve repair
                      stations’ compliance. FAA’s current qualifications for entry-level inspectors
                      require maintenance experience in a repair station, air carrier facility, or
                      military repair facility, and 3 years of supervisory experience. The manager
                      of the Human Resources Program Branch told us FAA requires that
                      inspectors have 3 years of supervisory experience because they must be
                      able to communicate orally and in writing with mechanics, engineers, and
                      managers. We did not determine whether FAA’s newly hired inspectors met
                      the agency’s qualification standards for new hires.




                      Page 68                                           GAO/RCED-98-21 Aviation Safety
                                        Chapter 4
                                        Actions Currently Under Way to Augment
                                        Oversight of Repair Stations




Figure 4.3: FAA Inspectors’ Responses
to Ideas for Improving Compliance
                                            100   Percentage of respondents
Through Inspector Qualifications


                                            80



                                            60



                                            40



                                            20



                                             0

                                                  Ensure that inspectors            Ensure that inspectors
                                                  have hands-on technical           have supervisory
                                                  experience in industry            experience in industry



                                                       Strongly or generally in favor
                                                       Neutral
                                                       Strongly or generally opposed



                                        Note: Percentages may not add to 100 because a few respondents said they had no basis to
                                        make a judgment.




                                        The training of new and existing inspectors is another area that has been a
                                        focus of attention. We and others have reported for several years that FAA’s
                                        aviation safety inspectors are not receiving needed training. Most recently,
                                        in October 1996 we issued a report recommending that FAA evaluate the
                                        impact of recent budget reductions on critical safety-related functions,
                                        including training, and report the results to the Congress through the
                                        appropriations process.4 FAA inspectors’ responses to our survey indicate
                                        that most inspectors continue to be concerned about the need for
                                        improved training. Specifically, 82 percent of the inspectors surveyed said
                                        they strongly or generally favored providing inspectors with maintenance


                                        4
                                         See Aviation Safety: New Airlines Illustrate Long-Standing Problems in FAA’s Inspection Program
                                        (GAO/RCED-97-2, Oct. 17, 1996).



                                        Page 69                                                         GAO/RCED-98-21 Aviation Safety
                           Chapter 4
                           Actions Currently Under Way to Augment
                           Oversight of Repair Stations




                           and avionics training, including hands-on training, as a way to improve
                           repair stations’ compliance with regulations. Over three-quarters of the
                           inspectors (80 percent) favored more training on inspection skills.
                           Additionally, 45 percent said that the inability to get needed training is at
                           least a minor reason why inspectors are not able to ensure repair stations’
                           compliance with all aspects of the regulations. These results add support
                           for our 1996 recommendation.

                           Inspectors also expressed their concern about inadequate training in
                           written comments on our survey forms. For example, one inspector stated
                           that inspectors need specific training on aircraft and systems. Another
                           inspector wrote, “I have completed FAA repair station certification and
                           surveillance course; however, that course does not educate in the
                           procedures for overhaul. To understand the product, I am relying on
                           personal experience I had before I joined FAA, with no recurrent training
                           on the actual product.” Another inspector stated he needed more hands-on
                           training on a turbine engine before he was sent to inspect it.


                           Because of significant technological advances in the aviation industry,
Upgrading                  current FAA regulations that prescribe the certification requirements for an
Certification and          estimated 145,000 mechanics and repairmen need to be updated. Aviation
Training of Aviation       maintenance is one of the most complex areas of the industry, and
                           aviation maintenance personnel must possess many technical skills.
Maintenance                Changes in aircraft technology have also significantly increased the need
Personnel                  for specialized training. FAA has been updating, consolidating, and
                           clarifying all its certification, training, experience, and currency
                           requirements for aviation maintenance personnel for a new rule (14 C.F.R.
                           part 66) entitled “Certification: Aviation Maintenance Personnel.”
                           According to FAA, some of the key features of this proposed rule include
                           the

                       •   creation of additional certificates and ratings for aviation maintenance
                           personnel,
                       •   expansion of current certification requirements, and
                       •   establishment of additional training and recurrent training requirements
                           for certified aviation maintenance personnel.

                           Initially, on August 17, 1994, the proposed rule was published in the
                           Federal Register for public comment. However, the final issuance of the
                           rule has been delayed because FAA officials decided that to avoid
                           confusion they need to combine the proposed rule with other rule



                           Page 70                                           GAO/RCED-98-21 Aviation Safety
                  Chapter 4
                  Actions Currently Under Way to Augment
                  Oversight of Repair Stations




                  revisions. According to FAA headquarters officials, FAA plans to reissue the
                  revised proposed rule in the Federal Register for public comment in
                  December 1997.


                  Although the various activities FAA has under way may help strengthen the
Conclusions       oversight of repair stations, none of them directly addresses the concerns
                  about inspection and follow-up that we discussed in chapters 2 and
                  3—namely the limited success in identifying problems through reviews by
                  individual inspectors of large facilities and the inadequate documentation
                  of efforts to correct deficiencies found during inspections. FAA’s initiatives
                  may help the air carriers—and the FAA inspectors who monitor those air
                  carriers—be more attentive to the work being performed by repair
                  stations, but they do not appear to have any direct link to improving the
                  quality of FAA’s inspections of repair stations or the speed and
                  thoroughness with which problems are resolved. Also, as FAA has struggled
                  to deal with a growing workload caused by new airlines and the greater
                  complexities of a deregulated environment, FAA has received a 32-percent
                  increase in the number of its inspectors since fiscal year 1994. To use
                  these additional resources as effectively as possible, FAA needs to
                  overcome its inspection program’s weaknesses in identifying problems at
                  repair stations and in documenting inspection results that need follow-up.

                  The results of our work also underscore the need for progress in several
                  areas that FAA is addressing by updating repair station regulations, hiring
                  new inspectors, and improving training programs. Progress on initiatives
                  for updating regulations on the oversight of repair stations and the
                  certification and training requirements for maintenance personnel has
                  been slow. These efforts may require additional management attention.


                  To ensure that outdated regulations governing the oversight of repair
Recommendation    stations and certification and training requirements for maintenance
                  personnel are updated as soon as possible, we recommend that the
                  Secretary of Transportation instruct the Administrator of FAA to expedite
                  the efforts to update the regulations and to establish and meet schedules
                  for completing the updates.


                  FAA agreed with the recommendation but did not indicate how or when it
Agency Comments   would be implemented.




                  Page 71                                           GAO/RCED-98-21 Aviation Safety
Appendix I

Objectives, Scope, and Methodology


                 In December 1996, the Ranking Minority Member of the Aviation
                 Subcommittee of the Senate Committee on Commerce, Science, and
                 Transportation, and Senator Ron Wyden asked us to examine FAA’s
                 oversight of repair stations. Specifically, we were asked to address the
                 following questions:

             •   What is the nature and scope of the oversight of repair stations, conducted
                 by FAA personnel?
             •   How well does FAA follow up on inspections to ensure that deficiencies in
                 repair stations’ operations are corrected once they have been identified?
             •   What steps has FAA taken to improve the oversight of repair stations?

                 Our analysis was based in part on agencywide data and in part on a
                 detailed review involving a cross-section of airlines, repair stations, FAA
                 offices, and FAA inspectors. Our industrywide data included the amount of
                 maintenance costs incurred by airlines. For a more in-depth look at how
                 air carriers were using repair stations, we selected eight air carriers for
                 review, choosing them in part because, like the industry as a whole, they
                 varied greatly in the extent to which they used repair stations (see table
                 I.1). At the eight airlines, we discussed the type and amount of
                 maintenance contracted to repair stations. Also, we discussed the impact
                 of recent FAA initiatives on the air carrier and the methods each uses to
                 oversee repair stations.




                 Page 72                                          GAO/RCED-98-21 Aviation Safety
                                        Appendix I
                                        Objectives, Scope, and Methodology




Table I.1: Airlines Included in GAO’s
Detailed Review                                                                          Percentage of       Nature of
                                                                                         maintenance         maintenance
                                                                   Nature of             performed by        activity done by
                                        Airline                    operations            repair stations     repair stations
                                        Alaska Airlines            Uses 74 aircraft on   10 (airframe)       Airframes, power
                                                                   the Pacific Coast     90 (power plants)   plants, radios,
                                                                   from Alaska to        75 (instruments     instruments and
                                                                   Mexico and the        and accessories)    accessories
                                                                   Russian Far East
                                        America West Airlines      Uses 102 aircraft  85                     Airframes, power
                                                                   on routes                                 plants, radios,
                                                                   throughout the                            instruments and
                                                                   U.S. and to Mexico                        accessories
                                                                   and Canada
                                        American Airlines          Uses 659 aircraft     26                  Power plants,
                                                                   on routes                                 radios, instruments
                                                                   throughout the                            and accessories,
                                                                   world                                     but primarily
                                                                                                             interior
                                                                                                             modifications
                                        Delta Airlines             Uses 542 aircraft     10                  Airframes, power
                                                                   on routes                                 plants, radios,
                                                                   throughout the                            instruments and
                                                                   world                                     accessories, but
                                                                                                             primarily
                                                                                                             modifications and
                                                                                                             predeparture
                                                                                                             checks
                                        Sierra Pacific Airlines    Uses 3 aircraft for   90-95               Airframes, power
                                                                   charter and                               plants, radios,
                                                                   supplemental                              instruments,
                                                                   services on a                             propellers, and
                                                                   contractual basis                         accessories
                                        Simmons Airlines           Uses 93 aircraft  12                      Airframes, power
                                                                   operating as                              plants, and
                                                                   American Eagle on                         accessories
                                                                   routes in the
                                                                   Southwest and
                                                                   Midwestern U.S.
                                        Southwest Airlines         Uses 242 aircraft   85                    Airframes, power
                                                                   on routes                                 plants, radios,
                                                                   throughout the U.S.                       instruments and
                                                                                                             accessories
                                        United Airlines            Uses 591 aircraft     7                   Airframes, power
                                                                   on routes                                 plants, radios,
                                                                   throughout the                            instruments and
                                                                   world                                     accessories, but
                                                                                                             primarily
                                                                                                             modification and
                                                                                                             routine
                                                                                                             maintenance



                                        Page 73                                                  GAO/RCED-98-21 Aviation Safety
                                         Appendix I
                                         Objectives, Scope, and Methodology




                                         We also visited 10 repair stations, choosing them in part because they
                                         represented a variety of locations (both domestic and foreign), types of
                                         repair station activities, and size of operations (see table I.2). At these
                                         repair stations, we observed maintenance capabilities, shipping and
                                         receiving procedures, and work package documentation. We also met with
                                         quality assurance officials to talk about internal controls over parts,
                                         suspected unapproved parts, and oversight by FAA and air carrier
                                         representatives.

Table I.2: Repair Stations Included in
Detailed Review                          Repair station and location                 Size and extent of operations
                                         Advanced Material Technologies, Inc.        Employs about 170 people and mainly
                                         Tempe, Arizona                              performs work on power plants and
                                                                                     accessories
                                         AeroControls, Inc.                          Employs about 450 people and performs
                                         Auburn, Washington                          all types of maintenance work except
                                                                                     power plants
                                         B.F. Goodrich Component Services Division   Employs about 20 people and works
                                         Tempe, Arizona                              exclusively on wheels and brakes; one of a
                                                                                     number of B.F. Goodrich repair stations
                                         Chromalloy Los Angeles                      Employs about 205 people and works
                                         Gardena, California                         exclusively on power plants; one of a
                                                                                     number of Chromalloy repair stations
                                         Evergreen Air Center                        Employs about 590 people and performs
                                         Marana, Arizona                             all types of work except instruments
                                         Greenwich Air Services, Inc.                Employs about 840 people and mainly
                                         Miami, Florida                              performs work on power plants and
                                                                                     accessories
                                         Lufthansa Technik AG                        Employs about 3,300 people and performs
                                         Frankfurt, Germany                          routine and contract maintenance for
                                                                                     Lufthansa and other carriers on airframes,
                                                                                     power plants, and accessories
                                         Precision Avionics and Instruments, Inc.    Employs about 35 people and works
                                         Atlanta, Georgia                            exclusively on electrical and electronic
                                                                                     instruments, components, and accessories
                                         Sabena Technic                              Employs about 1,600 people and performs
                                         Brussels National Airport, Belgium          work for Sabena and other carriers in all
                                                                                     maintenance areas
                                         Tramco, Inc.                                Employs about 2,200 people and mainly
                                         Everett, Washington                         performs work on airframes and
                                                                                     accessories; one of several B.F. Goodrich
                                                                                     repair stations

                                         We conducted reviews at FAA headquarters in Washington, D.C., four of the
                                         nine FAA regional offices (Northwest Mountain, Southern, Southwest, and
                                         Western-Pacific), 8 of FAA’s 86 Flight Standard district offices (Atlanta,




                                         Page 74                                                GAO/RCED-98-21 Aviation Safety
                                       Appendix I
                                       Objectives, Scope, and Methodology




                                       Dallas, Dallas/Fort Worth, Fort Worth, Los Angeles, Miami, Scottsdale, and
                                       Seattle), and six of the seven international offices located inside and
                                       outside the United States (Brussels, Dallas/Fort Worth, Frankfurt, London,
                                       Miami, and San Francisco). We selected the regional and Flight Standards
                                       district offices because of their geographical diversity and because the
                                       locations were responsible for overseeing both domestic and foreign
                                       repair stations. We reviewed FAA’s criteria, procedures, and other
                                       documents used to oversee domestic and foreign repair stations and
                                       discussed them with 86 principal maintenance and avionics inspectors
                                       responsible for overseeing repair stations and 17 inspectors responsible
                                       for air carriers that contract with repair stations. We also conducted
                                       detailed reviews of a judgmental sample of 488 repair station files to
                                       determine if inspection and follow-up activity was documented effectively.
                                       In addition, we analyzed data from FAA’s computer-based Program
                                       Tracking Reporting Subsystem for fiscal years 1993 through 1996 to
                                       determine if FAA effectively captured repair station inspection results.

                                       To gather information about the experiences of FAA inspectors overseeing
                                       repair stations and their views on ways to improve the oversight of repair
                                       stations, we conducted a mail survey. We mailed our survey to a sample of
                                       275 of the 601 FAA inspectors listed in FAA’s Vital Information Subsystem as
                                       being responsible for the oversight of one or more repair stations doing
                                       work for part 121 or 135 carriers during fiscal year 1996. In total, we
                                       received responses from 247 inspectors, a response rate of 90 percent. Of
                                       these, 202 completed the questionnaire, while 45 did not because they did
                                       not oversee any repair stations in fiscal year 1996. (See app. II of this
                                       report for additional information on the survey methodology and app. III
                                       for the summary results of that survey.) The 202 respondents were
                                       responsible for overseeing a total of 1,375 repair stations that performed
                                       maintenance for part 121 or 135 carriers, with the number of repair
                                       stations per inspector ranging from 1 to 37. Table I.3 shows the sizes of
                                       repair stations overseen by the inspectors responding to our survey.

Table I.3: Distribution of Repair
Stations Overseen by FAA Inspectors,   Size of repair station (number of
by Size                                employees)                                                                    Percentage of total
                                       More than 100                                                                                 17
                                       16-100                                                                                        35
                                       15 or fewer                                                                                   48
                                       Total                                                                                         100
                                       Note: Percentages can be generalized to the nearly 2,800 repair stations doing work for air
                                       carriers.




                                       Page 75                                                         GAO/RCED-98-21 Aviation Safety
Appendix I
Objectives, Scope, and Methodology




We also interviewed officials from the government agencies that are
responsible for reviewing FAA’s oversight of repair stations and air carriers:
the Office of the Inspector General, U.S. Department of Transportation;
the Air Carrier Survey and Analysis Office, Air Mobility Command Staff,
U.S. Air Force, Department of Defense; and the National Transportation
Safety Board.

We conducted our review from August 1996 through October 1997, in
accordance with generally accepted government auditing standards.




Page 76                                           GAO/RCED-98-21 Aviation Safety
Appendix II

Survey Methodology


              To examine the experiences and opinions of the FAA employees who
              inspect repair stations, we surveyed inspectors within FAA’s Flight
              Standards Service. We mailed our survey to a random sample of inspectors
              listed in FAA’s Vital Information Subsystem database as conducting
              inspections of repair stations that do work for air carriers. The survey
              asked for such information as the degree of success inspectors have in
              identifying all deficiencies, factors that hinder inspectors’ success in
              ensuring that repair stations comply with all aspects of the regulations,
              and ways to improve the compliance of repair stations. Our response rate
              for the survey was 90 percent. A summary of the responses is in appendix
              III.

              FAA’s database indicated that 601 inspectors were responsible for repair
              stations that do work for air carriers. We drew a random sample of 275
              inspectors. This sample size was designed to provide sampling errors of no
              more than 5 percent at the 95-percent confidence level.

              Since we used a sample (called a probability sample) of 275 FAA inspectors
              to develop our estimates, each estimate has a measurable precision, or
              sampling error, that may be expressed as a plus/minus figure. A sampling
              error indicates how closely we can reproduce from a sample the results
              that we would obtain if we were to take a complete count of the universe
              using the same measurement methods. By adding the sampling error to
              and subtracting it from the estimate, we can develop upper and lower
              bounds for each estimate. This range is called a confidence interval.
              Sampling errors and confidence intervals are stated at a certain
              confidence level—in this case, 95 percent. (See table II.1.) For example, a
              confidence interval, at the 95-percent confidence level, means that in 95
              out of 100 instances, the sampling procedure we used would produce a
              confidence interval containing the universe value we are estimating.




              Page 77                                         GAO/RCED-98-21 Aviation Safety
                                         Appendix II
                                         Survey Methodology




Table II.1: Sampling Errors for
Selected Percentages                     Percentage                                                                        Sampling error
                                         5                                                                                                   ±2
                                         10                                                                                                  ±3
                                         20                                                                                                  ±4
                                         30                                                                                                  ±5
                                         40                                                                                                  ±5
                                         50                                                                                                  ±5
                                         60                                                                                                  ±5
                                         70                                                                                                  ±5
                                         80                                                                                                  ±4
                                         90                                                                                                  ±3
                                         95                                                                                                  ±2
                                         Note: Sampling errors are calculated for the 95-percent confidence level using the finite
                                         population correction factor and 185 cases, the smallest number of valid cases for questions with
                                         finite categories.



                                         In addition, table I.3 in appendix I requires ratio estimates in order to
                                         calculate sampling errors. Table II.2 shows sampling errors for these
                                         estimates.

Table II.2: Sampling Errors for Repair
Stations by Number of Employees                                                                                          Sampling error
                                         Size of repair station                                                    (percentage added or
                                         (number of employees)                                Percentage                    subtracted)a
                                         More than 100                                                 16.6                             ±2.7
                                         16-100                                                        35.3                             ±3.5
                                         15 or fewer                                                   48.1                             ±4.1
                                         Total                                                       100.0                               n/a
                                         a
                                         Sampling errors calculated at the 95-percent confidence level.



                                         We conducted 11 pretests of our survey with inspectors in two of FAA’s
                                         nine regions, including 3 pretests with inspectors of foreign repair stations
                                         and 8 pretests with inspectors of domestic repair stations. Each pretest
                                         consisted of a visit with a single FAA employee by two or three GAO staff.
                                         The pretest attempted to simulate the actual survey experience by asking
                                         the employee to fill out the questionnaire while the GAO staff observed and
                                         unobtrusively took notes. Then the employee was interviewed about the
                                         questionnaire items to ensure that (1) the questions were readable and
                                         clear, (2) the terms were precise, (3) the survey did not place an undue




                                         Page 78                                                          GAO/RCED-98-21 Aviation Safety
Appendix II
Survey Methodology




burden on FAA employees that would result in a lack of cooperation, and
(4) the survey appeared independent and unbiased in its point of view. We
modified the final survey based on our pretesting results.

In addition to our pretesting, we had managers in FAA’s Flight Standards
Service in Washington, D.C., review our questionnaire, and we obtained
comments from a survey research psychologist at FAA’s Training and
Organizational Research Laboratory in Oklahoma City and from officials
of the union representing the FAA inspectors we surveyed. We incorporated
comments from these reviews as appropriate.

During the pretesting phase, it became evident that respondents
considered the survey questions to be sensitive. Specifically, some
respondents said less experienced inspectors might be afraid of
retribution if their answers were made public. To address these concerns,
we developed procedures to guarantee the complete anonymity of all
survey responses. To do this, we did not retain any identification of the
respondent on the survey booklet or return envelope. This procedure
prevented us from knowing the identity of the respondent for any of the
surveys returned to us. The use of a separate return postcard allowed us,
nevertheless, to track which respondents did and did not mail back survey
responses so that we could follow up with those who did not respond.
Pretest respondents told us these measures would encourage inspectors to
return the survey forms.

To increase the response to our survey, we mailed a prenotification letter
to respondents 1 week before we mailed the survey on January 31, 1997.
We also used three mailings after the survey mailing: (1) a reminder
postcard 1 week after the survey, (2) a reminder letter to nonrespondents
18 days after the survey, and (3) a replacement survey for recipients not
yet responding mailed 4 weeks after the survey. We received the last
survey included in our analysis on March 20, 1997.

We received survey responses from 247 inspectors for a response rate of
90 percent. Of those responding, 202 inspectors were assigned to oversee
repair stations that did work for air carriers during fiscal year 1996. Only
these respondents actually filled out our survey and are included in our
survey results in this report. Table II.3 shows a summary of the survey
returns.




Page 79                                          GAO/RCED-98-21 Aviation Safety
                                         Appendix II
                                         Survey Methodology




Table II.3: Summary of Returns to Mail
Surveys                                                                                                   Number of FAA inspectors
                                         Population size                                                                         601
                                         Total sample size                                                                       275
                                             Surveys returneda                                                                   247
                                             Eligible                                                                            202
                                             Not eligible                                                                         45
                                             Surveys not returned                                                                 28
                                         Response rate (number returned/number                                                    90%
                                         mailed)
                                         a
                                         Does not include surveys returned that were not filled out.




                                         Page 80                                                       GAO/RCED-98-21 Aviation Safety
Appendix III

Survey Questions and Responses




               Page 81           GAO/RCED-98-21 Aviation Safety
Appendix III
Survey Questions and Responses




Page 82                          GAO/RCED-98-21 Aviation Safety
Appendix III
Survey Questions and Responses




Page 83                          GAO/RCED-98-21 Aviation Safety
Appendix III
Survey Questions and Responses




Page 84                          GAO/RCED-98-21 Aviation Safety
Appendix III
Survey Questions and Responses




Page 85                          GAO/RCED-98-21 Aviation Safety
Appendix III
Survey Questions and Responses




Page 86                          GAO/RCED-98-21 Aviation Safety
Appendix III
Survey Questions and Responses




Page 87                          GAO/RCED-98-21 Aviation Safety
Appendix III
Survey Questions and Responses




Page 88                          GAO/RCED-98-21 Aviation Safety
Appendix III
Survey Questions and Responses




Page 89                          GAO/RCED-98-21 Aviation Safety
Appendix III
Survey Questions and Responses




Page 90                          GAO/RCED-98-21 Aviation Safety
Appendix IV

Major Contributors to This Report


               Robert Jerry Aiken
               Leslie Albin
               John H. Anderson, Jr.
               Steven N. Calvo
               Fran A. Featherston
               Thomas A. Kai
               Christopher M. Jones
               Julia A. Rachiele
               David A. Robinson
               Stanley G. Stenerson
               Mindi G. Weisenbloom




(341498)       Page 91                 GAO/RCED-98-21 Aviation Safety
Ordering Information

The first copy of each GAO report and testimony is free.
Additional copies are $2 each. Orders should be sent to the
following address, accompanied by a check or money order
made out to the Superintendent of Documents, when
necessary. VISA and MasterCard credit cards are accepted, also.
Orders for 100 or more copies to be mailed to a single address
are discounted 25 percent.

Orders by mail:

U.S. General Accounting Office
P.O. Box 37050
Washington, DC 20013

or visit:

Room 1100
700 4th St. NW (corner of 4th and G Sts. NW)
U.S. General Accounting Office
Washington, DC

Orders may also be placed by calling (202) 512-6000
or by using fax number (202) 512-6061, or TDD (202) 512-2537.

Each day, GAO issues a list of newly available reports and
testimony. To receive facsimile copies of the daily list or any
list from the past 30 days, please call (202) 512-6000 using a
touchtone phone. A recorded menu will provide information on
how to obtain these lists.

For information on how to access GAO reports on the INTERNET,
send an e-mail message with "info" in the body to:

info@www.gao.gov

or visit GAO’s World Wide Web Home Page at:

http://www.gao.gov




PRINTED ON    RECYCLED PAPER
United States                       Bulk Rate
General Accounting Office      Postage & Fees Paid
Washington, D.C. 20548-0001           GAO
                                 Permit No. G100
Official Business
Penalty for Private Use $300

Address Correction Requested