oversight

Aviation Safety: Additional FAA Efforts Could Enhance Safety Risk Management

Published by the Government Accountability Office on 2012-09-12.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                 United States Government Accountability Office

GAO              Report to Congressional Committees




September 2012
                 AVIATION SAFETY

                 Additional FAA
                 Efforts Could
                 Enhance Safety Risk
                 Management




GAO-12-898
                                                September 2012

                                                AVIATION SAFETY
                                                Additional FAA Efforts Could Enhance Safety Risk
                                                Management
Highlights of GAO-12-898, a report to
congressional committees.




Why GAO Did This Study                          What GAO Found
The nation’s aviation system is one of          The Federal Aviation Administration (FAA) and its business lines and offices are
the safest in the world, but with air           in different stages of their implementation of Safety Management Systems
travel projected to increase over the           (SMS). FAA finalized its agency-wide implementation plan in April 2012, and the
next 20 years, efforts to ensure the            Air Traffic Organization (ATO) has completed its SMS implementation, but other
continued safety of aviation are                FAA SMS efforts are in the early stages. FAA business lines, such as the
increasingly important. The FAA is              Aviation Safety Organization (AVS) and the Office of Airports (ARP), have SMS
seeking to further enhance safety by            guidance and plans largely in place and have begun to integrate related practices
shifting to a data-driven, risk-based           into their operations, but many implementation tasks remain incomplete, and
safety oversight approach—referred to
                                                officials and experts project that full SMS implementation could take many years.
as SMS. SMS implementation is
required for FAA and several of its             There are a number of key practices that can help agencies plan for and
business lines and the agency is taking         efficiently implement new projects, including large scale transformations such as
steps to require industry                       FAA’s SMS implementation, and FAA has many in place. For example, FAA has
implementation.                                 support from top leadership and a clear project mission. However, FAA has only
As requested, this report addresses             partially addressed other key practices such as developing a project plan to track
(1) the status of FAA’s implementation          SMS implementation, and FAA has not addressed performance-related practices
of SMS, (2) the extent to which FAA’s           such as establishing SMS performance measures or links between employees’
SMS efforts have been consistent with           performance standards and SMS.
key practices for successful planning
and implementation of a new program,            Alignment of FAA’s SMS Implementation with Key Practices
and (3) challenges FAA faces in
implementing SMS. To address these
issues, GAO reviewed FAA SMS
documents, compared FAA efforts to
key practices, and interviewed agency
and industry officials.

What GAO Recommends
GAO recommends that FAA develop
systems to: track SMS implementation,
evaluate employee performance as it
relates to SMS, and assess whether
SMS meets its goals and objectives;
conduct a workforce analysis for SMS;
and consider strategies to address
airports’ data concerns. The
Department of Transportation agreed             Several challenges remain that may affect FAA’s ability to effectively implement
to consider the recommendations and             SMS. FAA is taking steps to address some challenges and stakeholder
provided clarifying information about           concerns, but challenges related to data sharing and data quality; capacity to
SMS, which GAO incorporated.                    conduct SMS-based analyses and oversight; and standardization of policies and
                                                procedures could negatively affect FAA’s efforts to implement SMS in a timely
                                                and efficient manner. Further, FAA officials stated that SMS implementation will
                                                require some skills that agency employees do not have, but FAA has not yet
                                                assessed the skills of its workforce to identify specific gaps in employee
                                                expertise. In addition, while existing federal law protects any data collected for
View GAO-12-898. For more information,          SMS, any data airports collect could be subject to state-specific Freedom of
contact Gerald L. Dillingham, Ph.D., at (202)
512-2834 or dillinghamg@gao.gov.
                                                Information Act laws, a gap that could create a disincentive for airports to fully
                                                participate in SMS implementation.
                                                                                           United States Government Accountability Office
Contents


Letter                                                                                   1
              Background                                                                 4
              FAA and Its Business Lines Are at Different Stages of SMS
                Implementation                                                           8
              FAA’s SMS Approach Is Consistent with Many but Not All Key
                Practices for Successful Project Implementation                         15
              Addressing Key Implementation Practices and Other Challenges
                Could Enhance SMS Effectiveness                                         24
              Conclusions                                                               32
              Recommendations for Executive Action                                      33
              Agency Comments                                                           33

Appendix I    Objectives, Scope, and Methodology                                        35



Appendix II   GAO Contact and Staff Acknowledgments                                     38



Table
              Table 1: Airport Interviews                                               37


Figures
              Figure 1: The Four Components of Safety Management Systems                 5
              Figure 2: Organizational Structure to Coordinate SMS
                       Implementation across FAA Business Lines and Offices              7
              Figure 3: FAA’s SMS Guidance Links to International and
                       Government-wide SMS Requirements and Guidance                     9
              Figure 4: Timeline of Key SMS Implementation Activities across
                       FAA’s Business Lines and Offices, 2007 to 2013                   11
              Figure 5: Proposed and Potential Future Applicability of Federal
                       Aviation Regulation Part 5 to the Aviation Industry              13
              Figure 6: Alignment of FAA’s SMS Implementation with GAO-
                       Identified Key Practices for Successful Organizational
                       Transformations                                                  16




              Page i                                   GAO-12-898 Safety Management Systems
Abbreviations
AIR         Aircraft Certification Service
AFS         Flight Standards Service
ANG         Office of NextGen
ARP         Office of Airports
AST         Office of Commercial Space Transportation
ATO         Air Traffic Organization
AVP         Office of Accident Investigation and Prevention
AVS         Aviation Safety Organization
CFR         Code of Federal Regulations
FAA         Federal Aviation Administration
FOIA        Freedom of Information Act
ICAO        International Civil Aviation Organization
JPDO        Joint Planning and Development Office
NTSB        National Transportation Safety Board
SMS         Safety Management System
SRM         Safety Risk Management



This is a work of the U.S. government and is not subject to copyright protection in the
United States. The published product may be reproduced and distributed in its entirety
without further permission from GAO. However, because this work may contain
copyrighted images or other material, permission from the copyright holder may be
necessary if you wish to reproduce this material separately.




Page ii                                           GAO-12-898 Safety Management Systems
United States Government Accountability Office
Washington, DC 20548




                                   September 12, 2012

                                   The Honorable John D. Rockefeller, IV
                                   Chairman
                                   The Honorable Kay Bailey Hutchison
                                   Ranking Member
                                   Committee on Commerce, Science, and Transportation
                                   United States Senate

                                   The Honorable John L. Mica
                                   Chairman
                                   The Honorable Nick J. Rahall, II
                                   Ranking Member
                                   Committee on Transportation and Infrastructure
                                   House of Representatives

                                   The Honorable Thomas E. Petri
                                   Chairman
                                   The Honorable Jerry F. Costello
                                   Ranking Member
                                   Subcommittee on Aviation
                                   Committee on Transportation and Infrastructure
                                   House of Representatives

                                   The U.S. airspace system is one of the safest in the world, with no main
                                   line air carrier passenger fatalities in the U.S. in more than 10 years. 1 This
                                   record reflects the efforts of the Federal Aviation Administration (FAA),
                                   airlines, airports, manufacturers, the National Transportation Safety Board
                                   (NTSB), and others to continually improve aviation safety. However, with
                                   air travel projected to increase over the next 20 years, efforts to ensure
                                   the continued safety of aviation are increasingly important. FAA is
                                   attempting to further enhance aviation safety, in part, by shifting to a data-
                                   driven, risk-based safety oversight approach. This approach is becoming
                                   the standard throughout the global aviation industry and is recognized by
                                   aviation leaders such as the International Civil Aviation Organization



                                   1
                                    Main line air carriers are commercial airlines that use jets with over 90 seats, as
                                   compared to regional carriers, which use smaller piston, turboprop, and regional jet
                                   aircraft with up to and including 90 seats.




                                   Page 1                                            GAO-12-898 Safety Management Systems
(ICAO), the Joint Planning and Development Office (JPDO), and others
as the next step in the evolution of safety. 2

FAA is overseeing implementation of this new approach—called a safety
management system (SMS) approach—both within FAA and throughout
the U.S. aviation industry and is coordinating these efforts with the
international aviation community. Safety management systems represent
a proactive approach to safety and are intended to continually monitor all
aspects of aviation operations and collect appropriate data to identify
emerging safety problems before they result in death, injury, or significant
property damage. Under SMS, FAA will use the aviation safety data it
collects to identify conditions that could lead to aviation accidents or
incidents and to address such conditions through changes in the FAA’s
organization, processes, management, and culture. SMS adoption and
implementation is one of the biggest cultural and procedural
transformations in FAA history and will likely involve years of continuous
effort on the part of agency and industry officials.

You asked us to assess FAA’s implementation of SMS. To do so, we
addressed the following questions:

    (1) What is the status of FAA’s implementation of SMS?

    (2) To what extent have FAA’s SMS efforts been consistent with key
        practices for successful planning and implementation of a new
        program?

    (3) What challenges does FAA face in implementing SMS?

To determine the status of FAA’s implementation of SMS, we reviewed
FAA’s SMS orders and pilot project guidance, implementation plans, and
Notices of Proposed Rulemaking for Part 121 air carriers and Part 139




2
  ICAO is an agency of the United Nations that promotes the safe and orderly development
of international civil aviation worldwide. ICAO sets standards and regulations necessary
for aviation safety, security, and efficiency and serves as a forum for collaboration among
its 191 member states. JPDO was established by the U.S. Congress in 2003 to plan and
coordinate research and development for the Next Generation Air Transportation System
(NextGen). Pub. L. No. 108-176, §709. JPDO manages partnerships that include agencies
such as the Departments of Transportation, Defense, and Homeland Security.




Page 2                                            GAO-12-898 Safety Management Systems
airports. 3 We interviewed FAA SMS program managers across FAA
business lines and offices. We also reviewed international and FAA
guidance and SMS and NTSB recommendations to FAA related to SMS.
To assess the extent to which FAA’s efforts have been consistent with
key practices, we reviewed our reports and other literature on successful
project planning and implementation, particularly for large-scale
transformative projects, 4 and identified key practices applicable to FAA’s
SMS implementation. We then assessed FAA’s actions against the key
practices by examining such documents as FAA’s guidance and
implementation plans, and interviewing FAA officials. We determined
whether each key practice was addressed, partially addressed, or not
addressed by using criteria developed from prior GAO reports. For
example, we considered a practice partially addressed if FAA had
partially implemented, taken steps toward, or started but not completed
implementing it (see app. I for a discussion of our assessment). To
identify challenges FAA faces in implementing SMS, we reviewed our
prior work on long-standing FAA challenges, such as those related to
training and data, and interviewed aviation industry experts and FAA
officials. We also reviewed our prior work on performance measurement
and workforce analysis, and NTSB recommendations related to SMS. To
obtain industry views on challenges, we interviewed representatives from
airports and air carriers selected for size and geographic dispersion. We
also interviewed representatives of aviation industry associations and
reviewed written comments submitted by aviation stakeholders on two
FAA-issued Notices of Proposed Rulemaking that would require SMS for
some airports and air carriers. See appendix I for a more detailed
description of our scope and methodology.

We conducted this performance audit from September 2011 to
September 2012 in accordance with generally accepted government


3
 FAA issues operating certificates to air carriers offering scheduled, commercial air carrier
service under 14 CFR Part 121. For the purposes of this report, we will refer to these
carriers as commercial air carriers. FAA issues airport operating certificates to airports that
(1) serve unscheduled air carrier aircraft with more than 30 seats; or (2) serve scheduled
air carrier operations in aircraft with more than 9 seats under 14 CFR Part 139. 1(a)(1)
and (2). For the purposes of this report, we will refer to these as certificated airports.
4
 See GAO, Results-Oriented Cultures: Implementation Steps to Assist Mergers and
Organizational Transformations, GAO-03-669 (Washington, D.C.: July 2, 2003) and GAO,
Motor Carrier Safety: The Federal Motor Carrier Safety Administration Has Developed a
Reasonable Framework for Managing and Testing Its Comprehensive Safety Analysis
2010 Initiative, GAO-08-242R (Washington, D.C.: Dec. 20, 2007).




Page 3                                              GAO-12-898 Safety Management Systems
             auditing standards. Those standards require that we plan and perform the
             audit to obtain sufficient, appropriate evidence to provide a reasonable
             basis for our findings and conclusions based on our audit objectives. We
             believe the evidence obtained provides a reasonable basis for our
             findings and conclusions based on our audit objectives.


             SMS provides a top-down approach to managing safety risk, which FAA
Background   expects will improve aviation safety. SMS is not an additional safety
             program that is distinct from existing activities that accomplish an entity’s
             safety mission, but rather, a process for safety management that
             incorporates systematic procedures, practices, and policies. According to
             FAA, the overarching goal of SMS is to improve safety by helping ensure
             that the outcomes of any management or system activity incorporate
             informed, risk-based decision making. We reported in 2010 that FAA
             officials believe that successfully implementing SMS is critical to meeting
             the challenges of a rapidly changing and expanding aviation system. To
             achieve a higher level of safety in an already very safe system, FAA
             requires a more forward-thinking approach, which SMS provides, by
             addressing cultural and organizational problems that lead to safety
             hazards, identifying system-wide trends in aviation safety, and managing
             emerging hazards before they result in incidents or accidents.

             SMS implementation should bring about a fundamental shift in aviation
             safety oversight. For decades, the aviation industry and federal
             regulators, including FAA, have used data reactively to identify the
             causes of aviation accidents and incidents and take actions to prevent
             their recurrence. While FAA plans to continue to use data to analyze past
             safety events, it is also working to use data proactively to search for risks.
             FAA’s shift to the proactive approach of SMS is important because, as
             accidents have become increasingly rare, less information is available for
             reactive analyses of their causes. As a result, information that can be
             used to help identify accident and incident precursors has become more
             critical for accident prevention. Thus, the open sharing of safety
             information among aviation stakeholders and how FAA’s policies and
             procedures govern the reporting of safety information are essential to the
             success of SMS.

             SMS consists of four key components: (1) safety policy, (2) safety risk
             management, (3) safety assurance, and (4) safety promotion (see fig.1).
             Together, these four components are intended to provide a systematic
             approach to achieving acceptable levels of risk. FAA provides to its
             personnel detailed guidance on the principles underpinning these


             Page 4                                     GAO-12-898 Safety Management Systems
                                     components and the application of these components to aviation
                                     oversight in its official orders and other internal FAA guidance. To the
                                     industry, FAA provides this SMS guidance via advisory circulars and a
                                     dedicated page for the SMS program office on the FAA website.

Figure 1: The Four Components of Safety Management Systems




                                     FAA is undertaking the transition to SMS in coordination with the
                                     international aviation community, working with ICAO to adopt applicable
                                     global standards for safety management. ICAO requires SMS for the
                                     management of safety risk in air operations, maintenance organizations,
                                     air traffic services, and airports as well as certain flight-training operations
                                     and for organizations that design or manufacture aircraft. Further, ICAO
                                     has published safety management requirements for its member countries
                                     that mandate that civil aviation authorities—such as FAA—establish
                                     SMS. 5 ICAO first mandated SMS worldwide for air traffic service
                                     providers, such as air carriers and certified aerodromes, in 2001. ICAO
                                     later specified that member states should mandate SMS implementation
                                     for airports, air carriers, and others by 2009. FAA began SMS
                                     implementation in 2005, but FAA officials informed ICAO that the agency
                                     and industry would not be able to meet the 2009 deadline. ICAO is



                                     5
                                         ICAO refers to SMS as employed by civil aviation authorities as “State Safety Programs.”




                                     Page 5                                              GAO-12-898 Safety Management Systems
allowing FAA to take additional time in its efforts to implement SMS, with
the understanding that implementation is under way and that FAA is in
the midst of a rulemaking to require SMS for commercial air carriers.
ICAO officials stated that the United States is one of the leading
implementers of SMS worldwide and acknowledged that SMS
implementation in the U.S. aviation system may be more complicated
than in other countries because of the size and complexity of the U.S.
aviation industry. ICAO has not specified a date by which FAA is
expected to comply with the requirements to implement SMS in the
aviation system. There have also been actions within the United States to
encourage implementation of SMS. For instance, in 2007, NTSB
recommended that FAA require all commercial air carriers to establish an
SMS 6 and, in 2011, added SMS for all modes of transportation to the
NTSB’s Most Wanted List, identifying SMS as one of the most critical
changes needed to reduce the number of accidents and save lives.

Partially in response to the ICAO requirement, FAA added goals related
to SMS implementation to its 2009-2013 Flight Plan. 7 These are linked to
a requirement to implement SMS in three of FAA’s business lines—the
Air Traffic Organization (ATO), the Aviation Safety Organization (AVS),
and the Office of Airports (ARP)—and a goal to implement SMS policy in
all appropriate FAA organizations, which include the Office of Commercial
Space Transportation (AST) and the Office of NextGen (ANG). FAA is in
the process of implementing SMS within these business lines and offices
as well as in industry through rulemakings to require airports and
commercial air carriers to implement SMS. FAA designated AVS as the
lead for SMS implementation in September 2008. Within AVS, the Office
of Accident Investigation and Prevention’s (AVP) Safety Management and
Research Planning Division coordinates and manages SMS
implementation and operation across the agency, and so AVP serves as
the official SMS lead for the agency.

FAA has also established groups that work across the agency to
coordinate the agency’s implementation of SMS. The FAA SMS
Committee is comprised of managers from each of the four business lines
and one staff office currently implementing SMS, and reports to the FAA
SMS Executive Council, which is composed of Associate and Assistant


6
    NTSB recommendation A-07-010.
7
    Federal Aviation Administration, 2009-2013 Flight Plan is the agency’s strategic plan.




Page 6                                               GAO-12-898 Safety Management Systems
Administrators, their deputies, and other high-level FAA officials from
each business line or office (see fig. 2). Within some of the business
lines, there are offices devoted to specific aviation oversight functions that
are responsible for overseeing detailed implementation of SMS for those
functions. For example, the Flight Standards Service (AFS), a division of
AVS that provides safety oversight of commercial air carriers and others,
is taking steps to require SMS implementation by commercial air carriers
and is also working to integrate SMS into its internal activities. In addition,
the Aircraft Certification Service (AIR), a division of AVS that provides
safety oversight to aviation design and manufacturing firms, is leading
agency efforts to encourage SMS implementation for that industry sector,
while ARP is leading agency efforts to require SMS implementation for
certificated airports.

Figure 2: Organizational Structure to Coordinate SMS Implementation across FAA
Business Lines and Offices




SMS implementation will require changes to many of FAA’s operations.
As the agency and industry implement SMS, shifts will be necessary in
both the skills of FAA and industry staff and the tools that the agency
uses to monitor safety. FAA’s integration of SMS into its business
practices will also affect how the agency provides air navigation services
and oversees the aviation industry. Historically, FAA oversight of airlines,
airports, and other regulated entities has involved oversight of such things
as operations and maintenance. FAA will continue this oversight, but will
also apply SMS principles to its processes for oversight. The agency will
provide oversight of the safety management systems of service providers



Page 7                                      GAO-12-898 Safety Management Systems
                             such as air carriers and airports to help ensure that they are managing
                             safety within their operations through SMS. For example, AFS currently
                             provides oversight of the operations, maintenance, and safety data of
                             commercial air carriers and others. Once SMS is fully implemented, AFS
                             will continue to provide this oversight and will also conduct oversight of
                             the safety management systems that commercial air carriers and others
                             put in place.


                             ATO completed its implementation of SMS, but FAA and several of its
FAA and Its Business         other business lines and offices are in the early stages of implementation.
Lines Are at Different       Most FAA business lines and offices have guidance and plans for SMS
                             implementation in place and have begun to integrate SMS-related
Stages of SMS                practices into their operations, but many tasks remain and aviation
Implementation               officials and experts with whom we spoke project that full SMS
                             implementation will take many years.


FAA Recently Finalized Its   FAA finalized its agency-wide plan for SMS implementation in April 2012.
Agency-Wide                  The plan provides a road map for SMS implementation across the agency
Implementation Plan, but     and describes the activities that FAA business lines and offices will need
                             to complete by the end of 2015 to integrate SMS into their operations. 8
Full SMS Implementation
                             These activities will lead to outcomes including:
Is Likely to Take Many
Years                            •    revising and standardizing safety policies and safety risk
                                      management methodologies across FAA to ensure SMS
                                      principles are consistently addressed;

                                 •    improving organizational processes so that FAA business lines and
                                      offices can share safety data and information more easily; and

                                 •    coordinating communications to ensure a common understanding
                                      of SMS across the agency.

                             FAA began its agency-wide SMS implementation efforts in 2008, and in
                             September of that year issued a policy for implementation of a common
                             SMS within FAA. Among other things, the policy sets forth management


                             8
                              The year 2015 does not represent an estimated completion date for SMS
                             implementation. The implementation plan describes tasks through 2015, but FAA officials
                             estimate SMS implementation efforts will continue beyond 2015.




                             Page 8                                          GAO-12-898 Safety Management Systems
                                      principles to guide all of FAA in safety management and safety oversight
                                      activities and requires AVS, ARP, and ATO to develop and execute
                                      business line-specific plans for SMS implementation. In late 2008, FAA
                                      formed the agency-wide FAA SMS Committee to coordinate
                                      implementation efforts across FAA business lines and offices. Overall, the
                                      agency has taken a bottom-up approach to implementation, with some
                                      individual business lines and offices beginning implementation prior to
                                      agency-wide efforts. FAA has also taken steps to ensure that its plans for
                                      SMS implementation and policies align with international and
                                      government-wide requirements and technical guidance on SMS
                                      implementation, including ICAO’s Standards and Recommended
                                      Practices, the ICAO Safety Management Manual, and the JPDO SMS
                                      Standards. For instance, officials stated that they consulted international
                                      and government-wide guidance on SMS implementation when drafting
                                      agency implementation plans. (See fig. 3 for more information on
                                      alignment of FAA requirements with international and government-wide
                                      requirements and guidance on SMS.)

Figure 3: FAA’s SMS Guidance Links to International and Government-wide SMS Requirements and Guidance




                                      Although FAA has made progress, completion of SMS implementation
                                      across FAA is likely to take many years. FAA’s agency-wide SMS
                                      implementation plan includes tasks with estimated completion dates



                                      Page 9                                     GAO-12-898 Safety Management Systems
                         through 2015, and some implementation tasks may take even longer to
                         complete. For instance, a project plan that AVS officials developed to
                         track status of AVS SMS implementation tasks contained in its
                         implementation plan includes task completion dates through 2016.
                         According to FAA, the overall SMS implementation effort is an
                         evolutionary process that will not have a specific completion date. The
                         current implementation time frame is consistent with experts’ estimates of
                         how long it may take to implement SMS and with other large-scale
                         organizational transformations. For example, representatives from The
                         MITRE Corporation, which manages a federally funded research center
                         for FAA and assisted FAA in selected SMS implementation efforts, stated
                         that organizational transformations like SMS can take from 6 to 10 years.


ATO Completed SMS        ATO is the only entity among FAA and its business lines to have
Implementation in 2010   completed SMS implementation. ATO issued its internal SMS guidance in
                         March 2007 and finalized both its SMS implementation plan and its
                         updated SMS Manual in 2008. 9 According to ATO officials, ATO
                         completed SMS implementation in March 2010, and the FAA Air Traffic
                         Safety Oversight Service validated that ATO’s implementation of SMS
                         was complete. Officials stated that implementation within ATO was
                         simpler, in part, because it is the only branch of FAA that is considered an
                         aviation service provider and therefore did not have to conduct a
                         rulemaking for external entities as part of its SMS implementation. 10 With
                         the implementation phase complete, ATO is currently in the continuous
                         improvement phase of SMS. This means that ATO will continuously use
                         the SMS-based processes now in place to identify hazards, enact
                         strategies to mitigate the risks associated with those hazards, and assess
                         the extent to which the mitigations are working effectively. In addition,
                         FAA officials stated that ATO is working to improve its SMS operations,
                         will update guidance on SMS, and plans to perform audits of its SMS
                         functions on a regular basis. ATO officials added that they are working to



                         9
                          As part of FAA’s bottom-up approach to SMS implementation, ATO began
                         implementation in 2005, much earlier than other FAA business lines. Officials stated that
                         the agency did not initially intend to implement SMS across the entire agency.
                         10
                            The primary service of ATO is to move air traffic safely and efficiently. The controllers,
                         technicians, engineers, and support personnel employed by ATO provide air navigation
                         services directly to commercial and private aviation stakeholders, as well as the military.
                         Since ATO directly employs those providing the aviation service, it was not necessary for
                         FAA to conduct a rulemaking in order to require ATO to implement SMS.




                         Page 10                                             GAO-12-898 Safety Management Systems
                                        share lessons learned from their implementation efforts with other FAA
                                        business lines and to develop SMS tools and processes that can be
                                        commonly implemented across all FAA business lines.


Most FAA Business Lines                 With the exception of ATO, most FAA business lines and offices are in
and Offices Are in Early                the early stages of implementation, either in terms of integrating SMS into
Stages of Implementation                their internal processes or in terms of their efforts to prepare to provide
                                        oversight for proposed requirements for industry implementation of SMS.
                                        To date, much of the work of the FAA business lines has focused on
                                        efforts to draft implementation policies and guidance, train employees,
                                        and create tools for applying safety analyses and risk-based decision-
                                        making to safety oversight. (See fig. 4 for more information on the status
                                        of key SMS implementation efforts across FAA.)

Figure 4: Timeline of Key SMS Implementation Activities across FAA’s Business Lines and Offices, 2007 to 2013




                                        Page 11                                       GAO-12-898 Safety Management Systems
Aviation Safety Organization   AVS began its SMS implementation efforts in August 2006 and finalized
                               its SMS implementation plan in January 2012, which was then
                               incorporated into FAA’s overall plan for SMS implementation. Since 2006,
                               AVS and its seven services and offices have issued orders and other
                               guidance on SMS implementation; developed SMS training courses;
                               conducted voluntary pilot projects and rulemaking efforts on SMS
                               implementation for industry; and worked to begin integrating elements of
                               SMS into their operations. For example, AIR officials, who provide
                               oversight of aviation design and manufacturing firms, have developed a
                               central database that provides standard criteria for analyzing service data
                               in a risk-based manner. This should allow AIR inspectors and engineers
                               to rate the risk of potential safety issues and prioritize oversight to high
                               risk issues.

                               Some services and offices within AVS are in the midst of efforts to require
                               SMS for industry and are also operating voluntary pilot programs to
                               promote SMS implementation within industry. A final rule to require SMS
                               for commercial air carriers is expected to be issued in September 2012. 11
                               In 2007, AFS launched a pilot program to encourage voluntary
                               implementation of SMS by industry. 12

                               According to FAA officials, as part of its rulemaking efforts for commercial
                               air carriers, FAA and AVS are developing a new part in the Code of
                               Federal Regulations (CFR)—Part 5—that will describe SMS
                               implementation requirements for Part 121 certificate holders. In the future,
                               FAA may conduct rulemakings to require additional sectors of the aviation
                               industry to meet Part 5 requirements (see fig. 5). AVS officials stated that




                               11
                                  The Airline Safety and Federal Aviation Administration Extension Act of 2010 requires
                               FAA to issue a final rule for Part 121 air carriers by July 30, 2012. Pub. L. No. 111-216,
                               §215(c)(2), 124 Stat. 2348, 2366 (2010). According to FAA officials, the rule will be
                               applied to Part 121 operators as well as operators with dual certificates that conduct
                               operations under both 14 CFR Parts 121 and 135. The proposed rule will not apply to
                               operations conducted solely under 14 CFR Part 135. FAA officials stated that the
                               proposed rule is currently being reviewed by the Office of the Secretary of Transportation
                               and the Office of Management and Budget prior to final issuance.
                               12
                                  The AFS pilot project included Part 121 air carriers, Part 135 operators, and Part 145
                               repair stations. FAA regulations set forth certification procedures for aviation-related
                               products and parts under Federal Aviation Regulation Part 21, commuter and on-demand
                               aviation operations under Federal Aviation Regulation Part 135, and repair stations under
                               Federal Aviation Regulation Part 145.




                               Page 12                                           GAO-12-898 Safety Management Systems
efforts to establish SMS requirements more broadly across the aviation
industry will likely take many years.

Figure 5: Proposed and Potential Future Applicability of Federal Aviation
Regulation Part 5 to the Aviation Industry




Though FAA has not yet required SMS for air carriers or other parts of
industry, FAA has acted to encourage SMS implementation by industry
through voluntary pilot projects, and some aviation stakeholders have
chosen to implement SMS in advance of any federal requirement. Some
sectors of the aviation industry are farther along in their implementation of
SMS than others. For instance, FAA officials stated that a large majority
of commercial air carriers are in the process of implementing SMS. 13 As
of June 2012, over 90 percent of commercial air carriers operating under
Part 121 were participating in the AFS pilot program, which provides air
carriers with direct implementation support from FAA officials under a
more relaxed implementation time frame than is anticipated under an
eventual implementation regulation. Of these air carriers, three have
reached the final stage of SMS implementation. 14 However, most small air
carriers have not yet begun implementing SMS. In contrast to AFS, AIR is
at an earlier stage in its efforts to require SMS for the approximately


13
 Part 135 operators also hold air carrier certificates, but according to FAA officials, a
smaller proportion of these operators are participating in the AFS pilot project.
14
  FAA does not certify SMS implementation by commercial air carriers participating in the
pilot project as complete. However, FAA has laid out four phases of SMS implementation
for pilot project participants, and FAA officials meet with participants to verify whether or
not they have completed the steps included in each phase.




Page 13                                             GAO-12-898 Safety Management Systems
                     3,000 design and manufacturing firms it oversees. AIR began a voluntary
                     pilot project for SMS implementation by design and manufacturing firms in
                     2011 and has 11 pilot project participants. AIR officials stated that they
                     are in the process of launching a second aviation rulemaking committee
                     to continue to explore options to require SMS for design and
                     manufacturing firms. Officials also noted that AFS and AIR are working
                     together to share lessons learned and assist one another in their
                     implementation efforts.

Office of Airports   ARP is in the early stages of working to integrate SMS principles into its
                     oversight of airports, and recently took steps to reduce the scope of that
                     oversight. ARP initially planned to apply SMS-based oversight to all
                     certificated airports. Officials stated that ARP is currently limiting its SMS-
                     based oversight to large hub airports because of budget constraints and
                     will reassess its capacity to expand oversight to smaller airports in 2013.
                     ARP began its SMS implementation in 2010 and issued an internal order
                     to provide a basis for the integration of SMS into its operations later that
                     year. The office finalized its SMS implementation plan in September 2011
                     and has begun to make changes to its oversight. For instance, In June
                     2011, ARP began to apply SMS-based oversight to construction projects
                     at the 29 large hub airports in the United States. 15 Under this new
                     oversight framework, ARP staff assess proposed airport construction
                     projects using risk-based SMS principles, and airports need to
                     incorporate strategies to mitigate identified risks into their construction
                     plans prior to receiving ARP’s approval for the project. Like AVS, ARP is
                     also in the midst of a rulemaking to require SMS for all certificated
                     airports and has completed three voluntary SMS pilot projects for airports
                     from 2008 to 2011. 16 Thirty-one airports participated in at least one of
                     ARP’s SMS pilot projects. ARP is using information gathered through the
                     pilot projects to inform a planned advisory circular that will provide
                     additional guidance to airports on SMS implementation. The pilot projects
                     also allowed airports to share their SMS implementation practices with
                     other airports. The final rule to require SMS for Part 139 certificated



                     15
                      Large hub airports are those which enplane at least 1 percent of U.S. passenger
                     enplanements system-wide. See 49 U.S.C. § 47102 (10).
                     16
                       The first pilot project focused on airport creation of SMS implementation plans; the
                     second pilot project focused on SMS implementation for smaller airports, and the third
                     pilot project focused on SMS implementation at airports that participated in at least one of
                     the first two pilot projects.




                     Page 14                                            GAO-12-898 Safety Management Systems
                              airports is expected to be issued in April 2013 and, if implemented as
                              proposed, would require over 500 airports to implement SMS. 17

Offices of Commercial Space   Other FAA business lines are in varying stages of implementation. AST is
Transportation and NextGen    not currently required to implement SMS; however, AST is taking initial
                              steps toward integrating SMS into an existing set of safety management
                              processes. ANG is farther along in its implementation of SMS because of
                              its previous status as a part of ATO. 18 According to officials, ANG is
                              basing its implementation of SMS on policies and processes established
                              during ATO’s implementation of SMS. The officials stated that since ANG
                              will provide the systems and components that will be used by ATO to
                              manage air traffic, it made sense for ANG to develop its SMS based on
                              policies, processes, and systems established by ATO. Officials stated that
                              ANG completed its implementation plan in June 2012 and estimated that
                              ANG’s SMS implementation is about 70 percent complete.


                              There are a number of key practices and implementation steps that can
FAA’s SMS Approach            help agencies successfully plan for and implement new projects, including
Is Consistent with            large scale transformative ones, such as FAA’s implementation of SMS.
                              As we have previously reported, addressing these key practices can help
Many but Not All Key          an agency improve its efficiency, effectiveness, and accountability. 19 FAA
Practices for                 currently has many of these key factors in place, such as established
Successful Project            support from top leadership and a clear project mission; however, it has
                              only partially addressed other key practices, such as providing needed
Implementation                expertise and technology, and has yet to establish SMS performance
                              measures (see fig. 6).




                              17
                                While ARP has limited its SMS-based oversight efforts to large hub airports, the
                              proposed rulemaking would require Part 139 certificated airports of all sizes to integrate
                              SMS into their operations.
                              18
                                 In 2011, FAA reorganized some of its offices and, as part of the reorganization,
                              separated NextGen efforts from ATO.
                              19
                                   GAO-03-669 and GAO-08-242R.




                              Page 15                                            GAO-12-898 Safety Management Systems
Figure 6: Alignment of FAA’s SMS Implementation with GAO-Identified Key Practices for Successful Organizational
Transformations




Many Key Practices Are in               FAA has instituted many key practices that will help it prepare for and
Place                                   implement SMS across its business lines and offices.

                                        •   Top leadership: Top leaders from each FAA business line provide
                                            support for and actively participate in SMS implementation. As
                                            previously mentioned, FAA established the SMS Executive Council, a
                                            group of high-ranking FAA officials that provides executive-level
                                            guidance and conflict resolution for SMS-related issues across the



                                        Page 16                                      GAO-12-898 Safety Management Systems
       agency. In accordance with our key practices, the SMS Executive
       Council has the authority to make resource allocation decisions, but
       also confers decision-making authority where appropriate to the FAA
       SMS Committee.

       For instance, FAA officials told us that the SMS Executive Council
       retains the authority to make final decisions about changes to FAA’s
       implementation plan that affect policies or procedures for multiple
       business lines; the FAA SMS Committee has the authority to make
       decisions that relate to daily concerns that fall within the purview of its
       members. For example, committee members settled a disagreement
       between ATO and airport officials over whether an airport should
       conduct certain components of a safety risk management panel. At
       the time, FAA had not yet issued its safety risk management policy
       clarifying terms and requirements, so the airport and ATO each had
       its own distinct safety risk management definitions and processes.
       Working with ARP and ATO officials, committee members identified a
       compromise in which ATO protocols were followed, but any
       disagreements on terms or procedures were documented. ARP
       officials told us that FAA’s safety risk management policy, issued in
       April 2012, should help prevent this type of disagreement from
       occurring.

•      Clear project mission: FAA’s internal order requiring SMS
       implementation for ARP, ATO, and AVS clearly describes that FAA’s
       mission is to improve aviation safety and that implementing SMS and
       its components supports that mission. Each business line also has its
       own internal order requiring SMS implementation that mirrors this
       mission and goals.

•      Implementation team: AVP’s safety management division and the
       FAA SMS Committee, function jointly as FAA’s dedicated SMS
       implementation team. The team’s structure and actions align with our
       criteria for a strong and stable team 20 because it is composed of
       senior-level program managers from each business line, all of whom




20
     GA0-03-669.




Page 17                                        GAO-12-898 Safety Management Systems
     had received SMS training according to FAA officials. Despite some
     recent departures, its membership has been largely stable. 21

•    Leading practices: FAA shares information across business lines to
     identify lessons learned related to SMS implementation. For example,
     ATO assembled lessons learned from its SMS implementation into a
     presentation for the other business lines, and included tips such as
     encouraging others to implement a training program and monitor
     mitigations. According to FAA’s implementation plan, the agency
     plans to systematize the sharing of lessons learned by creating a
     central repository to collect and communicate safety lessons learned
     among its business lines and offices by September 30, 2013.

•    Troubleshooting: FAA has processes in place to manage SMS
     implementation across FAA, including troubleshooting unexpected
     problems. For example, the FAA SMS Committee meets monthly and
     manages agency-wide SMS implementation and any challenges that
     arise, and regularly briefs the SMS Executive Council, a briefing that
     includes a discussion of any issues or unexpected problems that
     could not be resolved at the committee level. For instance, when the
     Air Traffic Manager at an airport disagreed with airport officials
     regarding how to handle a potential safety issue with planes that were
     taking off on runways that were temporarily closed, the FAA SMS
     Committee elevated the issue to the SMS Executive Council, which
     resolved it. As we have previously reported, instituting practices like
     these can help an agency become more results-oriented, customer-
     focused, and collaborative.

Although FAA is still in the process of finalizing new requirements for
airports and air carriers to implement SMS, it has already taken some
steps to institute key practices for those efforts. For example, FAA
officials stated that the agency has taken steps to identify leading
practices during pilot projects by soliciting information from participating



21
  Our review of implementation practices focused on FAA’s internal implementation
efforts; however, FAA does not currently have a dedicated implementation team to
coordinate efforts to require industry SMS across FAA business lines. FAA officials stated
that the FAA SMS Committee is the appropriate group to handle industry SMS
implementation issues that cross FAA organizations, but committee officials told us that
SMS implementation in industry is implemented separately by AFS and ARP. In addition,
FAA reported that several FAA offices, including the Rulemaking Management Council
and the Office of Rulemaking, manage all of FAA’s rulemaking efforts.




Page 18                                           GAO-12-898 Safety Management Systems
                           airports and air carriers, and FAA officials told us they plan to incorporate
                           these lessons learned into rulemaking and guidance. ARP officials
                           reported that they encouraged pilot project participants to share lessons
                           learned directly with one another through studies and roundtable
                           discussions, and incorporated some of the lessons learned into FAA
                           advisory circulars. FAA has also made efforts to troubleshoot and
                           manage unexpected problems with pilot participants through meetings,
                           calls, and conferences with airport and air carrier officials to understand
                           their experiences. For example, AFS officials reported that they helped
                           officials from air carriers to understand when certain safety risk
                           management documentation and processes are necessary, and how they
                           could be adapted for a variety of changes made to carrier operations,
                           including smaller day-to-day changes. However, despite this assistance,
                           officials from some airports that participated in pilot projects reported that
                           they could have benefited from additional assistance from ARP, such as
                           clarification on the safety risk management component of SMS. In
                           addition, an official at one airport told us that he would have liked FAA to
                           facilitate conversations between airports of similar size to help them share
                           lessons learned.


FAA Efforts to Create a    Other steps FAA has taken in its SMS implementation efforts partially
Project Plan and Consult   align with key practices for implementing a new program.
and Assist Stakeholders
                           •   Project plan: Currently, the agency-wide project plan for SMS
Partially Align with Key       implementation is a single page of high-level milestones, which AVP
Practices                      officials monitor and report on to the SMS Executive Council. Also,
                               AVS has a detailed project plan for its own SMS implementation and
                               elements of agency-wide implementation for which AVP, as the
                               agency SMS lead, has responsibility. Officials stated that they have
                               plans to develop a system to monitor and track the progress of
                               activities needed to implement SMS, but FAA does not currently have
                               a system for tracking agency-wide SMS implementation, a key
                               practice particularly important during the initial planning phase of
                               project implementation. 22 However, given the scope and complexity of
                               SMS, a detailed, agency-wide project plan could help FAA track and
                               monitor the interim steps of SMS implementation across the agency.


                           22
                              Our review of implementation practices focused on FAA’s internal SMS implementation
                           efforts. However, FAA also does not currently have an agency-wide project plan to track
                           its efforts to oversee industry implementation.




                           Page 19                                          GAO-12-898 Safety Management Systems
       Without such a plan, it may be more difficult for FAA to identify
       problems or deviations from planned activities, putting both the
       timeliness and effectiveness of SMS implementation at risk.

•      Consulting with stakeholders: FAA has made efforts to consult with
       employees and stakeholders regarding its SMS implementation, but it
       has not yet developed a communications plan. Agencies should
       involve employees in planning, and incorporate employee feedback
       into new policies and procedures. 23 FAA involved its business line
       program managers and some of the managers’ staff by assigning
       them responsibility for the day-to-day tasks related to implementing
       SMS across the agency. FAA has involved other employees by
       soliciting questions and comments on SMS in town hall meetings and
       the online DOT site called “IdeaHub,” and by offering SMS training
       through each business line. ATO, ARP, and AVS all offer introductory
       SMS courses for their staff as well as additional related courses, such
       as an SMS course specifically for managers and ATO’s safety risk
       management course.

FAA has been working to implement SMS for the last 4 years, but the
agency does not have a communications plan or strategy for ensuring
that the SMS messages communicated to staff are consistent across the
agency. Instead, FAA relies on a more informal communications structure
in which each program manager staffed to the implementation team
communicates relevant information back to their respective business line.
The implementation team does not communicate any information directly
to employees, which could hinder the team’s ability to ensure consistency
in its message across FAA. ATO officials reported experiencing this
challenge at the beginning of ATO’s SMS implementation, when a lack of
clear requirements for communicating SMS information resulted in
variation in staff’s understanding of guidance. We have previously
reported that a communication plan or strategy can ensure consistency of
message, provide information to meet the specific needs of employees,
encourage two-way communication, and build trust. FAA plans to begin
working on a communications plan in September 2012, and is scheduled
to issue the plan at the end of February 2013. FAA officials also said they
are in the process of developing an internal SMS website for employees
to share information and ideas, which could enhance SMS
communications. However, until the communications plan is developed


23
     GAO-03-669.




Page 20                                      GAO-12-898 Safety Management Systems
and implemented, FAA’s employees may not receive timely or consistent
information on SMS or be as invested in its implementation as they might
otherwise be.

FAA’s approach to overseeing industry SMS implementation allowed for
additional two-way communication. 24 For example, FAA solicited views on
SMS implementation from airport and air carrier officials through voluntary
pilot projects described previously, and learned more about industry
perspectives through the formal rulemaking process—whereby an agency
issues a Notice of Proposed Rulemaking and is required to notify the
public and give them an opportunity to submit comments.

•    Providing technology and expertise: FAA has provided some SMS
     training and tools to its employees; however, it has not yet provided
     other tools important for SMS implementation. FAA officials reported
     that each business line has provided SMS training to staff. In addition,
     FAA recently developed a standardized Safety Risk Management
     (SRM) policy, which will assist employees across FAA by
     standardizing SRM terminology and clarifying confusion on the
     conduct of SRM across the agency. FAA plans to create a simple
     version of an agency-wide hazard-tracking system in the next 3 to 6
     months, but does not have plans to create a more complex system
     until August 2015, according to FAA’s SMS implementation plan. The
     simple version will draw from hazard-tracking systems already in
     place in some business lines, and summarize information from them
     to highlight broader hazards such as those that would affect multiple
     business lines. For instance, FAA officials stated that if ATO wanted
     to make a change to its operations at a particular airport, then ATO
     would be responsible for identifying associated hazards, risks, and
     risk mitigations and would also be responsible for assuming
     responsibility for the risk. However, if ATO determined that the airport
     was better equipped to mitigate the identified risks, then the airport
     and ARP would become more involved in designing risk mitigations
     and overseeing their implementation. FAA’s efforts to provide tools to
     help in SMS implementation are affected by differences in how data
     are collected and assessed across the agency. For example, these
     differences have held back agency efforts to model how changes to
     the national airspace system, such as increases to air travel, can



24
  Our review of implementation practices focused on FAA’s internal SMS implementation
efforts, so we do not comment on FAA’s overall communication strategy.




Page 21                                         GAO-12-898 Safety Management Systems
                                affect safety. We have previously reported on and made
                                recommendations related to FAA’s data challenges, 25 and also
                                discuss them later in this report. These data challenges mean that
                                FAA is not always able to perform comparisons across databases, a
                                challenge that that limits the usefulness of the data in identifying
                                possibly dangerous hazards. Identifying, monitoring, and mitigating
                                hazards is a key tenet of SMS, and without the proper technologies
                                and tools, FAA may not be able to do this as effectively.


FAA Has Yet to Integrate   FAA’s efforts do not align with two key practices for implementing a new
SMS into Employee          program.
Performance Plans and
                           •    Integrating SMS into employee performance plans: FAA does not
Establish Performance           consistently evaluate employees’ performance on SMS-related tasks.
Measures                        We have previously reported that effective performance management
                                systems create a clear linkage between individual performance and
                                organizational success, and include aligning individual performance
                                expectations with organizational goals. 26 FAA’s organizational mission
                                and goal, and that of SMS, is to improve safety, yet FAA officials told
                                us that the agency does not require employee performance plans to
                                include SMS-related tasks. Although officials reported that some
                                employees’ performance plans explicitly include SMS items, such as
                                providing SMS training or developing SMS policy, it is left to the
                                discretion of each business line whether SMS items are included. FAA
                                officials told us that SMS principles and methodologies will be
                                included in the performance plans of employees involved in writing
                                SMS policy and revising SMS processes, and will be incorporated into
                                the tasks of others once SMS implementation reaches those
                                individuals. However, currently, none of the business lines require
                                this. As such, FAA does not have a system for assessing the extent to
                                which staff are effectively supporting SMS, and FAA may not be able
                                to determine if staff are completing tasks and responsibilities
                                necessary for the successful implementation of SMS.



                           25
                            GAO, Aviation Safety: Improved Data Quality and Analysis Capabilities Are Needed as
                           FAA Plans a Risk-Based Approach to Safety Oversight, GAO-10-414 (Washington D.C.:
                           May 6, 2010).
                           26
                            GAO, Results-Oriented Culture: Creating a Clear Linkage Between Individual
                           Performance and Organizational Success, GAO-03-488 (Washington D.C.:
                           Mar. 14, 2003).




                           Page 22                                        GAO-12-898 Safety Management Systems
•    Measuring performance: FAA does not have performance measures
     in place to assess whether the SMS goals of improving safety are
     being achieved. FAA has broader safety-related performance
     measures, such as tracking rates of runway incursions and losses of
     separation, 27 but SMS-related performance measures could address
     intermediate safety issues, such as precursors to incursions or
     incidents. Such measures could help FAA track progress toward its
     broader safety measures. 28 FAA officials told us that AVS is a
     member of the Safety Management International Collaboration Group,
     a group formed in 2009 to address safety management-related topics,
     including performance measures. Most recently, FAA formed an
     agency-wide working group to study performance metrics for SMS
     implementation, and FAA’s implementation plan states that such
     metrics will be finalized in October 2014. However, FAA officials we
     spoke with acknowledged that they are at the very beginning phase of
     this process and, although already in the process of implementing
     SMS, have not yet identified metrics to measure safety results under
     an SMS system. 29 We have previously reported that performance
     information is critical for achieving results and maximizing the return
     on federal funds. 30 Performance measures should help FAA identify
     the extent to which SMS implementation will contribute to increased
     aviation safety—FAA’s stated overall goal for SMS—as well as help
     identify what changes could be made to improve SMS performance
     over time.


27
  Runway incursions are the unauthorized presence of an airplane, vehicle, or person on
the runway. Losses of separation involve a loss of the minimum required distance
between aircraft or as individual aircraft fly too close to terrain or obstructions.
28
   We previously reported that such measures could help the Transportation Security
Administration track its progress in securing transit and passenger rail systems. For more
information, see GAO, Surface Transportation Security: TSA Has Taken Actions to
Manage Risk, Improve Coordination, and Measure Performance, but Additional Actions
Would Enhance its Efforts, GAO-10-650T (Washington, D.C.: Apr. 21, 2010).
29
  We will discuss the challenges of developing SMS performance measures later in this
report.
30
  The Government Performance and Results Act of 1993 instituted a government-wide
requirement that agencies set goals and report annually on performance, and encouraged
federal agencies to conduct in-depth program evaluations to assess their program’s
impact or learn how to improve results. Pub. L. No. 103-62. Congress subsequently
passed the GPRA Modernization Act of 2010 (GPRAMA) which establishes a new
framework aimed at taking a more crosscutting and integrated approach to focusing on
results and improving government performance. Pub. L. No. 111-352, 124 Stat. 3866
(2011).




Page 23                                           GAO-12-898 Safety Management Systems
                      As previously mentioned, FAA has taken steps to address many of the
Addressing Key        practices associated with planning and implementing a new program.
Implementation        However, we identified six challenges that could negatively affect FAA’s
                      efforts to implement SMS in a timely and efficient manner:
Practices and Other
Challenges Could
                          1) the large scope and complexity of SMS implementation,
Enhance SMS
Effectiveness             2) resource and capacity constraints,

                          3) standardization of policies and processes,

                          4) data sharing and protection,

                          5) data quality and usefulness, and

                          6) development of performance measures to evaluate SMS
                             effectiveness.


Large Scope and       Implementing SMS is one of several major initiatives FAA has under way,
Complexity of SMS     and its sheer scope and complexity could affect, or be affected by,
                      concurrent FAA efforts such as NextGen or Unmanned Aircraft
                      Systems. 31 SMS requires changes in many of FAA’s operations: from the
                      way the agency tracks hazards to the way it oversees industry. SMS will
                      also require a transformation of FAA’s and the aviation industry’s safety
                      culture to one in which information and safety data are shared openly,
                      and errors are addressed through whatever action is necessary to
                      prevent them from happening in the future. FAA is making efforts to move
                      toward this new approach to safety, for instance by using data-sharing
                      systems that are protected from public disclosure to encourage voluntary
                      reporting of safety issues and enable more robust analysis of safety data
                      among FAA and air carriers. Moreover, as previously stated, each of
                      FAA’s business lines has its own role in implementing SMS that must be
                      coordinated across the agency. This is particularly challenging because
                      the business lines are at different stages of implementation and,
                      according to FAA officials, have historically operated independently.



                      31
                         Unmanned Aircraft Systems are remotely operated aircraft and vehicles that come in a
                      variety of shapes and sizes, and serve diverse purposes. GAO is currently conducting a
                      study on Unmanned Aircraft Systems.




                      Page 24                                          GAO-12-898 Safety Management Systems
                         The scope and complexity of SMS implementation may also be a
                         challenge for the aviation industry, and some stakeholders expressed
                         concerns both in interviews and in official comments on FAA’s Notices of
                         Proposed Rulemaking that eventual FAA requirements to implement SMS
                         need to allow for variation in airport and air carrier operations. For
                         example, officials from some smaller airlines and airports noted that SMS
                         implementation could require additional resources, such as staff and
                         software, which may not be readily available. In addition, officials from
                         some airports and air carriers were concerned that FAA’s final
                         requirements would be too prescriptive to allow entities to implement an
                         SMS program that best fit their organizational type, management
                         practices, and resources. Most stakeholders and experts we interviewed
                         stated that FAA could design SMS requirements for airports and airlines
                         that are scalable and flexible to accommodate this variation, which would
                         address these concerns. For instance, airport officials from smaller
                         airports told us that staff size limits their ability to assign a dedicated SMS
                         employee or safety director, while some officials at larger airports said
                         they were able to hire a SMS safety director or already had an
                         established safety director in place. Also, FAA’s SMS implementation pilot
                         project for airports found that 35 percent of participants planned to hire
                         additional staff to support SMS and 15 percent were not sure. FAA
                         officials have noted that they understand these scalability concerns, and
                         are taking them into consideration as they develop final SMS rules for
                         industry.


Resources and Capacity   SMS implementation across FAA will require some skills that agency
                         employees currently do not have, yet FAA has not formally assessed the
                         skills of its workforce to identify any gaps in the expertise required to
                         implement SMS or determined how to fill those gaps. In addition, FAA
                         officials stated that existing staff may not be able to be trained to fill SMS
                         implementation needs in all cases. For instance, FAA officials noted that
                         SMS implementation will require some engineers and other technical
                         employees to understand certain terminologies and have certain
                         knowledge, skills, and abilities, such as an enhanced ability to perform
                         complex modeling and analysis of aviation safety data to identify potential
                         safety hazards. AVS officials stated that to implement SMS, additional
                         employees with skills in analyzing data for hazards and associated risks
                         would be needed, along with additional training for existing staff. ARP
                         officials stated that the office might need program analysts with specific




                         Page 25                                    GAO-12-898 Safety Management Systems
data analysis skills to implement SMS. 32 ARP officials stated that they do
not expect to receive significantly more resources and, as previously
mentioned, have already had to reduce the scope of the office’s SMS-
based oversight because of insufficient staff. Stakeholders and experts
also questioned whether FAA currently has the resources and capacity
needed to fully implement SMS. For example, experts noted that FAA
may not have the requisite engineers and other staff to participate in
safety risk management efforts, or FAA inspectors to oversee individual
airport and air carrier SMS programs.

Despite these concerns, FAA has not yet conducted a strategic workforce
assessment to accurately determine the skills and staffing levels it needs
to manage SMS. Although FAA’s SMS implementation plan recommends
that business lines create such staffing analyses, none have done so. Nor
has FAA conducted an agency-wide workforce assessment for SMS. Our
internal control standards state that agencies should ensure that skill
needs are continually assessed to ensure workforces have the skills
necessary to help the agency meet its goals. 33 We have reported that
strategic workforce planning is an integral part of human capital
management and helps an agency, among other things, determine the
critical skills and competencies that will be needed to achieve current and
future programmatic results, and then develop strategies tailored to
address any gaps identified. 34

A workforce analysis could help FAA determine how to best address its
most critical needs in ways that account for budget limitations, such as
through retraining or shifting staff, rather than hiring additional employees.
Without conducting an agency-wide SMS workforce analysis, FAA cannot
be sure that it has sufficient staff, skills, or competencies to implement
SMS, thus putting its SMS implementation efforts at risk.




32
 FAA officials stated that ARP received approval to hire 26 additional staff in fiscal year
2012. However, most hiring is on hold until enactment of the fiscal year 2013 budget,
when ARP officials will be able to assess the office’s ability to continue to fund the
positions.
33
 GAO, Internal Control: Standards for Internal Control in the Federal Government,
GAO/AIMD-00-21.3.1 (Washington D.C.: Nov. 1999).
34
 GAO, Human Capital: Key Principles for Effective Strategic Workforce Planning,
GAO-04-39 (Washington D.C.: Dec. 11, 2003) and GAO/AIMD-00-21.3.1.




Page 26                                             GAO-12-898 Safety Management Systems
Standardization of SMS   SMS standardization across FAA business lines and offices is central to
Policies and Processes   implementation success, yet developing common systems for distinct
                         FAA business lines and offices has proved challenging. For example,
                         FAA realizes that the agency needs a common hazard-tracking system in
                         order to maximize SMS effectiveness, yet FAA officials and stakeholders
                         stated that it is difficult to develop such a system because each of FAA’s
                         business lines uses different hazard-related terms and definitions, and
                         often different data systems. These differences, in turn, prevent the
                         agency from performing simple comparisons across databases and have
                         delayed advances in using data analysis to proactively identify potential
                         safety hazards. FAA officials stated that the agency has recently taken
                         steps to make its databases interoperable, and also recently issued a
                         standardized policy for the safety risk management component of SMS.
                         Both of these steps may enhance FAA’s hazard-tracking and analysis
                         capabilities. The agency is also working with ICAO to address issues
                         related to standardization, such as adopting a collaborative approach to
                         increase the sharing of safety information internationally.

                         Industry officials are also concerned that FAA inspectors and certificate
                         management offices 35 may have different interpretations of SMS and
                         other regulations. We and others have previously reported that variation
                         in FAA’s interpretation of standards for certification and approval
                         decisions is a long-standing issue. 36 Industry stakeholders we interviewed
                         expressed concerns that a similar result could occur once final rules are
                         issued requiring airports and air carriers to implement SMS, and could
                         lead to airports or air carriers of similar size being held to different
                         standards of SMS implementation. FAA officials acknowledged that this is
                         a challenge for the agency and noted that the agency plans to provide
                         additional training to inspectors related to oversight of SMS. Additionally,
                         based on our 2010 recommendation, recent legislation directs FAA to




                         35
                            FAA Certificate Management Offices are located throughout the U.S. and employ staff
                         who specialize in the certification, surveillance, and inspection of major air carriers and
                         Flight Safety Training Centers.
                         36
                           GAO, Aviation Safety: Certification and Approval Processes Are Generally Viewed as
                         Working Well, but Better Evaluative Information Needed to Improve Efficiency, GAO-11-14
                         (Washington, D.C.: Oct. 7, 2010); Congressional Research Service, U.S. Airline Industry
                         Issues and Role of Congress, 7-5700, RL34467 (July 29, 2008).




                         Page 27                                             GAO-12-898 Safety Management Systems
                   establish an advisory body of government and industry representatives to
                   address the issue of inconsistent interpretation of regulations. 37

                   FAA’s organizational structure for SMS implementation may pose
                   challenges to standardization as well. For example, as previously
                   mentioned, AVP’s safety management division is the lead for SMS, and
                   AVP and the FAA SMS Committee share responsibility for implementing
                   SMS across the agency. Despite AVP’s role as lead for SMS
                   implementation, it does not have any additional authority compared to the
                   other business lines’ committee representatives, something that AVP
                   officials noted can make SMS implementation difficult. This could slow
                   decision-making, particularly around issues that require business lines to
                   come to a single decision, such as how to standardize policies.
                   Nevertheless, FAA officials acknowledged that having to collaborate to
                   implement an agency-wide SMS has improved communication among the
                   business lines. FAA will likely continue to face challenges standardizing
                   its policies and processes as standardization of this scale is not
                   something the agency has previously undertaken, and the need to
                   negotiate solutions across FAA business lines could take time.


Data Sharing and   Airport officials’ concerns about sharing and protecting their safety data
Protection         may reduce SMS effectiveness by limiting the ability of airports and FAA
                   to analyze safety data and identify trends. Although FAA has some data
                   protections in place, such as those established by the FAA Modernization
                   and Reform Act of 2012, which protects data that airports and air carriers
                   submit to FAA for SMS from federal Freedom of Information Act (FOIA)
                   requests, 38 any data airports collect and any data air carriers share with
                   airports could be subject to state-specific FOIA laws. Most certificated
                   U.S. airports are either owned by a state, a subdivision of a state, or a
                   local government body, and thus are subject to state laws, including state
                   FOIA laws. This means that data airports collect and submit to FAA for
                   SMS—such as information on hazards or other safety data—is protected
                   from federal FOIA public disclosure requests, but, according to officials
                   and experts, may be subject to public disclosure under state FOIA laws.
                   Air carriers are not directly subject to state FOIA laws because they are
                   privately owned. Nevertheless, officials and experts stated that these laws


                   37
                        Pub. L. No. 112-95, §313, 126 Stat. 11, 67 (2012).
                   38
                        Pub. L. No. 112-95, §310, 126 Stat. 11, 64 (2012).




                   Page 28                                              GAO-12-898 Safety Management Systems
could affect air carriers because any data they choose to share with
airports could then be subject to state FOIA laws. As a result, air carrier
officials told us they may be less likely to share safety information with
airports. Airport and airline officials’ primary concern is that the public
disclosure of such information could result in negative publicity or expose
them to legal liability in the event of an incident or accident. FAA officials
said that data protection and legal liability are two of the major concerns
throughout the aviation industry that could hinder the implementation of
SMS.

FAA officials told us that they intend to continue to promote and expand
safety information sharing efforts, but that airports could find ways to
structure their SMS implementation so that they realized safety benefits
while limiting the public release of air carrier safety information. In FAA’s
official response to comments on two Notices of Proposed Rulemaking,
FAA stated that airport officials are best situated to understand how to
comply with state laws. Nonetheless, we found consensus among NTSB
and many aviation stakeholders that FAA should seek congressional
action regarding the protection of airport data from state FOIA laws.

Data sharing can also be challenging within FAA. In 2011, we
recommended that FAA improve information sharing among its programs
because not doing so could limit the ability of FAA and others to analyze
safety data and understand safety trends. 39 The Department of
Transportation agreed that it must continue to promote and expand safety
information sharing efforts and safety practices in order to maximize the
effectiveness of safety data mining 40 to analyze trends and prioritize
safety efforts to address hazards before they lead to incidents or
accidents. However, our recommendation remains open. According to
officials, ICAO has also formed the Safety Information Exchange Study
Group to help enhance data protection and identify potential international
solutions.




39
 GAO, Aviation Safety: Enhanced Oversight and Improved Availability of Risk-Based
Data Could Further Improve Safety, GAO-12-24 (Washington, D.C: Oct. 5, 2011).
40
   Safety data mining involves the use of data based applications to look for hidden
patterns in groups of data that can be used to predict future behavior or occurrences that
may lead to an incident or accident.




Page 29                                           GAO-12-898 Safety Management Systems
Data Quality and   Long-standing issues with data quality and usefulness could negatively
Usefulness         affect FAA’s understanding of aspects of the safety of the aviation
                   industry and, consequently, affect SMS’s effectiveness. Obtaining
                   relevant data and understanding how to analyze those data to identify
                   potential hazards are major challenges that FAA will need to overcome. In
                   recent GAO reports, we commented on FAA’s lack of data to effectively
                   assess aviation trends for certain types of events and the safety
                   performance of certain industry sectors. 41 For instance, in April 2012, we
                   reported that for such events as runway excursions (when an aircraft
                   veers off or overruns a runway) and ramp accidents (incidents or injuries
                   that occur off the runway), a shortage of FAA data exists for analysis. 42
                   The Department of Transportation concurred with this and our
                   recommendations, and stated that the agency has taken steps to improve
                   to its data quality and usefulness. For example, the FAA SMS Committee
                   directed a working group to determine what safety data the agency is
                   going to collect and track and to recommend what kind of system will be
                   needed. However, FAA has not yet fully implemented several of our
                   recommendations aimed at improving its capability to use data for
                   aviation safety oversight, or several data-related NTSB recommendations
                   from recent years. For example, we recommended that FAA extend
                   standard quality controls, as appropriate, to the databases that support
                   aviation safety oversight to ensure that the data are as reliable and valid
                   as possible. By not fully addressing these challenges and
                   recommendations, FAA’s ability to comprehensively and accurately
                   assess and manage hazards and risk will be compromised, reducing the
                   ability of SMS to prevent incidents and accidents.


SMS Performance    The aviation community has widely acknowledged that developing SMS
Measures           performance measures is difficult, but without them, FAA will not be able
                   to gauge the direct impact of SMS on aviation safety. Some stakeholders
                   told us about ways in which SMS improved their organization’s
                   operations, and these examples could provide insight into possible SMS
                   performance measures. For instance, some airports and air carriers that
                   participated in FAA’s SMS pilot projects reported that SMS



                   41
                    GAO-12-24 and GAO, Aviation Safety: FAA Is Taking Steps to Improve Data, but
                   Challenges for Managing Safety Risks Remain, GAO-12-660T (Washington, D.C.:
                   Apr. 25, 2012).
                   42
                        GAO-12-660T.




                   Page 30                                       GAO-12-898 Safety Management Systems
implementation improved communication across their organizations,
helped them identify organizational gaps—such as those in internal
auditing and training—and decreased employees’ injuries, aircraft
damages, and insurance costs. Officials from the Flight Safety
Foundation 43 recommended that the extent to which SMS informs
management decision making, such as by redirecting resources or
shifting priorities, may be one way to measure SMS effectiveness. An
FAA official suggested that performance measures could be directed to
specific components of SMS, for instance tracking the number of risks
mitigated as a measure of safety risk management efficacy. We have
previously reported that agencies need to set quantifiable outcome-based
performance measures for significant agency activities, such as SMS, to
demonstrate how they intend to achieve their program goals and measure
the extent to which they have done so. 44 Performance measures allow an
agency to track its progress in achieving intended results, which can be
particularly important in the implementation stage of a new program such
as SMS. In our prior work we recommended that agencies develop
methods to accurately evaluate and measure the progress of
implementation, and develop contingency plans if the agency does not
meet its milestones to complete tasks. 45 FAA has established a working
group to study the issue and participates on two international
performance measures work groups: the Safety Management
International Collaboration Group and the aforementioned Safety
Information Exchange Study Group.




43
 The Flight Safety Foundation is an independent, nonprofit, international organization
engaged in research, auditing, education, advocacy, and publishing to improve aviation
safety.
44
  See GAO, The Results Act: An Evaluator’s Guide to Assessing Agency Annual
Performance Plans, GAO/GGD-10.1.20; (Washington, D.C.: April 1998); VA Health Care:
VA Should Better Monitor Implementation and Impact of Capital Asset Alignment
Decisions, GAO-07-408 (Washington, D.C.: Mar. 21, 2007); NextGen Air Transportation
System: FAA’s Metrics Can Be Used to Report on Status of Individual Programs, But Not
of Overall NextGen Implementation or Outcomes, GAO-10-629 (Washington, D.C. July
27, 2010); Motor Carrier Safety: More Assessment and Transparency Could Enhance
Benefits of New Oversight Program, GAO 11-858 (Washington, D.C.; Sept. 29, 2011).
45
 GAO, FAA Airspace Redesign: An Analysis of the New York/New Jersey/Philadelphia
Project, GAO-08-786 (Washington, D.C.: July 31, 2008).




Page 31                                          GAO-12-898 Safety Management Systems
              FAA is making progress implementing SMS, both within the agency and
Conclusions   for the aviation industry. However, SMS implementation represents a
              significant cultural and procedural shift in how the agency will conduct
              business internally and provide oversight to aviation stakeholders such as
              air carriers and airports, and by all estimates, this transformation will take
              many years to complete. Going forward, if FAA is to attain the full benefits
              of SMS, it will be important for the agency to remain committed to fully
              implementing SMS across its business lines. FAA has taken a number of
              steps that align with practices we identified as important to successful
              project planning and implementation, but has not addressed or has only
              partially addressed other key practices. These practices are important for
              large-scale transformative projects such as SMS, which require a
              dramatic shift in FAA’s approach to safety oversight and management. In
              the absence of these key practices, it may be difficult for FAA to prioritize
              projects or monitor SMS implementation and progress toward improving
              safety.

              Aviation safety is a shared responsibility among FAA, air carriers,
              airports, and others in the aviation industry, and efforts to improve safety
              will require the agency to overcome several challenges. The magnitude of
              SMS’s potential impact on aviation oversight and the complexity of
              implementation are both a benefit and a drawback for FAA, as SMS
              implementation could help ensure the continued safety of the U.S.
              aviation system, but could also affect implementation time frames for
              other large initiatives as the agency works in a resource-limited
              environment. FAA officials believe that SMS implementation will require
              some skills that employees do not currently have; however, FAA has not
              conducted an agency-wide workforce assessment. With agency
              resources and capacity in great demand, it will be important for the
              agency to maximize the efficiency of SMS implementation, both through
              efficient use of its workforce and creation of policies and systems that
              standardize and streamline implementation. In addition, data protection
              concerns from airport officials and others could prevent aviation
              stakeholders from fully embracing SMS implementation, thus hindering its
              effectiveness. Without assurance of protection from state FOIA laws,
              some aviation stakeholder may choose to collect only the bare minimum
              of safety-related data or may choose to limit the extent to which collected
              information is shared among aviation stakeholders. The agency also
              lacks sufficient data to effectively assess aviation trends for some events
              as well as the safety performance of certain industry sectors. The ability
              of FAA to identify safety risks, develop mitigation strategies, and measure
              outcomes is hindered by limited access to complete and meaningful data.



              Page 32                                    GAO-12-898 Safety Management Systems
                      To enhance the effectiveness of efforts to implement SMS and maximize
Recommendations for   the positive impact of SMS implementation on aviation safety, we
Executive Action      recommend that the Secretary of Transportation direct the FAA
                      Administrator to take the following five actions:

                      1. To better evaluate the effectiveness of the agency’s efforts to
                         implement SMS, develop a system to assess whether SMS meets its
                         goals and objectives by identifying and collecting related data on
                         performance measures.

                      2. To align strategic goals with employee efforts, develop a system to
                         evaluate employees’ performance as it relates to SMS.

                      3. To better manage implementation, develop a system to track and
                         report on SMS implementation across business lines.

                      4. To better leverage existing resources and facilitate SMS
                         implementation, conduct a workforce analysis to inventory existing
                         employee skills and abilities and develop strategies for addressing
                         any SMS-related gaps identified.

                      5. To maximize the positive impact of SMS implementation on aviation
                         safety, consider strategies to address airports’ concerns that may
                         negatively affect data collection and data sharing, including asking
                         Congress to provide additional protections for SMS data collected by
                         public entities.


                      We provided the Department of Transportation and NTSB with a draft of
Agency Comments       this report for review and comment. DOT and NTSB officials provided
                      technical comments, which we incorporated as appropriate and DOT
                      agreed to consider the recommendations. In addition, DOT officials stated
                      there is a need for FAA to have a common hazard-tracking system. FAA
                      has taken initial steps towards standardization by publishing FAA Order
                      8040.4A, Safety Risk Management Policy, which identifies terms and
                      definitions used for safety risk management. DOT also reinforced its
                      dedication to the success of SMS and noted its continued efforts to
                      improve its implementation plans with a measured, structured approach
                      to implementation.




                      Page 33                                  GAO-12-898 Safety Management Systems
We are sending copies of this report to the appropriate congressional
committees, DOT, NTSB, and interested parties, and others. In addition,
the report will be available at no charge on the GAO Web site at
http://www.gao.gov.

If you or your staff members have any questions about this report, please
contact me on (202) 512-2834 or at dillinghamg@gao.gov. Contact points
for our Office of Congressional Relations and Public Affairs may be found
on the last page of this report. Key contributors to this report are listed in
appendix II.




Gerald L. Dillingham, Ph.D.
Director
Physical Infrastructure Issues




Page 34                                    GAO-12-898 Safety Management Systems
Appendix I: Objectives, Scope, and
              Appendix I: Objectives, Scope, and
              Methodology



Methodology

              Our objective was to assess the Federal Aviation Administration’s (FAA)
              implementation of Safety Management Systems (SMS) and provide
              information on potential implementation challenges. To do so, we
              addressed the following questions:

                  (1) What is the status of FAA’s implementation of SMS?

                  (2) To what extent have FAA’s SMS efforts been consistent with key
                      practices for successful planning and implementation of a new
                      program?

                  (3) What challenges does FAA face in implementing SMS?

              To perform our review, we focused primarily on FAA’s implementation of
              SMS for its business lines as well as its preliminary efforts to require and
              oversee SMS implementation by industry. We conducted background
              research to identify literature related to SMS in aviation, and any
              challenges that agencies might face when implementing SMS. We also
              attended parts of a safety risk management panel on runway status lights
              conducted by FAA’s Air Traffic Organization (ATO) at Seattle-Tacoma
              International Airport in March 2012 as a means of learning more about
              SMS and related processes. During the data collection and drafting
              phases of this report, FAA was in the midst of rulemaking efforts to
              require SMS of Part 121 air carriers and Part 139 airports, so we did not
              comment on any draft or proposed regulatory guidance.

              To determine the status of FAA’s implementation of SMS, we reviewed
              FAA’s SMS orders and pilot project guidance, implementation plans, and
              Notices of Proposed Rulemaking for Part 121 air carriers and Part 139
              airports. We also reviewed international and FAA guidance on SMS
              issued by the International Civil Aviation Organization (ICAO) and the
              Joint Planning and Development Office (JPDO), respectively, and
              National Transportation Safety Board (NTSB) recommendations to FAA
              related to SMS. Finally, we interviewed FAA SMS program managers
              across FAA business lines and offices; industry experts we identified
              based on their knowledge and experience in industry, recommendations
              from aviation industry officials, and a search of SMS literature; and ICAO
              and NTSB officials.

              To assess the extent to which FAA’s efforts have been consistent with
              key practices, we reviewed our reports and other literature on successful
              project planning and implementation, particularly for large-scale
              transformative projects, and condensed the resulting list to eliminate


              Page 35                                   GAO-12-898 Safety Management Systems
Appendix I: Objectives, Scope, and
Methodology




duplication and overlap. To do this, we reviewed previous GAO reports
that highlighted practices associated with successful planning and
implementation of a new program. 1 We removed or consolidated any
duplicate items across the reports to create a single list of 10 criteria. We
then identified FAA’s actions related to these practices by reviewing FAA
guidance and agency documentation such as its SMS implementation
plans, conducting interviews with FAA officials across its business lines,
and using that information to assess the extent to which FAA had
addressed each practice. We determined whether each key practice was
addressed, partially addressed, or not addressed by using criteria
developed for prior GAO reports. As such, we considered a practice
“addressed” if FAA had instituted the practice; “partially addressed” if FAA
had shown some progress toward instituting, or started but not completed
the practice; and “not addressed” if FAA had made minimal or no
progress toward instituting the practice. The team made these coding
decisions together, with two analysts making initial judgments and team
management reviewing and confirming them.

To identify challenges FAA faces in implementing SMS, we reviewed our
prior work on long-standing FAA challenges, such as those related to
training and data, and interviewed aviation industry experts and FAA
officials mentioned above. We reviewed prior GAO work on performance
measurement and workforce analysis, Department of Transportation
Inspector General reports and NTSB recommendations related to SMS.
To obtain industry views on challenges, we interviewed officials from
selected airports and air carriers, industry associations representing
airports, air carriers, and pilots, and individuals with SMS experience
described above. We also reviewed and analyzed documents, including
language in the FAA Modernization and Reform Act of 2012 related to
data protection, and associated scholarly work. To supplement comments
received from the individuals we interviewed, we also reviewed comments
made by aviation stakeholders on the two Notices of Proposed
Rulemaking related to SMS.




1
  For examples, see GAO, Results-Oriented Cultures: Implementation Steps to Assist
Mergers and Organizational Transformations, GAO-03-669 (Washington, D.C.: July 2,
2003) and GAO, Managing for Results: GPRA Modernization Act Provides Important
Opportunities to Address Government Challenges, GAO-11-617T (Washington, D.C.:
May 10, 2011).




Page 36                                        GAO-12-898 Safety Management Systems
Appendix I: Objectives, Scope, and
Methodology




To obtain industry views on both SMS implementation practices and
associated challenges, we interviewed officials from selected airports and
air carriers, which we selected for diversity in size, location, participation
in FAA SMS pilot projects, and submission of comments on FAA’s two
Notices of Proposed Rulemaking related to SMS. (See table 1 for a list of
selected airports.)

Table 1: Airport Interviews

    Airport                                                     State
    Concord Regional                                            North Carolina
    Dallas/Fort Worth International                             Texas
    King County International                                   Washington
    Los Angeles International                                   California
    Manchester-Boston Regional                                  New Hampshire
    Pittsburgh International                                    Pennsylvania
    Sacramento International                                    California
    Seattle-Tacoma International                                Washington
    Sloulin Field International                                 North Dakota
    South Bend Regional                                         Indiana
Source: GAO.



We also interviewed officials from six air carriers: Delta, 2 GoJet, United, 3
Pinnacle, Southwest, and US Airways. Finally, we interviewed officials
with SMS knowledge and expertise, including experts from the Flight
Safety Foundation, Embry-Riddle Aeronautical University, John A. Volpe
National Transportation Systems Center, and MITRE Corporation.




2
    Northwest Airlines merged with Delta Air Lines in 2008.
3
    Continental Airlines merged with United Airlines in 2010.




Page 37                                              GAO-12-898 Safety Management Systems
Appendix II: GAO Contact and Staff
                  Appendix II: GAO Contact and Staff
                  Acknowledgments



Acknowledgments

                  Gerald L. Dillingham, Ph.D., (202) 512-2834, or dillinghamg@gao.gov
GAO Contact
                  In addition to the contact named above, Heather MacLeod (Assistant
Staff             Director); Elizabeth Curda; Leia Dickerson; Sarah Farkas; David Hooper;
Acknowledgments   Delwen Jones; Brooke Leary; Josh Ormond; Larry Thomas; and Elizabeth
                  Wood made key contributions to this report.




(540230)
                  Page 38                                GAO-12-898 Safety Management Systems
GAO’s Mission         The Government Accountability Office, the audit, evaluation, and
                      investigative arm of Congress, exists to support Congress in meeting its
                      constitutional responsibilities and to help improve the performance and
                      accountability of the federal government for the American people. GAO
                      examines the use of public funds; evaluates federal programs and
                      policies; and provides analyses, recommendations, and other assistance
                      to help Congress make informed oversight, policy, and funding decisions.
                      GAO’s commitment to good government is reflected in its core values of
                      accountability, integrity, and reliability.

                      The fastest and easiest way to obtain copies of GAO documents at no
Obtaining Copies of   cost is through GAO’s website (www.gao.gov). Each weekday afternoon,
GAO Reports and       GAO posts on its website newly released reports, testimony, and
                      correspondence. To have GAO e-mail you a list of newly posted products,
Testimony             go to www.gao.gov and select “E-mail Updates.”

Order by Phone        The price of each GAO publication reflects GAO’s actual cost of
                      production and distribution and depends on the number of pages in the
                      publication and whether the publication is printed in color or black and
                      white. Pricing and ordering information is posted on GAO’s website,
                      http://www.gao.gov/ordering.htm.
                      Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
                      TDD (202) 512-2537.
                      Orders may be paid for using American Express, Discover Card,
                      MasterCard, Visa, check, or money order. Call for additional information.
                      Connect with GAO on Facebook, Flickr, Twitter, and YouTube.
Connect with GAO      Subscribe to our RSS Feeds or E-mail Updates. Listen to our Podcasts.
                      Visit GAO on the web at www.gao.gov.
                      Contact:
To Report Fraud,
Waste, and Abuse in   Website: www.gao.gov/fraudnet/fraudnet.htm
                      E-mail: fraudnet@gao.gov
Federal Programs      Automated answering system: (800) 424-5454 or (202) 512-7470

                      Katherine Siggerud, Managing Director, siggerudk@gao.gov, (202) 512-
Congressional         4400, U.S. Government Accountability Office, 441 G Street NW, Room
Relations             7125, Washington, DC 20548

                      Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800
Public Affairs        U.S. Government Accountability Office, 441 G Street NW, Room 7149
                      Washington, DC 20548




                        Please Print on Recycled Paper.