United States General Accounting Office GAO Report to the Honorable Ron Wyden, U.S. Senate April 1997 AVIATION SAFETY FAA Has Begun Efforts to Make Data More Publicly Available GAO/RCED-97-137 United States GAO General Accounting Office Washington, D.C. 20548 Resources, Community, and Economic Development Division B-276714 April 25, 1997 The Honorable Ron Wyden United States Senate Dear Senator Wyden: Public concern about the safety of the nation’s aviation system escalated following the crashes of ValuJet flight 592 and TWA flight 800. The Congress and the public have expressed interest in having the Federal Aviation Administration (FAA) publish airline-specific safety data. In a letter to the Administrator of the FAA dated July 10, 1996, you and Senator Wendell Ford requested that the FAA work with the aviation community to recommend the best means to educate the public and to make available to them information about commercial aviation safety, while ensuring that the safety and integrity of the system is maintained. As agreed with your office, this report reviews FAA’s response to your request. Specifically, it addresses the following questions: (1) What actions has FAA taken to make aviation safety information more available to the public? (2) What has been the public demand for FAA’s aviation safety information? (3) What are FAA’s plans to expand the aviation safety information available to the public? In addition, you asked us for our observations about FAA’s progress in making safety information available to the public. FAA took a number of actions to provide aviation safety-related Results in Brief information to the public beginning in July 1996. FAA formed a working group of senior-level agency officials and adopted a strategy of providing aviation safety information to the public through a three-part effort: establishing an aviation safety information web site linked to the FAA’s Internet web site, publicizing significant enforcement actions,1 and undertaking a public education campaign on aviation safety. As of April 10, 1997, FAA has included four databases on its aviation safety Internet web site. Those databases include information on aviation accidents; other safety-related incidents; traffic data (e.g., departures made) reported by large commercial air carriers, which can be used to calculate comparative 1 FAA may initiate enforcement actions under its compliance and enforcement program in response to apparent or alleged violations of the laws governing federal aviation or of the Federal Aviation Regulations. Enforcement actions include administrative actions, such as warning notices and letters of correction; legal enforcement remedies, such as amending, suspending, or revoking airlines’ operating certificates; and punitive actions, such as imposing civil (financial) penalties and temporarily suspending certificates. Page 1 GAO/RCED-97-137 Aviation Safety B-276714 accident or incident rates; and the safety recommendations made by the National Transportation Safety Board (NTSB) to FAA. Since FAA first made its aviation safety site on the Internet available to the public, it has seen an approximate fourfold increase in the number of users who have accessed the web site each week. Usage has increased during those weeks when a public announcement related to the site has been made. In addition, FAA’s data indicate that users are spending more time using the site. It is too soon, however, to tell if these trends will continue. FAA plans to expand the number of databases that it posts on its aviation safety web site throughout the rest of 1997. It expects to incorporate information on the airlines’ composition (i.e., the make, models, and ages of aircraft in each airline’s fleet) and other indicators of aviation safety (e.g., data on near mid-air collisions). FAA is responsible for setting standards, assessing compliance, and taking Background enforcement actions to ensure that the airlines meet safety standards. To carry out this responsibility, FAA monitors the airlines’ compliance with the Federal Aviation Regulations through periodic inspections. Those regulations set the standards for the airlines’ operations and maintenance functions. A number of possible indicators of aviation safety exist. In a 1988 report, we identified and assessed potential ways of measuring the airlines’ performance in areas important to safety.2 The accident rate is a widely recognized measure of overall aviation safety.3 However, because accidents occur so infrequently, there are no statistically significant differences in the accident rates among similar airlines. Also, because accident rates reflect what has already happened, their relevance to accident prediction or prevention can be limited. Among the other measures discussed in that report were information on inspection results, unsafe incidents, airlines’ financial condition, pilots’ competence, and maintenance quality. 2 Aviation Safety: Measuring How Safely Individual Airlines Operate (GAO/RCED-88-61, Mar. 18, 1988). The report also reviewed government and academic research projects to develop basic information on factors influencing aviation safety. 3 The accident rate is computed by dividing the number of accidents by a measure of the airline’s activity, such as the number of operating hours, passenger miles, or departures. Page 2 GAO/RCED-97-137 Aviation Safety B-276714 Safety-related aviation information varies in the extent to which it is available to the public. In general, “availability” indicates whether or not information is protected from dissemination by federal law. For example, the National Transportation Safety Board, the official source of information on airline accidents, routinely publishes information on aviation accidents. On the other hand, the public can obtain some other information only after making a request through the Freedom of Information Act (FOIA). According to FAA, information on the enforcement actions against regulated entities (i.e., air carriers, airports, manufacturers, schools, or repair stations) has generally been available to the public only through FOIA requests, or when FAA elects, on a case-by-case basis, to publicize an enforcement action. FAA began to take a number of actions to provide aviation safety-related FAA’s Actions to information to the public in July 1996. The Administrator asked FAA’s Provide More Aviation Office of System Safety to assemble a working group of senior-level Safety Information to officials to determine how the FAA could most efficiently and effectively accomplish this task. In addition to FAA’s then-Deputy Administrator, the the Public group included representatives from FAA’s offices of Regulation and Certification, Chief Counsel, Government and Industry Affairs, Civil Aviation Security, and Public Affairs. FAA solicited comments from the public and from the aviation community on how best to educate the public about, and make information available on, commercial aviation safety. FAA contracted with a consultant to generate a discussion of and obtain feedback on the types of aviation safety data that FAA might make available to the public, the means by which such information might be distributed, and the issues and considerations that arise in the distribution of these data. The contractor’s draft report was made available for public comment through the Federal Register on November 13, 1996. According to senior FAA officials, in deciding what means the agency would use to provide greater information to the public, FAA recognized the challenges of availability and accessibility. FAA noted the growing use of the Internet as an expedient and cost-effective means to provide information, especially to those in government, the aviation industry, academia, and the media. As a result, FAA announced on January 29, 1997, that it would use the Internet to pursue all three of its information strategies: establishing an aviation safety information web site linked to Page 3 GAO/RCED-97-137 Aviation Safety B-276714 FAA’s Internet web site, publicizing significant enforcement actions, and undertaking a public education campaign on aviation safety. However, because broad sections of the general public may not have access to the Internet, FAA recognized that it might need to distribute safety information through some other supplementary means. FAA considered using toll-free telephone numbers to provide the public with certain safety information. However, on the basis of the experience of the National Highway Traffic Safety Administration, FAA decided that it lacked the staff resources to answer the large number of calls that it might receive.4 FAA subsequently decided to provide information, at least initially, through other public channels. As the Internet information effort develops, FAA expects to reassess the need for toll-free telephone access. FAA announced that beginning on February 1, 1997, it would issue press releases on newly issued enforcement actions concerning significant cases against regulated aviation entities that involve safety and security issues, including cases seeking civil penalties of $50,000 or more. As of April 16, FAA had issued press releases about three enforcement actions involving civil penalties, along with three instances in which it has revoked air carriers’ operating certificates. In addition to its normal procedures for issuing press releases, FAA has included them on its aviation safety information web site. FAA’s homepage is pictured in figure 1. 4 According to the National Highway Traffic Safety Administration’s budget submission to the Congress for fiscal year 1998, its Auto Safety Hotline received over 800,000 calls in calendar year 1995, and the number was expected to rise in 1996 and 1997. The Department of Transportation has requested $1.8 million for the hotline in fiscal year 1998. Among other things, the hotline provides consumers with information concerning motor vehicle safety, such as recall information on a consumer’s vehicle or child safety seat, and general information to increase consumers’ understanding and awareness of traffic safety. Page 4 GAO/RCED-97-137 Aviation Safety B-276714 Figure 1: FAA’s Internet Homepage Source: http://www.faa.gov Page 5 GAO/RCED-97-137 Aviation Safety B-276714 FAA began its public education campaign about aviation safety on April 2, 1997. On the basis of the public comments received on the consultant’s draft report, FAA determined that it needed to explore more effective ways of communicating with consumers about aviation safety. To complement its information-sharing efforts, FAA’s public education campaign is designed to help the public better understand the safety of the overall system. FAA prepared a short overview of the aviation safety system and included it on its aviation safety information web site. In addition to the press release and public education information, the aviation safety information web site includes a link to a web site maintained by the FAA’s Office of System Safety, where the public can access and search several of the principal sources of aviation safety data and information that are used by the federal government. It also includes an explanation of how to use the data and cautions about how those calculations should and should not be interpreted. Figure 2 shows the information presented on the web site on aviation safety data. Page 6 GAO/RCED-97-137 Aviation Safety B-276714 Figure 2: FAA’s Web Site on Aviation Safety Data Source: http://nasdac.faa.gov/internet/ FAA plans to make public various aviation safety-related databases over time. When it was first made available to the public, the web site included three aviation safety databases: Page 7 GAO/RCED-97-137 Aviation Safety B-276714 • The NTSB Aviation Accident/Incident Database, which is the official repository of aviation accident data and causal factors. NTSB generally defines an “accident” as an occurrence associated with the operation of an aircraft in which individuals are killed or suffer serious injury, or the aircraft is substantially damaged. An NTSB-defined incident is an occurrence, other than an accident, associated with the operation of an aircraft that affects or could affect the safety of operations. The NTSB database contains only selected incident reports. As of April 9, 1997, this database included a total of 37,696 records of aviation accidents and incidents, dating back to 1983. By far, the vast majority (34,073, or approximately 90.4 percent) concerned general aviation aircraft accidents and incidents; 3,623 records (9.6 percent) concerned large or commuter air carriers’ accidents and incidents. • The NTSB’s safety recommendations to FAA with FAA’s responses. NTSB uses information it gathers during accident investigations and the determination of probable cause to make safety recommendations to all elements of the transportation industry. The recipient of a recommendation must respond formally to the recommendation and specify what action is or is not being taken and why. This database includes the 3,471 recommendations made by NTSB to FAA since 1963, along with FAA’s responses.5 • The FAA’s Incident Data System, which contains a more extensive collection of records of aviation incidents—potentially hazardous events that do not meet the aircraft damage or personal injury thresholds contained in NTSB’s definition of an accident. As of April 9, 1997, this database included a total of 67,057 records of aviation incidents, dating back to 1978. As with the NTSB’s Aviation Accident/Incident Database, a relatively small percentage (28.0 percent) of the total number of records concerned incidents experienced by large or commuter air carriers. Users cannot readily retrieve complete copies of these three databases. Rather, users may browse (i.e., look at) individual records, count records (e.g., all accidents involving commuter air carriers during a given time period), or select particular reports on the basis of user-supplied words or phrases (e.g., smoke) and/or user-selected criteria, such as the aircraft’s category of operation.6 5 We have recently reported on FAA’s responsiveness to recommendations made by us, the Department of Transportation’s Inspector General, and NTSB. See Aviation Safety: FAA Generally Agrees With but Is Slow in Implementing Safety Recommendations (GAO/RCED-96-193, Sept. 23, 1996). 6 An air carrier’s category of operation refers to that part of the Federal Aviation Regulations under which the carrier operates. Large air carriers’ (airlines’) operations are covered by part 121 of the regulations, and commuter air carriers’ operations are covered by part 135. Page 8 GAO/RCED-97-137 Aviation Safety B-276714 FAA added another database on March 31, 1997, that provides the means by which the accident and incident information can be put into some context. FAA extracted this database—Airline Traffic Statistics—from information gathered by the Bureau of Transportation Statistics (BTS).7 It contains three selected measures of individual airlines’ operations: the number of departures, hours flown, and miles flown, by year, in domestic commercial service during the 5-year period from 1991 through 1995. Those statistics are the activity measures most frequently used to calculate accident and incident rates for the airlines. Unlike the first databases that FAA included on its web site, users cannot search the data on traffic statistics on the Internet. Users can, however, obtain a copy of this complete database from FAA’s web site, for use on their own computers. FAA includes warnings and disclaimers to explain the limitations of the databases it includes on its web sites. In general, these warnings and disclaimers state that the contents of the web sites are unofficial. FAA notes that the databases may not be complete and makes no certification about the accuracy of the data. Since FAA first established its aviation safety web site on the Internet, it Early Data Indicate has seen an approximately fourfold increase in the number of users who That Public Demand have accessed the safety data web site each week. FAA’s computers for Aviation Safety measure usage in several ways, and each indicates that usage of FAA’s site has grown since it was made public. The best measure of web site usage, Information on the according to FAA officials, is the number of users who have accessed the Internet Has Grown site. Although FAA cannot identify every individual user who accesses its site, it does count the number of users that access the web site over a period of time using a measure called a “user session.”8 In mid-January, before FAA publicly announced the availability of the web site, it averaged about 2,000 user sessions per week, even though the web site consisted mainly of a page explaining that the data will be available at a later date. After media attention about the availability of the web site in late January, the usage that week grew to almost 9,000 user sessions. After declining over several weeks, usage again grew after FAA added the searchable safety data to the site on February 28. FAA hosted about 8,200 user sessions 7 BTS is a unit of the Department of Transportation. BTS compiles, analyzes, and makes accessible information on the nation’s transportation system. The major U.S. airlines are required by federal regulation to provide specific financial, activity, and descriptive data to BTS. 8 As defined by FAA’s software, a user session is a period during which another computer is retrieving data from FAA’s Internet web site. Page 9 GAO/RCED-97-137 Aviation Safety B-276714 during the last week of March. Figure 3 illustrates the number of user sessions per week for the safety data web site. Figure 3: Total Weekly Usage of FAA’s Aviation Safety Data Web Site, for Weeks Ending January 4 Through April 12, 1997 10000 User sessions 9000 8000 7000 6000 5000 4000 3000 2000 1000 0 1/4 1/11 1/18 1/25 2/1a 2/8 2/15 2/22 3/1b 3/8 3/15 3/22 3/29 4/5 4/12 Week ending a Press conference introducing web site—1/29/97. b First searchable safety data put on web site—2/28/97. Source: GAO’s analysis of FAA’s data. In addition to an increase in the number of users, FAA’s data indicate that the public is utilizing the safety data web site more often than when it was first made available. First, FAA tracks the average time of each user session. The length of the average user session had grown to about 12 minutes in early April. Also, FAA tracks the number of times each user requests a file from FAA’s computers—called a “hit.”9 The average number of hits generated during each user session has also grown, from 18.6 in early January to as high as 31.9 in early April. (These data on the number of hits per user session are displayed in table 1.) According to FAA officials, 9 FAA’s software could generate several hits during each user session—one for the file containing the page itself and one for each of the graphic files displayed on the page. However, because of the way most users’ web software is designed, repeat visits to the same page may not be counted as hits. Page 10 GAO/RCED-97-137 Aviation Safety B-276714 the increases in both of these statistics indicate that users are finding the safety data more useful, possibly for research, than in the past. They added, however, that it is too early to tell if these trends will continue. Table 1: Weekly Hits per User Session for FAA’s Aviation Safety Data Web Average time of Average number Site, for Weeks Ending January 4 Number of user each user of hits per user Through April 12, 1997 Week ending sessions session (minutes) session 1/4/97 861 N/A 18.6 1/11/97 2,223 N/A 17.7 1/18/97 2,333 N/A 18.0 1/25/97 1,711 N/A 18.6 2/1/97 8,916 N/A 20.8 2/8/97 7,071 1.5 21.4 2/15/97 4,189 N/A 20.3 2/22/97 3,316 N/A 19.3 3/1/97 5,091 N/A 21.8 3/8/97 4,358 10.7 25.0 3/15/97 6,966 9.9 22.0 3/22/97 6,096 10.0 21.7 3/29/97 8,214 11.3 23.0 4/5/97 4,733 12.0 31.9 4/12/97 4,926 11.3 29.0 Note: N/A = Not available. Source: GAO’s analysis of FAA’s data. Finally, FAA’s computers also keep track of the host computer of each user who accesses the safety data site. The user’s host computer is operated by the organization that provides access to the Internet, whether that organization is an Internet service provider (such as America Online, Compuserve, or Netcom, that mainly serve the public) or another organization, such as Boeing. These data indicate that many of the host computers that access FAA’s site most frequently are operated by Internet service providers. Other frequent users are the Air Force and airlines such as Delta, which operate host computers that are generally available only to their employees. These same data indicate that about 10 percent of those who access FAA’s site are doing so from a computer located outside the United States—mostly from Germany and Canada. Page 11 GAO/RCED-97-137 Aviation Safety B-276714 Because FAA made the safety education material available only recently (on April 2, 1997), it has only limited information on the number of user sessions for that web site: FAA recorded 313 user sessions on that web site for the week ending April 5 and 515 for the week ending April 12. For the press releases on enforcement actions, however, FAA’s statistics indicate that weekly usage has generally fallen since FAA first made those press releases available, and fewer users have accessed this page than have accessed the safety data web site. Figure 4 shows the change in the number of weekly uses of the press release information since early February 1997. Figure 4: Total Weekly Usage of FAA’s Safety Press Release Internet Page, for 2500 User sessions Weeks Ending February 8 Through 2250 April 12, 1997 2000 1750 1500 1250 1000 750 500 250 0 2/8 2/15 2/22 3/1 3/8 3/15 3/22 3/29 4/5 4/12 Week ending Source: GAO’s analysis of FAA’s information. FAA plans to add other safety-related information to the web site gradually FAA’s Plans to Expand over time. By May 31, 1997, FAA plans to add data from the FAA National Information About Airspace Incident Monitoring System, which includes information on near Aviation Safety on Its mid-air collisions.10 On June 1, 1997, FAA expects to make available a quarterly report of enforcement actions in the safety and security areas Web Site 10 Near mid-air collisions are incidents associated with the operation of an aircraft in which a possibility of collision occurs as a result of an aircraft’s proximity of less than 500 feet to another aircraft. Page 12 GAO/RCED-97-137 Aviation Safety B-276714 against aviation entities. This report, which describes enforcement actions closed with a civil penalty or orders of certificate suspension or revocation, will cover the first quarter of 1997. Thereafter, FAA expects to issue its quarterly enforcement reports about 30 days after the end of each quarter. FAA has also indicated that it will expand the available information on airline traffic statistics in two ways. First, it will add data for 1996 as soon as it receives them from BTS in June or July. In addition, FAA expects to add traffic statistics for commuter airlines. At present, the traffic statistics that FAA has posted are limited to ones on domestic operations by large air carriers (i.e., generally those that operate aircraft with more than 60 seats). In addition, by the end of September 1997, FAA will develop a new database that will provide certain basic information about each air carrier, such as the number of specific makes, the models, and the ages of the aircraft flown by the carrier and the date when the carrier was certificated by the Department of Transportation (DOT) and FAA to operate.11 According to FAA officials, the agency has not yet decided how much data should be provided from the existing FAA databases or whether some information could be better provided by the individual air carriers, perhaps in conjunction with their trade associations, through direct links between their respective Internet web sites and FAA’s. According to FAA officials, the agency also intends to evaluate its efforts to provide safety information to the public, but not until March 1998, after the web site has been in operation for approximately 1 year. In an October 1996 report on aviation safety, we concluded that the time GAO’s Observations had come for FAA to begin the process that can lead to publishing on FAA’s Progress to airline-specific safety data.12 The report recommended that the Secretary Date of Transportation instruct the Administrator of FAA to study the feasibility of developing measurable criteria for what constitutes aviation safety, including those airline-specific, safety-related performance measures that 11 Before commencing operations, new airlines must obtain two separate authorizations from the DOT—“economic” authority from the Office of the Secretary of Transportation and “safety” authority from FAA. The Office of the Secretary of Transportation is responsible for assessing whether applicants have the managerial competence, disposition to comply with regulations, and financial resources necessary to operate a new airline. FAA uses a multiphased process to determine whether an applicant’s manuals, aircraft, facilities, and personnel meet federal safety standards. For additional information on these certification processes, see Certification of New Airlines: Department of Transportation Has Taken Action to Improve Its Certification Process (GAO/RCED-96-8, Jan. 11, 1996). 12 Aviation Safety: New Airlines Illustrate Long-Standing Problems in FAA’s Inspection Program (GAO/RCED-97-2, Oct. 17, 1996). Page 13 GAO/RCED-97-137 Aviation Safety B-276714 could be published for use by the traveling public. DOT concurred with that recommendation. FAA’s Internet web site represents a good first step toward providing the public with some aviation safety information. Providing the information in which FAA has the greatest confidence—NTSB’s accident/incident data, FAA’s incident data, and BTS’ traffic data—seems to be a reasonable approach. The early data on the usage of the web site indicate that the public has an interest in aviation safety data. FAA has said that evaluating its efforts will be an important aspect of its overall strategy of providing more information to the traveling public. We agree. Such an evaluation could help FAA determine whether it is meeting the needs of the traveling public and whether it should improve, refine, or expand its safety information, as well as improve the quality of the underlying data. It might also incorporate considerations of the extent to which the public finds these data easily usable, in view of the complexity and size of the posted databases. While it is too early to conduct an evaluation, FAA could begin the planning necessary to ensure that its evaluation produces meaningful results. We provided DOT and FAA with copies of a draft of this report. We met with Agency Comments DOT and FAA officials, including the Manager of FAA’s Safety Data Services Division, acting on behalf of the Deputy Assistant Administrator for System Safety. DOT and FAA officials agreed with the draft report’s overall message and provided editorial and technical comments that we incorporated as appropriate. The information in this report was developed through discussions with officials at FAA and analysis of data on the usage of FAA’s web site over time. We also reviewed previously issued GAO products, pertinent federal regulations, and FAA’s Internet web sites. We did not independently assess the quality of the data that FAA includes on its Internet web sites. We performed our review from March through mid-April 1997 in accordance with generally accepted government auditing standards. As you requested, unless you publicly announce its contents earlier, we plan no further distribution of this report for 30 days. We will then send copies to the Secretary of Transportation; the Director, Office of Page 14 GAO/RCED-97-137 Aviation Safety B-276714 Management and Budget; and other interested parties. We will also make copies available to others upon request. Major contributors to this report were Thomas Kai; Steve Martin; and James Sweetman, Jr. Please call me at (202) 512-3650 if you or your staff have further questions. Sincerely yours, Gerald L. Dillingham Associate Director, Transportation Issues (341531) Page 15 GAO/RCED-97-137 Aviation Safety Ordering Information The first copy of each GAO report and testimony is free. Additional copies are $2 each. Orders should be sent to the following address, accompanied by a check or money order made out to the Superintendent of Documents, when necessary. VISA and MasterCard credit cards are accepted, also. Orders for 100 or more copies to be mailed to a single address are discounted 25 percent. Orders by mail: U.S. General Accounting Office P.O. Box 6015 Gaithersburg, MD 20884-6015 or visit: Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders may also be placed by calling (202) 512-6000 or by using fax number (301) 258-4066, or TDD (301) 413-0006. Each day, GAO issues a list of newly available reports and testimony. To receive facsimile copies of the daily list or any list from the past 30 days, please call (202) 512-6000 using a touchtone phone. A recorded menu will provide information on how to obtain these lists. For information on how to access GAO reports on the INTERNET, send an e-mail message with "info" in the body to: email@example.com or visit GAO’s World Wide Web Home Page at: http://www.gao.gov PRINTED ON RECYCLED PAPER United States Bulk Rate General Accounting Office Postage & Fees Paid Washington, D.C. 20548-0001 GAO Permit No. G100 Official Business Penalty for Private Use $300 Address Correction Requested
Aviation Safety: FAA Has Begun Efforts to Make Data More Publicly Available
Published by the Government Accountability Office on 1997-04-25.
Below is a raw (and likely hideous) rendition of the original report. (PDF)