United States General Accounting Office GAO Report to Congressional Committees November 1999 CONSUMER PRODUCT SAFETY COMMISSION Additional Steps Needed to Assess Fire Hazards of Upholstered Furniture GAO/HEHS-00-3 United States GAO General Accounting Office Washington, D.C. 20548 Health, Education, and Human Services Division B-282713 November 17, 1999 The Honorable Christopher Bond Chairman The Honorable Barbara A. Mikulski Ranking Minority Member Subcommittee on VA, HUD, and Independent Agencies Committee on Appropriations United States Senate The Honorable James T. Walsh Chairman The Honorable Alan B. Mollohan Ranking Minority Member Subcommittee on VA, HUD, and Independent Agencies Committee on Appropriations House of Representatives In 1994, the Consumer Product Safety Commission (CPSC) began preliminary work on setting a flammability standard that would make it harder for sofas, chairs, and other upholstered furniture to catch fire. CPSC estimated at that time that nearly 700 people died annually from such fires. During the 5 years that the agency has been studying the potential standard, the need for such a standard has remained an issue of considerable debate. Proponents, such as fire protection groups, contend that without such a standard, the public is subject to an unnecessary risk. Opponents, including upholstered furniture manufacturers and the Small Business Administration, respond that the magnitude of the problem is not great enough to warrant the risks or added expense involved in treating fabrics with flame-retardant chemicals or taking other steps to make fabrics more flame-resistant. Opponents also contend that more cost-effective solutions may exist, such as making more use of smoke detectors to warn when furniture has caught fire. To issue a flammability standard, the CPSC commissioners must determine that the standard’s benefits bear a reasonable relationship to its costs.1 To do this, CPSC assesses in quantitative terms whether the lives and property saved would justify the additional expense or risks associated with building furniture that complies with the standard. In the conference 1 15 U.S.C. 1193 (j)(2)(B) (1994). Page 1 GAO/HEHS-00-3 Furniture Flammability Standard B-282713 report for CPSC’s 1999 appropriations act,2 the Congress directed us to review CPSC’s process for establishing a potential flammability standard.3 Because CPSC’s work on the potential standard is still in process, we did not conduct a comprehensive analysis of all aspects of the agency’s preliminary computation of the risks, costs, and benefits. As agreed with your offices, we focused our efforts primarily on analyzing CPSC’s approach to quantifying the fire hazards that are the basis for computing the standard’s potential benefits. We addressed the following questions: • What methodology does CPSC use to estimate the magnitude of the fire hazard that the standard would address? • How reliable is this methodology for producing sound estimates of the hazard that the standard would address? To answer these questions, we reviewed the process, including the underlying assumptions, that CPSC used to develop its initial estimates of the magnitude of fire hazards from upholstered furniture. We conducted limited tests of the effects of certain assumptions on the estimated benefits of the standard. We also looked at the capacity of CPSC’s methodology to produce sound fire hazard estimates when using the most current data available. We conducted our work between May and October 1999 in accordance with generally accepted government auditing standards. Appendix I describes our scope and methodology in more detail. Because no single national data source exists on the magnitude of hazards Results in Brief and losses caused by upholstered furniture fires, CPSC blends information from two different sources. One source provides national estimates on the total number of fires in four general categories and the extent of losses, but it provides no information about specific types of fires, such as upholstered furniture fires. The second source provides detailed information for specific types of fires, but for only a portion of all fires in the United States. CPSC uses the details from the second source of data and the national estimates from the first source to calculate national estimates of fire losses from the kinds of upholstered furniture fires the standard would address. 2 H.R. Rep. No. 105-769 at 267 (1998). 3 CPSC’s appropriations act directed the National Academy of Sciences to study the potential toxicological risks of all flame-retardant chemicals identified by CPSC and the Academy as likely candidates for use in making upholstered furniture resistant to ignition. Page 2 GAO/HEHS-00-3 Furniture Flammability Standard B-282713 At present, CPSC cannot ensure that its methodology provides a complete picture of the national fire losses that the potential standard would address. For example, • CPSC has not developed a statement of precision for the estimated losses from upholstered furniture fires. Without this, CPSC’s estimates of fire losses do not adequately disclose the range of benefits that may be associated with its potential standard. • CPSC’s methodology for calculating fire losses has the effect of including losses that are not likely to be addressed by the standard. Fire losses involving upholstered furniture are counted even though other factors not addressed by the standard may have been responsible, such as fires that are intentionally set. Also, for those fires for which the cause or origin is unknown, CPSC assumes that upholstered furniture fires will occur in the same proportion they occur in fires with a known cause. Our testing shows that these methods are likely to overstate fire losses that the standard would address, and as a result, they could have a material effect on the associated benefits expected from the potential standard. Various analyses can be used to assess the validity of underlying assumptions and ultimately strengthen CPSC estimates, but so far CPSC has not used them. We are recommending that, as CPSC continues to consider the need for a mandatory flammability standard for upholstered furniture, it should conduct additional analyses to identify the level of imprecision in the methodology’s fundamental assumptions and apply any necessary revisions to its cost-benefit analysis of the potential standard. Although CPSC has reported that upholstered furniture fires account for Background only 3 percent of all residential fires that occur each year, these fires take a high toll in human life and property damage. CPSC reports that of all the products the agency regulates, upholstered furniture is the leading cause of household fire death. Under the Flammable Fabrics Act,4 which authorizes the issuance of flammability standards for clothing, upholstery, and other products, CPSC has the authority to issue mandatory performance and labeling standards for upholstered products. To address the hazards of upholstered furniture fires, the National Association of State Fire Marshals petitioned CPSC in 1993 to issue a flammability standard for residential upholstered furniture by adopting 4 15 U.S.C. 1191 (1994). Page 3 GAO/HEHS-00-3 Furniture Flammability Standard B-282713 three California standards.5 In 1994, after studying national estimates on the incidence and severity of household fires, CPSC granted part of the petition. This part of the petition dealt with small open flames such as matches and candles. By a 2-to-1 vote, CPSC published an Advance Notice of Proposed Rulemaking to announce its consideration of issuing a flammability standard or other regulation to address the risks posed by these upholstered furniture fires. CPSC commissioners deferred action on the cigarette portion of the petition in order to first assess the effectiveness of and industry compliance with an existing voluntary furniture flammability standard for cigarettes. However, on the basis of their initial laboratory testing, CPSC staff now believe a small open-flame standard will also address cigarette-caused fires.6 As a result, they have taken that assumption into account in developing estimates of the fire losses the standard would address and the benefits it would produce. Proponents of a mandatory standard point to its ability to prevent death, injury, and property damage as the major benefits. Opponents recognize that these losses should be avoided, but they believe the potential costs associated with the proposed standard are too great. Possible costs cited include those related to health risks—both to employees of furniture manufacturers and to consumers—posed by flame-retardant chemicals that may be used; increased prices consumers would have to pay for upholstered furniture to cover a variety of additional manufacturing costs; diminished feel and texture of fabric treated with flame-retardant chemicals; and loss of consumer choice because some materials may be eliminated if they cannot be made flame-resistant. CPSC activities conducted since the Advance Notice of Proposed Rulemaking have included in-depth fire investigations, technical analyses and laboratory tests, development of a draft standard, and initial estimates of the standard’s potential costs and benefits. CPSC’s 1999 appropriations act prevents the standard from proceeding past this stage until CPSC fully considers the fundings and conclusions of the National Academy of Sciences’ congressionally mandated review on the potential toxicity of flame-retardant chemicals. This review is to be completed in January 2000. 5 California has standards for upholstered furniture fires started by large open flames in public buildings that lack a sprinkler system, upholstered furniture fires started by smoking materials, and those started by small open flames, such as candles. Separate standards were requested for small open flames and cigarettes because some materials have different probabilities of catching fire, depending on the heat source. 6 CPSC staff said that, for this reason, they will not take further action on the cigarette portion of the petition until after the technical work for the small-open-flame portion is completed. Page 4 GAO/HEHS-00-3 Furniture Flammability Standard B-282713 To estimate the incidence of fires and losses resulting from upholstered CPSC Uses Two Data furniture being ignited by cigarettes or small open flames, CPSC uses Sources to Estimate aggregate fire data from a national survey, detailed data on a portion of the Magnitude of the individual fires, and a methodology to combine the two databases. This approach is necessary because no single data source exists that provides Upholstered Furniture the information needed to estimate the magnitude of the problem the Fire Problem standard is intended to address. The aggregate survey data provide national fire loss estimates for four general categories of fires (such as residential or vehicle fires) but provide no information about detailed characteristics of the fires. This information is from a survey conducted by the National Fire Protection Association (NFPA), a nonprofit fire protection association. In contrast, the second source consists of detailed information linking fire losses—deaths, injuries, and damages—with specific types of fires, such as those originating in upholstered furniture. However, the detailed data do not provide national totals and are not collected in a way that permits them to be used by themselves for national projections. The detailed data are collected from more than a third of the nation’s fire departments and are found in the National Fire Incident Reporting System (NFIRS), a federal database. This database is maintained by the U.S. Fire Administration (USFA), within the Federal Emergency Management Agency. The NFIRS data contain information on more than 20 different fire characteristics, such as the source of the fire and what caught fire first. Because neither NFPA estimates nor the NFIRS data can produce a nationwide estimate of fire losses for specific types of fires, CPSC uses a methodology for combining the two. The process of blending the two sources of data was developed jointly by NFPA, CPSC, and USFA to approximate and report on fire trends for the fire-fighting community and the general public. CPSC, NFPA, and USFA use this general process to track and report on fires and fire losses nationally. They have all used this process for many years and try to apply it consistently so that conflicting estimates are not produced, which could confuse the public and those who use the data. The methodology is based on the proportional relationship of the NFIRS data to the NFPA estimates. If NFIRS data contain half of the total number of fires estimated by NFPA, for example, NFIRS numbers for specific types of fires are doubled to produce national estimates for these specific fires. The NFIRS data also include a number of fires for which the first item ignited is unknown. The methodology adds a proportional number of these fires and fire losses to the NFIRS fires known to have started in upholstered Page 5 GAO/HEHS-00-3 Furniture Flammability Standard B-282713 furniture. In the 1994 NFIRS data, for example, upholstered furniture fires constituted 3 percent of all fires with a known item of origin. Therefore, 3 percent of the fires with an unknown item of origin also would be allotted to upholstered furniture fires. CPSC has not fully addressed the uncertainty surrounding key data and Extent of Fire Losses underlying assumptions that it uses in developing national fire loss Is Uncertain estimates. Particularly important are (1) the need to account for imprecision surrounding the data used to make national fire loss estimates and (2) the assumption that the potential standard could address all upholstered furniture fires classified as being ignited by either small open flames or cigarettes. So far, CPSC has not accounted for this imprecision or tested the soundness of this assumption, and as a result, the full range of fire losses is not known. Our limited testing demonstrates the importance of these steps in developing sound estimates of fire losses that the standard is designed to address. CPSC Has Not Tested Key Several issues raise uncertainties about the underlying assumptions Issues related to the NFPA and NFIRS data (see table 1). These issues include the low response rates to the NFPA survey, the voluntary nature of reporting NFIRS data, and the fact that some upholstered furniture fires started by small open flames or cigarettes will not be addressed by the standard. These issues raise questions about whether the fire and fire loss data provided by NFIRS and NFPA result in a representative picture of the incidence and severity of residential fires that occur nationally. In relying on these data for standard-setting, it is important to understand whether the data are representative and, if they are not, whether they overestimate or underestimate the national fire problem. Page 6 GAO/HEHS-00-3 Furniture Flammability Standard B-282713 Table 1: Key Issues Potentially Affecting the Validity of CPSC Subject of Estimates preliminary Assumption Issues that raise uncertainty testing by GAO NFPA survey data NFPA national fire and fire Low response rate (21 percent of fire Yes loss estimates are departments surveyed)a and limited representative of fires review of nonrespondents results in nationwide. uncertainty about extent that national projections accurately represent the national fire problem. No outside review of the analysis No used to produce national estimates was conducted. Such a review, usually conducted by a party with a background in methodology and statistics, is commonly used to help identify flaws or constraints in estimating methodologies. Potential exists for survey No respondents to provide inaccurate information. Limited follow-up or corroboration of survey responses is a way to ascertain the degree to which these responses are accurate and supported. NFIRS data Any upholstered furniture fire Some upholstered furniture fires that Yes classified as being ignited by were started by cigarettes or small a small, open flame or open flames also involve other cigarette could be addressed factors that place them outside the by the potential standard. category of fires the standard addresses, raising concern that too many fire losses may be linked to the potential standard. Fire and fire loss data Voluntary reporting carries the Yes reported by local fire potential that certain types of fire departments are departments, such as large fire representative of fires that departments, will be over- or occur nationally. underrepresented in the data, or will only report major fires, which would affect the extent to which data are representative of the fire problem nationally. There is potential for data reported No by local fire departments to be inaccurate. Limited verification and corroboration of reported data is a way to ascertain the level of accuracy of the reported data. (Table notes on next page) Page 7 GAO/HEHS-00-3 Furniture Flammability Standard B-282713 a Fire departments protecting a larger number of people (over 50,000) responded to the survey at a rate more than twice that of departments protecting fewer people (under 50,000). As a result, NFPA reported that it received responses from fire departments that serve 40 percent of the population. Testing of Issues Showed We selectively tested three key issues that affect the validity of CPSC’s Potential for Inaccuracy estimates: • the extent to which the data used to project NFPA estimates provide nationally representative and precise estimates, • the extent to which data on fire losses that the potential standard does not address are included in developing estimates of fire losses the standard is expected to address, and • the extent to which fire departments in the NFIRS database are representative of fire departments nationally and provide nationally representative fire data. Our selective testing showed mixed results, indicating that CPSC needs to more fully analyze some of its assumptions in order to provide greater assurance that its fire loss estimates are valid. Representativeness and Because of the overall low response rate to the NFPA survey (21 percent), Precision of NFPA Estimates we conducted a limited test to assess the accuracy of NFPA estimates. Corroborating survey projections to another data source is a common way of assessing the representativeness of survey data. This type of comparison can be a general gauge of how well the survey represents the nation as a whole. To do this, we compared the NFPA death estimates with the Centers for Disease Control and Prevention (CDC) death estimates. CDC obtains this information from death certificates recorded in the 50 states and the District of Columbia. The data classify deaths by the underlying cause of death, which is determined from the death certificate information.7 Our testing indicated that CDC death data fall within the bounds of NFPA’s estimates. On the basis of this general test, the data appear to provide representative estimates of fire deaths. We did not assess the representativeness of the NFPA national estimates of injuries and property damage from fires. To provide a complete picture of fire losses, CPSC needs to account for the range of precision around NFPA’s estimates. Because the estimates are 7 While these data may not be 100-percent accurate either, they are subject to numerous tests and checks and are widely used within the health research community. Page 8 GAO/HEHS-00-3 Furniture Flammability Standard B-282713 based on a survey, they have a margin of error that needs to be incorporated into national upholstered furniture fire loss estimates. Survey researchers routinely report the margin of error in the results of their surveys and polls. CPSC has not accounted for the margin of error around NFPA estimates for deaths, injuries, and property damage. Although CPSC stated that the margin of error for the total number of fires (2.5 percent) was not significant for its estimates, we found that the margins of error for deaths and injuries were considerably larger than the margin of error for the total number of fires. In 1997, the largest margin of error was for the estimated number of deaths, which could be off by as much as 350 deaths (10 percent) in either direction. Because deaths, injuries, and property damages are the factors for estimating losses from the kinds of fires covered under a potential standard, they all need to be considered. By not obtaining and using all of this information in developing its estimates of upholstered furniture fire losses, CPSC runs the risk of conveying a false sense of precision about its results. CPSC’s Calculation of Fire CPSC’s methodology for estimating fire losses that the potential standard is Losses expected to address also warrants additional refinement. First, in estimating total fire losses, CPSC identifies all fires in NFIRS data known to have involved the ignition of upholstered furniture by small open flames or cigarettes.8 Second, when NFIRS data lack complete information to link fire losses to a specific type of fire, CPSC classifies a portion of these fires as upholstered furniture fires involving small open flames or cigarettes. Our work shows that both of these procedures are likely to overstate the fire losses the potential standard is designed to address, and as a result, they have a material effect on the benefits ascribed to the standard. With regard to counting fires for which the origin and cause are known, CPSC includes upholstered furniture fires that were classified in NFIRS as being started by small open flames or cigarettes. However, our examination of the same data shows that other factors were involved in the cause of some of these fires, making them fires the standard would not appear to address. For example, the standard is not designed to address fires intentionally set or those in which a small open flame ignites a paper or flammable liquid that is on or near an upholstered couch that in turn ignites the couch. CPSC, however, has not yet excluded these types of fires in its estimate. 8 As pointed out earlier, CPSC has determined that the potential standard will address both small-open-flame and cigarette fires, and as a result, CPSC estimates fire losses for both types of fires. Page 9 GAO/HEHS-00-3 Furniture Flammability Standard B-282713 CPSC’s allocation of unknown fire losses also warrants additional attention. In 1997, about 35 percent of the residential fire deaths in the NFIRS data were not identified by what first caught fire, such as upholstered furniture, draperies, chimneys, or walls. When NFIRS data lack complete information to link fire losses to a specific type of fire, CPSC allocates these losses on the basis of how the fire loss data are distributed across all fires in which the specific type of fire is known. In other words, if 19 percent of all residential fire deaths are attributed to upholstered furniture, the same percentage of deaths that stem from fires not attributed to specific types of fires is assigned to upholstered furniture fires. CPSC officials told us they proportionally allocated unknown fire loss data because they had no evidence that the data should be allocated differently. Methods and data exist that could categorize the origin and cause of fires better before calculating the fire losses the standard could possibly address. The NFIRS data are useful in this regard because information available in NFIRS allows certain types of fires to be ruled out as likely to be addressed by the standard. As a result, the NFIRS data can be used to test the effect of CPSC’s method of calculating total fire loss data. We tried such a test using NFIRS data for 1997. We analyzed detailed information on fires—those known to be upholstered furniture fires started by a small open flame or cigarette and those with an unknown origin or cause—looking for characteristics to identify fires that the potential standard would not appear to address. For example, fires we recategorized include those that were intentionally set, those involving electrical short circuits, and those involving flammable materials not used to upholster furniture (such as gases, flammable liquids, or cardboard). Our reclassification of fires reduced CPSC’s estimate of fire losses that the standard is designed to address by up to 152 deaths, 434 injuries, and $70 million in property damages (see app. I for more details on these reductions). Adjusting for this reclassification of fires could reduce the benefits CPSC attributes to the standard from $885 million to $610 million, or 31 percent of the total benefits. On the other hand, additional and more thorough investigation of some fires by CPSC could identify other factors that show that some of these fire losses would be prevented by the standard. Also, our test should not be taken as the most definitive or comprehensive approach to this issue. For example, more comprehensive analytical approaches to assess and allocate unknown fire losses could yield results that would make the estimates more precise. Our selective testing was Page 10 GAO/HEHS-00-3 Furniture Flammability Standard B-282713 done mainly to determine whether the underlying assumption of CPSC’s approach appears sound. Our results indicate that some of CPSC’s current assumptions have a level of uncertainty that is material and as such warrants attention. Representativeness of the NFIRS data are based on reports submitted by a little more than a third of Fire Departments That the 31,000 fire departments nationwide. Because the fire departments Report NFIRS Data reporting to NFIRS do so voluntarily, it is important to determine how representative this group is of fire departments and fire experiences as a whole. For example, 11 states have no fire departments reporting information to NFIRS, and only a portion of the fire departments in the other states participate. If certain types of fire departments, such as large urban fire departments, are more likely or less likely to report to NFIRS, or if fire departments tend to report only major fires, this could skew estimates of the fire problem. Testing the extent that NFIRS data are representative of fire and fire losses that occur nationally can be conducted in several ways. We did so by assessing the extent to which the types of fire departments reporting data to NFIRS resembled the types of fire departments nationally on the basis of two factors: the size of the fire department and whether it is predominantly a volunteer or a paid fire department. This analysis indicated that NFIRS-reporting fire departments fall into the various fire department size and type categories at roughly the same proportions as all fire departments across the country. As a result, any differences in fire losses that is related to fire department size or type are probably appropriately represented in the NFIRS data. Effect of Selected Testing Although the selected testing we conducted resulted in a decline in on Cost-Benefit Estimates estimated benefits expected from the potential standard, the tests should not be taken as a comprehensive assessment of CPSC’s overall cost-benefit calculation. For example, we did not review or evaluate CPSC’s assumptions about the effectiveness of the potential standard in preventing upholstered furniture fires ignited by small open flames or cigarettes, which also have a major impact on the estimated benefits of the standard. We also did not assess the validity or scientific basis for CPSC’s assumptions concerning the extent to which a small-open-flame standard will also address cigarette-caused fires. CPSC estimated that the potential standard would prevent 80 percent of the small-open-flame losses and Page 11 GAO/HEHS-00-3 Furniture Flammability Standard B-282713 50 percent of the cigarette-related losses.9 However, this key assumption is subject to further laboratory testing by CPSC. Also, as stated earlier we did not analyze assumptions and methodologies used to estimate the costs of the standard. It is important to recognize that the testing of other issues and assumptions—especially those relating to costs—may offset, at least in part, the decline in estimated potential benefits that our testing found.10 As a result, we present no estimates of the effect our testing has on tne net costs or benefits of the potential standard. Rather, our testing demonstrates the need for CPSC to further refine its analysis. To promulgate a mandatory standard, the CPSC commissioners must Conclusions determine that benefits produced by the standard bear a reasonable relationship to its costs. A high degree of assurance about the soundness of these estimates is an important part of accurately assessing this relationship. As matters stand, CPSC’s current approach is not likely to generate the necessary degree of assurance. To provide this assurance, CPSC needs to demonstrate, to a greater degree than now exists, the validity of the assumptions on which the estimate is based. This will require testing that so far has not been part of CPSC’s approach. To resolve issues surrounding the data and assumptions used in preparing Recommendations the cost-benefit analysis for a potential standard to protect against fire hazards associated with upholstered furniture, we recommend that the Chairman, CPSC, direct CPSC staff to conduct additional and more detailed analyses of key assumptions including, but not limited to, • assessing the precision surrounding NFPA national fire loss estimates and their impact on estimated benefits attributable to the standard and • identifying a more accurate method to calculate fire losses that could be addressed by the standard. We also recommend that any necessary revisions identified by these analyses should be incorporated into the cost-benefit analysis of the potential standard. 9 In applying this rate of effectiveness, CPSC also reduced cigarette-related losses by an additional 24 percent to account for the effect of the existing voluntary industry flammability standard. 10 It is also possible that this testing would lower the estimates even further. Page 12 GAO/HEHS-00-3 Furniture Flammability Standard B-282713 In its comments on our draft report, CPSC agreed with our Agency Comments recommendations. It stated that CPSC staff have already begun to address and Our Response the issues we raised as they refine their fire loss estimates for upholstered furniture. (See app. II for the complete text of CPSC’s comments.) However, CPSC stated that the report overall downplayed the positive findings about CPSC data and overemphasized the data problems. In fact, after reviewing our report, CPSC provided us with new information and perspective on the representativeness of the NFPA data. After considering this additional information, we modified our draft to agree with CPSC that our limited testing did not disclose any obvious data problems. However, this does not indicate that the data are fully reliable, because our testing did not address all the issues surrounding the underlying data. For example, NFPA and NFIRS data are based on data reported by individual fire departments that have not been subject to verification or corroboration. As a result, we believe our report, as modified, presents an appropriate characterization of our results and the remaining uncertainties surrounding the underlying data. CPSC also commented that our analysis overstated the significance of the issues surrounding NFIRS data. CPSC concluded that it was not appropriate for us to quantify the effect of these data problems and said that our analysis was flawed for two reasons: • Our analysis eliminated those fires that available information indicated were inconsistent with the scope of the potential standard. However, CPSC stated that this process could result in eliminating too many fires. • Our analysis erroneously adjusted the fire data where the cause was unknown. CPSC commented that we did not include the correct proportion of fires with unknown origin or source in arriving at a total number of fires that the potential standard is designed to address. We do not agree with CPSC’s conclusion. First, our approach was to determine whether data classification issues could have a material effect on CPSC’s estimated losses that the potential standard would address. We believe our analysis was a reasonable one for this purpose and that it shows a likelihood of significant effect that warrants attention. The report appropriately qualifies the results and acknowledges that our tests should not be taken as the most definitive or comprehensive approach. Rather, it is CPSC’s responsibility to develop more precise estimates, as it proceeds in its rulemaking process. Page 13 GAO/HEHS-00-3 Furniture Flammability Standard B-282713 Second, we believe that our approach properly distributed the number of “unknown” fires to the total number of fires that the potential standard is designed to address. In our opinion CPSC’s approach overstates the true proportion because it classifies as unknown some fires that available data indicate are not subject to the proposed regulation. Consequently, our methodology corrects the data to account for the ”greater chance of containing upholstered furniture cases.” In making this adjustment, contrary to CPSC’s comments, we did not eliminate fires from consideration. Rather, we reclassified fires as known not to be upholstered furniture fires that the potential standard is designed to address. We than recalculated the proportion of unknown fire losses on the basis of our reclassification. Finally, CPSC commented that our recalculation of its estimated fire losses, adjusted for reclassification of unknown fire origin or cause, would overstate the reduction in potential benefits. Our report indicates that, while this recalculation is an upper-bound estimate for this factor, further investigation and analysis could also influence the magnitude of estimated fire losses. CPSC’s continuing efforts to refine its analysis—of estimated costs as well as benefits associated with its proposed standard—will be central to its ongoing work to determine the merits of proceeding with a mandatory flammability standard. In addition to CPSC’s written comments, CPSC staff provided us with oral comments, which we incorporated, where appropriate, in the final report. We are sending copies of this report to the Honorable Ann Brown, Chairman, and the Honorable Thomas H. Moore and the Honorable Mary Sheila Gall, Commissioners, CPSC; and appropriate congressional committees. We will also make copies available to others upon request. If you or your staffs have any questions about this report, please contact me at (202) 512-7118 or Frank Pasquier at (206) 287-4861. Major contributors to this report include Tim S. Bushfield, Evan Stoll, and Stan Stenersen. Kathryn G. Allen Associate Director, Health Financing and Public Health Issues Page 14 GAO/HEHS-00-3 Furniture Flammability Standard Page 15 GAO/HEHS-00-3 Furniture Flammability Standard Contents Letter 1 Appendix I 18 Range of Information Sources Used 18 Scope and Analyzing the Methodology for Making National Fire Loss 18 Methodology Estimates Testing the Validity of Underlying Assumptions and the 19 Soundness of National Estimates Appendix II 21 Comments From the Consumer Product Safety Commission Tables Table 1: Key Issues Potentially Affecting the Validity of CPSC 7 Estimates Table I.1: Reduction in Fire Loss Estimates Based on 20 Recategorization of Fires that the Standard Is Not Designed to Address Abbreviations CDC Centers for Disease Control and Prevention CPSC Consumer Product Safety Commission NFIRS National Fire Incident Reporting System NFPA National Fire Protection Association USFA U.S. Fire Administration Page 16 GAO/HEHS-00-3 Furniture Flammability Standard Page 17 GAO/HEHS-00-3 Furniture Flammability Standard Appendix I Scope and Methodology Our work focused on the methodology and data that the Consumer Product Safety Commission (CPSC) is using to estimate national fire losses from the kinds of upholstered furniture fires that would be covered by the potential flammability standard now being considered. We used a variety of sources and methods to gather and analyze data on this issue. We obtained information from a wide variety of agencies, interest groups, Range of Information and other sources. Among the key sources were the following: Sources Used • CPSC officials, including those responsible for calculating national fire loss estimates and estimating the potential benefits that would result from the standard; • National Fire Protection Association (NFPA) officials responsible for conducting NFPA’s annual fire surveys, calculating the survey’s national fire and fire loss estimates, and involved in developing the methodology that combines NFPA’s survey projections with detailed fire data collected by local fire departments; • U.S. Fire Administration officials responsible for overseeing and analyzing the detailed fire and fire loss data reported to the agency’s National Fire Incident Reporting System (NFIRS); • officials from the National Center for Health Statistics and the National Center for Injury Prevention and Control at the Centers for Disease Control and Prevention (CDC); • officials representing various segments of the upholstered furniture manufacturing industry, such as fiber manufacturers and upholstered furniture manufacturers; and • private consultants who use the NFPA and NFIRS data to estimate national fire problems. Our work also included a review of relevant legislation and other documents related to CPSC’s approach to considering the potential standard. To address the study’s first question (the methodology used to estimate Analyzing the the magnitude of the fire hazard that would be addressed by the standard), Methodology for we obtained documentation describing the process and examples Making National Fire illustrating the data and major steps in the process from CPSC, NFPA, and the U.S. Fire Administration. In addition to reviewing documents and Loss Estimates conducting interviews to learn how these estimates had been developed, we also obtained and analyzed the 1997 NFIRS database of actual fire Page 18 GAO/HEHS-00-3 Furniture Flammability Standard Appendix I Scope and Methodology incidents as well as detailed information about the 1997 NFPA survey, including some of the characteristics of the fire departments that responded to it. To address the study’s second question (the reliability of this methodology Testing the Validity of to produce sound fire hazard estimates), we first identified the Underlying assumptions underlying key data and methodological steps in the process. Assumptions and the We reviewed each data source separately to identify the analytical and statistical limitations accompanying the assumptions on which the Soundness of National information was based. When we identified limitations, we compared Estimates CPSC’s process to date with various analytical and statistical methods for testing reliability. For example, we assessed the potential implication of NFPA’s response rate on the soundness of the data, determined what statistical techniques could help assess the effect of these uncertainties, and spoke with CPSC staff to determine the extent to which they had conducted their own tests of these uncertainties. We focused more detailed testing on three specific assumptions: the representativeness of data used to make NFPA’s national fire estimates; the representativeness of data in NFIRS; and CPSC’s classification of fires the standard would address, including fires with and without a known item of origin or ignition source. To assess the representativeness of the data used in the national fire estimates, we conducted two types of analysis. First, we compared NFPA’s national estimate of total fire deaths with the number of national fire deaths compiled by CDC. Because CDC obtains information from all death certificates in all 50 states and the District of Columbia, we concluded that the data were a valid basis for comparison. Second, we obtained from NFPA information on the margin of error around its national estimates of fires, deaths, injuries, and property damages. To assess the representativeness of data in NFIRS, we compared the distribution of NFIRS fire departments by size with the distribution of fire departments nationally. We did so by categorizing fire departments by using the size of the population they protected as a proxy for fire department size. To do this, we needed more data than were available in NFIRS. We matched NFIRS fire departments with the NFPA database and used the NFPA population data in those cases where a fire department appeared on both databases. We also conducted a similar analysis comparing the distribution of fire departments based on the type of fire fighting personnel employed (paid or volunteer) and the severity of fires reported by each category of fire department. Page 19 GAO/HEHS-00-3 Furniture Flammability Standard Appendix I Scope and Methodology To evaluate CPSC’s calculation of fire losses expected to be addressed by the potential standard, we used data available in NFIRS to recategorize fires the standard is not designed to address. We conducted this analysis using information from CPSC and 1997 NFIRS data. To identify those fires the potential standard is not designed to address, we used other information in the NFIRS database such as the type of material ignited first, the situation that resulted in contact between a heat source and flammable material, the equipment involved in the ignition, and the area of the house where the fire started. As a result of these analyses, adjustments were made to (1) the raw number of upholstered furniture fires and fire losses known to have resulted from small open flames or cigarettes; (2) the raw number of fires with an unknown origin and/or heat source that are allocated to upholstered furniture fires the standard is intended to address; and (3) the weights, or multipliers, used to project the raw NFIRS numbers into national estimates. Table I.1 shows the effect of these adjustments on fire loss estimates that the standard is designed to address. Table I.1: Reduction in Fire Loss Estimates Based on Recategorization Recategorized Recategorized of Fires That the Standard Is Not fires with a fires with an Designed to Address known source or unknown source Fire loss type origin or origin Total Deaths 87 65 152 Injuries 296 138 434 Property damage $48 million $22 million $70 million To determine the impact these adjustments have on CPSC’s estimate of potential benefits, we used a two-step process. First, we compared the total cost of fire losses based on national estimates resulting from our adjustments with the total costs from CPSC’s national fire loss estimates. Then we applied the difference between CPSC’s and our total cost estimates to CPSC’s estimate of potential benefit from the standard. A general limitation of our analysis was that it did not review the scientific basis for the effectiveness of the potential standard. Laboratory testing to show the extent that the standard would prevent different types of upholstered furniture fires is ongoing, according to CPSC. Rather, the scope of our work was limited to reviewing whether the methodology for using existing data was sufficiently reliable to produce sound fire loss estimates. Page 20 GAO/HEHS-00-3 Furniture Flammability Standard Appendix II Comments From the Consumer Product Safety Commission Page 21 GAO/HEHS-00-3 Furniture Flammability Standard Appendix II Comments From the Consumer Product Safety Commission Page 22 GAO/HEHS-00-3 Furniture Flammability Standard Appendix II Comments From the Consumer Product Safety Commission Page 23 GAO/HEHS-00-3 Furniture Flammability Standard Appendix II Comments From the Consumer Product Safety Commission Page 24 GAO/HEHS-00-3 Furniture Flammability Standard Appendix II Comments From the Consumer Product Safety Commission Page 25 GAO/HEHS-00-3 Furniture Flammability Standard Appendix II Comments From the Consumer Product Safety Commission Page 26 GAO/HEHS-00-3 Furniture Flammability Standard Appendix II Comments From the Consumer Product Safety Commission Page 27 GAO/HEHS-00-3 Furniture Flammability Standard Appendix II Comments From the Consumer Product Safety Commission (101823) Page 28 GAO/HEHS-00-3 Furniture Flammability Standard Ordering Information The first copy of each GAO report and testimony is free. Additional copies are $2 each. Orders should be sent to the following address, accompanied by a check or money order made out to the Superintendent of Documents, when necessary. VISA and MasterCard credit cards are accepted, also. Orders for 100 or more copies to be mailed to a single address are discounted 25 percent. Orders by mail: U.S. General Accounting Office P.O. Box 37050 Washington, DC 20013 or visit: Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders may also be placed by calling (202) 512-6000 or by using fax number (202) 512-6061, or TDD (202) 512-2537. Each day, GAO issues a list of newly available reports and testimony. To receive facsimile copies of the daily list or any list from the past 30 days, please call (202) 512-6000 using a touchtone phone. A recorded menu will provide information on how to obtain these lists. For information on how to access GAO reports on the INTERNET, send an e-mail message with "info" in the body to: email@example.com or visit GAO’s World Wide Web Home Page at: http://www.gao.gov PRINTED ON RECYCLED PAPER United States Bulk Rate General Accounting Office Postage & Fees Paid Washington, D.C. 20548-0001 GAO Permit No. G100 Official Business Penalty for Private Use $300 Address Correction Requested
Consumer Product Safety Commission: Additional Steps Needed to Assess Fire Hazards of Upholstered Furniture
Published by the Government Accountability Office on 1999-11-17.
Below is a raw (and likely hideous) rendition of the original report. (PDF)