oversight

Consumer Product Safety Commission: Additional Steps Needed to Assess Fire Hazards of Upholstered Furniture

Published by the Government Accountability Office on 1999-11-17.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                United States General Accounting Office

GAO             Report to Congressional Committees




November 1999
                CONSUMER PRODUCT
                SAFETY COMMISSION
                Additional Steps Needed
                to Assess Fire Hazards of
                Upholstered Furniture




GAO/HEHS-00-3
      United States
GAO   General Accounting Office
      Washington, D.C. 20548

      Health, Education, and
      Human Services Division

      B-282713

      November 17, 1999

      The Honorable Christopher Bond
      Chairman
      The Honorable Barbara A. Mikulski
      Ranking Minority Member
      Subcommittee on VA, HUD,
        and Independent Agencies
      Committee on Appropriations
      United States Senate

      The Honorable James T. Walsh
      Chairman
      The Honorable Alan B. Mollohan
      Ranking Minority Member
      Subcommittee on VA, HUD,
        and Independent Agencies
      Committee on Appropriations
      House of Representatives

      In 1994, the Consumer Product Safety Commission (CPSC) began
      preliminary work on setting a flammability standard that would make it
      harder for sofas, chairs, and other upholstered furniture to catch fire. CPSC
      estimated at that time that nearly 700 people died annually from such fires.
      During the 5 years that the agency has been studying the potential
      standard, the need for such a standard has remained an issue of
      considerable debate. Proponents, such as fire protection groups, contend
      that without such a standard, the public is subject to an unnecessary risk.
      Opponents, including upholstered furniture manufacturers and the Small
      Business Administration, respond that the magnitude of the problem is not
      great enough to warrant the risks or added expense involved in treating
      fabrics with flame-retardant chemicals or taking other steps to make
      fabrics more flame-resistant. Opponents also contend that more
      cost-effective solutions may exist, such as making more use of smoke
      detectors to warn when furniture has caught fire.

      To issue a flammability standard, the CPSC commissioners must determine
      that the standard’s benefits bear a reasonable relationship to its costs.1 To
      do this, CPSC assesses in quantitative terms whether the lives and property
      saved would justify the additional expense or risks associated with
      building furniture that complies with the standard. In the conference

      1
       15 U.S.C. 1193 (j)(2)(B) (1994).



      Page 1                              GAO/HEHS-00-3 Furniture Flammability Standard
                       B-282713




                       report for CPSC’s 1999 appropriations act,2 the Congress directed us to
                       review CPSC’s process for establishing a potential flammability standard.3
                       Because CPSC’s work on the potential standard is still in process, we did
                       not conduct a comprehensive analysis of all aspects of the agency’s
                       preliminary computation of the risks, costs, and benefits. As agreed with
                       your offices, we focused our efforts primarily on analyzing CPSC’s approach
                       to quantifying the fire hazards that are the basis for computing the
                       standard’s potential benefits. We addressed the following questions:

                   •   What methodology does CPSC use to estimate the magnitude of the fire
                       hazard that the standard would address?
                   •   How reliable is this methodology for producing sound estimates of the
                       hazard that the standard would address?

                       To answer these questions, we reviewed the process, including the
                       underlying assumptions, that CPSC used to develop its initial estimates of
                       the magnitude of fire hazards from upholstered furniture. We conducted
                       limited tests of the effects of certain assumptions on the estimated
                       benefits of the standard. We also looked at the capacity of CPSC’s
                       methodology to produce sound fire hazard estimates when using the most
                       current data available. We conducted our work between May and
                       October 1999 in accordance with generally accepted government auditing
                       standards. Appendix I describes our scope and methodology in more
                       detail.


                       Because no single national data source exists on the magnitude of hazards
Results in Brief       and losses caused by upholstered furniture fires, CPSC blends information
                       from two different sources. One source provides national estimates on the
                       total number of fires in four general categories and the extent of losses,
                       but it provides no information about specific types of fires, such as
                       upholstered furniture fires. The second source provides detailed
                       information for specific types of fires, but for only a portion of all fires in
                       the United States. CPSC uses the details from the second source of data and
                       the national estimates from the first source to calculate national estimates
                       of fire losses from the kinds of upholstered furniture fires the standard
                       would address.




                       2
                        H.R. Rep. No. 105-769 at 267 (1998).
                       3
                        CPSC’s appropriations act directed the National Academy of Sciences to study the potential
                       toxicological risks of all flame-retardant chemicals identified by CPSC and the Academy as likely
                       candidates for use in making upholstered furniture resistant to ignition.


                       Page 2                                         GAO/HEHS-00-3 Furniture Flammability Standard
                 B-282713




                 At present, CPSC cannot ensure that its methodology provides a complete
                 picture of the national fire losses that the potential standard would
                 address. For example,

             •   CPSC has not developed a statement of precision for the estimated losses
                 from upholstered furniture fires. Without this, CPSC’s estimates of fire
                 losses do not adequately disclose the range of benefits that may be
                 associated with its potential standard.
             •   CPSC’s methodology for calculating fire losses has the effect of including
                 losses that are not likely to be addressed by the standard. Fire losses
                 involving upholstered furniture are counted even though other factors not
                 addressed by the standard may have been responsible, such as fires that
                 are intentionally set. Also, for those fires for which the cause or origin is
                 unknown, CPSC assumes that upholstered furniture fires will occur in the
                 same proportion they occur in fires with a known cause. Our testing
                 shows that these methods are likely to overstate fire losses that the
                 standard would address, and as a result, they could have a material effect
                 on the associated benefits expected from the potential standard. Various
                 analyses can be used to assess the validity of underlying assumptions and
                 ultimately strengthen CPSC estimates, but so far CPSC has not used them.

                 We are recommending that, as CPSC continues to consider the need for a
                 mandatory flammability standard for upholstered furniture, it should
                 conduct additional analyses to identify the level of imprecision in the
                 methodology’s fundamental assumptions and apply any necessary
                 revisions to its cost-benefit analysis of the potential standard.


                 Although CPSC has reported that upholstered furniture fires account for
Background       only 3 percent of all residential fires that occur each year, these fires take
                 a high toll in human life and property damage. CPSC reports that of all the
                 products the agency regulates, upholstered furniture is the leading cause
                 of household fire death. Under the Flammable Fabrics Act,4 which
                 authorizes the issuance of flammability standards for clothing, upholstery,
                 and other products, CPSC has the authority to issue mandatory
                 performance and labeling standards for upholstered products.

                 To address the hazards of upholstered furniture fires, the National
                 Association of State Fire Marshals petitioned CPSC in 1993 to issue a
                 flammability standard for residential upholstered furniture by adopting


                 4
                  15 U.S.C. 1191 (1994).



                 Page 3                              GAO/HEHS-00-3 Furniture Flammability Standard
B-282713




three California standards.5 In 1994, after studying national estimates on
the incidence and severity of household fires, CPSC granted part of the
petition. This part of the petition dealt with small open flames such as
matches and candles. By a 2-to-1 vote, CPSC published an Advance Notice
of Proposed Rulemaking to announce its consideration of issuing a
flammability standard or other regulation to address the risks posed by
these upholstered furniture fires. CPSC commissioners deferred action on
the cigarette portion of the petition in order to first assess the
effectiveness of and industry compliance with an existing voluntary
furniture flammability standard for cigarettes. However, on the basis of
their initial laboratory testing, CPSC staff now believe a small open-flame
standard will also address cigarette-caused fires.6 As a result, they have
taken that assumption into account in developing estimates of the fire
losses the standard would address and the benefits it would produce.

Proponents of a mandatory standard point to its ability to prevent death,
injury, and property damage as the major benefits. Opponents recognize
that these losses should be avoided, but they believe the potential costs
associated with the proposed standard are too great. Possible costs cited
include those related to health risks—both to employees of furniture
manufacturers and to consumers—posed by flame-retardant chemicals
that may be used; increased prices consumers would have to pay for
upholstered furniture to cover a variety of additional manufacturing costs;
diminished feel and texture of fabric treated with flame-retardant
chemicals; and loss of consumer choice because some materials may be
eliminated if they cannot be made flame-resistant.

CPSC activities conducted since the Advance Notice of Proposed
Rulemaking have included in-depth fire investigations, technical analyses
and laboratory tests, development of a draft standard, and initial estimates
of the standard’s potential costs and benefits. CPSC’s 1999 appropriations
act prevents the standard from proceeding past this stage until CPSC fully
considers the fundings and conclusions of the National Academy of
Sciences’ congressionally mandated review on the potential toxicity of
flame-retardant chemicals. This review is to be completed in January 2000.




5
 California has standards for upholstered furniture fires started by large open flames in public
buildings that lack a sprinkler system, upholstered furniture fires started by smoking materials, and
those started by small open flames, such as candles. Separate standards were requested for small open
flames and cigarettes because some materials have different probabilities of catching fire, depending
on the heat source.
6
 CPSC staff said that, for this reason, they will not take further action on the cigarette portion of the
petition until after the technical work for the small-open-flame portion is completed.


Page 4                                            GAO/HEHS-00-3 Furniture Flammability Standard
                        B-282713




                        To estimate the incidence of fires and losses resulting from upholstered
CPSC Uses Two Data      furniture being ignited by cigarettes or small open flames, CPSC uses
Sources to Estimate     aggregate fire data from a national survey, detailed data on a portion of
the Magnitude of the    individual fires, and a methodology to combine the two databases. This
                        approach is necessary because no single data source exists that provides
Upholstered Furniture   the information needed to estimate the magnitude of the problem the
Fire Problem            standard is intended to address. The aggregate survey data provide
                        national fire loss estimates for four general categories of fires (such as
                        residential or vehicle fires) but provide no information about detailed
                        characteristics of the fires. This information is from a survey conducted by
                        the National Fire Protection Association (NFPA), a nonprofit fire protection
                        association. In contrast, the second source consists of detailed
                        information linking fire losses—deaths, injuries, and damages—with
                        specific types of fires, such as those originating in upholstered furniture.
                        However, the detailed data do not provide national totals and are not
                        collected in a way that permits them to be used by themselves for national
                        projections. The detailed data are collected from more than a third of the
                        nation’s fire departments and are found in the National Fire Incident
                        Reporting System (NFIRS), a federal database. This database is maintained
                        by the U.S. Fire Administration (USFA), within the Federal Emergency
                        Management Agency. The NFIRS data contain information on more than 20
                        different fire characteristics, such as the source of the fire and what
                        caught fire first.

                        Because neither NFPA estimates nor the NFIRS data can produce a
                        nationwide estimate of fire losses for specific types of fires, CPSC uses a
                        methodology for combining the two. The process of blending the two
                        sources of data was developed jointly by NFPA, CPSC, and USFA to
                        approximate and report on fire trends for the fire-fighting community and
                        the general public. CPSC, NFPA, and USFA use this general process to track
                        and report on fires and fire losses nationally. They have all used this
                        process for many years and try to apply it consistently so that conflicting
                        estimates are not produced, which could confuse the public and those
                        who use the data.

                        The methodology is based on the proportional relationship of the NFIRS
                        data to the NFPA estimates. If NFIRS data contain half of the total number of
                        fires estimated by NFPA, for example, NFIRS numbers for specific types of
                        fires are doubled to produce national estimates for these specific fires. The
                        NFIRS data also include a number of fires for which the first item ignited is
                        unknown. The methodology adds a proportional number of these fires and
                        fire losses to the NFIRS fires known to have started in upholstered



                        Page 5                             GAO/HEHS-00-3 Furniture Flammability Standard
                          B-282713




                          furniture. In the 1994 NFIRS data, for example, upholstered furniture fires
                          constituted 3 percent of all fires with a known item of origin. Therefore,
                          3 percent of the fires with an unknown item of origin also would be
                          allotted to upholstered furniture fires.


                          CPSC  has not fully addressed the uncertainty surrounding key data and
Extent of Fire Losses     underlying assumptions that it uses in developing national fire loss
Is Uncertain              estimates. Particularly important are (1) the need to account for
                          imprecision surrounding the data used to make national fire loss estimates
                          and (2) the assumption that the potential standard could address all
                          upholstered furniture fires classified as being ignited by either small open
                          flames or cigarettes. So far, CPSC has not accounted for this imprecision or
                          tested the soundness of this assumption, and as a result, the full range of
                          fire losses is not known. Our limited testing demonstrates the importance
                          of these steps in developing sound estimates of fire losses that the
                          standard is designed to address.


CPSC Has Not Tested Key   Several issues raise uncertainties about the underlying assumptions
Issues                    related to the NFPA and NFIRS data (see table 1). These issues include the
                          low response rates to the NFPA survey, the voluntary nature of reporting
                          NFIRS data, and the fact that some upholstered furniture fires started by
                          small open flames or cigarettes will not be addressed by the standard.
                          These issues raise questions about whether the fire and fire loss data
                          provided by NFIRS and NFPA result in a representative picture of the
                          incidence and severity of residential fires that occur nationally. In relying
                          on these data for standard-setting, it is important to understand whether
                          the data are representative and, if they are not, whether they overestimate
                          or underestimate the national fire problem.




                          Page 6                              GAO/HEHS-00-3 Furniture Flammability Standard
                                  B-282713




Table 1: Key Issues Potentially
Affecting the Validity of CPSC                                                                             Subject of
Estimates                                                                                                  preliminary
                                  Assumption                       Issues that raise uncertainty           testing by GAO
                                  NFPA survey data
                                  NFPA national fire and fire      Low response rate (21 percent of fire Yes
                                  loss estimates are               departments surveyed)a and limited
                                  representative of fires          review of nonrespondents results in
                                  nationwide.                      uncertainty about extent that national
                                                                   projections accurately represent the
                                                                   national fire problem.
                                                                   No outside review of the analysis       No
                                                                   used to produce national estimates
                                                                   was conducted. Such a review,
                                                                   usually conducted by a party with a
                                                                   background in methodology and
                                                                   statistics, is commonly used to help
                                                                   identify flaws or constraints in
                                                                   estimating methodologies.
                                                                   Potential exists for survey             No
                                                                   respondents to provide inaccurate
                                                                   information. Limited follow-up or
                                                                   corroboration of survey responses is
                                                                   a way to ascertain the degree to
                                                                   which these responses are accurate
                                                                   and supported.
                                  NFIRS data
                                  Any upholstered furniture fire   Some upholstered furniture fires that Yes
                                  classified as being ignited by   were started by cigarettes or small
                                  a small, open flame or           open flames also involve other
                                  cigarette could be addressed     factors that place them outside the
                                  by the potential standard.       category of fires the standard
                                                                   addresses, raising concern that too
                                                                   many fire losses may be linked to the
                                                                   potential standard.
                                  Fire and fire loss data          Voluntary reporting carries the         Yes
                                  reported by local fire           potential that certain types of fire
                                  departments are                  departments, such as large fire
                                  representative of fires that     departments, will be over- or
                                  occur nationally.                underrepresented in the data, or will
                                                                   only report major fires, which would
                                                                   affect the extent to which data are
                                                                   representative of the fire problem
                                                                   nationally.
                                                                   There is potential for data reported    No
                                                                   by local fire departments to be
                                                                   inaccurate. Limited verification and
                                                                   corroboration of reported data is a
                                                                   way to ascertain the level of
                                                                   accuracy of the reported data.

                                                                                                      (Table notes on next page)




                                  Page 7                                     GAO/HEHS-00-3 Furniture Flammability Standard
                                  B-282713




                                  a
                                   Fire departments protecting a larger number of people (over 50,000) responded to the survey at
                                  a rate more than twice that of departments protecting fewer people (under 50,000). As a result,
                                  NFPA reported that it received responses from fire departments that serve 40 percent of the
                                  population.




Testing of Issues Showed          We selectively tested three key issues that affect the validity of CPSC’s
Potential for Inaccuracy          estimates:

                              •   the extent to which the data used to project NFPA estimates provide
                                  nationally representative and precise estimates,
                              •   the extent to which data on fire losses that the potential standard does not
                                  address are included in developing estimates of fire losses the standard is
                                  expected to address, and
                              •   the extent to which fire departments in the NFIRS database are
                                  representative of fire departments nationally and provide nationally
                                  representative fire data.

                                  Our selective testing showed mixed results, indicating that CPSC needs to
                                  more fully analyze some of its assumptions in order to provide greater
                                  assurance that its fire loss estimates are valid.

Representativeness and            Because of the overall low response rate to the NFPA survey (21 percent),
Precision of NFPA Estimates       we conducted a limited test to assess the accuracy of NFPA estimates.
                                  Corroborating survey projections to another data source is a common way
                                  of assessing the representativeness of survey data. This type of
                                  comparison can be a general gauge of how well the survey represents the
                                  nation as a whole. To do this, we compared the NFPA death estimates with
                                  the Centers for Disease Control and Prevention (CDC) death estimates. CDC
                                  obtains this information from death certificates recorded in the 50 states
                                  and the District of Columbia. The data classify deaths by the underlying
                                  cause of death, which is determined from the death certificate
                                  information.7 Our testing indicated that CDC death data fall within the
                                  bounds of NFPA’s estimates. On the basis of this general test, the data
                                  appear to provide representative estimates of fire deaths. We did not
                                  assess the representativeness of the NFPA national estimates of injuries and
                                  property damage from fires.

                                  To provide a complete picture of fire losses, CPSC needs to account for the
                                  range of precision around NFPA’s estimates. Because the estimates are


                                  7
                                   While these data may not be 100-percent accurate either, they are subject to numerous tests and
                                  checks and are widely used within the health research community.



                                  Page 8                                         GAO/HEHS-00-3 Furniture Flammability Standard
                             B-282713




                             based on a survey, they have a margin of error that needs to be
                             incorporated into national upholstered furniture fire loss estimates. Survey
                             researchers routinely report the margin of error in the results of their
                             surveys and polls. CPSC has not accounted for the margin of error around
                             NFPA estimates for deaths, injuries, and property damage. Although CPSC
                             stated that the margin of error for the total number of fires (2.5 percent)
                             was not significant for its estimates, we found that the margins of error for
                             deaths and injuries were considerably larger than the margin of error for
                             the total number of fires. In 1997, the largest margin of error was for the
                             estimated number of deaths, which could be off by as much as 350 deaths
                             (10 percent) in either direction. Because deaths, injuries, and property
                             damages are the factors for estimating losses from the kinds of fires
                             covered under a potential standard, they all need to be considered. By not
                             obtaining and using all of this information in developing its estimates of
                             upholstered furniture fire losses, CPSC runs the risk of conveying a false
                             sense of precision about its results.

CPSC’s Calculation of Fire   CPSC’s methodology for estimating fire losses that the potential standard is
Losses                       expected to address also warrants additional refinement. First, in
                             estimating total fire losses, CPSC identifies all fires in NFIRS data known to
                             have involved the ignition of upholstered furniture by small open flames or
                             cigarettes.8 Second, when NFIRS data lack complete information to link fire
                             losses to a specific type of fire, CPSC classifies a portion of these fires as
                             upholstered furniture fires involving small open flames or cigarettes. Our
                             work shows that both of these procedures are likely to overstate the fire
                             losses the potential standard is designed to address, and as a result, they
                             have a material effect on the benefits ascribed to the standard.

                             With regard to counting fires for which the origin and cause are known,
                             CPSC includes upholstered furniture fires that were classified in NFIRS as
                             being started by small open flames or cigarettes. However, our
                             examination of the same data shows that other factors were involved in
                             the cause of some of these fires, making them fires the standard would not
                             appear to address. For example, the standard is not designed to address
                             fires intentionally set or those in which a small open flame ignites a paper
                             or flammable liquid that is on or near an upholstered couch that in turn
                             ignites the couch. CPSC, however, has not yet excluded these types of fires
                             in its estimate.




                             8
                              As pointed out earlier, CPSC has determined that the potential standard will address both
                             small-open-flame and cigarette fires, and as a result, CPSC estimates fire losses for both types of fires.



                             Page 9                                           GAO/HEHS-00-3 Furniture Flammability Standard
B-282713




CPSC’s allocation of unknown fire losses also warrants additional attention.
In 1997, about 35 percent of the residential fire deaths in the NFIRS data
were not identified by what first caught fire, such as upholstered furniture,
draperies, chimneys, or walls. When NFIRS data lack complete information
to link fire losses to a specific type of fire, CPSC allocates these losses on
the basis of how the fire loss data are distributed across all fires in which
the specific type of fire is known. In other words, if 19 percent of all
residential fire deaths are attributed to upholstered furniture, the same
percentage of deaths that stem from fires not attributed to specific types
of fires is assigned to upholstered furniture fires. CPSC officials told us they
proportionally allocated unknown fire loss data because they had no
evidence that the data should be allocated differently.

Methods and data exist that could categorize the origin and cause of fires
better before calculating the fire losses the standard could possibly
address. The NFIRS data are useful in this regard because information
available in NFIRS allows certain types of fires to be ruled out as likely to be
addressed by the standard. As a result, the NFIRS data can be used to test
the effect of CPSC’s method of calculating total fire loss data.

We tried such a test using NFIRS data for 1997. We analyzed detailed
information on fires—those known to be upholstered furniture fires
started by a small open flame or cigarette and those with an unknown
origin or cause—looking for characteristics to identify fires that the
potential standard would not appear to address. For example, fires we
recategorized include those that were intentionally set, those involving
electrical short circuits, and those involving flammable materials not used
to upholster furniture (such as gases, flammable liquids, or cardboard).
Our reclassification of fires reduced CPSC’s estimate of fire losses that the
standard is designed to address by up to 152 deaths, 434 injuries, and
$70 million in property damages (see app. I for more details on these
reductions). Adjusting for this reclassification of fires could reduce the
benefits CPSC attributes to the standard from $885 million to $610 million,
or 31 percent of the total benefits. On the other hand, additional and more
thorough investigation of some fires by CPSC could identify other factors
that show that some of these fire losses would be prevented by the
standard.

Also, our test should not be taken as the most definitive or comprehensive
approach to this issue. For example, more comprehensive analytical
approaches to assess and allocate unknown fire losses could yield results
that would make the estimates more precise. Our selective testing was



Page 10                              GAO/HEHS-00-3 Furniture Flammability Standard
                             B-282713




                             done mainly to determine whether the underlying assumption of CPSC’s
                             approach appears sound. Our results indicate that some of CPSC’s current
                             assumptions have a level of uncertainty that is material and as such
                             warrants attention.


Representativeness of the    NFIRS  data are based on reports submitted by a little more than a third of
Fire Departments That        the 31,000 fire departments nationwide. Because the fire departments
Report NFIRS Data            reporting to NFIRS do so voluntarily, it is important to determine how
                             representative this group is of fire departments and fire experiences as a
                             whole. For example, 11 states have no fire departments reporting
                             information to NFIRS, and only a portion of the fire departments in the
                             other states participate. If certain types of fire departments, such as large
                             urban fire departments, are more likely or less likely to report to NFIRS, or
                             if fire departments tend to report only major fires, this could skew
                             estimates of the fire problem.

                             Testing the extent that NFIRS data are representative of fire and fire losses
                             that occur nationally can be conducted in several ways. We did so by
                             assessing the extent to which the types of fire departments reporting data
                             to NFIRS resembled the types of fire departments nationally on the basis of
                             two factors: the size of the fire department and whether it is
                             predominantly a volunteer or a paid fire department. This analysis
                             indicated that NFIRS-reporting fire departments fall into the various fire
                             department size and type categories at roughly the same proportions as all
                             fire departments across the country. As a result, any differences in fire
                             losses that is related to fire department size or type are probably
                             appropriately represented in the NFIRS data.


Effect of Selected Testing   Although the selected testing we conducted resulted in a decline in
on Cost-Benefit Estimates    estimated benefits expected from the potential standard, the tests should
                             not be taken as a comprehensive assessment of CPSC’s overall cost-benefit
                             calculation. For example, we did not review or evaluate CPSC’s
                             assumptions about the effectiveness of the potential standard in
                             preventing upholstered furniture fires ignited by small open flames or
                             cigarettes, which also have a major impact on the estimated benefits of the
                             standard. We also did not assess the validity or scientific basis for CPSC’s
                             assumptions concerning the extent to which a small-open-flame standard
                             will also address cigarette-caused fires. CPSC estimated that the potential
                             standard would prevent 80 percent of the small-open-flame losses and




                             Page 11                             GAO/HEHS-00-3 Furniture Flammability Standard
                      B-282713




                      50 percent of the cigarette-related losses.9 However, this key assumption is
                      subject to further laboratory testing by CPSC.

                      Also, as stated earlier we did not analyze assumptions and methodologies
                      used to estimate the costs of the standard. It is important to recognize that
                      the testing of other issues and assumptions—especially those relating to
                      costs—may offset, at least in part, the decline in estimated potential
                      benefits that our testing found.10 As a result, we present no estimates of
                      the effect our testing has on tne net costs or benefits of the potential
                      standard. Rather, our testing demonstrates the need for CPSC to further
                      refine its analysis.


                      To promulgate a mandatory standard, the CPSC commissioners must
Conclusions           determine that benefits produced by the standard bear a reasonable
                      relationship to its costs. A high degree of assurance about the soundness
                      of these estimates is an important part of accurately assessing this
                      relationship. As matters stand, CPSC’s current approach is not likely to
                      generate the necessary degree of assurance. To provide this assurance,
                      CPSC needs to demonstrate, to a greater degree than now exists, the
                      validity of the assumptions on which the estimate is based. This will
                      require testing that so far has not been part of CPSC’s approach.


                      To resolve issues surrounding the data and assumptions used in preparing
Recommendations       the cost-benefit analysis for a potential standard to protect against fire
                      hazards associated with upholstered furniture, we recommend that the
                      Chairman, CPSC, direct CPSC staff to conduct additional and more detailed
                      analyses of key assumptions including, but not limited to,

                  •   assessing the precision surrounding NFPA national fire loss estimates and
                      their impact on estimated benefits attributable to the standard and
                  •   identifying a more accurate method to calculate fire losses that could be
                      addressed by the standard.

                      We also recommend that any necessary revisions identified by these
                      analyses should be incorporated into the cost-benefit analysis of the
                      potential standard.


                      9
                       In applying this rate of effectiveness, CPSC also reduced cigarette-related losses by an additional
                      24 percent to account for the effect of the existing voluntary industry flammability standard.
                      10
                          It is also possible that this testing would lower the estimates even further.



                      Page 12                                             GAO/HEHS-00-3 Furniture Flammability Standard
                       B-282713




                       In its comments on our draft report, CPSC agreed with our
Agency Comments        recommendations. It stated that CPSC staff have already begun to address
and Our Response       the issues we raised as they refine their fire loss estimates for upholstered
                       furniture. (See app. II for the complete text of CPSC’s comments.)

                       However, CPSC stated that the report overall downplayed the positive
                       findings about CPSC data and overemphasized the data problems. In fact,
                       after reviewing our report, CPSC provided us with new information and
                       perspective on the representativeness of the NFPA data. After considering
                       this additional information, we modified our draft to agree with CPSC that
                       our limited testing did not disclose any obvious data problems. However,
                       this does not indicate that the data are fully reliable, because our testing
                       did not address all the issues surrounding the underlying data. For
                       example, NFPA and NFIRS data are based on data reported by individual fire
                       departments that have not been subject to verification or corroboration.
                       As a result, we believe our report, as modified, presents an appropriate
                       characterization of our results and the remaining uncertainties
                       surrounding the underlying data.

                       CPSC also commented that our analysis overstated the significance of the
                       issues surrounding NFIRS data. CPSC concluded that it was not appropriate
                       for us to quantify the effect of these data problems and said that our
                       analysis was flawed for two reasons:

                   •   Our analysis eliminated those fires that available information indicated
                       were inconsistent with the scope of the potential standard. However, CPSC
                       stated that this process could result in eliminating too many fires.
                   •   Our analysis erroneously adjusted the fire data where the cause was
                       unknown. CPSC commented that we did not include the correct proportion
                       of fires with unknown origin or source in arriving at a total number of fires
                       that the potential standard is designed to address.

                       We do not agree with CPSC’s conclusion. First, our approach was to
                       determine whether data classification issues could have a material effect
                       on CPSC’s estimated losses that the potential standard would address. We
                       believe our analysis was a reasonable one for this purpose and that it
                       shows a likelihood of significant effect that warrants attention. The report
                       appropriately qualifies the results and acknowledges that our tests should
                       not be taken as the most definitive or comprehensive approach. Rather, it
                       is CPSC’s responsibility to develop more precise estimates, as it proceeds in
                       its rulemaking process.




                       Page 13                             GAO/HEHS-00-3 Furniture Flammability Standard
B-282713




Second, we believe that our approach properly distributed the number of
“unknown” fires to the total number of fires that the potential standard is
designed to address. In our opinion CPSC’s approach overstates the true
proportion because it classifies as unknown some fires that available data
indicate are not subject to the proposed regulation. Consequently, our
methodology corrects the data to account for the ”greater chance of
containing upholstered furniture cases.” In making this adjustment,
contrary to CPSC’s comments, we did not eliminate fires from
consideration. Rather, we reclassified fires as known not to be
upholstered furniture fires that the potential standard is designed to
address. We than recalculated the proportion of unknown fire losses on
the basis of our reclassification.

Finally, CPSC commented that our recalculation of its estimated fire losses,
adjusted for reclassification of unknown fire origin or cause, would
overstate the reduction in potential benefits. Our report indicates that,
while this recalculation is an upper-bound estimate for this factor, further
investigation and analysis could also influence the magnitude of estimated
fire losses. CPSC’s continuing efforts to refine its analysis—of estimated
costs as well as benefits associated with its proposed standard—will be
central to its ongoing work to determine the merits of proceeding with a
mandatory flammability standard. In addition to CPSC’s written comments,
CPSC staff provided us with oral comments, which we incorporated, where
appropriate, in the final report.


We are sending copies of this report to the Honorable Ann Brown,
Chairman, and the Honorable Thomas H. Moore and the Honorable Mary
Sheila Gall, Commissioners, CPSC; and appropriate congressional
committees. We will also make copies available to others upon request.

If you or your staffs have any questions about this report, please contact
me at (202) 512-7118 or Frank Pasquier at (206) 287-4861. Major
contributors to this report include Tim S. Bushfield, Evan Stoll, and Stan
Stenersen.




Kathryn G. Allen
Associate Director, Health Financing
  and Public Health Issues

Page 14                            GAO/HEHS-00-3 Furniture Flammability Standard
Page 15   GAO/HEHS-00-3 Furniture Flammability Standard
Contents



Letter                                                                                           1


Appendix I                                                                                      18
                    Range of Information Sources Used                                           18
Scope and           Analyzing the Methodology for Making National Fire Loss                     18
Methodology           Estimates
                    Testing the Validity of Underlying Assumptions and the                      19
                      Soundness of National Estimates

Appendix II                                                                                     21

Comments From the
Consumer Product
Safety Commission
Tables              Table 1: Key Issues Potentially Affecting the Validity of CPSC               7
                      Estimates
                    Table I.1: Reduction in Fire Loss Estimates Based on                        20
                      Recategorization of Fires that the Standard Is Not Designed to
                      Address




                    Abbreviations

                    CDC        Centers for Disease Control and Prevention
                    CPSC       Consumer Product Safety Commission
                    NFIRS      National Fire Incident Reporting System
                    NFPA       National Fire Protection Association
                    USFA       U.S. Fire Administration


                    Page 16                           GAO/HEHS-00-3 Furniture Flammability Standard
Page 17   GAO/HEHS-00-3 Furniture Flammability Standard
Appendix I

Scope and Methodology


                           Our work focused on the methodology and data that the Consumer
                           Product Safety Commission (CPSC) is using to estimate national fire losses
                           from the kinds of upholstered furniture fires that would be covered by the
                           potential flammability standard now being considered. We used a variety
                           of sources and methods to gather and analyze data on this issue.


                           We obtained information from a wide variety of agencies, interest groups,
Range of Information       and other sources. Among the key sources were the following:
Sources Used
                       •   CPSC  officials, including those responsible for calculating national fire loss
                           estimates and estimating the potential benefits that would result from the
                           standard;
                       •   National Fire Protection Association (NFPA) officials responsible for
                           conducting NFPA’s annual fire surveys, calculating the survey’s national fire
                           and fire loss estimates, and involved in developing the methodology that
                           combines NFPA’s survey projections with detailed fire data collected by
                           local fire departments;
                       •   U.S. Fire Administration officials responsible for overseeing and analyzing
                           the detailed fire and fire loss data reported to the agency’s National Fire
                           Incident Reporting System (NFIRS);
                       •   officials from the National Center for Health Statistics and the National
                           Center for Injury Prevention and Control at the Centers for Disease
                           Control and Prevention (CDC);
                       •   officials representing various segments of the upholstered furniture
                           manufacturing industry, such as fiber manufacturers and upholstered
                           furniture manufacturers; and
                       •   private consultants who use the NFPA and NFIRS data to estimate national
                           fire problems.

                           Our work also included a review of relevant legislation and other
                           documents related to CPSC’s approach to considering the potential
                           standard.


                           To address the study’s first question (the methodology used to estimate
Analyzing the              the magnitude of the fire hazard that would be addressed by the standard),
Methodology for            we obtained documentation describing the process and examples
Making National Fire       illustrating the data and major steps in the process from CPSC, NFPA, and
                           the U.S. Fire Administration. In addition to reviewing documents and
Loss Estimates             conducting interviews to learn how these estimates had been developed,
                           we also obtained and analyzed the 1997 NFIRS database of actual fire



                           Page 18                             GAO/HEHS-00-3 Furniture Flammability Standard
                          Appendix I
                          Scope and Methodology




                          incidents as well as detailed information about the 1997 NFPA survey,
                          including some of the characteristics of the fire departments that
                          responded to it.


                          To address the study’s second question (the reliability of this methodology
Testing the Validity of   to produce sound fire hazard estimates), we first identified the
Underlying                assumptions underlying key data and methodological steps in the process.
Assumptions and the       We reviewed each data source separately to identify the analytical and
                          statistical limitations accompanying the assumptions on which the
Soundness of National     information was based. When we identified limitations, we compared
Estimates                 CPSC’s process to date with various analytical and statistical methods for
                          testing reliability. For example, we assessed the potential implication of
                          NFPA’s response rate on the soundness of the data, determined what
                          statistical techniques could help assess the effect of these uncertainties,
                          and spoke with CPSC staff to determine the extent to which they had
                          conducted their own tests of these uncertainties. We focused more
                          detailed testing on three specific assumptions: the representativeness of
                          data used to make NFPA’s national fire estimates; the representativeness of
                          data in NFIRS; and CPSC’s classification of fires the standard would address,
                          including fires with and without a known item of origin or ignition source.

                          To assess the representativeness of the data used in the national fire
                          estimates, we conducted two types of analysis. First, we compared NFPA’s
                          national estimate of total fire deaths with the number of national fire
                          deaths compiled by CDC. Because CDC obtains information from all death
                          certificates in all 50 states and the District of Columbia, we concluded that
                          the data were a valid basis for comparison. Second, we obtained from NFPA
                          information on the margin of error around its national estimates of fires,
                          deaths, injuries, and property damages.

                          To assess the representativeness of data in NFIRS, we compared the
                          distribution of NFIRS fire departments by size with the distribution of fire
                          departments nationally. We did so by categorizing fire departments by
                          using the size of the population they protected as a proxy for fire
                          department size. To do this, we needed more data than were available in
                          NFIRS. We matched NFIRS fire departments with the NFPA database and used
                          the NFPA population data in those cases where a fire department appeared
                          on both databases. We also conducted a similar analysis comparing the
                          distribution of fire departments based on the type of fire fighting
                          personnel employed (paid or volunteer) and the severity of fires reported
                          by each category of fire department.



                          Page 19                            GAO/HEHS-00-3 Furniture Flammability Standard
                                      Appendix I
                                      Scope and Methodology




                                      To evaluate CPSC’s calculation of fire losses expected to be addressed by
                                      the potential standard, we used data available in NFIRS to recategorize fires
                                      the standard is not designed to address. We conducted this analysis using
                                      information from CPSC and 1997 NFIRS data. To identify those fires the
                                      potential standard is not designed to address, we used other information
                                      in the NFIRS database such as the type of material ignited first, the situation
                                      that resulted in contact between a heat source and flammable material, the
                                      equipment involved in the ignition, and the area of the house where the
                                      fire started.

                                      As a result of these analyses, adjustments were made to (1) the raw
                                      number of upholstered furniture fires and fire losses known to have
                                      resulted from small open flames or cigarettes; (2) the raw number of fires
                                      with an unknown origin and/or heat source that are allocated to
                                      upholstered furniture fires the standard is intended to address; and (3) the
                                      weights, or multipliers, used to project the raw NFIRS numbers into national
                                      estimates. Table I.1 shows the effect of these adjustments on fire loss
                                      estimates that the standard is designed to address.

Table I.1: Reduction in Fire Loss
Estimates Based on Recategorization                               Recategorized       Recategorized
of Fires That the Standard Is Not                                    fires with a       fires with an
Designed to Address                                             known source or     unknown source
                                      Fire loss type                       origin           or origin             Total
                                      Deaths                                  87                  65                152
                                      Injuries                               296                 138                434
                                      Property damage                 $48 million         $22 million        $70 million

                                      To determine the impact these adjustments have on CPSC’s estimate of
                                      potential benefits, we used a two-step process. First, we compared the
                                      total cost of fire losses based on national estimates resulting from our
                                      adjustments with the total costs from CPSC’s national fire loss estimates.
                                      Then we applied the difference between CPSC’s and our total cost estimates
                                      to CPSC’s estimate of potential benefit from the standard.

                                      A general limitation of our analysis was that it did not review the scientific
                                      basis for the effectiveness of the potential standard. Laboratory testing to
                                      show the extent that the standard would prevent different types of
                                      upholstered furniture fires is ongoing, according to CPSC. Rather, the scope
                                      of our work was limited to reviewing whether the methodology for using
                                      existing data was sufficiently reliable to produce sound fire loss estimates.




                                      Page 20                              GAO/HEHS-00-3 Furniture Flammability Standard
Appendix II

Comments From the Consumer Product
Safety Commission




              Page 21    GAO/HEHS-00-3 Furniture Flammability Standard
Appendix II
Comments From the Consumer Product
Safety Commission




Page 22                              GAO/HEHS-00-3 Furniture Flammability Standard
Appendix II
Comments From the Consumer Product
Safety Commission




Page 23                              GAO/HEHS-00-3 Furniture Flammability Standard
Appendix II
Comments From the Consumer Product
Safety Commission




Page 24                              GAO/HEHS-00-3 Furniture Flammability Standard
Appendix II
Comments From the Consumer Product
Safety Commission




Page 25                              GAO/HEHS-00-3 Furniture Flammability Standard
Appendix II
Comments From the Consumer Product
Safety Commission




Page 26                              GAO/HEHS-00-3 Furniture Flammability Standard
Appendix II
Comments From the Consumer Product
Safety Commission




Page 27                              GAO/HEHS-00-3 Furniture Flammability Standard
           Appendix II
           Comments From the Consumer Product
           Safety Commission




(101823)   Page 28                              GAO/HEHS-00-3 Furniture Flammability Standard
Ordering Information

The first copy of each GAO report and testimony is free.
Additional copies are $2 each. Orders should be sent to the
following address, accompanied by a check or money order
made out to the Superintendent of Documents, when
necessary. VISA and MasterCard credit cards are accepted, also.
Orders for 100 or more copies to be mailed to a single address
are discounted 25 percent.

Orders by mail:

U.S. General Accounting Office
P.O. Box 37050
Washington, DC 20013

or visit:

Room 1100
700 4th St. NW (corner of 4th and G Sts. NW)
U.S. General Accounting Office
Washington, DC

Orders may also be placed by calling (202) 512-6000
or by using fax number (202) 512-6061, or TDD (202) 512-2537.

Each day, GAO issues a list of newly available reports and
testimony. To receive facsimile copies of the daily list or any
list from the past 30 days, please call (202) 512-6000 using a
touchtone phone. A recorded menu will provide information on
how to obtain these lists.

For information on how to access GAO reports on the INTERNET,
send an e-mail message with "info" in the body to:

info@www.gao.gov

or visit GAO’s World Wide Web Home Page at:

http://www.gao.gov




PRINTED ON    RECYCLED PAPER
United States                       Bulk Rate
General Accounting Office      Postage & Fees Paid
Washington, D.C. 20548-0001           GAO
                                 Permit No. G100
Official Business
Penalty for Private Use $300

Address Correction Requested