oversight

Consumer Product Safety Commission: Better Data Needed to Help Identify and Analyze Potential Hazards

Published by the Government Accountability Office on 1997-09-29.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                  United States General Accounting Office

GAO               Report to the Chairmen, Committee on
                  Commerce, Science, and Transportation,
                  U.S. Senate, and the Committee on
                  Commerce, House of Representatives

September 1997
                  CONSUMER PRODUCT
                  SAFETY COMMISSION
                  Better Data Needed to
                  Help Identify and Analyze
                  Potential Hazards




GAO/HEHS-97-147
      United States
GAO   General Accounting Office
      Washington, D.C. 20548

      Health, Education, and
      Human Services Division

      B-274042

      September , 1997

      The Honorable John McCain
      Chairman, Committee on Commerce,
        Science, and Transportation
      United States Senate

      The Honorable Thomas J. Bliley, Jr.
      Chairman, Committee on Commerce
      House of Representatives

      Every year, children are hurt while using playground equipment; users or
      spectators are injured when fireworks explode unexpectedly; and
      homeowners are cut while operating chain saws and lawn mowers.
      Created to protect consumers from “unreasonable risk of injury,” the U.S.
      Consumer Product Safety Commission (CPSC) oversees these and about
      15,000 other consumer products ranging from kitchen appliances and
      children’s toys to hot tubs and garage door openers. With a budget of
      about $42.5 million, CPSC carries out its mission by (1) enforcing federal
      consumer product regulations (by recalling products from store shelves,
      for example) and (2) conducting projects to address products with
      potential hazards not covered by existing regulations.1 These projects may
      result in CPSC issuing new regulations concerning specific products,
      assisting in the development of voluntary industry standards, or providing
      information to consumers about how to use the products safely.

      Contending that the agency is ineffectively allocating its resources, CPSC’s
      critics have voiced dissatisfaction with the selection of certain agency
      projects and have questioned the validity of CPSC’s risk assessment and
      cost-benefit analyses supporting those projects. In addition, congressional
      and interest group critics have questioned the agency’s procedures for
      ensuring the accuracy of manufacturer-specific information before
      releasing it to the public,

      contending that such releases can mar the reputation of responsible
      corporate citizens.

      In light of these concerns, you asked us to review CPSC’s project selection,
      use of cost-benefit analysis and risk assessment, and information release
      procedures. Specifically, this report (1) identifies the criteria CPSC uses to

      1
       Projects vary widely in scope, and CPSC has no standard definition of what constitutes a project. For
      our review, we defined a “project” as work CPSC conducted on any specific consumer product that
      was associated with a potential hazard or hazards not covered by existing regulation.



      Page 1                                 GAO/HEHS-97-147 Consumer Product Safety Commission
                   B-274042




                   select projects and reviews the information it relies upon in making these
                   choices, (2) assesses the information CPSC draws on to perform risk
                   assessment and cost-benefit analyses and evaluates the agency’s
                   methodology for conducting cost-benefit analyses, and (3) describes CPSC’s
                   procedures for releasing manufacturer-specific information to the public
                   and reports whether evidence exists that CPSC violated its statutory
                   requirements concerning the release of such information.

                   To address these objectives, we reviewed internal CPSC documents,
                   relevant legislation and regulations, and the literature on cost-benefit
                   analysis and on consumer product safety issues. We interviewed CPSC
                   commissioners and staff, four former commissioners, consumer
                   advocates, industry representatives, and outside experts to obtain their
                   perspectives on CPSC’s work. We identified CPSC projects by compiling from
                   various agency documents a list of 115 potential product hazards
                   examined by the agency from January 1990 to September 30, 1996, and we
                   reviewed available agency documentation on each of these projects. We
                   examined the agency’s internal databases to obtain project information
                   and to assess the agency’s information on product hazards. In reviewing
                   CPSC’s cost-benefit analyses, we consulted with experts to develop
                   objective criteria that reflected elements commonly used in the evaluation
                   of cost-benefit analyses. These evaluation questions were designed to elicit
                   whether CPSC conducted comprehensive analyses and reported them in
                   sufficient detail. However, they do not make up a complete measure of the
                   quality of a cost-benefit analysis; for example, our evaluation assessed
                   whether these elements were included but not how well they were
                   measured or incorporated. We reviewed CPSC’s internal procedures
                   concerning information clearance and release, and we examined relevant
                   legal cases in this area. For more detailed information on our scope and
                   methodology, see appendix I.


                   Although CPSC has established criteria to help select new projects, with the
Results in Brief   agency’s current data these criteria can be measured only imprecisely, if at
                   all. The criteria for selecting projects include the number of
                   product-related injuries, chronic illnesses, and deaths. However, although
                   CPSC has described itself as “data driven,” its information on
                   product-related injuries and deaths is often sketchy. For example, because
                   the agency’s measure of injuries generally includes only hospital
                   emergency room reports, CPSC has an incomplete picture of injuries.
                   Similarly, CPSC’s data on product-related deaths understate the total
                   number of deaths and are available to the agency only with a 2-year time



                   Page 2                       GAO/HEHS-97-147 Consumer Product Safety Commission
B-274042




lag. In addition, CPSC does not maintain a comprehensive list or description
of either its past or ongoing projects. This makes it more difficult not only
for agency management to monitor current projects but also for staff and
commissioners to assess and prioritize the need for new projects in
different hazard areas. As a result, CPSC has insufficient data on both
internal agency efforts and external product hazards to assess the impact
and cost of each project, either when it is selected or after it has been
implemented.

To help evaluate alternative methods of addressing potential hazards, CPSC
may perform a risk assessment to estimate the likelihood of injury
associated with a hazard or conduct a cost-benefit analysis to assess the
potential effects of a proposed regulation. Although CPSC does not
complete either a risk assessment or cost-benefit analysis for every
project, the agency conducts these analyses more often than it is required
to by law. Nevertheless, CPSC’s data are often insufficient to support a
thorough application of these analytical techniques—a problem that
frequently arises in doing both risk assessment and cost-benefit analysis.
To evaluate relative risks, it is usually necessary to have information on
how many consumers use the product—information that CPSC frequently
does not have. In addition, risk assessment of consumer products requires
measurement of the number of harmful incidents. CPSC’s imprecise and
incomplete death and injury data make risk assessment and cost-benefit
analysis at best less reliable and at worst impossible to do. Furthermore,
the cost-benefit analyses conducted by CPSC between 1990 and 1996 were
often not comprehensive, and the reports on these analyses were not
sufficiently detailed. For example, experts generally agree that sensitivity
analysis—a technique that enables the reader to determine which
assumptions, data limitations, or parameters are most important to the
conclusions—should be incorporated in cost-benefit analyses. Most of
CPSC’s cost-benefit analyses did not include such information. We are
making recommendations to the Chairman of CPSC to improve the agency’s
project selection and cost-benefit analyses.

CPSC has established procedures to implement statutory requirements
concerning the release of manufacturer-specific information. When
releasing information to the public that identifies a specific
manufacturer—for example, in a safety alert or recall notice—CPSC is
required to verify the information and allow the manufacturer an
opportunity to comment. Evidence from the industry and from legal cases
suggests that CPSC has met its statutory requirements in this area.
Individuals within CPSC, as well as some industry representatives and



Page 3                       GAO/HEHS-97-147 Consumer Product Safety Commission
             B-274042




             consumer groups, expressed dissatisfaction with the requirements of this
             law. Some of these individuals have proposed statutory changes that range
             from reducing to expanding the current requirements.


             CPSC was created in 1972 under the Consumer Product Safety Act (P.L.
Background   92-573) to regulate consumer products that pose an unreasonable risk of
             injury, to assist consumers in using products safely, and to promote
             research and investigation into product-related deaths, injuries, and
             illnesses. CPSC currently has three commissioners, who are responsible for
             establishing agency policy.2 One of these commissioners is designated the
             chairman; the chairman directs all the executive and administrative
             functions of the agency.

             The Consumer Product Safety Act consolidated federal safety regulatory
             activity relating to consumer products within CPSC. As a result, in addition
             to its responsibilities for protecting against product hazards in general,
             CPSC also administers four laws that authorize various performance
             standards for specific consumer products. These laws are the Flammable
             Fabrics Act (June 3, 1953, c.164), which authorizes flammability standards
             for clothing, upholstery, and other fabrics; the Federal Hazardous
             Substances Act (P.L. 86-613), which authorizes the regulation of
             substances that are toxic, corrosive, combustible, or otherwise hazardous;
             the Poison Prevention Packaging Act of 1970 (P.L. 91-601), which
             authorizes requirements for child-resistant packaging for certain drugs and
             other household substances; and the Refrigerator Safety Act of 1956
             (Aug. 2, 1956, c.890), which establishes safety standards for household
             refrigerators.

             In fiscal year 1997, CPSC carries out this broad mission with a budget of
             about $42.5 million and a full-time-equivalent staff of 480. As figure 1
             shows, after adjusting for inflation, the agency’s budget has decreased by
             about 60 percent since 1974. Similarly, CPSC’s current staffing level
             represents 43 percent fewer positions as compared with the agency’s 1974
             staff.




             2
              The Consumer Product Safety Act provides for the appointment of five commissioners by the
             President of the United States for staggered 7-year terms. However, since 1986, no more than three
             commissioners have served at one time.



             Page 4                                 GAO/HEHS-97-147 Consumer Product Safety Commission
                                            B-274042




Figure 1: CPSC Funding, in Inflation-Adjusted Dollars, 1974-96

CPSC Funding (in Thousands)
90,000

80,000

70,000

60,000

50,000

40,000

30,000

20,000

10,000

     0
     1974   1976    1978      1980   1982   1984     1986    1988     1990     1992     1994     1996
     Year


                                            Notes: Budget figures were obtained from CPSC. The 1993 budget figure excludes a special
                                            appropriation for office space relocation. Budget figures were adjusted for inflation using the
                                            Gross Domestic Product deflator for federal nondefense spending, with 1992 as the base year.




                                            CPSC uses a number of regulatory and nonregulatory tools to reduce
                                            injuries and deaths associated with consumer products. Under several of
                                            the acts that it administers, CPSC has the authority to issue regulations that
                                            establish performance or labeling standards for consumer products. For
                                            example, in 1993, CPSC issued regulations under the Consumer Product
                                            Safety Act requiring disposable cigarette lighters to be child-resistant. If
                                            CPSC determines that there is no feasible standard that would sufficiently
                                            address the danger, CPSC may issue regulations to ban the manufacture and
                                            distribution of the product. In addition, under the Consumer Product
                                            Safety Act, if a product violates a safety regulation or presents a
                                            “substantial hazard,” CPSC may order a product recall, in which the item is
                                            removed from store shelves and consumers are alerted to return the item




                                            Page 5                                GAO/HEHS-97-147 Consumer Product Safety Commission
B-274042




for repair, replacement, or refund.3 CPSC can also impose civil penalties for
violations of federal safety standards.

Although CPSC has these broad regulatory powers, much of the agency’s
efforts are carried out using nonregulatory methods. In addition to
federally mandated product safety standards, many consumer products
are covered by voluntary standards. These voluntary standards, which are
often established by private standard-setting groups, do not have the force
of law. However, many voluntary standards are widely accepted by
industry.4 The 1981 amendments to the Consumer Product Safety Act
require CPSC to defer to a voluntary standard—rather than issue a
mandatory regulation—if CPSC determines that the voluntary standard
adequately addresses the hazard and that there is likely to be substantial
compliance with the voluntary standard. As a result, voluntary standards
development is an important tool in CPSC’s hazard-reduction efforts. For
example, in 1996 CPSC helped a private group develop a voluntary standard
to address the risk of children getting their heads stuck between the slats
of toddler beds, and in 1995 CPSC assisted a standard-setting group in
upgrading safety standards to prevent fires associated with Christmas tree
lights.

CPSC also addresses product hazards by providing information to
consumers on safety practices that can help prevent product-related
accidents. For example, to encourage consumers to use electricity
safely—and particularly to promote the use of ground fault circuit
interrupters—CPSC conducted a far-reaching publicity campaign that
included radio public service announcements, messages printed on
carryout bags for hardware stores, a joint press conference with industry
representatives, presentations on television’s Home Shopping Network,
and promotional letters to real estate and home inspection associations. In
addition to its own active efforts to disseminate information, CPSC provides
considerable amounts of information in response to requests from the
public. Like other federal agencies, CPSC must comply with the Freedom of
Information Act (FOIA) when responding to requests from the public for
information. A notable feature of FOIA is its presumption in favor of
disclosure: any person has the right to inspect and copy any government

3
In practice, CPSC rarely uses its regulatory power to order a recall, but works cooperatively with
manufacturers to carry out recalls.
4
 Voluntary standards may benefit manufacturers by giving consumers added confidence in a product,
providing some degree of protection from product liability, and allowing manufacturers to benefit
from the safety expertise developed by voluntary standards groups. In addition, although federal law
does not compel manufacturers to comply with voluntary standards, state or local regulations may
incorporate some voluntary standards regarding consumer products, and some retailers prefer to carry
only those goods that comply with the applicable voluntary standards.



Page 6                                 GAO/HEHS-97-147 Consumer Product Safety Commission
B-274042




records unless the documents requested fall within one of the exemptions
to the act (for example, disclosure of trade secrets). FOIA requests may
come to CPSC from regulated industries, the press, consumer groups, or
individuals. During calendar year 1995, CPSC responded to 16,424 formal
requests made under FOIA.

CPSC’s resource base and extensive jurisdiction require the agency to select
among potential product hazards. New initiatives may come to CPSC in
several ways. First, any person may file a petition asking CPSC to issue,
amend, or revoke a regulation. Petitions, which can be as simple as a letter
or as formal and detailed as a legal brief, have come to CPSC from doctors
and nurses, consumers and advocacy groups, and industry representatives.
CPSC may grant or deny a petition either in full or in part. Even when CPSC
denies a petition and declines to issue a regulation, it may still begin a
project to address the hazard by promoting a voluntary standard or
conducting a consumer education campaign. For example, a project on
heat tapes (heated wraps for exposed pipes) originated with a petition
from a concerned consumer. CPSC denied the petition for a mandatory
standard, but conducted a research study and review of the existing
voluntary standard for heat tapes.

Second, CPSC receives some product hazard projects from the Congress.
The Congress may require CPSC to study a wide-ranging product area. For
example, the Consumer Product Safety Improvement Act of 1990 resulted
in a large body of work on products affecting indoor air quality, including
wood stoves, kerosene heaters, and carpets. The Congress may also direct
CPSC to impose a specific regulation, such as when it directed CPSC to
require additional labeling on toys intended for children aged 3 to 6
warning parents of possible choking hazards when the toy is used by
children under age 3.5

Finally, CPSC commissioners and agency staff may initiate projects or
suggest areas to address. CPSC gathers death and injury information to help
identify potential product hazards and also obtains input from the public.
The agency maintains a toll-free hot line and an Internet site that the
public can use to notify agency staff of a possible product hazard. In
addition, CPSC holds public meetings to get input on possible hazards to
address and on which hazards should receive priority. For example, CPSC
increased its efforts to remove drawstrings from children’s clothing after
receiving a letter from a woman whose daughter was strangled when her
jacket string caught on a playground slide.

5
 This mandate was imposed in the Child Safety Protection Act (P.L. 103-267, June 16, 1994).



Page 7                                 GAO/HEHS-97-147 Consumer Product Safety Commission
                        B-274042




                        The selection of projects to address new product hazards takes place at
                        different levels of the agency throughout the year. For a petition, the
                        commissioners decide whether the product hazard warrants further
                        agency involvement, and the commissioners vote on whether to grant or
                        deny the petition. If a project is believed to have a high potential for
                        regulatory action or involve a substantial amount of agency resources, the
                        commissioners decide whether to pursue it. Projects of this caliber are
                        often noted in the agency’s annual budget and operating plan, which must
                        also be approved by commissioner vote.6 Staff request a decision on such
                        a project by preparing a briefing package about the product hazard for the
                        commissioners, who vote to begin the regulatory process, take some other
                        action, or terminate the project. Agency staff generally may decide to
                        initiate projects that are unlikely to result in regulation, and no briefing
                        package is sent to the commissioners for a vote.7 (Of the 115 CPSC projects
                        we identified, 80 (70 percent) were detailed in briefing packages.)

                        The scope of the agency’s projects varies greatly, and CPSC has no standard
                        definition of what constitutes a project. A project might cover general
                        product areas, such as fire hazards, or address only a specific product, like
                        cigarette lighters. A project might require undertaking an extensive
                        research study or providing technical assistance to a group that is
                        developing a voluntary standard.


                        The bulk of CPSC’s workload is made up of projects selected by the agency
CPSC Has Limited        rather than by the Congress. CPSC has established criteria to help in project
Information Available   selection, such as the numbers of deaths and injuries associated with a
to Assist in Project    product. However, CPSC is unable to accurately measure these criteria
                        because its data on potential hazards are incomplete. In addition, CPSC
Selection               does not maintain systematic information on past and ongoing projects,
                        which makes it difficult to assess and prioritize the need for new projects
                        in different hazard areas. The lack of comprehensive data on individual
                        product hazards and on agency initiatives raises questions about CPSC’s
                        ability to evaluate its own effectiveness—which it is now required to do
                        under the Government Performance and Results Act of 1993 (the Results
                        Act).




                        6
                         Commission votes are also taken in several other instances, such as initiating a regulation, the use of
                        the agency logo by outside groups, and the application of civil penalties.
                        7
                         For more information about the agency’s organization and structure, see app. II.



                        Page 8                                  GAO/HEHS-97-147 Consumer Product Safety Commission
                                              B-274042




Agency Has Considerable                       CPSC has wide latitude over which potential product hazards it targets for
Discretion in Project                         regulatory and nonregulatory action. Although it has little or no discretion
Selection                                     over projects mandated by the Congress, CPSC can choose to accept or
                                              reject suggestions that are submitted by petition or proposed by the
                                              agency staff. As shown in figure 2, 59 percent of CPSC projects were
                                              initiated by CPSC, 30 percent originated from a petition, and about
                                              11 percent resulted from congressional mandates. Of the 115 projects the
                                              agency worked on from January 1, 1990, to September 30, 1996, 97 (about
                                              90 percent) were chosen by the agency.8 Data were unavailable to assess
                                              the extent to which staff suggestions for projects were accepted or
                                              rejected. Of the petitions filed with CPSC between January 1, 1990, and
                                              September 30, 1996, 60 percent resulted in projects (32 percent by granting
                                              the petition in whole or in part, and 27 percent by denying the petition to
                                              establish a mandatory regulation but creating a nonregulatory project). In
                                              27 percent of cases, CPSC decided that no action was needed or that
                                              existing actions or standards were sufficient to address the issue raised by
                                              the petition. In the remaining cases, a decision is still pending or the
                                              petition was withdrawn before a decision was rendered.


Figure 2: Origination of CPSC Projects,
January 1, 1990-September 30, 1996
                                                                                                    Petition
                                                                                                    30.0%




                                                                                                               Mandate
                                                                                                               11.0%
                                              Internal
                                              59.0%




CPSC Project Selection                        CPSC has established criteria for setting agency priorities and selecting
Criteria Open to                              potential hazards to address. These criteria, which are incorporated in
Differences in Emphasis                       agency regulations, include the following:
and Interpretation                        •   the frequency of injuries and deaths resulting from the hazard;
                                          •   the severity of the injuries resulting from the hazard;


                                              8
                                               For a listing of these 115 projects, see app. III.



                                              Page 9                                     GAO/HEHS-97-147 Consumer Product Safety Commission
                              B-274042




                          •   addressability—that is, the extent to which the hazard is likely to be
                              reduced through CPSC action—agency regulations note that the cause of
                              the hazard should be analyzed to help determine the extent to which
                              injuries can reasonably be expected to be reduced or eliminated through
                              CPSC action;
                          •   the number of chronic illnesses and future injuries predicted to result from
                              the hazard;
                          •   preliminary estimates of the costs and benefits to society resulting from
                              CPSC action;
                          •   unforeseen nature of the risk—that is, the degree to which consumers are
                              aware of the hazard and its consequences;
                          •   vulnerability of the population at risk—whether some individuals (such as
                              children) may be less able to recognize or escape from potential hazards
                              and therefore may require a relatively higher degree of protection;
                          •   probability of exposure to the product hazard—that is, how many
                              consumers are exposed to the potential hazard, or how likely a typical
                              consumer is to be exposed to the hazard; and
                          •   other—additional criteria to be considered at the discretion of CPSC.

Vulnerable Populations,       CPSC’s regulations allow for considerable freedom in applying these
Numbers of Deaths and         criteria; commissioners and staff can base their project selections on what
Injuries, and Causality       they perceive as the most important factors. For example, the regulations
Emphasized Over Other         do not specify whether any criterion should be given more weight than the
Selection Criteria            others, nor must all criteria be applied to every potential project. Indeed,
                              our interviews with present and former commissioners and our review of
                              CPSC briefing packages revealed a pattern in which three criteria—the
                              numbers of deaths and injuries, the causality of injuries, and the
                              vulnerability of the population at risk—were more strongly emphasized
                              than the others. In addition, each of the commissioners we interviewed
                              identified some criteria as being more important than others for project
                              selection. For example, one commissioner indicated that the number of
                              deaths and injuries was most important, while another commissioner
                              included awareness of the hazard in a list of several criteria she believed
                              were most important. However, there was considerable agreement among
                              the commissioners about the importance of several criteria. The
                              commissioners cited two criteria—vulnerability of population and number
                              of deaths and injuries—as especially important for project selection. In
                              addition, several—but not all—commissioners emphasized causality of
                              injuries. None of the other criteria was emphasized by more than one or
                              two commissioners.




                              Page 10                      GAO/HEHS-97-147 Consumer Product Safety Commission
                                 B-274042




                                 Because the commissioners use their judgment in applying these criteria,
                                 there is no systematic checklist or scoring system that would enable us to
                                 determine which factors were considered most important for a particular
                                 product. However, information related to some or all of these criteria is
                                 sometimes contained in briefing packages and other documents. Our
                                 review of CPSC project documentation showed that information on
                                 vulnerable populations and the numbers of deaths and injuries associated
                                 with the product was likely to be compiled at some time during the
                                 project, but information associated with other criteria was less likely to be
                                 documented. For example, of the 115 projects we reviewed, death and
                                 injury information was available in 97 cases. However, only 26 cases
                                 included information on exposure to the hazard, a less-emphasized
                                 criterion.

                                 Although data were insufficient to compare the universe of possible
                                 projects with the ones selected by CPSC, the characteristics of CPSC projects
                                 appear generally consistent with the stronger emphasis on death and
                                 injury data and on vulnerable populations expressed by the current and
                                 former commissioners. For example, while 76 of the 115 projects we
                                 examined were directed at least partially at a vulnerable population group,
                                 only 13 projects mentioned chronic illness. However, although the number
                                 of deaths and injuries associated with product hazards was almost always
                                 available in project documentation, there was no pattern of only those
                                 projects with high numbers of injuries or deaths being selected. Of the 97
                                 projects that had death and injury statistics, 19 showed fewer than 50
                                 injuries and/or deaths associated with the product. The estimated number
                                 of annual injuries associated with product hazards ranged from 1 to
                                 162,100 (for baseball injuries), and the estimated number of deaths
                                 associated with product hazards ranged from zero to a high of 3,600
                                 annually (for smoke detectors). This wide range is consistent with CPSC
                                 staff’s statement that there is no threshold for the number of deaths and
                                 injuries that would require acceptance or rejection of a project.

Commissioners Interpret Some     Although the commissioners and former commissioners we interviewed
Selection Criteria Differently   generally agreed on the criteria they emphasized for project selection, they
                                 expressed very different views on how some of these criteria should be
                                 interpreted. For example, several commissioners viewed vulnerable
                                 populations as focusing on children, while others highlighted additional
                                 segments of the population that they considered vulnerable. One
                                 commissioner also listed low-income and poorly educated consumers as
                                 vulnerable populations, and another expressed concern that the elderly
                                 were especially vulnerable to injury from product hazards. Project



                                 Page 11                      GAO/HEHS-97-147 Consumer Product Safety Commission
B-274042




documentation focused on children more frequently than on other
population segments thought to be at special risk. Many projects we
examined contained no information in the documentation that indicated a
particular population was being considered vulnerable. However, of the 76
projects for which information was available on special populations, 69
(91 percent) mentioned children.9

Industry observers, consumer advocates, current and former
commissioners, and others expressed widely diverging views on how to
apply causality of injuries in selecting projects. All seven commissioners
we interviewed mentioned this as an important criterion, and several
stressed causal factors. A major issue surrounding the application of
causality is determining the appropriate level of protection the agency
should provide when a product hazard results, at least in part, from
consumer behavior. For example, a consumer advocate stated that
regulatory action may be necessary whatever the cause of the incident if
children who were incapable of protecting themselves get hurt. Similarly,
another individual told us that CPSC should deal with potential hazards on
the basis of the behavior that actually took place, not the behavior that
might be expected or considered reasonable. However, other individuals
asserted that CPSC should address only those hazards that result from
products that are defective—that is, products that create a hazard even
when used as intended by the manufacturer. Some industry
representatives stated that it was inappropriate for CPSC to take action
concerning a product if the product was “misused” by the consumer.

Complicating this debate is the difficulty of defining misuse of the product
or negligence of the consumer. For example, the appropriate degree of
parental supervision is frequently an issue with children’s products. One of
the agency’s more controversial projects illustrates this point. CPSC staff
conducted a project to investigate the deaths of children using baby bath
seats or rings. In these incidents, infants slipped out of the seat and
drowned in the bathtub when the parent or caregiver stepped out of the
room and left the child unsupervised, despite warning signs on the seats
not to leave children unattended. The Commission disagreed on the proper
course of action, largely because of differing views on causality. In 1994,
the staff recommended that the Commission issue an Advance Notice of
Proposed Rulemaking (ANPR), the first step in the regulatory process. The
staff argued (and one commissioner agreed) that some parents will leave a

9
 Many of these projects involved children’s products, for which only children would be affected, such
as cribs or baby walkers. Other projects dealt with products the general public would be exposed to
but for which children would be more likely to receive injury, including mouthwashes with high
concentrations of alcohol and automatic garage doors that could crush a child when they close.



Page 12                                GAO/HEHS-97-147 Consumer Product Safety Commission
                               B-274042




                               young child alone in the bathtub regardless of a warning not to. However,
                               in voting against issuing an ANPR, the other two commissioners stated that
                               they believed regulation was not appropriate because the lack of
                               supervision, not the product, caused the tragedies.

                               CPSC staff have also encountered other instances in which the behavior of
                               consumers might be viewed as inappropriate. For example, the role of
                               alcohol and drug use in accidents can also raise questions about the
                               appropriate level of regulatory protection.10 In addition, a 1991 CPSC study
                               found that at least 33 percent of bicycle accidents involved behaviors such
                               as performing stunts and going too fast. Similarly, a 1991 CPSC study of
                               fires associated with heat tapes found that at least 38 percent of the heat
                               tapes had been installed improperly. In each of these cases, no regulatory
                               action was taken; in the case of bicycles, the staff did recommend
                               increasing efforts to encourage consumers to take safety precautions such
                               as using lights at night and wearing helmets.


Data Systems Provide           CPSC uses data from internal management systems and from external
Insufficient Information to    sources to assist in project selection. CPSC collects information on
Measure Selection Criteria,    product-related deaths and injuries to provide information for project
                               selection as well as to perform risk assessments and cost-benefit analyses.
Monitor All Projects, or       Furthermore, the agency maintains a computerized management
Evaluate Results               information system (MIS) that contains information on some of its major
                               activities and is used by the agency to develop its annual budget. Both
                               these internal and external data are of limited value. The inadequacy of the
                               information raises questions about CPSC’s ability to make informed project
                               selection decisions so that agency resources are being spent efficiently.

Significant Gaps Exist in      CPSC has developed a patchwork of independent data systems to provide
CPSC’s Data on                 information on deaths and injuries associated with consumer products. To
Product-Related Injuries and   obtain estimates of the number of injuries associated with specific
Deaths                         consumer products, CPSC relies on its National Electronic Injury
                               Surveillance System (NEISS). NEISS gathers information from the emergency
                               room records of a nationally representative sample of 101 hospitals. CPSC
                               also obtains information on fatalities by purchasing a selected group of
                               death certificates from the states. It supplements this information with
                               anecdotal reports from individual consumers and with data from private
                               organizations such as fire-prevention groups and poison control centers.

                               10
                                Alcohol and drug use contributed to an estimated 1.35 million injury-related visits to hospital
                               emergency departments in 1992. See Cheryl R. Nelson and Barbara J. Stussman, “Alcohol- and
                               Drug-Related Visits to Hospital Emergency Departments: 1992 National Hospital Ambulatory Medical
                               Care Survey,” Advance Data, No. 251 (Aug. 10, 1994).



                               Page 13                              GAO/HEHS-97-147 Consumer Product Safety Commission
B-274042




Because neither NEISS nor death certificate data provide detailed
information on hazard patterns or injury causes, CPSC investigates selected
incidents to obtain more detailed information. In addition, CPSC sometimes
uses mathematical modeling techniques or conducts special surveys to
obtain information on product exposure. (For more information on CPSC’s
data sources, see app. V.)

CPSC’s data give the agency only limited assistance in applying its project
selection criteria. (These criteria, the measures used for each, and major
data limitations are given in table 1.) CPSC’s injury and death data allow the
agency to piece together at best an incomplete view of the incidents that
result from consumer product hazards. Product-related injuries may be
treated in a variety of ways—in an emergency room, in a physician’s office,
or through an outpatient clinic, for example. As figure 3 illustrates, CPSC
obtains systematic surveillance information only on deaths and on injuries
treated in the emergency room; injuries treated in other settings (such as
physicians’ offices) are not represented in CPSC’s surveillance data.




Page 14                      GAO/HEHS-97-147 Consumer Product Safety Commission
                                      B-274042




Figure 3: Types of Death and Injury
Data Covered by CPSC’s Systematic
                                        Treated in
Surveillance Information
                                        Physician’s                                  Death
                                                                Treated in
                                          Office
                                                                   ER


                                       Treated in                             Hospitalized:
                                       Outpatient        Treated at            Admitted
                                         Clinic            Home               Through ER



                                                          Hospitalized:
                                                           Admitted                   Not
                                          Near
                                                           Through                  Treated
                                          Miss
                                                           Physician


                                             Partially Covered
                                             Covered
                                             Not Covered
                                      Note: Death data are considered partially included because CPSC obtains death certificates for
                                      selected causes of death. For the other sources of treatment, CPSC may obtain some anecdotal
                                      information from consumer reports or other sources (see app. V). The actual numbers of injuries
                                      treated in each setting is unknown.




                                      A “near miss” refers to an incident in which a product-related injury nearly
                                      occurred but was narrowly averted. In its regulations that address
                                      priority-setting, CPSC states that such incidents can be as important as
                                      actual injuries in identifying potential hazards.




                                      Page 15                               GAO/HEHS-97-147 Consumer Product Safety Commission
                                           B-274042




Table 1: CPSC Regulatory
Priority-Setting Criteria and Systematic   Criterion                Measure used by CPSC             Major limitations
Information Available                      Number of deaths         Number of reported deaths        Incomplete because not all
                                                                    where consumer product           certificates are gathered and
                                                                    involvement can be inferred.     not all product-related
                                                                                                     incidents are coded.
                                           Number of injuries       Estimated number of injuries     Generally omits injuries treated
                                                                    involving consumer products      in other settings.
                                                                    treated in emergency rooms.
                                           Severity of injuries     Estimated percentage of          Not representative of the
                                                                    emergency-room-treated           severity of all injuries treated in
                                                                    injuries requiring               all settings.
                                                                    hospitalization.
                                           Chronic illnesses        Some limited information from    Little systematic information.
                                                                    emergency room diagnosis
                                                                    information.
                                           Predicted future injuries Prediction based on NEISS       Of questionable validity
                                                                     data.                           because of changes in
                                                                                                     medical care over time.
                                           Vulnerable populations   Percentage of                    Incomplete. Information
                                                                    emergency-room-treated           available only on age, not on
                                                                    injuries or deaths involving     other vulnerable populations,
                                                                    children or the elderly.         such as persons with
                                                                                                     disabilities.
                                           Exposure                 Various measures, including      Exposure surveys are time
                                                                    sales, estimated products in     consuming and expensive; not
                                                                    use, and population.             done for all projects; done only
                                                                                                     after project is well under way.
                                           Addressability/          Judgment based on cause of       Often impossible to make an
                                           causation                incident, hazard pattern, and    informed judgment until project
                                                                    information on product design;   is well under way;
                                                                    investigations of selected       investigations are time
                                                                    incidents may be conducted to    consuming and expensive.
                                                                    obtain this information.
                                           Preliminary cost-benefit Results of analysis.             Quality data are frequently not
                                           analysis                                                  available; limited accuracy in
                                                                                                     early stages of project.

                                           CPSC staff identified the lack of data on injuries treated in physicians’
                                           offices and other settings as a key concern. Because CPSC’s data sets reveal
                                           only a portion of the injury picture, the agency underestimates the total
                                           numbers of deaths and injuries associated with any given consumer
                                           product. The extent of this undercount is unknown. For example,
                                           researchers report widely varying estimates of the percentage of injuries
                                           that are treated in emergency rooms as opposed to other medical settings.
                                           For example, a 1991 study by researchers at RAND found that




                                           Page 16                           GAO/HEHS-97-147 Consumer Product Safety Commission
B-274042




approximately 65 percent of injuries were treated in the emergency room.11
 However, recent data indicate that the number of injury-related visits to
physicians’ offices alone were more than double the number of
injury-related visits to the emergency room.12 CPSC’s estimates of
product-related deaths are also undercounted, for two reasons. First, for
budgetary reasons, the agency purchases only a subset of the total number
of death certificates from states. Second, CPSC death counts include only
those cases in which product involvement can be inferred from the
information on the death certificate, and in some cases, product-related
information is not recorded.

Even if a reliable figure was available to determine the exact percentage of
product-related injuries that were treated in emergency rooms, this
percentage would not necessarily apply to any specific type of
product-related injury. For example, even if it was established that
40 percent of all product-related injuries were treated in emergency
rooms, the percentage of bunk bed injuries treated in emergency rooms
might be much larger or smaller. The setting in which injuries are treated
depends on a wide array of factors that vary among individuals, across
geographic regions, and among different types of injuries. Research
indicates that African Americans are more likely to use the emergency
room than Caucasians are.13 Access to the emergency room or to a
physician also depends on the type of medical insurance a person has. For
example, health maintenance organizations (HMO) often place restrictions
on reimbursement for emergency room care, and HMO membership as a
percentage of the total population varies widely from state to state.14 In
addition, injuries that occur at night, when most physicians’ offices are
closed, may be more likely to be treated in the emergency room. As a
result, it is unlikely that CPSC could approximate the number of injuries
associated with a specific product by using data that apply to all consumer
products as a group.

The incompleteness of CPSC’s injury information also hampers its ability to
reliably discern long-term trends in injuries, which is not only a criterion

11
 Deborah R. Hensler, and others, Compensation for Accidental Injuries in the United States (Santa
Monica, Calif.: RAND, 1991).
12
 See Susan M. Schappert, “National Ambulatory Medical Care Survey: 1992 Summary,”Advance Data
No. 253 (Aug. 18, 1994), and Linda F. McCaig, “National Hospital Ambulatory Medical Care Survey:
1992 Emergency Department Summary,” Advance Data, No. 245 (Mar. 2, 1994).
13
 See Linda F. McCaig, “National Hospital Ambulatory Medical Care Survey: 1992 Emergency
Department Summary,” p. 1.
14
  1996 figures from InterStudy show that while only 1.2 percent of individuals in Mississippi belonged
to HMOs, 44.8 percent of individuals in Oregon were HMO members. See The InterStudy Competitive
Edge: HMO Industry Report 6.2, InterStudy Publications (1996), p. 28.


Page 17                                GAO/HEHS-97-147 Consumer Product Safety Commission
B-274042




for project selection but also an important factor for evaluating the
success of CPSC’s injury-reduction efforts and determining the need for
follow-up actions. The relative sizes of the pieces of the injury puzzle in
figure 3 are unknown but appear to change over time. For example,
hospitalizations decreased by 5 percent on a per capita basis between 1982
and 1994, while between 1983 and 1993, hospital outpatient clinics saw a
53-percent increase in visits on a per capita basis.15 As a result, it is
impossible to determine whether any change in the number of emergency
room visits represents a true change in injuries or a shift to other medical
settings.16

According to CPSC staff, identifying chronic illnesses associated with
consumer products is nearly impossible. CPSC staff stated that little is
known about many chronic illness hazards that may be associated with
potentially dangerous substances, and even less information is available
about which consumer products may contain these substances. Chronic
illnesses are especially likely to be underestimated in CPSC’s NEISS data
because they are underrepresented among emergency room visits and
because product involvement is more difficult to ascertain. Similarly,
consumer product involvement is very seldom recorded on death
certificates in cases of chronic illnesses.

CPSC’s  surveillance data also give an incomplete picture of the severity of
incidents. Although the data capture many relatively severe injuries—that
is, those that result in death or require treatment in an emergency medical
facility—data are missing for individuals who are admitted to the hospital
through their physician rather than through the emergency room.
Potentially less severe cases—for example, those treated in physicians’
offices, walk-in medical centers, or hospital outpatient clinics—are not
represented at all in CPSC’s

systematic surveillance data systems, and consequently, CPSC has no data
on some consumer product problems that may result in numerous but
potentially less severe injuries.

Sketchy information about accident victims also limits CPSC’s ability to
assess which consumer product hazards have a disproportionate impact
on vulnerable populations. NEISS and death certificates provide only the

15
  These figures were derived from data provided by the American Hospital Association.
16
  Precise estimates on the impact of changing treatment patterns are rare, and the impact may vary
among local areas. However, one research team found that because of the growth in alternative
facilities in their local area, emergency rooms handled about half of outpatient injury cases—down
from a previous estimate of 75 percent. See Julian A. Waller, Joan M. Skelly, and John H. Davis,
“Treated Injuries in Northern Vermont,” Accident Analysis and Prevention, 27 (6) (1995), pp. 819-28.


Page 18                                GAO/HEHS-97-147 Consumer Product Safety Commission
                            B-274042




                            age of the victim; no systematic or comprehensive information is available
                            to determine whether a given hazard has a special impact on other
                            vulnerable populations, such as persons with disabilities. A former
                            commissioner told us that the lack of other demographic information such
                            as race, income, and disability status made it difficult for her to know
                            which subpopulations were predominantly affected by a particular hazard.
                            Another commissioner echoed this concern, and said that such
                            information would be useful in targeting public information campaigns on
                            certain hazards to those groups that needed the information most.

                            CPSC staff identified the need for additional exposure data as a major
                            concern. However, they also told us that obtaining information on
                            exposure to products and establishing causation requires special efforts
                            that can be time consuming and costly. Although CPSC’s priority-setting
                            criteria include exposure to the hazard, exposure data are generally not
                            included in CPSC’s ongoing data collection efforts. As a result, exposure is
                            assessed either not at all or further along in the project, precluding the use
                            of exposure as an effective criterion for project selection. Similarly, CPSC’s
                            emergency room and death certificate data provide little information on
                            the circumstances surrounding the incident. As a result, CPSC staff perform
                            follow-up investigations of selected incidents to obtain additional detail.
                            These investigations may include detailed interviews with victims and/or
                            witnesses, police or fire reports, photographs of the product and/or
                            accident site, laboratory testing of the product, or recreations of the
                            incident. As with exposure data, these investigations are not conducted for
                            every project and are completed only after a project is well under way.
                            Thus, assessment of causation at the project selection stage is unavoidably
                            speculative.

Agency Project Management   CPSC conducts a number of projects annually, but staff were unable to
Information Incomplete or   provide a comprehensive list of projects the agency had worked on in the
Unavailable                 6-year period we examined. CPSC was also unable to verify the
                            completeness of the project list we compiled from agency documents and
                            interviews with staff. According to CPSC staff, internal management
                            systems do not contain this information and such a list could be compiled
                            only by relying on institutional memories of staff members who had been
                            with the agency long enough to know which products the agency had
                            addressed. Without systematic and comprehensive information on its past
                            efforts, CPSC cannot assess whether some hazard areas have been
                            overrepresented and whether agency resources might be more efficiently
                            employed.




                            Page 19                      GAO/HEHS-97-147 Consumer Product Safety Commission
                              B-274042




                              CPSC also lacks information on the characteristics of, resources used on, or
                              outcomes of individual projects. CPSC’s MIS tracks contract dollars and staff
                              time by accounting codes that cover some specific projects and general
                              categories, such as compliance work, which are composed of numerous
                              activities. According to agency officials, CPSC’s MIS generally cannot
                              provide descriptive information on individual projects, such as when a
                              project was started or concluded; the number of staff days used; what
                              aspect of the product was addressed; whether the project originated from
                              a petition, congressional mandate or other source; or what action was
                              taken to address the hazard (mandatory standard, voluntary standard, or
                              public information campaign, for example). In addition, CPSC staff told us
                              that two separate projects involving the same or similar products at
                              different times may be assigned the same MIS code. As a result, even if a
                              project appears to be tracked in the MIS, reliable inferences cannot be
                              drawn from MIS data.

CPSC Has Limited Ability to   CPSC’s limited data on deaths and injuries, combined with its lack of
Evaluate Agency Impact        information on projects, reduce the agency’s ability to evaluate the impact
                              of its work, a process it is now required to undertake under recently
                              passed legislation. The Results Act requires every federal agency to
                              evaluate the effectiveness of its efforts starting in fiscal year 1999. The
                              Results Act is aimed at increasing the investment return of tax dollars by
                              improving agencies’ performance. Under the Results Act, an agency is to
                              set mission-related goals and measure progress toward these goals to
                              evaluate agency impact. CPSC has preliminarily identified results-oriented
                              goals in four areas: (1) reducing head injuries to children, (2) reducing
                              deaths from fires, (3) reducing deaths from carbon monoxide poisoning,
                              and (4) reducing deaths from electrocutions. However, the limitations in
                              CPSC’s injury and death data raise a question about how well CPSC will be
                              able to evaluate the effectiveness of agency actions in these and other
                              areas.


                              CPSC uses two analytical tools—risk assessment and cost-benefit
CPSC Uses                     analysis—to assist in making decisions on regulatory and nonregulatory
COST-BENEFIT                  methods to address potential hazards. Risk assessment involves estimating
Analysis and Risk             the likelihood of an adverse event (such as injury or death). For example,
                              CPSC estimated that the risk of death from an accident involving an
Assessment, but               all-terrain vehicle (ATV) was about 1 death for every 10,000 ATVs in use in
Improvements Are              1994. Cost-benefit analysis details and compares the expected effects of a
                              proposed regulation or policy, including both the positive results
Needed in Data and            (benefits) and the negative consequences (costs). Although cost-benefit
Methodology

                              Page 20                      GAO/HEHS-97-147 Consumer Product Safety Commission
                             B-274042




                             analysis may not be applicable to every decision and may not be the only
                             factor appropriately considered in a decision, it can be a useful
                             decision-making tool. The Congress requires CPSC to perform cost-benefit
                             analysis before issuing certain regulations, and CPSC has conducted
                             cost-benefit analysis for these regulations and in other situations in which
                             it was not required. Although perfectly complete and accurate data are
                             rarely available for any analysis, CPSC’s data are frequently inadequate to
                             support detailed, thorough, and careful risk assessment and cost-benefit
                             analysis. In addition, CPSC’s cost-benefit analyses are frequently not
                             comprehensive, and the reports on these analyses are not sufficiently
                             detailed. Improvements in the agency’s methodology and in the quality of
                             the underlying data are necessary to ensure the clarity and accuracy of
                             CPSC’s risk assessments and cost-benefit analyses.



CPSC Performs                Cost-benefit analysis can help decisionmakers by organizing and
Cost-Benefit Analysis More   aggregating all the relevant information to clarify the nature of the
Often Than Required by       trade-offs involved in a decision. Although cost-benefit analysis may not be
                             appropriately used as the sole criterion for making a decision, a
Law                          well-constructed cost-benefit analysis can highlight crucial factors, expose
                             possible biases, and facilitate informed decisions even when it is
                             impossible to measure all the potential effects of a specific regulatory
                             proposal. The Congress has required CPSC to perform and publish a
                             cost-benefit analysis when issuing a regulation (such as a mandatory
                             standard or product ban) under the Consumer Product Safety Act. In
                             addition, CPSC is also required to conduct cost-benefit analyses before
                             issuing regulations under the authority of portions of the Federal
                             Hazardous Substances Act (specified labeling provisions are exempt from
                             this requirement) and the Flammable Fabrics Act.17

                             Because most of the agency’s projects do not involve mandatory
                             regulation, relatively few CPSC projects conducted between January 1,
                             1990, and September 30, 1996, were subject to these requirements. We
                             identified 8 cost-benefit analyses that CPSC performed in accordance with
                             these requirements, and an additional 21 analyses it conducted in
                             situations in which it was not required. For example, CPSC performed
                             cost-benefit analyses in eight instances in which it was considering issuing
                             requirements for child-resistant packaging under the Poison Prevention



                             17
                               As an independent agency, CPSC is not required to comply with Executive Order 12866, which
                             requires federal agencies to perform a cost-benefit analysis and submit it to the Office of Management
                             and Budget (OMB) for review whenever the agency issues a regulation that is economically significant.



                             Page 21                                GAO/HEHS-97-147 Consumer Product Safety Commission
                            B-274042




                            Packaging Act, which does not require cost-benefit analysis.18 CPSC
                            frequently conducts cost-benefit analysis with respect to regulatory
                            procedures, whether or not it is required to do so. However, a complete
                            cost-benefit analysis is done less frequently for voluntary standards
                            projects or information and education efforts, although some economic
                            information may be generated to assist such projects. In addition to the
                            complete cost-benefit analyses, we identified an additional 23 cases in
                            which some information was provided on some economic benefits or
                            costs.

                            Before issuing a mandatory regulation, CPSC is required to consider the
                            degree and nature of the risk of injury the regulation is designed to
                            eliminate or reduce. However, CPSC usually does not conduct a formal
                            numerical risk assessment before issuing a regulation, and the law does
                            not require it. We found 24 risk assessments conducted by CPSC between
                            January 1, 1990, and September 30, 1996; only 4 of these were associated
                            with regulatory action.


Agency Data Are Generally   Both risk assessment and cost-benefit analysis require extensive data. Risk
Insufficient to Support     assessment requires information both on the adverse event and on
Thorough and Detailed       exposure to the precipitating circumstances. For example, when CPSC
                            performed a risk assessment on floor furnaces, the agency estimated the
Analysis                    number of previous injuries associated with floor furnaces and the number
                            of floor furnaces in use.19 Similarly, when CPSC performed a risk
                            assessment to examine the risk of contracting cancer from dioxin traces in
                            common paper products, the agency used information from laboratory
                            studies on dioxin’s link to cancer and also incorporated data on exposure
                            to paper products.20 Because cost-benefit analysis includes a
                            comprehensive delineation of the expected effects of a given proposal, a
                            careful and thorough cost-benefit analysis will also be very data intensive
                            and rich with detail.

                            CPSC’s data systems are frequently unable to adequately meet the extensive
                            demands for information posed by risk assessment and cost-benefit
                            analysis. As a result, the agency’s estimates of risks, costs, and benefits are


                            18
                             For child-resistant packaging regulations, CPSC is legally required to consider only whether requiring
                            child-resistant packaging is “technically feasible, practicable, and appropriate.”
                            19
                              CPSC found that the risk of injury associated with floor furnaces was small in terms of the number of
                            incidents but relatively higher when considered in relation to the number of products in use.
                            20
                             CPSC’s study found that the risk from this type of dioxin exposure was negligible—at most, 5 cases
                            per billion people exposed—and as a result no action was taken.



                            Page 22                                GAO/HEHS-97-147 Consumer Product Safety Commission
B-274042




less accurate because they reflect the substantial limitations of the
underlying data. Available information does not permit us to determine the
potential impact of better data on the results of CPSC’s cost-benefit
analyses and risk assessments. Some of these data weaknesses tend to
make product hazard risks seem larger, and other problems tend to make
the same risks appear smaller. For example, because CPSC’s data
undercount the deaths and injuries associated with particular consumer
products, estimates of risk—and the potential benefits of reducing that
risk—appear smaller. However, CPSC’s surveillance data provide
information only on whether a product was involved in an accident, not on
whether the product caused the accident. At least at the initial stages of a
project, this can make the risks assessed by CPSC—and the benefits of
reducing those risks—appear larger.

For risk assessment, CPSC must also obtain information on exposure to the
hazard. For example, to assess the risk associated with aluminum ladders,
CPSC obtained estimates of the number of ladders available for use and on
the number of times each year the ladder was used. Obtaining exposure
data presents special challenges for CPSC. Because the product definition
that relates to a particular hazard is often relatively narrow, existing data
sources frequently offer insufficient detail. For example, CPSC was unable
to use Census sources to determine the number of saunas in the United
States because saunas were included in a broader classification of
products when government data were collected. In addition, CPSC staff told
us that it is often difficult to find accurate information on the number of
products that are in households and available for use.

CPSC sometimes responds to these challenges by using mathematical
modeling techniques or easier-to-obtain proxy measures (such as
population) to estimate product exposure. In addition, for a few
large-scale projects, CPSC has incurred the substantial expenses necessary
to conduct its own detailed exposure survey. For example, CPSC conducted
a survey of households that asked detailed questions on matches and
disposable cigarette lighters—the number purchased, where they are
generally kept, how they are used, and other details. However, for the
majority of the projects we reviewed, CPSC did not gather any data on
exposure. Of the 80 projects we reviewed for which briefing packages
were prepared, only 26 included information on exposure to the hazard,
and CPSC’s risk assessments were confined to 24 cases between 1990 and
1996—approximately 21 percent of all projects conducted over that time
period.




Page 23                      GAO/HEHS-97-147 Consumer Product Safety Commission
                         B-274042




CPSC Cost-Benefit        The methodology used to conduct a cost-benefit analysis will frequently
Analyses Are Often Not   depend on the circumstances and the context of the analysis. For this
Comprehensive and Not    reason, there is no complete set of standards for evaluating the quality of
                         an individual cost-benefit analysis. However, the professional literature
Reported in Sufficient   offers some guidance for analysts, and certain specific elements are
Detail                   frequently mentioned as essential for cost-benefit analysis. For example,
                         because cost-benefit analysis is meant to be a complete delineation of the
                         expected effects of a proposed action, all potential impacts (even those
                         that cannot be quantified) should be discussed. To ensure that the reader
                         is able to make an informed judgment, it is important to be explicit about
                         the underlying data, methodology, and assumptions. Accordingly, the
                         literature suggests that all methodological choices and assumptions should
                         be detailed, all limitations pertaining to the data should be revealed, and
                         measures of uncertainty should be provided to allow the reader to take
                         into account the precision of the underlying data. Similarly, the literature
                         calls for sensitivity analysis, which enables the reader to determine which
                         assumptions, values, and parameters of the cost-benefit analysis are most
                         important to the conclusions.

                         On the basis of our review of the cost-benefit literature, we developed a
                         list of the elements that are frequently used in evaluating cost-benefit
                         analysis. This list, and a description of all the factors we examined, is in
                         appendix IV.21 Although we compared each of these elements with each of
                         CPSC’s analyses, not all elements were applicable to each case. For
                         example, in some cases, the circumstances indicated by a given
                         element—such as reliance on statistical data—were not found, and those
                         cases were treated as not applicable to that element. In addition, for some
                         elements it was not always possible to determine whether CPSC’s analysis
                         was consistent with the element. For these reasons, and to emphasize
                         those areas that we viewed as most critical, we reported only the
                         evaluation results that relate to key elements, applied to the majority of
                         CPSC’s analyses, and for which a determination was possible in all or nearly
                         all cases.

                         Our review of all the cost-benefit analyses that CPSC conducted between
                         January 1, 1990, and October 31, 1996, showed that for many—but not
                         all—elements, CPSC’s analyses were not comprehensive and not reported in
                         sufficient detail (see table 2). For example, CPSC provided descriptive
                         information on proposals and also provided information on a variety of
                         reasonable alternatives in almost 100 percent of cases. However, CPSC

                         21
                          These elements are also similar to guidance OMB issued in January 1996 for preparing economic
                         analyses of significant regulatory actions.



                         Page 24                              GAO/HEHS-97-147 Consumer Product Safety Commission
                                       B-274042




                                       analyses generally did not provide measures of uncertainty for the
                                       underlying data. Estimates derived from samples are subject to sampling
                                       error, which can be especially large when the estimates are projected from
                                       relatively fewer cases. In only 17 percent of its analyses did CPSC provide
                                       any statistical information on the precision of the underlying estimates.
                                       Similarly, when estimates are based on a relatively small sample size,
                                       projections are generally not considered reliable. CPSC analysts cautioned
                                       the reader against making conclusions based on small sample data only
                                       45 percent of the time. In addition, some of CPSC’s data sets have a known
                                       upward or downward bias because of the way the data were constructed.
                                       For example, CPSC’s estimates of deaths based on CPSC’s death certificate
                                       database will be understated, and when estimates of incidents are based
                                       only on investigated or reported cases (such as cases reported to CPSC’s
                                       hot line), two potential biases are likely to be introduced into the analysis:
                                       (1) the estimates are likely to be biased downward by nonreporting and
                                       (2) the incidents reported tend to be the more severe ones. In only
                                       53 percent of applicable cases did CPSC’s analysis inform the reader of
                                       known limitations inherent in the data being used for cost-benefit analysis.

Table 2: Evaluation of CPSC Analyses
Shows Inconsistencies With Several                                                                                           Percentage of
Evaluation Elements                                                                                                       CPSC’s analyses
                                                                                                                       that were consistent
                                       Evaluation element                                                                 with this element
                                       Provided descriptive information about a well-defined proposal                                            98
                                       Addressed multiple alternatives                                                                           95
                                       Reported measures of precision for underlying data                                                        17
                                       Cautioned reader about making inferences from data with a small
                                       sample size                                                                                               45
                                       Reported known biases in underlying data                                                                  53
                                       Provided any sensitivity analysis information                                                             26
                                       Included all important categories of benefits and costs                                                   54
                                       Considered “risk-risk” trade-offsa                                                                        49
                                       a
                                        A “risk-risk” trade-off refers to an action to decrease a hazard’s risk that unintentionally increases
                                       that or another risk.



                                       We identified several other areas in which CPSC analyses could benefit
                                       from improvement. For example, researchers agree that sensitivity
                                       analysis—a technique that enables the reader to determine which
                                       assumptions, data limitations, or parameters are most important to the
                                       conclusions—should be incorporated in cost-benefit analyses. CPSC usually
                                       did not provide sensitivity analysis information. For example, agency



                                       Page 25                                 GAO/HEHS-97-147 Consumer Product Safety Commission
B-274042




briefing packages did not include any information on how CPSC’s injury
cost estimates were derived or what factors were the largest components
of injury costs. CPSC applies a statistical model to injury estimates to derive
a figure for injury cost. The model that computed injury cost estimates
accounts for a number of components, including medical costs, forgone
wages, and pain and suffering. With only one exception, CPSC briefing
packages provided only the total cost, without any information on the
derivation of those costs or the individual components. In addition, CPSC
provided only an average injury cost, not a range of injury cost estimates.
For situations in which injuries differ in severity, or for projects in which
severity is probably overstated or understated in the data, the reader
would find such information useful.

Forty-six percent of CPSC analyses did not consider the full range of costs
and benefits likely to result from regulation. For example, CPSC analysts
frequently omitted mentioning intangible costs and/or benefits (costs or
benefits that are difficult to quantify, such as loss of consumer enjoyment)
or potential indirect effects (such as changes in the prices of related
goods). In addition, CPSC frequently excluded risk-risk considerations from
its evaluation of the costs and benefits of potential actions. Sometimes
actions taken to reduce one risk can have the unintended effect of
increasing that or another risk. Individuals may take more or fewer
precautions in response to a change in a product’s safety features, and this
behavior can result in an increase in the risk the intervention was designed
to mitigate. For example, in establishing a standard for child-resistant
packaging that was also “senior-friendly,” CPSC considered that because
child-resistant medicine bottles can be difficult to open, a grandparent
may leave the cap off the bottle, creating an even greater risk than would
be the case with the original cap. Although CPSC considered such factors in
some cases, only 49 percent of its analyses reflected potential risk-risk
trade-offs.

CPSC has not established internal procedures that require analysts to
conduct comprehensive analyses and report them in sufficient detail. For
example, according to CPSC staff, the agency has little written guidance
about what factors should be included in cost-benefit analyses, what
methodology should be used to incorporate these factors, and how the
results should be presented. Staff also told us that CPSC analyses are not
generally subject to external peer review. Such reviews can serve as an
important mechanism for enhancing the quality and credibility of the
analyses that are used to help make key agency decisions.




Page 26                       GAO/HEHS-97-147 Consumer Product Safety Commission
                            B-274042




                            To help minimize the possibility that a product might be unfairly
CPSC Has Established        disparaged, in section 6(b) of the Consumer Product Safety Act the
Procedures for              Congress imposed restrictions on the disclosure of manufacturer-specific
Complying With              information by CPSC.22 Before CPSC can disclose any information that
                            identifies a manufacturer,23 the agency must
Statutory
Requirements for        •   take “reasonable steps” to verify the accuracy of the information and to
                            ensure that disclosure is fair,
Releasing               •   notify the manufacturer that the information is subject to release, and
Manufacturer-Specific   •   provide the manufacturer with an opportunity to comment on the
Information                 information.

                            If the manufacturer requests that its comments be included in CPSC’s
                            disclosure of the information, CPSC can release the information only if
                            accompanied by the manufacturer’s comments. If the manufacturer
                            objects to the release even if its comments are included, it can challenge
                            CPSC in U.S. district court to block disclosure. These restrictions on the
                            release of information apply not only to information the agency issues on
                            its own—such as a press release—but also to information disclosed in
                            response to a request under FOIA. In addition, section 6(b) also requires
                            CPSC to establish procedures to ensure that releases of information that
                            reflect on the safety of a consumer product or class of products are
                            accurate and not misleading, regardless of whether the information
                            disclosed identifies a specific manufacturer.

                            CPSC has established procedures to implement these requirements,
                            including requiring technical staff to “sign off” on information releases and
                            notifying manufacturers. Evidence from several sources—industry
                            sources, published legal decisions, and agency retractions—suggests that
                            CPSC has complied with its statutory requirements. CPSC staff and
                            commissioners, industry representatives, and consumer advocates
                            expressed a wide variety of opinions on the effectiveness of these
                            requirements, and some individuals favored specific changes.




                            22
                             An exception to these restrictions is given if CPSC has declared that the product is an “imminent
                            hazard” under section 12 of the Consumer Product Safety Act.
                            23
                              These restrictions also apply even if the manufacturer is not named but the information would allow
                            the reader to readily identify the manufacturer from the context. For example, if there is only one
                            manufacturer of a product that is identified in the information, the information may be subject to
                            restriction even if the manufacturer’s name is not given.



                            Page 27                                GAO/HEHS-97-147 Consumer Product Safety Commission
                           B-274042




CPSC Has Established       Part of CPSC’s mission is to provide the public with information to help
Procedures for Verifying   individuals use consumer products safely. CPSC disseminates information
and Clearing Information   through its own initiatives and also in response to requests from the
                           public. For example, CPSC informs both consumers and businesses about
for Release                product hazards through product recall notices, provision of information
                           at trade shows and special events, and a telephone hot line and Internet
                           site. In addition, CPSC responds to thousands of telephone and written
                           requests for information each year.

                           CPSC’s mission and its responsibility under FOIA require the agency to
                           disseminate a great deal of information. However, because much of this
                           information is about specific products or manufacturers, CPSC’s
                           information disclosure is often restricted under section 6(b). In its
                           regulation implementing section 6(b), CPSC established several measures
                           designed to ensure compliance with the statutory requirements. These
                           measurers include obtaining written verification from consumers of the
                           information they report to the agency, notifying manufacturers by certified
                           mail when manufacturer-specific information is requested, and giving
                           manufacturers the option of having their comments published along with
                           any information being disclosed.

                           CPSC’s procedures outline several steps to verify all information before it is
                           released. For example, CPSC checks each report the agency receives from
                           consumers about incidents involving potentially hazardous products to
                           ensure that CPSC’s records accurately reflect the consumer’s version of the
                           incident. Agency procedures require staff to send a written description of
                           each incident back to the person who reported it with a request that he or
                           she review it and state if any information needs to be corrected or
                           supplemented.24 The commission staff review each of these incident
                           reports for discrepancies or any obvious inaccuracies. Once they have
                           been checked and confirmed with the consumer, incident reports are
                           made available to the public upon request. If the reports contain
                           information that would identify a specific manufacturer, they are subject
                           to 6(b) requirements regarding disclosure.

                           CPSC also investigates events surrounding selected product-related injuries
                           or incidents. Investigation reports provide details about incident sequence,
                           human behavior factors, and product involvement. The reports generally
                           contain the consumer’s version of what happened and the observations of
                           witnesses, fire and police investigators, and others. Investigations may

                           24
                             The cover letter that accompanies the incident report also asks recipients if they would like to have
                           their names withheld if information about the incident is made public.



                           Page 28                                 GAO/HEHS-97-147 Consumer Product Safety Commission
B-274042




also include follow-up inspections at retail stores or service centers.
However, neither investigations nor incident reports include the
manufacturer’s view of the incident. Its point of view may be expressed in
comments it submits before the report is released. Like incident reports,
investigation files are available to the public upon request and are subject
to 6(b) requirements.

CPSC has issued clearance procedures to cover situations in which
commissioners or staff initiate public disclosures—for example, when the
Commission publishes the results of agency research. These procedures
are intended to verify any information—oral or written—released by the
Commission, regardless of whether the information identifies a
manufacturer. Under CPSC’s guidelines, each assistant or associate
executive director whose area of responsibility is involved must review
the information and indicate approval for the release in writing.25 After all
other review has been completed, the Office of the General Counsel must
also review and approve the release. Press releases with respect to
product recalls are written and issued jointly by CPSC and the affected
manufacturer. In addition, CPSC’s clearance procedures for press releases
state that final clearance must be obtained from the Office of the
Chairman of the Commission.26 In addition, CPSC staff told us that the
current chairman’s policy of coordinating media inquiries through the
Office of Public Affairs is intended to ensure that information provided is
in compliance with section 6(b).

CPSC has also established procedures to implement the notification and
comment provisions of section 6(b). Before CPSC releases information in
response to an FOIA request, an information specialist determines whether
a manufacturer could be readily identified. CPSC staff said that agency
policy is to clearly and narrowly identify hazardous products (including by
manufacturer) whenever possible, in order to prevent the person receiving
the information from confusing safe products with unsafe products of the
same type. However, if an information request is broad, like “all bicycle
accidents,” names of manufacturers are removed before the information is
released, according to CPSC staff. If the requested information could
identify a manufacturer, then staff review the information for appropriate
exemptions (such as trade secrets), and delete portions as appropriate.
The manufacturer is given 20 calendar days in which to review and


25
  For a description of CPSC’s directorates, see app. II.
26
 An exception is made for those hazards considered by the Compliance staff as most serious, for
which clearance must be obtained from a majority of the commissioners.



Page 29                                  GAO/HEHS-97-147 Consumer Product Safety Commission
                             B-274042




                             comment on a summary of the information CPSC plans to release.27
                             Because CPSC often receives multiple requests for the same information,
                             the agency informs manufacturers that it will not send them copies of
                             subsequent requests for the same information unless specifically requested
                             to do so. However, according to CPSC staff, more than 80 percent of the
                             manufacturers that submit 6(b) comments routinely request such
                             notification. In calendar year 1993 (the most recent year for which data
                             were available), CPSC sent out 487 notices to manufacturers and received
                             154 responses (32 percent). Twenty-five manufacturers (5 percent)
                             contested the accuracy of the information or claimed that the proposed
                             disclosure would be unfair.

                             If a manufacturer fails to comment, the information can be released 30
                             days from the date CPSC notified the manufacturer. After taking the
                             manufacturer’s comments into account, CPSC may decide to disclose
                             incident information despite the firm’s objection if, for example, the
                             comments lack specific information to support a claim of inaccuracy or a
                             request for confidentiality. If CPSC chooses to disclose information over a
                             manufacturer’s objection, it must release the manufacturer’s comments
                             along with the other information, unless the manufacturer requests
                             otherwise. In addition, if CPSC decides to release information and the
                             manufacturer objects, the manufacturer has 10 working days to go to court
                             and seek to enjoin CPSC from disclosing the information. Manufacturers
                             have sued CPSC to prohibit disclosure of records only 11 times since the
                             agency was founded, and CPSC was prohibited from releasing the
                             information in 2 of these cases.28


Evidence Suggests That       Information from three sources of evidence—industry, published legal
CPSC Complies With Legal     cases, and data on retractions—suggest that CPSC complies with its
Requirements Regarding       statutory requirements concerning information release. Industry sources,
                             even those otherwise critical of the agency, told us that CPSC generally
the Release of Information   does a good job of keeping proprietary information confidential as
                             required by law. Our review of published legal decisions found no rulings
                             that CPSC violated its statutory requirements concerning the release of
                             information. Retractions by CPSC are also rare. If CPSC finds that it has
                             disclosed inaccurate or misleading information that reflects adversely on
                             any consumer product or class of consumer products or on any


                             27
                              Since small companies may lack the technical ability to comment, CPSC staff told us that they will
                             sometimes help small businesses to formulate their comments.
                             28
                                Currently, Daisy Manufacturing, Inc. v. CPSC is on appeal, after a judge ruled in CPSC’s favor in U.S.
                             district court.



                             Page 30                                 GAO/HEHS-97-147 Consumer Product Safety Commission
                           B-274042




                           manufacturer, it must publish a retraction. Any retraction must be made in
                           the same manner in which CPSC released the original information.
                           According to CPSC, it has published only three such retractions. Two of
                           these retractions, in 1984 and 1994, were made in response to requests
                           from firms. A third retraction, in 1990, was issued after CPSC discovered
                           that a report in its public reading room had mistakenly included inaccurate
                           information.


Industry Observers,        Industry observers, CPSC staff, and consumer groups expressed a wide
Consumer Groups, and       range of opinions on the effectiveness of section 6(b). In response to our
Others Suggested Changes   inquiries, some CPSC commissioners and former commissioners said that
                           these restrictions serve a useful purpose and should not be changed.
to 6(b) Requirements       However, CPSC’s current chairman, industry and advocacy group
                           representatives, and others expressed dissatisfaction with 6(b), and some
                           of these suggested possible changes. Although these individuals raised
                           issues about the extent of the protection afforded to manufacturers and
                           the resources necessary to ensure compliance, we did not assess whether
                           the specific suggestions were necessary or feasible.

                           CPSC’s chairman, other CPSC officials, former commissioners, and the
                           representative of a consumer advocacy group stated that compliance with
                           6(b) is costly for CPSC and delays the agency in getting information out to
                           the public. Although CPSC has not estimated the cost of complying with
                           6(b), agency staff told us that it takes much more staff time to respond to
                           FOIA requests that come under 6(b) than it does to respond to FOIA requests
                           that do not involve company names. To reduce the burden of complying
                           with these requirements, CPSC staff have suggested that the notification
                           requirement that gives manufacturers 20 days in which to comment should
                           apply only the first time an item is released. Some have suggested that
                           instead of requiring CPSC to verify information from consumer complaints,
                           the agency should be allowed to issue such information with an explicit
                           disclaimer that CPSC has not taken a position on the validity of the
                           consumer’s report.

                           Instead of reducing CPSC’s verification requirements, some industry
                           representatives suggested expanding them. These manufacturers stated
                           that before CPSC releases incident information, it should substantiate the
                           information rather than relying on a consumer’s testimony. Industry
                           representatives stated—and CPSC staff confirmed—that many of the
                           requests for CPSC information come from attorneys for plaintiffs in product
                           liability suits. As a result, some industry representatives expressed



                           Page 31                     GAO/HEHS-97-147 Consumer Product Safety Commission
              B-274042




              concern that unsubstantiated consumer complaints could be used against
              them in product liability litigation. They suggested that 6(b) should require
              CPSC to substantiate all incident reports by investigating them before they
              can be disclosed instead of merely checking with the consumer as it does
              now. However, CPSC officials told us that investigations—which are time
              consuming and costly—can be conducted only on a small proportion of
              specially selected cases because of limited resources.

              Industry representatives also said that the current restrictions do not
              provide sufficient protection when information is released on product
              groups instead of on the products of an individual manufacturer. Several
              industry representatives expressed concern that producers of safer
              products may be unfairly maligned when CPSC releases information about a
              group of products, only some of which may be associated with a safety
              problem. According to some of these industry representatives, CPSC should
              extend protection to product groups similar to the safeguards
              manufacturers receive under 6(b).

              Retailers’ representatives also suggested specific changes to CPSC’s
              information release requirements. They said that retailers do not receive
              timely notice of recalls because CPSC has interpreted the law to prohibit
              advance notification of retailers. Consequently, the retailers said that they
              sometimes receive notice of recalls at the same time as their customers
              and have no time to prepare. For example, when consumers come in with
              recalled products, the retailer may not yet know whether the manufacturer
              has agreed to replace the product, refund the purchase price, or provide
              some other remedy. Retailers’ representatives suggested amending 6(b) to
              give 5 business days’ advance notice to retailers before the public
              announcement of a recall. CPSC officials said that typically manufacturers
              are and should be the ones to contact the retailers and make all
              arrangements for a recall. Although they disagreed on the need for a
              statutory change, both CPSC staff and a major retailers’ association said
              that they were trying to work out a more satisfactory arrangement.


              CPSC’s current data provide insufficient information to monitor ongoing
Conclusions   projects and to determine whether potential projects adhere to the
              agency’s selection criteria. Moreover, inadequate agency data often
              prevent CPSC from conducting risk assessments on projects, potentially
              limiting the agency’s ability to target resources to the hazards presenting
              the greatest risks. The lack of sufficient data, combined with
              methodological problems, also makes CPSC’s cost-benefit analyses less



              Page 32                      GAO/HEHS-97-147 Consumer Product Safety Commission
B-274042




useful than they could be. With more detailed information on both internal
resources and external product hazards, CPSC would be better able to
assure the Congress and taxpayers that its resources are expended wisely.

We identified several key areas where CPSC management could improve its
collection and analysis of external data. For example, CPSC would be better
able to make informed decisions on potential agency projects if it had
additional statistically reliable and timely data in several areas, including
(1) injuries treated outside of hospital emergency rooms; (2) exposure to
consumer products and product-related hazards; (3) chronic illnesses
related to consumer products; and (4) hazards that disproportionately
affect certain vulnerable populations, such as low-income individuals and
consumers with disabilities.

In addition, project selection and implementation could be improved if
commissioners and staff had tracking information on CPSC projects, such
as starting and ending dates, project origin, project costs (including staff
days and contract costs), and agency actions taken to address the
potential hazard. Such information could assist the commissioners in
monitoring ongoing projects, targeting new efforts on the basis of previous
work, and assessing the allocation of resources across current projects
and among major hazard areas.

CPSC could also benefit from an improved methodology for cost-benefit
analysis. A stronger methodological base for CPSC’s cost-benefit analyses,
including more complete documentation, would promote sound regulatory
decision-making and could improve the quality of the input and comment
CPSC receives during the regulatory process.


Without these improved data, CPSC will remain unable to accurately apply
measurable criteria in choosing projects or to rigorously assess relative
risks among potential hazards. Some of this necessary information—the
need for more representative and complete injury and exposure data, for
example—may require a significant investment of resources, so CPSC may
need to prioritize these data needs. In doing so, is important for CPSC to
draw on the insight of individuals outside the agency to ensure that all
available alternatives for obtaining these data are explored. Some of the
other information CPSC needs, however, could be compiled internally at
relatively little additional cost and effort. For example, more detailed
information on individual projects could be collected within the agency.
Similarly, the methodological problems we identified in CPSC’s cost-benefit
analysis could be remedied without additional data.



Page 33                      GAO/HEHS-97-147 Consumer Product Safety Commission
                      B-274042




                      We recommend that the Chairman of CPSC take the following actions:
Recommendations
                  •   Improve the quality of CPSC’s injury, death, and exposure data by
                      consulting with experts both within and outside the agency to
                      (1) prioritize CPSC’s needs for additional statistically valid surveillance data
                      on injuries and deaths related to consumer products and on exposure to
                      consumer products and product-related hazards, (2) investigate the
                      feasibility and cost of alternative means of obtaining these data, and
                      (3) design data systems to collect and analyze this information.
                  •   Direct agency staff to develop and implement a project management
                      tracking system to compile information on current agency projects. For
                      each project, such a system should include, at a minimum, a description of
                      the hazard addressed, start and end dates, project origin, and major
                      agency action resulting from it.
                  •   Direct agency staff to develop and implement procedures to ensure that all
                      cost-benefit analyses performed on behalf of CPSC are comprehensive and
                      reported in sufficient detail, including providing measures of precision for
                      underlying data, incorporating information on all important costs and
                      benefits, and performing sensitivity analysis.


                      We received two separate sets of comments from CPSC’s
Agency Comments       commissioners—one from Chairman Brown and Commissioner Moore and
                      one from Commissioner Gall. CPSC staff also submitted some technical
                      comments, which we incorporated in the report as appropriate.

                      Chairman Brown and Commissioner Moore stated that they are
                      considering our recommendations and that in some respects our
                      recommendations parallel efforts already under way at the Commission.
                      However, they disagreed with some of the specific findings of our report.
                      They stated that CPSC’s actions are based on solid injury and death
                      estimates and that CPSC (1) employs sound economic analyses that are
                      appropriate for the circumstances, (2) tracks projects to monitor the
                      progress of its work, and (3) has been successful in dramatically reducing
                      the threat to consumers from unsafe products. We concluded that CPSC’s
                      death and injury data are generally insufficient to support the agency’s
                      project selection process for two reasons: (1) CPSC has little or no data on
                      several project selection criteria and (2) CPSC’s data on its other project
                      selection criteria exhibit significant gaps. Similarly, our finding that CPSC’s
                      cost-benefit analyses were not comprehensive and not reported in
                      sufficient detail supports our conclusion that these analyses were less
                      useful than they could have been in the agency’s decision-making process.



                      Page 34                       GAO/HEHS-97-147 Consumer Product Safety Commission
B-274042




Because CPSC’s current management information system operates at a very
high level of generality and (according to CPSC staff) does not produce
consistent, reliable information, we recommend that the agency
implement an improved tracking system that would provide enough
information to monitor the projects selected and the resources spent for
each hazard. We did not review whether CPSC’s actions had been
successful in reducing the number of injuries and deaths associated with
consumer products. The full text of Chairman Brown and Commissioner
Moore’s detailed comments and our response are in appendix VI.

Commissioner Gall stated in her comments that she is looking forward to
implementing many of the reforms we recommended. She also stated that
she agrees with our conclusion that CPSC could improve the way it gathers
information, including reassessing the need for injury data gathered
outside of hospital emergency rooms. Commissioner Gall also agreed that
CPSC needs an improved system to track ongoing activities or projects and
that sensitivity analysis, measures of uncertainty, and risk-risk analysis
should be incorporated into CPSC analyses. However, she also commented
that additional GAO analysis of the implications of inadequate data could
have been helpful. Unfortunately, available information does not permit us
to determine the impact of better-quality data on the decisions made by
CPSC. In addition, Commissioner Gall said that she believes the Compliance
function of CPSC also warrants further review. Such a review is outside the
scope of this report. Commissioner Gall’s detailed comments and our
response are in appendix VII.


As agreed with your office, unless you publicly announce its contents
earlier, we plan no further distribution of this report until 30 days after its
issue date. At that time, we will send copies to the commissioners of the
Consumer Product Safety Commission and make copies available to
others upon request.

If you or your staff have any questions about this report, please contact me
at (202) 512-7014. Major contributors are listed in appendix VIII.




Carlotta C. Joyner
Director, Education
  and Employment Issues


Page 35                       GAO/HEHS-97-147 Consumer Product Safety Commission
Contents



Letter                                                                                             1


Appendix I                                                                                        40

Scope and
Methodology
Appendix II                                                                                       43

Agency Organization
and Structure
Appendix III                                                                                      47

CPSC Projects,
1990-96
Appendix IV                                                                                       59
                        Background                                                                62
A Review of the         Conceptual, Analytical, and Technical Issues Arise When                   64
Literature on             Applying COST-BENEFIT Analysis
COST-BENEFIT
Analysis
Appendix V                                                                                        76
                        CPSC Relies Heavily on Hospital Emergency Room Reports to                 76
Death, Injury, and        Obtain Estimates of Injuries
Incident Data Sources   CPSC Obtains Most of Its Fatality Information From Selected               79
                          Death Certificates
Used by CPSC            CPSC Investigates Selected Incidents to Obtain Detailed                   81
                          Information About the Causes of Deaths and Injuries
                        CPSC Uses Modeling Techniques and Special Surveys to Estimate             82
                          Exposure Information
                        CPSC Obtains Some Information Under Reporting Requirements                83
                          for Manufacturers and Other Businesses




                        Page 36                    GAO/HEHS-97-147 Consumer Product Safety Commission
                       Contents




Appendix VI                                                                                       85
                       GAO Comments                                                              102
Comments From the
Chairman and
Commissioner Moore,
Consumer Product
Safety Commission,
and GAO’S Evaluation
Appendix VII                                                                                     106
                       GAO Comments                                                              110
Comments From
Commissioner Gall,
Consumer Product
Safety Commission,
and GAO’S Evaluation
Appendix VIII                                                                                    111

GAO Contacts and
Staff
Acknowledgments
Bibliography                                                                                     112


Tables                 Table 1: CPSC Regulatory Priority-Setting Criteria and Systematic          16
                         Information Available
                       Table 2: Evaluation of CPSC Analyses Shows Inconsistencies                 25
                         With Several Evaluation Elements
                       Table III.1: CPSC Projects, 1990-96                                        48
                       Table IV.1: Frequently Used Elements of Evaluation of                      60
                         Cost-Benefit Analysis

Figures                Figure 1: CPSC Funding, in Inflation-Adjusted Dollars, 1974-96              5
                       Figure 2: Origination of CPSC Projects, January 1,                          9
                         1990-September 30, 1996




                       Page 37                     GAO/HEHS-97-147 Consumer Product Safety Commission
Contents




Figure 3: Types of Death and Injury Data Covered by CPSC’s                15
  Systematic Surveillance Information
Figure II.1: CPSC Organization Chart                                      46




Abbreviations

AAPCC      American Association of Poison Control Centers
ANPR       Advance Notice of Proposed Rulemaking
ATV        all-terrain vehicle
CO         carbon monoxide
CPSC       Consumer Product Safety Commission
FOIA       Freedon of Information Act
HMO        health maintenance organization
IPII       Injury or Potential Injury Incident
LP         liquified petroleum
MECAP      Medical Examiners’ and Coroners’ Alert Program
MIS        management information system
NEISS      National Electronic Injury Surveillance System
NFIRS      National Fire Reporting System
NFPA       National Fire Protection Association
OMB        Office of Management and Budget


Page 38                    GAO/HEHS-97-147 Consumer Product Safety Commission
Page 39   GAO/HEHS-97-147 Consumer Product Safety Commission
Appendix I

Scope and Methodology


             This report provides information on project selection, analytical research,
             and information release procedures at the Consumer Product Safety
             Commission (CPSC), which we gathered from a variety of sources inside
             and outside the agency. For example, we interviewed current CPSC staff,
             including analysts and line managers, to obtain information about CPSC’s
             data and procedures. We also interviewed the three current CPSC
             commissioners and four former commissioners. In addition, we
             interviewed representatives of manufacturers, retailers, trade associations,
             consumer groups, and academic and other experts to obtain their
             perspectives on CPSC’s activities. We reviewed the legislation governing
             CPSC’s activities and the associated regulations. We also reviewed
             statements by public and private organizations and the academic research
             literature dealing with consumer protection issues and the technical
             literature on cost-benefit analysis and risk assessment.

             We gathered extensive information from CPSC documents. We reviewed
             every agency budget request, operating plan, mid-year review, and
             regulatory agenda from 1990 to 1996. We reviewed all CPSC’s annual reports
             from fiscal years 1986 to 1996. We also obtained all CPSC Federal Register
             notices from January 1993 to May 1996 and selected Federal Register
             notices from January 1990 to December 1992. We reviewed all CPSC press
             releases from September 1992 to February 1996. We also reviewed 644
             CPSC briefing packages prepared from January 1, 1990, to September 30,
             1996, including all briefing packages that concerned a specific, identifiable
             potential product hazard under consideration for regulatory, voluntary, or
             information-gathering activities. We excluded matters exclusively related
             to compliance with and enforcement of existing standards, civil penalties,
             and internal management issues as well as other items not related to a
             specific potential product hazard. In addition, we reviewed other agency
             documents, including information-clearing procedures, documentation of
             data systems, consumer education materials, internal memorandums and
             correspondence, and selected documents downloaded from CPSC’s Internet
             site.

             In describing CPSC’s project selection, we drew on our interviews with
             commissioners, former commissioners, and agency staff, and we reviewed
             the project selection criteria in CPSC’s regulations. Although we described
             the criteria used by the agency, we did not make judgments about the
             merit of individual criteria or about whether any criterion was
             appropriately applied in any specific case. We compiled a list of agency
             projects on the basis of the documents we reviewed. We included not only
             major regulatory efforts but also smaller-scale projects. We gathered



             Page 40                      GAO/HEHS-97-147 Consumer Product Safety Commission
Appendix I
Scope and Methodology




information on the characteristics of these projects; for example, we
obtained information on how the projects originated, what action CPSC
took, and (for most) how many deaths and injuries were associated with
the hazard. CPSC officials examined our list and provided some information
that was not readily available in agency documents. However, because
CPSC’s internal data management system does not track agency projects,
neither we nor CPSC were able to verify the accuracy and
comprehensiveness of this information. We also reviewed agency
documentation and the technical literature and interviewed outside
experts to obtain information on CPSC’s data-gathering systems.

To examine CPSC’s cost benefit analysis, we reviewed the technical
literature and developed a set of objective evaluation questions to elicit
descriptions of and to evaluate the analytical work. These evaluation
questions were designed to indicate only whether CPSC’s cost benefit
analyses were consistent with elements that are commonly used to
evaluate whether cost-benefit analyses are comprehensive and reported in
sufficient detail. Our review did not assess whether CPSC’s analyses were
the best that could be done on any particular topic. We reviewed these
evaluation questions with two leading experts in the field of risk
assessment and cost-benefit analysis: Professor John Mendeloff of the
University of Pittsburgh and Professor John Graham of Harvard
University. Then we examined all CPSC’s cost-benefit analyses from
January 1990 through September 1996 to see how they measured up
against these evaluation questions. In making these assessments, we
reviewed all the information available to the commissioners in the written
record; that is, we did not confine our analysis to the portion of the
briefing packages that dealt explicitly with cost-benefit analysis or risk
assessment, and we examined all the briefing packages that pertained to
that particular project. Although we evaluated CPSC’s cost-benefit analyses
using a wide-ranging set of evaluation questions, we reported results of
our analysis only for those questions where: (1) the question was
applicable to the majority of the CPSC analyses we reviewed; and (2) we
were able to make a determination of whether the analyses were
consistent with the evaluation question in applicable cases.

To review CPSC’s information release process, we reviewed internal agency
procedures, discussed the information release requirements with agency
officials and industry representatives, and reviewed the relevant legal
cases. We provided information on CPSC’s information release procedures
but did not audit whether CPSC complied with these procedures in
releasing information. Instead, we relied on other sources—industry



Page 41                      GAO/HEHS-97-147 Consumer Product Safety Commission
Appendix I
Scope and Methodology




sources, published legal decisions, and retractions—to assess whether this
readily available evidence suggested that CPSC had violated its statutory
requirements in releasing manufacturer-specific information.

Our review was conducted between August 1996 and May 1997 in
accordance with generally accepted government auditing standards.




Page 42                     GAO/HEHS-97-147 Consumer Product Safety Commission
Appendix II

Agency Organization and Structure


                  The Consumer Product Safety Act (P.L. 92-573) provides for the
                  appointment of five commissioners by the President of the United States
                  for staggered 7-year terms. Not more than three of the commissioners may
                  be affiliated with the same political party. The President appoints the
                  commissioners, subject to Senate confirmation. However, the President
                  cannot directly overrule an agency decision or fire a commissioner for
                  making an unpopular decision. Since 1986, there have been no more than
                  three commissioners at one time. In fulfilling the provisions of the
                  government in the Sunshine Act (P.L. 94-409),29 in general, commissioners
                  must open to the public any meeting held for the purpose of disposing of
                  CPSC business—that is, if any two of the three commissioners want to
                  discuss agency business, generally public notice must first be given. On a
                  daily basis, communication among CPSC offices takes place at the staff
                  level.

                  The chairman, the principal executive officer of the Commission, directs
                  all executive and administrative functions of the agency. The chairman
                  oversees the appointment and supervision of all agency personnel except
                  those employed in the immediate offices of other commissioners. CPSC
                  annually elects a vice chairman to act in the absence or disability of the
                  chairman or in case of a vacancy in the office of chairman.

                  As figure II.1 shows, six offices report directly to the chairman:

              •   the Office of the Secretary manages the agency’s records, publishes CPSC’s
                  public meetings calendar, and administers the requirements of the
                  Freedom of Information Act;
              •   the Office of Congressional Relations serves as CPSC’s liaison with the
                  Congress;
              •   the Office of General Counsel provides advice and counsel to the agency
                  on legal matters;
              •   the Office of the Inspector General is an independent office that
                  undertakes activities to prevent and detect waste, fraud, and abuse; and
              •   the Office of Equal Opportunity and Minority Enterprise monitors
                  compliance with equal opportunity employment laws.

                  The agency’s executive director, who is appointed by and reports directly
                  to the chairman, is responsible for overseeing the day-to-day management
                  of the agency. The executive director’s office also houses the agency’s
                  small business ombudsman, who acts as a liaison to small businesses,

                  29
                   The purpose of the Sunshine Act is to provide the public with information regarding the decision
                  making processes of the federal government while protecting both the rights of individuals and the
                  ability of the government to carry out its responsibilities.



                  Page 43                                GAO/HEHS-97-147 Consumer Product Safety Commission
    Appendix II
    Agency Organization and Structure




    providing technical assistance concerning CPSC programs and regulations,
    among other things.

    The executive director has line authority over nine operating directorates:

•   The Office of Compliance is responsible for compliance with and
    administrative enforcement of CPSC regulations, including product
    standards. The office initiates investigations on safety hazards of products
    already in the consumer marketplace or being offered for import. It
    negotiates and subsequently monitors corrective action plans designed to
    give public notice of hazards and recall defective or noncomplying
    products.
•   The Office of Hazard Identification and Reduction manages many of CPSC’s
    activities that involve identifying, examining, and remedying new product
    hazards. The office’s responsibilities include collecting and analyzing data
    to identify hazards and hazard patterns, carrying out CPSC’s regulatory and
    voluntary standards development projects, and coordinating international
    activities related to consumer product safety. Serving in various groups
    under the director of hazard identification and reduction are the
    epidemiologists and statisticians, who provide information on particular
    products; engineers and human factors specialists, who help develop
    design remedies for product hazards; and economists, who provide market
    information and perform cost-benefit analysis on commission projects.
•   The Office of Information and Public Affairs is CPSC’s touchstone with
    consumers and the media. It prepares and publishes brochures, booklets,
    fact sheets, and safety alerts providing safety information to consumers on
    products used in the home environment. The Office of Information and
    Public Affairs also handles requests from the media for information and
    access to CPSC staff and prepares press releases announcing CPSC actions
    or decisions.
•   The Office of Information Services manages the agency’s toll-free hot line,
    Internet, and fax-on-demand services. Within the Office of Information
    Services, CPSC’s Information Clearinghouse provides summary information
    on product-related injuries in response to requests from the public.30
•   The Office of the Budget is responsible for overseeing the development of
    CPSC’s budget.
•   The Office of Human Resources Management provides support to the
    agency in recruitment and placement, position classification, training, and
    other personnel areas.



    30
      If such a request can not be filled without identifying the manufacturer, however, the request is sent
    to the freedom of information officer for processing.



    Page 44                                 GAO/HEHS-97-147 Consumer Product Safety Commission
    Appendix II
    Agency Organization and Structure




•   The Office of Planning and Evaluation assists with long-term planning
    efforts and manages and ensures agency compliance with paperwork
    reduction regulations. In addition, the Office of Planning and Evaluation is
    currently preparing for the implementation of the Government
    Performance and Results Act (the Results Act) and reviewing the
    effectiveness of the agency’s outreach efforts.
•   The Directorate for Field Operations coordinates the activities of CPSC’s
    128 field staff located in 38 cities across the country. CPSC field staff carry
    out a wide range of agency activities, including conducting investigations
    of injury incidents; acting as liaisons with state and local organizations;
    working with the local press to support consumer education campaigns;
    and inspecting manufacturers, importers, distributors, and retailers to
    ensure compliance with safety regulations and standards. To complement
    the efforts of the field staff at the local level, CPSC contracts some product
    safety work to state and local entities; commissions state and local
    officials to function as officials of CPSC for the purpose of conducting
    investigations, inspections, recalls, and sample collections; and transmits
    information on CPSC programs and activities to states.
•   The Directorate for Administration is responsible for executing general
    administrative policies.




    Page 45                             GAO/HEHS-97-147 Consumer Product Safety Commission
                                       Appendix II
                                       Agency Organization and Structure




Figure II.1: CPSC Organization Chart




                                       Page 46                             GAO/HEHS-97-147 Consumer Product Safety Commission
Appendix III

CPSC Projects, 1990-96


               CPSC has addressed many different product hazards. We identified 115
               projects the agency worked on between January 1, 1990, and
               September 30, 1996. Because CPSC does not maintain a list of projects it
               has worked on or any project characteristics, such as project origin or
               result, we used various agency documents, such as budget requests,
               annual operating plans, regulatory agendas, and project briefing
               documents, to compile our list.

               We attempted to determine how each project came to the attention of the
               agency, for example, through a petition, congressional direction, or
               internal initiation. This was not always clear. For example, in three cases,
               a petition was submitted and subsequently denied, but the issue was later
               mandated by the Congress as a project the agency must address. For
               example, a petition concerning the safety of bicycle helmets submitted in
               1990 was denied. In 1994, the Congress passed the Children’s Bicycle
               Helmet Safety Act of 1994, which requires CPSC to establish a mandatory
               standard. We counted such projects as originating through the Congress.
               In other cases, we categorized projects on the basis of information in CPSC
               documents.

               We also attempted to identify the most significant action that resulted
               from the agency’s efforts. Some projects were ongoing for several years
               and involved more than one agency action, such as information campaigns
               conducted in conjunction with voluntary standards efforts. For example,
               agency work on carbon monoxide (CO) detectors resulted in a voluntary
               standard in 1992, a “CO Awareness Week” information campaign in 1994,
               and public hearings on the issue in 1996. In cases such as these, we
               attempted to identify the activity with the greatest long-lasting impact;
               usually this was a voluntary or mandatory standard. Also, projects
               classified as “voluntary standard activity” include those for which a new
               voluntary standard was created, an existing voluntary standard was
               revised, or staff are currently participating in voluntary standard activities.

               In addition, we identified whether a risk assessment or cost-benefit
               analysis was completed for a given project, using CPSC briefing packages.
               We defined a formal risk assessment to include only those cases for which
               a numerical estimate of unit risk was calculated; we defined a complete
               cost-benefit analysis to include those projects for which both economic
               costs and benefits were explicitly compared, even if quantitative estimates
               were not made for all economic factors considered. Some risk-related or
               economic information was provided for many of the projects for which no
               formal, complete risk assessment or cost-benefit analysis was performed.



               Page 47                       GAO/HEHS-97-147 Consumer Product Safety Commission
                                      Appendix III
                                      CPSC Projects, 1990-96




Table III.1: CPSC Projects, 1990-96
                                                                                          Origin
                                                                     Initiated by the                         Initiated by
                                      Product                              Congress           Petition             agency
                                      Fire/gas codes and standards
                                      Camping heaters                                                                   x
                                      Carbon monoxide
                                      detectors                                                                         x
                                      Children’s sleep-wear
                                      flammability                                                   x
                                      Cigarette lighters                                             x
                                      Cigarette safety                             x
                                      Elderly sleepwear
                                      flammability                                                                      x
                                      Fire safety devices                                                               x
                                                                                                      c
                                      Fireworks, fuse-burn time                                      x
                                      Fireworks, large and small
                                      reloadable shell                                                                  x
                                      Fireworks, multiple tube
                                      mine and shell                                                                    x
                                      Flexible gas connectors                                                           x
                                      Floor furnaces                                                 x
                                      Fuel gas detectors                                                                x
                                      Gas control valves,
                                      automatic                                                                         x
                                      Gas furnaces                                                                      x
                                      Gas grills, 20-lb. systems                                                        x
                                      Gas systems,
                                      over-pressurization                                            x
                                      Heaters, unvented gas
                                      space heaters                                                                     x
                                      Liquified petroleum (LP)
                                      gas odorant fade                                                                  x
                                      Mattress/bedding fires                                                            x
                                      Multipurpose lighters                                          x
                                      Range/oven fires                                                                  x
                                      Smoke detectors                                                                   x
                                      Upholstered furniture                                          x
                                      Water heaters                                                                     x
                                      Sports and Recreation
                                      All-terrain vehicles (ATV)                                                        x
                                                         g
                                      Baseball injuries                                                                 x
                                      Baseball bases                                                                    x



                                      Page 48                          GAO/HEHS-97-147 Consumer Product Safety Commission
                         Appendix III
                         CPSC Projects, 1990-96




                   Agency action                                                 Analytical efforts
      Mandatory          Voluntary      Information and                     Cost-benefit    Risk assessment
Ban    standard   standard activitya          education        Otherb      analysis done               done


                                   x

                                   x

  x                                                                  x
              x                                                                         x                  x
                                                                     x

                                                                     x
                                                                     x
              x                                                                         x

  x                                                                                     x                  x

              x                                                                         x
                                   x
                                                                     x                                     x
                                                                     x

                                   x
                                   x
                                   x

              x

                                   x

                                   x
                                                                     x
                                                                     xd                 x
                                                                     x
                                   x
                                   xe
                                   x                                                    x


                                                                     xf                                    x
                                                                     x
                                   x
                                                                                                  (continued)


                         Page 49                          GAO/HEHS-97-147 Consumer Product Safety Commission
Appendix III
CPSC Projects, 1990-96




                                                   Origin
                              Initiated by the                         Initiated by
Product                             Congress           Petition             agency
Baseball, chest protectors
for young batters                                                                x
Baseball, face-guards on
helmets                                     x
Baseball, safety baseballs
(soft)                                                                           x
Bicycles                                                                         x
Bicycle reflectors                                                               x
Bicycle/recreation helmets                  xh
In-line skates                                                                   x
Model rocket motors                                           x
Pools, barriers for
swimming pools and spas                                                          x
Pools, diving injuries                                                           x
Pools, swimming pool
covers                                                        x
Saunas                                                        x
Soccer injuries                                                                  x
Soccer goals                                                                     x
Spas and hot tubs                                             x
Electrical/power codes and standards
Garage door openers                         x
Go-carts                                                                         x
Hair dryers                                                   x
Heaters, portable electric                                    x
Heat tapes                                                    x
Holiday lights                                                                   x
Home electrical system
fires                                                                            x
Ladders, portable
aluminum                                                      x
Lamps, portable                                                                  x
Microwave ovens                                                                  x
Mowers, riding                                                                   x
Mowers, walk-behind                                                              x
Receptacle outlets                                                               x
Shock protection devices
(GFCI)                                                                           x




Page 50                         GAO/HEHS-97-147 Consumer Product Safety Commission
                         Appendix III
                         CPSC Projects, 1990-96




                   Agency action                                                Analytical efforts
      Mandatory          Voluntary     Information and                     Cost-benefit     Risk assessment
Ban    standard   standard activitya         education        Otherb      analysis done                done

                                   x

              x                                                     x                  x

                                   x
                                                     x                                                    x
                                                                    x
              x                                                                                           x
                                                                    x
                                                                    x

                                   x
                                   x

                                   x
                                   x
                                                                    x
                                   x
                                   x                                                                      x


              x                                                                        xi
                                   x
                                   x
                                                                    x
                                                                    x                  x
                                   x

                                   x

                                                                    x                  x                  x
                                   x
                                   x
                                   x                                                   x                  x
                                                                    x
                                   x

                                   x                                                                      x
                                                                                                 (continued)




                         Page 51                         GAO/HEHS-97-147 Consumer Product Safety Commission
Appendix III
CPSC Projects, 1990-96




                                                   Origin
                              Initiated by the                         Initiated by
Product                             Congress           Petition             agency
Thermoplastics in
electrical products                                                              x
Children’s products
Baby bath seats                                                                  x
Baby walkers                                                  x
Bean bag chairs                                                                  x
Bunk beds                                                     x
Child restraints on grocery
carts                                                         x
Choking
hazards—balloons, balls,
marbles, x small figures,
pom-poms
Clacker balls                                                 x
Crib corner post
extensions                                                    x
Cribs, nonfull size                                                              x
Crib toys                                                     x
Drawstrings on children’s
clothing                                                                         x
Electronic video games                                        x
5-gallon buckets                                                                 x
Furniture tipover                                                                x
High chairs                                                                      x
Infant cushions                                                                  x
Infant suffocation                                                               x
Lead in paint                                                                    x
Playground equipment,
home                                                          x
Playground equipment,
public                                                                           x
Playground equipment,
soft                                                                             x
Playground surfacing                                                             x
Play yards                                                                       x
Strollers                                                                        x
Toddler beds                                                                     x
Window blind cords                                                               x
Window guards                                                                    x
Chemical/poison prevention



Page 52                         GAO/HEHS-97-147 Consumer Product Safety Commission
                         Appendix III
                         CPSC Projects, 1990-96




                   Agency action                                                 Analytical efforts
      Mandatory          Voluntary      Information and                     Cost-benefit    Risk assessment
Ban    standard   standard activitya          education        Otherb      analysis done               done

                                                                     x


                                                      x
                                   xd                                                   x                  x
                                   x
                                   x

                                                      x                                 x                  x



  x                                                                  x
              x                                                                         x

                                   x                                                                       x
                                   x
                                   x

                                   x
              x                                                                         x
                                   x                                                                       x
                                   x
                                   x
  x                                                                                     x
                                                      x
                                                                     x

                                   x                                                    x                  x

                                   x                                                                       xj

                                   x
                                   x
                                   x
                                   x
                                   x
                                   x
                                   x


                                                                                                  (continued)


                         Page 53                          GAO/HEHS-97-147 Consumer Product Safety Commission
Appendix III
CPSC Projects, 1990-96




                                                   Origin
                              Initiated by the                         Initiated by
Product                             Congress           Petition             agency
Acetone in 1-gallon
containers                                                    xk
Architectural glazing                                         x
Art materials                               x
Charcoal labeling                                             x
Child-resistant,
adult-friendly packaging
of drugs                                                                         x
Child-resistant packaging
of certain dietary products
with iron powders                                             x
Child-resistant packaging
of dibucaine                                                                     x
Child-resistant packaging
of ibuprofen                                                                     x
Child-resistant packaging
of drugs with loperamide                                                         x
Child-resistant packaging
of glue removers with
acetonitrile                                                  xl
Child-resistant packaging
of hair wave neutralizers                                     xl
Child-resistant packaging
of lidocaine                                                                     x
Child-resistant packaging
of mouthwashes with
ethanol                                                       x
Child-resistant packaging
of naproxen                                                                      x
Child-resistant packaging
of spot remover with
naptha                                                        xm
Coal- and woodburning
stove emissions                             x
Dichlorobenzene                                                                  x
Dioxin in paper products                                                         x
Exemption to
child-resistant packaging
of mouthwashes with
ethanol                                     x
Indoor air quality                          x
Indoor air quality—carpet
emissions x




Page 54                         GAO/HEHS-97-147 Consumer Product Safety Commission
                         Appendix III
                         CPSC Projects, 1990-96




                   Agency action                                                Analytical efforts
      Mandatory          Voluntary     Information and                     Cost-benefit    Risk assessment
Ban    standard   standard activitya         education        Otherb      analysis done               done

                                                                    x
                                                                    x                  x
              x
              x                                                                        x                  x


              x                                                                        x


              x

              x                                                                        x

              x                                                                        x

              x                                                                        x


              x                                                                        x

              x                                                                        x

              x                                                                        x


              x                                                                        x

              x


                                                                    x

                                                                    x                                     x
                                                                    x                                     x
                                                                    x                                     x



                                                     x
                                                                    x

                                   x                                                   x
                                                                                                 (continued)


                         Page 55                         GAO/HEHS-97-147 Consumer Product Safety Commission
Appendix III
CPSC Projects, 1990-96




                                                    Origin
                               Initiated by the                         Initiated by
Product                              Congress           Petition             agency
Indoor air
quality—formaldehyde in
pressed x wood products
Indoor air quality—glycol
ethers x
Indoor air
quality—heating,
ventilation, and x
air-conditioning systems
Indoor air
quality—kerosene heaters
x
Indoor air
quality—portable x room
humidifiers
Lead poisoning abatement                                                          x
Methylene chloride in
paint strippers                                                x
Sulfuric acid drain cleaners                                   x
Other
Waterbeds                                                      x
Total 115                                   13               34                  68




Page 56                          GAO/HEHS-97-147 Consumer Product Safety Commission
                         Appendix III
                         CPSC Projects, 1990-96




                   Agency action                                                 Analytical efforts
      Mandatory          Voluntary      Information and                     Cost-benefit    Risk assessment
Ban    standard   standard activitya          education        Otherb      analysis done               done


              x

                                                      x



                                                      x


              x


              x
                                    x

                                                                     x                                    x
                                                                     x


                                    x                                                                     x
  2          21                    55                5             32                 29                 24




                         Page 57                          GAO/HEHS-97-147 Consumer Product Safety Commission
Appendix III
CPSC Projects, 1990-96




a
 “Voluntary standard activity” includes efforts in which agency staff participated in the
development of a new voluntary standard or revision of an existing voluntary standard. This
participation included regularly attending meetings of a standards development group; taking an
active part in discussions; and providing research or other support, information and education
programs, and administrative assistance.
b
 “Other” includes projects that were completed with no agency action, terminated, currently in
process or pending, or completed with a study or report.
c
 Two petitions were submitted for fuse-burn time. The first, submitted in 1991, was denied. The
second, submitted in 1996, resulted in a change to an existing mandatory standard.
d
 CPSC published an Advance Notice of Proposed Rulemaking (ANPR). This is the first step
toward enacting a mandatory standard.
e
 CPSC published an ANPR in 1994, which is the first step toward enacting a mandatory standard.
The ANPR also solicited offers to develop or modify a voluntary standard. During fiscal year 1996,
CPSC worked with an industry group on voluntary standards development.
f
    CPSC has worked with states to develop state ATV legislation.
g
 The baseball injuries project started in the 1980s and resulted in a special report on baseball
injuries and protective equipment. The projects—baseball bases, chest protectors, faceguards
on helmets, and safety baseballs—all grew out of this umbrella project and, as such, are counted
separately.
h
 A petition was also submitted in 1990. The Congress mandated this project in the Children’s
Bicycle Helmet Safety Act of 1994. Project origin is counted as congressionally initiated.
i
The cost-benefit analysis for garage door openers applied to the labeling and record-keeping
requirements only, not to the provisions of the safety standard.
j
The risk assessment for playground equipment examined the hazard associated with exposure to
arsenic from wooden equipment. The risk assessment concluded that the risk was small. Most of
CPSC’s efforts concerning playground equipment are related to injuries from falls or impacts of
equipment and are not related to chemical exposure.
k
    Petition, submitted in 1995, was withdrawn.
l
One petition was submitted in 1988 by the American Association of Poison Control Centers
requesting CPSC to require child-resistant packaging both for household glue removers
containing acetonitrile and home cold wave permanent neutralizers containing sodium bromate or
potassium bromate.
m
    Petition, submitted in 1991, was withdrawn.




Page 58                                  GAO/HEHS-97-147 Consumer Product Safety Commission
Appendix IV

A Review of the Literature on
COST-BENEFIT Analysis


               Cost-benefit analysis can be described as an analytical technique that
               details the expected positive and negative effects of a given policy
               proposal (expected benefits and costs). To construct this framework,
               researchers approach a policy question needing considerable information
               about the potential remedy. Because cost-benefit analysis requires an
               inclusive approach to evaluating a proposal’s impact, the proposal itself
               must be well defined, and some information must be known about its
               impact. Once the expected benefits and costs of the proposal are
               thoroughly identified and delineated, the next step is to place a value on
               each benefit or cost. Frequently, these expected costs and benefits are
               expressed in numerical or monetary terms to facilitate a comparison of the
               aggregate costs and benefits. If all relevant factors can be translated into
               monetary terms, the decision rule suggested by this proceeding is to
               accept the proposal if its aggregate benefits exceed its aggregate costs.31

               Although at first the concept behind cost-benefit analysis seems relatively
               straightforward, the application of cost-benefit analysis is not. The
               practical difficulties associated with measuring effects, quantifying results,
               and accounting for uncertainty (to name only a few issues) can create a
               gap between the way cost-benefit analysis is described in theory and the
               way it is implemented in practice. For this reason, experts generally agree
               that analysts should be comprehensive in including all important factors
               and be explicit in their description of the underlying data, methodology,
               and assumptions.

               This appendix describes many of the major methodological issues that
               often arise in cost-benefit analysis and outlines specific elements that are
               frequently used to evaluate cost-benefit analyses. The methodology used
               to conduct a cost-benefit analysis frequently varies depending on the
               circumstances and the context of the analysis. For this reason, there is no
               complete set of standards for evaluating the quality of an individual
               cost-benefit analysis. However, the professional literature offers some
               guidance for analysts, and certain specific elements are frequently used to
               determine whether a given analysis meets a minimum threshold of
               comprehensiveness and openness. These elements are necessary, but not
               sufficient, for a quality analysis.


               31
                 Variations of cost-benefit analysis include cost-effectiveness analysis, which relates the costs and
               outcomes of alternative methods to achieve the same desired goal. For example, one
               cost-effectiveness analysis compared the cost per case detected for different methods of treating the
               same medical condition. Because only cost-benefit analysis is relevant to CPSC’s activities, related
               techniques are not considered here. For more information on these and other related analytical tools,
               see Michael F. Drummond, Greg L. Stoddart, and George W. Torrance, Methods for the Economic
               Evaluation of Health Care Programmes (New York: Oxford University Press, 1996).



               Page 59                                GAO/HEHS-97-147 Consumer Product Safety Commission
                                         Appendix IV
                                         A Review of the Literature on
                                         COST-BENEFIT Analysis




                                         From the extensive cost-benefit literature, we developed objective
                                         evaluation questions to evaluate cost-benefit analyses performed by CPSC.
                                         These evaluation questions are summarized in table IV.1. Although these
                                         evaluation questions are not a comprehensive measure of the quality of an
                                         analysis, they were designed to reflect whether an analysis is
                                         comprehensive and reported in sufficient detail. We applied these
                                         questions to each of CPSC’s analyses, but not all evaluation questions were
                                         applicable to each case. In addition, for some questions it was not always
                                         possible to determine whether CPSC’s analysis complied with the particular
                                         element reflected in the question. For these reasons, and to emphasize
                                         those areas that we considered as most critical, we reported only the
                                         evaluation results that related to key elements and that applied to the
                                         majority of CPSC’s analyses; these eight are shown in bold in table IV.1.

Table IV.1: Frequently Used Elements
of Evaluation of Cost-Benefit Analysis   Number          Evaluation question
                                         Evaluating potential courses of action
                                         1               Was descriptive information given about well-defined alternative
                                                         courses of action?
                                         2               Was more than one alternative course of action considered in the
                                                         analysis?
                                         3               Was evidence given to support the degree to which the proposal was
                                                         assumed to mitigate the problem?
                                         Considering all important factors for analysis
                                         4               Were all important categories of potential costs and benefits
                                                         included in the analysis? (For example, did the analysis include [where
                                                         applicable] intangible benefits and costs, health and safety benefits,
                                                         costs of compliance, upfront costs, price changes, and changes in
                                                         consumer surplus?)
                                         5               Were the effects of intangible costs and benefits detailed explicitly, even
                                                         if they could not be quantified? (For example, did the analysis include
                                                         [where applicable] consumer utility?)
                                         6               Did the analysis show how the existence of intangible costs or benefits
                                                         could affect the conclusions?
                                         7a              Were possible indirect effects discussed? (For example, did the analysis
                                                         include [where applicable] price changes of related goods or likely
                                                         changes in market concentration?)
                                         7b              If indirect effects were measured quantitatively, were the calculations
                                                         and assumptions behind this measurement discussed in detail?
                                         7c              If indirect effects were measured qualitatively, did the analysis show how
                                                         the existence of these factors could affect the conclusions?
                                         Considering interdependent factors
                                         8               Were possible issues of standing identified?
                                         9               Was an incremental (marginal) analysis of costs and benefits performed?
                                                                                                                        (continued)



                                         Page 60                            GAO/HEHS-97-147 Consumer Product Safety Commission
Appendix IV
A Review of the Literature on
COST-BENEFIT Analysis




Number          Evaluation question
10              Were major policy interdependencies identified? (For example, did the
                analysis discuss the implications if another agency also had jurisdiction
                over the product, or if the proposal could create a conflict with existing
                government policy?)
Considering distribution issues
11              Did the analysis make clear the distribution of gains and losses? (For
                example, who would pay the costs and who would get the benefits of the
                proposal?)
12a             Was the impact of distribution of gains and losses assessed? (For
                example, would the proposal be likely to have a greater adverse impact
                on small businesses, potentially leading to increased market
                concentration?)
12b             Were distributional weights employed in the analysis?
12c             If weights were used, was information provided about the basis for those
                weights?
13a             Is the standard assumption that relative market prices are insensitive to
                the policy change likely to hold in this case—that is, are there likely to be
                few or no macroeconomic effects?
13b             If no, was this impact discussed or considered in the analysis?
Considering risk-risk factors and potential offsetting behaviors
14              Were risk-risk or offsetting behavior concerns identified and
                considered in the analysis?
Using the willingness-to-pay approach to value risk reduction
15              Were willingness-to-pay measures used to value reductions in the risk of
                death or injury?
16              Is the numerical value of a statistical life used in this analysis consistent
                with the literature?
17              Is the numerical value of a reduction in the chance of injury consistent
                with the literature?
Discounting future costs and benefits
18a             Are costs and benefits that occur at different points discounted for
                differences in timing?
18b             Does the discount rate or rates used include the suggested rate by the
                Office of Management and Budget?
18c             Does the discount rate or rates used include the Treasury bill rate for the
                time horizon of the analysis?
18d             Does the discount rate or rates used include a lower “social” discount
                rate?
Considering data issues
19a             Where applicable, were the implications of unverified or unverifiable data
                provided by an interested party identified and discussed?
19b             Where applicable, were the implications of uncertainty surrounding a
                dose-response model identified and discussed?
19c             Where applicable, were the implications of uncertainty surrounding
                crossspecies extrapolation identified and discussed?
                                                                                  (continued)


Page 61                            GAO/HEHS-97-147 Consumer Product Safety Commission
             Appendix IV
             A Review of the Literature on
             COST-BENEFIT Analysis




             Number            Evaluation question
             19d               Where applicable, were the implications of the use of data relying on
                               investigated or reported cases identified and discussed?
             19e               Where applicable, were the implications of survival bias in the underlying
                               data identified and discussed?
             19f               Where applicable, were the implications of small sample sizes
                               identified and discussed?
             19g               Where applicable, were the implications of other known biases in
                               the underlying data identified and discussed?
             20                If the underlying data were derived from a statistical sample, were
                               appropriate measures of precision provided?
             Performing sensitivity analysis
             21a               Was sensitivity analysis performed on any parameter in the
                               analysis?
             21b               Was sensitivity analysis performed on the value of a statistical life?
             21c               Was sensitivity analysis performed on the value of injury reduction?
             21d               Was sensitivity analysis performed on the discount rate?
             21e               Was sensitivity analysis performed on the precision of the underlying
                               data?
             21f               Was sensitivity analysis performed on other important parameters?

             Note: We reported results for questions that are in bold. We viewed these measures as especially
             important, and they applied to the majority of CPSC analyses and a determination could be made
             for these questions in each case.




             As an organizing framework, cost-benefit analysis can help a
Background   decisionmaker to organize and aggregate all the relevant information in a
             way that can clarify the nature of the trade-offs involved in a decision. At
             the same time, by providing a framework to convert dissimilar effects to a
             common measurement, cost-benefit analysis can allow this information to
             be weighed and aggregated to help make a decision. A well-constructed
             cost-benefit analysis can highlight crucial factors, expose possible biases,
             and perhaps expand the openness of the decision-making process by
             clarifying the factors on which the decision was based—whether these
             factors are purely economic criteria or include other social factors. A key
             advantage of carefully built cost-benefit analysis is that it promotes
             explicit rather than implicit decision-making, even when it is impossible to
             monetize or even quantify all the potential effects of a given regulatory
             proposal.

             Despite the value of this analytical tool, in some situations using
             cost-benefit analysis as the sole basis for decision-making may be



             Page 62                               GAO/HEHS-97-147 Consumer Product Safety Commission
Appendix IV
A Review of the Literature on
COST-BENEFIT Analysis




inappropriate. For example, alternatives that would eliminate basic human
rights or dramatically increase income inequality may be viewed as
morally unacceptable. Similarly, when uncertainty about possible effects
is so pervasive that attempting to identify potential costs and benefits
would amount to nothing more than uninformed speculation, cost-benefit
analysis probably has little to offer.32 In addition, for minor decisions it
may not be necessary to employ a rigorous, time- and resource-consuming
decision-making process, and a less detailed cost-benefit analysis—or
none at all—may be more appropriate. Virtually all observers agree that
the appropriate role for cost-benefit analysis—sole decision-making rule,
input into decision-making, or not done at all—will depend on the context
in which the particular decision is being considered.

To some individuals, application of an abstract analytical technique such
as cost-benefit analysis is especially unpalatable in certain situations. For
example, the idea of applying a value to “saving life” may be distasteful,
because we regard our own lives as “priceless” or of infinite value.
However, although our lives may be priceless, avoiding risks often means
forgoing time, convenience, enjoyment, or other opportunities—all of
which do have a price. Thus, policy interventions that can affect life
expectancy pose an unavoidable problem—to refuse to consider the value
of potentially lifesaving interventions is to implicitly value them at zero,
and to consider any potentially lifesaving activity as infinitely valuable
implies that individuals would never take any action that involves risk
(driving a car, for example). The literature on cost-benefit analysis makes
a key distinction in this area. For ethical reasons, most practitioners
consider it inappropriate to use cost-benefit analysis to evaluate
alternatives that would (with certainty) affect the life expectancy of a
given, known individual. But government policy does not usually involve
making such decisions. Instead, policy questions typically center on
actions that may bring about small changes in the statistical life
expectancy of anonymous members of a large group. For example, when
CPSC considered imposing a mandatory regulation on large, multiple-tube
fireworks, the agency estimated that such a standard could reduce the
number of individuals that die in related accidents by one anonymous

32
  However, one should use caution in assuming that this situation applies. Sometimes the conclusions
of a cost-benefit analysis can be quite robust even in the face of substantial uncertainty. For example, a
recent analysis of the benefits and costs of reducing emissions of acrylonitrile concluded that
imposing strict standards on acrylonitrile emissions fails reasonable cost-benefit tests, despite as much
as a threefold factor of uncertainty in acrylonitrile emissions, additional uncertainty in the amount of
exposure to acrylonitrile, and a probable but unmeasurable upward bias in the estimate of cancer risk.
(See John A. Haigh, David Harrison, Jr., and Albert L. Nichols, “Benefit-Cost Analysis of Environmental
Regulation: Case Studies of Hazardous Air Pollution,” ch. 1 in The Moral Dimensions of Public Policy
Choice: Beyond the Market Paradigm, eds., John Martin Gilroy and Maurice Wade (Pittsburgh, Pa.:
University of Pittsburgh Press, 1992), pp. 15-57.



Page 63                                 GAO/HEHS-97-147 Consumer Product Safety Commission
                         Appendix IV
                         A Review of the Literature on
                         COST-BENEFIT Analysis




                         consumer over a 3-year period. On a daily basis, each of us makes such
                         trade-offs between perfect safety and other things we value, such as the
                         convenience of driving a car or the excitement of skiing down a mountain.
                         Such assessments, which place a value on reductions in risk, are viewed as
                         appropriate, whereas valuing a given person’s life may be viewed as less
                         reasonable.

                         Similarly, a decision to surrender basic rights—such as liberty—may be
                         unacceptable on moral grounds, and so cost-benefit analysis might not be
                         applicable in a situation involving such rights. In addition, some
                         individuals object to cost-benefit analysis in circumstances in which other
                         species may be affected adversely. For example, one individual objected
                         to a proposal that would allow increased pollution in a particular river
                         because the proposal would adversely affect the fish and other species in
                         the river. Cost-benefit analysis would probably be unable to address this
                         objection because there is no accepted method to place a value on the
                         losses suffered by the fish.


                         Over the past 3 decades, a substantial economic literature has explored
Conceptual,              the conceptual, analytical, and technical issues posed by the application of
Analytical, and          cost-benefit analysis. From this literature, generally accepted standards of
Technical Issues Arise   professional practice have emerged that cover a wide range of research
                         methods. As one experienced researcher has said, “Good studies follow
When Applying            procedures that are in accord with economic theory for estimating
COST-BENEFIT             benefits and costs, provide a clear statement of all assumptions, point out
                         uncertainties where they exist, and suggest realistic margins of error.”33
Analysis                 Even when undertaken by careful and competent researchers, cost-benefit
                         analysis can sometimes be difficult to interpret, especially when
                         uncertainty is substantial and information is incomplete.


Conceptual Issues in     Some of the issues underlying the application of cost-benefit analysis are
Cost-Benefit Analysis    conceptual, and the researcher may make a different judgment in different
                         circumstances. With respect to these issues, being complete and explicit is
                         important—the consensus in the literature is that while there may be no
                         single method for dealing with these issues that is universally appropriate,
                         the researcher must be clear and direct in detailing how the issue was
                         addressed in the context of the analysis.



                         33
                           See Eban Goodstein, “Benefit-Cost Analysis at the EPA,” The Journal of Socio-Economics, 24
                         (2) (1995), pp. 375-89.



                         Page 64                               GAO/HEHS-97-147 Consumer Product Safety Commission
                                Appendix IV
                                A Review of the Literature on
                                COST-BENEFIT Analysis




Intangible or Unquantifiable    Ideally, a cost-benefit analysis involves translating each impact into a
Factors Can Pose Dilemmas for   common measurement (such as dollars) for comparison. However, some
Researchers                     effects may be difficult or impossible to measure or quantify. For example,
                                a researcher evaluating an alternative outpatient mental health treatment
                                program realized that the greater independence afforded by the outpatient
                                program could create increased anxiety—or, alternatively, higher
                                self-esteem—for some participants. However, these intangible effects
                                would be very difficult to measure and even more difficult to quantify.34
                                Some individuals have criticized certain individual practitioners of
                                cost-benefit analysis for ignoring or de-emphasizing aspects of proposed
                                changes that cannot be easily quantified. Although it is necessary for such
                                effects to be described and emphasized appropriately, the existence of
                                such intangibles does not necessarily limit the value of the analysis. For
                                example, in the discussion of mental health programs, the researchers
                                found that patients in the experimental program experienced higher
                                satisfaction and reported having a greater number of social relationships.
                                These qualitative benefits were important to the analysis even though they
                                could not be valued in dollars. In addition, if it can be shown explicitly that
                                the value of the intangibles would be unlikely to change the conclusion,
                                then cost-benefit analysis has played a valuable role by considering
                                intangible effects explicitly, even if it is not possible to consider them
                                quantitatively. (We addressed this issue in questions 5 and 6 of table IV.1.)

Measuring Indirect and          Researchers must also decide whether or not to include indirect or
Secondary Costs and Benefits    secondary effects that may result from the proposal. For example, a
Can Be Difficult                change in tax rules may not only have an initial, direct effect on
                                individuals’ income but may also create a secondary ripple (or
                                “multiplier”) effect on the economy as a whole. Similarly, a medical
                                treatment that prolongs life can be expected to have a secondary effect on
                                health care costs as individuals live longer.

                                A researcher who wants to include such secondary impacts will frequently
                                find a measurement challenge, because determining the magnitude (or
                                perhaps even the existence) of many secondary effects involves answering
                                a “what if?” type of question, and frequently without relevant historical
                                experience. For example, individuals who receive improved treatment for
                                hypertension may contract more prolonged or costly illnesses in the
                                future.35 Any attempt to measure the impact of these indirect future costs

                                34
                                  See Burton A. Weisbrod, “Benefit-Cost Analysis of a Controlled Experiment: Treating the Mentally
                                Ill,” Journal of Human Resources, 16 (4) (1981), pp. 523-48.
                                35
                                 For an example of such an analysis, see W.B. Stason and M.C. Weinstein, “Allocation of Resources to
                                Manage Hypertension,” The New England Journal of Medicine, 296 (13) (1977), pp. 732-39.



                                Page 65                                GAO/HEHS-97-147 Consumer Product Safety Commission
                               Appendix IV
                               A Review of the Literature on
                               COST-BENEFIT Analysis




                               would require the researcher to predict the future health status and health
                               care costs of the patient group in the event that the treatment had or had
                               not been available. In addition, the existence of some secondary effects is
                               dependent on one or more key assumptions. For example, an economic
                               multiplier effect is generally thought to take place only in an economy
                               operating below its productive capacity. A researcher who wants to
                               include such an effect must thus determine whether or not this
                               assumption holds, and then decide how large an effect should be included.

                               Secondary impacts may be considered part of the cost-benefit analysis or
                               extraneous to it. The proper choice in each circumstance probably
                               depends on the likelihood that the secondary effects in fact exist, their
                               probable importance, and the ability to measure them. However, like
                               intangible effects, potential secondary or indirect effects should be
                               detailed even if they are not included quantitatively. Furthermore, should
                               such effects be included, the assumptions underlying their existence and
                               measurement must be revealed explicitly to allow the reader to make an
                               informed judgment. (We addressed this issue in question 7 of table IV.1.)

Issues of “standing” May       In some circumstances, it may be difficult to define the point of view from
Require Clarification of the   which the cost-benefit analysis is calculated—that is, who has “standing,”
Scope of the Analysis          or whose costs and benefits should be included in the analysis. For most
                               cost-benefit calculations involving government policy, the analysis is
                               appropriately done at the level of “society” (rather than, for example,
                               considering only the impact on individuals in a particular state or locality),
                               so that a wide range of implications is considered. However, in some
                               circumstances, it is important to make the point of view explicit. For
                               example, when policy effects may cross national boundaries, “society”
                               may be best defined on a broader basis. If the cost-benefit analysis is
                               considering policies to reduce acid rain, for example, considering the
                               effects of acid rain and pollution reduction on the Canadians who share
                               U.S. weather patterns may be important. Similarly, analysts may wish to
                               consider the impact of product regulation on foreign producers as well as
                               on U.S. consumers.

                               Even when the unit of analysis is clear, occasionally some members of that
                               unit are not afforded standing under cost-benefit analysis. For example,
                               suppose that a particular policy proposal is likely to result in a decrease in
                               property crime. If a thief steals a television, and both the victim and the
                               thief are given standing, then the gain to the thief could offset the loss to
                               the victim in a cost-benefit accounting. “Society,” however, is clearly
                               worse off. Unless we try to measure the psychological and sociological



                               Page 66                         GAO/HEHS-97-147 Consumer Product Safety Commission
                                  Appendix IV
                                  A Review of the Literature on
                                  COST-BENEFIT Analysis




                                  costs of crime, it may make more sense not to afford the thief standing in
                                  the cost-benefit analysis. Such issues arise infrequently but should be
                                  made clear if they could affect the interpretation of the analysis. (We
                                  addressed this issue in question 8 of table IV.1.)

Taking the Status Quo as          Some cost-benefit analyses have been criticized for excluding relevant
Baseline, Cost-Benefit Analysis   factors on what appears to be an ad hoc basis. Without including such
Should Account for Relevant       factors, or providing an explicit justification for excluding them, a
Factors and Constraints           researcher limits the value of the analysis. However, while it is important
                                  to consider a variety of alternative actions, it is equally crucial to adopt a
                                  realistic view of the possible alternatives. For example, political and
                                  agency realities often place meaningful constraints on options available
                                  for consideration. If these constraints are not incorporated into the
                                  analysis, the results could be “pie in the sky” recommendations that
                                  would be of little use of the decisionmaker. (We addressed these issues in
                                  questions 1, 2, 4, and 10 of table IV.1.)

                                  Similarly, it is important that cost-benefit analysis be conducted in a way
                                  that evaluates the change in aggregate costs and benefits, with current
                                  conditions serving as a baseline. For the properties of economic efficiency
                                  to hold, cost-benefit analysis must take what economists call a marginal or
                                  incremental approach—that is, it must consider only the changes that
                                  would result from the proposed intervention. For example, a proposal to
                                  renovate a museum should be evaluated, for cost-benefit purposes, on the
                                  basis of the incremental cost of the renovation, not on the basis of the
                                  original cost of constructing the entire museum. (We addressed this issue
                                  in question 9 of table IV.1.)

Issues of Distribution May Be     By definition, cost-benefit analysis is a method for considering the
Considered Within or Outside      aggregate effects of a given proposal. If the aggregate expected benefits of
the Cost-Benefit Context          the proposal exceed the aggregate expected costs, the proposal is said to
                                  pass the cost-benefit test. This does not necessarily mean that adopting the
                                  proposal would improve the condition of each individual; instead, it
                                  implies only that when the expected gains and losses to all individuals are
                                  added up, the total expected gains exceed the total expected losses.36



                                  36
                                    Economists make a distinction between events that lead to a “Pareto improvement” and those that
                                  lead only to a “potential Pareto improvement.” If a change leads to a Pareto improvement, then as a
                                  result of the change no individual will be worse off, and at least one individual will be better off.
                                  However, if a change leads to a “potential Pareto improvement,” then the aggregate gains exceed the
                                  aggregate losses, but some individuals will gain and others will lose. In this situation, it would be
                                  theoretically possible for the “winners” to compensate the “losers” and leave no individual worse off
                                  than before if redistribution itself is costless.



                                  Page 67                                GAO/HEHS-97-147 Consumer Product Safety Commission
Appendix IV
A Review of the Literature on
COST-BENEFIT Analysis




By adding up individual gains and losses to determine the effect on society
as a whole, cost-benefit analysis implicitly assumes that each individual’s
gains or losses should be valued equally with any other individual’s gains
or losses. As a result, cost-benefit analysis is “neutral” with respect to the
distribution of gains and losses. To put it another way, the “fairness” of
how gains and losses are distributed is generally not included in the
calculations underlying a cost-benefit analysis.

Sometimes a decisionmaker might want to address such issues of fairness.
For example, if a proposal would involve redistributing income, the loss of
one dollar to a wealthy person may be viewed as less consequential than
the gain of one dollar to a poor person.37 In order to address such an issue,
the researcher or decisionmaker can consider issues of distribution inside
or outside the cost-benefit context. One method of examining such issues
might be to analyze the distributional impact of a proposal separately, and
consider fairness issues—along with the cost-benefit calculation—as
another factor in the decision-making process. Under these circumstances,
if the proposal is viewed as having negative distributional consequences,
and it would be costly or difficult to redistribute the gains and losses, then
the proposal might be rejected even though it would otherwise meet a
cost-benefit test.

Occasionally a researcher who wants to consider distributional issues
explicitly chooses to incorporate distributional consequences into the
mathematics of the cost-benefit calculation. Instead of simply adding up
the costs and benefits accruing to each individual, the researcher uses a
mathematical formula that applies different weights to different
individuals. These weights could be based on a number of factors,
depending on the characteristics of the “fairness” issues being addressed.
For example, if the proposal would affect income directly, weights could
be based on income (with greater weight being applied to gains in income
experienced by low-income individuals). Weights could also be based on
other circumstances; for example, greater weight could be placed on
vulnerable populations or on future generations. Similarly, if a change was
proposed in a given program, greater weight could be applied to the


37
  There is some limited ground in utilitarian economic theory for this assumption. Economic theory
states that individuals will generally, at some point, experience “diminishing marginal utility of
income”—that is, receiving one additional dollar of income brings less additional satisfaction (or
“utility”) as the individual becomes wealthier. However, the amount of satisfaction deriving from any
good or service, including income, cannot be compared across individuals. As a result, while economic
theory suggests that it is plausible that at some point giving one dollar to a rich person produces less
total “utility” than giving one dollar to a poor person, economic theory also tells us that it is generally
not possible to measure the circumstances under which this assumption might (or might not) be
correct.



Page 68                                  GAO/HEHS-97-147 Consumer Product Safety Commission
                               Appendix IV
                               A Review of the Literature on
                               COST-BENEFIT Analysis




                               smaller number of program participants (who would be more significantly
                               affected) than the rest of society (who would be affected only indirectly).
                               A weighting scheme would be most helpful to decisionmakers and outside
                               observers, however, if the formula was detailed explicitly and the analysis
                               was accompanied by a sensitivity analysis, so the effect of the
                               distributional weights was clear to the reader. (We addressed these issues
                               in questions 11 and 12 of table IV.1.)


Implementation Issues in       While a number of conceptual issues arise in cost-benefit analysis for
Cost-Benefit Analysis          which the appropriate answer depends on circumstances, there are also a
                               number of methodological or implementation issues about which there is
                               widespread agreement. For example, years of debate on the appropriate
                               valuation method for risk reduction have largely been resolved in the favor
                               of “willingness to pay” measures, and the importance of sensitivity
                               analysis is generally recognized. While the choice of discount rate has long
                               been a subject of controversy, with some practitioners arguing for the use
                               of market rates and others advocating a lower “social” discount rate, the
                               literature generally recognizes the value of multiple rates. Finally, there is
                               virtually universal agreement on the importance of reliable data and
                               careful risk measurement to cost-benefit analysis.

Costs and Benefits Should Be   Frequently, the various consequences of the proposal under discussion
Adjusted for Differences in    will differ in when they occur. For example, changing the labeling
Timing                         requirements on bags of charcoal could reduce carbon monoxide deaths
                               years later and also result in an immediate, one-time increase in industry
                               costs. As individuals and as a society, we generally prefer to have dollars
                               or resources now than at some time in the future because we can benefit
                               from them in the interim. In addition, if we acquire $1 next year instead of
                               today, we give up the opportunity to invest that dollar and earn interest on
                               our investment. As a result, it is generally agreed that future dollar cost
                               and benefit streams should be reduced or “discounted” to reflect
                               differences in timing.

                               The rate at which this adjustment is made is usually done on a
                               case-by-case basis. Many analysts choose to use the market rate of
                               interest—for example, the rate payable on government bonds with a time
                               horizon comparable to the analysis. Some researchers have argued that
                               the discount rate should be set somewhat higher because, presumably,
                               economic growth will leave future generations better off. Others have
                               advocated using a “social” discount rate, which is generally lower than the
                               market rate, on the grounds that society’s interest in the welfare of future



                               Page 69                         GAO/HEHS-97-147 Consumer Product Safety Commission
                                 Appendix IV
                                 A Review of the Literature on
                                 COST-BENEFIT Analysis




                                 generations implies that the discount rate for all of society should be
                                 lower than the rates chosen by individuals. Experts on cost-benefit
                                 analysis generally encourage researchers to use multiple rates—both to
                                 assess the sensitivity of the results to the chosen discount rate and to
                                 facilitate comparison across studies. (We addressed issues of discounting
                                 in question 18 of table IV.1.)

Comparability With Other         Ideally, proponents of extensive cost-benefit analysis would like to be able
Analyses Could Influence         to order and prioritize regulatory interventions across agencies. A
Selection of Parameters          potential mechanism for this type of coordination is provided in Executive
                                 Order 12866, which requires agencies to submit to the Office of
                                 Management and Budget (OMB) cost-benefit analyses for regulations with
                                 an estimated impact on the economy of $100 million or more.38 OMB has
                                 suggested or prescribed certain rules (such as a presentation format
                                 and/or a specific discount rate) to facilitate comparability across agencies
                                 and to help promote quality analyses.39 While a common set of
                                 assumptions or parameters may facilitate comparability by a central
                                 government authority, so that priorities may be set across as well as within
                                 agencies, some of these assumptions may fit the circumstances of the
                                 given analyses less well. For example, a given discount rate may be more
                                 appropriate for one project (for which the benefits and costs are spread
                                 out over a long period) than another (with costs and benefits spread over a
                                 shorter period). Therefore, there may be a tension between making an
                                 analysis comparable to others and customizing it to fit a unique situation.
                                 (We addressed these issues in questions 10 and 18 of table IV.1.)

Potential Future Price Changes   Typically, cost-benefit analysis assumes that the intervention considered
Can Add Uncertainty to           has only a small or negligible effect on relative prices throughout the
Analysis                         economy. Generally, this assumption makes sense; however, for major
                                 changes (such as a big change in tax law) the assumption is clearly
                                 inappropriate. In these cases, it is much harder to put a value on the
                                 potential impact of the proposed change. For example, a large cut in the
                                 capital gains tax could affect a wide range of investment behavior, and
                                 may even affect how much some individuals choose to work and how
                                 much they are paid. A researcher trying to place a value on the potential
                                 consequences of a capital gains tax cut might introduce an error by
                                 valuing changes in productivity using the wage rates that prevailed before
                                 the cut, because, in reality, once the new policy has been implemented,
                                 wages rates might change. In such a situation, it is especially important for

                                 38
                                   CPSC, as an independent agency, is not required to follow these procedures.
                                 39
                                  The evaluation factors we examined parallel OMB’s suggestions; in some areas, we have
                                 operationalized these concepts to create more specific and objective evaluation questions.



                                 Page 70                                GAO/HEHS-97-147 Consumer Product Safety Commission
                                   Appendix IV
                                   A Review of the Literature on
                                   COST-BENEFIT Analysis




                                   the researcher to point out the potential mismeasurement, and (if feasible)
                                   try to model its effect. (We addressed these issues in question 13 of table
                                   IV.1.)

Researchers Recognize the          Every cost-benefit analysis will include some level of uncertainty or
Importance of Sensitivity          imprecision, or reflect a methodological choice that not everyone will
Analysis to Inform the Reader      necessarily agree with. Careful analysts will identify critical sources of
                                   uncertainty or controversy, and revise or test the analysis quantitatively or
                                   qualitatively to identify how or whether these areas affect the conclusions
                                   reached. If large variations in measurement or assumption do not alter the
                                   conclusions, then the researcher and the decisionmaker can have greater
                                   confidence in the original results. However, if the conclusions of the
                                   analysis can change depending on methodological choices or variable
                                   measurement, then the researcher may want to try to improve the
                                   measurement of the original variables. If this is not possible, then the
                                   analysis may be of more limited value—an implication the decisionmaker
                                   would need to know. (We addressed these issues in question 21 of table
                                   IV.1.)

Reliable Underlying Data           Despite the importance of sensitivity analysis, even the most careful and
Crucial to Reliable Cost-Benefit   elaborate sensitivity analysis often cannot sufficiently compensate for
Analysis                           poor underlying data. Obviously, if key variables are defined poorly or
                                   measured imprecisely, the quality of the cost-benefit analysis will suffer.
                                   Because many variables used in research are measured from surveys or
                                   samples, sampling error may be unavoidable. The researcher can use a
                                   sensitivity analysis to test the potential for sampling error to affect the
                                   conclusions; however, if the survey instrument itself is flawed, the results
                                   may be unreliable even beyond the degree indicated by sampling error.
                                   Similarly, researchers may face a difficult problem when key data (such as
                                   the cost of compliance) are provided for the analysis by an interested
                                   party (such as the industry). If the interested party provides the data, that
                                   party may have an incentive to provide biased, inaccurate, or misleading
                                   information. In such cases, it is important to try to verify the data to the
                                   extent possible; if not, the researcher needs to at least note the source of
                                   the underlying information and allow the reader to make an informed
                                   judgment about the reliability of the final analysis. (We addressed
                                   potential data issues in questions 19 and 20 of table IV.1.)


Measuring and Valuing              Frequently, the benefits of a particular proposal involve a reduction in risk
Risk Reduction Under               of injury or death. In these situations, the quality of the cost-benefit
Cost-Benefit Analysis              analysis will also depend substantially on the estimates of the proposal’s



                                   Page 71                         GAO/HEHS-97-147 Consumer Product Safety Commission
                                 Appendix IV
                                 A Review of the Literature on
                                 COST-BENEFIT Analysis




                                 impact on these risks. However, it is often difficult to accurately measure
                                 and value risk reduction. A number of issues arise, including the effect of
                                 the proposal on the level of risk (especially when individual behavior is a
                                 factor in determining the size of the risk) and measuring individuals’
                                 willingness to take risks in exchange for rewards.

Considerable Uncertainty Often   Measuring the effect of a proposal on the level of risk can be especially
Surrounds Risk Measurement       difficult. Generally, it is not realistic to assume that any regulatory
and Risk Assessment              intervention will reduce the risk level to zero—removing an environmental
                                 carcinogen, for example, will probably reduce the number of cancer
                                 deaths, but is unlikely to eliminate cancer entirely. Considerable
                                 uncertainty may be inevitable when extrapolating a dose-response model.
                                 Sometimes large doses of a potentially dangerous substance, given over a
                                 short period of time, are used to predict the results of long-term exposure.
                                 When the model moves across species—for example, predicting cases of
                                 cancer in humans based on experiments with laboratory animals—another
                                 source of uncertainty is introduced.

                                 Measurement problems in the underlying data can make it hard to predict
                                 future risk from epidemiological data as well. For example, when
                                 longitudinal data are used, there may arise a “survival bias” in that the
                                 analysis could be biased if it excludes individuals who die or otherwise
                                 move out of the data set. Similarly, when some cases go unreported, the
                                 data may understate the size and/or overstate the severity of the hazard.
                                 For rare hazards, limited variation in the epidemiological data can make
                                 measurement and prediction of risk more difficult. In addition, some data
                                 may be known to produce biased estimates (because of exclusions or
                                 potential double-counting, for example). Data problems such as these (and
                                 the statistical or analytical methods used to deal with them) should be
                                 pointed out to the reader to increase the opportunity for informed
                                 judgment. (We addressed these issues in questions 3 and 19 of table IV.1.)

Risk-Risk Trade-Offs and         Sometimes actions taken to reduce one risk can have the unintended
Individuals’ Behavioral          effect of increasing that or another risk. For example, unforeseen
Responses to Policy Changes      consequences arose in the 1970s when CPSC issued regulations requiring
Can Affect the Level of Risk     children’s sleepwear to meet flammability standards. Manufacturers used
                                 a chemical called Tris to meet these standards, but later it was discovered
                                 that Tris posed a cancer risk. Changes in individual behavior can also
                                 create uncertainty in predicting the level of risk because valuations of
                                 costs and benefits of a proposed action are often based on historical
                                 behavior. For example, valuations of a proposed new highway may be
                                 based on the number of individuals currently traveling the route, their



                                 Page 72                         GAO/HEHS-97-147 Consumer Product Safety Commission
                              Appendix IV
                              A Review of the Literature on
                              COST-BENEFIT Analysis




                              commuting times, and other factors. However, once the new highway is
                              built, some people may make extra trips in the area (shopping in different
                              stores, for example) than they did before. Some people who had driven at
                              off-hours to avoid congestion on the old road may now travel at peak
                              hours on the new one. Similarly, changes in public policy may cause
                              individuals or firms to change the technology they use, the amount of time
                              they spend at leisure or at work, or the amount they invest in innovation.

                              A special type of behavioral change—often referred to as “offsetting
                              behavior”—occurs when individuals change the amount of precautions
                              they take in response to a change in policy. For example, having an air bag
                              in the car or being required to wear a motorcycle helmet might make some
                              drivers feel safer, so they exercise less caution on the road. Sometimes
                              this offsetting behavior can result in an increase in the risk the
                              intervention was designed to mitigate. For example, because
                              child-resistant medicine bottles can be difficult to open, a grandparent
                              may leave the cap off the bottle, creating an even greater risk than would
                              be posed with a non-child-resistent cap. If such changes in consumer
                              behavior are foreseeable, the analysis will be improved if the researcher
                              points out such possibilities, and it will be even more useful if reasonable
                              attempts can be made to measure the impact. (We addressed these factors
                              in question 14 of table IV.1.)

Willingness-To-Pay Measures   Measuring the benefit of risk reduction requires placing a value on
Viewed as Appropriate for     avoiding death and/or injury. Several approaches are available for this
Valuing Risk but Require      task, but two have been used in cost-benefit analysis: (1) the “human
Careful Estimation            capital” approach, in which death or injury is valued at the market value
                              of the lost production it causes plus the medical costs expended and
                              (2) the “willingness-to-pay” approach, which attempts to measure directly
                              individuals’ willingness to pay for reducing the risk of death or injury.40 In
                              early cost-benefit analysis, the human capital approach was used, largely
                              because lost wages and medical costs were relatively easy to measure.
                              However, this approach is not preferred today because of a number of
                              shortcomings. First, if this approach were taken literally to apply to
                              individuals, then persons who do not produce output in the
                              marketplace—such as the elderly or homemakers—are not valued—an
                              assumption that is clearly ethically and economically inappropriate. In
                              addition, the human capital approach is unable to take into account costs
                              to the individual of death or injury such as pain and suffering. Also, it is
                              difficult to apply an “average” value based on human capital calculations

                              40
                               This terminology has been borrowed from W.K. Viscusi in “The Valuation of Risks to Life and
                              Health,” in Benefits Assessment: The State of the Art, eds., Judith D. Bentkover, Vincent T. Covello,
                              and Jeryl Mumpower (Dordrecht, Holland: Reidel Publishing Company, 1986), pp. 193-210.



                              Page 73                                 GAO/HEHS-97-147 Consumer Product Safety Commission
Appendix IV
A Review of the Literature on
COST-BENEFIT Analysis




to a statistically anonymous member of a large group. Finally, we are all
more valuable than the sum of what we produce, and these amounts could
not be included in a human-capital-based formulation. Thus, human capital
measurements are generally viewed as a lower boundary on the value of
avoiding death and injury, and are less preferred than the more recent
willingness-to-pay measures.

In part because of dissatisfaction with human capital measures as used in
cost-benefit analysis, and facilitated by newly available large micro-level
data sets, economists measure the benefits of reducing death or injury by
calculating the consumer’s willingness to pay for small reductions in the
probability of injury or death. These calculations have been done several
ways. Some approaches attempt to glean willingness-to-pay measures
from observed behavior in the marketplace. For example, one approach
examines pay differentials for jobs with different risks. Another approach
looks at the payoff to consumers who purchase safety devices, such as
smoke detectors and air bags in automobiles. These measures have the
advantage of being based on actual observed behavior in the marketplace
rather than on an artificial experimental situation. However, the validity of
such measures is based on the questionable assumption that workers and
consumers are sufficiently knowledgeable about the risks they face and
potential of different occupations or safety devices to alleviate those risks.
For many purposes, practitioners of cost-benefit analysis can select an
appropriate value from the range of research already done in this field
without performing the actual analysis themselves. However, for other
analyses, especially those involving unique risk-taking situations, it may be
wiser to gather new data to construct an estimate that is based on
circumstances as close as possible to those being studied.

Another common method for valuing risks is known as contingent
valuation, in which such values are elicited by observing responses or
behavior on a survey or in a controlled experiment. For example,
researchers surveyed individual shoppers on how much of a premium they
would be willing to pay for pesticide-free grapefruit.41 These methods have
the advantage of being able to provide information on areas that cannot be
addressed with market data. However, this characteristic could also be a
weakness—the very artificiality of the situation could make the consumer
make a less deliberate choice or could limit the usefulness of applying this
measure to other situations. This method does entail some technical
requirements—for example, it may be useful to perform statistical tests on

41
 Jean C. Buzzby, Richard C. Ready, and Jerry R. Skees, “Contingent Valuation in Food Policy Analysis:
A Case Study of a Pesticide-Residue Risk Reduction,” Journal of Agricultural and Applied Economics,
27 (2) (Dec. 1995), pp. 613-25.



Page 74                               GAO/HEHS-97-147 Consumer Product Safety Commission
Appendix IV
A Review of the Literature on
COST-BENEFIT Analysis




the distributional assumptions when constructing contingent valuation
measures. (We addressed these issues in questions 15 through 17 of table
IV.1.)




Page 75                         GAO/HEHS-97-147 Consumer Product Safety Commission
Appendix V

Death, Injury, and Incident Data Sources
Used by CPSC

                        In order to target its resources and analyze the costs and benefits of
                        projects or potential projects, CPSC must obtain data on injuries and deaths
                        related to particular product hazards. CPSC relies on a patchwork of
                        independent data systems to address this need. However, not only does
                        each of these data sources have its own internal limitations, but together
                        CPSC’s data sources present an incomplete—and potentially
                        distorted—picture of consumer-product-related injuries and deaths. The
                        implications of this lack of data range from reducing CPSC’s ability to apply
                        regulatory project selection criteria to limiting the agency’s ability to
                        estimate the impact of its regulatory actions.



CPSC Relies Heavily
on Hospital
Emergency Room
Reports to Obtain
Estimates of Injuries

NEISS System            CPSC obtains most of its injury information from its National Electronic
                        Injury Surveillance System (NEISS), which gathers information from
                        hospital emergency room records. NEISS provides national estimates about
                        the number and severity of emergency-room-treated injuries associated
                        with, although not necessarily caused by, consumer products in the United
                        States. To accomplish this, a stratified probability sample of hospitals is
                        drawn that is representative of all hospitals with emergency departments
                        in the United States and its territories.42 CPSC constructs national estimates
                        of the number of injuries associated with individual consumer products on
                        the basis of reports from this sample of 101 hospitals.

                        NEISS data result from information abstracted from hospital emergency
                        room records by coders trained by CPSC staff. The data are coded and
                        entered, at each site, into a personal computer programmed for this
                        purpose. The software checks the data entries for consistency. The data
                        collections are linked nightly to a CPSC permanent central database. Data
                        about product-related injuries are available to CPSC staff within 72 hours
                        after the accident for most of the injuries reported. These daily inputs are
                        reviewed by CPSC staff for quality and for identifying possible emerging

                        42
                         The hospitals are sampled from a universe in which each hospital has at least six beds and provides
                        24-hour emergency service.



                        Page 76                                GAO/HEHS-97-147 Consumer Product Safety Commission
Appendix V
Death, Injury, and Incident Data Sources
Used by CPSC




hazards. The timeliness of the data also allows staff to observe seasonal or
episodic variations. For example, during a 30-day period surrounding the
Fourth of July in 1990, CPSC gathered extra data through NEISS for a special
study on injuries involving fireworks.

The unit of analysis for NEISS is the injured person. Other key
characteristics coded include the date of treatment, age and gender of
patient, injury diagnosis, body part affected, disposition of case, product
involved, and accident location. In addition, important details about the
injury and the injured person are provided in NEISS. For example, the
address and phone number of the injured person is included, permitting
follow-up investigations about the nature and cause of the injury. There
are about 900 product codes, ranging from abrasive cleaners to youth
chairs, which the coders use to specify the product involved.43 Consumer
products are coded to allow for a great deal of specificity in the estimates.
For example, a “hand saw” would be differentiated from a “portable
circular power saw,” and a “bicycle-mounted baby carrier” would be
specified differently from a “backpack baby carrier.” However, NEISS’
coding system that describes the diagnosis of the primary injury and the
body part injured is not overly specific, and because it is unique it cannot
be directly compared with similar data from other databases.

Despite the extent of valuable information provided by NEISS, this system
also has significant limitations. One important consideration relates to the
nature and size of the NEISS sample design. NEISS, throughout its history,
has been designed so that only national—not state, local, or
regional—estimates can be made. Thus, NEISS cannot detect interregional
or interstate differences, and may also be limited in its ability to identify
emerging product hazard patterns that are focused in specific states or
regions.

A limitation in the NEISS data for which CPSC has received criticism is the
lack of information that would assist in assessing causality—that is,
information that would establish (or provide a starting point to
establishing) whether the product in question caused the accident or
merely was involved in the accident. NEISS contains neither “N”
codes—which describe the nature of the injury—nor “E” codes—which
help explain how the injury happened. E codes briefly describe the
circumstances of the accident that produced the injury. For example, E
codes could help distinguish among falls that occurred on stairs, from a

43
 Customarily excluded from NEISS are injury data involving automobiles and motorcycles, trains,
boats, planes, firearms, food, illegal drugs, pesticides, cosmetics, medical devices, assaults and suicide
attempts, and occupational injuries.



Page 77                                 GAO/HEHS-97-147 Consumer Product Safety Commission
                           Appendix V
                           Death, Injury, and Incident Data Sources
                           Used by CPSC




                           ladder, or off a roof. NEISS’ short narrative section sometimes contains this
                           type of information, but NEISS does not generally provide much information
                           on the circumstances surrounding the product’s involvement.


Other Sources of Data on   CPSC  augments the NEISS injury reports with other sources of anecdotal
Injuries                   information. For example, CPSC maintains an Injury or Potential Injury
                           Incident (IPII) database of reports the agency receives about injuries or
                           incidents involving consumer products. These reports come from a variety
                           of sources, including news clips; consumer complaints, including calls to
                           CPSC’s hot line; and public reports of product liability suits. Although the
                           IPII can provide only anecdotal information, this database sometimes
                           contains more detailed information than can be found in NEISS. For
                           example, IPII records can contain specific information about the product
                           involved—such as the manufacturer and date of purchase—that are
                           generally not found in NEISS. The IPII also serves as a source for cases to
                           investigate. This is particularly important for hazards for which NEISS
                           provides relatively few cases.

                           CPSC has also purchased data on poisonings in the form of the American
                           Association of Poison Control Centers (AAPCC) database. This database is
                           composed of reports from approximately 65 poison control centers
                           throughout the country.44 Reports from the AAPCC database contain such
                           information as the number of phone calls received by participating centers
                           concerning possible ingestion of a product and the number of individuals
                           who reported experiencing symptoms related to the ingestion. But these
                           reports contain little other information—for example, they do not show
                           how much of the poison was ingested.

                           Although NEISS provides some information on fire-related injuries, CPSC
                           obtains additional data on fires from the National Fire Protection
                           Association (NFPA) and the U.S. Fire Administration. NFPA, a private
                           organization, conducts an annual survey of fire departments that is
                           designed to make statistically valid national estimates of the total number
                           of fires experienced nationally each year. However, the NFPA survey does
                           not collect much detailed information about the characteristics of
                           individual fires. To augment the NFPA information, CPSC relies on the more
                           detailed information provided in the National Fire Reporting System




                           44
                             The number of poison control centers reporting cases to the AAPCC database varied over the years,
                           ranging from 47 centers in 1984 to 67 in 1996.



                           Page 78                               GAO/HEHS-97-147 Consumer Product Safety Commission
                       Appendix V
                       Death, Injury, and Incident Data Sources
                       Used by CPSC




                       (NFIRS), which is compiled by the U.S. Fire Administration.45 NFIRS data can
                       provide information on the ages of the victim or victims, and the type of
                       dwelling (apartment versus single-family home, for example). However,
                       these data are available only with a long lag; a CPSC official we interviewed
                       in October 1996 told us that the most recent data he had available at that
                       time was from 1993.


                       CPSC obtains the majority of its information on fatalities from purchasing
CPSC Obtains Most of   death certificates from the states and culling information from them to
Its Fatality           determine which deaths were related to consumer products. (Like the
Information From       NEISS emergency room reports, the death certificates establish only
                       involvement—not causality—of a consumer product.) CPSC does not
Selected Death         purchase the complete annual set of death certificates from each state;
Certificates           instead, the agency buys death certificates according to selected E codes.46
                        CPSC purchases about 8,000 death certificates annually; of these,
                       approximately 50 percent will be related to a consumer product. Over the
                       years, in response to declining budgets, CPSC has reduced even further the
                       number of E codes for which it purchases certificates. Death certificates
                       include the date, place, cause of death, age, gender, race, and residence of
                       the deceased.

                       Although death certificates cannot indicate whether the consumer product
                       was “at fault” in a death, they do provide some information on the
                       underlying circumstances. For example, the E code would specify that a
                       person was killed in an accident caused by electric current. Of course, the
                       quality of the data depends on the care with which causation is
                       determined and reported. It is likely that this differs from locality to
                       locality as well as among individual doctors. However, there are some
                       objective indications that the quality of reporting causes of death has been
                       improving; for example, the proportion of cases that are categorized under




                       45
                         Although NFIRS is a large dataset, it does not capture data uniformly across states. CPSC generally
                       assumes that NFIRS data accurately depict the distribution of characteristics across fires, and then
                       applies the NFIRS proportions to the NFPA data to approximate these characteristics on a national
                       level.
                       46
                         According to agency officials, the major categories of E codes the agency does not purchase involve
                       falls (which infrequently involve consumer products) and fires (for which the agency relies on other
                       sources). This means that fall-related deaths associated with consumer products—such as falls from
                       playground equipment—will be undercounted. In 1995, CPSC conducted a pilot in which it purchased
                       a complete set of E codes from one state to determine whether it should change its mix of E codes.
                       CPSC decided that the set it was buying was still the best choice given the budget for death certificate
                       data.



                       Page 79                                 GAO/HEHS-97-147 Consumer Product Safety Commission
Appendix V
Death, Injury, and Incident Data Sources
Used by CPSC




ill-defined conditions has been falling.47 Although death certificates do not
constitute a sample of known probability or a complete count of fatalities,
and thus statistically reliable estimates cannot be made, the geographic
information contained in death certificates may help to identify state and
regional patterns as well as those that are national in scope.

However, death certificate data also have substantial limitations. First,
there is an extensive lag—usually 2 years—before data become available.
Therefore, death certificates are not very useful in timely identifying
emerging hazards. Furthermore, a number of factors contribute to
obtaining rather sketchy causal information. For example, details are quite
limited, which inhibit determining the degree to which a consumer
product was involved in the death, let alone whether the product was
defective or hazardous. Certificates are frequently completed without the
benefit of autopsy information to establish the precise cause of death,
either because an autopsy was not performed or because the certificate
was filled out before the autopsy took place. There is substantial variation
in coding practices and the level of detail available from state to state.48
The information available about the injured individual is very limited, and
the number of cases for most categories is very small; such data therefore
limit the kinds and amount of analyses CPSC can perform and the
conclusions it can draw.

CPSC augments its death certificate data with other sources where possible.
For example, CPSC has instituted a Medical Examiners’ and Coroners’ Alert
Project (MECAP), to provide more timely fatality information that lends
itself to follow-up investigations. CPSC has engaged some 100 coroners and
medical examiners from across the country to report potential
product-related hazards. This project produces about 2,000 reports
annually, and CPSC staff credit the coroners’ reports with alerting them to a
suffocation hazard concerning infant cushions, which eventually led CPSC
to recall existing products and ban future production. In addition to the
MECAP data, CPSC also records some reports of fatalities in the IPII, NEISS,
and NFIRS databases.




47
  See National Center for Health Statistics, Advance Report of Final Mortality Statistics for 1994, 45
(3) (Sept. 30, 1996), p. 74.
48
 For more information on this variation, see New England Journal of Medicine, Vol. 313 (1985), pp.
1285-86, and U.S. Department of Health and Human Services, Morbidity and Mortality Weekly Report,
Vol. 37 (1988), pp. 191-4.



Page 80                                 GAO/HEHS-97-147 Consumer Product Safety Commission
                        Appendix V
                        Death, Injury, and Incident Data Sources
                        Used by CPSC




                        In order to effectively target resources, identify hazard patterns, and
CPSC Investigates       determine the appropriate remedy for particular product hazards, CPSC
Selected Incidents to   needs detailed analyses of the causes of reported incidents. CPSC’s data
Obtain Detailed         sets generally do not provide such information. The NEISS data do not
                        include E codes (standardized, if brief and incomplete, descriptions of
Information About the   incident circumstances), nor do they include detailed information about
Causes of Deaths and    how the incident happened. Some information may be provided in the
                        short, free-text comment area of the NEISS report, but generally few such
Injuries                details are recorded. Death certificates do include E codes, but product
                        involvement is often difficult to ascertain.

                        As a result, CPSC staff perform follow-up investigations on selected cases to
                        develop additional information about each incident. Some of these
                        investigations are conducted entirely by telephone, while others are
                        conducted at the accident site. These investigations may include detailed
                        interviews with victims and/or witnesses, police or fire reports,
                        photographs of the product and/or the accident site, laboratory testing of
                        the product involved, or re-creations of the incidents. For example, in 1996
                        CPSC staff investigated an incident in which a baby’s leg was scratched as it
                        was caught between the slats of her crib. As part of the on-site
                        investigation, the CPSC investigator interviewed the child’s mother,
                        examined the child, examined and photographed the crib, and interviewed
                        staff at the store where the crib was purchased.

                        The CPSC staff we interviewed told us that investigations, particularly
                        on-site investigations, were an important source of information on
                        established projects. The additional detail these investigations gathered
                        helps determine causality and identify hazard patterns, leading analysts to
                        the appropriate remedies. For example, investigations revealed that very
                        few bicycle accidents were related to mechanical problems, and as a
                        result, CPSC staff decided not to recommend any changes to existing
                        bicycle standards. In addition, investigations may provide key evidence to
                        help identify and correct compliance problems. For example, the
                        investigator who reviewed the crib incident found that the crib in question
                        appeared to violate several mandatory safety standards.




                        Page 81                           GAO/HEHS-97-147 Consumer Product Safety Commission
                     Appendix V
                     Death, Injury, and Incident Data Sources
                     Used by CPSC




                     In order to produce a numerical risk assessment, CPSC must have some
CPSC Uses Modeling   information on the extent to which individuals are exposed to a particular
Techniques and       product hazard. Exposure information can take many different forms, and
Special Surveys to   the best measure of exposure will depend on the characteristics of the
                     particular product hazard. For example, one measure of exposure might
Estimate Exposure    be the number of products in use, while another measure might be the
Information          number of hours a person spends using the product, and another measure
                     might take into account the intensity with which a product is used. If a
                     product is for one-time use only and is usually used soon after it is sold
                     (such as fireworks), number of products sold might be a reasonable proxy
                     for exposure. However, when a product is more durable, and used for a
                     longer period, and often “handed down” to another user, such as a baby
                     high chair, it would probably be more reasonable to base estimates of
                     exposure on the number of products in use. In cases where the potential
                     hazard is especially ubiquitous (like air pollution, for example), population
                     measures (the number living near the source or the number of children
                     under 5, for example) may be reasonable. For some hazards, it may be
                     important to account for the intensity of use. For example, the probability
                     of developing cancer from exposure to a wood stove may depend on how
                     often the stove is used, how large a space it heats, and how many times the
                     stove door is opened to add wood. Similarly, bicycles could be used very
                     frequently and very intensely (every day in urban traffic) or infrequently
                     and not intensely (once a season on the bike path in the park); thus, a
                     good exposure measure would take such factors into account.

                     CPSC does not conduct a formal, numerical risk assessment for each
                     project it undertakes. Of the 115 CPSC projects we reviewed, only 24
                     included a numerical assessment of risk. CPSC most frequently relied on
                     estimates of products in use to provide the exposure information for its
                     risk assessments. In 65 percent of the cases where an epidemiological risk
                     assessment was performed, CPSC based its exposure measure on an
                     estimate of the number of products in use.49 In an additional 5 percent,
                     CPSC obtained information on the actual number of products in use. In
                     30 percent of cases, CPSC used a population-based measure, and in
                     10 percent of cases, CPSC used a sales measure.50 We did not evaluate
                     whether the exposure measure CPSC chose was appropriate for each case.

                     CPSC conducted special surveys on bicycles and on cigarette lighters and
                     matches, for instance, to develop exposure information. The survey

                     49
                      About half of these estimates of products in use were derived from modeling and about half from
                     special surveys.
                     50
                       Percentages do not total 100 because in two cases CPSC used more than one measure of exposure.



                     Page 82                               GAO/HEHS-97-147 Consumer Product Safety Commission
                    Appendix V
                    Death, Injury, and Incident Data Sources
                    Used by CPSC




                    questions were designed to obtain information on the number of products
                    in the home, the intensity of use, the characteristics of the households
                    using the product, and the usual patterns of use. For example, the bicycle
                    survey included questions on the number of hours spent biking, the ages,
                    education, and income of household members, whether riding was done
                    most often on streets, sidewalks, or bike paths, and whether riders used
                    helmets.

                    Where special surveys are not practical—because of time, resource, or
                    other limitations—CPSC sometimes uses mathematical modeling
                    techniques to estimate the number of products in use. These models can
                    take sales information and information on the life of the product to
                    estimate the number of units in use. For example, CPSC used such a model
                    to estimate how many of the portable heaters made before a 1991 revision
                    to a voluntary standard were still in use.


                    CPSC may also receive information about potential product hazards
CPSC Obtains Some   through industry reporting requirements. Although this information is used
Information Under   mostly for identifying and addressing compliance problems, it may also
Reporting           help identify new hazards. Companies are legally obligated to report to
                    CPSC information they receive that indicates a consumer product they
Requirements for    distribute is potentially hazardous. Under section 15 of the Consumer
Manufacturers and   Product Safety Act, manufacturers (including importers), distributors, and
                    retailers of consumer products must notify CPSC if they obtain information
Other Businesses    that a product (1) fails to comply with a consumer product safety
                    regulation or a voluntary consumer product safety standard, (2) contains a
                    defect that could create a substantial product hazard, or (3) creates an
                    unreasonable risk of serious injury or death.

                    In addition to these reporting requirements, manufacturers and importers
                    of a consumer product must report to CPSC if (1) a particular model of a
                    consumer product is the subject of at least three civil actions that have
                    been filed in federal or state court, (2) each suit alleges the involvement of
                    that model in death or grievous bodily injury, and (3) within a specified
                    2-year period at least three of the actions resulted in a final settlement
                    involving the manufacturer or importer or in a judgment for the plaintiff.
                    Manufacturers must file a report within 30 days after the settlement or
                    judgment in the third such civil action.

                    CPSC may receive requests for information that firms have reported to it
                    under section 15 requirements. The law limits CPSC’s disclosure of any



                    Page 83                           GAO/HEHS-97-147 Consumer Product Safety Commission
Appendix V
Death, Injury, and Incident Data Sources
Used by CPSC




information identifying a manufacturer and further limits CPSC’s release of
information that firms have provided under these requirements. Reports
on civil lawsuits may not be publicly disclosed by CPSC or subpoenaed or
otherwise obtained from CPSC through discovery in any civil action or
administrative procedure.




Page 84                           GAO/HEHS-97-147 Consumer Product Safety Commission
Appendix VI

Comments From the Chairman and
Commissioner Moore, Consumer Product
Safety Commission, and GAO’S Evaluation
Note: GAO comments
supplementing those in the
report text appear at the
end of this appendix.




                             Page 85   GAO/HEHS-97-147 Consumer Product Safety Commission
Appendix VI
Comments From the Chairman and
Commissioner Moore, Consumer Product
Safety Commission, and GAO’S Evaluation




Page 86                         GAO/HEHS-97-147 Consumer Product Safety Commission
Appendix VI
Comments From the Chairman and
Commissioner Moore, Consumer Product
Safety Commission, and GAO’S Evaluation




Page 87                         GAO/HEHS-97-147 Consumer Product Safety Commission
Appendix VI
Comments From the Chairman and
Commissioner Moore, Consumer Product
Safety Commission, and GAO’S Evaluation




Page 88                         GAO/HEHS-97-147 Consumer Product Safety Commission
                 Appendix VI
                 Comments From the Chairman and
                 Commissioner Moore, Consumer Product
                 Safety Commission, and GAO’S Evaluation




See comment 1.




See comment 2.




                 Page 89                         GAO/HEHS-97-147 Consumer Product Safety Commission
Appendix VI
Comments From the Chairman and
Commissioner Moore, Consumer Product
Safety Commission, and GAO’S Evaluation




Page 90                         GAO/HEHS-97-147 Consumer Product Safety Commission
                 Appendix VI
                 Comments From the Chairman and
                 Commissioner Moore, Consumer Product
                 Safety Commission, and GAO’S Evaluation




See comment 3.




See comment 4.




See comment 5.




                 Page 91                         GAO/HEHS-97-147 Consumer Product Safety Commission
                 Appendix VI
                 Comments From the Chairman and
                 Commissioner Moore, Consumer Product
                 Safety Commission, and GAO’S Evaluation




See comment 6.




See comment 7.




                 Page 92                         GAO/HEHS-97-147 Consumer Product Safety Commission
                 Appendix VI
                 Comments From the Chairman and
                 Commissioner Moore, Consumer Product
                 Safety Commission, and GAO’S Evaluation




See comment 8.




See comment 3.




                 Page 93                         GAO/HEHS-97-147 Consumer Product Safety Commission
Appendix VI
Comments From the Chairman and
Commissioner Moore, Consumer Product
Safety Commission, and GAO’S Evaluation




Page 94                         GAO/HEHS-97-147 Consumer Product Safety Commission
                 Appendix VI
                 Comments From the Chairman and
                 Commissioner Moore, Consumer Product
                 Safety Commission, and GAO’S Evaluation




See comment 9.




                 Page 95                         GAO/HEHS-97-147 Consumer Product Safety Commission
                  Appendix VI
                  Comments From the Chairman and
                  Commissioner Moore, Consumer Product
                  Safety Commission, and GAO’S Evaluation




See comment 10.




See comment 11.




                  Page 96                         GAO/HEHS-97-147 Consumer Product Safety Commission
Appendix VI
Comments From the Chairman and
Commissioner Moore, Consumer Product
Safety Commission, and GAO’S Evaluation




Page 97                         GAO/HEHS-97-147 Consumer Product Safety Commission
                  Appendix VI
                  Comments From the Chairman and
                  Commissioner Moore, Consumer Product
                  Safety Commission, and GAO’S Evaluation




See comment 12.




                  Page 98                         GAO/HEHS-97-147 Consumer Product Safety Commission
Appendix VI
Comments From the Chairman and
Commissioner Moore, Consumer Product
Safety Commission, and GAO’S Evaluation




Page 99                         GAO/HEHS-97-147 Consumer Product Safety Commission
Appendix VI
Comments From the Chairman and
Commissioner Moore, Consumer Product
Safety Commission, and GAO’S Evaluation




Page 100                        GAO/HEHS-97-147 Consumer Product Safety Commission
Appendix VI
Comments From the Chairman and
Commissioner Moore, Consumer Product
Safety Commission, and GAO’S Evaluation




Page 101                        GAO/HEHS-97-147 Consumer Product Safety Commission
               Appendix VI
               Comments From the Chairman and
               Commissioner Moore, Consumer Product
               Safety Commission, and GAO’S Evaluation




               The following are GAO’s comments on Chairman Brown and Commissioner
               Moore’s letter dated July 24, 1997.


               1. As we discuss in the report, CPSC relies on a number of different sources
GAO Comments   for injury incident data. (Descriptive information on all CPSC’s data sources
               is provided in app. V.) However, only one source—the NEISS system—is
               capable of providing statistically reliable, representative information, and
               the NEISS system covers only those injuries treated in hospital emergency
               rooms. Although CPSC obtains information from other sources, these data
               are anecdotal and thus their usefulness is very limited in estimating injury
               prevalence. As a result, we have recommended that CPSC consult with
               experts both inside and outside the agency to prioritize its additional data
               needs and to explore the feasibility of options for obtaining these data.
               Changes over the past decade in the health care market, including the
               growth of ambulatory care, changes in reimbursement procedures, and
               improved health services research have the potential to make such
               additional data collection more feasible and less costly than it was a
               decade ago when some of these assessments were last made.

               2. Figure 3 correctly states that CPSC does not obtain systematic
               surveillance data on injuries treated outside the emergency room. In the
               text of the report, we state that the number of injuries treated in each
               setting is unknown. We added a similar statement to the figure to
               emphasize this point.

               3. In several places in their comments, Chairman Brown and
               Commissioner Moore refer to CPSC’s investigations of selected incidents to
               obtain more information. We discussed these investigations in our report.
               We believe that these investigations provide valuable information on
               causation, characteristics of accident victims, and hazard patterns, and we
               agree that some information gleaned from investigations is not obtainable
               from surveillance data. However, according to CPSC staff, this information
               is available to the agency only after a project is well under way, not at the
               initial stage of project selection. As a result, CPSC has little information on
               these important factors to assist in project selection, and evaluation of
               these criteria at the project selection stage is thus unavoidably speculative.

               4. Descriptive information on this source of information was provided on
               in appendix V.




               Page 102                        GAO/HEHS-97-147 Consumer Product Safety Commission
Appendix VI
Comments From the Chairman and
Commissioner Moore, Consumer Product
Safety Commission, and GAO’S Evaluation




5. For reasons we discussed in the report, relying on aggregate data (data
not specific to a particular consumer product) to address the limitations of
CPSC’s surveillance data is problematic. In addition, in the briefing
packages we reviewed, the estimates of injury incidents usually referred
only to NEISS cases and were not extended with modeling techniques;
because these techniques were used so infrequently for this purpose, we
did not assess their application in this report.

6. In referring to the agency’s monitoring of other aggregated sources of
injury data, Chairman Brown and Commissioner Moore state that
“Tracking of this type is sufficient to assure the adequacy of our data, as
we use it.” We disagree. Examining trend information from other sources
(without a rigorous application to CPSC’s own specific needs) is not
sufficient to compensate for (or even measure the magnitude of) the
limitations of CPSC’s injury surveillance data.

7. We believe this comment may reflect a misunderstanding of our point.
We do not mean to suggest that CPSC incorrectly relies on NEISS to provide
information on chronic illnesses. Rather, we are pointing out that CPSC has
virtually no systematic data on chronic illnesses. As we stated in the
report, on page 20, we agree with Chairman Brown and Commissioner
Moore (and with the CPSC staff we interviewed) that such data are often
difficult to obtain. However, chronic illness is listed as a criterion for CPSC
project selection, and CPSC has little information to assist in applying this
criterion. Accordingly, we are pleased with the statement by Brown and
Moore that CPSC will consider additional surveillance methods to obtain
more information on chronic illnesses.

8. We acknowledge in the report that CPSC uses other sources to
supplement its death certificate data. However, our interviews with CPSC
staff and our review of agency documents confirmed that death
certificates are the most important source of death data for CPSC. In the
briefing packages we reviewed, 80 percent of the calculations for numbers
of deaths were based in whole or in large part on death certificate data,
and death certificates were the sole source of death information in
36 percent of all CPSC briefing packages (a far greater percentage than for
any other single source). Death data were usually reported in CPSC briefing
packages with a lag of 2 or more years, especially when death certificates
were the sole source of data.

9. We agree with Chairman Brown and Commissioner Moore that there is
no single mold or perfect set of criteria for evaluating a cost-benefit



Page 103                        GAO/HEHS-97-147 Consumer Product Safety Commission
Appendix VI
Comments From the Chairman and
Commissioner Moore, Consumer Product
Safety Commission, and GAO’S Evaluation




analysis, and we have revised the report to further emphasize this point.
We believe that Chairman Brown and Commissioner Moore may have
misunderstood the purpose of our evaluation questions. We did not “derive
a particular methodology” for cost-benefit analysis, nor do we mean to
suggest that CPSC should follow some “formula” for conducting analyses
that does not leave room for competent professional judgment. However,
to say that there is no perfect “formula” for cost-benefit analysis does not
imply that all methodological choices are equally consistent with rigorous
and comprehensive professional work. Although no litmus test exists for a
“good” analysis, the professional literature offers some basic, minimum
elements that are commonly used in evaluating cost-benefit analyses.
These elements, which are based on the principles of transparency and
completeness, are generally considered necessary, although not sufficient,
for a good analysis. Although flexibility may be sometimes necessary in
the assumptions or models underlying a cost-benefit analysis, the elements
we used—including full disclosure of data limitations, sensitivity analysis,
and incorporating all important costs and benefits—are appropriate to a
wide range of situations.

10. Chairman Brown and Commissioner Moore are correct in stating that
we evaluated 29 CPSC cost-benefit analyses that were completed between
January 1, 1990, and September 30, 1996. We identified these 29 analyses
as complete on the basis of CPSC’s statements—specifically, we considered
a cost-benefit analysis to be complete only if an explicit comparison was
made between aggregate costs and benefits. In addition, although the
analyses may have been prepared at different stages of the project, we
based our review on all available documentation on the project, and we
reported results only for the evaluation questions that applied to the
majority of cases and for which a clear determination could be made. We
did not report results separately for the eight regulatory analyses that were
required by law, because there were relatively few of these. However, we
found no substantial differences between these 8 and the remaining 21 in
terms of how they performed, compared with commonly used elements of
evaluation of cost-benefit analysis. Therefore, we are confident that our
results present an accurate assessment of CPSC’s cost-benefit analyses, and
we recommend that the agency implement changes to ensure that its
analyses are comprehensive and reported in sufficient detail.

11. We believe that, as now constructed, CPSC’s method for tracking
projects operates at too high a level of generality and provides too little
information to give a comprehensive, accurate picture of the agency’s
activities either at a given point or over a longer period. CPSC staff told us,



Page 104                        GAO/HEHS-97-147 Consumer Product Safety Commission
Appendix VI
Comments From the Chairman and
Commissioner Moore, Consumer Product
Safety Commission, and GAO’S Evaluation




and our review of agency documentation confirmed, that the Management
Information System (MIS) usually tracks most agency activities only at a
very general level. For example, CPSC’s 1996 year-end MIS report lists some
specific projects such as “upholstered furniture” and “range fires,” but
most projects are accounted for under either broad umbrella codes such
as “sports and recreation” or “children’s projects,” or under activity codes
such as “investigations,” “product safety assessment,” or “emerging
problems.” In addition, CPSC staff told us that reliable inferences on
resources spent cannot be drawn from MIS data because of limitations in
the computer system and because no consistent rule exists about how
staff time in different directorates is recorded to project codes. As a result,
CPSC staff were unable to generate a comprehensive list of projects or to
provide accurate information about resources allocated to those projects.
We recommend an improved tracking system that would provide enough
information to monitor the projects selected and resources spent for each
specific consumer product hazard. We believe that as CPSC develops its
planned accounting system, it should attempt to make it as compatible as
practicable with the recommended tracking system. Nevertheless, we
believe that whether or not it implements its planned accounting system,
CPSC can and should improve its ability to track projects.


12. Our interviews with present and former commissioners revealed a
pattern by which certain of CPSC’s regulatory criteria have historically been
given greater emphasis in CPSC’s project selection process. Our objective
was to describe the process as it was related to us; we have not taken a
position on whether this process is appropriate. We have added a
statement to our methodology section to emphasize this point.




Page 105                        GAO/HEHS-97-147 Consumer Product Safety Commission
Appendix VII

Comments From Commissioner Gall,
Consumer Product Safety Commission, and
GAO’S Evaluation
Note: GAO comments
supplementing those in the
report text appear at the
end of this appendix.




See comment 1.




                             Page 106   GAO/HEHS-97-147 Consumer Product Safety Commission
Appendix VII
Comments From Commissioner Gall,
Consumer Product Safety Commission, and
GAO’S Evaluation




Page 107                        GAO/HEHS-97-147 Consumer Product Safety Commission
                 Appendix VII
                 Comments From Commissioner Gall,
                 Consumer Product Safety Commission, and
                 GAO’S Evaluation




See comment 2.




See comment 3.




                 Page 108                        GAO/HEHS-97-147 Consumer Product Safety Commission
Appendix VII
Comments From Commissioner Gall,
Consumer Product Safety Commission, and
GAO’S Evaluation




Page 109                        GAO/HEHS-97-147 Consumer Product Safety Commission
               Appendix VII
               Comments From Commissioner Gall,
               Consumer Product Safety Commission, and
               GAO’S Evaluation




               The following are GAO’s comments on Commissioner Gall’s letter dated
               July 23, 1997.


               1. Our interviews with present and former commissioners revealed a
GAO Comments   pattern by which certain of CPSC’s regulatory criteria have historically been
               given greater emphasis in CPSC’s project selection process. Our objective
               was to describe the process as it was related to us; we have not taken a
               position on whether this process is appropriate. We have added a
               statement to our methodology section to emphasize this point.

               2. Commissioner Gall states that we did not analyze the relative
               importance of the deficiencies we found in CPSC’s data and methodology.
               However, as we stated in the report, available information does not permit
               us to determine the impact of better-quality data on the decisions CPSC
               made. The limitations we found in CPSC’s data have a variety of potentially
               conflicting impacts, precluding us from determining exactly how the
               results of the analysis might change if improved data were available. For
               example, because CPSC’s injury estimates are often confined to injuries
               treated in hospital emergency rooms, CPSC’s estimates will generally
               understate the actual number of injuries associated with a consumer
               product. However, CPSC’s systematic injury and death data can generally
               tell only whether a product was involved in an accident—not whether the
               product caused or contributed to the accident. As a result, this can make
               the risks assessed by CPSC appear larger than they might actually be.
               Similarly, we cannot determine how improved exposure data would
               change the relative importance of the risks assessed by CPSC. We agree
               with Commissioner Gall that not all projects will merit the same level of
               data or analysis. However, our review of CPSC raises questions about the
               agency’s ability to obtain and analyze data necessary to support rigorous
               analysis of important agency projects.

               3. We agree with Commissioner Gall that resource considerations should
               enter into CPSC’s decisions to undertake new data collection. For this
               reason, we recommended an overall feasibility study for CPSC to prioritize
               among its data needs and investigate new options for obtaining additional
               information.




               Page 110                        GAO/HEHS-97-147 Consumer Product Safety Commission
Appendix VIII

GAO Contacts and Staff Acknowledgments


                  Lawrence J. Horinko, Assistant Director, (202) 512-7001
GAO Contacts      Sarah L. Glavin, Project Manager, (202) 512-7180


                  In addition to those named above, the following individuals made
Staff             important contributions to this report: Sheila A. Nicholson, Analyst,
Acknowledgments   gathered and analyzed data on CPSC’s information release procedures;
                  Nancy K. Kintner-Meyer, Senior Evaluator, compiled and analyzed
                  information on CPSC projects; George Bogart, Senior Attorney, provided
                  legal assistance; Harold Wallach, Senior Analyst, assisted in the analysis of
                  CPSC’s data systems; and Charles Jeszeck served as Assistant Director for
                  the project in its early stages.




                  Page 111                     GAO/HEHS-97-147 Consumer Product Safety Commission
Bibliography


               Abkowitz, M., and P. Der-Ming Cheng. “Developing a Risk/Cost
               Framework for Routing Truck Movements of Hazardous Materials.”
               Accident Analysis and Prevention, 20 (1988), pp. 39-51.

               Abraham, John. “Negotiation and Accommodation in Expert Medical Risk
               Assessment and Regulation: An Institutional Analysis of the Benoxaprofen
               Case.” Policy Sciences, 27 (1994), pp. 53-76.

               Adams, John. “The Emperor’s Old Clothes: The Curious Comeback of
               Cost-Benefit Analysis.” Environmental Values, 2 (1993), pp. 247-60.

               Adler, Robert S. “From ’Model Agency’ to Basket Case—Can the
               Consumer Product Safety Commission Be Redeemed?” Administrative
               Law Review, 42 (Winter 1989), pp. 61-129.

               Alberini, Anna. “Testing Willingness-to-Pay Models of Discrete Choice
               Contingent Valuation Survey Data.” Land Economics, 71 (1) (Feb. 1995),
               pp. 83-95.

               Allen, Julius. Cost Benefit Analysis of Federal Regulation: A Review and
               Analysis of Developments, 1978-1984. Report No. 84-74 E. Washington,
               D.C.: Congressional Research Service, May 15, 1984.

               Allison, Paul D. Event History Analysis: Regression for Longitudinal Event
               Data, Sage University Paper Series: Quantitative Applications in the Social
               Sciences, series no. 07-046. Newbury Park, Calif.: Sage Publications, 1984.

               American Academy of Pediatrics. “Hospital Discharge Data on Injury: The
               Need for E Codes.” Policy Statement in AAP News, Mar. 1992.

               Arrow, Kenneth J., and others. Benefit-Cost Analysis in Environmental,
               Health, and Safety Regulation: A Statement of Principles. American
               Enterprise Institute, The Annapolis Center, and Resources for the Future.
               Washington, D.C.: AEI Press, 1996.

               Asch, Peter. Consumer Safety Regulation: Putting a Price on Life and
               Limb. New York: Oxford University Press, 1988.

               Ault, Eric B. “CPSC’s Voluntary Standards: An Assessment and a Paradox.”
               The Frontier of Research in the Consumer Interest. Eds. E. Scott Maynes
               and ACCI Research Committee. Columbia, Mo.: American Council on
               Consumer Interests, 1986, pp. 77-81.



               Page 112                     GAO/HEHS-97-147 Consumer Product Safety Commission
Bibliography




Bailey, Martin J. Reducing Risks to Life: Measurement of the Benefits.
Washington, D.C.: American Enterprise Institute, 1980.

Baker, S.P., and others. The Injury Fact Book, 2nd ed. New York: Oxford
University Press, 1992.

Battiato, Salvatore Enrico. “Cost-Benefit Analysis and the Theory of
Resource Allocation.” Alan Williams and Emilio Giardina. Efficiency in the
Public Sector. Aldershot, Hants, England: Edward Elgar Publishing, 1993,
pp. 26-42.

Berlage, L., and R. Renard. “The Discount Rate in Cost-Benefit Analysis
and the Choice of a Numeraire.” Oxford Economic Papers, 37 (1985), pp.
691-9.

Berlau, John. “Play (Regulated) Ball
“ Reason, Dec. 1996, pp. 71-3.

Bhat, M., and S. Li. “Consumer-Product-Related Tooth Injuries Treated in
Hospital Emergency Rooms: United States, 1979-1987.” Community
Dentistry and Oral Epidemiology, 18 (3) (June 1990), pp. 133-8.

Bird, Peter J.W.N. “A Note on Relative Price Changes in Cost-Benefit
Analysis.” Public Finance, 35 (2) (1980), pp. 318-23.

Blackorby, Charles, and David Donaldson. “A Review Article: The Case
Against the Use of the Sum of Compensating Variations in Cost-Benefit
Analysis.” Canadian Journal of Economics, 23 (3) (Aug. 1990), pp. 471-94.

_____. “Welfare Ratios and Distributionally Sensitive Cost-Benefit
Analysis.” Journal of Public Economics, 34 (1987), pp. 265-90.

Boardman, Anthony, Aidan Vining, and W.G. Waters II. “Costs and Benefits
Through Bureaucratic Lenses: Example of a Highway Project.” Journal of
Policy Analysis and Management, 12 (3) (1993), pp. 532-55.

Bordley, Robert F. “Making Social Trade-offs Among Lives, Disabilities,
and Costs.” Journal of Risk and Uncertainty, 9 (1994), pp. 135-49.

Boustead, Thomas. The Law and Economics of Administrative Law: A
Statistical Analysis of the Consumer Product Safety Commission’s Petition
Process. Ph.D. Dissertation, Fordham University, May 1995.



Page 113                    GAO/HEHS-97-147 Consumer Product Safety Commission
Bibliography




Boyle, M.H., and others. “Economic Evaluation of Neonatal Intensive Care
of Very Low Birthweight Infants.” New England Journal of Medicine, 308
(1983), pp. 1330-37.

Bradfield, R.E. “Social Cost-Benefit Analysis in Project Evaluation: A
Practical Framework for Decision Making.” Journal of Studies in
Economics and Econometrics, 13 (2) (1989), pp. 25-35.

Brent, Robert J. “The Cost-Benefit Analysis of Government Loans.” Public
Finance Quarterly, 19 (1) (Jan. 1991), pp. 43-66.

Breyer, Stephen. Breaking the Vicious Circle. Cambridge, Mass.: Harvard
University Press, 1993.

Burt, Catharine E. “Injury-Related Visits to Hospital Emergency
Departments: United States, 1992.” Advance Data, No. 261 (Feb. 1, 1995).

Buzby, Jean C., Richard C. Ready, and Jerry R. Skees. “Contingent
Valuation in Food Policy Analysis: A Case Study of a Pesticide-Residue
Risk Reduction.” Journal of Agricultural and Applied Economics, 27
(2) (Dec. 1995), pp. 613-25.

Campen, James T. Benefit, Cost, and Beyond: The Political Economy of
Benefit-Cost Analysis. Cambridge, Mass.: Ballinger Publishing Company,
1986.

Cesario, Frank J. “Benefit-Cost Analysis Under Pricing Constraints.”
Applied Economics, 13 (1981), pp. 215-24.

Chan, Yuk-Shee, and Anthony M. Marino. “Regulation of Product Safety
Characteristics Under Imperfect Observability.” Journal of Regulatory
Economics, 6 (1994), pp. 177-95.

Chirinko, Robert S., and Edward P. Harper, Jr. “Buckle Up or Slow Down?
New Estimates of Offsetting Behavior and Their Implications for
Automobile Safety Regulation.” Journal of Policy Analysis and
Management, 12 (2) (1993), pp. 270-96.

The Institute of Medicine and the National Research Council, Commission
on Life Sciences, Committee on Trauma Research. Injury in America: A
Continuing Public Health Problem. Washington, D.C.: National Academy
Press, 1985.



Page 114                     GAO/HEHS-97-147 Consumer Product Safety Commission
Bibliography




Cost-Benefit Analysis: Wonder Tool or Mirage? Report by the U.S. House
of Representatives Subcommittee on Oversight and Investigation,
Committee on Interstate and Foreign Commerce, Dec. 1980.

Cowen, Tyler. “The Scope and Limits of Preference Sovereignty.”
Economics and Philosophy, 9 (1993), pp. 253-69.

Cox, Louis Anthony (Jr.). “Theory of Regulatory Benefits Assessment:
Econometric and Expressed Preference Approaches.” Benefits
Assessment: The State of the Art. Eds. Judith D. Bentkover, Vincent T.
Covello, and Jeryl Mumpower. Dordrecht, Holland: D. Reidel Publishing
Company, 1986, pp. 85-160.

Crandall, Robert W. “The Use of Cost-Benefit Analysis in Product Safety
Regulation.” The Frontier of Research in the Consumer Interest. Eds. E.
Scott Maynes and ACCI Research Committee. Columbia, Mo.: American
Council on Consumer Interests, 1986, pp. 61-75.

Crews, Clyde Wayne. “Ten Thousand Commandments: Regulatory Trends
1981-92 and the Prospect for Reform.” Journal of Regulation and Social
Cost, Vol. 2 (Mar. 1993), pp. 105-50.

Cummings, Ronald G., Louis Anthony Cox, Jr., and A. Myrick Freeman III.
“General Methods for Risk Assessment.” Benefits Assessment: The State of
the Art. Eds. Judith D. Bentkover, Vincent T. Covello, and Jeryl
Mumpower. Dordrecht, Holland: D. Reidel Publishing Company, 1986, pp.
161-92.

Curtin, Leo, and Ronald Krystynak. “An Economic Framework for
Assessing Foodborne Disease Control Strategies With an Application to
Salmonella Control in Poultry.” The Economics of Food Safety. Ed. Julie
A. Caswell. London: Elsevie, 1991, pp. 131-51.

Dasguptha, A.K., and D.W. Pearce. Cost-Benefit Analysis: Theory and
Practice. London: Macmillan, 1972.

Davis, Miriam. Health Risk Assessment Research at the Department of
Energy: Background Information. Washington, D.C.: U.S. Office of
Technology Assessment, Sept. 15, 1992.

Davis, Yvette, and others. “An Evaluation of the National Electronic Injury
Surveillance System for Use in Monitoring Nonfatal Firearm Injuries and



Page 115                     GAO/HEHS-97-147 Consumer Product Safety Commission
Bibliography




Obtaining National Estimates.” Journal of Safety Research, 27
(2) (Summer 1996), pp. 83-91.

Dehar, Mary Anne, Sally Casswell, and Paul Duignan. “Formative and
Process Evaluation of Health Promotion and Disease Prevention
Programs.” Evaluation Review, 17 (2) (Apr. 1993), pp. 204-20.

Drummond, Michael F. “Cost Benefit Analysis in Health and Health Care:
Fine in Practice, but Does It Work in Theory?” Alan Williams and Emilio
Giardina. Efficiency in the Public Sector. Aldershot, Hants, England:
Edward Elgar Publishing, 1993, pp. 106-28.

Drummond, Michael F., Greg L. Stoddart, and George W. Torrance.
Methods for Economic Evaluation of Health Care Programmes. New York:
Oxford University Press, 1996.

Drummond, Michael F., and others. “Users’ Guide to the Medical
Literature: How to Use an Article on Economic Analysis of Clinical
Practice.” Journal of the American Medical Association, 277 (19) (May 21,
1997), pp. 1552-57.

Evans, J.S., N.S. Hawkins, and J.D. Graham. “Uncertainty Analysis and the
Value of Information: Monitoring for Radon in the Home.” Journal of Air
Pollution Control Association, 38 (1988), pp. 1380-85.

Evans, L., and M.C. Frick. “Helmet Effectiveness in Preventing Motorcycle
Driver and Passenger Fatalities.” Accident Analysis and Prevention, 20
(1988), pp. 447-58.

Fingar, Ann R., Richard S. Hopkins, and Marjorie Nelson. “Work-Related
Injuries in Athens County, 1982 to 1986: A Comparison of Emergency
Department and Workers’ Compensation Data.” Journal of Occupational
Medicine, 34 (8) (Aug. 1992), pp. 779-87. Finkel, Adam M. Comparing Risks
Thoughtfully. Washington, D.C.: U.S. Office of Technology Assessment,
Sept. 1994.

_____. “Who’s Exaggerating?” Discover, May 1996, pp. 45-54.

Fischhoff, Baruch. Ranking Risks. Washington, D.C.: U.S. Office of
Technology Assessment, Feb. 1994.




Page 116                    GAO/HEHS-97-147 Consumer Product Safety Commission
Bibliography




_____, and Louis Anthony Cox, Jr. “Conceptual Framework for Regulatory
Benefits Assessment.” Benefits Assessment: The State of the Art. Eds.
Judith D. Bentkover, Vincent T. Covello, and Jeryl Mumpower. Dordrecht,
Holland: D. Reidel Publishing Company, 1986, pp. 51-84.

Fise, Mary Ellen. “Consumer Product Safety Regulation,” Chapter 11 in
Regulation and Consumer Protection. Dame Publications, Inc., 1995, pp.
277-94.

_____. The CPSC: Guiding or Hiding From Product Safety? Washington,
D.C.: Consumer Federation of America, May 1987.

_____, and M. Kristen Rand. Pennies for Consumer Protection? A Report
on the CPSC’s 1991 Budget. Washington, D.C.: Consumer Federation of
America, May 1990.

Fisher, Ann, Maria Pavlova, and Vincent Covello, eds. Evaluation and
Effective Risk Communication Workshop Proceedings. Washington, D.C.:
Interagency Task Force on Environmental Cancer and Heart and Lung
Disease, Committee on Public Education and Communication, Jan. 1991.

Fisher, Ann, Lauraine G. Chestnut, and Daniel M. Violette. “The Value of
Reducing Risks of Death: A Note on New Evidence.” Journal of Policy
Analysis and Management, 8 (1) (1989), pp. 88-100.

Fuchs, Edward Paul, and James Anderson. “Institutionalizing Cost-Benefit
Analysis in Regulatory Agencies.” Research in Public Policy Analysis and
Management, 4 (1987), pp. 187-211.

Fung, V.A., J.C. Barrett, and J. Huff. “Reviews and Commentaries: The
Carcinogenesis Bioassay in Perspective: Application in Identifying Human
Cancer Hazards.” Environmental Health Perspectives, 103 (1995), pp. 7-8.

Gerner, Jennifer L. “Product Safety: A Review.” The Frontier of Research
in the Consumer Interest, eds. E. Scott Maynes and ACCI Research
Committee. Columbia, Mo.: American Council on Consumer Interests,
1986, pp. 37-59.

Gillroy, John Martin, and Maurice Wade, eds. The Moral Dimensions of
Public Policy Choice. Pittsburgh, Pa.: University of Pittsburgh Press, 1992.




Page 117                     GAO/HEHS-97-147 Consumer Product Safety Commission
Bibliography




Gold, Marthe R., and others. Cost-Effectiveness in Health and Medicine.
New York: Oxford University Press, 1996.

Goldenhar, Linda, and Paul A. Schulte. “Intervention Research in
Occupational Health and Safety.” Journal of Occupational Medicine, 36
(7), pp. 763-75.

Goldstein, B. “Risk Management Will Not Be Improved by Mandating
Numerical Uncertainty Analysis for Risk Assessment.” University of
Cincinnati Law Review, Vol. 63, pp. 1599-1610.

Goodstein, Eban. “Benefit-Cost Analysis at the EPA.” Journal of
Socio-Economics, 24 (2) (1995), pp. 375-89.

Graham, John D. “Making Sense of Risk: An Agenda for Congress.” Risks,
Costs and Lives Saved: Getting Better Results from Regulation. Ed Robert
W. Hahn. Washington, D.C.: AEI Press, 1996, pp. 183-207.

_____. “Technology, Behavior, and Safety: An Empirical Study of
Automobile Occupant-Protection Regulation.” Policy Sciences, 17 (1984),
pp. 141-51.

_____, and Younghee Lee. “Behavioral Response to Safety Regulation: The
Case of Motorcycle Helmet-Wearing Legislation.” Policy Sciences, 19
(1986), pp. 253-73.

Graves, Edmund J., and Brenda S. Gillum. “1994 Summary: National
Hospital Discharge Survey.” Advance Data, No. 278 (Oct. 3, 1996).

Gray, George M. The Challenge of Risk Characterization. Washington,
D.C.: U.S. Office of Technology Assessment, Nov. 1992.

Greenberg, David H. “Conceptual Issues in Cost/Benefit Analysis of
Welfare-to-Work Programs.” Contemporary Policy Issues, 10 (Oct. 1992),
pp. 51-64.

Gregory, Robin, Thomas C. Brown, and Jack L. Knetsch. “Valuing Risks to
the Environment.” The Annals of the American Academy of Political and
Social Science. Eds. Howard Kunreuther and Paul Slovic. Vol. 545
(May 1996), pp. 54-63.




Page 118                    GAO/HEHS-97-147 Consumer Product Safety Commission
Bibliography




Griffin, Ronald. “On the Meaning of Economic Efficiency in Policy
Analysis.” Land Economics, 71 (1) (Feb. 1995), pp. 1-15.

Guyer, Bernard, and Susan S. Gallagher. “An Approach to the
Epidemiology of Childhood Injuries.” Pediatric Clinics of North America,
32 (1) (Feb. 1985), pp. 5-15.

Haas, C.N., and others. “Risk Assessment of Viruses in Drinking Water.”
Risk Analysis, Vol. 13, pp. 545-52.

Hadden, Susan G. Read the Label: Reducing Risk by Providing
Information. Boulder, Co.: Westview Press, 1986.

Haddix, A.C., and others, eds. Prevention Effectiveness: A Guide to
Decision Analysis and Economic Evaluation. New York: Oxford University
Press, 1996.

Hahn, Robert W. “Improving Regulation: Steps Toward Reform.”
Statement before the U.S. Senate Subcommittee on Financial Management
and Accountability, Committee on Govermental Affairs, Sept. 25, 1996.

_____. “A Preliminary Estimate of Some Indirect Costs of Environmental
Regulation.” Unpublished paper, Feb. 1995.

_____. “Regulatory Reform: A Legislative Agenda.” Statement before the
U.S. Senate Committee on Governmental Affairs, Feb. 8, 1995.

_____. “Regulatory Reform: What Do the Government’s Numbers Tell Us?”
Risks, Costs and Lives Saved: Getting Better Results From Regulation. Ed.
Robert W. Hahn. Washington, D.C.: AEI Press, 1996, pp. 208-54.

_____. “Why We Need to Balance the Costs and Benefits of Regulation.”
Statement before the U.S. House of Representatives, Subcommittee on
Oversight and Investigations, Committee on Economic and Educational
Opportunities, and the Subcommittee on Paperwork and Regulation,
Committee on Small Business, Feb. 2, 1995.

Hall, Margaret Jean, and Maria F. Owings. “Hospitalizations for Injury and
Poisoning in the United States, 1991.” Advance Data, No. 252 (Oct. 7,
1994).




Page 119                    GAO/HEHS-97-147 Consumer Product Safety Commission
Bibliography




Hammitt, J.K., and J.A.K. Cave. Research Planning for Food Safety: A
Value of Information Approach. RAND R-3946-ASPE/NCTR. Santa Monica,
Calif.: Rand, 1991.

Hanke, Steve. “On the Feasibility of Benefit-Cost Analysis.” Public Policy,
29 (2) (Spring 1981), pp. 147-57.

Haskins, Jack B. “Evaluative Research on the Effects of Mass
Communication Safety Campaigns: A Methodological Critique.” Journal of
Safety Research, 2 (2) (June 1970) pp. 86-8.

Haupt, Barbara. “1982 Summary: National Hospital Discharge Survey.”
Advance Data, No. 95 (Dec. 27, 1983).

Havrilesky, Thomas. “The Persistent Misapplication of the Hedonic
Damages Concept to Wrongful Death and Personal Injury Litigation.”
Journal of Forensic Economics, 8 (1) (1995), pp. 49-54.

Heimann, Christopher M., and others. “Project: The Impact of Cost-Benefit
Analysis on Federal Administrative Law.” Administrative Law Review, 42
(Fall 1990), pp. 545-654.

Henderson, J. Vernon. “Peak Shifting and Cost-Benefit Miscalculations.”
Regional Science and Urban Economics, 22 (1992), pp. 103-121.

Hensler, Deborah R., and others. Compensation for Accidential Injuries in
the United States. Santa Monica, Calif.: RAND, The Institute for Civil
Justice, 1991.

Hildred, William, and Fred Beauvais. “An Instrumentalist Critique of
’Cost-Utility Analysis.’” Journal of Economic Issues, 29 (4) (Dec. 1995), pp.
1083-96.

Hoehn, John P., and Alan Randall. “A Satisfactory Benefit Cost Indicator
From Contingent Valuation.” Journal of Environmental Economics and
Management, 14 (1987), pp. 226-47.

_____. “Too Many Proposals Pass the Benefit-Cost Test.” American
Economic Review, 79 (3) (June 1989), pp. 544-51.

_____. “Too Many Proposals Pass the Benefit-Cost Test: Reply.” American
Economic Review, 81 (5) (Dec. 1991), pp. 1450-52.



Page 120                     GAO/HEHS-97-147 Consumer Product Safety Commission
Bibliography




Hoffer, George E., Stephen W. Pruitt, and Robert J. Reilly. “The Impact of
Product Recalls on the Wealth of Sellers: A Reexamination.” Journal of
Political Economy, 96 (3) (1988).

Hopkins, Thomas D. “The Costs of Federal Regulation.” Journal of
Regulation and Social Costs, Mar. 1992, pp. 5-31.

Horowitz, John K., and Richard T. Carson. “Discounting Statistical Lives.”
Journal of Risk and Uncertainty, 3 (1990), pp. 403-13.

Hubin, Donald C. “The Moral Justification of Benefit-Cost Analysis.”
Economics and Philosophy, 10 (1994), pp. 169-94.

Jarrell, Gregg A., and Sam Peltzman. “The Impact of Product Recalls on
the Wealth of Sellers.” George J. Stigler. Chicago Studies in Political
Economy. Chicago, Ill.: Chicago Press, 1988, pp. 612-34.

Johansson, Per-Olov. “Altruism in Cost-Benefit Analysis.” Environmental
and Resource Economics, 2 (1992), pp. 605-13.

Joksch, H. “Critique of Peltzman’s Study: The Effects of Automobile Safety
Regulation.” Accident Analysis and Prevention, 8 (1976), pp. 213-14.

Keeney, Ralph L. “The Role of Values in Risk Management.” The Annals of
the American Academy of Political and Social Science. Eds. Howard
Kunreuther and Paul Slovic. Vol. 545 (May 1996). Philadelphia: Sage
Periodicals Press, pp. 126-34.

Kemper, Peter, David A. Long, and Craig Thornton. “A Benefit-Cost
Analysis of the Supported Work Experiment.” The National Supported
Work Demonstration. Robinson G. Hollister, Peter Kemper, and Rebecca
A. Maynard, eds. Madison, Wis.: University of Wisconsin Press, 1984, pp.
239-85.

Kerton, Robert, and Richard Bodell. “Quality, Choice and the Economics
of Concealment: The Marketing of Lemons.” Journal of Consumer Affairs,
29 (1) (Summer 1995), pp. 1-28.

Kopp, Raymond J. “Why Existence Value SHOULD Be Used in
Cost-Benefit Analysis.” Journal of Policy Analysis and Management, 11
(1) (1992), pp. 123-30.




Page 121                     GAO/HEHS-97-147 Consumer Product Safety Commission
Bibliography




Langlois, Jean A., and others. Improving the E Coding of Hospitalizations
for Injury: Do Hospital Records Contain Adequate Documentation?”
American Journal of Public Health, 85 (9) (Sept. 1995), pp. 1261-65.

Laughery, Kenneth R., David R. Lovvoll, and Meredith L. McQuilkin.
“Allocation of Responsibility for Child Safety.” Proceedings of the Human
Factors and Ergonomics Society 40th Annual Meeting. Santa Monica,
Calif.: Human Factors and Ergonomics Society, 1996, pp. 810-13.

Lave, Lester B. “Benefit Cost Analysis: Do the Benefits Exceed the Costs?”
Risks, Costs and Lives Saved: Getting Better Results From Regulation. Ed.
Robert W. Hahn. Washington, D.C.: AEI Press, 1996, pp. 104-34.

Layard, Richard. Cost-Benefit Analysis. Middlesex, England: Penguin
Books, 1972.

Lescohier, Ilana, Susan S. Gallagher, and Bernard Guyer. “Not by
Accident.” Issues in Science and Technology, Summer 1990, pp. 35-42.

Lichtenberg, Erik. “Conservatism in Risk Assessment and Food Safety
Policy.” The Economics of Food Safety. Ed. Julie A. Caswell. London:
Elsevier, 1991, pp. 89-102.

Linneman, Peter. “The Effects of Consumer Safety Standards: The 1973
Mattress Flammability Standard.” George J. Stigler. Chicago Studies in
Political Economy. Chicago, Ill.: University of Chicago Press, 1988, pp.
441-60.

Litan, Robert E., and William D. Nordhaus. Reforming Federal Regulation.
New Haven: Yale University Press, 1983.

Loomis, John B., and Pierre H. duVair. “Evaluating the Effect of
Alternative Risk Communication Devices on Willingness to Pay: Results
From a Dichotomous Choice Contingent Valuation Experiment.” Land
Economics, 69 (3) (Aug. 1993), pp. 287-98.

Lovvoll, David R., and others. “Responsibility for Product Safety in the
Work Environment.” Proceedings of the Human Factors and Ergonomics
Society 40th Annual Meeting. Santa Monica, Calif.: Human Factors and
Ergonomics Society, 1996, pp. 814-17.

MacLean, D. Values at Risk. Totowa, N.J.: Rowman and Allanheld, 1986.



Page 122                     GAO/HEHS-97-147 Consumer Product Safety Commission
Bibliography




Magat, Wesley A., and Michael J. Moore. “Consumer Product Safety
Regulation in the United States and the United Kingdom: the Case of
Bicycles.” Rand Journal of Economics, 27 (1) (Spring 1996), pp. 148-64.

Magat, Wesley A., and W. Kip Viscusi. Informational Approaches to
Regulation. Cambridge, Mass.: MIT Press, 1992.

Mayer, Michelle, and Felicia B. LeClere. “Injury Prevention Measures in
Households with Children in the United States, 1990.” Advance Data, No.
250 (May 31, 1994).

McCaig, Linda F. “National Hospital Ambulatory Medical Care Survey:
1992 Emergency Department Summary.” Advance Data, No. 245 (Mar. 2,
1994).

McGarity, Thomas O. “Media Quality, Technology, and Cost Benefit
Blancing Strategies for Health and Environmental Regulation.” Law and
Contemporary Problems, 46 (3) (1983), pp. 159-233.

_____, and Sidney A. Shapiro. Workers at Risk: The Failed Promise of the
Occupational Safety and Health Administration. Westport, Conn.: Praeger
Press, 1993.

McGinnis, J.M., and W.H. Foege. “Actual Causes of Death in the United
States.” Journal of the American Medical Association, Vol. 270, pp. 2207-12.

McKean, Kevin. “They Fly in the Face of Danger.” Discover, Apr. 1986, pp.
48-58.

Miller, James C. III, and Bruce Yandle, eds. Benefit-Cost Analyses of Social
Regulation: Case Studies From the Council on Wage and Price Stability.
Washington, D.C.: American Enterprise Institute, 1979.

Miller, Ted R., and Diane C. Lestina. “Patterns in U.S. Medical
Expenditures and Utilization for Injury, 1987.” American Journal of Public
Health, 86 (1) (Jan. 1996), pp. 89-93.

Mishan, E.J. Cost-Benefit Analysis. London: Unwin Hyman, 1988.

Moore, John L. Cost Benefit Analysis: Issues in Its Use in Regulation.
Report No. 95-760 ENR. Washington, D.C.: Congressional Research
Service, June 28, 1995.



Page 123                     GAO/HEHS-97-147 Consumer Product Safety Commission
Bibliography




Morgan, M. Granger, and Max Henrion. Uncertainty: A Guide to Dealing
With Uncertainty in Quantitative Risk and Policy Analysis. Cambridge,
Mass.: Cambridge University Press, 1990.

Nash, Chris. “Cost-benefit Analysis of Transport Projects.” Alan Williams
and Emilio Giardina. Efficiency in the Public Sector. Aldershot, Hants,
England: Edward Elgar Publishing, 1993, pp. 83-105.

National Committee for Injury Prevention and Control. Injury Prevention:
Meeting the Challenge. New York: Oxford University Press, 1989.

Needleman, Carolyn, and Martin L. Needleman. “Qualitative Methods for
Intervention Research.” American Journal of Industrial Medicine, 29
(1996), pp. 329-37.

Nelson, Cheryl R., and Barbara J. Stussman. “Alcohol- and Drug-Related
Visits to Hospital Emergency Departments: 1992 National Hospital
Ambulatory Medical Care Survey.” Advance Data, No. 251 (Aug. 10, 1994).

Newbery, David. “The Isolation Paradox and the Discount Rate for
Benefit-Cost Analysis: A Comment.” Quarterly Journal of Economics,
Feb. 1990, pp. 235-8.

Oi, Walter Y. “Safety at What Price?” American Economic Review, AEA
Papers and Proceedings, 85 (2) (May 1995), pp. 68-71.

Ozonoff, Victoria Vespe, Susan Tan-Torres, and Catherine W. Barber.
“Assessment of E-Coding Practices and Costs in Massachusetts Hospitals.”
Public Health Reports, 108 (5) (Sept. -Oct. 1993), pp. 633-6.

Page, Talbot. “A Framework for Unreasonable Risk in the Toxic
Substances Control Act (TSCA).” Management of Assessed Risk for
Carcinogens. Ed. William J. Nicholson. Annals of the New York Academy
of Sciences, New York: 1981.

Page, Talbot, and Paolo F. Ricci. “A Cost-Benefit Perspective for Risk
Assessment.” Principles of Health Risk Assessment. Ed. Paolo F. Ricci.
Englewood Cliffs, N.J.: Prentice-Hall, Inc., 1985, pp. 37-65.

Pease, William S. The Impact of Scientific Research on Federal Agency
Guidelines for Conducting Health Risk Assessments. Washington, D.C.:
U.S. Office of Technology Assessment, Dec. 18, 1992.



Page 124                    GAO/HEHS-97-147 Consumer Product Safety Commission
Bibliography




Peltzman, Sam. “The Effect of Automobile Safety Regulation.” George J.
Stigler. Chicago Studies in Political Economy. Chicago, Ill. : University of
Chicago Press, 1988, pp. 349-403 (originally published in Journal of
Political Economy, 83, 1975, pp. 677-725).

Petty, Ross D. “Bicycle Safety: A Case Study in Regulatory Review.”
Regulation, 17 (2) (1994), pp. 22-24.

_____. “Regulating Product Safety: The Informational Role of the U.S.
Federal Trade Commission.” Journal of Consumer Policy, 18 (1995), pp.
387-415.

Pollack, E.S., and D.G. Keimig, eds. Counting Injuries and Illnesses in the
Workplace: Proposals for a Better System. New York: National Academy
Press, 1987.

Price, Colin. “Does Social Cost-Benefit Analysis Measure Overall Utility
Change?” Economics Letters, 26 (1988), pp. 357-61.

Proulx, Guylene, and Joelle Pineau. “The Impact of Age on Occupants’
Behavior During a Residential Fire.” Unpublished paper, 1996.

Quiggin, John. “Existence Value and Cost-Benefit Analysis: A Third View.”
Journal of Policy Analysis and Management, 12 (1) (1993), pp. 195-9.

_____. “Too Many Proposals Pass the Benefit-Cost Test: Comment.”
American Economic Review, 81 (5) (Dec. 1991), pp. 1446-9.

Ramirez, Jorge, and others. “Ex-Post Analysis of Flood Control:
Benefit-Cost Analysis and the Value of Information.” Water Resources
Research, 24 (8) (Aug. 1988), pp. 1397-1405.

Rauchschwalbe, Renae, and N. Clay Mann. “Pediatric Window-Cord
Strangulations in the United States, 1981-1995.” Journal of the American
Medical Association, 277 (21) (June 4, 1997), pp. 1696-98.

Rice, Dorothy P., and Wendy Max. “Annotation: The High Cost of Injuries
in the United States.” American Journal of Public Health, 86
(1) (Jan. 1996), pp. 14-15.




Page 125                      GAO/HEHS-97-147 Consumer Product Safety Commission
Bibliography




Risa, Alf Erling. “Public Regulation of Private Accident Risk: The Moral
Hazard of Technological Improvements.” Journal of Regulatory
Economics, 4 (1992), pp. 335-46.

Rivara, Frederick P., and others. “Cost Estimates for Statewide Reporting
of Injuries by E Coding Hospital Discharge Abstract Data Base Systems.”
Public Health Reports, 105 (6) (Nov.-Dec. 1990), pp. 635-38.

Robins, Marcia P. “An Examination of the Effect of CPSC Prescription Drug
Child-Resistant Packaging Requirements on Child Fatalities.” Unpublished
paper, Nov. 1989. Rodgers, Gregory B. “The Effectiveness of Helmets in
Reducing All-Terrain Vehicle Injuries and Deaths.” Accident Analysis and
Prevention, 22 (1) (1990), pp. 47-58.

_____. “Evaluating Product-Related Hazards at the Consumer Product
Safety Commission: The Case of All-Terrain Vehicles.” Evaluation Review,
Feb. 1990, pp. 3-21.

_____, with P. Rubin. “Cost-Benefit Analysis of All-Terrain Vehicles at the
CPSC.” Risk Analysis, 9 (1989), pp. 63-9.


Rosenthal, Donald H., and Robert H. Nelson. “Why Existence Value Should
NOT Be Used in Cost-Benefit Analysis.” Journal of Policy Analysis and
Management, 11 (1) (1992), pp. 116-22.

Rubin, Paul H., R. Dennis Murphy, and Gregg Jarrell. “Risky Products,
Risky Stocks.” Regulation, No. 1 (1988), pp. 35-9.

Rubinfeld, Daniel L., and Gregory B. Rodgers. “Evaluating the Injury Risk
Associated With All-Terrain Vehicles: An Application of Bayes’ Rule.”
Journal of Risk and Uncertainty, 5 (1992), pp. 145-58.

Schappert, Susan M. “National Ambulatory Medical Care Survey: 1992
Summary.” Advance Data, No. 253 (Aug. 18, 1994).

Schierow, Linda-Jo. “Risk Analysis and Cost-Benefit Analysis of
Environmental Regulations. 94-961 ENR. Washington, D.C. : U.S.
Congressional Research Service, Dec. 2, 1994.

Schroeder, Elinor P., and Sidney A. Shapiro. “Responses to Occupational
Disease: The Role of Markets, Regulation, and Information.” Georgetown
Law Journal, 72 (1983), pp. 1231-1306.



Page 126                     GAO/HEHS-97-147 Consumer Product Safety Commission
Bibliography




Selbst, S., M.D. Baker, and M. Shames. “Bunk Bed Injuries.” American
Journal of Diseases in Children, 144 (6) (June 1990), pp. 721-23.

Sen, Amartya. “Approaches to the Choice of Discount Rates for Social
Benefit-Cost Analysis.” Chapter 8 in Resources, Values, and Development.
Cambridge, Mass.: Harvard University Press, 1984, pp. 172-203.

Senturia, Yvonne D., and others. “Exposure Corrected Risk Estimates for
Childhood Product Related Injuries.” American Journal of Public Health,
1993, pp. 473-77.

Shew, Russel, and Rachel Dardis. “An Economic Analysis of Child
Restraints.” Journal of Consumer Policy, 18 (1995), pp. 417-31.

Smith, V. Kerry. “A Conceptual Overview of the Foundations of
Benefit-Cost Analysis.” Benefits Assessment: The State of the Art. Eds.
Judith D. Bentkover, Vincent T. Covello, and Jeryl Mumpower. Dordrecht,
Holland: D. Reidel Publishing Company, 1986, pp. 13-34.

_____. “Uncertainty, Benefit-Cost Analysis, and the Treatment of Option
Value.” Journal of Environmental Economics and Management, 14 (1987),
pp. 283-92.

_____, and others. “Can Public Information Programs Affect Risk
Perceptions?” Journal of Policy Analysis and Management, 9 (1) (1990),
pp. 41-59.

Spitzer, Hugh. Case Studies to Demonstrate the Impact of Research on
Assessment of Carcinogenic Risk. Washington, D.C.: U.S. Office of
Technology Assessment, Dec. 1, 1992.

Stason, W.B., and M.C. Weinstein. “Allocation of Resources to Manage
Hypertension.” New England Journal of Medicine, 296 (1977), pp. 732-9.

Stayner, Leslie, and others. “Approaches for Assessing the Efficacy of
Occupational Health and Safety Standards.” American Journal of Industrial
Medicine, 29 (1996), pp. 353-7.

_____. “National Hospital Ambulatory Medical Care Survey: 1994
Emergency Department Summary.” Advance Data, No. 275 (May 17, 1996).




Page 127                    GAO/HEHS-97-147 Consumer Product Safety Commission
Bibliography




Stussman, Barbara J. “National Hospital Ambulatory Medical Care Survey:
1993 Emergency Department Summary.” Advance Data, No. 271 (Jan. 25,
1996).

Tengs, Tammy O., and John D. Graham. “The Opportunity Costs of
Haphazard Social Investments in Life Saving.” Risks, Costs and Lives
Saved: Getting Better Results from Regulation. Ed. Robert W. Hahn.
Washington, D.C.: AEI Press, 1996, pp. 167-82.

Thacker, Stephen B., and others. “Assessing Prevention Effectiveness
Using Data to Drive Program Decisions.” Public Health Reports, 109
(2) (Mar.-Apr. 1994), pp. 187-94.

Thomas, L.G. “Revealed Bureaucratic Preference: Priorities of the
Consumer Product Safety Commission.” Rand Journal of Economics, 19
(1) (Spring 1988), pp. 102-13.

Thompson, M.S., J.L. Read, and M. Laing. “Feasibility of Willingness-to-Pay
Measuresment in Chronic Arthritis.” Medical Decision Making, 4
(2) (1984), pp. 195-215.

Tolchin, Susan J., and Martin Tolchin. “Particles of Truth: Cotton Dust and
Cost-Benefit Analysis.” Dismantling America: The Rush to Deregulate,
Boston, Mass.: Houghton Mifflin Company, 1983, pp. 111-41.

Tolley, G., K. Kenkel, and R. Rabian, eds. Valuing Health for Policy: An
Economic Approach. Chicago, Ill.: University of Chicago Press, 1994.

Trinkoff, Alison, and Peggy L. Parks. “Prevention Strategies for Infant
Walker-Related Injuries.” Public Health Reports, 108 (6) (Nov.-Dec. 1993),
pp. 784-8.

Trumbull, William. “Reply to Whittington and MacRae.” Journal of Policy
Analysis and Management, 9 (4) (1990), pp. 548-50.

_____. “Who Has Standing in Cost-Benefit Analysis?” Journal of Policy
Analysis and Management, 9 (2) (1990), pp. 201-18.

U.S. Commission on Risk Assessment and Risk Management. Risk
Assessment and Risk Management in Regulatory Decision-Making.
Washington, D.C.: 1997.




Page 128                     GAO/HEHS-97-147 Consumer Product Safety Commission
Bibliography




U.S. Department of Health and Human Services, Centers for Disease
Control and Prevention. “A Framework for Assessing the Effectiveness of
Disease and Injury Prevention.” Morbidity and Mortality Weekly Report,
Recommendations and Reports, Vol. 41, No. RR-3 (Mar. 27, 1992), pp. 1-13.

_____. “Guidelines for Death Scene Investigation of Sudden Unexplained
Infant Deaths.” Morbidity and Mortality Weekly Report, Recommendations
and Reports, Vol. 45, No. RR-10 (June 21, 1996), pp. 1-31.

_____. “Injuries and Deaths Associated With the Use of
Snowmobiles—Maine, 1991-1996.” Morbidity and Mortality Weekly Report,
Vol. 46, No. 1 (Jan. 10, 1997), pp. 1-4.

_____. Inventory of Federal Data Systems in the United States for Injury
Surveillance, Research and Prevention Activities. Washington, D.C.:
May 1996.

U.S. Department of Health and Human Services, Centers for Disease
Control and Prevention, National Center for Health Statistics. “Advance
Report of Final Mortality Statistics, 1994.” Monthly Vital Statistics Report,
45 (3) (Supplement, Sept. 30, 1996).

_____. “Hospitalizations for Injury and Poisoning in the United States,
1991.” Advance Data, No. 252 (Oct. 7, 1994).

U.S. Department of Health and Human Services, Public Health Service.
Healthy People 2000: Midcourse Review and 1995 Revisions. Washington,
D.C.: 1995.

_____. Healthy People 2000: National Health Promotion and Disease
Objectives. Washington, D.C.: 1991.

U.S. General Accounting Office. Peer Review: EPA’s Implementation
Remains Uneven. Sept. 24, 1996, GAO/RCED-96-236.

U.S. Office of Management and Budget. Guidelines and Discount Rates for
Benefit-Cost Analysis of Federal Programs. Circular A-94. Washington,
D.C.: Oct. 29, 1992.

_____. Regulatory Program of the United States Government. Bulletin No.
91-04. Washington, D.C.: Nov. 26, 1990.




Page 129                      GAO/HEHS-97-147 Consumer Product Safety Commission
Bibliography




U.S. Office of Technology Assessment. Researching Health Risks.
OTA-BBS-570. Washington, D.C.: Jan. 1993.

_____. Risks to Students in School. OTA-ENV-633. Washington, D.C.: Sept.
1995.

U.S. Physician Payment Review Commission, A Comparison of Alternative
Approaches to Risk Measurement. Washington, D.C.: Dec. 1994.

van Ravenswaay, Eileen O., and John P. Hoehn. “The Impact of Health
Risk Information on Food Demand: A Case Study of Alar and Apples.” The
Economics of Food Safety. Ed. Julie A. Caswell. London: Elsevier, 1991,
pp. 155-174.

Veasie, M.A., and others. “Epidemiologic Research on the Etiology of
Injuries at Work.” Annual Review of Public Health, 15 (1994), pp. 203-21.

Viscusi, W. Kip. “The Dangers of Unbounded Commitments to Regulate
Risk.” Risks, Costs and Lives Saved: Getting Better Results From
Regulation. Ed. Robert W. Hahn. Washington, D.C.: AEI Press, 1996, pp.
135-66.

_____. Regulating Consumer Product Safety. Washington, D.C.: American
Enterprise Institute, 1984.

_____. Tax Policy and the Economy. Ed. James M. Porterba, Vol. 9, 1995, p.
180.

_____. “The Valuation of Risks to Life and Health: Guidelines for Policy
Analysis.” Benefits Assessment: The State of the Art. Eds. Judith D.
Bentkover, Vincent T. Covello, and Jeryl Mumpower. Dordrecht, Holland:
D. Reidel Publishing Company, 1986, pp. 193-210.

_____. “The Value of Risks to Life and Health. Journal of Economic
Literature, 31 (Dec. 1993), pp. 1912-46.

_____, and Richard J. Zeckhauser. “Hazard Communication: Warnings and
Risk.” The Annals of the American Academy of Political and Social
Science. Eds. Howard Kunreuther and Paul Slovic. Vol 545 (May 1996).
Philadelphia: Sage Periodicals Press, pp. 106-15.




Page 130                     GAO/HEHS-97-147 Consumer Product Safety Commission
Bibliography




_____, and Wesley A. Magat, eds. Learning About Risk: Consumer and
Worker Responses to Hazard Information. Cambridge, Mass.: Harvard
University Press, 1987.

Waller, Julian A. “Reflections on a Half Century of Injury Control.”
American Journal of Public Health, 84 (4) (Apr. 1994), pp. 664-70.

_____, Joan M. Skelly, and John H. Davis. “Treated Injuries in Northern
Vermont.” Accident Analysis and Prevention, 27 (6) (1995), pp. 819-28.

Warr, Peter G., and Brian D. Wright. “The Isolation Paradox and the
Discount Rate for Benefit-Cost Analysis.” Quarterly Journal of Economics,
96 (1) (Feb. 1981), pp. 129-45. Weinstein, M.C., and others.
“Recommendations of the Panel on Cost-Effectiveness in Health and
Medicine.” Journal of the American Medical Association, 276 (15) (1996),
pp. 1253-8.

Weisbrod, Burton A. “Benefit-Cost Analysis of a Controlled Experiment:
Treating the Mentally Ill.” Journal of Human Resources, 26 (4) (1981), pp.
523-48.

Weiss, K.B., P.J. Gergen, and T.A. Hodgson. “An Economic Evaluation of
Asthma in the United States.” New England Journal of Medicine, 326, pp.
862-6.

Wheelwright, Jeff. “The Air of Ostrava: Pollution and Risk Assessment in
the Czech Republic.” Discover, 17 (5) (May 1996), pp. 56-64.

Whittington, Dale, and Duncan MacRae, Jr. “Comment: Judgments About
Who Has Standing in Cost-Benefit Analysis.” Journal of Policy Analysis
and Management, 9 (4) (1990), pp. 536-47.

Wildasin, David E. “Indirect Distributional Effects in Benefit-Cost Analysis
of Small Projects.” The Economic Journal, 98 (Sept. 1988), pp. 801-7.

Williams, Alan. “Cost-Benefit Analysis: Applied Welfare Economics or
General Decision Aid?” Alan Williams and Emilio Giardina. Efficiency in
the Public Sector. Aldershot, Hants, England: Edward Elgar Publishing,
1993, pp. 65-82.




Page 131                     GAO/HEHS-97-147 Consumer Product Safety Commission
           Bibliography




           Wood, William W. “Cost-Benefit Analysis of Small Business Assistance: Do
           Entrepreneurs Really Need ’Assisting’?” Journal of Private Enterprise, 10
           (1) (Summer 1994), pp. 13-21.

           Zeckhauser, Richard J., and W. Kip Viscusi. “The Risk Management
           Dilemma.” The Annals of the American Academy of Political and Social
           Science. Eds. Howard Kunreuther and Paul Slovic. Vol. 545 (May 1996).
           Philadelphia: Sage Periodicals Press, pp. 144-55.

           Zerbe, Richard O. “Comment: Does Benefit Cost Analysis Stand Alone?
           Rights and Standing.” Journal of Policy Analysis and Management, 10
           (1) (1991), pp. 96-105.




(205333)   Page 132                    GAO/HEHS-97-147 Consumer Product Safety Commission
Ordering Information

The first copy of each GAO report and testimony is free.
Additional copies are $2 each. Orders should be sent to the
following address, accompanied by a check or money order
made out to the Superintendent of Documents, when
necessary. VISA and MasterCard credit cards are accepted, also.
Orders for 100 or more copies to be mailed to a single address
are discounted 25 percent.

Orders by mail:

U.S. General Accounting Office
P.O. Box 37050
Washington, DC 20013

or visit:

Room 1100
700 4th St. NW (corner of 4th and G Sts. NW)
U.S. General Accounting Office
Washington, DC

Orders may also be placed by calling (202) 512-6000
or by using fax number (202) 512-6061, or TDD (202) 512-2537.

Each day, GAO issues a list of newly available reports and
testimony. To receive facsimile copies of the daily list or any
list from the past 30 days, please call (202) 512-6000 using a
touchtone phone. A recorded menu will provide information on
how to obtain these lists.

For information on how to access GAO reports on the INTERNET,
send an e-mail message with "info" in the body to:

info@www.gao.gov

or visit GAO’s World Wide Web Home Page at:

http://www.gao.gov




PRINTED ON    RECYCLED PAPER
United States                       Bulk Rate
General Accounting Office      Postage & Fees Paid
Washington, D.C. 20548-0001           GAO
                                 Permit No. G100
Official Business
Penalty for Private Use $300

Address Correction Requested