oversight

Disinfectants: EPA Lacks Assurance They Work

Published by the Government Accountability Office on 1990-08-30.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

   EPA Lacks Assurance
   They Work



                                                            84
                                                  lllllllW
                                                       IM
                                                   142343
                                                         ll


RESTItIcTED --Not
                            qELEjSED
                      to be re ease outside the
General Accounting Office unless specifkally
approved by the Office of Congressional
Relations.
    Reeources, Community,   and
    Economic Development    Division

    B-236877
    August 30,199O

   The Honorable John Conyers, Jr., Chairman
   The Honorable Frank Horton, Ranking Minority Member
   Committee on Government Operations
   House of Representatives
   The Honorable Mike Synar, Chairman
   The Honorable William F. Clinger, Jr., Ranking Minority
     Member
   Environment, Energy and Natural ResourcesSubcommittee,
   Committee on Government Operations
   House of Representatives
   This report respondsto your request that we review the Environmental Protection Agency’s
   (EPA) activities to regulate the efficacy of disinfectants. Specifically, it describesthe nature
   of the scientific controversies surrounding EPA-recommended      methods and performance
   standards for testing the efficacy of disinfectants, efforts EPAhas made to obtain objective
   research to resolve these controversies, the adequacy of EPA’Sinternal controls to ensure the
   quality and integrity of registrant-submitted efficacy data, post-registration efforts EPAand
   the states have implemented to ensure that disinfectants on the market are effective, and the
   need and options for a laboratory to research and test the efficacy of disinfectants.
   Unless you publicly releaseits contents earlier, we will make no further distribution of this
   report until 30 days from the date of this letter. At that time, we will send copies of the
   report to appropriate congressionalcommittees; the Administrator, EPA;and other interested
   parties.
   This report was prepared under the direction of Richard L. Hembra, Director, Environmental
   Protection Issues,whom you may contact at (202) 276-6111if you or your staff have any
   questions. Other major contributors are listed in appendix II.




$exP@
   Assistant Comptroller General
Executive Summary


                   Disinfectants-about a $1 billion per year market-are used to kill
Purpose            germs on inanimate surfaces and objects in hospitals, schools,restau-
                   rants, and homes.Becauseusers cannot seewhether disinfectants kill
                   bacteria, fungi, and viruses, the use of ineffective disinfectants posesa
                   threat to public health and wastes consumer dollars.
                   Mounting concernsabout whether hospital and household disinfectants
                   work as claimed and the adequacy of the Environmental Protection
                   Agency’s (EPA) disinfectants program led the HouseCommittee on Gov-
                   ernment Operations to request that GAOreview EPA’Sregulation of the
                   efficacy of disinfectants.


Background         Under the Federal Insecticide, Fungicide, and Rodenticide Act (FIFRA),
                   EPAgenerally must register (license) pesticide products, including disin-
                   fectants, before they are marketed. EPAmay register a pesticide product
                   only if EPAdetermines that it is effective, when used as claimed, without
                   causing an unreasonablerisk to health or the environment. For most
                   pesticides, EPAallows the marketplace to regulate product performance
                   (efficacy) becauseusers can seewhether the pesticide is effective
                   against the target pest. However, registrants of disinfectants intended to
                   protect public health must submit efficacy data to substantiate each
                   product performance claim and use.
                   Until 1982, EPAconducted limited preregistration confirmatory and post-
                   registration enforcement tests on disinfectants at its laboratory facilities
                   in Beltsville, Maryland. EPAdiscontinued disinfectant testing in 1982 pri-
                   marily becauseof budget constraints. Currently, EPArelies on its review
                   of registrant-submitted efficacy data to register disinfectants. As of Sep-
                   tember 1, 1989, about 4,100 disinfectants for public-health use were reg-
                   istered with EPA,representing about 18 percent of approximately 23,000
                   registered pesticide products.

                   EPAdoesnot know whether disinfectants kill the germs claimed on
Results in Brief   product labels for four reasons.First, although the validity of methods
                   and performance standards used to assessthe efficacy of disinfectants
                   has been the source of scientific controversy for over a decade,EPAdoes
                   not independently test disinfectants before registering them and lacks
                   criteria to assessthe validity of registrant-proposed test methods and
                   modifications. Second,EPAhas made little progress in resolving these
                   controversies becauseof budget constraints and inadequate research
                   management.Third, EPAlacks sufficient internal controls to ensure the


                   Page 2                     GAO/ltCED.BO-122 EPA Lacka A~wce   Dbinfectanta   Work
                           Executive Siummary




                           quality and integrity of the data that registrants submit on disinfectant
                           efficacy. Fourth, EPAlacks an enforcement strategy to ensure that, once
                           registered, disinfectants sold and distributed in the marketplace work as
                           claimed.

                           The extent to which ineffective disinfectants are marketed is unknown.
                           Although the scientific controversies cloud the issue somewhat, evi-
                           dencefrom EPA,the states, and others suggeststhat up to 20 percent of
                           disinfectants on the market may be ineffective.


Principal Findings

Validity of Test Methods   EPAlacks assurancethat    the test methods and performance standards
                           used by registrants to substantiate disinfectant efficacy claims are
                           valid. EPAprimarily relies on standard-setting organizations, such as the
                           Association of Official Analytical Chemists (AOAC)and the industry itself
                           to develop test methods and performance standards. However, these
                           methods and standards have been embroiled in scientific controversies
                           for over a decade.For example, scientists have debated whether the
                           AOACUse-Dilution Method, the most widely used test method, is repro-
                           ducible, accurate, and precise, and whether the performance standard
                           (pass/fail criterion) established by EPAis valid. Although EPAbelieves
                           that the existing methods and standards are acceptablefor registering
                           and enforcing disinfectant efficacy claims, the controversies have
                           impaired the credibility of the disinfectant program, An ad hoc
                           industry/state group recently has developeda test method to replace
                           the AOACUse-Dilution Method and is expected to present the results of
                           its research to the AOACin September 1990. EPAofficials believe that the
                           new method is reproducible and reliable and will consider whether to
                           require that disinfectants be retested using it after AOACconsidersit for
                           adoption.
                           EPAhas contributed    to the controversies by accepting test methods and
                           modifications without criteria and independent laboratory data for eval-
                           uating their validity. For example, EPAacceptsthree different test
                           methods to demonstrate that disinfectants kill tuberculosis bacteria. At
                           least one product tested under two of the methods produced substan-
                           tially different results. Although EPAhas registered the product on the
                           basis of one of the methods, EPAlacks the laboratory information needed
                           to explain the differences in results between the methods.


                           Page 9                    GAO/RCED-99-139 EPA Lacks hsurtuwe   Didnfectants   Work
                                                                                                   .


                            Executive Summary




                            Although EPAhas been aware of the scientific controversies for years, it
                            has made little progress in resolving them becauseof problems in con-
                            ducting neededresearch.EPA'Sg-year, $384,000, cooperative agreements
                            with the University of North Carolina did not fulfill EPA'Sresearch
                            objectives to improve disinfectant efficacy methods becauseEPAinade-
                            quately managedthe agreements.EPAhas also made little progress in
                            conducting additional research becauseof budget constraints. In April
                            1990, EPAannouncedthat it would spend $600,000 for research on cer-
                            tain disinfectant efficacy methods and estimated an additional $1.2 mil-
                            lion will be needed.


Controls Over Quality and   To ensure the quality and integrity of registrant-submitted disinfectant
Integrity of Data           data, EPAreviews the data prior to registration and performs laboratory
                            inspections and data audits. GAO,however, found internal control weak-
                            nessesin these programs. For example, EPAhas not inspected the
                            majority of labs that have performed disinfectant efficacy studies. In
                            fact, EPAwas aware of only 12 of the 92 labs that had performed these
                            studies. Although these programs need to be improved they, in them-
                            selves,are not an adequate substitute for a preregistration program to
                            selectively test disinfectant efficacy by an independent laboratory. Data
                            reviewers, lab inspectors, and data auditors generally cannot identify
                            casesin which registrants have selectively submitted data indicating
                            that their disinfectants work becausethey generally do not observethe
                            tests in progress and no physical evidenceremains from the tests
                            conducted.


Monitoring and              EPA'Sregistration  processby itself cannot provide assurancethat disin-
Enforcement of Marketed     fectants are effective becauseregistrants could market ineffective
                            batches, either intentionally or inadvertently, after registering them.
Disinfectants               However, EPAdoesnot enforce the efficacy claims of disinfectants on the
                            market. EPAdiscontinued its limited enforcement testing program in
                            1982 primarily becauseof budget constraints. Since 1982, EPAhas
                            looked to the states, user groups, and the industry to enforce efficacy
                            claims, However, GAOfound few states and no users monitoring disinfec-
                            tant efficacy becauseof cost concerns.Only five states test disinfectants
                            for efficacy, and these states have limited programs. Moreover, EPA
                            lacks a strategy to channel complaints about potentially ineffective dis-
                            infectants from the states, user groups, and the industry and to take
                            appropriate enforcement action against disinfectants found to be inef-
                            fective. Although EPAneedsto resolve the scientific controversies that
                            surround disinfectant efficacy test methods and performance standards,


                            Page 4                    GAO/RCED-90-139 EPA Lacks Assurance Disinfectants Work
                  these controversies should not prevent EPAfrom developing an enforce-
                  ment strategy, in conjunction with the states, user groups and industry,
                  to ensure that marketed disinfectants work as claimed. Public health
                  and consumer welfare may be compromisedwithout such assurance.

                  GAOis making recommendationsto the Administrator, EPA,to         correct
Recommendations   deficiencies and restore credibility in the disinfectant program,
                  including (1) developing a plan to resolve the scientific controversies
                  that surround disinfectant efficacy test methods and performance stan-
                  dards; (2) developing and publishing a policy that establishescriteria
                  for evaluating the validity of new test methods and modifications,
                  including criteria for determining when independent laboratory data are
                  neededfor validation; (3) improving internal controls over its current
                  programs to ensure the quality and integrity of registrant-submitted
                  efficacy data and conducting preregistration tests to selectively verify
                  registrant claims; (4) establishing an enforcement strategy in conjunc-
                  tion with the states, user groups, and industry to ensure that marketed
                  disinfectants work as claimed; and (5) preparing a cost-benefit analysis
                  of alternatives for a laboratory facility to research and test the efficacy
                  of disinfectants, including the option of charging fees to register disin-
                  fectants to help finance such a facility.

                  GAOdid  not obtain official agency commentson this report. GAOdid,
Agency Comments   however, discussthe factual content of the report with EPAofficials and
                  has included their commentswhere appropriate. EPAofficials generally
                  agreed with the accuracy of the facts but believed that, as presented,
                  the report could be misread and suggestedchangesfor presenting the
                  facts. GAOmade somerevisions to the report on the basis of EPA'Scom-
                  ments. GAObelievesthat the report is a fair and accurate presentation of
                  the issues.




                  Page 6                     GAO/RCED-99-139 EPA Lacks Asmrance Disinfectants Work
Contents


Executive Summary                                                                                      2

Chapter 1                                                                                             8
Introduction             Federal Regulation of Disinfectants                                          8
                         Disinfectant Types and Uses                                                  9
                         Overview of Efficacy Data Requirements                                      12
                         Disinfectants and Infection Control                                         14
                         Prior GAO Reports                                                           16
                         Objectives,Scope,and Methodology                                            16

Chapter 2                                                                                            20
EPA Lacks Assurance      Scientific Controversy Over Methods and Standards
                         Methods AcceptanceProcessInadequate
                                                                                                     20
                                                                                                     24
That Efficacy Test       Limited ProgressMade in Resolving Controversies                             28
Methods Are Valid        Conclusions                                                                 34
                         Recommendations                                                             35

Chapter 3
Controls Over Quality/   WeaknessesImpair Data and Lab Reviews
                         Preregistration Testing Neededto Ensure Data Integrity
Integrity of Efficacy    Internal Controls Inadequate
Data Questionable        Conclusions
                         Recommendations

Chapter 4
Limited MOInitoring/     Federal Enforcement Testing Discontinued                                    47
TX-LT-------
          --^ of
I!d1lOI’CeItlerlt        Nonfederal Testing Limited                                                  48
                         SomeMarketed Disinfectants May Be Ineffective                               53
Registered               Conclusions                                                                 65
DiGnfectants             Recommendations                                                             55

Chapter 5                                                                                            67
Alternatives for a       Need for and Objectivesof a Disinfectant Laboratory
                         SuggestedCriteria for Evaluating Alternatives
                                                                                                     57
                                                                                                     68
Disinfectant             Alternatives for a Laboratory Facility                                      69
Laboratory               Feesfor a Disinfectant Laboratory                                           61
           Y             Conclusions                                                                 61
                         Recommendations                                                             62



                         Page 6                   GAO/RCED-90-139 EPA Lacka Assurance Didnfectmta   Work
             Content4




Appendixes   Appendix I: Affiliations of Individuals GAO Contacted                    64
             Appendix II: Major Contributors to This Report                           66

Glossary                                                                              67

Tables       Table 1.1: SelectedDisinfectant Efficacy Claims                          10
             Table 1.2: Estimated Average Annual Amount of                            12
                 Disinfectants Sold in the United States, 1986-87
             Table 1.3: EPA-RecommendedMethods for Testing                             13
                 Disinfectants Intended for Use on Hard Surfaces

Figures      Figure 1.1: RegisteredDisinfectant Efficacy Claims by                     11
                 Type
             Figure 4.1: Disinfectant Enforcement SamplesTested by                    60
                 North Carolina, Florida, and Mississippi, 1983-89




             Abbreviations

             ADA        American Dental Association
             AOAC       Association of Official Analytical Chemists
             CDC        Centers for DiseaseControl
             EPA        Environmental Protection Agency
             FDA        Food and Drug Administration
             FIFRA      Federal Insecticide, Fungicide, and Rodenticide Act
             FMFIA      Federal Managers’ Financial Integrity Act of 1982
             GAO        General Accounting Office
             GLP        good laboratory practice
             LDIAD      Laboratory Data Integrity Assurance Division
             NSF        National Sanitation Foundation
             OCM        Office of Compliance Monitoring
             OPP        Office of Pesticide Programs
             OPTS       Office of Pesticidesand Toxic Substances
             PDMS       Pesticide Document ManagementSystem
             PPIS       Pesticide Product Information System
                        Toxic SubstancesControl Act
             UNC        University of North Carolina


             Pqge 7                     GAO/RCED-90-139 EPA La&m Auumnce DblnMtan~   Work
Chapter 1

Introduction


                        Disinfectants are used almost everywhere people want to kill disease-
                        causing microorganisms-in households,hospitals, schools,restaurants,
                        day care centers, dairy farms, and a host of other places.About $1 bil-
                        lion a year is spent on disinfectants to kill bacteria, fungi, and viruses in
                        bathrooms, kitchens, and offices; on medical and dental instruments,
                        diaper pails, and eating utensils; and at many other locations, Although
                        the role of the inanimate environment in transmitting infections has not
                        been completely defined, the use of disinfectants is considered an impor-
                        tant part of infection control programs. In fact, health-care organiza-
                        tions recommend,and many public health ordinances require, their use.


Federal Regulation of   The Environmental Protection Agency (EPA) regulates disinfectants as
                        pesticides under the Federal Insecticide, Fungicide, and Rodenticide Act,
Disinfectants           as amended(FIFRA).~Under F-IFFU,disinfectants generally must be regis-
                        tered (licensed) by EPAbefore they may be sold, held for sale, or distrib-
                        uted in commerce.EPAmay register a disinfectant only if EPAdetermines
                        that it is effective, when used as directed, without causing an unreason-
                        able risk to public health or the environment. EPArequires disinfectant
                        firms to submit, among other things, data demonstrating that their prod-
                        ucts are effective as claimed before EPAwill register them. As of Sep-
                        tember 1, 1989, about 4,100 disinfectants were registered with EPA.
                        These disinfectants represented about 18 percent of approximately
                        23,000 registered pesticide products.

                        As permitted under FIFFtA,EPAhas waived all requirements for pesticide
                        firms to submit efficacy data except for (1) disinfectants and (2) pesti-
                        cides that claim to control vertebrates that may transmit diseasesto
                        humans, such as rodents, birds, and skunks. EPAretains the requirement
                        for disinfectants becauseusers cannot seewhether disinfectants kill
                        microorganisms that may causehuman disease,such as the bacteria that
                        causefood poisoning, and becausethe use of an ineffective disinfectant



                        ‘As defined by EPA, “disinfectant” refers to only one of several types of antimicrobial pesticides,
                        which, with some exceptions, are substances intended to inhibit or destroy microorganisms (bacteria,
                        fungi, viruses, and spores). However, we use the term “disinfectant” in this report to broadly describe
                        all antimicrobial pesticides intended to protect public health. For expianations of specific disinfectant
                        efficacy claims, see table 1.1 and the glossary.


                        “Disinfectants, which are used on inanimate surfaces, are distinct from antiseptics, which are used on
                        skin and other living tissue. Antiseptics are regulated by the Food and Drug Administration. Disinfec-
                        tants that are used in or on medical devices, such as dialysis machines, are regulated by both EPA and
                        the Food and Drug Administration.



                        Page 8                                GAO/RCED-99-139 EPA Lacks Assurance Dhinfectanta             Work
    .



                     chapter 1
                     Introduction




                     posesa threat to public health. By contrast, EPAdoesnot require regis-
                     trants to submit efficacy data for other pesticides becauseusers gener-
                     ally can tell whether they work, and the marketplace can regulate
                     product performance. In addition, registrants of pesticides that target
                     microorganisms that do not causediseasesin humans, such as those that
                     target slime-forming or odor-causingbacteria, are not required to submit
                     efficacy data. EPAdoes,however, require all registrants to be able to
                     show that their products are effective on demand for such data.

                     EPA’SOffice of  Pesticidesand Toxic Substances(OPTS)    is responsible for
                     regulating pesticides, including disinfectants. Within OPTS, the Office of
                     Pesticide Programs (OPP) and the Office of Compliance Monitoring (OCM)
                     are responsible for evaluating pesticides for registration and for plan-
                     ning and coordinating pesticide compliance/enforcement activities,
                     respectively. The Antimicrobial Program Branch (formerly the Disinfec-
                     tants Branch), within OPP,is responsible for registering disinfectants.
                     Within the branch, the Efficacy Evaluation and Technical Management
                     Section is responsible for approving and recommendingmethods for
                     testing the efficacy of disinfectants and evaluating registrant-submitted
                     efficacy data.


                     EPAregisters disinfectants
                                              with a variety of efficacy claims for use in
Disinfectant Types   many areas of the inanimate environment. Disinfectant types and uses
and Uses             range from products intended to kill bacteria on hard surfaces in bath-
                     rooms to products intended to chemically sterilize medical instruments
                     in hospitals.
                     A disinfectant may claim one or more of a number of types of efficacy.
                     The types of efficacy claims a disinfectant may make depend on, among
                     other things, the types of microorganisms the disinfectant targets (e.g.,
                     tuberculosis or a polio virus) and the disinfectant’s intended level of
                     activity (e.g., a reduction in the level of the microorganism or a complete
                     kill). (For a list of selecteddisinfectant efficacy claims, seetable 1.l.)




                     Page 9                       GAO/RCED-9&139 EPA Lacks Assurance Didnfectanta   Work
                                            Chapter 1
                                            Introduction




Table 1.1: Selected Disinfectant Efficacy
Clalmr
                                            Sterilizer                    The disinfectant, sometimes called a sporicide, is intended to
                                                                          destroy or eliminate viruses and all living bacteria, fungi, and their
                                                                          spores. (The claim denotes killing all microorganisms, including the
                                                                          highly resistant spore forms, and indicates that the disinfectant will
                                                                          produce the highest level of disinfection possible.)
                                            Tuberculocide                 The disinfectant is intended to destroy or inactivate tuberculosis
                                                                          bacteria. (Tuberculocidal claims are often used by medical users of
                                                                          disinfectants as an indicator of product strength because
                                                                          tuberculosis bacteria are more difficult to kill than most other
                                                                          species of bacteria.)
                                            Disinfectant                  The disinfectant is intended to destroy or inactivate one or more
                                                                          major species of bacteria, depending upon whether the disinfectant
                                                                          makes a “limited,” ” aeneral,” or “hospital” disinfectant claim.
                                            Fungicide                     The disinfectant is intended to destroy fungi.
                                            Virucide                      The disinfectant is intended to destroy or inactivate one or more
                                                                          specific viruses named on the disinfectant’s label.
                                            Sanitizer                     The disinfectant is intended to reduce the number of living bacteria
                                                                          or viable virus oarticles.
                                            Source: Prepared by GAO on the basis of EPA disinfectant efficacy data requirements and the definition
                                            of disinfectant   from footnote 1.


                                            We used EPA’S Pesticide Product Information System (PPIS) to obtain
                                            EPA’S best available data on registered disinfectant claims. According to
                                            the system, about 4,100 disinfectants3were registered with EPA as of
                                            September 1, 198ga4These disinfectants made about 8,000 different effi-
                                            cacy claims or, on average,about 2 claims per disinfectant. The most
                                            common efficacy claim was “disinfectant.” (Seefig. 1.1.)




                                            “EPA registers both end-use products (products for sale at the retail level) and manufacturing-use
                                            products (active ingredients for use in end-use products). Because EPA requires a complete efficacy
                                            profile only on end-use products, we did not include manufacturing-use products in our counts of
                                            disinfectants.


                                            4The 1999 amendments to FIFRA imposed user fees on pesticide registrants to help fund an acceler-
                                            ated review of older pesticides and expedited registration of new pesticides. In 1989, EPA canceled
                                            over 6,000 disinfectant registrations because registrants decided to abandon their registrations rather
                                            than pay the required fees.



                                            Page 10                                GAO/RCED-90-139 EPA Lacks Assurance Lbidnfectanta Work
                                        Chapter 1
                                        Introduction




Flgure 1.l : Reglrtlsred Dlrlnfrctant




                                                                                              2%
Efficacy Claim8 by Typo
                                             I                                                Virucide
                                                                                              Fungicide




                                                                                              1%
                                                                                              Tuberculocide


                                                                                              Sterilizer




                                                                                              Disinfectant




                                                                                              Sanitizer
                                        Dam are for end-use products as of September 1,1989.

                                        Water purifier daims were not included because they represented less than 1 percent of all
                                        registered disinfectant efficacy claims as of September 1, 1989.

                                        Source: Prepared by GAO on the basis of data from the EPA Pesticide Product Information System


                                        In addition to the types of efficacy claimed (e.g., tuberculocidal), each
                                        registered disinfectant must specify the use patterns for which the dis-
                                        infectant is recommended.Broad categoriesof use patterns include the
                                        use of a disinfectant (1) to kill microorganisms on hard surfaces, (2) to
                                        kill microorganisms on fabrics or textiles, (3) to control microbial pests
                                        associatedwith human or animal wastes, and (4) to treat water systems.
                                        Registrants typically label their disinfectants for use at specific sites
                                        within these broad categories.For example, a disinfectant intended for
                                        use on hard surfaces may be labeled for use on counter tops, medical
                                        instruments, floors, or other types of hard surfaces.

                                        According to PPIS, as of September 1, 1989, the approximately 4,100 dis-
                                        infectants made about 18,100 specific use site claims. About 76 percent


                                        Page 11                             GAO/RCED-90-139 EPA Lacks Assurance Disinfectants Work
                                      Chapter 1
                                      Introduction




                                      of the disinfectants, or 3,200, were registered for use on at least 1 hard
                                      surface site. Someof the common hard surfaces for which disinfectants
                                      were registered were the surfaces in bathrooms, hospitals, and eating
                                      establishments, and at commercial/industrial sites.

                                      EPAhas limited   data on the size of the disinfectant market. However,
                                      data from a private market research firm indicates that the disinfectant
                                      market is about $1 billion per year at the retail level in the United
                                      States. In addition, as table 1.2 illustrates, the estimated average annual
                                      amount of disinfectants sold represented a substantial amount of all
                                      pesticides sold in the United States during 1986-87.
Table 1.2: Estimated Average Annual
Amount of Dirinfectanta Sold in the   Pounds and gallons in thousands
United States, 1985-87                                                                     Amount sold
                                                                                    Percentage of                           Percentage
                                                                   Dry or solid        dry or solid        Liquid              of liquid
                                                                     chemical            chemical       chemical              chemical
                                                                 product (Ibs.)         pesticides  product (gal.)           pesticides
                                      Pesticides                      8,773,799                  100           1,067,308              100
                                      Disinfectants                   1,182,203                   13             542,606               51
                                      Disinfectants used
                                      on hard surfaces                  486.207                     6           493,159               46
                                      Source: Prepared by GAO on the basis of data from the PPIS and the FIFRA and Toxic Substances
                                      Control Act Enforcement System.



                                      EPArequires the manufacturer     or registrant to develop and submit data
Overview of Efficacy                  on a disinfectant for each claim and use of the product. EPAhas pub-
Data Requirements                     lished guidelines that recommendspecific methods and minimum test
                                      specifications for registrants to use to test the efficacy of disinfectants.
                                      The guidelines also contain performance standards (pass/fail criteria)
                                      that disinfectants must meet to make efficacy claims.


Description of Test                   EPA'SPesticide AssessmentGuidelines and supplemental technical gui-
Methods                               dance briefs contain recommendedmethods for testing disinfectants,
                                      standards for conducting acceptabletests, and instructions on inter-
                                      preting and reporting data. Table 1.3 lists the EPA-recommended
                                      methods for demonstrating specific efficacy claims of disinfectants used
                                      on hard surfaces.
                  II




                                      Page 12                             GAO/RCED-90-139 EPA Lacks Assurance Disinfectants Work
                                            Chapter 1
                                            Introduction




Table 1.3: EPA-Recommended Method8 for Testing Disinfectants Intended for Use on Hard Surfaces
lvr3e of claim                         EPA-recommended method(s)                  Test oraanismk)
Sterilizer
.- _.^-._- .--_ ---- _..-.-                 AOAC Sporicidal Test                              Bacillus subtilis and Clostridium sporogenes
Tuberculocide                               AOAC Tuberculocidal Activity Method               Mycobacterium tuberculosis
                                            Modified AOAC Tuberculocidal       Activity
                                            Method
                                            ;;;;Jative     Tuberculocidal   Activity Test

Hospital disinfectant                       AOAC Use-Dilution Method                          Salmonella choleraesuis, Staphylococcus
                                                                                              aureus, and Pseudomonas aeruginosa
.- __- .._.__._ _-_-_-____-. __--           AOAC Germicidal Spray Products Test
General disinfectant                        AOAC Use-Dilution Method                          Salmonella choleraesuis and Staphylococcus
                                                                                              aureus
                                            AOAC Germicidal Spray Products Test
Limited disinfectant                        AOAC Use-Dilution Method                          Salmonella choleraesuis or Staphylococcus
                                                                                              aureus
.._._....~. .   ----.--    ..- .__-     -   AOAC Germicidal Spray Products Test
Fungicide                                   AOAC Fungicidal Test                              Trichophyton   mentagrophytes

                                            AOAC Use-Dilution Method

                                            AOAC Germicidal Spray Products Test
Virucide                                    EPA virucidal test parameters                     Specific virus claimed
Sanitizing rinse (food-contact surfaces)    AOAC Available Chlorine Germicidal                Escherichia coli; and Salmonella typhi or
                                            Equivalent Concentration Method                   Staphylococcus aureus

                                            AOAC Germicidal and Detergent Sanitizers
.-.- ..-.     -.-.“.-- -~~ .                Method
Sanitizer (inanimate, nonfood-contact       EPA sanitizer test parameters                     Staphylococcus aureus; and Klebsiella
surfaces)                                                                                     pneumoniae aberrant or Enterobacter
                                                                                              aerogenes
                                            Source: Prepared by GAO on the basis of EPA’s Pesticide Assessment Guidelines.


                                            Most of the EPA-recommended     methods were developedunder the aus-
                                            pices of the Association of Official Analytical Chemists (AOAC),an inde-
                                            pendent, international standard-setting organization. AOAC'Sprimary
                                            purpose is to develop and validate standardized chemical and biological
                                            analytical methods that will perform with the necessaryaccuracy and
                                            precision under usual laboratory conditions to be recognizedas “offi-
                                            cial” methods by the courts and others. According to the AOACAssistant
                                            Executive Director, a collaborative study, or round robin test, is con-
                                            ducted for each method to provide an unbiased evaluation of the per-
                                            formance of an analytical method through the analysis of a number of
                                            identical samplesby a number of different laboratories.




                                            Page 13                               GAO/RCED-90-139 EPA Lacks Assurauce DMnfectant~ Work
                        Chapter 1
                        Introduction




                        In evaluating registrant-submitted efficacy data on disinfectants, EPA
                        determines whether the data are adequateto satisfy its data require-
                        ments and guidelines. This determination involves considering the
                        design and conduct of the test, including whether generally accepted
                        methods were used, whether a sufficient number of measurementswere
                        made to achieve statistical reliability, whether sufficient controls were
                        built into the test, whether the test was conducted in conformity with
                        the design, whether good laboratory practices were observed,and
                        whether the results were reproducible.


Description of          The Pesticide AssessmentGuidelines also contain performance stan-
Performance Standards   dards that EPA requires disinfectant products to meet to make specific
                        efficacy claims. Failure of a product to meet the specified testing or per-
                        formance requirements is consideredevidencethat the product is
                        unlikely to be effective as claimed in actual use.
                        The performance standards vary depending on the claim intended and
                        the test method employed. For example, for a hospital disinfectant, EPA
                        recommendsthat three batches be tested using the AOAC Use-Dilution
                        Method against three different microorganisms using 60 test tubes per
                        microorganism per batch. If the product fails to kill a microorganism in
                        2 or more out of the 60 tubes for any microorganism/batch, then the
                        product is consideredto have failed the test for the specific microor-
                        ganism tested.

                        Although the role of the inanimate environment in transmitting infec-
Disinfectants and       tions has not been completely defined, doctors, dentists, restaurant
Infection Control       owners, consumers,and others consider disinfectants to play an impor-
                        tant part in infection control. Health-care providers, as well as others,
                        rely on EPA’S registration of disinfectants as evidencethat purchased
                        products work as claimed.
                        Infection control is a serious concern for health-care providers.
                        According to the Centers for DiseaseControl (WC), about 6 percent of all
                        patients acquire an infection while hospitalized. Hospital-acquired, or
                        nosocomial,infections prolong hospital stays, increasepatient care
                        costs, and, in somecases,causedeath. According to one estimate,




                        Page 14                    GAO/RCED-90-139 EPA Lacks Assurance DMnfectanta Work
Chapter 1
IntroducWon




nosocomialinfections may causeapproximately 20,000 deaths and con-
tribute to about 60,000 more deaths annually.” Furthermore, according
to one infection control expert, nosocomial infections rank among the 10
most frequent causesof death in the United States.‘;In addition, interest
in disinfectant efficacy has increased in responseto the growing num-
bers of immune deficient patients, who are susceptible to infections.
Although medical experts generally believe that most nosocomial infec-
tions are transmitted from person to person rather than from the inani-
mate environment, the role of the inanimate environment in
transmitting infections has not been completely defined. Researchhas
linked at least someinfections, including fatalities, to contaminated
endoscopesand other medical instruments, but their exact contribution
to the overall nosocomial infection rate is unknown. In addition, contro-
versy exists over the extent to which inanimate objects (e.g., stetho-
scopes)and environmental surfaces (e.g., floors, walls, sink drains) that
comeinto contact with intact skin can transmit infections. Lastly, many
infections from inanimate objects may never be detected becauseso
much time can elapsebetween infection and the onset of illness that the
source of the infection is difficult to trace.

Health-care providers generally use disinfectants to treat medical
instruments they cannot otherwise sterilize or disposeof. For example,
disinfectants are used on sensitive medical instruments, such as fiber-
optic endoscopes,that cannot be sterilized using heat and on instru-
ments that need to be sterilized quickly between uses.In addition, many
health-care providers view the use of disinfectants on objects/surfaces
that comeinto contact with intact skin as a necessarypart of infection
control, in the absenceof evidencethat these objects/surfaces play a
negligible role in transmitting infections,
Infection control also is a serious concern in the food-processingand
food-service industries, since food can be easily contaminated by dis-
ease.The total amount of food-borne illness in the United States is
unknown, but outbreaks occur frequently. Both industries use sanitizers
on food-contact surfaces (e.g., food-processingequipment and utensils)
to reduce the likelihood that food may becomecontaminated.
“James M. Hughes and William R. Jarvis, “Epidemiology of Nosocomial Infections,” in Manual of
Clinical Microbiology, 4th ed., ed. Edwin H. Lennett et al. (Washington, DC.: American?%i%j%r
Microbiology, 1986).


“Robert W. Haley, Managing Hospital Infection Control for Cost-Effectiveness (Chicago: American
Hospital Publishing, Inc., 1986).



Page 15                            GAO/RCRD-99-139 EPA Lacks Assurance Disinfectants Work
                        Chapter 1
                        Introduction




                        We have issued two reports in the past addressingthe need to better
Prior GAO Reports       regulate the efficacy of disinfectants. In 1968, we reported that many
                        pesticides subject to enforcement-seizureaction, including somedisinfec-
                        tants that the federal government found to be ineffective, may have
                        remained on the market7 In 1974, we questioned EPA’Sreliance on regis-
                        trant-submitted data for registering pesticides with high rates of effi-
                        cacy failures, such as disinfectants. We also found that EPAdid not
                        always cancel disinfectant registrations or require registrants to delete
                        efficacy claims from the labels of repeatedly ineffective disinfectants.R

                        In a January 27, 1989, letter, and at subsequentmeetings with their
Objectives, Scope,and   staffs, the Chairman and Ranking Minority Member of the HouseCom-
Methodology             mittee on Government Operations, and the Chairman and Ranking
                        Minority Member of the Environment, Energy and Natural Resources
                        Subcommittee,HouseCommittee on Government Operations, asked us to
                        review EPA’Sefforts to regulate the efficacy of disinfectants. In partic-
                        ular, the Committee and Subcommitteeagreedthat we would address
                        the following questions:

                        What is the nature of the scientific controversies surrounding EPA-rec-
                        on-unendedmethods for testing the efficacy of disinfectants?
                        What action has EPAtaken to obtain objective research to resolve the
                        scientific controversies that surround disinfectant efficacy test methods
                        and performance standards?
                        DoesEPAhave sufficient internal controls to ensure the quality and
                        integrity of registrant-submitted disinfectant efficacy data?
                        What post-registration enforcement procedures have EPAand the states
                        implemented to ensure that disinfectants on the market are effective?
                        Doesa need exist for a post-registration program to monitor disinfec-
                        tants on the market and, if so, what options exist for structuring such a
                        program?

                        EPAbelievesthat   all disinfectants are critical to protecting public health.
                        However, we decided to concentrate our review on EPA’Sregulation of
                        the efficacy of disinfectants registered for use on hard surfaces because
                        over 76 percent of all disinfectants were registered for use on at least
                        one hard-surface use site. In addition, these disinfectants are most

                        ‘Need to Improve Regulatory Enforcement Procedures Involving Pesticides (5133192, Sept. 10,
                         1968).
                        *Pesticides: Actions Needed to Protect the Consumer From Defective Products (B133192, May 23,
                        1974).



                        Page 16                           GAO/RCED-99-139 EPA Lacks Assurance Didnfectanti        Work
Chapter 1
Introduction




affected by the scientific controversies surrounding the test methods
used to substantiate efficacy.
To understand EPA'Sefficacy data requirements and determine the
nature of the scientific controversies surrounding EPA-recommended
methods for testing disinfectant efficacy (seech. 2), we (1) reviewed,
and interviewed EPAofficials about, EPA’Sdisinfectant efficacy data
requirements and procedures for accepting test methods, procedures,
and modifications; (2) observed microbiologists at two state laboratories
perform efficacy tests; (3) interviewed sourcesknowledgeable about the
test methods and controversies, including individuals affiliated with
federal and state agencies,standard-setting organizations, professional
and trade associations,scientific associations,universities, and commer-
cial laboratories (see app. I); (4) attended several conferencesand meet-
ings held by the AOAC,  the standard-setting organization whose methods
are the subject of scientific controversy, and discusseddisinfectant effi-
cacy test methods and related issueswith participants; and (5) reviewed
scientific and medical literature on disinfectant efficacy and efficacy
test methods.
To determine what steps EPAhas taken to obtain objective research to
resolve the controversies (seech. 2), we (1) interviewed the EPAgrant,
project, and quality assuranceofficers responsible for managing and
overseeingEPA’Scooperative agreementson efficacy test research with
the University of North Carolina at Chapel Hill (UNC); officials within
the EPAAntimicrobial Program Branch; and the two lead UNCresearchers
and the statistician participating in the cooperative agreements;(2)
reviewed EPAfinancial assistanceagreement regulations, policies, and
procedures; (3) reviewed EPAgrant and project officer files and ~JNC
records, including financial statements, progress reports, and technical
reports; (4) interviewed individuals outside EPAabout the research per-
formed under the agreements;and (5) reviewed EPA’SJune 1987 strategy
paper on improving its disinfectant program and updates to the
strategy. We were unable to determine whether EPAhad obtained objec-
tive research becauseEPAinadequately managedthe cooperative
agreements.
To determine whether EPAhas sufficient internal controls in place to
ensure the quality and integrity of registrant-submitted disinfectant
efficacy data (seech. 3), we (1) reviewed the Federal Managers’ Finan-
cial Integrity Act of 1982 (FMFIA)and federal guidelines on internal con-
trols, (2) reviewed EPA’S1983 through 1989 annual internal control



Page 17                   GAO/RCED-90-139 EPA Lacks Assurance Disinfectants Work
Chapter 1
Introduction




reports and other EPAdocumentson EPA'Sinternal controls, (3) inter-
viewed EPAofficials to determine what procedures EPAhas implemented
to ensure the quality and integrity of the efficacy data that registrants
submit and whether the procedures are effective, and (4) reviewed files
from EPAinspections/audits over about a 4-year period at laboratories
that have conducted disinfectant efficacy studies.
To determine what post-registration procedures EPAand the states had
implemented to ensure that disinfectants on the market are effective
(seech. 4), we (1) visited and/or interviewed officials from states that
monitor the efficacy of marketed disinfectants and obtained available
testing and enforcement data; (2) interviewed several officials repre-
senting states that do not test disinfectants to determine why; (3) visited
EPA'Sdisinfectant laboratory facility in Beltsville, Maryland and
reviewed existing records from the testing program maintained at the
lab; and (4) interviewed EPAofficials about EPA’Sdecision to discontinue
a limited program to test disinfectants for efficacy. To determine which
states monitor the efficacy of marketed disinfectants, we relied prima-
rily on EPAto survey the states through the EPAregional pesticide branch
chiefs.

To determine whether a need exists for a post-registration program to
monitor disinfectants on the market and what options exist for struc-
turing such a program (seech. 5), we (1) interviewed individuals within
EPA,the disinfectants industry, the health-care community, state agen-
cies, and academia;(2) reviewed available disinfectant laboratory cost
estimates; and (3) reviewed the history of legislative proposals to
resume a disinfectant-testing program at EPA’SBeltsville laboratory.
To identify the number of registered disinfectant products and claims
and the amount of disinfectants sold in the United States, we used data
from EPA’SPesticide Product Information System and FIFRAand Toxic
SubstancesControl Act (TSCA)Enforcement System (FIFRAsec.7 annual
production report data). To identify laboratories that conducted disin-
fectant efficacy studies submitted to EPA,we used EPA’SPesticide Docu-
ment ManagementSystem. (We will express our concernsabout the
accuracy and completenessof disinfectants data in these systems in a
separate letter to be issued shortly to the Administrator, EPA.)
To understand the role of disinfectants in preventing diseasetransmis-
sion, we (1) reviewed medical literature on disinfectants and infection




Page 18                    GAO/RCED-99-139 EPA Lacks Assurance Disinfectants Work
Chapter 1
Introduction




control, (2) interviewed representatives from the infection control com-
munity and reviewed their infection control guidelines, and (3) inter-
viewed infection control personnel at a large urban teaching hospital
about infection control procedures and watched them demonstrate these
procedures.
We did not (1) attempt to resolve the scientific controversies about dis-
infectant efficacy test methods by independently analyzing the validity
of the methods, (2) review the efficacy of any individual disinfectant or
classof disinfectants, (3) addressEPA'Sefforts to monitor disinfectant
efficacy claims made in product advertising, or (4) assessEPA'Sknowl-
edge and regulation of the toxicity of disinfectants.
Our review was conducted from January through November 1989 and
updated with information gathered through June 1990 in accordance
with generally acceptedgovernment auditing standards. As requested,
we did not obtain official agency commentson this report. We did, how-
ever, discussthe factual content of the report with EPAofficials and
have included their commentswhere appropriate. EPAofficials generally
agreedwith the accuracy of the facts, but believed that as presented the
report could be misread, and suggestedchangesfor presenting them. We
made somerevisions to the report on the basis of EPA'Scomments and
believe that the report is a fair and accurate presentation of the issues.




Page 19                   GAO/RCED-90-139 EPA Lacks Assurance Disinfectnuts Work
                                                                                                J          e
Chapter 2

WA Lacks AssuranceThat Efficacy Test
MethodsAre Valid

                             Doctors,janitors, consumers,and others rely on EPA'Sreview and
                             approval of registrant-submitted data for assurancethat disinfectants
                             work as claimed. However, we found that this reliance may be
                             unfounded because(1) scientific consensusis lacking on the validity of
                             the test methods and standards EPArecommendsto registrants for sub-
                             stantiating disinfectant claims and (2) EPAlacks criteria for assessing
                             the validity of registrant-proposed test methods and standards and does
                             not independently validate test methods before permitting their use. We
                             question whether EPAhas adequately attempted to resolve the contro-
                             versies. EPAinadequately managedthe limited research that it has
                             funded to help resolve the scientific controversies, and the research has
                             proved to be controversial itself. Further, EPAhas made limited progress
                             in conducting additional research to resolve the controversies becauseof
                             budget constraints.

                             EPA'Sregulations require   that pesticide registration test methods be sta-
Scientific Controversy       tistically reliable, generally acceptable,and reproducible. However,
Over Methods and             almost all of the EPA-recommended    efficacy test methods and perform-
Standards                    ante standards for the approximately 3,200 disinfectants used on hard
                             surfaces have been embroiled in scientific controversy for over a
                             decade.Various scientists and officials from EPA,state governments,
                             academia,industry, commercial laboratories, scientific associations,and
                             user groups disagreeover whether
                         . the test methods contain uncontrollable variables that produce inconsis-
                           tent and nonreproducible results,
                         l the existing laboratory test methods adequately simulate performance
                           of a disinfectant in actual use, and
                         . the performance standards (pass/fail criteria) for existing methods are
                           valid or should be changed.
                             Although EPAand state officials acknowledge that the existing disinfec-
                             tant efficacy methods and standards may need to be improved, EPAand
                             certain state officials argue that the existing methods and standards
                             have not been invalidated, are the only available methods/standards,
                             and are acceptablefor registration and enforcement purposes.
                             According to these officials, inconsistent test resu!ts on possibly margin-
                             ally effective and ineffective products may be at least as much respon-
                             sible for the alleged problems with disinfectant efficacy test methods
                             and standards as the methods and standards themselves.Further, sev-
                             eral state officials and others believe certain registrants and others have



                             Page20                     GAO/&cED4O-12O
                                                                    EPALa& Amuance Dbinfectm~taWork
                        Chapter2
                        EPALmkaAsmranceThat EfficacyTeat
                        Methods Are Valid




                        raised problems with the methods and standards to divert the attention
                        of enforcement review from products that are possibly ineffective.
                        Wewere unable to disentangle valid criticisms of test methods and per-
                        formance standards from possible industry self-interest or possible
                        industry-funded research biasesbecauseof the mutually dependent
                        relationship that exists between researchersand industry. EPAdoesnot
                        believe that it registers ineffective disinfectants. However, we found
                        that the validity of EPA’Srecommendeddisinfectant efficacy methods
                        and standards has been disputed to such an extent that the credibility
                        and use of these methods and standards to substantiate disinfectant
                        efficacy claims have been impaired.


Variabilities in Test   Methods recommendedby EPAfor testing disinfectant efficacy have been
Methods Alleged         widely criticized by industry, academia,and others for producing highly
                        variable results. Researchersdisagreeover the extent and causesof the
                        variability and over how the methods can be improved or replaced.
                        Industry and other critics have alleged that extreme variability inherent
                        in the test methods and laboratory procedures raise doubts about
                        whether the test results can be repeated within the samelab and
                        whether test results can be reproduced by different labs. Specific con-
                        troversies related to test variabilities involve questions like the fol-
                        lowing: (1) Doesthe design of the methods or the efficacy of products
                        and chemical compoundstested account for inconsistent results? (2) To
                        what extent can variability in laboratory procedures,operator tech-
                        nique and experience, and materials used be controlled or reduced?and
                        (3) Do EPA’Srecommendedmethods, someof which were developedover
                        20 years ago, lack the necessaryaccuracy and precision expected of test
                        methods today?
                        Most criticisms have focused on the AOAC Use-Dilution Method and the
                        AOACTuberculocidal Activity Method. In fact, the AOAC has actively con-
                        sidered repealing the two test methods during the past 3 years because
                        of reports that results from the test methods could not be consistently
                        reproduced.1However, becauseof similarities in design, concernsabout
                        test variabilities in the AOACUse-Dilution and Tuberculocidal methods


                        ‘The AOAC repealed the Tuberculocidal Activity Method at its general meeting in September 1988.
                        The AOAC Board of Directors reinstated the method in March 1989, following an objection raised by
                        EPA in December 1988 that the AOAC membership voted on the basis of erroneous information
                        presented at the 1988 meeting.




                        Page21                             GAO/RCED-99-139
                                                                        EPALacksAssuranceDiainfectanteWork
                          Chapter 2
                          EPA Lacke Assurance That Efficacy Test
                          Methods Are Valid




                          may also apply to other disinfectant efficacy tests, such as the AQAC
                          Sporicidal Test.


Ability of Lab Tests to   Controversy exists over the extent to which EPA-recommended      efficacy
Simulate Actual Use       tests should and do simulate actual use and whether they provide a suf-
                          ficient margin of safety to allow for expected variations in actual use
Questioned                conditions. EPAhas generally assumedthat if a disinfectant fails to per-
                          form as claimed under the recommendedefficacy tests conducted in a
                          laboratory, then it will fail to perform under actual use conditions. How-
                          ever, critics from industry, academia,and other organizations claim that
                          the laboratory tests may not accurately predict how a disinfectant will
                          perform in actual use becausethe surfaces, number and resistanceof
                          microorganisms, presenceof organic matter (e.g., blood), disinfectant
                          concentration, ambient temperature at which a disinfectant is used, and
                          amount of time a disinfectant is exposedto a contaminated surface
                          (referred to as “contact time”), among other things, encounteredunder
                          actual use conditions may differ significantly from laboratory test con-
                          ditions, In fact, someinfection control experts have advised disinfectant
                          users to extend the contact time on EPA-registeredlabels to compensate
                          for the unknown margin of safety in EPA-recommended      efficacy tests.

                          Central to the laboratory simulation question is the controversy over
                          carrier-based versus suspension-baseddisinfectant efficacy test
                          methods.”Scientists disagree over whether carrier-based methods pro-
                          vide a greater representative link between laboratory tests and actual
                          use than suspensiontests and whether this purported advantage is
                          offset by an unacceptableincrease in variable test results. Further, some
                          scientists question whether labs can bias test results by identifying and
                          selectively using carriers with a higher probability of yielding negative
                          results. Controversy also exists over whether microorganisms are more
                          resistant to chemical disinfectants in carrier-based methods than in sus-
                          pension-basedmethods and over whether one type of material used for
                          the carriers is better than another. An industry/state ad hoc group,
                          working under the auspicesof the AOAC,    and other scientists have
                          recently concluded that carrier-based methods can be refined to reduce
                          variable test results by using different carrier materials (e.g., glass
                          versus stainless steel) and more stringent laboratory procedures.How-
                          ever, other scientists argue whether certain carrier materials, such as
                          “Carrier methods test a product’s effectiveness against a test microorganism dried on a “carrier” (a
                          small cylinder called a “penicylinder”). Suspension methods test a product’s effectiveness against a
                          test microorganism suspended in solution. The purpose of the carrier is to simulate the surface on
                          which the product would be used.



                          Page 22                              GAO/RCED-99-139 EPA Lacks Assurance Disinfectanta Work
                          Chapter 2
                          EPALackr,AINIUFMC~   That   EMcncy Test
                          Methods Are Valid




                          glass,truly represent environmental surfaces likely to be treated with
                          disinfectants, They also question whether, even granting that these
                          materials do approximate real surfaces, their use reducesvariability to
                          an acceptablelevel.

                          Most of the EPA-recommended    methods for demonstrating efficacy for
                          various types of disinfectant claims on hard surfaces are carrier-based
                          methods. Although EPAgenerally prefers carrier-based methods because
                          they presumably have a closer link with reality, it has accepteda few
                          suspension-basedmethods for disinfectant claims on hard surfaces, such
                          as for tuberculocidal claims, when it has believed that the claims would
                          be as stringent as the claims made on the basis of carrier methods.


Validity of Performance   Much of the scientific and regulatory controversy over the efficacy test
Standards Unknown         methods focuseson EPA’Sperformance standards (pass/fail criteria) for
                          the methods. Someindustry officials and researchersassert that the
                          existing performance standards are invalid and need to be changed.
                          Conversely, EPAbelieves that the standards are valid for registration
                          purposes,even though these standards were adopted many years ago
                          without the benefit of statistical analysesthat would be performed for
                          new standards today. EPAplans to continue using the standards to reg-
                          ister disinfectant efficacy claims unless investigations, confirming alle-
                          gations that they are invalid, show that these standards need to be
                          changed.
                          According to someindustry members,EPA’Sexisting performance stan-
                          dards are too stringent-in the caseof the AOACSporicidal Test, nearly
                          impossible to achieve- and such stringency explains why certain regis-
                          tered disinfectant claims could not be substantiated in collaborative
                          studies and state enforcement labs. Membersof industry argue, for
                          example, that EPA’Sstandard for a hospital disinfectant claim is arbi-
                          trary. This standard permits only 1 failure out of each set of 60 test
                          tubes when tested by the AOACUse-Dilution Method against three dif-
                          ferent microorganisms. Industry membersclaim that such a standard
                          doesnot adequately allow for the probability that other factors in the
                          test (e.g., variation in the number of test microorganisms in each test
                          tube, variation in the carriers) could explain why an effective disinfec-
                          tant might sometimesfail the test.
                          SomeEPAofficials believe membersof industry have tried to encourage
                          EPAto relax its performance standards to avoid possible enforcement
                          action on ineffective disinfectants. Despite the alleged variabilities in


                          Page 23                          GAO/RCED-99-139 EPA Lacks Assurance Disinfectants Work
                     Chapter 2
                     EPA Lacks Assurance That Efficacy Test
                     Methods Are Valid




                     efficacy test results and stringency in performance standards, the data
                     that EPAreceivesfrom registrants rarely show such variabilities or that
                     disinfectants fail to meet existing standards. Researchersand state
                     enforcement labs have been unable to substantiate claims for somereg-
                     istered disinfectants. Further, according to EPA/stateofficials, certain
                     registered disinfectants have failed state and federal enforcement tests
                     by such a wide margin that the disinfectants tested would be judged
                     ineffective by almost any performance standard. EPAofficials and others
                     have suggestedthat becausesometypes and concentrations of active
                     ingredients yield highly variable test results, disinfectants with these
                     ingredients may need to be reformulated with greater concentrations or
                     with different ingredients, or claim longer contact times. Industry offi-
                     cials counter that increasing the concentration of active ingredients in
                     their disinfectants will increaseproduct costs and toxicity and may not
                     improve the “effectiveness” of the disinfectants in actual use. (We dis-
                     cussthe possibility that someregistrants may have selectively sub-
                     mitted data for registration in ch. 3 and the need for post-registration
                     enforcement tests in ch. 4.)

                     Disagreementalso exists over whether and how the existing perform-
                     ance standards should be changed.Someresearchersargue that the
                     alleged variabilities in the test methods need to be resolved before the
                     performance standards can be modified since changing the standards to
                     allow for test variabilities may increasethe probability that EPAwould
                     accept an ineffective product. However, according to membersof
                     industry, the standards can be modified on the basis of theoretical calcu-
                     lations while the methods are being improved.

                     EPA'Sprocessfor accepting disinfectant efficacy test methods has con-
Methods Acceptance   tributed to the controversies surrounding the methods. Over the years,
ProcessInadequate    EPAhas acceptedsomedisinfectant efficacy test methods and modifica-
                     tions to the methods-usually to register product claims at industry’s
                     urging-that have not been developed and acceptedby independent
                     standard-setting organizations. EPAhas acceptedthe methods and modi-
                     fications on the basis of internal, and in somecasesexternal, scientific
                     peer review and regulatory judgment. However, EPAlacks criteria for
                     assessingthe validity of proposed new test methods or significantly
                     modified test methods. Furthermore, although EPAdoesnot indepen-
                     dently validate methods and doesnot require registrant-proposed
                     methods and significant modifications to undergo collaborative testing
                     and evaluation prior to acceptance,EPAscientists told us that they
                     cannot evaluate the validity of new methods or significant modifications


                     Page 24                         GAO/WED-99-139   EPALacka Assurance Dimlnfectante Work
                             Chapter 2
                             EPA Lacks Aseurance That Efficacy Test
                             Methods Are Valid




                             without a laboratory evaluation. EPA’Sexperience in accepting alterna-
                             tive methods for substantiating tuberculocidal and other claims suggests
                             that EPAneedscriteria for assessingthe validity of registrant-proposed
                             methods and significant modifications, including criteria for determining
                             when laboratory data, such as from a collaborative study, are neededto
                             demonstrate validity.


Lack of Criteria to Assess   Under EPAregulations, registrants may use any test method to demon-
Validity                     strate product performance as long as the test method used meets the
                             purpose of the test standards specified in EPA’SPesticide Assessment
                             Guidelines and provides data of suitable quality and completenessas
                             typified by the methods that EPArecommendsin the guidelines. EPAalso
                             allows registrants to modify the recommendedmethods to make them
                             suitable for a particular product claim, such as effectiveness against
                             target microorganisms in the presenceof organic matter and hard water.
                             However, under the guidelines, registrants are responsible for demon-
                             strating the validity of the test method or modification selectedto sub-
                             stantiate product efficacy. According to the Director, Office of Pesticide
                             Programs, consistent testing with uniform and valid methods is essential
                             for judging the comparability of efficacy test results.

                             As noted in chapter 1, EPAhas primarily relied on the A~ACand other
                             independent standard-setting organizations as sourcesfor the recom-
                             mended methods published in the guidelines. However, becausethese
                             organizations have not published methods for demonstrating all types of
                             disinfectant efficacy claims, EPAguidelines do not contain standard
                             methods for demonstrating all types of claims. For example, the guide-
                             lines do not contain a standardized method for registrants to use to
                             demonstrate that products are virucidal when used on hard surfaces.
                             Instead, EPA’Sguidelines contain minimum specifications (parameters)
                             with which registrants must comply to make these types of efficacy
                             claims.
                             EPAhas primarily relied on scientists in the Antimicrobial Program
                             Branch to review the validity of proposed new methods and modifica-
                             tions. In caseswhere registrants have proposed an alternative method
                             that represents a major departure from existing EPA-approvedmethods,
                             EPAhas consulted the scientific community, including the FIFRAScientific
                             Advisory Panel,”the Centers for DiseaseControl (CDC),and the Food and

                             3The FIFFU Scientific Advisory Panel is a statutorily created panel of experts convened to review
                             qajor pesticide decisions or regulations and to give advice to the Administrator, EPA.



                             Page 26                             GAO/RCED-99439 EPA Lack8 Amurance DMnfectantr              Work
                         Chapter 2
                         EPA Imka Amurauce That EfYicacy Test
                         Methods Are Valid




                         Drug Administration (FDA),before accepting the method. However, EPA
                         has not developed written criteria that methods must meet for EPAto
                         consider them valid. Further, EPAhas recognizedthat laboratory evalua-
                         tions are sometimesneededto assessthe validity of new methods and
                         significantly modified methods. In particular, according to EPAscientists,
                         it is difficult for them to determine, in the absenceof laboratory evalua-
                         tions, the extent to which new test methods and significantly modified
                         methods are reproducible and reliable.
                         EPAofficials  argue that EPAneedsthe regulatory flexibility to accept new
                         or alternative methods to respond to changesin infection control needs
                         and product claims. Further, EPAofficials claim that collaborative
                         testing of methods prior to acceptanceis a lengthy processusually
                         requiring 2 or more years to complete and that EPAlacks the laboratory
                         facilities and personnel to validate registrant-submitted test methods
                         independently. However, EPAofficials acknowledge that collaborative
                         testing of methods is scientifically ideal and that reviewing registrant-
                         proposed methods and modifications is an insufficient mechanism for
                         evaluating the comparability of results obtained with multiple methods
                         for substantiating one type of efficacy claim.


Problems With Multiple   Since about 1976, EPAhas received conflicting reports about the effec-
Tuberculocidal Methods   tiveness of glutaraldehyde-based products when tested by the AQAC
                         Tuberculocidal Activity Method-a carrier-based test. (Glutaraldehyde
                         is one of the most widely used active ingredients for hospital disinfec-
                         tants.) Researchersfrom a leading manufacturer of glutaraldehyde-
                         basedproducts submitted evidenceto EPAindicating that none of six
                         glutaraldehyde-based products tested, including the manufacturer’s
                         product, met their label claims of killing tuberculosis bacteria in 10 or 20
                         minutes at 2@@(about 6&F) when tested under the AoACmethod. The
                         researchersclaimed that glutaraldehyde was more sensitive to tempera-
                         ture than previously thought and that variables in the method, particu-
                         larly the carrier, led to inconsistent and erroneous results. EPAhas
                         reported that it obtained similar inconsistenciesin its preregistration
                         testing of tuberculocidal products using the AOACmethod at its testing
                         facility in Beltsville, Maryland, between 1971 and 1979. In 1983, the
                         industry researchers,also key ~o~c officials, submitted a quantitative
                         suspensiontest, intended to replace the AOACTuberculocidal Activity
                         Method, to EPA.
                         In May 1986, on the basis of internal review and its analysis of two sep-
                         arate FIFRAScientific Advisory Panel subpanel reviews, EPAconcluded


                         Page 20                       GAO/RCED-99-139 EPA Lacks Assurance Disinfectants Work
chapter 2
EPA Lacka Assurance That Efficacy Test
Methods Are Valid




that both the AOA~ method and the proposed quantitative suspension
method had merit. EPAdecided to allow registrants to choosefrom one of
three testing options to substantiate tuberculocidal claims. In June 1986,
EPArequired registrants of all tuberculocidal products to submit new
data to support the claim using the new quantitative method, the AOA~
Tuberculocidal Activity Method, or the AOAC method with substantial
modification of the contact time and/or temperature. EPAalso required
registrants of certain chemical classesto submit test data from a second,
independent laboratory to validate test results from the first laboratory.
(In 1987, the AOACedited its tuberculocidal activity method to state that
the method had not beenvalidated for glutaraldehyde-basedproducts.)
As of February 1989,44 out of 144 products had satisfied EPA'Srequest
for data; the remaining products either deleted their tuberculocide label
claims or were suspendedor canceled.Of the 44,11 used the new
method, 26 used the standard AOAC method, and 8 used the modified
method. The 19 products that relied on either the new method or the
modified AOAC method resulted in label claims of either use at higher
temperature or different contact time or both.

Although most registrants submitted data using only one of the test
options, several registrants developed data using more than one test
option. According to EPA,the data indicated that one glutaraldehyde-
basedproduct passedboth the standard AOAC method and the quantita-
tive method but at significantly different contact times. Under the stan-
dard AOAC method, the product was tuberculocidal (i.e., killed 100
percent of the test bacteria) in 10 minutes at 2OC, but under the quanti-
tative method, the product was tuberculocidal in 65 minutes at 2OC.
Despite the disparity in test results, EPAacceptedthe label claim for the
product on the basis of the standard AOACtest becauseit had allowed
registrants the option of choosingwhich test to use. EPAconcluded that
There is no technical information available on which to base a scientific judgment of
the reason(s) for the significant differences in required contact time for
tuberculocidal effectiveness. It is unknown whether the difference is: an unintended
confirmation of the questionable efficacy test results alleged to occur when
glutaraldehydes are tested by the standard AOAC method; attributable to some defi-
ciency in the quantitative method; or attributable to other testing or product related
factors.

EPA has acknowledged that  controlled laboratory studies are neededto
comparatively evaluate the three test options. EPA has also acknowl-
edgedthat the quantitative test needsto be collaboratively evaluated.


Page 27                         GAO/RCED-99-139 EPA Lacks Assurance Disinf’ectants Work
                        chapter 2
                        EPALacksAssuranceThat EfficacyTest
                        MethodsAre Valid




                        Despite these limitations, EPAcontinues to allow registrants to choose
                        from among the three options to support tuberculocidal claims.
                        Although 7 years have passedsince the quantitative method was first
                        submitted to EPA,the agency still lacks definitive, independent labora-
                        tory data showing that the AOACTuberculocidal Activity Method is
                        invalid or that the quantitative suspensionmethod is valid. In addition,
                        the quantitative suspensiontest has yet to be collaboratively tested and
                        approved by the A~ACbecauseof statistical design problems and lack of
                        laboratory participation in a collaborative study. EPAhas beentrying to
                        comparatively evaluate the AOACTuberculocidal Activity Method and
                        the quantitative suspensiontest at its laboratory facilities in Beltsville,
                        Maryland, but the lab has had problems with equipment and with
                        growing the test organism. As part of a 1990 research initiative, dis-
                        cussedbelow, EPAplans to fund research to assessthe validity of the
                        AOACTuberculocidal Activity Method.


Other Test Method       The tuberculocide example is not unique. EPAhas acceptedother disin-
Problems                fectant efficacy methods and significant modifications to methods that
                        have not undergone collaborative studies to validate test procedures.
                        For example, EPAhas allowed registrants to modify test methods by
                        adding hard water and organic matter to simulate actual use without
                        knowing how these additions affect the validity of the methods used.
                        EPAhas also acceptedsimilar    efficacy claims for different disinfectants
                        on the basis of multiple methods without laboratory data assessing
                        whether the methods yield comparable results. For example, EPAallows
                        registrants to use either a modified form of the AOACUse-Dilution
                        Method or the AOACFungicidal Test to make fungicidal claims. However,
                        in one case,the State of Florida tested and failed a product using the
                        AoACFungicidal Test but could not take any enforcement action because
                        EPAhad registered the product on the basis of passing test results the
                        registrant had submitted using the AOACUse-Dilution Method.

                        Although EPAhas known about the scientific controversies surrounding
Limited Progress Made   its recommendeddisinfectant efficacy test methods and standards for
in Resolving            most of the last decade,it has not made much progress in resolving the
Controversies           controversies, Between 1983 and 1989, EPAfunded limited research to
                        improve disinfectant efficacy methods. However, the research did not




                        Page28                      GAO/RCED-99-139
                                                                 EPALacksAssuranceDisinfectantsWork
                      Chapter 2
                      EPA Lacks Atwurance That Efficacy Test
                      Methods Am Valid




                      fulfill EPA'Sobjectives becauseEPAinadequately managedit. Further-
                      more, although EPAdeveloped a strategy in 1987 to improve the disinfec-
                      tants program that included, as its most important element, proposed
                      research on test methods, EPAhas made limited progress in completing
                      the research becauseof budgetary constraints. A new research initiative
                      is a step in the right direction, but more work remains to be done.


ResearchNot Managed   Between October 1983 and October 1989, EPAspent about $384,000 on
Well                  two consecutivecooperative agreementswith the University of North
                      Carolina at Chapel Hill (UNC) to update and improve the AOACUse-Dilu-
                      tion Method, Tuberculocidal Activity Method, and Sporicidal Test.4UNC
                      researchersinvestigated 19 presumed deficiencies in the AOACUse-Dilu-
                      tion Method and conducted two collaborative studies to evaluate vari-
                      abilities in that method and in a slightly modified version of it. In
                      addition, they conducted preliminary investigations on an alternative
                      quantitative suspensionmethod, conducted preliminary investigations
                      on the AOACTuberculocidal Activity Method, and conducted preliminary
                      evaluations of tuberculocidal claims of certain products, as well as other
                      investigations. On the basis of the research conducted, UNCresearchers
                      concluded that the AQACUse-Dilution Method and the slightly modified
                      version were subject to extreme inter-laboratory variability, mostly due
                      to the carrier, and should not be used for registration or enforcement
                      purposes.The researchersrecommendedthat an alternative method,
                      such as a quantitative suspensiontest, replace the AOACUse-Dilution
                      Method.
                      EPAbelieves that   the UNCresearchersdid not fulfill the primary intent of
                      the cooperative agreementsbecausethey focused on examining deficien-
                      cies in the AOACUse-Dilution Method rather than correcting the deficien-
                      cies to reduce variability and establish new pass/fail criteria. EPA
                      officials strongly criticized UNC'Sresearch conclusionsbecauseof a dis-
                      pute on the relative merits of carrier- versus suspension-baseddisinfec-
                      tant efficacy test methods. These officials do not believe that the UNC
                      research results definitively show that the A~AC Use-Dilution Method is
                      valid or invalid becauseof methodological limitations in the research.
                      They also believe that the research results might reflect deficiencies in
                      registered products rather than irreparable problems with the method.

                      4The total value of the cooperative agreements was $404,340, with EPA contributing $383,709 and
                      UNC contributing $20,631. On February 28, 1990, EPA extended the project period of the second
                      agreement to May 23, 1990, at no cost to the federal government to allow UNC researchers time to
                      complete their final report.



                      Page 29                            GAO/RCED-99-139 EPA Lacks Assurance Disinfectants Work
Chapter 2
EPALacksAmmnceThatEfflcacyTeet
MethodeAreValid




In fact, EPAofficials believe that the UNC research showed that the pri-
mary sourceof the alleged variability in the AOACUse-Dilution Method-
the stainless steel carrier-can be mitigated by using glass carriers and
has endorsedthe efforts of the ad hoc industry/state group to this end.
Disappointed with the UNCresearch, EPAreduced the amount of funding
under the cooperative agreementduring the last 2 years and delayed
funding while deciding whether to continue the agreement.
In their own defense,the UNCresearcherspointed out that EPAofficials,
especially those in the Antimicrobial Program Branch, were not highly
critical of the research until 198’7,when the AOACbeganto consider
repealing the AOACUse-Dilution Method on the basis of the UNCresearch
results. According to the UNCresearchers,EPA'Scriticisms of the
research may be the reaction of someoneshooting the messengerof bad
news. The researchersclaimed that EPAofficials may be unwilling to
accept the prospect that the most widely used disinfectant method, upon
which thousands of product claims to protect the public health are
based,may be invalid. The researchersfurther assertedthat the cost
and administrative burden of reregistering thousands of product claims
on the basis of a new method may be prohibitive in the eyes of EPAoffi-
cials who managea program already consideredto be of lower priority
than that of other pesticide programs.
EPA'Sinadequate managementof       the UNCcooperative agreementsmay
have contributed to the controversy over the research.Under EPAregu-
lations, a cooperative agreementis a form of financial assistancein
which EPAexpects to be substantially involved in the project funded.
EPA'Sproject officer for the UNCcooperative agreements(the EPApro-
gram official designatedto manageand monitor the project), also a key
AOACofficial in disinfectant method development, did not appear to be
substantially involved in managing and monitoring the direction and pri-
ority of research at UNCto fulfill EPA'Sresearch objective. Under the
terms of the cooperative agreements,the project officer, in consultation
with the UNCresearchers,was responsible for determining the nature
and scopeof tasks performed under the agreements.
The records of the project, grant, and quality assuranceofficers show
that the EPAproject officer allowed the UNC researchersto develop the
work plans for the cooperative agreements.However, the project officer
and other EPAofficials later criticized UNC'Sresearch direction and the
scopeof work and tasks UNCperformed. For example, the project officer
criticized the methodology of UNC'S secondcollaborative study on a mod-
ified version of the AOACUse-Dilution Method after UNCcompleted the


Page 30
                      Chapter
                           2
                      EPA Lacks Aeeurance That Efficacy Test
                      Methods Arc Valid




                      work. In addition, according to the project officer, he did not insist on
                      the quarterly progressreports the secondcooperative agreementcalled
                      for from UNCbecausehe did not think that sufficient funds were avail-
                      able under the cooperative agreementfor UNCto complete these reports.
                      The dispute between EPAand UNCover the research may have been com-
                      pounded by a lack of communication and coordination within EPAand a
                      lack of top managementinvolvement in the early years of the coopera-
                      tive agreements.The EPAproject officer for the cooperative agreements
                      worked in the Biological and Economic Analysis Division. This division
                      is separate from the Registration Division, which contains the Antimi-
                      crobial Program Branch. Although the UNCresearch was intended to
                      support the Antimicrobial Program Branch, which registers disinfec-
                      tants, it appears that the two divisions did not communicate on the
                      scopeof work and results of the research,since the Antimicrobial Pro-
                      gram Branch did not comment on the cooperative agreementresearch
                      until 1987.
                      EPAand UNChave also disputed other issuesunder the cooperative
                      agreements.EPAhas criticized UNC'Sresearchersfor investigating the
                      claimed tuberculocidal efficacy of individual products rather than inves-
                      tigating the validity of the AOACTuberculocidal Activity Method. EPAhas
                      also criticized the UNCresearchersfor publicizing the results of their
                      research on tuberculocidal disinfectants before sharing them with EPAas
                      required in the secondcooperative agreement.In both situations, how-
                      ever, it appears that EPAofficials did not closely manageand monitor
                      the cooperative agreement and expresseddissatisfaction with the
                      researchersand research results only after the work was completed and
                      publicized.
                      We do not know whether EPA'Scriticisms of UNC’Sresearch are valid,
                      reflect reasonabledifferences in scientific judgment, or reflect a defen-
                      sive position and lack of acceptanceof the research results. However, if
                      EPAis ever to resolve the scientific controversies that surround disinfec-
                      tant efficacy test methods and performance standards, it must do a
                      better job of managing the research.


Limited Progress in   In responseto mounting criticisms from industry, the public-health com-
Conducting Planned    munity, disinfectant users, the Congress,and others, EPAdeveloped a
                      strategy to improve its regulation of disinfectants in 1987, which pro-
Research              posed research on the methods and standards EPArecommends.How-
                      ever, EPAhas made limited progress in conducting this research because


                      Page 31                         GAO/RCED-90-139 EPA Lacks Assurance Disinfectants Work
Chapter 2
EPA Lacks Aasurance That Efficacy Test
Methods Are Valid




of budget constraints. A recent EPAresearch initiative is a step in the
right direction, but it will be several years before EPAfully addressesthe
controversies surrounding disinfectant efficacy methods and standards.
In addition, the regulation of disinfectant efficacy may be hampered by
the lack of an adequate, federal laboratory facility to conduct disinfec-
tant efficacy method research and development.
In 1986, EPAconveneda multi-office work group to analyze alleged
problems with disinfectant efficacy test methods and also hired a micro-
biologist consultant to provide expertise on the issues.The work group
identified deficiencies in several critical areas of the disinfectants pro-
gram, including those areas embroiled in the scientific controversies dis-
cussedabove, and explored several approachesfor resolving the
deficiencies that would not depend on a large-scale,federally operated
testing program. The work group concludedthat
The most critical and most recognized deficiency is the current lack of credibility of
the standard efficacy test methods utilized in the registration program, particularly
the AOAC test procedures. Any attempt to improve the [disinfectant] efficacy pro-
gram is contingent upon re-establishing the credibility of the existing methods;
updating/revising the existing methods; or developing new methods.

To resolve the deficiencies, the work group developed a five-point
strategy that included improving or replacing existing efficacy methods.
The work group estimated that EPA would need about $1.6 million in
contract funds to evaluate the methods.

Between 1987 and 1990, EPA made limited progress in conducting its
planned research becauseof competing program priorities and budget
constraints, according to the Chief, Antimicrobial Program Branch.
According to a June 1990 strategy update, “Becauseof budget con-
straints, EPA’S past efforts in conducting evaluations of current test
methods have been limited.”
Recognizingits resource limitations, EPA challenged industry, user
groups, and others to coordinate resourcesto achieve mutual research
goals. In responseto this challenge,the aforementioned ad hoc industry/
state group, funded by individual participating laboratories, conducted
a collaborative study of a new hard-surface carrier test in 1989. EPA offi-
cials believe that the study results show that the test, which they con-
sider to be a modified version of the AOAC Use-Dilution Method, is
reproducible and reliable. EPA officials expect that the ad hoc group will
present its final study report to the AOAC in September 1990 and that the


Page 32                         GAO/RCED-90-139 EPA Lacks Assurance Disinfectanta Work
Chapter 2
EPA Lacka Asmmnce That EfYicacy Test
Methods Are Valid




AOAC will consider replacing the existing AOACUse-Dilution Method with
the new method. EPAwill consider what regulatory actions, if any, to
take on the basis of the new test after it is consideredby the AOACand
after additional research is completed on the effects of using hard water
and organic matter in the method and on the appropriate performance
standard for the method, according to the Chief, Antimicrobial Program
Branch.
In April 1990, EPAannouncedthat it plans to support three cooperative
agreementsover a 2-year project period to conduct research and devel-
opment on validating, revising, or replacing the AoACTuberculocidal
Activity, AOACSporicidal, and EPAvirucidal test methods and perform-
ance standards. EPAplans to spend approximately $600,000 in fiscal
year 1990 funds on this research.EPAplans to award the cooperative
agreementsby September30, 1990.
Although EPA'Sfunding initiative represents a substantial increase in
commitment to resolve the scientific controversies,the Chief of the Anti-
microbial Program Branch believes that the initial funding will be insuf-
ficient to resolve the controversies completely. She estimates that EPA
may need to spend an additional $1.2 million to completely research the
alleged problems with the methods and standards.

If the new research initiative is not well managed,EPA,industry, disin-
fectant users, consumers,and the public health will be no better off than
they are today. EPAhas designatedthe Antimicrobial Program Branch,
within the Registration Division, as the project office for the new coop-
erative agreementsinstead of the Biological and Economic Effects Divi-
sion, which was the project office for the UNCcooperative agreements.
Further, EPAhas announcedthat it “intends to be involved in the test
methodology research and development by approving work by stages,
approving any subcontracts, conducting on-site visits/inspections at rea-
sonable intervals, co-authoring published reports relative to the funded
study, and halting research activity if the intent, approach, or antici-
pated phasesleading to the accomplishment of the study are not being
achieved, or have been revised without prior Agency approval.”
Although this approach may resolve someof the managementproblems
EPAexperienced with the UNC cooperative agreements,EPAmight want to
request that the FIFRAScientific Advisory Panel provide advice on the
research direction and results. EPA'S1987 strategy paper called for the
panel’s assistanceto help evaluate research priorities, among other
things, but as of June 1990, EPAhad not convenedthe panel for this


Page 33                       GAO/RCED-99-139 EPA Lacks Assurance Disinfectants Work
              Chapter 2
              EPA Lacks Aseurance That Eff¶cacy Test
              Methods Are Valid




              purpose. In addition to guidance on research needsand priorities, the
              FIFRAScientific Advisory Panel could provide independent confirmation
              of the need for any regulatory changesthat might be indicated by the
              research,such as the need to reregister disinfectants on the basis of new
              methods and standards. An EPAdecision to involve the FIFRAScientific
              Advisory Panel in this manner would be consistent with the internal
              control principle of separating key functions in a transaction to ensure
              that effective checks and balancesexist.
              Many of the disinfectant efficacy test methods in dispute were devel-
              oped by scientists working at the federal government’s laboratory facili-
              ties in Beltsville, Maryland. EPAdiscontinued testing disinfectants at
              Beltsville in 1982 primarily becauseof budget constraints. Although EPA
              kept the laboratory open to assist in a limited capacity on method evalu-
              ation, EPAofficials readily admit that EPAlacks the personnel and facili-
              ties to do the necessarymethodology research.According to the current
              laboratory supervisor, EPA’SBeltsville laboratory facilities would not
              pass EPA’Sown good laboratory practice (GLP) regulations. (Ch. 5 dis-
              cussesoptions for a disinfectant laboratory.)

              Disinfectants are used to reduce the risk of transmitting infectious dis-
Conclusions   eases.If disinfectants fail to work as claimed, then those using disinfec-
              tants in restaurants, child day care centers, hospitals, homes, and other
              places may be placing themselves and others at risk and wasting their
              money.
              Doctors,janitors, consumers,and others rely on EPA’Sregistration of dis-
              infectants as assurancethat these products work. However, scientific
              controversy exists over the validity of the methods and standards that
              EPArecommendsthat registrants follow in order to substantiate claims.
              Although EPAand state officials have acceptedallegations that there
              may be problems with the methods and standards, they believe that the
              existing methods and standards are acceptablefor registration and
              enforcement purposes. Nonetheless,the scientific controversies over the
              adequacy of these methods and standards have impaired the credibility
              of EPA’Sregistration of disinfectant efficacy claims.
              Although disinfectant efficacy test methods and performance standards
              have been embroiled in scientific controversies for over a decade,EPA
              has made limited progress in resolving them becauseof inadequate
              research managementand budget constraints. EPAdid not manageand
              monitor its cooperative agreementswith UNC well becauseit lacked


              Page 34                         GAO/RCED-90-139 EPA Lacks Assurance Disinfectants Work
                  Chapter 2
                  EPA J.mke Assurance That EfYicacy Teat
                  Methods Are Valid




                  internal communication and agreementon research tasks and results.
                  EPAalso has made limited progress in conducting additional research
                  becauseof budget constraints. In April 1990, EPAannouncedthat it
                  planned to spend approximately $600,000 for research on the scientific
                  controversies surrounding certain disinfectant efficacy methods over a
                  Z-year period. Although this initiative is significant, more research,
                  requiring time and additional resources,will be needed.
                  Although we do not expect EPAto ever eliminate all scientific disputes
                  over these methods and standards, the agency could better protect the
                  public health by raising the degreeof certainty about the validity of dis-
                  infectant efficacy methods and standards. A detailed plan that would
                  describe a research strategy to resolve the controversies surrounding
                  the existing disinfectant efficacy test methods and performance stan-
                  dards would help guide this effort by establishing milestones and cost
                  estimates. In addition, the FIFF~AScientific Advisory Panel could assist in
                  developing the plan and overseeingEPA'Sresearch direction and
                  management.

                  EPA'Sprocessfor   accepting and recommending disinfectant efficacy test
                  methods and standards has contributed to the scientific controversies
                  surrounding methods and standards. EPAhas acceptedmethods and
                  modifications to methods without independent laboratory data that
                  demonstrate the validity of the procedures or standards. Furthermore,
                  EPAlacks criteria for assessingthe validity of new methods and any sig-
                  nificant modifications to methods. EPAcould better ensure that future
                  registrant-proposed disinfectant efficacy methods and modifications are
                  valid by establishing specific criteria for ascertaining validity, including
                  criteria for determining when independent laboratory data, such as col-
                  laborative study data, are neededto assessthe validity of proposed
                  methods or modifications.

                  To increasethe degreeof certainty that disinfectant efficacy test
Recommendations   methods and standards are valid, we recommendthat the Adminis-
                  trator, EPA,develop a detailed plan, including cost estimates and mile-
                  stones,to resolve the controversies surrounding existing methods and
                  standards. The plan should include a research strategy that addresses
                  problems with the alleged variability in test methods, adequacy of lab
                  tests to simulate actual use, and the validity of performance standards,
                  as discussedin this chapter. Further, we recommendthat the Adminis-
                  trator, EPA,convenethe FIFRAScientific Advisory Panel to assist in
                  developing the plan and overseeingthe research strategy direction and


                  Page 35                         GAO/RCED-99-139 EPA Lacks Assurance Diinfectmts   Work
Chapter 2
EPA Lacka Assurance That Efficmy Test
Methods Are Valid




management.(Seech. 6 for options on establishing a laboratory facility
to assist in researching and developing disinfectant efficacy test
methods.)

In addition, we recommendthat the Administrator, EPA, develop and
publish a policy that establishesspecific criteria for evaluating the
validity of new disinfectant efficacy test methods and modifications to
methods, including criteria for determining when independent labora-
tory data, such as data from a collaborative study, are neededto demon-
strate the validity of proposed methods and modifications.




Page 36                        GAO/RCED-99-139 EPA Lacka Assurance Dbinfectants   Work
Chapter 3

Controls Over Quality/Integrity of Efficacy
Data Questionable

                         Even with improvements in its programs to ensure the validity of the
                         test methods used to support the efficacy of disinfectants, EPAwould
                         continue to lack assurancethat disinfectants work as claimed. EPArelies
                         on registrant-submitted efficacy data to make decisionsabout whether
                         to register individual disinfectants for specific claims and uses,and on
                         its data review, lab inspection, and data audit programs to ensure the
                         quality and integrity of the data. However, we found weaknessesin
                         these programs that EPAneedsto address.We also found that a preregis-
                         tration-testing program is neededto supplement these programs because
                         they generally do not enable EPAto identify casesin which registrants
                         have selectively submitted incomplete disinfectant efficacy data or have
                         deliberately submitted invalid data. We believe EPA'Slack of sufficient
                         control over the quality and integrity of registrant-submitted disinfec-
                         tant efficacy data is a material weaknessin EPA'Sinternal controls that
                         should be, but has not been, reported to the President, as required by
                         the Federal Managers’ Financial Integrity Act of 1982 (FMFIA).

                         EPAreviews registrant-submitted disinfectant efficacy data to determine
WeaknessesImpair         whether proposed label claims are supported by the data and whether
Data and Lab Reviews     registrants have made any major mistakes in performing efficacy tests,
                         interpreting test results, or translating the results into label claims,
                         according to the Head, Efficacy Evaluation and Technical Management
                         Section, Antimicrobial Program Branch. In addition, EPAinspects the
                         labs that generatethe data and audits laboratory study records to assess
                         the quality and integrity of the data and the competenceof the laborato-
                         ries that have performed the studies. However, we found that: (1) EPA
                         has inspected/audited only about 10 percent of all of the labs that per-
                         formed disinfectant efficacy studies submitted to EPAover about a 4-
                         year period and has identified only about 13 percent of all the labs that
                         performed these studies; (2) EPAinspectors and auditors may be unable
                         to evaluate adequately the capability of labs to perform these studies;
                         (3) EPAhas been slow to prepare and processreports from inspections
                         and audits at labs performing these studies; and (4) EPAlacks program
                         guidance for conducting data reviews, lab inspections, and data audits
                         relating to these studies.


Not All Labs Inspected   The Laboratory Data Integrity Assurance Division (LDIAD), within the
                         Office of Compliance Monitoring (OCM), is responsible for, among other
            Y            things, inspecting laboratories that perform studies submitted to EPAto
                         support pesticide registrations and for auditing these studies. The objec-
                         tives of good laboratory practice (GLP) inspections are to ensure that a


                         Page 37                   GAO/RCED-99-139 EPA Lacks Aeeurance Didnfectants   Work
                      Chapter 3
                      Controls Over Quality/Integrity    of EMcacy
                      Data Questionable




                      lab follows specific test methods, adheresto standard operating proce-
                      dures, keeps records in compliance with GLPregulations, and conforms
                      to required safety and quality assuranceprocedures so that lab prac-
                      tices do not compromise the quality and integrity of data submitted to
                      EPAfor registration purp0ses.l The objectives of data audits are to
                      ensure that the data underlying a study are both present at the lab and
                      fully substantiate the study results by comparing the data from the
                      study submitted to EPAwith records on the study in the lab. A data
                      auditor typically audits a study at the sametime that an inspector
                      inspects the lab that performed the study. LDIAD'Sgoals are to inspect all
                      labs performing efficacy studies approximately every 2 years, audit one
                      or more efficacy studies from each lab inspected, and inspect high-
                      volume labs first.

                      Between January 1,1986, and September30,1989, LDIAD conducted 13
                      lab inspections and 14 data audits at 9 individual labs that performed
                      disinfectant efficacy tests. These 9 labs represent only about 10 percent
                      of the 92 labs that had generated disinfectant efficacy data received by
                      EPAbetween January 1,1985, and June 26,1989. The 9 labs that LDIAD
                      inspected generated an estimated 40 percent of the 1,148 disinfectant
                      efficacy studies that EPAreceived during that period. LDIAD audited 109
                      studies at the 9 labs.

                      LDIAD did not inspect most of the labs that had performed disinfectant
                      efficacy studies. In fact, LDUD identified only 12 (about 13 percent) of
                      the 92 labs that had performed these studies. LDIAD was unaware of
                      most of the labs, including somehigh-volume labs, becauseit had not
                      used EPA’S Pesticide Document ManagementSystem (PDMS)to identify
                      labs for inspections/audits, This system is a central archive primarily
                      consisting of documents that registrants have submitted to EPAto sup-
                      port pesticide registrations, and we used it to identify labs that had per-
                      formed disinfectant efficacy studies submitted to EPA.


Lab Capability Not    Existing EPA program guidance specifies that lab inspections are per-
Adequately Assessed   formed while a study is in progress,that they provide the inspector with
                      an opportunity to observe laboratory techniques, and that they ensure
                      that labs follow specific test methods correctly. Various officials from


                       ‘In August 1989, EPA published GLP regulations governing efficacy studies. These regulations
                      became effective on October 16, 1989. Before this date, EPA could not enforce good laboratory prac-
                      tices of laboratories that performed disinfectant efficacy tests.



                      Page 38                             GAO/RCJZD-90-139 EPA Lacks Assurance lMdnfectants Work
Chapter 9
Controls Over QuaRty/Integrlty   of EfTleaey
Data Questionable




EPA,the states that operate disinfectant efficacy testing programs, com-
mercial labs, and industry, as well as the AOAC,have stated that the
results of a disinfectant efficacy test are very sensitive to small varia-
tions in the way the technician performs the test procedures and that
lab personnel must have extensive experienceto perform disinfectant
efficacy tests correctly. However, according to a June 1987 EPAstrategy
document, many of the labs testing disinfectants for efficacy do not
follow prescribed, standard efficacy test methods.

Although the tests are operator-sensitive, lab inspectors and data audi-
tors generally do not observe disinfectant efficacy tests in progress
becauseEPAhas no means of identifying most of the studies ultimately
submitted to EPAwhile they are still in progress, according to the Chief,
Scientific Support Branch, LDIAD. In addition, the branch chief told us
that disinfectant efficacy tests generally are so short in duration that it
is difficult for LDIADto schedule an inspection at a lab performing such a
test before the test is complete. OneEPAefficacy data auditor told us
that inspectors and auditors do not usually seelabs performing efficacy
tests becausemost labs do not run efficacy tests on a continuous basis.
The five EPAdata auditors who have audited disinfectant efficacy
studies, as well as state lab officials and researchers,told us that a
check sample program could ensure the quality of the data registrants
submit better than data audits can. EPAcurrently requires states that
have cooperative pesticide enforcement agreementswith EPAto partici-
pate in a check sample program. Under the program, EPAsendsthe state
labs samplesof pesticide formulations and residues for analysis to
ensure that they perform analytical tests correctly, EPAcould adopt a
similar program for laboratories that conduct disinfectant efficacy
studies to ensure that they are capable of performing efficacy tests cor-
rectly. Under the program, EPAwould send samplesof formulations of
known efficacy to a lab, ask the lab to test the samplesfor efficacy, then
check the lab’s results against the known efficacy of the samples.




Page 39                           GAO/RCED-99-139 EPA Lacks Assurance Disinfectants Work
                                Chapter 3
                                Controls Over QuaRty/Integrlty    of Efficacy
                                Data Cjueetionable




ProcessingInspection/           LDIAD’S target time frame for conducting inspections and audits and
Audit Reports Takes             processingreports is 112 days (about 3.6 months). However, LDIAD had,
                                as of November 13,1989, taken 639 days (about 1.6 years) on average
Nearly Five Times Longer        to perform and processreports from inspections/audits at labs per-
Than Estimated                  forming disinfectant efficacy studies.2The times ranged from 161 to 963
                                days (about 6 months to over 2.6 years). The 6 inspection/audit reports
                                that remained open as of November 13,1989, had been open an average
                                of 360 days. The longest report had been open over 1.6 years since the
                                lab was first inspected.
                                LDIADhas been slow in processingthese lab inspection and data audit
                                reports for three reasons,according to the Chief, Scientific Support
                                Branch. First, most lab inspectors and data auditors are not directly
                                accountableto LDIAD. Instead, most lab inspectors are stationed in EPA’S
                                regional offices and have other noninspection-related duties that the
                              1 regional offices consider of higher priority than the duty of preparing
                                lab inspection reports. Second,according to this official, until recently,
                                the EPA headquarters staff available to perform inspections and process
                                inspection/audit reports at EPA headquarters was too small. Third, EPA
                                did not have enforceable GLPregulations for efficacy studies until
                                October 16, 1989, so LDIAD consideredprocessinginspection/audit
                                reports relating to labs performing these studies to be a low priority.
                                The Director, LDIAD, acknowledged that EPA has been slow in processing
                                inspection/audit reports related to disinfectant efficacy studies as well
                                as other types of registration studies. To addressthe problem, LDIAD is
                                developing a simplified reporting format and a procedure for processing
                                those reports that indicate a violation first.


Program Guidance Lacking Scientists in the Antimicrobial Program Branch review registrant-sub-
                                mitted reports summarizing the results of efficacy tests before regis-
                                tering disinfectant efficacy claims. As of June 1990, EPA had not
                                completed final guidelines for these scientists to use in conducting effi-
                                cacy data reviews, although it had prepared guidelines for performing
                                reviews of most other types of pesticide registration data (e.g., chemical,
                                toxicological, environmental, and ecological data). With such guidelines,
                                EPA could better ensure that its reviewers identify all potential problems
                                with efficacy data. EPA could not estimate when final guidelines would
                                be published.

                                ‘This finding is based on the 9 of 10 inspection/audit reports EPA had prepared and completely
                                processed as of November 13, 1989, and for which EPA could provide information.




                                Page 40                             GAO/RCED-90.139 EPA Lacks Assurance Disinfectants Work
                        Ckapter 3
                        Cmtrob Over Qufdlty/Integrity        of Effhwy
                        Data Questionable




                        Similarly, as of June 1990, EPAhad not developedguidance for con-
                        ducting inspections and data audits at labs that have performed disin-
                        fectant efficacy studies. Instead, according to the Chief, Scientific
                        Support Branch, LDIAD, inspectors have beenusing EPA'Smanual gov-
                        erning inspections at labs that have performed health-effects studies.
                        According to this official, EPA'Sefficacy data auditors have relied on
                        their professional judgment to perform the audits, (Although two of the
                        auditors have developedquestionnaires for use in conducting efficacy
                        data audits, not all five use them.)

                        In addition, as of June 1990, EPAhad not published guidelines specifying
                        the types of inspection/audit findings that would prompt registration
                        and/or enforcement action by EPAand the type of action EPAshould take
                        in each case.Although the Registration Division has developed a stan-
                        dard operating procedure for managing the review and disposition of
                        inspection/audit reports, the procedure doesnot specify criteria for
                        evaluating what registration action, if any, the division should take on
                        the basis of report findings. An LDIAD work group has developed interim
                        guidelines on taking enforcement action that are undergoing internal
                        review.

                        Even if the weaknesseswe found in EPA'Sdata review, lab inspection,
Preregistration         and data audit programs were corrected, the programs generally would
Testing Neededto        not enable EPAto identify casesin which registrants have selectively
Ensure Data Integrity   submitted incomplete disinfectant efficacy data. EPA'Sposition is that
                        registrants are required by FIFFWto submit all data indicating that a dis-
                        infectant may not be effective as registered when registrants are aware
                        that such data exist.” Evidence exists that someregistrants have sub-
                        mitted to EPAefficacy test data indicating that their disinfectants work
                        but may have withheld other test data indicating that these disinfec-
                        tants do not work as claimed. Furthermore, if a registrant deliberately



                        “Section 6(a)(2) of FIFRA states:

                        “If at any time after the registration of a pesticide the registrant has additional factual information
                        regarding unreasonable adverse effects on the environment of the pesticide, he shall submit such
                        information to the Administrator.” 7 U.S.C. sec. 136d(a)(2).
                        As interpreted by EPA, “unreasonable adverse effects on the environment” include information con-
                        cerning the efficacy of disinfectants. Under an EPA regulation, 40 C.F.R. sec. 16260(f)(3), an appli-
                        cant for registration also must submit any information that would be required under sec. 6(a)(2) if
                        the product were registered.




                        Page 41                               GAO/RCED-99-139 EPA Lacks Assurance Disinfectants Work
Chapter 3
Controls Over Quality/Integrity   of Efficacy
Data Questionable




submitted invalid data to EPA,or a commercial lab deliberately sub-
mitted invalid data to a registrant, EPA'Sdata reviewers, lab inspectors,
and data auditors probably would be unable to tell.
Although EPAmaintains that FIFFZAprohibits registrants from submitting
selective data, EPA-recommended    methods for testing disinfectant effi-
cacy provide registrants with an opportunity to submit efficacy data
selectively. Given the alleged variability in certain disinfectant efficacy
tests (seech. Z), a registrant and/or testing facility could run an efficacy
test repeatedly until the formulation tested passed,and the registrant
could submit only the passing results to EPA(i.e., a registrant could
submit selective data to EPA).
Despite this opportunity, EPA'Sexisting programs for validating efficacy
data generally do not enable EPAto recognizeselective data. According
to the Head, Efficacy Evaluation and Technical ManagementSection,
Antimicrobial Program Branch, data reviewers cannot identify situa-
tions in which EPAhas received selective data becausethe reviewer sees
only what was submitted rather than all tests conducted on a disinfec-
tant. According to EPA’Sfive efficacy data auditors, data auditors gener-
ally cannot identify casesin which registration data are selective. The
auditors told us that they rely on labs to identify the records they
should audit, and labs could provide them with selective records.
During the course of our review, we found someevidencethat regis-
trants have submitted selective data to EPA.For example, one data
auditor found evidenceof selective data during a data audit-passing
and failing data on a high-volume, household product whose registration
file contained only passing data. According to the data auditor, he found
the set of failure data by chance.The data auditor who discovered the
set of failure data and other efficacy data auditors told us that they
would have no way of knowing if a registrant withheld data from an
audit. In addition, representatives of two registrants told us that they
had obtained variable (both pass and fail) efficacy results on disinfec-
tants but had submitted only passing results.

The belief that registrants submit selective data to EPAis widespread.
The Chief, EPAAntimicrobial Program Branch, and the Head of the
branch’s Efficacy Evaluation and Technical ManagementSection told us
they believe that registrants submit selective data, and EPA'SJune 1990
disinfectant program strategy paper stated that “the practice of not
reporting failing/adverse test results is widespread.” In addition, other
EPAofficials, state officials, the UNCresearchers,and others all told us


Page 42                            GAO/RCED-99-139 EPA Lacks Assurance Disinfectants Work
Chapter 3
Controla Over Quality/Integrity   of Efficacy
Data Questionable




that they believe someregistrants submit selective data. In fact, mem-
bers of the disinfectant industry openly joked about submitting selective
data to EPAat a widely attended national meeting on disinfectant effi-
cacy test methods that we and EPAofficials attended in 1989. As noted
in chapter 2, registrants have argued that the existing test methods
yield variable results and the performance standards are too stringent.
However, these criticisms, and the fact that EPArarely receivesdata
showing that disinfectants do not work, suggestthe possibility that
someregistrants could have submitted selective data to EPAto register
their products.
Even if the variabilities in the test methods were reduced to limit a reg-
istrant’s opportunity to submit selective data, neither lab inspectors nor
data auditors can practically observethese tests in progress, and no
physical evidence(e.g., test tubes) remains from the tests once they are
completed. As a result, data reviewers and auditors must rely on the
registrant’s word about the procedures followed in a test, the disinfec-
tant formulation tested, and the test results, according to EPAofficials.
In the event that a registrant or lab facility were deliberately to deviate
from an EPA-approvedefficacy test method (e.g., by running the test at a
higher temperature than reported); run a test with a disinfectant formu-
lation that would pass the test, rather than the formulation the regis-
trant planned to market; or record that a disinfectant formulation
passedan efficacy test when the formulation failed, data reviewers or
auditors probably would not be in a position to identify the deviation
and question the test results.

Although a program to test disinfectants before registering them could
resolve these issues,EPAdoesnot currently operate such a program. At
one time, EPAoperated a limited preregistration-testing program to
verify sporicidal claims and selectedtuberculocidal claims. EPAdiscon-
tinued preregistration tuberculocidal testing in 1979 becauseof inconsis-
tent test results and discontinued all other preregistration tests in 1982.
EPA’Srecords from the testing program show that at least someregis-
trants changedproposed efficacy claims on the basis of EPA’Stests to
make them more protective (e.g., lengthened contact times). For
example, one registrant lengthened its proposed exposure period for the
sporicidal efficacy of a disinfectant from 6 hours to 10 hours in 1974
after the disinfectant failed a preregistration test. EPAwould not need to
test all disinfectants for efficacy before registering them. Instead, EPA
could target such a program to those disinfectant claims of greatest
importance to public health, such as sporicidal claims, to those products



Page 43                            GAO/RCED-90-139 EPA Lacks Assurance Disinfectants Work
                    Ckapter 3
                    Controla Over Quality/Integrity   of EfYlcacy
                    Data Queetionable




                    with suspectedefficacy problems, and/or to other products as deter-
                    mined by the Administrator.

                    EPA'Sinability  to ensure the quality and integrity of registrant-submitted
Internal Controls   disinfectant efficacy data is, in our opinion, a material weaknessin
Inadequate          internal controls under FMFIA. Under FMFIA, each federal agency must
                    establish and maintain a system of internal controls to provide reason-
                    able assurancethat, among other things, program and administrative
                    activities are effectively managedto achieve the goals of the agency.

                    GAOand the Office of    Managementand Budget guidance on internal con-
                    trols require agenciesto identify, in writing, objectives for each agency
                    activity that are specific, complete, logical, and applicable to the specific
                    activity; as well as techniques that will provide reasonableassurance
                    that the objectives will be accomplished.In addition, federal agencies
                    are required to identify, in an annual report to the President, material
                    weaknessesin internal controls. A material weaknessexists in an
                    agency’sinternal control systems when, among other things, the agency
                    lacks reasonableassurancethat the objectives of the system are being
                    accomplishedand that the weaknesswould significantly impair the ful-
                    fillment of the agency’smission and/or would deprive the public of a
                    neededservice.

                    EPArelies on registrant-submitted   data to support disinfectant efficacy
Conclusions         claims. As noted in chapter 2, we found that the validity of disinfectant
                    efficacy methods and performance standards has been questioned and
                    that EPA'Sprocessfor accepting test methods is inadequate. However,
                    even if EPAaddressedthese problems, we believe that EPAstill would
                    lack sufficient controls to ensure the quality and integrity of registrant-
                    submitted disinfectant efficacy data.
                    We found a number of weaknessesin the data review, lab inspection,
                    and data audit programs. First, EPAhas not identified all labs that have
                    performed disinfectant efficacy studies to be inspected/audited. EPA
                    should use the Pesticide Document ManagementSystem which, though
                    limited, contains the best available information for identifying these
                    labs. Second,EPA'Slab inspection and data audit programs are unable to
                    completely assessthe capabilities of labs that perform disinfectant effi-
                    cacy tests becausethese tests are operator-sensitive and inspectors/
                    auditors generally do not observethem in progress.A check sample pro-
                    gram could provide EPAwith greater assurancethat laboratories that


                    Page 44                            GAO/RCED-90-139 EPA Lacks Assurance DMnhctant.s   Work
                  Chapter 3
                  ~mol# Over Cluallty/Integrity   of Efficacy
                  Data Que&hable




                  perform these tests for registrants are capable of performing them cor-
                  rectly. Third, EPA has been slow to prepare and processreports from
                  inspections/audits related to disinfectant efficacy studies. The timeli-
                  nessof these reports is more important now that EPA'SGw regulations
                  encompassdisinfectant efficacy studies. A review of OCM'Sinternal con-
                  trols for ensuring that inspections/audits are processedon time could
                  prevent problems with timeliness in the future. Fourth, EPAhas not pub-
                  lished guidelines neededto ensure that data reviewers, lab inspectors,
                  and data auditors identify all potential problems with disinfectant effi-
                  cacy studies and that EPAtakes appropriate registration and/or enforce-
                  ment action.
                  Although EPAcan improve its data review, lab inspection, and data audit
                  programs, we believe that the only way for EPAto determine whether a
                  registrant has submitted selective data or has deliberately submitted
                  invalid data is by testing the product. Evidence exists to suggestthat, in
                  at least somecases,pre-registration government tests to verify
                  sporicidal efficacy data led to more protective label claims. We are not
                  suggestingthat EPAtest all disinfectants for efficacy before registering
                  them. Instead, EPAcould target such a program to those disinfectant
                  claims of greatest importance to public health and/or products with sus-
                  pected efficacy problems. Data reviews, lab inspections, and data audits
                  cannot ensure the integrity of registrant-submitted data. A preregistra-
                  tion-testing program would complement those activities and better
                  ensure the integrity of the data.
                  We believe that EPA'Slack of sufficient controls to ensure the quality and
                  integrity of registrant-submitted disinfectant efficacy studies is a mate-
                  rial weaknessin EPA'Sinternal controls that should be, but has not been,
                  reported to the President as required by FMFIA. Until this weaknessis
                  corrected, EPAmay be compromising public health by registering ineffec-
                  tive products.

                  To improve EPAcontrols over the quality and integrity of registrant-sub-
Recommendations   mitted data, we recommendthat the Administrator, EPA,implement a
                  preregistration-testing program to verify selecteddisinfectant efficacy
                  data. (Seech. 6 for a discussionof options for establishing a laboratory
                  facility to assist in such a program.) The Administrator could target pre-
                  registration tests on those claims that are of the greatest public health
                  significance and/or products with suspectedefficacy problems.




                  Page 46                           GAO/lUXtMM%139 EPA La&t Assurance Disinfectanta Work
    Chapter 3
    Controls Over Quality/Integrity   of Efficacy
    Data Questionable




    To improve the effectiveness of the data review, lab inspection, and
    data audit programs, we recommendthat the Administrator, EPA,
l   direct the Laboratory Data Integrity Assurance Division to identify all
    laboratories that have performed efficacy studies submitted to EPAto
    support disinfectant registrations and meet the division’s goal of
    inspecting these labs at least every 2 years (at a minimum, direct LDIAD
    to use the Office of Pesticide Programs Pesticide Document Management
    System, which contains the best available information for identifying
    the labs);
l   direct LDIADto establish a check sample program as part of the lab
    inspection program to better assessthe ability of labs to perform disin-
    fectant efficacy tests;
l   direct the Office of Compliance Monitoring to review its internal con-
    trols for ensuring that inspections/audits are processedon time (for
    example, ensure that inspectors/auditors are held accountablein their
    performance standards and appraisals for meeting processingtime
    frames); and
l   direct the Office of Pesticide Programs and the Office of Compliance
    Monitoring to develop and implement specific guidance for data
    reviewers, lab inspectors, and data auditors to follow; further, direct
    these offices to develop, publish for comment, and implement detailed
    policies and guidelines to decide what registration and/or enforcement
    action to take on the basis of findings from lab inspections and data
    audits.
    We recommendthat in his next annual internal control report to the
    President, the Administrator, EPA,report the lack of sufficient controls
    to ensure the quality and integrity of registrant-submitted disinfectant
    efficacy data as a material weakness.We also recommendthat the
    Administrator, EPA,include in his report a plan delineating specific cor-
    rective actions and time frames.




    Page 46                            GAO/RCED-99-139 EPA Lacks Assurance Disinfectanta Work
Limited Monitoring/E                                                          of Registered
Disidectmts

                       Even were EPAto implement improvements in its processesfor regis-
                       tering efficacy claims (e.g., independent validation of test methods and
                       preregistration testing), these improvements would not provide suffi-
                       cient assurancethat disinfectants on the market were effective. Regis-
                       trants could, intentionally or inadvertently, manufacture and sell
                       ineffective batches of disinfectants after registering them. EPAlacks
                       assurancethat, once registered, disinfectants work as claimed because
                       EPAstopped monitoring disinfectants on the market for efficacy in 1982.
                       Becausestates and disinfectant users generally do not monitor disinfec-
                       tants for efficacy, EPAis relying, in effect, on the industry to regulate
                       itself. Despite the scientific controversies over efficacy test results, his-
                       torical enforcement and other data suggestthat as many as 20 percent
                       of the disinfectant batches on the market do not work as claimed. As a
                       result, disinfectant users may be placing themselves and others at risk
                       from infection and spending money unnecessarily.

                       Until 1982, EPAoperated a limited post-registration testing program to
Federal Enforcement    verify certain efficacy claims of marketed disinfectants at its laboratory
Testing Discontinued   facilities in Beltsville, Maryland. According to the best available infor-
                       mation, the program was limited in size, scope,and operation. EPAdis-
                       continued the program becauseof budget constraints and expected that
                       states, users, and the disinfectants industry would assumemost of its
                       responsibility for ensuring the efficacy of marketed disinfectants.’

                       From 1970, after EPAwas established and took over the pesticide pro-
                       grams from the U.S. Department of Agriculture, through 1982, the Belts-
                       ville laboratory tested, on average,207 disinfectant samples annually.
                       However, the number of samplestested each year dropped substantially
                       after 1972. From 1970 through 1972, the laboratory tested an average
                       of 610 samples annually. By contrast, from 1973 through 1982, the labo-
                       ratory tested an averageof 127 samples annually, testing only 47 sam-
                       ples in 1982. Furthermore, the Beltsville laboratory did not verify all
                       types of efficacy claims. According to the microbiologist at the labora-
                       tory, the laboratory tested no products from the marketplace for
                       sporicidal claims and, after the mid-19709, few products for use on sur-
                       faces other than hard surfaces.


                       ‘Although EPA discontinued post-registration testing, it kept the Beltsville laboratory open for use on
                       an as-needed basis (e.g., to perform confirmatory efficacy tests on disinfectant enforcement samples
                       from the states).




                       Page 47                              GAO/RCED-99-139 EPA Lacks Assurance Dbhfectan~              Work
                        Chapter 4
                        Limtted Monitoring/Enforcement   of
                        Registered Msinfectanta




                        According to EPAcongressionaltestimony, EPAdiscontinued its post-reg-
                        istration testing program primarily because(1) the level of testing was
                        inadequate and was creating a false senseof security among users and
                        the public about the efficacy of disinfectants on the market and (2)
                        budget constraints prevented EPAfrom conducting what it consideredto
                        be an adequate level of testing. EPAofficials believed that by discontin-
                        uing the testing, EPAcould direct more time and effort to evaluating and
                        improving the test methods. EPAofficials believed that the states, the
                        user community, and the disinfectants industry would take a more
                        active role in monitoring the efficacy of disinfectants on the market and
                        that EPAcould establish a system for processingcomplaints from these
                        sourcesabout actual or suspectedproduct failures.

                        However, EPAhad little reason to expect that states and disinfectant
                        users would assumeEPA’Sresponsibility for monitoring disinfectant effi-
                        cacy or to assumethat a complaint system would be developedbecause
                        of a lack of EPAleadership. Few states and no users were testing at the
                        time, and no others expressedan interest in testing. In addition, without
                        a laboratory facility or provision to contract for laboratory services,EPA
                        had no mechanism for channelling and verifying complaints about
                        potentially ineffective disinfectants from the states, user groups, and
                        the industry and for initiating appropriate enforcement action against
                        disinfectants found to be ineffective.


                        Although EPAdiscontinued its efficacy-monitoring program in 1982,
Nonfederal Testing      nonfederal monitoring efforts remain limited. As of July 1989, we were
Limited                 able to identify only three states- Florida, North Carolina, and Missis-
                        sippi-that were monitoring the efficacy of disinfectants in the market-
                        place under their pesticide enforcement programs and only two states-
                        Wisconsin and Virginia- that were testing disinfectants under other
                        state programs. Furthermore, we did not identify any users or related
                        groups, including the health-care, restaurant, or food-processingindus-
                        tries, that have implemented comprehensive,routine monitoring pro-
                        grams. In effect, EPAhas left the burden of monitoring disinfectant
                        efficacy to the industry itself.


State Testing Limited   Although Florida, North Carolina, and Mississippi monitor the efficacy
                        of disinfectants on the market, the scopeof their testing programs and
             ii         authority of their enforcement programs are limited, and the amount of
                        testing they have performed has decreasedin recent years. In addition,



                        Page 48                          GAO/RCED-99-139 EPA Lacks Assurance Disinfectants Work
Chapter 4
Limited Monitdng/Enforcement   of
Registered Dhilnfectan~




scientific controversies surrounding the test methods have affected
their efforts to enforce the efficacy of disinfectants on the market.
The types and amounts of enforcement testing the states perform is lim-
ited. Florida, North Carolina, and Mississippi do not verify all types of
disinfectant efficacy claims. For example, none tests products for
sporicidal, tuberculocidal, or virucidal efficacy. According to officials in
these states, the tests are too time-consuming and expensive. Further-
more, the total amount of enforcement testing these states have per-
formed has been dropping overall since EPAdiscontinued its testing
program. The total number of samplesthe states tested decreasedfrom
962 in 1986 to 500 in 1989. (Seefig. 4.1.) The state of Mississippi has
virtually discontinued testing. In fiscal year 1989,the state tested only
14 disinfectant samplesfor efficacy.




Page 49                        GAO/RCED-99-139 EPA Lacks Assurance Disinfectants Work
                                       Chapter4
                                       Limited Monitoring/Enforcement       of
                                       Registered Disinfectanta




Figure 4.1: Dlsinfectant Enforcement
Samples Tested by North Carolina,
                                       Entorcomont Samples
Florida, and Mississippi, 1983-89
                                       1909

                                        900

                                        so0

                                        700
                                        Boo

                                        so0

                                        400

                                        Boo

                                        200

                                        199

                                           0

                                                 1993    1994      199s      1996     1997      1999      1999
                                                Yoan


                                                t-l     MiSSiSSippi
                                                        Florida
                                                        NorthCarolina

                                       Florida and Mississippi data are for fiscal years (July 1 through June 30). NO&I Carolina data are for
                                       calendar years.

                                       Source: Prepared by GAO on the basis of data from the states of North Carolina, Florida, and
                                       Mississippi.


                                       To someextent, the scientific controversies surrounding the test
                                       methods and performance standards have affected the states’ efforts to
                                       enforce the efficacy of disinfectants on the market. For example, the
                                       state of Florida has not attempted to issue complaints against any regis-
                                       trant whose disinfectants have failed efficacy tests since 1986.
                                       According to the Director, Florida Division of Inspection, the state does
                                       not feel comfortable basing legal action on results from efficacy tests
                                       until controversies surrounding the AOACUse-Dilution Method are
                                       resolved.
                                       Although Florida, North Carolina, and Mississippi have tested and failed
                                       at least somedisinfectants registered for use in other states, states do



                                       Page SO                              GAO/RCED-90-139 EPA Lacks Assurance Disinfectants Work
                                                                                    -
Chapter 4
Limited Monitoring/Enforcement   of
Registered Dieinfecta.nts




not have authority to regulate the efficacy of disinfectants in the mar-
ketplace outside of their boundaries. In at least two cases,registrants
have respondedto state enforcement action by recalling disinfectants
from the marketplace and disinfectant users within the state or by
deleting specific efficacy claims from disinfectant labels, but have con-
tinued to market the disinfectants or make the claims in other states. In
one case,a producer agreed to recall from a single state market a phe-
no&based disinfectant marketed under nine different brand names
after the product failed state efficacy tests. However, the disinfectant,
which is registered for use on floors, walls, and other hard surfaces in
hospitals, nurseries, rest rooms, telephone booths, and elsewhere, con-
tinues to be sold in all other states, according to the registrant’s cus-
tomer service representative. In the other case,the registrant of two
nationally marketed disinfectants removed claims that the products
were effective in hard water from the product labels in one state after
one of the products failed state efficacy tests. However, the registrant
continues to market the two products (one of which is for use on floors,
walls, basins, and on instruments in hospitals, nursing homes,schools,
and medical and dental offices) with the claims in other states,
according to the registrant’s customer service representative.
States other than Florida, North Carolina, and Mississippi have been
reluctant to start testing programs. We interviewed officials from sev-
eral of the states with high pesticide enforcement funding from EPAand/
or a high number of disinfectant producers located within their bounda-
ries. The most common reason they offered for not operating testing
programs was that they do not have the resourcesneededto establish
and operate a testing facility. Although EPAhas provided most states
with funding for pesticide enforcement activities, the amount of funding
decreasedby about 25 percent in real terms (i.e., after adjusting for
changesin price levels) from federal fiscal year 1983 to 1989, and,
according to the Chief, EPAGrants and Evaluation Branch, EPAhas not
required the states to make disinfectant efficacy testing a pesticide
enforcement priority. Instead of a testing program, most of these state
officials said they rely on EPA'Sjudgment in registering disinfectants as
assurancethat they are effective.
The states of Wisconsin and Virginia operate testing programs but not
under their pesticide enforcement programs. The Wisconsin Department
of Health and Social Servicesrequires the efficacy of sanitizers intended
for use on food contact surfaces (except those containing inorganic
hypochlorites) to be tested under its program to regulate the milk and
restaurant industries. The Commonwealth of Virginia discontinued an


Page 61                          GAO/RCED-90-139 EPA Lacks Assurance Disinfectants Work
                    Chapter 4
                    Limited Monitoring/Enforcement   of
                    Registered Disinfectants




                    enforcement testing program sometime around 1982 because,according
                    to the Supervisor, Virginia Office of Pesticide Regulation, the state
                    found that the AOACUse-Dilution Method did not yield results that were
                    reproducible enough to take enforcement action against registrants
                    whose samples failed the test. Despite problems with the method, the
                    state has continued to test disinfectant samplesthat manufacturers
                    submit in order to compete for state contracts for sometypes of efficacy
                    claims.


User Programs Not   Individuals and organizations both within and outside of the health-care
Established         industry told us that they were not aware of any hospitals, doctors, or
                    dentists that test disinfectants for efficacy. According to several health-
                    care representatives, hospitals and other users do not have the
                    resourcesand/or the expertise necessaryto operate a program to mon-
                    itor the efficacy of disinfectants on the market. Instead, they rely on
                    EPA'Sregistration of disinfectants as assurancethat disinfectants on the
                    market are effective. Furthermore, restaurants and food-processing
                    facilities do not test the sanitizers they use for efficacy, according to
                    representatives from these industries.

                    Someresearchersaffiliated with users (as well as others) have tested
                    disinfectants, or are planning to test disinfectants, but have not estab-
                    lished monitoring programs. For example, since 1976, Clinical Research
                    Associates,an independent research organization dedicated to evalu-
                    ating dental materials, devices,and concepts,has been testing the effi-
                    cacy of selecteddisinfectants used by dentists. As of February 1990, we
                    had identified two organizations, the American Dental Association (ADA)
                    and the National Sanitation Foundation (NSF), that were considering
                    testing disinfectants for efficacy. However, if ADA were to test disinfec-
                    tants, ADA would only test tuberculocidal disinfectants and would not
                    test products routinely. Moreover, both programs would depend on vol-
                    untary participation by registrants.
                    Although several individuals and organizations affiliated with disinfec-
                    tant users have complained or submitted data to EPAindicating that spe-
                    cific disinfectants do not work as claimed, EPAhas not established the
                    system for processingthese complaints that it proposed as part of its
                    1987 disinfectant program improvement strategy, Furthermore, EPAhas
                    not yet addressedthe question of how it would verify the complaints
                    and take appropriate enforcement action against disinfectants found to
                    be ineffective.



                    Page 52                          GAO/RCED-90-139 EPA Lacks Assurance Disinfectants Work
                           Chapter 4
                           Lhhd     Monitoring/Enforcement   of
                           Registered Msinfectanta




Industry Self-Regulation   The burden of monitoring the efficacy of disinfectants has fallen on the
Limited                    industry itself becauseEPAno longer tests disinfectants, few states test
                           and their programs are limited, users do not generally test, and EPAlacks
                           an effective complaint system. However, market forces cannot be relied
                           upon to control disinfectant efficacy problems becauseusers cannot
                           visually identify ineffective products. Furthermore, although regis-
                           trants can test competitors’ products and have challenged competitors’
                           claims, EPAhas been unable to resolve conflicting claims becauseit lacks
                           the laboratory facilities necessaryto do so, according to the Chief, Anti-
                           microbial Program Branch. Consequently, industry self-regulation
                           appears limited. A June 1990 update to EPA’SJune 1987 strategy paper
                           on improving the disinfectant program acknowledgedthat someregis-
                           trants make unregistered and/or exaggeratedefficacy claims for disin-
                           fectants and that someregistrants market unregistered disinfectants.


SomeMarketed               Although the scientific controversies surrounding disinfectant efficacy
                           test results discussedin chapter 2 cloud the issue somewhat, historical
Disinfectants May Be       data on specific products from EPA and the states, along with data from
Ineffective                industry, academia,and other sources,indicate that somedisinfectants
                           on the market are ineffective. Although the true percentageof ineffec-
                           tive disinfectants on the market is unknown, EPAacknowledged in 1983
                           that as many as 20 percent of marketed disinfectants may be ineffec-
                           tive. Consequently,public health and consumerwelfare may be at risk
                           from disinfectants that do not work as claimed.

                           Historical test-failure data indicate that EPAand the states have been
                           unable to substantiate disinfectant claims for certain products. Between
                           1978 and 1982,the last 6 years during which EPAtested disinfectants, an
                           averageof 42 percent of all disinfectant samplestested by the lab failed
                           efficacy tests. In the years since the EPAlab closed(1983-89), 22 percent,
                           4 percent, and 2 percent of all disinfectant samplestested by the states
                           of Florida, North Carolina, and Mississippi, respectively, have failed
                           efficacy tests. During 1988 and 1989, about 30 percent of all sanitizer
                           samplestested under the state of Wisconsin program failed, and about
                           40 percent of the samplestested by the Commonwealth of Virginia
                           failed.
                           Although many products have failed disinfectant efficacy tests in EPA’S
                           and the states’ laboratories, the data generally are not comparable. More
                           specifically, EPAand the states have used different schemesfor sampling
                           disinfectants from the marketplace, used different performance stan-
                           dards to assessefficacy, and tested different types of claims. For


                           Page 63                           GAO/RCED-99-139 EPA Lacks Assurance Disinfectants Work
Chapter 4
Limited Monitoring/Enforcement   of
Registered Disinfectante




example, Florida has focused on sampling and testing disinfectants that
frequently fail efficacy tests and/or are intended for hospital use,
whereas Mississippi has tested products at random from the market-
place. North Carolina and Florida have used pass-fail criteria for disin-
fectants in efficacy tests that are more lenient than EPA'Sregistration
standards. Mississippi has not tested disinfectants for efficacy in the
presenceof organic matter, while Florida has. Virginia tests disinfec-
tants to determine whether they are effective in hard water and in the
presenceof organic matter regardlessof whether the products are regis-
tered with those claims.

While the data from state and federal enforcement testing are not com-
parable, they suggestthat an unknown number of disinfectants on the
market may be ineffective. Somedisinfectants have failed enforcement
tests by a wide margin. At least two states have found disinfectants con-
taminated with bacteria. Somedisinfectants have failed efficacy tests
repeatedly in a single state, in multiple regulatory labs (state and/or
EPA), or tests performed by registrants themselves at regulatory labs.
For example, a disinfectant registered for use on floors, walls, showers,
and other surfaces in hospitals, nursing homes,and schoolsfailed effi-
cacy tests on multiple occasionsin EPA'Slab and two state laboratories.
Similarly, a disinfectant registered for use in hospitals, nursing homes,
food-processingfacilities, office buildings, schoolsand recreation facili-
ties failed 28 efficacy tests in one state from 1983 to 1988.
In addition, data from industry, academia,and others also raise ques-
tions about whether somedisinfectant batches in the marketplace work
as claimed on their labels. For example, in a collaborative study of the
AOACUse-Dilution Method, three of four disinfectant manufacturers’ lab-
oratories unknowingly tested and failed their own products.
While several officials from EPA,industry, and the states, and most
others we talked to believe that disinfectants on the market are gener-
ally effective, various sourcesestimate that between 5 to 20 percent of
disinfectants on the market do not work as claimed. EPAofficials, them-
selves,believe that somedisinfectants on the market may be ineffective.
In 1983, the Director of the Compliance Monitoring Staff (now the Office
of Compliance Monitoring) estimated that a random sample of disinfec-
tants would reveal a failure rate of up to 20 percent. He concluded that,
“This is an unacceptable rate of failure for products with direct public
health significance.” In a 1983 congressionalhearing, the Director of the
Office of Pesticide Programs stated that, given the types of failure rates
the Beltsville laboratory had found, it was apparent that many batches


Page 54                          GAO/RCED-90-139
                                              EPALacks Assurance Disinfectants Work
                  Chapter 4
                  Limited MonitoriugjEnforcement   of
                  Regiet4wed Disinfectanta




                  of ineffective products had reached the marketplace with the limited
                  level of testing conducted, Furthermore, EPAand state officials, as noted
                  in chapter 2, have stated that the conflicting test results obtained in
                  research on disinfectant efficacy test methods may result as much from
                  ineffective products as from problems with the methods themselves.


                  EPAlacks assurancethat     all disinfectants on the market work as
Conclusions       claimed. After a disinfectant is registered, its formulation could be
                  altered, intentionally or inadvertently, and ineffective batches intro-
                  duced into the marketplace. Although EPAhas acknowledgedthis possi-
                  bility, the agency discontinued its limited testing and enforcement
                  program in 1982. Becausestate and user monitoring efforts are limited,
                  EPA’Sdecision to stop enforcement testing was, in effect, a decision to let
                  industry regulate the efficacy of marketed disinfectants. However, EPA
                  lacks a formal system to channel complaints about potentially ineffec-
                  tive disinfectants from competitors, users, and others, and to take
                  appropriate enforcement action against ineffective disinfectants.
                  Greater leadership on EPA’Spart to develop a strategy that pools
                  resourcesfrom the states, user groups, industry and others to identify
                  potentially ineffective disinfectants and that specifies the appropriate
                  enforcement action against disinfectants found to be ineffective, would
                  provide better assurancethat disinfectants on the market work as
                  claimed. Without such an enforcement strategy, EI’A’Spolicy of “let the
                  buyer beware” for disinfectants may be compromising public health and
                  consumer protection. While we recognizethat EPAneedsto resolve the
                  scientific controversies that surround disinfectant efficacy test methods
                  and performance standards (discussedin ch. 2), these controversies
                  should not prevent EPAfrom establishing and implementing an enforce-
                  ment strategy to ensure the efficacy of disinfectants in the marketplace.

                  We recommendthat the Administrator, EPA,develop, publish for com-
Recommendations   ment, and implement an enforcement strategy to ensure that marketed
                  disinfectants work as claimed. This strategy should specify (1) the
                  mechanismsand procedures for identifying potentially ineffective disin-
                  fectants; (2) the procedures for investigating and verifying complaints
                  about potentially ineffective disinfectants, including, where necessary,
                  the use of independent laboratory testing; and (3) the criteria and proce-
                  dures for initiating registration and/or enforcement action against disin-
                  fectants found to be ineffective. In light of federal budget constraints,
                  we also recommendthat the Administrator explore options for pooling
                  resourcesfrom the states, user groups, and industry to implement a


                  Page 65                          GAO/RCED-90-139 EPA Lacks Assurance Disinfectants Work
Chapter 4
Limtted Monitoring/Enforcement   of
RegisteredDiehuectantB




national disinfectant efficacy enforcement strategy. (Seech. 6 for a dis-
cussionof options for establishing a laboratory facility to assist in an
enforcement program.)




Page 56                          GAO/RCED-99-139 EPA Lacks Assurance Disinfectants Work
Chapter 6

Altemtives for a Disinfectant Laboratory


                      Many people in industry, the state regulatory laboratories, universities,
                      scientific associations,user groups, and other organizations believe an
                      independent laboratory facility is neededto research and selectively test
                      the efficacy of disinfectants. Somebelieve that EPA'Sefforts to resolve
                      the scientific controversies that surround disinfectant efficacy test
                      methods may languish without a federal laboratory facility. EPAofficials
                      have objected to such a facility because(1) EPAcannot afford to resume
                      methodology research and testing disinfectants at its limited laboratory
                      facility in Beltsville, Maryland, given its limited budget and competing
                      pesticide program priorities, and (2) they do not think the federal gov-
                      ernment should test these products. However, fees charged for the privi-
                      lege of obtaining a disinfectant registration could help offset the costs of
                      a disinfectant laboratory facility. Also, public health may be endangered
                      without such a resource.
                      This chapter briefly identifies possible alternatives for a laboratory
                      facility, considering (1) the need for and objectives of a laboratory
                      facility, (2) suggestedcriteria to evaluate alternatives, (3) a list of pos-
                      sible alternatives, and (4) the option of fees. The chapter is not a cost/
                      benefit assessmentof various alternatives for a disinfectant laboratory
                      facility. Rather, the purpose of this chapter is to briefly present the pros
                      and cons of the alternatives.

                      Previous chapters in this report discussedthe need for a laboratory
Need for and          facility to research and test the efficacy of disinfectants. Although their
Objectives of a       opinions differ, various critics of EPA'Sregulation of disinfectants from
Disinfectant          industry, academia,user groups, and other organizations have collec-
                      tively identified five objectives, listed below, of a disinfectant labora-
Laboratory            tory facility,
                  .   Researchand development: the facility should research and develop dis-
                      infectant efficacy test methods and performance standards that are
                      accurate and precise, objectively evaluate the validity of registrant-pro-
                      posed alternative methods and modifications, and actively participate in
                      the scientific exchangeof information.
                      Preregistration tests: the facility should conduct a preregistration-
                      testing program to verify selective disinfectant efficacy claims and
                      ensure the quality and integrity of registrant-submitted data.
                      Laboratory development: the facility should improve and standardize
                      laboratory procedures,promote good laboratory practices, train labora-
                      tory technicians, and conduct a check sample program to improve the



                      Page 67                    GAO/RCED-99-139 EPA Lacks Assurance Disinfectants Work
                          Chapter 6
                          Altemathe   for a Dbinfect.ant   Laboratory




                          capability of commercial and other laboratories conducting disinfectant
                          efficacy tests.
                        . Post-registration tests: the facility should conduct a post-registration
                          testing program to verify selective disinfectant efficacy claims in the
                          marketplace and assist in enforcement cases.
                        . Referencelaboratory: the facility should serve as the reference labora-
                          tory on questions of test methodology and procedures,referee disputes,
                          and confirm test sample results from state government laboratories.
                          With respect to the testing functions, the laboratory facility would not
                          need to test every disinfectant product or claim. Proponents of a disin-
                          fectant laboratory facility do not support the idea of routine batch-
                          testing to duplicate or replace manufacturers’ quality control programs.
                          Rather, the laboratory could focus its efforts on those product claims
                          that are of the greatest significance to public health, such as sporicidal
                          and tuberculocidal claims, randomly check the efficacy of disinfectants
                          in the marketplace, and selectively target those products or chemicals
                          with suspectedefficacy problems.


                          Many factors or criteria could be used to evaluate alternatives for
SuggestedCriteria for     meeting the objectives of a disinfectant laboratory facility listed above.
Evaluating                However, we suggestfour criteria for evaluating alternatives: (1) inde-
Alternatives              pendence,(2) authority, (3) quality, and (4) cost.
                          The need for a laboratory facility that is independent has been demon-
                          strated in the scientific controversies surrounding disinfectant efficacy
                          test methods. Market forces and competitive pressuresmake it difficult
                          to ascertain the validity of scientific disputes. A laboratory facility that
                          provides information to        on the validity of disinfectant efficacy test
                                                           EPA

                          methods and performance standards should be relatively isolated from
                          the profit motive of the industry. The facility should be above reproach
                          to allegations of conflict of interest or the appearanceof such conflict.
                          An independent laboratory facility needsthe authority to obtain and
                          test product samples,validate test methods, verify registrants’ claims,
                          establish laboratory procedures, and referee disputes.

                          The laboratory facility and personnel should be of high quality. The lab-
                          oratory would be unable to restore credibility in disinfectant regulation
                          if its equipment and its personnel were not at the forefront of microbi-
                          ology and infection control.



                          Page 68                                GAO/RCED-90-139 EPA Lacks Assurance Diiinfectants   Work
                      Chapter5
                      Alt.ematlves for a Disinfectant Laboratory




                      Although all of the criteria consideredimply cost tradeoffs, the effi-
                      ciency and practicality of each alternative must be considered.We have
                      not prepared detailed cost estimates for each alternative; rather, we
                      have consideredwhether the alternatives appear efficient and practical.


                      Several alternatives or a combination of alternatives could be developed
Alternatives for a    for operating a laboratory facility to test the efficacy of disinfectants
Laboratory Facility   and perform other functions. However, we suggestfour alternatives: (1)
                      the federal government, (2) a commercial laboratory, (3) state govern-
                      ment(s), and (4) a nonprofit association or foundation.
                      Many proponents of a disinfectant laboratory facility support the idea
                      of the federal government operating the laboratory, particularly of its
                      resuming disinfectant testing in Beltsville, Maryland. A federally oper-
                      ated facility would satisfy the criterion of independence.EPA,the most
                      likely agencyto operate such a lab, has the authority to run it, but is not
                      required to do so under FIFR,A. Legislation has been consideredin pre-
                      vious congressionalsessionsto require EPAto establish standards for
                      disinfectants. Quality and cost would be major considerations in estab-
                      lishing a federal laboratory, especially in resuming testing at the Belts-
                      ville facility. Both EPAand its critics acknowledge that EPAwould need
                      new, highly skilled personnel to operate a testing facility. In addition,
                      EPA'Sexisting facility would need extensive renovation becausesomeof
                      the equipment and supplies at the lab are obsolete and in disrepair.

                      Cost estimates for establishing a federal disinfectant laboratory range
                      from $200,000 to $2 million; annual operating cost estimates range from
                      $300,000 to $600,000, depending on the size and scopeof operations.
                      Although we were unable to find official historical budget documents
                      from EPA'Sdisinfectant laboratory, various records in the files indicate
                      that EPAmay have spent upwards of $300,000 in fiscal year 1981, the
                      year before the laboratory stopped testing disinfectants. This amount
                      may not cover the costs of a federal laboratory today becauseof infla-
                      tion and becausea testing program today may be larger in scopeand
                      size than the one conducted by the facility in 1981.
                      Although EPAcurrently is responsible for registering and enforcing the
                      efficacy of these products, several proponents of a federally operated
                      disinfectant laboratory have suggestedthat the cnc or the FDAoperate
                      the laboratory either alone or in cooperation with EPA.Critics of EPA
                      argue that it lacks personnel with the expertise in infection control prac-
                      tices necessaryto effectively regulate disinfectants. They also argue


                      Page 69                           GAO/RCED-90-139 EPA Lacks Amurance Disinfectants Work
Chapter 5
Alternatives   for a Disinfectant Laboratory




that the disinfectant program will always be of lower priority than
other pesticide programs within EPAand that perhaps it is a program
that is misplaced.
Rather than operate its own disinfectant laboratory, EPAcould contract
for the necessarylaboratory serviceswith a commercial facility. This
arrangement might offer EPAmaximum flexibility in directing the focus
of work each year. In addition, since the private sector currently retains
the best available scientific and technical personnel and laboratory
equipment, EPAmight be able to contract for the quality of services
needed.However, this alternative might sacrifice independenceif the
contract laboratory has or has had a closelink with the regulated
industry. In addition, EPAhas been criticized frequently by the Congress
for contracting out too many of its responsibilities. (We did not estimate
how much it would cost for EPAto contract out for a disinfectant
laboratory.)
A third alternative for a disinfectant laboratory is for the states to
operate their own laboratories. However, this alternative has already
proven unsuccessful.Most states do not operate a disinfectant labora-
tory today and are unlikely to open labs in the near future becauseof
cost constraints. Further, while the states would be independent, they
would lack the necessaryauthority to test prdducts not sold in their
state. Users and consumerswould be unfairly disadvantaged if their
states were unable to afford a laboratory facility while neighboring
states could afford one. In addition, having 60 states research improve-
ments to disinfectant efficacy test methods and test disinfectants would
be grossly inefficient.
A fourth alternative could be for a nonprofit associationor foundation
to operate a disinfectant laboratory. As discussedin chapter 4, the
American Dental Association and the National Sanitation Foundation
are considering whether and when to establish their own testing pro-
grams. While these efforts to help protect public health are commend-
able, they may not be able to achieve the objectives of a disinfectants
laboratory outlined above becausethey depend on registrants to volun-
tarily cooperate and submit their products for testing. In addition, these
organizations lack authority to take any registration and/or enforce-
ment actions on the basis of their tests.




Page 00                              GAO/RCED-99-139 EPA Lacks Amwrame Dihfectants   Work
                         Chapter 5
                         Alternatives   for a Disinfectant Laboratory




                         EPAhas objected to resuming testing disinfectants at its Beltsville labora-
Feesfor a Disinfectant   tory, in part, becauseof a lack of resources,However, fees might be one
Laboratory               way to help offset the costs of a disinfectant laboratory. Currently, the
                         1988 amendmentsto FIFRA(known as FIFRA‘88) prohibit EPAfrom
                         charging such fees. In addition, fees, which industry has opposed,pre-
                         sent other obstaclesthat need to be considered.
                         EPA’Sprimary   pesticide program objective has been to implement FIFRA
                         ‘88, which involves reviewing the risks and benefits of all pesticides
                         first registered before November 1, 1984-a processknown as “reregis-
                         tration.” FIFRA‘88 imposesa one-time reregistration fee and an annual
                         maintenance fee to help offset the costs of accelerating reregistration
                         and expediting new registrations. The FIFRA‘88 reregistration fees are
                         $150,000 per active ingredient but are waived for small volume active
                         ingredients (including small volume disinfectant active ingredients) and
                         reduced for small businesses,The annual maintenance fee is designedto
                         provide about $14 million each year. The FIFRA‘88 fees do not explicitly
                         provide funds for operating a disinfectant laboratory. Furthermore,
                         FIFRA ‘88 prohibits EPAfrom charging other fees until September30,
                         1997, when the FIFRA‘88 fee provisions expire. Consequently, the Con-
                         gresswould have to amend FIFRA‘88 to allow EPAto charge disinfectant
                         registrants a fee specifically designatedfor operating a disinfectant
                         laboratory.
                         The industry has opposeduser fees for several reasonsbut mostly
                         becauseit views product registrations as a public rather than private
                         benefit. However, charging registrants user fees to finance a disinfec-
                         tant laboratory, in order for EPAto recover the costs of testing their
                         products, is analogousto charging user fees for EPAto acceleratereregis-
                         tration of older pesticides and expedite registration of new pesticides.
                         Furthermore, although fees would increasethe cost of registering and
                         marketing disinfectants, if registrants want to market disinfectants it
                         seemsfair that they finance the cost of demonstrating that their prod-
                         ucts work as claimed.

                         Earlier chapters of this report have discussedthe need for a laboratory
Conclusions              facility to research methods for testing disinfectant efficacy and to test
                         the efficacy of disinfectants. Several alternative ways exist for oper-
              Y          ating a disinfectant laboratory, but EPAlacks the resourcesneededto
                         finance such a laboratory. However, fees on disinfectant registrations
                         might be one way to help finance a disinfectant laboratory. Becauseof
                         the statutory prohibition on new pesticide registration fees and the


                         Page 61                             GAO/RCED-99-139 EPA Lacks Ammxnce Disinf’ectant~ Work
                  Chapter 5
                  Alternatives   for a DMdect.ant   Laboratory




                  unknown effects that disinfectant fees might have on the marketplace,
                  what is neededis a detailed cost/benefit assessmentof alternatives for a
                  disinfectant laboratory, including the option of assessingfees to help
                  finance such a facility.


                  We recommendthat the Administrator, EPA,develop a detailed cost/ben-
Recommendations   efit analysis of alternatives for operating a laboratory facility to
                  research and test the efficacy of disinfectants, including the option of
                  charging fees to register disinfectants to help finance such a facility, and
                  submit the results of its analysis to the Congressso that the Congress
                  may weigh the advantagesand disadvantagesof various alternatives.




                  Page 62                              GAO/RCED-90-139 EPA Lacks Assurance DMnfectanta Work
Page 63
Aonendix I

Affiliations of Individuals GAO Contacted


                          Not all individuals GAOcontacted officially representedthese organiza-
                          tions, but they were employed by or affiliated with these organizations.
                          Someof the individuals contacted from nongovernmental organizations
                          were also employeesof disinfectant registrants.


Federal Agencies          Centers for DiseaseControl
                          Environmental Protection Agency
                          Food and Drug Administration


State Governments         California
                          Florida
                          Georgia
                          Illinois
                          Iowa
                          Mississippi
                          New York
                          North Carolina
                          Texas
                          Virginia
                          Wisconsin


Standard-Setting          American Society for Testing and Materials
Organizations             Association of Official Analytical Chemists
                          National Sanitation Foundation


Professional and Trade    American Dental Association
Associations              American Hospital Association
                          Association for Practitioners in Infection Control
                          Chemical Specialties Manufacturers Association
                          Joint Commissionon Accreditation of Healthcare Organizations
                          National Environmental Health Association
                          National Food ProcessorsAssociation
                          National Restaurant Association


Scientific Associations   American Society for Microbiology
                          Clinical ResearchAssociates




                          Page 64                   GAO/RCRD-99-139 EPA Lacks Assurance Disinfectanta Work
   .

                          AffUiatio~~ of Indlviduala GAO Contacted




Universities              University of Missouri-Columbia
                          University of North Carolina at Chapel Hill


Commercial Laboratories   Gibraltar Biological Laboratories, Inc.
                          Hill Top Biolabs, Inc.




               Y




                          Page 65                          GAO/RCED-99-139 EPA Lacb Assurance DMnfectanta Work
Appendix II

Major Contributors to This Report


                          Peter F. Guerrero, Associate Director
Resources,                J. Kevin Donohue,Assistant Director
Community, and            William M. Layden, Evaluator-in-Charge
Economic                  Karen Simpson,Staff Evaluator
                          Scott W. Weaver,Staff Evaluator
Development   Division,   Sarah-Ann Moessbauer,Operations ResearchAnalyst
Washington, D.C.

                          Doreen StolzenbergFeldman, Senior Attorney
Office of the General
Counsel, Washington,
DC.




                          Page 66                 GAO/RCED-90-139 EPA La&s Arsmrance MeMectante   Work
Giossary


Accuracy                   The closenessof an observedresult to the true or acceptedresult.


Antimicrobial Pesticides   With someexceptions, substancesand mixtures of substances,intended
                           for inhibiting the growth of, or destroying any bacteria, fungi patho-
                           genic to people and other animals; or viruses declared to be pests and
                           existing in any environment. (For purposes of this report, we have
                           referred to all antimicrobial pesticides for public-health use as
                           “disinfectants.“)


Bacteria                   Small microorganisms with a relatively primitive cellular organization.


Collaborative Study        A study involving a number of laboratories analyzing the samesamples
                           by the samemethod for the purpose of generating performance data on
                           the method when a competent analyst usesit exactly as written. (Per-
                           formance data include any values that indicate the reliability, applica-
                           bility, and practicability that can be expected from the method.)


Disinfectant               As used in this report, any pesticide used on inanimate surfaces or
                           objects and intended to inhibit or destroy bacteria, fungi, or spores
                           causing human disease.


Efficacy                   The capacity of a pesticide product when used according to label direc-
                           tions to control the target pest. (As used in this report, the term “effi-
                           cacy” is synonymous with the terms “product performance“ and
                           “effectiveness.“)


Fungicide                  As defined in this report, a disinfectant intended to destroy fungi.


Fungi                      A group of organisms devoid of chlorophyll that cannot manufacture
                           their own food.


Nosocomial Infection       An infection that occurs during or sometimesafter hospitalization and
                           was not present or incubating at the time of the patient’s admission.



                           Page 07                    GAO/RCED-90-139 EPA Lacks Assurance Diiinfectante   Work
                       Glossary




Precision              Agreement among repeat observations made under the sameconditions.


Reliability            A criterion used to evaluate the validity of a method by measuring the
                       method’s ability, when used by qualified analysts, to produce data of a
                       predictable degreeof precision and accuracy.


Repeatability          The variability in successiveresults obtained with the samemethod on
                       identical test material and under the sameconditions (same operator,
                       same apparatus, samelaboratory, and sametime).


Reproducibility        The variability in individual results obtained with the samemethod on
                       identical test material but under different conditions (different operator,
                       different apparatus, different laboratory, and/or different time).


Sterilizer/Sporicide   As defined in this report, a disinfectant intended to destroy or eliminate
                       viruses and all living bacteria, fungi, and their spores.


Tuberculocide          An agent that is intended to destroy or inactivate tuberculosis bacteria.


Viricide               An agent that is intended to destroy or inactivate one or more speciesof
                       virus.


Virus                  Any of a group of submicroscopicinfective agentsthat are regarded
                       either as simple microorganisms or as complex molecules.(Viruses are
                       capable of growth and multiplication only in connection with living
                       cells.)




(onerm)                Page 69                    GAO/RCED-90-139 EPA Lacks Assurance Disinfectants Work
    Orders   may also t)tl placed   by calling   (202) 2756241.




i