EPA Lacks Assurance They Work 84 lllllllW IM 142343 ll RESTItIcTED --Not qELEjSED to be re ease outside the General Accounting Office unless specifkally approved by the Office of Congressional Relations. Reeources, Community, and Economic Development Division B-236877 August 30,199O The Honorable John Conyers, Jr., Chairman The Honorable Frank Horton, Ranking Minority Member Committee on Government Operations House of Representatives The Honorable Mike Synar, Chairman The Honorable William F. Clinger, Jr., Ranking Minority Member Environment, Energy and Natural ResourcesSubcommittee, Committee on Government Operations House of Representatives This report respondsto your request that we review the Environmental Protection Agency’s (EPA) activities to regulate the efficacy of disinfectants. Specifically, it describesthe nature of the scientific controversies surrounding EPA-recommended methods and performance standards for testing the efficacy of disinfectants, efforts EPAhas made to obtain objective research to resolve these controversies, the adequacy of EPA’Sinternal controls to ensure the quality and integrity of registrant-submitted efficacy data, post-registration efforts EPAand the states have implemented to ensure that disinfectants on the market are effective, and the need and options for a laboratory to research and test the efficacy of disinfectants. Unless you publicly releaseits contents earlier, we will make no further distribution of this report until 30 days from the date of this letter. At that time, we will send copies of the report to appropriate congressionalcommittees; the Administrator, EPA;and other interested parties. This report was prepared under the direction of Richard L. Hembra, Director, Environmental Protection Issues,whom you may contact at (202) 276-6111if you or your staff have any questions. Other major contributors are listed in appendix II. $exP@ Assistant Comptroller General Executive Summary Disinfectants-about a $1 billion per year market-are used to kill Purpose germs on inanimate surfaces and objects in hospitals, schools,restau- rants, and homes.Becauseusers cannot seewhether disinfectants kill bacteria, fungi, and viruses, the use of ineffective disinfectants posesa threat to public health and wastes consumer dollars. Mounting concernsabout whether hospital and household disinfectants work as claimed and the adequacy of the Environmental Protection Agency’s (EPA) disinfectants program led the HouseCommittee on Gov- ernment Operations to request that GAOreview EPA’Sregulation of the efficacy of disinfectants. Background Under the Federal Insecticide, Fungicide, and Rodenticide Act (FIFRA), EPAgenerally must register (license) pesticide products, including disin- fectants, before they are marketed. EPAmay register a pesticide product only if EPAdetermines that it is effective, when used as claimed, without causing an unreasonablerisk to health or the environment. For most pesticides, EPAallows the marketplace to regulate product performance (efficacy) becauseusers can seewhether the pesticide is effective against the target pest. However, registrants of disinfectants intended to protect public health must submit efficacy data to substantiate each product performance claim and use. Until 1982, EPAconducted limited preregistration confirmatory and post- registration enforcement tests on disinfectants at its laboratory facilities in Beltsville, Maryland. EPAdiscontinued disinfectant testing in 1982 pri- marily becauseof budget constraints. Currently, EPArelies on its review of registrant-submitted efficacy data to register disinfectants. As of Sep- tember 1, 1989, about 4,100 disinfectants for public-health use were reg- istered with EPA,representing about 18 percent of approximately 23,000 registered pesticide products. EPAdoesnot know whether disinfectants kill the germs claimed on Results in Brief product labels for four reasons.First, although the validity of methods and performance standards used to assessthe efficacy of disinfectants has been the source of scientific controversy for over a decade,EPAdoes not independently test disinfectants before registering them and lacks criteria to assessthe validity of registrant-proposed test methods and modifications. Second,EPAhas made little progress in resolving these controversies becauseof budget constraints and inadequate research management.Third, EPAlacks sufficient internal controls to ensure the Page 2 GAO/ltCED.BO-122 EPA Lacka A~wce Dbinfectanta Work Executive Siummary quality and integrity of the data that registrants submit on disinfectant efficacy. Fourth, EPAlacks an enforcement strategy to ensure that, once registered, disinfectants sold and distributed in the marketplace work as claimed. The extent to which ineffective disinfectants are marketed is unknown. Although the scientific controversies cloud the issue somewhat, evi- dencefrom EPA,the states, and others suggeststhat up to 20 percent of disinfectants on the market may be ineffective. Principal Findings Validity of Test Methods EPAlacks assurancethat the test methods and performance standards used by registrants to substantiate disinfectant efficacy claims are valid. EPAprimarily relies on standard-setting organizations, such as the Association of Official Analytical Chemists (AOAC)and the industry itself to develop test methods and performance standards. However, these methods and standards have been embroiled in scientific controversies for over a decade.For example, scientists have debated whether the AOACUse-Dilution Method, the most widely used test method, is repro- ducible, accurate, and precise, and whether the performance standard (pass/fail criterion) established by EPAis valid. Although EPAbelieves that the existing methods and standards are acceptablefor registering and enforcing disinfectant efficacy claims, the controversies have impaired the credibility of the disinfectant program, An ad hoc industry/state group recently has developeda test method to replace the AOACUse-Dilution Method and is expected to present the results of its research to the AOACin September 1990. EPAofficials believe that the new method is reproducible and reliable and will consider whether to require that disinfectants be retested using it after AOACconsidersit for adoption. EPAhas contributed to the controversies by accepting test methods and modifications without criteria and independent laboratory data for eval- uating their validity. For example, EPAacceptsthree different test methods to demonstrate that disinfectants kill tuberculosis bacteria. At least one product tested under two of the methods produced substan- tially different results. Although EPAhas registered the product on the basis of one of the methods, EPAlacks the laboratory information needed to explain the differences in results between the methods. Page 9 GAO/RCED-99-139 EPA Lacks hsurtuwe Didnfectants Work . Executive Summary Although EPAhas been aware of the scientific controversies for years, it has made little progress in resolving them becauseof problems in con- ducting neededresearch.EPA'Sg-year, $384,000, cooperative agreements with the University of North Carolina did not fulfill EPA'Sresearch objectives to improve disinfectant efficacy methods becauseEPAinade- quately managedthe agreements.EPAhas also made little progress in conducting additional research becauseof budget constraints. In April 1990, EPAannouncedthat it would spend $600,000 for research on cer- tain disinfectant efficacy methods and estimated an additional $1.2 mil- lion will be needed. Controls Over Quality and To ensure the quality and integrity of registrant-submitted disinfectant Integrity of Data data, EPAreviews the data prior to registration and performs laboratory inspections and data audits. GAO,however, found internal control weak- nessesin these programs. For example, EPAhas not inspected the majority of labs that have performed disinfectant efficacy studies. In fact, EPAwas aware of only 12 of the 92 labs that had performed these studies. Although these programs need to be improved they, in them- selves,are not an adequate substitute for a preregistration program to selectively test disinfectant efficacy by an independent laboratory. Data reviewers, lab inspectors, and data auditors generally cannot identify casesin which registrants have selectively submitted data indicating that their disinfectants work becausethey generally do not observethe tests in progress and no physical evidenceremains from the tests conducted. Monitoring and EPA'Sregistration processby itself cannot provide assurancethat disin- Enforcement of Marketed fectants are effective becauseregistrants could market ineffective batches, either intentionally or inadvertently, after registering them. Disinfectants However, EPAdoesnot enforce the efficacy claims of disinfectants on the market. EPAdiscontinued its limited enforcement testing program in 1982 primarily becauseof budget constraints. Since 1982, EPAhas looked to the states, user groups, and the industry to enforce efficacy claims, However, GAOfound few states and no users monitoring disinfec- tant efficacy becauseof cost concerns.Only five states test disinfectants for efficacy, and these states have limited programs. Moreover, EPA lacks a strategy to channel complaints about potentially ineffective dis- infectants from the states, user groups, and the industry and to take appropriate enforcement action against disinfectants found to be inef- fective. Although EPAneedsto resolve the scientific controversies that surround disinfectant efficacy test methods and performance standards, Page 4 GAO/RCED-90-139 EPA Lacks Assurance Disinfectants Work these controversies should not prevent EPAfrom developing an enforce- ment strategy, in conjunction with the states, user groups and industry, to ensure that marketed disinfectants work as claimed. Public health and consumer welfare may be compromisedwithout such assurance. GAOis making recommendationsto the Administrator, EPA,to correct Recommendations deficiencies and restore credibility in the disinfectant program, including (1) developing a plan to resolve the scientific controversies that surround disinfectant efficacy test methods and performance stan- dards; (2) developing and publishing a policy that establishescriteria for evaluating the validity of new test methods and modifications, including criteria for determining when independent laboratory data are neededfor validation; (3) improving internal controls over its current programs to ensure the quality and integrity of registrant-submitted efficacy data and conducting preregistration tests to selectively verify registrant claims; (4) establishing an enforcement strategy in conjunc- tion with the states, user groups, and industry to ensure that marketed disinfectants work as claimed; and (5) preparing a cost-benefit analysis of alternatives for a laboratory facility to research and test the efficacy of disinfectants, including the option of charging fees to register disin- fectants to help finance such a facility. GAOdid not obtain official agency commentson this report. GAOdid, Agency Comments however, discussthe factual content of the report with EPAofficials and has included their commentswhere appropriate. EPAofficials generally agreed with the accuracy of the facts but believed that, as presented, the report could be misread and suggestedchangesfor presenting the facts. GAOmade somerevisions to the report on the basis of EPA'Scom- ments. GAObelievesthat the report is a fair and accurate presentation of the issues. Page 6 GAO/RCED-99-139 EPA Lacks Asmrance Disinfectants Work Contents Executive Summary 2 Chapter 1 8 Introduction Federal Regulation of Disinfectants 8 Disinfectant Types and Uses 9 Overview of Efficacy Data Requirements 12 Disinfectants and Infection Control 14 Prior GAO Reports 16 Objectives,Scope,and Methodology 16 Chapter 2 20 EPA Lacks Assurance Scientific Controversy Over Methods and Standards Methods AcceptanceProcessInadequate 20 24 That Efficacy Test Limited ProgressMade in Resolving Controversies 28 Methods Are Valid Conclusions 34 Recommendations 35 Chapter 3 Controls Over Quality/ WeaknessesImpair Data and Lab Reviews Preregistration Testing Neededto Ensure Data Integrity Integrity of Efficacy Internal Controls Inadequate Data Questionable Conclusions Recommendations Chapter 4 Limited MOInitoring/ Federal Enforcement Testing Discontinued 47 TX-LT------- --^ of I!d1lOI’CeItlerlt Nonfederal Testing Limited 48 SomeMarketed Disinfectants May Be Ineffective 53 Registered Conclusions 65 DiGnfectants Recommendations 55 Chapter 5 67 Alternatives for a Need for and Objectivesof a Disinfectant Laboratory SuggestedCriteria for Evaluating Alternatives 57 68 Disinfectant Alternatives for a Laboratory Facility 69 Laboratory Feesfor a Disinfectant Laboratory 61 Y Conclusions 61 Recommendations 62 Page 6 GAO/RCED-90-139 EPA Lacka Assurance Didnfectmta Work Content4 Appendixes Appendix I: Affiliations of Individuals GAO Contacted 64 Appendix II: Major Contributors to This Report 66 Glossary 67 Tables Table 1.1: SelectedDisinfectant Efficacy Claims 10 Table 1.2: Estimated Average Annual Amount of 12 Disinfectants Sold in the United States, 1986-87 Table 1.3: EPA-RecommendedMethods for Testing 13 Disinfectants Intended for Use on Hard Surfaces Figures Figure 1.1: RegisteredDisinfectant Efficacy Claims by 11 Type Figure 4.1: Disinfectant Enforcement SamplesTested by 60 North Carolina, Florida, and Mississippi, 1983-89 Abbreviations ADA American Dental Association AOAC Association of Official Analytical Chemists CDC Centers for DiseaseControl EPA Environmental Protection Agency FDA Food and Drug Administration FIFRA Federal Insecticide, Fungicide, and Rodenticide Act FMFIA Federal Managers’ Financial Integrity Act of 1982 GAO General Accounting Office GLP good laboratory practice LDIAD Laboratory Data Integrity Assurance Division NSF National Sanitation Foundation OCM Office of Compliance Monitoring OPP Office of Pesticide Programs OPTS Office of Pesticidesand Toxic Substances PDMS Pesticide Document ManagementSystem PPIS Pesticide Product Information System Toxic SubstancesControl Act UNC University of North Carolina Pqge 7 GAO/RCED-90-139 EPA La&m Auumnce DblnMtan~ Work Chapter 1 Introduction Disinfectants are used almost everywhere people want to kill disease- causing microorganisms-in households,hospitals, schools,restaurants, day care centers, dairy farms, and a host of other places.About $1 bil- lion a year is spent on disinfectants to kill bacteria, fungi, and viruses in bathrooms, kitchens, and offices; on medical and dental instruments, diaper pails, and eating utensils; and at many other locations, Although the role of the inanimate environment in transmitting infections has not been completely defined, the use of disinfectants is considered an impor- tant part of infection control programs. In fact, health-care organiza- tions recommend,and many public health ordinances require, their use. Federal Regulation of The Environmental Protection Agency (EPA) regulates disinfectants as pesticides under the Federal Insecticide, Fungicide, and Rodenticide Act, Disinfectants as amended(FIFRA).~Under F-IFFU,disinfectants generally must be regis- tered (licensed) by EPAbefore they may be sold, held for sale, or distrib- uted in commerce.EPAmay register a disinfectant only if EPAdetermines that it is effective, when used as directed, without causing an unreason- able risk to public health or the environment. EPArequires disinfectant firms to submit, among other things, data demonstrating that their prod- ucts are effective as claimed before EPAwill register them. As of Sep- tember 1, 1989, about 4,100 disinfectants were registered with EPA. These disinfectants represented about 18 percent of approximately 23,000 registered pesticide products. As permitted under FIFFtA,EPAhas waived all requirements for pesticide firms to submit efficacy data except for (1) disinfectants and (2) pesti- cides that claim to control vertebrates that may transmit diseasesto humans, such as rodents, birds, and skunks. EPAretains the requirement for disinfectants becauseusers cannot seewhether disinfectants kill microorganisms that may causehuman disease,such as the bacteria that causefood poisoning, and becausethe use of an ineffective disinfectant ‘As defined by EPA, “disinfectant” refers to only one of several types of antimicrobial pesticides, which, with some exceptions, are substances intended to inhibit or destroy microorganisms (bacteria, fungi, viruses, and spores). However, we use the term “disinfectant” in this report to broadly describe all antimicrobial pesticides intended to protect public health. For expianations of specific disinfectant efficacy claims, see table 1.1 and the glossary. “Disinfectants, which are used on inanimate surfaces, are distinct from antiseptics, which are used on skin and other living tissue. Antiseptics are regulated by the Food and Drug Administration. Disinfec- tants that are used in or on medical devices, such as dialysis machines, are regulated by both EPA and the Food and Drug Administration. Page 8 GAO/RCED-99-139 EPA Lacks Assurance Dhinfectanta Work . chapter 1 Introduction posesa threat to public health. By contrast, EPAdoesnot require regis- trants to submit efficacy data for other pesticides becauseusers gener- ally can tell whether they work, and the marketplace can regulate product performance. In addition, registrants of pesticides that target microorganisms that do not causediseasesin humans, such as those that target slime-forming or odor-causingbacteria, are not required to submit efficacy data. EPAdoes,however, require all registrants to be able to show that their products are effective on demand for such data. EPA’SOffice of Pesticidesand Toxic Substances(OPTS) is responsible for regulating pesticides, including disinfectants. Within OPTS, the Office of Pesticide Programs (OPP) and the Office of Compliance Monitoring (OCM) are responsible for evaluating pesticides for registration and for plan- ning and coordinating pesticide compliance/enforcement activities, respectively. The Antimicrobial Program Branch (formerly the Disinfec- tants Branch), within OPP,is responsible for registering disinfectants. Within the branch, the Efficacy Evaluation and Technical Management Section is responsible for approving and recommendingmethods for testing the efficacy of disinfectants and evaluating registrant-submitted efficacy data. EPAregisters disinfectants with a variety of efficacy claims for use in Disinfectant Types many areas of the inanimate environment. Disinfectant types and uses and Uses range from products intended to kill bacteria on hard surfaces in bath- rooms to products intended to chemically sterilize medical instruments in hospitals. A disinfectant may claim one or more of a number of types of efficacy. The types of efficacy claims a disinfectant may make depend on, among other things, the types of microorganisms the disinfectant targets (e.g., tuberculosis or a polio virus) and the disinfectant’s intended level of activity (e.g., a reduction in the level of the microorganism or a complete kill). (For a list of selecteddisinfectant efficacy claims, seetable 1.l.) Page 9 GAO/RCED-9&139 EPA Lacks Assurance Didnfectanta Work Chapter 1 Introduction Table 1.1: Selected Disinfectant Efficacy Clalmr Sterilizer The disinfectant, sometimes called a sporicide, is intended to destroy or eliminate viruses and all living bacteria, fungi, and their spores. (The claim denotes killing all microorganisms, including the highly resistant spore forms, and indicates that the disinfectant will produce the highest level of disinfection possible.) Tuberculocide The disinfectant is intended to destroy or inactivate tuberculosis bacteria. (Tuberculocidal claims are often used by medical users of disinfectants as an indicator of product strength because tuberculosis bacteria are more difficult to kill than most other species of bacteria.) Disinfectant The disinfectant is intended to destroy or inactivate one or more major species of bacteria, depending upon whether the disinfectant makes a “limited,” ” aeneral,” or “hospital” disinfectant claim. Fungicide The disinfectant is intended to destroy fungi. Virucide The disinfectant is intended to destroy or inactivate one or more specific viruses named on the disinfectant’s label. Sanitizer The disinfectant is intended to reduce the number of living bacteria or viable virus oarticles. Source: Prepared by GAO on the basis of EPA disinfectant efficacy data requirements and the definition of disinfectant from footnote 1. We used EPA’S Pesticide Product Information System (PPIS) to obtain EPA’S best available data on registered disinfectant claims. According to the system, about 4,100 disinfectants3were registered with EPA as of September 1, 198ga4These disinfectants made about 8,000 different effi- cacy claims or, on average,about 2 claims per disinfectant. The most common efficacy claim was “disinfectant.” (Seefig. 1.1.) “EPA registers both end-use products (products for sale at the retail level) and manufacturing-use products (active ingredients for use in end-use products). Because EPA requires a complete efficacy profile only on end-use products, we did not include manufacturing-use products in our counts of disinfectants. 4The 1999 amendments to FIFRA imposed user fees on pesticide registrants to help fund an acceler- ated review of older pesticides and expedited registration of new pesticides. In 1989, EPA canceled over 6,000 disinfectant registrations because registrants decided to abandon their registrations rather than pay the required fees. Page 10 GAO/RCED-90-139 EPA Lacks Assurance Lbidnfectanta Work Chapter 1 Introduction Flgure 1.l : Reglrtlsred Dlrlnfrctant 2% Efficacy Claim8 by Typo I Virucide Fungicide 1% Tuberculocide Sterilizer Disinfectant Sanitizer Dam are for end-use products as of September 1,1989. Water purifier daims were not included because they represented less than 1 percent of all registered disinfectant efficacy claims as of September 1, 1989. Source: Prepared by GAO on the basis of data from the EPA Pesticide Product Information System In addition to the types of efficacy claimed (e.g., tuberculocidal), each registered disinfectant must specify the use patterns for which the dis- infectant is recommended.Broad categoriesof use patterns include the use of a disinfectant (1) to kill microorganisms on hard surfaces, (2) to kill microorganisms on fabrics or textiles, (3) to control microbial pests associatedwith human or animal wastes, and (4) to treat water systems. Registrants typically label their disinfectants for use at specific sites within these broad categories.For example, a disinfectant intended for use on hard surfaces may be labeled for use on counter tops, medical instruments, floors, or other types of hard surfaces. According to PPIS, as of September 1, 1989, the approximately 4,100 dis- infectants made about 18,100 specific use site claims. About 76 percent Page 11 GAO/RCED-90-139 EPA Lacks Assurance Disinfectants Work Chapter 1 Introduction of the disinfectants, or 3,200, were registered for use on at least 1 hard surface site. Someof the common hard surfaces for which disinfectants were registered were the surfaces in bathrooms, hospitals, and eating establishments, and at commercial/industrial sites. EPAhas limited data on the size of the disinfectant market. However, data from a private market research firm indicates that the disinfectant market is about $1 billion per year at the retail level in the United States. In addition, as table 1.2 illustrates, the estimated average annual amount of disinfectants sold represented a substantial amount of all pesticides sold in the United States during 1986-87. Table 1.2: Estimated Average Annual Amount of Dirinfectanta Sold in the Pounds and gallons in thousands United States, 1985-87 Amount sold Percentage of Percentage Dry or solid dry or solid Liquid of liquid chemical chemical chemical chemical product (Ibs.) pesticides product (gal.) pesticides Pesticides 8,773,799 100 1,067,308 100 Disinfectants 1,182,203 13 542,606 51 Disinfectants used on hard surfaces 486.207 6 493,159 46 Source: Prepared by GAO on the basis of data from the PPIS and the FIFRA and Toxic Substances Control Act Enforcement System. EPArequires the manufacturer or registrant to develop and submit data Overview of Efficacy on a disinfectant for each claim and use of the product. EPAhas pub- Data Requirements lished guidelines that recommendspecific methods and minimum test specifications for registrants to use to test the efficacy of disinfectants. The guidelines also contain performance standards (pass/fail criteria) that disinfectants must meet to make efficacy claims. Description of Test EPA'SPesticide AssessmentGuidelines and supplemental technical gui- Methods dance briefs contain recommendedmethods for testing disinfectants, standards for conducting acceptabletests, and instructions on inter- preting and reporting data. Table 1.3 lists the EPA-recommended methods for demonstrating specific efficacy claims of disinfectants used on hard surfaces. II Page 12 GAO/RCED-90-139 EPA Lacks Assurance Disinfectants Work Chapter 1 Introduction Table 1.3: EPA-Recommended Method8 for Testing Disinfectants Intended for Use on Hard Surfaces lvr3e of claim EPA-recommended method(s) Test oraanismk) Sterilizer .- _.^-._- .--_ ---- _..-.- AOAC Sporicidal Test Bacillus subtilis and Clostridium sporogenes Tuberculocide AOAC Tuberculocidal Activity Method Mycobacterium tuberculosis Modified AOAC Tuberculocidal Activity Method ;;;;Jative Tuberculocidal Activity Test Hospital disinfectant AOAC Use-Dilution Method Salmonella choleraesuis, Staphylococcus aureus, and Pseudomonas aeruginosa .- __- .._.__._ _-_-_-____-. __-- AOAC Germicidal Spray Products Test General disinfectant AOAC Use-Dilution Method Salmonella choleraesuis and Staphylococcus aureus AOAC Germicidal Spray Products Test Limited disinfectant AOAC Use-Dilution Method Salmonella choleraesuis or Staphylococcus aureus .._._....~. . ----.-- ..- .__- - AOAC Germicidal Spray Products Test Fungicide AOAC Fungicidal Test Trichophyton mentagrophytes AOAC Use-Dilution Method AOAC Germicidal Spray Products Test Virucide EPA virucidal test parameters Specific virus claimed Sanitizing rinse (food-contact surfaces) AOAC Available Chlorine Germicidal Escherichia coli; and Salmonella typhi or Equivalent Concentration Method Staphylococcus aureus AOAC Germicidal and Detergent Sanitizers .-.- ..-. -.-.“.-- -~~ . Method Sanitizer (inanimate, nonfood-contact EPA sanitizer test parameters Staphylococcus aureus; and Klebsiella surfaces) pneumoniae aberrant or Enterobacter aerogenes Source: Prepared by GAO on the basis of EPA’s Pesticide Assessment Guidelines. Most of the EPA-recommended methods were developedunder the aus- pices of the Association of Official Analytical Chemists (AOAC),an inde- pendent, international standard-setting organization. AOAC'Sprimary purpose is to develop and validate standardized chemical and biological analytical methods that will perform with the necessaryaccuracy and precision under usual laboratory conditions to be recognizedas “offi- cial” methods by the courts and others. According to the AOACAssistant Executive Director, a collaborative study, or round robin test, is con- ducted for each method to provide an unbiased evaluation of the per- formance of an analytical method through the analysis of a number of identical samplesby a number of different laboratories. Page 13 GAO/RCED-90-139 EPA Lacks Assurauce DMnfectant~ Work Chapter 1 Introduction In evaluating registrant-submitted efficacy data on disinfectants, EPA determines whether the data are adequateto satisfy its data require- ments and guidelines. This determination involves considering the design and conduct of the test, including whether generally accepted methods were used, whether a sufficient number of measurementswere made to achieve statistical reliability, whether sufficient controls were built into the test, whether the test was conducted in conformity with the design, whether good laboratory practices were observed,and whether the results were reproducible. Description of The Pesticide AssessmentGuidelines also contain performance stan- Performance Standards dards that EPA requires disinfectant products to meet to make specific efficacy claims. Failure of a product to meet the specified testing or per- formance requirements is consideredevidencethat the product is unlikely to be effective as claimed in actual use. The performance standards vary depending on the claim intended and the test method employed. For example, for a hospital disinfectant, EPA recommendsthat three batches be tested using the AOAC Use-Dilution Method against three different microorganisms using 60 test tubes per microorganism per batch. If the product fails to kill a microorganism in 2 or more out of the 60 tubes for any microorganism/batch, then the product is consideredto have failed the test for the specific microor- ganism tested. Although the role of the inanimate environment in transmitting infec- Disinfectants and tions has not been completely defined, doctors, dentists, restaurant Infection Control owners, consumers,and others consider disinfectants to play an impor- tant part in infection control. Health-care providers, as well as others, rely on EPA’S registration of disinfectants as evidencethat purchased products work as claimed. Infection control is a serious concern for health-care providers. According to the Centers for DiseaseControl (WC), about 6 percent of all patients acquire an infection while hospitalized. Hospital-acquired, or nosocomial,infections prolong hospital stays, increasepatient care costs, and, in somecases,causedeath. According to one estimate, Page 14 GAO/RCED-90-139 EPA Lacks Assurance DMnfectanta Work Chapter 1 IntroducWon nosocomialinfections may causeapproximately 20,000 deaths and con- tribute to about 60,000 more deaths annually.” Furthermore, according to one infection control expert, nosocomial infections rank among the 10 most frequent causesof death in the United States.‘;In addition, interest in disinfectant efficacy has increased in responseto the growing num- bers of immune deficient patients, who are susceptible to infections. Although medical experts generally believe that most nosocomial infec- tions are transmitted from person to person rather than from the inani- mate environment, the role of the inanimate environment in transmitting infections has not been completely defined. Researchhas linked at least someinfections, including fatalities, to contaminated endoscopesand other medical instruments, but their exact contribution to the overall nosocomial infection rate is unknown. In addition, contro- versy exists over the extent to which inanimate objects (e.g., stetho- scopes)and environmental surfaces (e.g., floors, walls, sink drains) that comeinto contact with intact skin can transmit infections. Lastly, many infections from inanimate objects may never be detected becauseso much time can elapsebetween infection and the onset of illness that the source of the infection is difficult to trace. Health-care providers generally use disinfectants to treat medical instruments they cannot otherwise sterilize or disposeof. For example, disinfectants are used on sensitive medical instruments, such as fiber- optic endoscopes,that cannot be sterilized using heat and on instru- ments that need to be sterilized quickly between uses.In addition, many health-care providers view the use of disinfectants on objects/surfaces that comeinto contact with intact skin as a necessarypart of infection control, in the absenceof evidencethat these objects/surfaces play a negligible role in transmitting infections, Infection control also is a serious concern in the food-processingand food-service industries, since food can be easily contaminated by dis- ease.The total amount of food-borne illness in the United States is unknown, but outbreaks occur frequently. Both industries use sanitizers on food-contact surfaces (e.g., food-processingequipment and utensils) to reduce the likelihood that food may becomecontaminated. “James M. Hughes and William R. Jarvis, “Epidemiology of Nosocomial Infections,” in Manual of Clinical Microbiology, 4th ed., ed. Edwin H. Lennett et al. (Washington, DC.: American?%i%j%r Microbiology, 1986). “Robert W. Haley, Managing Hospital Infection Control for Cost-Effectiveness (Chicago: American Hospital Publishing, Inc., 1986). Page 15 GAO/RCRD-99-139 EPA Lacks Assurance Disinfectants Work Chapter 1 Introduction We have issued two reports in the past addressingthe need to better Prior GAO Reports regulate the efficacy of disinfectants. In 1968, we reported that many pesticides subject to enforcement-seizureaction, including somedisinfec- tants that the federal government found to be ineffective, may have remained on the market7 In 1974, we questioned EPA’Sreliance on regis- trant-submitted data for registering pesticides with high rates of effi- cacy failures, such as disinfectants. We also found that EPAdid not always cancel disinfectant registrations or require registrants to delete efficacy claims from the labels of repeatedly ineffective disinfectants.R In a January 27, 1989, letter, and at subsequentmeetings with their Objectives, Scope,and staffs, the Chairman and Ranking Minority Member of the HouseCom- Methodology mittee on Government Operations, and the Chairman and Ranking Minority Member of the Environment, Energy and Natural Resources Subcommittee,HouseCommittee on Government Operations, asked us to review EPA’Sefforts to regulate the efficacy of disinfectants. In partic- ular, the Committee and Subcommitteeagreedthat we would address the following questions: What is the nature of the scientific controversies surrounding EPA-rec- on-unendedmethods for testing the efficacy of disinfectants? What action has EPAtaken to obtain objective research to resolve the scientific controversies that surround disinfectant efficacy test methods and performance standards? DoesEPAhave sufficient internal controls to ensure the quality and integrity of registrant-submitted disinfectant efficacy data? What post-registration enforcement procedures have EPAand the states implemented to ensure that disinfectants on the market are effective? Doesa need exist for a post-registration program to monitor disinfec- tants on the market and, if so, what options exist for structuring such a program? EPAbelievesthat all disinfectants are critical to protecting public health. However, we decided to concentrate our review on EPA’Sregulation of the efficacy of disinfectants registered for use on hard surfaces because over 76 percent of all disinfectants were registered for use on at least one hard-surface use site. In addition, these disinfectants are most ‘Need to Improve Regulatory Enforcement Procedures Involving Pesticides (5133192, Sept. 10, 1968). *Pesticides: Actions Needed to Protect the Consumer From Defective Products (B133192, May 23, 1974). Page 16 GAO/RCED-99-139 EPA Lacks Assurance Didnfectanti Work Chapter 1 Introduction affected by the scientific controversies surrounding the test methods used to substantiate efficacy. To understand EPA'Sefficacy data requirements and determine the nature of the scientific controversies surrounding EPA-recommended methods for testing disinfectant efficacy (seech. 2), we (1) reviewed, and interviewed EPAofficials about, EPA’Sdisinfectant efficacy data requirements and procedures for accepting test methods, procedures, and modifications; (2) observed microbiologists at two state laboratories perform efficacy tests; (3) interviewed sourcesknowledgeable about the test methods and controversies, including individuals affiliated with federal and state agencies,standard-setting organizations, professional and trade associations,scientific associations,universities, and commer- cial laboratories (see app. I); (4) attended several conferencesand meet- ings held by the AOAC, the standard-setting organization whose methods are the subject of scientific controversy, and discusseddisinfectant effi- cacy test methods and related issueswith participants; and (5) reviewed scientific and medical literature on disinfectant efficacy and efficacy test methods. To determine what steps EPAhas taken to obtain objective research to resolve the controversies (seech. 2), we (1) interviewed the EPAgrant, project, and quality assuranceofficers responsible for managing and overseeingEPA’Scooperative agreementson efficacy test research with the University of North Carolina at Chapel Hill (UNC); officials within the EPAAntimicrobial Program Branch; and the two lead UNCresearchers and the statistician participating in the cooperative agreements;(2) reviewed EPAfinancial assistanceagreement regulations, policies, and procedures; (3) reviewed EPAgrant and project officer files and ~JNC records, including financial statements, progress reports, and technical reports; (4) interviewed individuals outside EPAabout the research per- formed under the agreements;and (5) reviewed EPA’SJune 1987 strategy paper on improving its disinfectant program and updates to the strategy. We were unable to determine whether EPAhad obtained objec- tive research becauseEPAinadequately managedthe cooperative agreements. To determine whether EPAhas sufficient internal controls in place to ensure the quality and integrity of registrant-submitted disinfectant efficacy data (seech. 3), we (1) reviewed the Federal Managers’ Finan- cial Integrity Act of 1982 (FMFIA)and federal guidelines on internal con- trols, (2) reviewed EPA’S1983 through 1989 annual internal control Page 17 GAO/RCED-90-139 EPA Lacks Assurance Disinfectants Work Chapter 1 Introduction reports and other EPAdocumentson EPA'Sinternal controls, (3) inter- viewed EPAofficials to determine what procedures EPAhas implemented to ensure the quality and integrity of the efficacy data that registrants submit and whether the procedures are effective, and (4) reviewed files from EPAinspections/audits over about a 4-year period at laboratories that have conducted disinfectant efficacy studies. To determine what post-registration procedures EPAand the states had implemented to ensure that disinfectants on the market are effective (seech. 4), we (1) visited and/or interviewed officials from states that monitor the efficacy of marketed disinfectants and obtained available testing and enforcement data; (2) interviewed several officials repre- senting states that do not test disinfectants to determine why; (3) visited EPA'Sdisinfectant laboratory facility in Beltsville, Maryland and reviewed existing records from the testing program maintained at the lab; and (4) interviewed EPAofficials about EPA’Sdecision to discontinue a limited program to test disinfectants for efficacy. To determine which states monitor the efficacy of marketed disinfectants, we relied prima- rily on EPAto survey the states through the EPAregional pesticide branch chiefs. To determine whether a need exists for a post-registration program to monitor disinfectants on the market and what options exist for struc- turing such a program (seech. 5), we (1) interviewed individuals within EPA,the disinfectants industry, the health-care community, state agen- cies, and academia;(2) reviewed available disinfectant laboratory cost estimates; and (3) reviewed the history of legislative proposals to resume a disinfectant-testing program at EPA’SBeltsville laboratory. To identify the number of registered disinfectant products and claims and the amount of disinfectants sold in the United States, we used data from EPA’SPesticide Product Information System and FIFRAand Toxic SubstancesControl Act (TSCA)Enforcement System (FIFRAsec.7 annual production report data). To identify laboratories that conducted disin- fectant efficacy studies submitted to EPA,we used EPA’SPesticide Docu- ment ManagementSystem. (We will express our concernsabout the accuracy and completenessof disinfectants data in these systems in a separate letter to be issued shortly to the Administrator, EPA.) To understand the role of disinfectants in preventing diseasetransmis- sion, we (1) reviewed medical literature on disinfectants and infection Page 18 GAO/RCED-99-139 EPA Lacks Assurance Disinfectants Work Chapter 1 Introduction control, (2) interviewed representatives from the infection control com- munity and reviewed their infection control guidelines, and (3) inter- viewed infection control personnel at a large urban teaching hospital about infection control procedures and watched them demonstrate these procedures. We did not (1) attempt to resolve the scientific controversies about dis- infectant efficacy test methods by independently analyzing the validity of the methods, (2) review the efficacy of any individual disinfectant or classof disinfectants, (3) addressEPA'Sefforts to monitor disinfectant efficacy claims made in product advertising, or (4) assessEPA'Sknowl- edge and regulation of the toxicity of disinfectants. Our review was conducted from January through November 1989 and updated with information gathered through June 1990 in accordance with generally acceptedgovernment auditing standards. As requested, we did not obtain official agency commentson this report. We did, how- ever, discussthe factual content of the report with EPAofficials and have included their commentswhere appropriate. EPAofficials generally agreedwith the accuracy of the facts, but believed that as presented the report could be misread, and suggestedchangesfor presenting them. We made somerevisions to the report on the basis of EPA'Scomments and believe that the report is a fair and accurate presentation of the issues. Page 19 GAO/RCED-90-139 EPA Lacks Assurance Disinfectnuts Work J e Chapter 2 WA Lacks AssuranceThat Efficacy Test MethodsAre Valid Doctors,janitors, consumers,and others rely on EPA'Sreview and approval of registrant-submitted data for assurancethat disinfectants work as claimed. However, we found that this reliance may be unfounded because(1) scientific consensusis lacking on the validity of the test methods and standards EPArecommendsto registrants for sub- stantiating disinfectant claims and (2) EPAlacks criteria for assessing the validity of registrant-proposed test methods and standards and does not independently validate test methods before permitting their use. We question whether EPAhas adequately attempted to resolve the contro- versies. EPAinadequately managedthe limited research that it has funded to help resolve the scientific controversies, and the research has proved to be controversial itself. Further, EPAhas made limited progress in conducting additional research to resolve the controversies becauseof budget constraints. EPA'Sregulations require that pesticide registration test methods be sta- Scientific Controversy tistically reliable, generally acceptable,and reproducible. However, Over Methods and almost all of the EPA-recommended efficacy test methods and perform- Standards ante standards for the approximately 3,200 disinfectants used on hard surfaces have been embroiled in scientific controversy for over a decade.Various scientists and officials from EPA,state governments, academia,industry, commercial laboratories, scientific associations,and user groups disagreeover whether . the test methods contain uncontrollable variables that produce inconsis- tent and nonreproducible results, l the existing laboratory test methods adequately simulate performance of a disinfectant in actual use, and . the performance standards (pass/fail criteria) for existing methods are valid or should be changed. Although EPAand state officials acknowledge that the existing disinfec- tant efficacy methods and standards may need to be improved, EPAand certain state officials argue that the existing methods and standards have not been invalidated, are the only available methods/standards, and are acceptablefor registration and enforcement purposes. According to these officials, inconsistent test resu!ts on possibly margin- ally effective and ineffective products may be at least as much respon- sible for the alleged problems with disinfectant efficacy test methods and standards as the methods and standards themselves.Further, sev- eral state officials and others believe certain registrants and others have Page20 GAO/&cED4O-12O EPALa& Amuance Dbinfectm~taWork Chapter2 EPALmkaAsmranceThat EfficacyTeat Methods Are Valid raised problems with the methods and standards to divert the attention of enforcement review from products that are possibly ineffective. Wewere unable to disentangle valid criticisms of test methods and per- formance standards from possible industry self-interest or possible industry-funded research biasesbecauseof the mutually dependent relationship that exists between researchersand industry. EPAdoesnot believe that it registers ineffective disinfectants. However, we found that the validity of EPA’Srecommendeddisinfectant efficacy methods and standards has been disputed to such an extent that the credibility and use of these methods and standards to substantiate disinfectant efficacy claims have been impaired. Variabilities in Test Methods recommendedby EPAfor testing disinfectant efficacy have been Methods Alleged widely criticized by industry, academia,and others for producing highly variable results. Researchersdisagreeover the extent and causesof the variability and over how the methods can be improved or replaced. Industry and other critics have alleged that extreme variability inherent in the test methods and laboratory procedures raise doubts about whether the test results can be repeated within the samelab and whether test results can be reproduced by different labs. Specific con- troversies related to test variabilities involve questions like the fol- lowing: (1) Doesthe design of the methods or the efficacy of products and chemical compoundstested account for inconsistent results? (2) To what extent can variability in laboratory procedures,operator tech- nique and experience, and materials used be controlled or reduced?and (3) Do EPA’Srecommendedmethods, someof which were developedover 20 years ago, lack the necessaryaccuracy and precision expected of test methods today? Most criticisms have focused on the AOAC Use-Dilution Method and the AOACTuberculocidal Activity Method. In fact, the AOAC has actively con- sidered repealing the two test methods during the past 3 years because of reports that results from the test methods could not be consistently reproduced.1However, becauseof similarities in design, concernsabout test variabilities in the AOACUse-Dilution and Tuberculocidal methods ‘The AOAC repealed the Tuberculocidal Activity Method at its general meeting in September 1988. The AOAC Board of Directors reinstated the method in March 1989, following an objection raised by EPA in December 1988 that the AOAC membership voted on the basis of erroneous information presented at the 1988 meeting. Page21 GAO/RCED-99-139 EPALacksAssuranceDiainfectanteWork Chapter 2 EPA Lacke Assurance That Efficacy Test Methods Are Valid may also apply to other disinfectant efficacy tests, such as the AQAC Sporicidal Test. Ability of Lab Tests to Controversy exists over the extent to which EPA-recommended efficacy Simulate Actual Use tests should and do simulate actual use and whether they provide a suf- ficient margin of safety to allow for expected variations in actual use Questioned conditions. EPAhas generally assumedthat if a disinfectant fails to per- form as claimed under the recommendedefficacy tests conducted in a laboratory, then it will fail to perform under actual use conditions. How- ever, critics from industry, academia,and other organizations claim that the laboratory tests may not accurately predict how a disinfectant will perform in actual use becausethe surfaces, number and resistanceof microorganisms, presenceof organic matter (e.g., blood), disinfectant concentration, ambient temperature at which a disinfectant is used, and amount of time a disinfectant is exposedto a contaminated surface (referred to as “contact time”), among other things, encounteredunder actual use conditions may differ significantly from laboratory test con- ditions, In fact, someinfection control experts have advised disinfectant users to extend the contact time on EPA-registeredlabels to compensate for the unknown margin of safety in EPA-recommended efficacy tests. Central to the laboratory simulation question is the controversy over carrier-based versus suspension-baseddisinfectant efficacy test methods.”Scientists disagree over whether carrier-based methods pro- vide a greater representative link between laboratory tests and actual use than suspensiontests and whether this purported advantage is offset by an unacceptableincrease in variable test results. Further, some scientists question whether labs can bias test results by identifying and selectively using carriers with a higher probability of yielding negative results. Controversy also exists over whether microorganisms are more resistant to chemical disinfectants in carrier-based methods than in sus- pension-basedmethods and over whether one type of material used for the carriers is better than another. An industry/state ad hoc group, working under the auspicesof the AOAC, and other scientists have recently concluded that carrier-based methods can be refined to reduce variable test results by using different carrier materials (e.g., glass versus stainless steel) and more stringent laboratory procedures.How- ever, other scientists argue whether certain carrier materials, such as “Carrier methods test a product’s effectiveness against a test microorganism dried on a “carrier” (a small cylinder called a “penicylinder”). Suspension methods test a product’s effectiveness against a test microorganism suspended in solution. The purpose of the carrier is to simulate the surface on which the product would be used. Page 22 GAO/RCED-99-139 EPA Lacks Assurance Disinfectanta Work Chapter 2 EPALackr,AINIUFMC~ That EMcncy Test Methods Are Valid glass,truly represent environmental surfaces likely to be treated with disinfectants, They also question whether, even granting that these materials do approximate real surfaces, their use reducesvariability to an acceptablelevel. Most of the EPA-recommended methods for demonstrating efficacy for various types of disinfectant claims on hard surfaces are carrier-based methods. Although EPAgenerally prefers carrier-based methods because they presumably have a closer link with reality, it has accepteda few suspension-basedmethods for disinfectant claims on hard surfaces, such as for tuberculocidal claims, when it has believed that the claims would be as stringent as the claims made on the basis of carrier methods. Validity of Performance Much of the scientific and regulatory controversy over the efficacy test Standards Unknown methods focuseson EPA’Sperformance standards (pass/fail criteria) for the methods. Someindustry officials and researchersassert that the existing performance standards are invalid and need to be changed. Conversely, EPAbelieves that the standards are valid for registration purposes,even though these standards were adopted many years ago without the benefit of statistical analysesthat would be performed for new standards today. EPAplans to continue using the standards to reg- ister disinfectant efficacy claims unless investigations, confirming alle- gations that they are invalid, show that these standards need to be changed. According to someindustry members,EPA’Sexisting performance stan- dards are too stringent-in the caseof the AOACSporicidal Test, nearly impossible to achieve- and such stringency explains why certain regis- tered disinfectant claims could not be substantiated in collaborative studies and state enforcement labs. Membersof industry argue, for example, that EPA’Sstandard for a hospital disinfectant claim is arbi- trary. This standard permits only 1 failure out of each set of 60 test tubes when tested by the AOACUse-Dilution Method against three dif- ferent microorganisms. Industry membersclaim that such a standard doesnot adequately allow for the probability that other factors in the test (e.g., variation in the number of test microorganisms in each test tube, variation in the carriers) could explain why an effective disinfec- tant might sometimesfail the test. SomeEPAofficials believe membersof industry have tried to encourage EPAto relax its performance standards to avoid possible enforcement action on ineffective disinfectants. Despite the alleged variabilities in Page 23 GAO/RCED-99-139 EPA Lacks Assurance Disinfectants Work Chapter 2 EPA Lacks Assurance That Efficacy Test Methods Are Valid efficacy test results and stringency in performance standards, the data that EPAreceivesfrom registrants rarely show such variabilities or that disinfectants fail to meet existing standards. Researchersand state enforcement labs have been unable to substantiate claims for somereg- istered disinfectants. Further, according to EPA/stateofficials, certain registered disinfectants have failed state and federal enforcement tests by such a wide margin that the disinfectants tested would be judged ineffective by almost any performance standard. EPAofficials and others have suggestedthat becausesometypes and concentrations of active ingredients yield highly variable test results, disinfectants with these ingredients may need to be reformulated with greater concentrations or with different ingredients, or claim longer contact times. Industry offi- cials counter that increasing the concentration of active ingredients in their disinfectants will increaseproduct costs and toxicity and may not improve the “effectiveness” of the disinfectants in actual use. (We dis- cussthe possibility that someregistrants may have selectively sub- mitted data for registration in ch. 3 and the need for post-registration enforcement tests in ch. 4.) Disagreementalso exists over whether and how the existing perform- ance standards should be changed.Someresearchersargue that the alleged variabilities in the test methods need to be resolved before the performance standards can be modified since changing the standards to allow for test variabilities may increasethe probability that EPAwould accept an ineffective product. However, according to membersof industry, the standards can be modified on the basis of theoretical calcu- lations while the methods are being improved. EPA'Sprocessfor accepting disinfectant efficacy test methods has con- Methods Acceptance tributed to the controversies surrounding the methods. Over the years, ProcessInadequate EPAhas acceptedsomedisinfectant efficacy test methods and modifica- tions to the methods-usually to register product claims at industry’s urging-that have not been developed and acceptedby independent standard-setting organizations. EPAhas acceptedthe methods and modi- fications on the basis of internal, and in somecasesexternal, scientific peer review and regulatory judgment. However, EPAlacks criteria for assessingthe validity of proposed new test methods or significantly modified test methods. Furthermore, although EPAdoesnot indepen- dently validate methods and doesnot require registrant-proposed methods and significant modifications to undergo collaborative testing and evaluation prior to acceptance,EPAscientists told us that they cannot evaluate the validity of new methods or significant modifications Page 24 GAO/WED-99-139 EPALacka Assurance Dimlnfectante Work Chapter 2 EPA Lacks Aseurance That Efficacy Test Methods Are Valid without a laboratory evaluation. EPA’Sexperience in accepting alterna- tive methods for substantiating tuberculocidal and other claims suggests that EPAneedscriteria for assessingthe validity of registrant-proposed methods and significant modifications, including criteria for determining when laboratory data, such as from a collaborative study, are neededto demonstrate validity. Lack of Criteria to Assess Under EPAregulations, registrants may use any test method to demon- Validity strate product performance as long as the test method used meets the purpose of the test standards specified in EPA’SPesticide Assessment Guidelines and provides data of suitable quality and completenessas typified by the methods that EPArecommendsin the guidelines. EPAalso allows registrants to modify the recommendedmethods to make them suitable for a particular product claim, such as effectiveness against target microorganisms in the presenceof organic matter and hard water. However, under the guidelines, registrants are responsible for demon- strating the validity of the test method or modification selectedto sub- stantiate product efficacy. According to the Director, Office of Pesticide Programs, consistent testing with uniform and valid methods is essential for judging the comparability of efficacy test results. As noted in chapter 1, EPAhas primarily relied on the A~ACand other independent standard-setting organizations as sourcesfor the recom- mended methods published in the guidelines. However, becausethese organizations have not published methods for demonstrating all types of disinfectant efficacy claims, EPAguidelines do not contain standard methods for demonstrating all types of claims. For example, the guide- lines do not contain a standardized method for registrants to use to demonstrate that products are virucidal when used on hard surfaces. Instead, EPA’Sguidelines contain minimum specifications (parameters) with which registrants must comply to make these types of efficacy claims. EPAhas primarily relied on scientists in the Antimicrobial Program Branch to review the validity of proposed new methods and modifica- tions. In caseswhere registrants have proposed an alternative method that represents a major departure from existing EPA-approvedmethods, EPAhas consulted the scientific community, including the FIFRAScientific Advisory Panel,”the Centers for DiseaseControl (CDC),and the Food and 3The FIFFU Scientific Advisory Panel is a statutorily created panel of experts convened to review qajor pesticide decisions or regulations and to give advice to the Administrator, EPA. Page 26 GAO/RCED-99439 EPA Lack8 Amurance DMnfectantr Work Chapter 2 EPA Imka Amurauce That EfYicacy Test Methods Are Valid Drug Administration (FDA),before accepting the method. However, EPA has not developed written criteria that methods must meet for EPAto consider them valid. Further, EPAhas recognizedthat laboratory evalua- tions are sometimesneededto assessthe validity of new methods and significantly modified methods. In particular, according to EPAscientists, it is difficult for them to determine, in the absenceof laboratory evalua- tions, the extent to which new test methods and significantly modified methods are reproducible and reliable. EPAofficials argue that EPAneedsthe regulatory flexibility to accept new or alternative methods to respond to changesin infection control needs and product claims. Further, EPAofficials claim that collaborative testing of methods prior to acceptanceis a lengthy processusually requiring 2 or more years to complete and that EPAlacks the laboratory facilities and personnel to validate registrant-submitted test methods independently. However, EPAofficials acknowledge that collaborative testing of methods is scientifically ideal and that reviewing registrant- proposed methods and modifications is an insufficient mechanism for evaluating the comparability of results obtained with multiple methods for substantiating one type of efficacy claim. Problems With Multiple Since about 1976, EPAhas received conflicting reports about the effec- Tuberculocidal Methods tiveness of glutaraldehyde-based products when tested by the AQAC Tuberculocidal Activity Method-a carrier-based test. (Glutaraldehyde is one of the most widely used active ingredients for hospital disinfec- tants.) Researchersfrom a leading manufacturer of glutaraldehyde- basedproducts submitted evidenceto EPAindicating that none of six glutaraldehyde-based products tested, including the manufacturer’s product, met their label claims of killing tuberculosis bacteria in 10 or 20 minutes at 2@@(about 6&F) when tested under the AoACmethod. The researchersclaimed that glutaraldehyde was more sensitive to tempera- ture than previously thought and that variables in the method, particu- larly the carrier, led to inconsistent and erroneous results. EPAhas reported that it obtained similar inconsistenciesin its preregistration testing of tuberculocidal products using the AOACmethod at its testing facility in Beltsville, Maryland, between 1971 and 1979. In 1983, the industry researchers,also key ~o~c officials, submitted a quantitative suspensiontest, intended to replace the AOACTuberculocidal Activity Method, to EPA. In May 1986, on the basis of internal review and its analysis of two sep- arate FIFRAScientific Advisory Panel subpanel reviews, EPAconcluded Page 20 GAO/RCED-99-139 EPA Lacks Assurance Disinfectants Work chapter 2 EPA Lacka Assurance That Efficacy Test Methods Are Valid that both the AOA~ method and the proposed quantitative suspension method had merit. EPAdecided to allow registrants to choosefrom one of three testing options to substantiate tuberculocidal claims. In June 1986, EPArequired registrants of all tuberculocidal products to submit new data to support the claim using the new quantitative method, the AOA~ Tuberculocidal Activity Method, or the AOAC method with substantial modification of the contact time and/or temperature. EPAalso required registrants of certain chemical classesto submit test data from a second, independent laboratory to validate test results from the first laboratory. (In 1987, the AOACedited its tuberculocidal activity method to state that the method had not beenvalidated for glutaraldehyde-basedproducts.) As of February 1989,44 out of 144 products had satisfied EPA'Srequest for data; the remaining products either deleted their tuberculocide label claims or were suspendedor canceled.Of the 44,11 used the new method, 26 used the standard AOAC method, and 8 used the modified method. The 19 products that relied on either the new method or the modified AOAC method resulted in label claims of either use at higher temperature or different contact time or both. Although most registrants submitted data using only one of the test options, several registrants developed data using more than one test option. According to EPA,the data indicated that one glutaraldehyde- basedproduct passedboth the standard AOAC method and the quantita- tive method but at significantly different contact times. Under the stan- dard AOAC method, the product was tuberculocidal (i.e., killed 100 percent of the test bacteria) in 10 minutes at 2OC, but under the quanti- tative method, the product was tuberculocidal in 65 minutes at 2OC. Despite the disparity in test results, EPAacceptedthe label claim for the product on the basis of the standard AOACtest becauseit had allowed registrants the option of choosingwhich test to use. EPAconcluded that There is no technical information available on which to base a scientific judgment of the reason(s) for the significant differences in required contact time for tuberculocidal effectiveness. It is unknown whether the difference is: an unintended confirmation of the questionable efficacy test results alleged to occur when glutaraldehydes are tested by the standard AOAC method; attributable to some defi- ciency in the quantitative method; or attributable to other testing or product related factors. EPA has acknowledged that controlled laboratory studies are neededto comparatively evaluate the three test options. EPA has also acknowl- edgedthat the quantitative test needsto be collaboratively evaluated. Page 27 GAO/RCED-99-139 EPA Lacks Assurance Disinf’ectants Work chapter 2 EPALacksAssuranceThat EfficacyTest MethodsAre Valid Despite these limitations, EPAcontinues to allow registrants to choose from among the three options to support tuberculocidal claims. Although 7 years have passedsince the quantitative method was first submitted to EPA,the agency still lacks definitive, independent labora- tory data showing that the AOACTuberculocidal Activity Method is invalid or that the quantitative suspensionmethod is valid. In addition, the quantitative suspensiontest has yet to be collaboratively tested and approved by the A~ACbecauseof statistical design problems and lack of laboratory participation in a collaborative study. EPAhas beentrying to comparatively evaluate the AOACTuberculocidal Activity Method and the quantitative suspensiontest at its laboratory facilities in Beltsville, Maryland, but the lab has had problems with equipment and with growing the test organism. As part of a 1990 research initiative, dis- cussedbelow, EPAplans to fund research to assessthe validity of the AOACTuberculocidal Activity Method. Other Test Method The tuberculocide example is not unique. EPAhas acceptedother disin- Problems fectant efficacy methods and significant modifications to methods that have not undergone collaborative studies to validate test procedures. For example, EPAhas allowed registrants to modify test methods by adding hard water and organic matter to simulate actual use without knowing how these additions affect the validity of the methods used. EPAhas also acceptedsimilar efficacy claims for different disinfectants on the basis of multiple methods without laboratory data assessing whether the methods yield comparable results. For example, EPAallows registrants to use either a modified form of the AOACUse-Dilution Method or the AOACFungicidal Test to make fungicidal claims. However, in one case,the State of Florida tested and failed a product using the AoACFungicidal Test but could not take any enforcement action because EPAhad registered the product on the basis of passing test results the registrant had submitted using the AOACUse-Dilution Method. Although EPAhas known about the scientific controversies surrounding Limited Progress Made its recommendeddisinfectant efficacy test methods and standards for in Resolving most of the last decade,it has not made much progress in resolving the Controversies controversies, Between 1983 and 1989, EPAfunded limited research to improve disinfectant efficacy methods. However, the research did not Page28 GAO/RCED-99-139 EPALacksAssuranceDisinfectantsWork Chapter 2 EPA Lacks Atwurance That Efficacy Test Methods Am Valid fulfill EPA'Sobjectives becauseEPAinadequately managedit. Further- more, although EPAdeveloped a strategy in 1987 to improve the disinfec- tants program that included, as its most important element, proposed research on test methods, EPAhas made limited progress in completing the research becauseof budgetary constraints. A new research initiative is a step in the right direction, but more work remains to be done. ResearchNot Managed Between October 1983 and October 1989, EPAspent about $384,000 on Well two consecutivecooperative agreementswith the University of North Carolina at Chapel Hill (UNC) to update and improve the AOACUse-Dilu- tion Method, Tuberculocidal Activity Method, and Sporicidal Test.4UNC researchersinvestigated 19 presumed deficiencies in the AOACUse-Dilu- tion Method and conducted two collaborative studies to evaluate vari- abilities in that method and in a slightly modified version of it. In addition, they conducted preliminary investigations on an alternative quantitative suspensionmethod, conducted preliminary investigations on the AOACTuberculocidal Activity Method, and conducted preliminary evaluations of tuberculocidal claims of certain products, as well as other investigations. On the basis of the research conducted, UNCresearchers concluded that the AQACUse-Dilution Method and the slightly modified version were subject to extreme inter-laboratory variability, mostly due to the carrier, and should not be used for registration or enforcement purposes.The researchersrecommendedthat an alternative method, such as a quantitative suspensiontest, replace the AOACUse-Dilution Method. EPAbelieves that the UNCresearchersdid not fulfill the primary intent of the cooperative agreementsbecausethey focused on examining deficien- cies in the AOACUse-Dilution Method rather than correcting the deficien- cies to reduce variability and establish new pass/fail criteria. EPA officials strongly criticized UNC'Sresearch conclusionsbecauseof a dis- pute on the relative merits of carrier- versus suspension-baseddisinfec- tant efficacy test methods. These officials do not believe that the UNC research results definitively show that the A~AC Use-Dilution Method is valid or invalid becauseof methodological limitations in the research. They also believe that the research results might reflect deficiencies in registered products rather than irreparable problems with the method. 4The total value of the cooperative agreements was $404,340, with EPA contributing $383,709 and UNC contributing $20,631. On February 28, 1990, EPA extended the project period of the second agreement to May 23, 1990, at no cost to the federal government to allow UNC researchers time to complete their final report. Page 29 GAO/RCED-99-139 EPA Lacks Assurance Disinfectants Work Chapter 2 EPALacksAmmnceThatEfflcacyTeet MethodeAreValid In fact, EPAofficials believe that the UNC research showed that the pri- mary sourceof the alleged variability in the AOACUse-Dilution Method- the stainless steel carrier-can be mitigated by using glass carriers and has endorsedthe efforts of the ad hoc industry/state group to this end. Disappointed with the UNCresearch, EPAreduced the amount of funding under the cooperative agreementduring the last 2 years and delayed funding while deciding whether to continue the agreement. In their own defense,the UNCresearcherspointed out that EPAofficials, especially those in the Antimicrobial Program Branch, were not highly critical of the research until 198’7,when the AOACbeganto consider repealing the AOACUse-Dilution Method on the basis of the UNCresearch results. According to the UNCresearchers,EPA'Scriticisms of the research may be the reaction of someoneshooting the messengerof bad news. The researchersclaimed that EPAofficials may be unwilling to accept the prospect that the most widely used disinfectant method, upon which thousands of product claims to protect the public health are based,may be invalid. The researchersfurther assertedthat the cost and administrative burden of reregistering thousands of product claims on the basis of a new method may be prohibitive in the eyes of EPAoffi- cials who managea program already consideredto be of lower priority than that of other pesticide programs. EPA'Sinadequate managementof the UNCcooperative agreementsmay have contributed to the controversy over the research.Under EPAregu- lations, a cooperative agreementis a form of financial assistancein which EPAexpects to be substantially involved in the project funded. EPA'Sproject officer for the UNCcooperative agreements(the EPApro- gram official designatedto manageand monitor the project), also a key AOACofficial in disinfectant method development, did not appear to be substantially involved in managing and monitoring the direction and pri- ority of research at UNCto fulfill EPA'Sresearch objective. Under the terms of the cooperative agreements,the project officer, in consultation with the UNCresearchers,was responsible for determining the nature and scopeof tasks performed under the agreements. The records of the project, grant, and quality assuranceofficers show that the EPAproject officer allowed the UNC researchersto develop the work plans for the cooperative agreements.However, the project officer and other EPAofficials later criticized UNC'Sresearch direction and the scopeof work and tasks UNCperformed. For example, the project officer criticized the methodology of UNC'S secondcollaborative study on a mod- ified version of the AOACUse-Dilution Method after UNCcompleted the Page 30 Chapter 2 EPA Lacks Aeeurance That Efficacy Test Methods Arc Valid work. In addition, according to the project officer, he did not insist on the quarterly progressreports the secondcooperative agreementcalled for from UNCbecausehe did not think that sufficient funds were avail- able under the cooperative agreementfor UNCto complete these reports. The dispute between EPAand UNCover the research may have been com- pounded by a lack of communication and coordination within EPAand a lack of top managementinvolvement in the early years of the coopera- tive agreements.The EPAproject officer for the cooperative agreements worked in the Biological and Economic Analysis Division. This division is separate from the Registration Division, which contains the Antimi- crobial Program Branch. Although the UNCresearch was intended to support the Antimicrobial Program Branch, which registers disinfec- tants, it appears that the two divisions did not communicate on the scopeof work and results of the research,since the Antimicrobial Pro- gram Branch did not comment on the cooperative agreementresearch until 1987. EPAand UNChave also disputed other issuesunder the cooperative agreements.EPAhas criticized UNC'Sresearchersfor investigating the claimed tuberculocidal efficacy of individual products rather than inves- tigating the validity of the AOACTuberculocidal Activity Method. EPAhas also criticized the UNCresearchersfor publicizing the results of their research on tuberculocidal disinfectants before sharing them with EPAas required in the secondcooperative agreement.In both situations, how- ever, it appears that EPAofficials did not closely manageand monitor the cooperative agreement and expresseddissatisfaction with the researchersand research results only after the work was completed and publicized. We do not know whether EPA'Scriticisms of UNC’Sresearch are valid, reflect reasonabledifferences in scientific judgment, or reflect a defen- sive position and lack of acceptanceof the research results. However, if EPAis ever to resolve the scientific controversies that surround disinfec- tant efficacy test methods and performance standards, it must do a better job of managing the research. Limited Progress in In responseto mounting criticisms from industry, the public-health com- Conducting Planned munity, disinfectant users, the Congress,and others, EPAdeveloped a strategy to improve its regulation of disinfectants in 1987, which pro- Research posed research on the methods and standards EPArecommends.How- ever, EPAhas made limited progress in conducting this research because Page 31 GAO/RCED-90-139 EPA Lacks Assurance Disinfectants Work Chapter 2 EPA Lacks Aasurance That Efficacy Test Methods Are Valid of budget constraints. A recent EPAresearch initiative is a step in the right direction, but it will be several years before EPAfully addressesthe controversies surrounding disinfectant efficacy methods and standards. In addition, the regulation of disinfectant efficacy may be hampered by the lack of an adequate, federal laboratory facility to conduct disinfec- tant efficacy method research and development. In 1986, EPAconveneda multi-office work group to analyze alleged problems with disinfectant efficacy test methods and also hired a micro- biologist consultant to provide expertise on the issues.The work group identified deficiencies in several critical areas of the disinfectants pro- gram, including those areas embroiled in the scientific controversies dis- cussedabove, and explored several approachesfor resolving the deficiencies that would not depend on a large-scale,federally operated testing program. The work group concludedthat The most critical and most recognized deficiency is the current lack of credibility of the standard efficacy test methods utilized in the registration program, particularly the AOAC test procedures. Any attempt to improve the [disinfectant] efficacy pro- gram is contingent upon re-establishing the credibility of the existing methods; updating/revising the existing methods; or developing new methods. To resolve the deficiencies, the work group developed a five-point strategy that included improving or replacing existing efficacy methods. The work group estimated that EPA would need about $1.6 million in contract funds to evaluate the methods. Between 1987 and 1990, EPA made limited progress in conducting its planned research becauseof competing program priorities and budget constraints, according to the Chief, Antimicrobial Program Branch. According to a June 1990 strategy update, “Becauseof budget con- straints, EPA’S past efforts in conducting evaluations of current test methods have been limited.” Recognizingits resource limitations, EPA challenged industry, user groups, and others to coordinate resourcesto achieve mutual research goals. In responseto this challenge,the aforementioned ad hoc industry/ state group, funded by individual participating laboratories, conducted a collaborative study of a new hard-surface carrier test in 1989. EPA offi- cials believe that the study results show that the test, which they con- sider to be a modified version of the AOAC Use-Dilution Method, is reproducible and reliable. EPA officials expect that the ad hoc group will present its final study report to the AOAC in September 1990 and that the Page 32 GAO/RCED-90-139 EPA Lacks Assurance Disinfectanta Work Chapter 2 EPA Lacka Asmmnce That EfYicacy Test Methods Are Valid AOAC will consider replacing the existing AOACUse-Dilution Method with the new method. EPAwill consider what regulatory actions, if any, to take on the basis of the new test after it is consideredby the AOACand after additional research is completed on the effects of using hard water and organic matter in the method and on the appropriate performance standard for the method, according to the Chief, Antimicrobial Program Branch. In April 1990, EPAannouncedthat it plans to support three cooperative agreementsover a 2-year project period to conduct research and devel- opment on validating, revising, or replacing the AoACTuberculocidal Activity, AOACSporicidal, and EPAvirucidal test methods and perform- ance standards. EPAplans to spend approximately $600,000 in fiscal year 1990 funds on this research.EPAplans to award the cooperative agreementsby September30, 1990. Although EPA'Sfunding initiative represents a substantial increase in commitment to resolve the scientific controversies,the Chief of the Anti- microbial Program Branch believes that the initial funding will be insuf- ficient to resolve the controversies completely. She estimates that EPA may need to spend an additional $1.2 million to completely research the alleged problems with the methods and standards. If the new research initiative is not well managed,EPA,industry, disin- fectant users, consumers,and the public health will be no better off than they are today. EPAhas designatedthe Antimicrobial Program Branch, within the Registration Division, as the project office for the new coop- erative agreementsinstead of the Biological and Economic Effects Divi- sion, which was the project office for the UNCcooperative agreements. Further, EPAhas announcedthat it “intends to be involved in the test methodology research and development by approving work by stages, approving any subcontracts, conducting on-site visits/inspections at rea- sonable intervals, co-authoring published reports relative to the funded study, and halting research activity if the intent, approach, or antici- pated phasesleading to the accomplishment of the study are not being achieved, or have been revised without prior Agency approval.” Although this approach may resolve someof the managementproblems EPAexperienced with the UNC cooperative agreements,EPAmight want to request that the FIFRAScientific Advisory Panel provide advice on the research direction and results. EPA'S1987 strategy paper called for the panel’s assistanceto help evaluate research priorities, among other things, but as of June 1990, EPAhad not convenedthe panel for this Page 33 GAO/RCED-99-139 EPA Lacks Assurance Disinfectants Work Chapter 2 EPA Lacks Aseurance That Eff¶cacy Test Methods Are Valid purpose. In addition to guidance on research needsand priorities, the FIFRAScientific Advisory Panel could provide independent confirmation of the need for any regulatory changesthat might be indicated by the research,such as the need to reregister disinfectants on the basis of new methods and standards. An EPAdecision to involve the FIFRAScientific Advisory Panel in this manner would be consistent with the internal control principle of separating key functions in a transaction to ensure that effective checks and balancesexist. Many of the disinfectant efficacy test methods in dispute were devel- oped by scientists working at the federal government’s laboratory facili- ties in Beltsville, Maryland. EPAdiscontinued testing disinfectants at Beltsville in 1982 primarily becauseof budget constraints. Although EPA kept the laboratory open to assist in a limited capacity on method evalu- ation, EPAofficials readily admit that EPAlacks the personnel and facili- ties to do the necessarymethodology research.According to the current laboratory supervisor, EPA’SBeltsville laboratory facilities would not pass EPA’Sown good laboratory practice (GLP) regulations. (Ch. 5 dis- cussesoptions for a disinfectant laboratory.) Disinfectants are used to reduce the risk of transmitting infectious dis- Conclusions eases.If disinfectants fail to work as claimed, then those using disinfec- tants in restaurants, child day care centers, hospitals, homes, and other places may be placing themselves and others at risk and wasting their money. Doctors,janitors, consumers,and others rely on EPA’Sregistration of dis- infectants as assurancethat these products work. However, scientific controversy exists over the validity of the methods and standards that EPArecommendsthat registrants follow in order to substantiate claims. Although EPAand state officials have acceptedallegations that there may be problems with the methods and standards, they believe that the existing methods and standards are acceptablefor registration and enforcement purposes. Nonetheless,the scientific controversies over the adequacy of these methods and standards have impaired the credibility of EPA’Sregistration of disinfectant efficacy claims. Although disinfectant efficacy test methods and performance standards have been embroiled in scientific controversies for over a decade,EPA has made limited progress in resolving them becauseof inadequate research managementand budget constraints. EPAdid not manageand monitor its cooperative agreementswith UNC well becauseit lacked Page 34 GAO/RCED-90-139 EPA Lacks Assurance Disinfectants Work Chapter 2 EPA J.mke Assurance That EfYicacy Teat Methods Are Valid internal communication and agreementon research tasks and results. EPAalso has made limited progress in conducting additional research becauseof budget constraints. In April 1990, EPAannouncedthat it planned to spend approximately $600,000 for research on the scientific controversies surrounding certain disinfectant efficacy methods over a Z-year period. Although this initiative is significant, more research, requiring time and additional resources,will be needed. Although we do not expect EPAto ever eliminate all scientific disputes over these methods and standards, the agency could better protect the public health by raising the degreeof certainty about the validity of dis- infectant efficacy methods and standards. A detailed plan that would describe a research strategy to resolve the controversies surrounding the existing disinfectant efficacy test methods and performance stan- dards would help guide this effort by establishing milestones and cost estimates. In addition, the FIFF~AScientific Advisory Panel could assist in developing the plan and overseeingEPA'Sresearch direction and management. EPA'Sprocessfor accepting and recommending disinfectant efficacy test methods and standards has contributed to the scientific controversies surrounding methods and standards. EPAhas acceptedmethods and modifications to methods without independent laboratory data that demonstrate the validity of the procedures or standards. Furthermore, EPAlacks criteria for assessingthe validity of new methods and any sig- nificant modifications to methods. EPAcould better ensure that future registrant-proposed disinfectant efficacy methods and modifications are valid by establishing specific criteria for ascertaining validity, including criteria for determining when independent laboratory data, such as col- laborative study data, are neededto assessthe validity of proposed methods or modifications. To increasethe degreeof certainty that disinfectant efficacy test Recommendations methods and standards are valid, we recommendthat the Adminis- trator, EPA,develop a detailed plan, including cost estimates and mile- stones,to resolve the controversies surrounding existing methods and standards. The plan should include a research strategy that addresses problems with the alleged variability in test methods, adequacy of lab tests to simulate actual use, and the validity of performance standards, as discussedin this chapter. Further, we recommendthat the Adminis- trator, EPA,convenethe FIFRAScientific Advisory Panel to assist in developing the plan and overseeingthe research strategy direction and Page 35 GAO/RCED-99-139 EPA Lacks Assurance Diinfectmts Work Chapter 2 EPA Lacka Assurance That Efficmy Test Methods Are Valid management.(Seech. 6 for options on establishing a laboratory facility to assist in researching and developing disinfectant efficacy test methods.) In addition, we recommendthat the Administrator, EPA, develop and publish a policy that establishesspecific criteria for evaluating the validity of new disinfectant efficacy test methods and modifications to methods, including criteria for determining when independent labora- tory data, such as data from a collaborative study, are neededto demon- strate the validity of proposed methods and modifications. Page 36 GAO/RCED-99-139 EPA Lacka Assurance Dbinfectants Work Chapter 3 Controls Over Quality/Integrity of Efficacy Data Questionable Even with improvements in its programs to ensure the validity of the test methods used to support the efficacy of disinfectants, EPAwould continue to lack assurancethat disinfectants work as claimed. EPArelies on registrant-submitted efficacy data to make decisionsabout whether to register individual disinfectants for specific claims and uses,and on its data review, lab inspection, and data audit programs to ensure the quality and integrity of the data. However, we found weaknessesin these programs that EPAneedsto address.We also found that a preregis- tration-testing program is neededto supplement these programs because they generally do not enable EPAto identify casesin which registrants have selectively submitted incomplete disinfectant efficacy data or have deliberately submitted invalid data. We believe EPA'Slack of sufficient control over the quality and integrity of registrant-submitted disinfec- tant efficacy data is a material weaknessin EPA'Sinternal controls that should be, but has not been, reported to the President, as required by the Federal Managers’ Financial Integrity Act of 1982 (FMFIA). EPAreviews registrant-submitted disinfectant efficacy data to determine WeaknessesImpair whether proposed label claims are supported by the data and whether Data and Lab Reviews registrants have made any major mistakes in performing efficacy tests, interpreting test results, or translating the results into label claims, according to the Head, Efficacy Evaluation and Technical Management Section, Antimicrobial Program Branch. In addition, EPAinspects the labs that generatethe data and audits laboratory study records to assess the quality and integrity of the data and the competenceof the laborato- ries that have performed the studies. However, we found that: (1) EPA has inspected/audited only about 10 percent of all of the labs that per- formed disinfectant efficacy studies submitted to EPAover about a 4- year period and has identified only about 13 percent of all the labs that performed these studies; (2) EPAinspectors and auditors may be unable to evaluate adequately the capability of labs to perform these studies; (3) EPAhas been slow to prepare and processreports from inspections and audits at labs performing these studies; and (4) EPAlacks program guidance for conducting data reviews, lab inspections, and data audits relating to these studies. Not All Labs Inspected The Laboratory Data Integrity Assurance Division (LDIAD), within the Office of Compliance Monitoring (OCM), is responsible for, among other Y things, inspecting laboratories that perform studies submitted to EPAto support pesticide registrations and for auditing these studies. The objec- tives of good laboratory practice (GLP) inspections are to ensure that a Page 37 GAO/RCED-99-139 EPA Lacks Aeeurance Didnfectants Work Chapter 3 Controls Over Quality/Integrity of EMcacy Data Questionable lab follows specific test methods, adheresto standard operating proce- dures, keeps records in compliance with GLPregulations, and conforms to required safety and quality assuranceprocedures so that lab prac- tices do not compromise the quality and integrity of data submitted to EPAfor registration purp0ses.l The objectives of data audits are to ensure that the data underlying a study are both present at the lab and fully substantiate the study results by comparing the data from the study submitted to EPAwith records on the study in the lab. A data auditor typically audits a study at the sametime that an inspector inspects the lab that performed the study. LDIAD'Sgoals are to inspect all labs performing efficacy studies approximately every 2 years, audit one or more efficacy studies from each lab inspected, and inspect high- volume labs first. Between January 1,1986, and September30,1989, LDIAD conducted 13 lab inspections and 14 data audits at 9 individual labs that performed disinfectant efficacy tests. These 9 labs represent only about 10 percent of the 92 labs that had generated disinfectant efficacy data received by EPAbetween January 1,1985, and June 26,1989. The 9 labs that LDIAD inspected generated an estimated 40 percent of the 1,148 disinfectant efficacy studies that EPAreceived during that period. LDIAD audited 109 studies at the 9 labs. LDIAD did not inspect most of the labs that had performed disinfectant efficacy studies. In fact, LDUD identified only 12 (about 13 percent) of the 92 labs that had performed these studies. LDIAD was unaware of most of the labs, including somehigh-volume labs, becauseit had not used EPA’S Pesticide Document ManagementSystem (PDMS)to identify labs for inspections/audits, This system is a central archive primarily consisting of documents that registrants have submitted to EPAto sup- port pesticide registrations, and we used it to identify labs that had per- formed disinfectant efficacy studies submitted to EPA. Lab Capability Not Existing EPA program guidance specifies that lab inspections are per- Adequately Assessed formed while a study is in progress,that they provide the inspector with an opportunity to observe laboratory techniques, and that they ensure that labs follow specific test methods correctly. Various officials from ‘In August 1989, EPA published GLP regulations governing efficacy studies. These regulations became effective on October 16, 1989. Before this date, EPA could not enforce good laboratory prac- tices of laboratories that performed disinfectant efficacy tests. Page 38 GAO/RCJZD-90-139 EPA Lacks Assurance lMdnfectants Work Chapter 9 Controls Over QuaRty/Integrlty of EfTleaey Data Questionable EPA,the states that operate disinfectant efficacy testing programs, com- mercial labs, and industry, as well as the AOAC,have stated that the results of a disinfectant efficacy test are very sensitive to small varia- tions in the way the technician performs the test procedures and that lab personnel must have extensive experienceto perform disinfectant efficacy tests correctly. However, according to a June 1987 EPAstrategy document, many of the labs testing disinfectants for efficacy do not follow prescribed, standard efficacy test methods. Although the tests are operator-sensitive, lab inspectors and data audi- tors generally do not observe disinfectant efficacy tests in progress becauseEPAhas no means of identifying most of the studies ultimately submitted to EPAwhile they are still in progress, according to the Chief, Scientific Support Branch, LDIAD. In addition, the branch chief told us that disinfectant efficacy tests generally are so short in duration that it is difficult for LDIADto schedule an inspection at a lab performing such a test before the test is complete. OneEPAefficacy data auditor told us that inspectors and auditors do not usually seelabs performing efficacy tests becausemost labs do not run efficacy tests on a continuous basis. The five EPAdata auditors who have audited disinfectant efficacy studies, as well as state lab officials and researchers,told us that a check sample program could ensure the quality of the data registrants submit better than data audits can. EPAcurrently requires states that have cooperative pesticide enforcement agreementswith EPAto partici- pate in a check sample program. Under the program, EPAsendsthe state labs samplesof pesticide formulations and residues for analysis to ensure that they perform analytical tests correctly, EPAcould adopt a similar program for laboratories that conduct disinfectant efficacy studies to ensure that they are capable of performing efficacy tests cor- rectly. Under the program, EPAwould send samplesof formulations of known efficacy to a lab, ask the lab to test the samplesfor efficacy, then check the lab’s results against the known efficacy of the samples. Page 39 GAO/RCED-99-139 EPA Lacks Assurance Disinfectants Work Chapter 3 Controls Over QuaRty/Integrlty of Efficacy Data Cjueetionable ProcessingInspection/ LDIAD’S target time frame for conducting inspections and audits and Audit Reports Takes processingreports is 112 days (about 3.6 months). However, LDIAD had, as of November 13,1989, taken 639 days (about 1.6 years) on average Nearly Five Times Longer to perform and processreports from inspections/audits at labs per- Than Estimated forming disinfectant efficacy studies.2The times ranged from 161 to 963 days (about 6 months to over 2.6 years). The 6 inspection/audit reports that remained open as of November 13,1989, had been open an average of 360 days. The longest report had been open over 1.6 years since the lab was first inspected. LDIADhas been slow in processingthese lab inspection and data audit reports for three reasons,according to the Chief, Scientific Support Branch. First, most lab inspectors and data auditors are not directly accountableto LDIAD. Instead, most lab inspectors are stationed in EPA’S regional offices and have other noninspection-related duties that the 1 regional offices consider of higher priority than the duty of preparing lab inspection reports. Second,according to this official, until recently, the EPA headquarters staff available to perform inspections and process inspection/audit reports at EPA headquarters was too small. Third, EPA did not have enforceable GLPregulations for efficacy studies until October 16, 1989, so LDIAD consideredprocessinginspection/audit reports relating to labs performing these studies to be a low priority. The Director, LDIAD, acknowledged that EPA has been slow in processing inspection/audit reports related to disinfectant efficacy studies as well as other types of registration studies. To addressthe problem, LDIAD is developing a simplified reporting format and a procedure for processing those reports that indicate a violation first. Program Guidance Lacking Scientists in the Antimicrobial Program Branch review registrant-sub- mitted reports summarizing the results of efficacy tests before regis- tering disinfectant efficacy claims. As of June 1990, EPA had not completed final guidelines for these scientists to use in conducting effi- cacy data reviews, although it had prepared guidelines for performing reviews of most other types of pesticide registration data (e.g., chemical, toxicological, environmental, and ecological data). With such guidelines, EPA could better ensure that its reviewers identify all potential problems with efficacy data. EPA could not estimate when final guidelines would be published. ‘This finding is based on the 9 of 10 inspection/audit reports EPA had prepared and completely processed as of November 13, 1989, and for which EPA could provide information. Page 40 GAO/RCED-90.139 EPA Lacks Assurance Disinfectants Work Ckapter 3 Cmtrob Over Qufdlty/Integrity of Effhwy Data Questionable Similarly, as of June 1990, EPAhad not developedguidance for con- ducting inspections and data audits at labs that have performed disin- fectant efficacy studies. Instead, according to the Chief, Scientific Support Branch, LDIAD, inspectors have beenusing EPA'Smanual gov- erning inspections at labs that have performed health-effects studies. According to this official, EPA'Sefficacy data auditors have relied on their professional judgment to perform the audits, (Although two of the auditors have developedquestionnaires for use in conducting efficacy data audits, not all five use them.) In addition, as of June 1990, EPAhad not published guidelines specifying the types of inspection/audit findings that would prompt registration and/or enforcement action by EPAand the type of action EPAshould take in each case.Although the Registration Division has developed a stan- dard operating procedure for managing the review and disposition of inspection/audit reports, the procedure doesnot specify criteria for evaluating what registration action, if any, the division should take on the basis of report findings. An LDIAD work group has developed interim guidelines on taking enforcement action that are undergoing internal review. Even if the weaknesseswe found in EPA'Sdata review, lab inspection, Preregistration and data audit programs were corrected, the programs generally would Testing Neededto not enable EPAto identify casesin which registrants have selectively Ensure Data Integrity submitted incomplete disinfectant efficacy data. EPA'Sposition is that registrants are required by FIFFWto submit all data indicating that a dis- infectant may not be effective as registered when registrants are aware that such data exist.” Evidence exists that someregistrants have sub- mitted to EPAefficacy test data indicating that their disinfectants work but may have withheld other test data indicating that these disinfec- tants do not work as claimed. Furthermore, if a registrant deliberately “Section 6(a)(2) of FIFRA states: “If at any time after the registration of a pesticide the registrant has additional factual information regarding unreasonable adverse effects on the environment of the pesticide, he shall submit such information to the Administrator.” 7 U.S.C. sec. 136d(a)(2). As interpreted by EPA, “unreasonable adverse effects on the environment” include information con- cerning the efficacy of disinfectants. Under an EPA regulation, 40 C.F.R. sec. 16260(f)(3), an appli- cant for registration also must submit any information that would be required under sec. 6(a)(2) if the product were registered. Page 41 GAO/RCED-99-139 EPA Lacks Assurance Disinfectants Work Chapter 3 Controls Over Quality/Integrity of Efficacy Data Questionable submitted invalid data to EPA,or a commercial lab deliberately sub- mitted invalid data to a registrant, EPA'Sdata reviewers, lab inspectors, and data auditors probably would be unable to tell. Although EPAmaintains that FIFFZAprohibits registrants from submitting selective data, EPA-recommended methods for testing disinfectant effi- cacy provide registrants with an opportunity to submit efficacy data selectively. Given the alleged variability in certain disinfectant efficacy tests (seech. Z), a registrant and/or testing facility could run an efficacy test repeatedly until the formulation tested passed,and the registrant could submit only the passing results to EPA(i.e., a registrant could submit selective data to EPA). Despite this opportunity, EPA'Sexisting programs for validating efficacy data generally do not enable EPAto recognizeselective data. According to the Head, Efficacy Evaluation and Technical ManagementSection, Antimicrobial Program Branch, data reviewers cannot identify situa- tions in which EPAhas received selective data becausethe reviewer sees only what was submitted rather than all tests conducted on a disinfec- tant. According to EPA’Sfive efficacy data auditors, data auditors gener- ally cannot identify casesin which registration data are selective. The auditors told us that they rely on labs to identify the records they should audit, and labs could provide them with selective records. During the course of our review, we found someevidencethat regis- trants have submitted selective data to EPA.For example, one data auditor found evidenceof selective data during a data audit-passing and failing data on a high-volume, household product whose registration file contained only passing data. According to the data auditor, he found the set of failure data by chance.The data auditor who discovered the set of failure data and other efficacy data auditors told us that they would have no way of knowing if a registrant withheld data from an audit. In addition, representatives of two registrants told us that they had obtained variable (both pass and fail) efficacy results on disinfec- tants but had submitted only passing results. The belief that registrants submit selective data to EPAis widespread. The Chief, EPAAntimicrobial Program Branch, and the Head of the branch’s Efficacy Evaluation and Technical ManagementSection told us they believe that registrants submit selective data, and EPA'SJune 1990 disinfectant program strategy paper stated that “the practice of not reporting failing/adverse test results is widespread.” In addition, other EPAofficials, state officials, the UNCresearchers,and others all told us Page 42 GAO/RCED-99-139 EPA Lacks Assurance Disinfectants Work Chapter 3 Controla Over Quality/Integrity of Efficacy Data Questionable that they believe someregistrants submit selective data. In fact, mem- bers of the disinfectant industry openly joked about submitting selective data to EPAat a widely attended national meeting on disinfectant effi- cacy test methods that we and EPAofficials attended in 1989. As noted in chapter 2, registrants have argued that the existing test methods yield variable results and the performance standards are too stringent. However, these criticisms, and the fact that EPArarely receivesdata showing that disinfectants do not work, suggestthe possibility that someregistrants could have submitted selective data to EPAto register their products. Even if the variabilities in the test methods were reduced to limit a reg- istrant’s opportunity to submit selective data, neither lab inspectors nor data auditors can practically observethese tests in progress, and no physical evidence(e.g., test tubes) remains from the tests once they are completed. As a result, data reviewers and auditors must rely on the registrant’s word about the procedures followed in a test, the disinfec- tant formulation tested, and the test results, according to EPAofficials. In the event that a registrant or lab facility were deliberately to deviate from an EPA-approvedefficacy test method (e.g., by running the test at a higher temperature than reported); run a test with a disinfectant formu- lation that would pass the test, rather than the formulation the regis- trant planned to market; or record that a disinfectant formulation passedan efficacy test when the formulation failed, data reviewers or auditors probably would not be in a position to identify the deviation and question the test results. Although a program to test disinfectants before registering them could resolve these issues,EPAdoesnot currently operate such a program. At one time, EPAoperated a limited preregistration-testing program to verify sporicidal claims and selectedtuberculocidal claims. EPAdiscon- tinued preregistration tuberculocidal testing in 1979 becauseof inconsis- tent test results and discontinued all other preregistration tests in 1982. EPA’Srecords from the testing program show that at least someregis- trants changedproposed efficacy claims on the basis of EPA’Stests to make them more protective (e.g., lengthened contact times). For example, one registrant lengthened its proposed exposure period for the sporicidal efficacy of a disinfectant from 6 hours to 10 hours in 1974 after the disinfectant failed a preregistration test. EPAwould not need to test all disinfectants for efficacy before registering them. Instead, EPA could target such a program to those disinfectant claims of greatest importance to public health, such as sporicidal claims, to those products Page 43 GAO/RCED-90-139 EPA Lacks Assurance Disinfectants Work Ckapter 3 Controla Over Quality/Integrity of EfYlcacy Data Queetionable with suspectedefficacy problems, and/or to other products as deter- mined by the Administrator. EPA'Sinability to ensure the quality and integrity of registrant-submitted Internal Controls disinfectant efficacy data is, in our opinion, a material weaknessin Inadequate internal controls under FMFIA. Under FMFIA, each federal agency must establish and maintain a system of internal controls to provide reason- able assurancethat, among other things, program and administrative activities are effectively managedto achieve the goals of the agency. GAOand the Office of Managementand Budget guidance on internal con- trols require agenciesto identify, in writing, objectives for each agency activity that are specific, complete, logical, and applicable to the specific activity; as well as techniques that will provide reasonableassurance that the objectives will be accomplished.In addition, federal agencies are required to identify, in an annual report to the President, material weaknessesin internal controls. A material weaknessexists in an agency’sinternal control systems when, among other things, the agency lacks reasonableassurancethat the objectives of the system are being accomplishedand that the weaknesswould significantly impair the ful- fillment of the agency’smission and/or would deprive the public of a neededservice. EPArelies on registrant-submitted data to support disinfectant efficacy Conclusions claims. As noted in chapter 2, we found that the validity of disinfectant efficacy methods and performance standards has been questioned and that EPA'Sprocessfor accepting test methods is inadequate. However, even if EPAaddressedthese problems, we believe that EPAstill would lack sufficient controls to ensure the quality and integrity of registrant- submitted disinfectant efficacy data. We found a number of weaknessesin the data review, lab inspection, and data audit programs. First, EPAhas not identified all labs that have performed disinfectant efficacy studies to be inspected/audited. EPA should use the Pesticide Document ManagementSystem which, though limited, contains the best available information for identifying these labs. Second,EPA'Slab inspection and data audit programs are unable to completely assessthe capabilities of labs that perform disinfectant effi- cacy tests becausethese tests are operator-sensitive and inspectors/ auditors generally do not observethem in progress.A check sample pro- gram could provide EPAwith greater assurancethat laboratories that Page 44 GAO/RCED-90-139 EPA Lacks Assurance DMnhctant.s Work Chapter 3 ~mol# Over Cluallty/Integrity of Efficacy Data Que&hable perform these tests for registrants are capable of performing them cor- rectly. Third, EPA has been slow to prepare and processreports from inspections/audits related to disinfectant efficacy studies. The timeli- nessof these reports is more important now that EPA'SGw regulations encompassdisinfectant efficacy studies. A review of OCM'Sinternal con- trols for ensuring that inspections/audits are processedon time could prevent problems with timeliness in the future. Fourth, EPAhas not pub- lished guidelines neededto ensure that data reviewers, lab inspectors, and data auditors identify all potential problems with disinfectant effi- cacy studies and that EPAtakes appropriate registration and/or enforce- ment action. Although EPAcan improve its data review, lab inspection, and data audit programs, we believe that the only way for EPAto determine whether a registrant has submitted selective data or has deliberately submitted invalid data is by testing the product. Evidence exists to suggestthat, in at least somecases,pre-registration government tests to verify sporicidal efficacy data led to more protective label claims. We are not suggestingthat EPAtest all disinfectants for efficacy before registering them. Instead, EPAcould target such a program to those disinfectant claims of greatest importance to public health and/or products with sus- pected efficacy problems. Data reviews, lab inspections, and data audits cannot ensure the integrity of registrant-submitted data. A preregistra- tion-testing program would complement those activities and better ensure the integrity of the data. We believe that EPA'Slack of sufficient controls to ensure the quality and integrity of registrant-submitted disinfectant efficacy studies is a mate- rial weaknessin EPA'Sinternal controls that should be, but has not been, reported to the President as required by FMFIA. Until this weaknessis corrected, EPAmay be compromising public health by registering ineffec- tive products. To improve EPAcontrols over the quality and integrity of registrant-sub- Recommendations mitted data, we recommendthat the Administrator, EPA,implement a preregistration-testing program to verify selecteddisinfectant efficacy data. (Seech. 6 for a discussionof options for establishing a laboratory facility to assist in such a program.) The Administrator could target pre- registration tests on those claims that are of the greatest public health significance and/or products with suspectedefficacy problems. Page 46 GAO/lUXtMM%139 EPA La&t Assurance Disinfectanta Work Chapter 3 Controls Over Quality/Integrity of Efficacy Data Questionable To improve the effectiveness of the data review, lab inspection, and data audit programs, we recommendthat the Administrator, EPA, l direct the Laboratory Data Integrity Assurance Division to identify all laboratories that have performed efficacy studies submitted to EPAto support disinfectant registrations and meet the division’s goal of inspecting these labs at least every 2 years (at a minimum, direct LDIAD to use the Office of Pesticide Programs Pesticide Document Management System, which contains the best available information for identifying the labs); l direct LDIADto establish a check sample program as part of the lab inspection program to better assessthe ability of labs to perform disin- fectant efficacy tests; l direct the Office of Compliance Monitoring to review its internal con- trols for ensuring that inspections/audits are processedon time (for example, ensure that inspectors/auditors are held accountablein their performance standards and appraisals for meeting processingtime frames); and l direct the Office of Pesticide Programs and the Office of Compliance Monitoring to develop and implement specific guidance for data reviewers, lab inspectors, and data auditors to follow; further, direct these offices to develop, publish for comment, and implement detailed policies and guidelines to decide what registration and/or enforcement action to take on the basis of findings from lab inspections and data audits. We recommendthat in his next annual internal control report to the President, the Administrator, EPA,report the lack of sufficient controls to ensure the quality and integrity of registrant-submitted disinfectant efficacy data as a material weakness.We also recommendthat the Administrator, EPA,include in his report a plan delineating specific cor- rective actions and time frames. Page 46 GAO/RCED-99-139 EPA Lacks Assurance Disinfectanta Work Limited Monitoring/E of Registered Disidectmts Even were EPAto implement improvements in its processesfor regis- tering efficacy claims (e.g., independent validation of test methods and preregistration testing), these improvements would not provide suffi- cient assurancethat disinfectants on the market were effective. Regis- trants could, intentionally or inadvertently, manufacture and sell ineffective batches of disinfectants after registering them. EPAlacks assurancethat, once registered, disinfectants work as claimed because EPAstopped monitoring disinfectants on the market for efficacy in 1982. Becausestates and disinfectant users generally do not monitor disinfec- tants for efficacy, EPAis relying, in effect, on the industry to regulate itself. Despite the scientific controversies over efficacy test results, his- torical enforcement and other data suggestthat as many as 20 percent of the disinfectant batches on the market do not work as claimed. As a result, disinfectant users may be placing themselves and others at risk from infection and spending money unnecessarily. Until 1982, EPAoperated a limited post-registration testing program to Federal Enforcement verify certain efficacy claims of marketed disinfectants at its laboratory Testing Discontinued facilities in Beltsville, Maryland. According to the best available infor- mation, the program was limited in size, scope,and operation. EPAdis- continued the program becauseof budget constraints and expected that states, users, and the disinfectants industry would assumemost of its responsibility for ensuring the efficacy of marketed disinfectants.’ From 1970, after EPAwas established and took over the pesticide pro- grams from the U.S. Department of Agriculture, through 1982, the Belts- ville laboratory tested, on average,207 disinfectant samples annually. However, the number of samplestested each year dropped substantially after 1972. From 1970 through 1972, the laboratory tested an average of 610 samples annually. By contrast, from 1973 through 1982, the labo- ratory tested an averageof 127 samples annually, testing only 47 sam- ples in 1982. Furthermore, the Beltsville laboratory did not verify all types of efficacy claims. According to the microbiologist at the labora- tory, the laboratory tested no products from the marketplace for sporicidal claims and, after the mid-19709, few products for use on sur- faces other than hard surfaces. ‘Although EPA discontinued post-registration testing, it kept the Beltsville laboratory open for use on an as-needed basis (e.g., to perform confirmatory efficacy tests on disinfectant enforcement samples from the states). Page 47 GAO/RCED-99-139 EPA Lacks Assurance Dbhfectan~ Work Chapter 4 Limtted Monitoring/Enforcement of Registered Msinfectanta According to EPAcongressionaltestimony, EPAdiscontinued its post-reg- istration testing program primarily because(1) the level of testing was inadequate and was creating a false senseof security among users and the public about the efficacy of disinfectants on the market and (2) budget constraints prevented EPAfrom conducting what it consideredto be an adequate level of testing. EPAofficials believed that by discontin- uing the testing, EPAcould direct more time and effort to evaluating and improving the test methods. EPAofficials believed that the states, the user community, and the disinfectants industry would take a more active role in monitoring the efficacy of disinfectants on the market and that EPAcould establish a system for processingcomplaints from these sourcesabout actual or suspectedproduct failures. However, EPAhad little reason to expect that states and disinfectant users would assumeEPA’Sresponsibility for monitoring disinfectant effi- cacy or to assumethat a complaint system would be developedbecause of a lack of EPAleadership. Few states and no users were testing at the time, and no others expressedan interest in testing. In addition, without a laboratory facility or provision to contract for laboratory services,EPA had no mechanism for channelling and verifying complaints about potentially ineffective disinfectants from the states, user groups, and the industry and for initiating appropriate enforcement action against disinfectants found to be ineffective. Although EPAdiscontinued its efficacy-monitoring program in 1982, Nonfederal Testing nonfederal monitoring efforts remain limited. As of July 1989, we were Limited able to identify only three states- Florida, North Carolina, and Missis- sippi-that were monitoring the efficacy of disinfectants in the market- place under their pesticide enforcement programs and only two states- Wisconsin and Virginia- that were testing disinfectants under other state programs. Furthermore, we did not identify any users or related groups, including the health-care, restaurant, or food-processingindus- tries, that have implemented comprehensive,routine monitoring pro- grams. In effect, EPAhas left the burden of monitoring disinfectant efficacy to the industry itself. State Testing Limited Although Florida, North Carolina, and Mississippi monitor the efficacy of disinfectants on the market, the scopeof their testing programs and ii authority of their enforcement programs are limited, and the amount of testing they have performed has decreasedin recent years. In addition, Page 48 GAO/RCED-99-139 EPA Lacks Assurance Disinfectants Work Chapter 4 Limited Monitdng/Enforcement of Registered Dhilnfectan~ scientific controversies surrounding the test methods have affected their efforts to enforce the efficacy of disinfectants on the market. The types and amounts of enforcement testing the states perform is lim- ited. Florida, North Carolina, and Mississippi do not verify all types of disinfectant efficacy claims. For example, none tests products for sporicidal, tuberculocidal, or virucidal efficacy. According to officials in these states, the tests are too time-consuming and expensive. Further- more, the total amount of enforcement testing these states have per- formed has been dropping overall since EPAdiscontinued its testing program. The total number of samplesthe states tested decreasedfrom 962 in 1986 to 500 in 1989. (Seefig. 4.1.) The state of Mississippi has virtually discontinued testing. In fiscal year 1989,the state tested only 14 disinfectant samplesfor efficacy. Page 49 GAO/RCED-99-139 EPA Lacks Assurance Disinfectants Work Chapter4 Limited Monitoring/Enforcement of Registered Disinfectanta Figure 4.1: Dlsinfectant Enforcement Samples Tested by North Carolina, Entorcomont Samples Florida, and Mississippi, 1983-89 1909 900 so0 700 Boo so0 400 Boo 200 199 0 1993 1994 199s 1996 1997 1999 1999 Yoan t-l MiSSiSSippi Florida NorthCarolina Florida and Mississippi data are for fiscal years (July 1 through June 30). NO&I Carolina data are for calendar years. Source: Prepared by GAO on the basis of data from the states of North Carolina, Florida, and Mississippi. To someextent, the scientific controversies surrounding the test methods and performance standards have affected the states’ efforts to enforce the efficacy of disinfectants on the market. For example, the state of Florida has not attempted to issue complaints against any regis- trant whose disinfectants have failed efficacy tests since 1986. According to the Director, Florida Division of Inspection, the state does not feel comfortable basing legal action on results from efficacy tests until controversies surrounding the AOACUse-Dilution Method are resolved. Although Florida, North Carolina, and Mississippi have tested and failed at least somedisinfectants registered for use in other states, states do Page SO GAO/RCED-90-139 EPA Lacks Assurance Disinfectants Work - Chapter 4 Limited Monitoring/Enforcement of Registered Dieinfecta.nts not have authority to regulate the efficacy of disinfectants in the mar- ketplace outside of their boundaries. In at least two cases,registrants have respondedto state enforcement action by recalling disinfectants from the marketplace and disinfectant users within the state or by deleting specific efficacy claims from disinfectant labels, but have con- tinued to market the disinfectants or make the claims in other states. In one case,a producer agreed to recall from a single state market a phe- no&based disinfectant marketed under nine different brand names after the product failed state efficacy tests. However, the disinfectant, which is registered for use on floors, walls, and other hard surfaces in hospitals, nurseries, rest rooms, telephone booths, and elsewhere, con- tinues to be sold in all other states, according to the registrant’s cus- tomer service representative. In the other case,the registrant of two nationally marketed disinfectants removed claims that the products were effective in hard water from the product labels in one state after one of the products failed state efficacy tests. However, the registrant continues to market the two products (one of which is for use on floors, walls, basins, and on instruments in hospitals, nursing homes,schools, and medical and dental offices) with the claims in other states, according to the registrant’s customer service representative. States other than Florida, North Carolina, and Mississippi have been reluctant to start testing programs. We interviewed officials from sev- eral of the states with high pesticide enforcement funding from EPAand/ or a high number of disinfectant producers located within their bounda- ries. The most common reason they offered for not operating testing programs was that they do not have the resourcesneededto establish and operate a testing facility. Although EPAhas provided most states with funding for pesticide enforcement activities, the amount of funding decreasedby about 25 percent in real terms (i.e., after adjusting for changesin price levels) from federal fiscal year 1983 to 1989, and, according to the Chief, EPAGrants and Evaluation Branch, EPAhas not required the states to make disinfectant efficacy testing a pesticide enforcement priority. Instead of a testing program, most of these state officials said they rely on EPA'Sjudgment in registering disinfectants as assurancethat they are effective. The states of Wisconsin and Virginia operate testing programs but not under their pesticide enforcement programs. The Wisconsin Department of Health and Social Servicesrequires the efficacy of sanitizers intended for use on food contact surfaces (except those containing inorganic hypochlorites) to be tested under its program to regulate the milk and restaurant industries. The Commonwealth of Virginia discontinued an Page 61 GAO/RCED-90-139 EPA Lacks Assurance Disinfectants Work Chapter 4 Limited Monitoring/Enforcement of Registered Disinfectants enforcement testing program sometime around 1982 because,according to the Supervisor, Virginia Office of Pesticide Regulation, the state found that the AOACUse-Dilution Method did not yield results that were reproducible enough to take enforcement action against registrants whose samples failed the test. Despite problems with the method, the state has continued to test disinfectant samplesthat manufacturers submit in order to compete for state contracts for sometypes of efficacy claims. User Programs Not Individuals and organizations both within and outside of the health-care Established industry told us that they were not aware of any hospitals, doctors, or dentists that test disinfectants for efficacy. According to several health- care representatives, hospitals and other users do not have the resourcesand/or the expertise necessaryto operate a program to mon- itor the efficacy of disinfectants on the market. Instead, they rely on EPA'Sregistration of disinfectants as assurancethat disinfectants on the market are effective. Furthermore, restaurants and food-processing facilities do not test the sanitizers they use for efficacy, according to representatives from these industries. Someresearchersaffiliated with users (as well as others) have tested disinfectants, or are planning to test disinfectants, but have not estab- lished monitoring programs. For example, since 1976, Clinical Research Associates,an independent research organization dedicated to evalu- ating dental materials, devices,and concepts,has been testing the effi- cacy of selecteddisinfectants used by dentists. As of February 1990, we had identified two organizations, the American Dental Association (ADA) and the National Sanitation Foundation (NSF), that were considering testing disinfectants for efficacy. However, if ADA were to test disinfec- tants, ADA would only test tuberculocidal disinfectants and would not test products routinely. Moreover, both programs would depend on vol- untary participation by registrants. Although several individuals and organizations affiliated with disinfec- tant users have complained or submitted data to EPAindicating that spe- cific disinfectants do not work as claimed, EPAhas not established the system for processingthese complaints that it proposed as part of its 1987 disinfectant program improvement strategy, Furthermore, EPAhas not yet addressedthe question of how it would verify the complaints and take appropriate enforcement action against disinfectants found to be ineffective. Page 52 GAO/RCED-90-139 EPA Lacks Assurance Disinfectants Work Chapter 4 Lhhd Monitoring/Enforcement of Registered Msinfectanta Industry Self-Regulation The burden of monitoring the efficacy of disinfectants has fallen on the Limited industry itself becauseEPAno longer tests disinfectants, few states test and their programs are limited, users do not generally test, and EPAlacks an effective complaint system. However, market forces cannot be relied upon to control disinfectant efficacy problems becauseusers cannot visually identify ineffective products. Furthermore, although regis- trants can test competitors’ products and have challenged competitors’ claims, EPAhas been unable to resolve conflicting claims becauseit lacks the laboratory facilities necessaryto do so, according to the Chief, Anti- microbial Program Branch. Consequently, industry self-regulation appears limited. A June 1990 update to EPA’SJune 1987 strategy paper on improving the disinfectant program acknowledgedthat someregis- trants make unregistered and/or exaggeratedefficacy claims for disin- fectants and that someregistrants market unregistered disinfectants. SomeMarketed Although the scientific controversies surrounding disinfectant efficacy test results discussedin chapter 2 cloud the issue somewhat, historical Disinfectants May Be data on specific products from EPA and the states, along with data from Ineffective industry, academia,and other sources,indicate that somedisinfectants on the market are ineffective. Although the true percentageof ineffec- tive disinfectants on the market is unknown, EPAacknowledged in 1983 that as many as 20 percent of marketed disinfectants may be ineffec- tive. Consequently,public health and consumerwelfare may be at risk from disinfectants that do not work as claimed. Historical test-failure data indicate that EPAand the states have been unable to substantiate disinfectant claims for certain products. Between 1978 and 1982,the last 6 years during which EPAtested disinfectants, an averageof 42 percent of all disinfectant samplestested by the lab failed efficacy tests. In the years since the EPAlab closed(1983-89), 22 percent, 4 percent, and 2 percent of all disinfectant samplestested by the states of Florida, North Carolina, and Mississippi, respectively, have failed efficacy tests. During 1988 and 1989, about 30 percent of all sanitizer samplestested under the state of Wisconsin program failed, and about 40 percent of the samplestested by the Commonwealth of Virginia failed. Although many products have failed disinfectant efficacy tests in EPA’S and the states’ laboratories, the data generally are not comparable. More specifically, EPAand the states have used different schemesfor sampling disinfectants from the marketplace, used different performance stan- dards to assessefficacy, and tested different types of claims. For Page 63 GAO/RCED-99-139 EPA Lacks Assurance Disinfectants Work Chapter 4 Limited Monitoring/Enforcement of Registered Disinfectante example, Florida has focused on sampling and testing disinfectants that frequently fail efficacy tests and/or are intended for hospital use, whereas Mississippi has tested products at random from the market- place. North Carolina and Florida have used pass-fail criteria for disin- fectants in efficacy tests that are more lenient than EPA'Sregistration standards. Mississippi has not tested disinfectants for efficacy in the presenceof organic matter, while Florida has. Virginia tests disinfec- tants to determine whether they are effective in hard water and in the presenceof organic matter regardlessof whether the products are regis- tered with those claims. While the data from state and federal enforcement testing are not com- parable, they suggestthat an unknown number of disinfectants on the market may be ineffective. Somedisinfectants have failed enforcement tests by a wide margin. At least two states have found disinfectants con- taminated with bacteria. Somedisinfectants have failed efficacy tests repeatedly in a single state, in multiple regulatory labs (state and/or EPA), or tests performed by registrants themselves at regulatory labs. For example, a disinfectant registered for use on floors, walls, showers, and other surfaces in hospitals, nursing homes,and schoolsfailed effi- cacy tests on multiple occasionsin EPA'Slab and two state laboratories. Similarly, a disinfectant registered for use in hospitals, nursing homes, food-processingfacilities, office buildings, schoolsand recreation facili- ties failed 28 efficacy tests in one state from 1983 to 1988. In addition, data from industry, academia,and others also raise ques- tions about whether somedisinfectant batches in the marketplace work as claimed on their labels. For example, in a collaborative study of the AOACUse-Dilution Method, three of four disinfectant manufacturers’ lab- oratories unknowingly tested and failed their own products. While several officials from EPA,industry, and the states, and most others we talked to believe that disinfectants on the market are gener- ally effective, various sourcesestimate that between 5 to 20 percent of disinfectants on the market do not work as claimed. EPAofficials, them- selves,believe that somedisinfectants on the market may be ineffective. In 1983, the Director of the Compliance Monitoring Staff (now the Office of Compliance Monitoring) estimated that a random sample of disinfec- tants would reveal a failure rate of up to 20 percent. He concluded that, “This is an unacceptable rate of failure for products with direct public health significance.” In a 1983 congressionalhearing, the Director of the Office of Pesticide Programs stated that, given the types of failure rates the Beltsville laboratory had found, it was apparent that many batches Page 54 GAO/RCED-90-139 EPALacks Assurance Disinfectants Work Chapter 4 Limited MonitoriugjEnforcement of Regiet4wed Disinfectanta of ineffective products had reached the marketplace with the limited level of testing conducted, Furthermore, EPAand state officials, as noted in chapter 2, have stated that the conflicting test results obtained in research on disinfectant efficacy test methods may result as much from ineffective products as from problems with the methods themselves. EPAlacks assurancethat all disinfectants on the market work as Conclusions claimed. After a disinfectant is registered, its formulation could be altered, intentionally or inadvertently, and ineffective batches intro- duced into the marketplace. Although EPAhas acknowledgedthis possi- bility, the agency discontinued its limited testing and enforcement program in 1982. Becausestate and user monitoring efforts are limited, EPA’Sdecision to stop enforcement testing was, in effect, a decision to let industry regulate the efficacy of marketed disinfectants. However, EPA lacks a formal system to channel complaints about potentially ineffec- tive disinfectants from competitors, users, and others, and to take appropriate enforcement action against ineffective disinfectants. Greater leadership on EPA’Spart to develop a strategy that pools resourcesfrom the states, user groups, industry and others to identify potentially ineffective disinfectants and that specifies the appropriate enforcement action against disinfectants found to be ineffective, would provide better assurancethat disinfectants on the market work as claimed. Without such an enforcement strategy, EI’A’Spolicy of “let the buyer beware” for disinfectants may be compromising public health and consumer protection. While we recognizethat EPAneedsto resolve the scientific controversies that surround disinfectant efficacy test methods and performance standards (discussedin ch. 2), these controversies should not prevent EPAfrom establishing and implementing an enforce- ment strategy to ensure the efficacy of disinfectants in the marketplace. We recommendthat the Administrator, EPA,develop, publish for com- Recommendations ment, and implement an enforcement strategy to ensure that marketed disinfectants work as claimed. This strategy should specify (1) the mechanismsand procedures for identifying potentially ineffective disin- fectants; (2) the procedures for investigating and verifying complaints about potentially ineffective disinfectants, including, where necessary, the use of independent laboratory testing; and (3) the criteria and proce- dures for initiating registration and/or enforcement action against disin- fectants found to be ineffective. In light of federal budget constraints, we also recommendthat the Administrator explore options for pooling resourcesfrom the states, user groups, and industry to implement a Page 65 GAO/RCED-90-139 EPA Lacks Assurance Disinfectants Work Chapter 4 Limtted Monitoring/Enforcement of RegisteredDiehuectantB national disinfectant efficacy enforcement strategy. (Seech. 6 for a dis- cussionof options for establishing a laboratory facility to assist in an enforcement program.) Page 56 GAO/RCED-99-139 EPA Lacks Assurance Disinfectants Work Chapter 6 Altemtives for a Disinfectant Laboratory Many people in industry, the state regulatory laboratories, universities, scientific associations,user groups, and other organizations believe an independent laboratory facility is neededto research and selectively test the efficacy of disinfectants. Somebelieve that EPA'Sefforts to resolve the scientific controversies that surround disinfectant efficacy test methods may languish without a federal laboratory facility. EPAofficials have objected to such a facility because(1) EPAcannot afford to resume methodology research and testing disinfectants at its limited laboratory facility in Beltsville, Maryland, given its limited budget and competing pesticide program priorities, and (2) they do not think the federal gov- ernment should test these products. However, fees charged for the privi- lege of obtaining a disinfectant registration could help offset the costs of a disinfectant laboratory facility. Also, public health may be endangered without such a resource. This chapter briefly identifies possible alternatives for a laboratory facility, considering (1) the need for and objectives of a laboratory facility, (2) suggestedcriteria to evaluate alternatives, (3) a list of pos- sible alternatives, and (4) the option of fees. The chapter is not a cost/ benefit assessmentof various alternatives for a disinfectant laboratory facility. Rather, the purpose of this chapter is to briefly present the pros and cons of the alternatives. Previous chapters in this report discussedthe need for a laboratory Need for and facility to research and test the efficacy of disinfectants. Although their Objectives of a opinions differ, various critics of EPA'Sregulation of disinfectants from Disinfectant industry, academia,user groups, and other organizations have collec- tively identified five objectives, listed below, of a disinfectant labora- Laboratory tory facility, . Researchand development: the facility should research and develop dis- infectant efficacy test methods and performance standards that are accurate and precise, objectively evaluate the validity of registrant-pro- posed alternative methods and modifications, and actively participate in the scientific exchangeof information. Preregistration tests: the facility should conduct a preregistration- testing program to verify selective disinfectant efficacy claims and ensure the quality and integrity of registrant-submitted data. Laboratory development: the facility should improve and standardize laboratory procedures,promote good laboratory practices, train labora- tory technicians, and conduct a check sample program to improve the Page 67 GAO/RCED-99-139 EPA Lacks Assurance Disinfectants Work Chapter 6 Altemathe for a Dbinfect.ant Laboratory capability of commercial and other laboratories conducting disinfectant efficacy tests. . Post-registration tests: the facility should conduct a post-registration testing program to verify selective disinfectant efficacy claims in the marketplace and assist in enforcement cases. . Referencelaboratory: the facility should serve as the reference labora- tory on questions of test methodology and procedures,referee disputes, and confirm test sample results from state government laboratories. With respect to the testing functions, the laboratory facility would not need to test every disinfectant product or claim. Proponents of a disin- fectant laboratory facility do not support the idea of routine batch- testing to duplicate or replace manufacturers’ quality control programs. Rather, the laboratory could focus its efforts on those product claims that are of the greatest significance to public health, such as sporicidal and tuberculocidal claims, randomly check the efficacy of disinfectants in the marketplace, and selectively target those products or chemicals with suspectedefficacy problems. Many factors or criteria could be used to evaluate alternatives for SuggestedCriteria for meeting the objectives of a disinfectant laboratory facility listed above. Evaluating However, we suggestfour criteria for evaluating alternatives: (1) inde- Alternatives pendence,(2) authority, (3) quality, and (4) cost. The need for a laboratory facility that is independent has been demon- strated in the scientific controversies surrounding disinfectant efficacy test methods. Market forces and competitive pressuresmake it difficult to ascertain the validity of scientific disputes. A laboratory facility that provides information to on the validity of disinfectant efficacy test EPA methods and performance standards should be relatively isolated from the profit motive of the industry. The facility should be above reproach to allegations of conflict of interest or the appearanceof such conflict. An independent laboratory facility needsthe authority to obtain and test product samples,validate test methods, verify registrants’ claims, establish laboratory procedures, and referee disputes. The laboratory facility and personnel should be of high quality. The lab- oratory would be unable to restore credibility in disinfectant regulation if its equipment and its personnel were not at the forefront of microbi- ology and infection control. Page 68 GAO/RCED-90-139 EPA Lacks Assurance Diiinfectants Work Chapter5 Alt.ematlves for a Disinfectant Laboratory Although all of the criteria consideredimply cost tradeoffs, the effi- ciency and practicality of each alternative must be considered.We have not prepared detailed cost estimates for each alternative; rather, we have consideredwhether the alternatives appear efficient and practical. Several alternatives or a combination of alternatives could be developed Alternatives for a for operating a laboratory facility to test the efficacy of disinfectants Laboratory Facility and perform other functions. However, we suggestfour alternatives: (1) the federal government, (2) a commercial laboratory, (3) state govern- ment(s), and (4) a nonprofit association or foundation. Many proponents of a disinfectant laboratory facility support the idea of the federal government operating the laboratory, particularly of its resuming disinfectant testing in Beltsville, Maryland. A federally oper- ated facility would satisfy the criterion of independence.EPA,the most likely agencyto operate such a lab, has the authority to run it, but is not required to do so under FIFR,A. Legislation has been consideredin pre- vious congressionalsessionsto require EPAto establish standards for disinfectants. Quality and cost would be major considerations in estab- lishing a federal laboratory, especially in resuming testing at the Belts- ville facility. Both EPAand its critics acknowledge that EPAwould need new, highly skilled personnel to operate a testing facility. In addition, EPA'Sexisting facility would need extensive renovation becausesomeof the equipment and supplies at the lab are obsolete and in disrepair. Cost estimates for establishing a federal disinfectant laboratory range from $200,000 to $2 million; annual operating cost estimates range from $300,000 to $600,000, depending on the size and scopeof operations. Although we were unable to find official historical budget documents from EPA'Sdisinfectant laboratory, various records in the files indicate that EPAmay have spent upwards of $300,000 in fiscal year 1981, the year before the laboratory stopped testing disinfectants. This amount may not cover the costs of a federal laboratory today becauseof infla- tion and becausea testing program today may be larger in scopeand size than the one conducted by the facility in 1981. Although EPAcurrently is responsible for registering and enforcing the efficacy of these products, several proponents of a federally operated disinfectant laboratory have suggestedthat the cnc or the FDAoperate the laboratory either alone or in cooperation with EPA.Critics of EPA argue that it lacks personnel with the expertise in infection control prac- tices necessaryto effectively regulate disinfectants. They also argue Page 69 GAO/RCED-90-139 EPA Lacks Amurance Disinfectants Work Chapter 5 Alternatives for a Disinfectant Laboratory that the disinfectant program will always be of lower priority than other pesticide programs within EPAand that perhaps it is a program that is misplaced. Rather than operate its own disinfectant laboratory, EPAcould contract for the necessarylaboratory serviceswith a commercial facility. This arrangement might offer EPAmaximum flexibility in directing the focus of work each year. In addition, since the private sector currently retains the best available scientific and technical personnel and laboratory equipment, EPAmight be able to contract for the quality of services needed.However, this alternative might sacrifice independenceif the contract laboratory has or has had a closelink with the regulated industry. In addition, EPAhas been criticized frequently by the Congress for contracting out too many of its responsibilities. (We did not estimate how much it would cost for EPAto contract out for a disinfectant laboratory.) A third alternative for a disinfectant laboratory is for the states to operate their own laboratories. However, this alternative has already proven unsuccessful.Most states do not operate a disinfectant labora- tory today and are unlikely to open labs in the near future becauseof cost constraints. Further, while the states would be independent, they would lack the necessaryauthority to test prdducts not sold in their state. Users and consumerswould be unfairly disadvantaged if their states were unable to afford a laboratory facility while neighboring states could afford one. In addition, having 60 states research improve- ments to disinfectant efficacy test methods and test disinfectants would be grossly inefficient. A fourth alternative could be for a nonprofit associationor foundation to operate a disinfectant laboratory. As discussedin chapter 4, the American Dental Association and the National Sanitation Foundation are considering whether and when to establish their own testing pro- grams. While these efforts to help protect public health are commend- able, they may not be able to achieve the objectives of a disinfectants laboratory outlined above becausethey depend on registrants to volun- tarily cooperate and submit their products for testing. In addition, these organizations lack authority to take any registration and/or enforce- ment actions on the basis of their tests. Page 00 GAO/RCED-99-139 EPA Lacks Amwrame Dihfectants Work Chapter 5 Alternatives for a Disinfectant Laboratory EPAhas objected to resuming testing disinfectants at its Beltsville labora- Feesfor a Disinfectant tory, in part, becauseof a lack of resources,However, fees might be one Laboratory way to help offset the costs of a disinfectant laboratory. Currently, the 1988 amendmentsto FIFRA(known as FIFRA‘88) prohibit EPAfrom charging such fees. In addition, fees, which industry has opposed,pre- sent other obstaclesthat need to be considered. EPA’Sprimary pesticide program objective has been to implement FIFRA ‘88, which involves reviewing the risks and benefits of all pesticides first registered before November 1, 1984-a processknown as “reregis- tration.” FIFRA‘88 imposesa one-time reregistration fee and an annual maintenance fee to help offset the costs of accelerating reregistration and expediting new registrations. The FIFRA‘88 reregistration fees are $150,000 per active ingredient but are waived for small volume active ingredients (including small volume disinfectant active ingredients) and reduced for small businesses,The annual maintenance fee is designedto provide about $14 million each year. The FIFRA‘88 fees do not explicitly provide funds for operating a disinfectant laboratory. Furthermore, FIFRA ‘88 prohibits EPAfrom charging other fees until September30, 1997, when the FIFRA‘88 fee provisions expire. Consequently, the Con- gresswould have to amend FIFRA‘88 to allow EPAto charge disinfectant registrants a fee specifically designatedfor operating a disinfectant laboratory. The industry has opposeduser fees for several reasonsbut mostly becauseit views product registrations as a public rather than private benefit. However, charging registrants user fees to finance a disinfec- tant laboratory, in order for EPAto recover the costs of testing their products, is analogousto charging user fees for EPAto acceleratereregis- tration of older pesticides and expedite registration of new pesticides. Furthermore, although fees would increasethe cost of registering and marketing disinfectants, if registrants want to market disinfectants it seemsfair that they finance the cost of demonstrating that their prod- ucts work as claimed. Earlier chapters of this report have discussedthe need for a laboratory Conclusions facility to research methods for testing disinfectant efficacy and to test the efficacy of disinfectants. Several alternative ways exist for oper- Y ating a disinfectant laboratory, but EPAlacks the resourcesneededto finance such a laboratory. However, fees on disinfectant registrations might be one way to help finance a disinfectant laboratory. Becauseof the statutory prohibition on new pesticide registration fees and the Page 61 GAO/RCED-99-139 EPA Lacks Ammxnce Disinf’ectant~ Work Chapter 5 Alternatives for a DMdect.ant Laboratory unknown effects that disinfectant fees might have on the marketplace, what is neededis a detailed cost/benefit assessmentof alternatives for a disinfectant laboratory, including the option of assessingfees to help finance such a facility. We recommendthat the Administrator, EPA,develop a detailed cost/ben- Recommendations efit analysis of alternatives for operating a laboratory facility to research and test the efficacy of disinfectants, including the option of charging fees to register disinfectants to help finance such a facility, and submit the results of its analysis to the Congressso that the Congress may weigh the advantagesand disadvantagesof various alternatives. Page 62 GAO/RCED-90-139 EPA Lacks Assurance DMnfectanta Work Page 63 Aonendix I Affiliations of Individuals GAO Contacted Not all individuals GAOcontacted officially representedthese organiza- tions, but they were employed by or affiliated with these organizations. Someof the individuals contacted from nongovernmental organizations were also employeesof disinfectant registrants. Federal Agencies Centers for DiseaseControl Environmental Protection Agency Food and Drug Administration State Governments California Florida Georgia Illinois Iowa Mississippi New York North Carolina Texas Virginia Wisconsin Standard-Setting American Society for Testing and Materials Organizations Association of Official Analytical Chemists National Sanitation Foundation Professional and Trade American Dental Association Associations American Hospital Association Association for Practitioners in Infection Control Chemical Specialties Manufacturers Association Joint Commissionon Accreditation of Healthcare Organizations National Environmental Health Association National Food ProcessorsAssociation National Restaurant Association Scientific Associations American Society for Microbiology Clinical ResearchAssociates Page 64 GAO/RCRD-99-139 EPA Lacks Assurance Disinfectanta Work . AffUiatio~~ of Indlviduala GAO Contacted Universities University of Missouri-Columbia University of North Carolina at Chapel Hill Commercial Laboratories Gibraltar Biological Laboratories, Inc. Hill Top Biolabs, Inc. Y Page 65 GAO/RCED-99-139 EPA Lacb Assurance DMnfectanta Work Appendix II Major Contributors to This Report Peter F. Guerrero, Associate Director Resources, J. Kevin Donohue,Assistant Director Community, and William M. Layden, Evaluator-in-Charge Economic Karen Simpson,Staff Evaluator Scott W. Weaver,Staff Evaluator Development Division, Sarah-Ann Moessbauer,Operations ResearchAnalyst Washington, D.C. Doreen StolzenbergFeldman, Senior Attorney Office of the General Counsel, Washington, DC. Page 66 GAO/RCED-90-139 EPA La&s Arsmrance MeMectante Work Giossary Accuracy The closenessof an observedresult to the true or acceptedresult. Antimicrobial Pesticides With someexceptions, substancesand mixtures of substances,intended for inhibiting the growth of, or destroying any bacteria, fungi patho- genic to people and other animals; or viruses declared to be pests and existing in any environment. (For purposes of this report, we have referred to all antimicrobial pesticides for public-health use as “disinfectants.“) Bacteria Small microorganisms with a relatively primitive cellular organization. Collaborative Study A study involving a number of laboratories analyzing the samesamples by the samemethod for the purpose of generating performance data on the method when a competent analyst usesit exactly as written. (Per- formance data include any values that indicate the reliability, applica- bility, and practicability that can be expected from the method.) Disinfectant As used in this report, any pesticide used on inanimate surfaces or objects and intended to inhibit or destroy bacteria, fungi, or spores causing human disease. Efficacy The capacity of a pesticide product when used according to label direc- tions to control the target pest. (As used in this report, the term “effi- cacy” is synonymous with the terms “product performance“ and “effectiveness.“) Fungicide As defined in this report, a disinfectant intended to destroy fungi. Fungi A group of organisms devoid of chlorophyll that cannot manufacture their own food. Nosocomial Infection An infection that occurs during or sometimesafter hospitalization and was not present or incubating at the time of the patient’s admission. Page 07 GAO/RCED-90-139 EPA Lacks Assurance Diiinfectante Work Glossary Precision Agreement among repeat observations made under the sameconditions. Reliability A criterion used to evaluate the validity of a method by measuring the method’s ability, when used by qualified analysts, to produce data of a predictable degreeof precision and accuracy. Repeatability The variability in successiveresults obtained with the samemethod on identical test material and under the sameconditions (same operator, same apparatus, samelaboratory, and sametime). Reproducibility The variability in individual results obtained with the samemethod on identical test material but under different conditions (different operator, different apparatus, different laboratory, and/or different time). Sterilizer/Sporicide As defined in this report, a disinfectant intended to destroy or eliminate viruses and all living bacteria, fungi, and their spores. Tuberculocide An agent that is intended to destroy or inactivate tuberculosis bacteria. Viricide An agent that is intended to destroy or inactivate one or more speciesof virus. Virus Any of a group of submicroscopicinfective agentsthat are regarded either as simple microorganisms or as complex molecules.(Viruses are capable of growth and multiplication only in connection with living cells.) (onerm) Page 69 GAO/RCED-90-139 EPA Lacks Assurance Disinfectants Work Orders may also t)tl placed by calling (202) 2756241. i
Disinfectants: EPA Lacks Assurance They Work
Published by the Government Accountability Office on 1990-08-30.
Below is a raw (and likely hideous) rendition of the original report. (PDF)