GAO svpt ~‘llli~(‘t’ l!b!H) DISINFECTANTS Concerns Over the Integrity of EPA’s Data Bases 142460 United States General Accounting Office Washington, DC. 20648 Resources, Community, and Economic Development Division R-2408 11 September 21, 1990 The Honorable William K. Reilly Administrator Environmental Protection Agency Dear Mr. Reilly: As you know, the Environmental Protection Agency (EPA) relies on pesti- cide data systems to make a variety of program management, budg- etary, enforcement, and regulatory decisions about pesticides, including disinfectants,’ and for providing information about pesticides to the Congress, industry, and the public. In our August 1990 report,2 we men- tioned our concerns about the accuracy and completeness of disinfectant data in these systems. This report explains these concerns, specifically in terms of three systems: (1) the Pesticide Product Information System ' (PPIS), (2) the Pesticide Document Management System (PDMS), and (3) the Federal Insecticide, Fungicide, and Rodenticide Act and Toxic Sub- stances Control Act Enforcement System (FATES). Although we did not evaluate EPA'S management of these three com- Results in Brief puter-based systems, we nonetheless identified several problems with the integrity of the data in these systems, problems that may limit the extent to which the data can support EPA'S disinfectant program and its managers. Specifically, the systems contained inaccurate and/or incom- plete data or were missing data on disinfectants. For example, although 1~1sis intended to include essential regulatory data on disinfectant product claims, as much as 60 percent of the data in the system may be inaccurate or incomplete, according to some EPA officials. Furthermore, although intended to capture relevant information, the systems could not be used to identify all types of disinfectants and their product per- formance claims. Although these problems could adversely affect EPA'S regulation of disinfectants, and perhaps the regulation of other pesti- cides, EPA has not fully addressed these problems. ‘As defined by EPA, “disinfectant” refers to only one of several types of antimicrobial pesticides, which, with some exceptions, are substances intended to inhibit or destroy microorganisms (bacteria, fungi, viruses, and spores). However, we use the term “disinfectant” in this report to broadly describe all antimicrobial pesticides intended to protect public health. ‘Disinfectants: EPA Lacks Assurance They Work (GAO/RCED-90-139, Aug. 30,199O). Page 1 GAO/RCED-90-232 EPA’s Data on DMnfectanta B-240811 Under the Federal Insecticide, Fungicide, and Rodenticide Act, as Background amended (FIFRA), EPA generally must register (license) pesticides, including disinfectants, before they may be sold, held for sale, or distrib- uted in commerce. Before registering a pesticide, EPA must determine that it is effective, when used as directed, without causing an unreason- able risk to public health or the environment. To make this determina- tion, EPA requires registrants to submit data on health and environmental effects and, in the case of disinfectants, on product per- formance (efficacy). Because EPA has not evaluated most pesticides used today against current testing requirements, the agency is now in the process of reregistering older pesticides, including disinfectants, on the basis of these requirements. Within EPA, the Office of Pesticides and Toxic Substances (OPTS) regu- lates disinfectants and other pesticides. EPA developed PPIS, PDMS, and FATES for use in performing pesticide registration and FIFRA compliance activities. PPIS is intended to capture basic data on pesticide product registrations, including data on product labels, such as the product’s name and regis- tration number, the name and address of the product’s registrant, the product’s active ingredient(s), the product’s pesticide type, and the uses for which the product is registered. The Office of Pesticide Programs (OPP) uses data from PPIS for a variety of purposes, including the identi- fication of registered pesticide products for which additional data are needed before they can be reregistered. In addition, whenever new evi- dence suggests that an ingredient contained in a pesticide may cause sig- nificant public health or environmental concerns, OPP uses PPIS to identify the products that contain the ingredient so that the agency may conduct a detailed analysis of the risks and benefits of continued regis- tration of such products. PDMS serves as a central archive of the documents (referred to as studies) that registrants have submitted to EPA to support pesticide registrations. The system contains a catalogue of bibliographic informa- tion about each document under a Master Record Identifier (MRID) number. This information includes, among other things, the registration number and active ingredient(s) of the product to which the document pertains and the name and location of the laboratory that generated the data cited in the document. The Information Services Branch, OPP, main- tains the system. OPP currently uses the system for several purposes, one of which is to locate studies on health and environmental effects. These studies are needed to make decisions on whether to reregister pesticides. Page 2 GAO/RCED@O-232 EPA’s Data on Disinfectanta R-240811 On the basis of a recommendation we made in our recent disinfectants report, the Office of Compliance Monitoring (OCM) may also begin using PDMS to identify labs to inspect and studies to audit under its good labo- ratory practices program. FATES contains data on, among other things, the types and amounts of pesticides, including disinfectants, produced during the current year and the amounts sold or distributed during the previous year. Section 7 of FIFRA and EPA regulations generally require certain establishments to report this information to EPA. Specifically, establishments that produce pesticide products for use in, export from, or import to the United States or that produce substances that will be used as active ingredients in pes- ticides must report this information to the EPA regional office that serves the area where the establishment is located. The Compliance Division, OCM has overall responsibility for managing the system. The states, most of which perform FIFRA compliance activities under cooperative agree- ments with EPA, use information from the system to identify all the loca- tions at which pesticides are produced, so that they can inspect producers for compliance with FIFRA. In addition, OPP uses FATES produc- tion data in risk/benefit assessments of some types of pesticides to sup- plement estimates of the amount of these pesticides used annually. When we attempted to use data from the three systems we found that SomeKey Data some key data on disinfectants were inaccurate, incomplete, and/or Inaccurate, Incomplete missing. In addition, we found inconsistencies in some data that both and/or Missing PPIS and FATES are supposed to contain. Since EPA officials use the data in these three systems for registration and enforcement purposes, problems with the integrity of the data could impair, among other things, EPA'S ability to make sound regulatory decisions about disinfec- tants or EPA'S ability to identify all labs that generate disinfectant effi- cacy studies for inspection. The following examples illustrate the reasons for our concerns. First, PPIS contained inaccurate data on the number of disinfectants reg- istered to kill tuberculosis bacteria. In June 1986, EPA required all regis- trants of tuberculocidal disinfectants to submit new data on the efficacy of their products. By February 1989, registrants of 44 out of 144 disin- fectants subject to the notice had satisfied EPA'S request for data. As for the remaining disinfectants, registrants either deleted their tuberculocidal label claims or EPA suspended or canceled their registra- tions. As of September 1, 1989, however, data files in PPIS still showed Page3 GAO/RCED-SO-232EPA'sDataonDMnfectanta -~ - R-240811 that 130, rather than 44, disinfectants were registered with tuberculo- tide claims. According to the Chief, Systems Branch, the extent to which pesticide data in the system are inaccurate or incomplete is unknown but may be in the 60-percent range. Various EPA officials told us that data quality problems exist because EPA lacks adequate procedures for ensuring that data are accurately coded for and entered into the system and for ensuring that the data remain up to date once entered. Because much of the data in the system may be inaccurate or incomplete, EPA may not be able to identify accurately all disinfectants, or perhaps other pesticides, that may require additional data for the agency to assess their health and environmental effects, among other things. Second, PDMS was missing some data on the disinfectant efficacy studies that registrants had submitted to EPA between January 1, 1985, and June 26, 1989. Data files on about 30 percent of the disinfectant efficacy studies catalogued in the system were missing information on the labo- ratory that generated the studies. In addition, we could not identify all disinfectant efficacy studies that had been submitted to EPA because, in some cases, EPA had catalogued groups of disinfectant efficacy studies submitted by registrants, rather than separately cataloguing individual studies. According to an official in the Information Services Branch, the problems are a result of the relatively low priority EPA has given to auto- mating efficacy data on pesticides, including disinfectant efficacy data, and the lack of adequate quality control procedures for entering data into the system. Until EPA resolves these problems, the agency will not be able to meet the goals of its good laboratory practices program by identifying and inspecting all labs that generate disinfectant efficacy studies, nor will it be able to perform other important tasks. Third, an analysis that EPA conducted-matching disinfectant registra- tions from PPISand production data in FATES-indicated that FATESdoes not contain production data for some disinfectants. Specifically, FATES contained no matching production data for 25 percent of the active dis- infectant registrations in PPIS.~As previously noted, Section 7 of FIFRA requires that all registered pesticide-producing establishments annually report production data. EPAofficials did not know why 25 percent of the disinfectants lacked FATESproduction data. EPAofficials suggested that 3EPAconsidered a pesticide registration active if the registrant opted to pay a fee in 1989 to maintain the registration. At our request, EPA matched data from PPIS on disinfectants first registered before 1987 and considered active in 1989 with 1987 production data from FATE!+--the most recent produc- tion data FATFS contained at the time of our review. Page 4 GAO/RCED-90.232 EPA’s Data on DMnfectants B-240811 some producers might not produce disinfectants each year. Other pos- sible explanations are that producers are not submitting required pro- duction data to EPA or that EPA'S regional offices are not properly or completely entering all the data that are submitted. As of July 1990, OCM officials managing the data base had not yet determined how much, if any, production data were missing and why data might be missing. To the extent that some data have been missing, EPA may not have provided complete information to the states on pesticide producers that should be inspected, and EPA risk/benefit decisions relying on FATES data may have been impaired. Fourth, although some of the data elements in the systems were intended to contain the same data, there are strong indications that they do not. For example, both PPIS and FATES classify products according to the type of pesticide they are, and both can be used to identify regis- tered disinfectants. However, as noted above, we found that FATES did not contain production information on all of the products that PPIS indi- cated as active disinfectants. In addition, we found that when we matched the pesticides classified as disinfectants in PPIS with production amounts in FATES, we obtained a different value for total annual produc- tion in the disinfectants industry than when we used FATE%,itself to iden- tify the disinfectants. According to the Chief, Compliance Branch, OCM has not attempted to determine whether pesticides are classified consist- ently as disinfectants in both systems. To the extent that these systems do not contain complete and consistent information on registered disin- fectants, the quality of EPA analyses based on the data may be impaired. EPA does not consistently classify and code disinfectants in a way that Data BasesCannot completely distinguishes them from other types of antimicrobial pesti- Identify All cides in the three systems. In addition, the data bases in these systems Disinfectants or do not contain information on all types of disinfectant efficacy claims. Because of these data base limitations, EPA cannot accurately identify all Claims registered disinfectants or those disinfectants making various types of efficacy claims. EPA regulations require registrants to submit efficacy data on disinfec- tants but not on most other types of pesticides, For example, registrants of antimicrobials which target microorganisms that do not cause dis- eases in humans, such as those that target slime-forming or odor-causing bacteria, are not required to submit efficacy data. In addition, EPA'S reg- istration guidelines specify data requirements that vary according to the Page 5 GAO/RCED-90-232 EPA’s Data on Disinfectants B-240611 types of efficacy claims a registrant proposes for a disinfectant. How- ever, EPA did not design the coding systems used in PPIS, PDMS, or FATES so that they completely distinguish disinfectants from other types of anti- microbial pesticides. Furthermore, EPA did not design the three data sys- tems so that they could be used to identify disinfectants registered for (1) “limited,” “general,” or “hospital” efficacy; (2) efficacy in hard water or the presence of organic matter; or (3) efficacy on hard surfaces or other types of surfaces. We recognize that the three systems were designed to support different functions. However, EPA relies on the systems to identify disinfectants subject to different registration requirements, risk/benefit assessments, and a variety of other regulatory activities. Because EPA cannot accu- rately identify all registered disinfectants or those disinfectants making various types of efficacy claims, these activities may be impaired. For example, EPA may be unable to completely identify the disinfectant effi- cacy claims that should be retested on the basis of new methods for testing efficacy that are being developed. officials generally agreed with our concerns about the disinfectant EPA Has Not Fully EPA data in EPA information systems and told us that they plan to resolve Addressed Data some of the concerns through ongoing efforts to replace certain informa- Integrity Problems tion in PPIS and examine ways to improve the data in FATES as part of EPA'S efforts to convert this system to a new data-base management system. However, EPA'S plans for the two systems are incomplete and do not address all of our concerns on the integrity of the data in these sys- tems. In addition, EPA is not planning to make any changes to PDMS. As of July 1990, an EPA initiative was underway to replace the pesticide product label information in PPIS with new data from registrants as part of their efforts to develop a new system called the Label Use Informa- tion System, EPA is developing this new system to provide users with automated access to accurate data on pesticide label information, including data on the pesticide’s type, recommendations for use of the pesticide, the pest(s) the pesticide controls, and how the pesticide should be applied. Eventually, the new system will be electronically linked to PPIS. Further, according to EPA officials, EPA has taken several steps to ensure that the label information in the new system is more accurate than that in PPIS. However, according to the Chief, Systems Branch, even when the new system is in place, EPA will lack adequate procedures for ensuring that data are accurately coded for entry into the system, accurately entered into the system, and kept up to date once Page 6 GAO/RCED-90-232 EPA’s Data on Didnfectant~ entered. In addition, EPA has been revising the coding system used in PPIS to make it, among other things, easier to identify pesticides subject to various data requirements in EPA'S registration guidelines; however, as of July 1990, the new coding system was incomplete. Also as of July 1990, an initiative was underway to convert FATES to a new data-base management system and to make other improvements to the system by October 1990. According to a planning document, the stimulus for the conversion is an agencywide shift to a new type of data management system; however, OCM is using the opportunity to make other changes to the system. According to the Chief, Compliance Branch, EPA agrees with the data integrity problems that we have identi- fied with respect to data on disinfectants in the system and will investi- gate them further as part of the efforts to implement the new data-base management system. However, because OCM is in the initial stages of planning improvements to the system, OCM has not reached the point of making specific plans for resolving these data problems. EPA had not made any plans to address the data problems we identified in PDMS as of July 1990. According to the Head, Information Resources Development Section, Information Services Branch, EPA does not plan to assess the accuracy or completeness of the data on efficacy studies in the system or to change the scheme used to code the studies in the system until automating efficacy data is made a higher priority in EPA. Our work with these three EPA data systems indicated problems with the Conclusions integrity of disinfectant data in the systems. We found that the systems contained inaccurate and/or incomplete data or were missing data on registered disinfectants. In addition, we found that the data could not be used to identify all disinfectants or efficacy claims that EPA has regis- tered. We did not assess the full extent of the risks from the data integ- rity problems we identified. However, these data problems may be adversely affecting EPA management and regulatory decisions as well as the usefulness of information EPA provides to the Congress, industry, and the public on registered disinfectants. Although EPA officials generally agreed with the data problems we iden- tified, EPA has not yet formulated specific plans to resolve all of them and needs to do so. In particular, although EPA officials believe the lack of adequate procedures to ensure that data are entered into the systems accurately, completely, and consistently and that the data are kept up to date is responsible for many of the data problems we identified, EPA has Page 7 GAO/RCED=BO-232EPA’s Data on Meinfectants R-240811 not taken corrective action. Furthermore, EPA'S lack of adequate proce- dures to enter data into the systems may jeopardize the integrity of data on pesticides other than disinfectants. As a result, EPA needs to deter- mine the extent to which these data integrity problems apply to data on other pesticides, since EPA relies on the data for a variety of program management, budgetary, enforcement, and regulatory decisions. We recommend that you direct OPTSto develop and implement a strategy Recommendations for resolving the data integrity problems that we have identified in the data on disinfectants in PPIS, PDMS, and FATES, especially the need to establish procedures to ensure that data are accurately, completely, and consistently entered into automated systems and that the data are kept up to date. We also recommend that you direct OPTS to determine the extent to which the data integrity problems we identified apply to data on pesticides other than disinfectants and take corrective action as warranted. As previously noted, we did not evaluate EPA'S management of OPTS information systems or its plans to update and/or improve the systems. Instead, we became concerned about the integrity of the data in PPIS, PDMS, and FATES while using data from the systems for our review of EPA'S disinfectants programs. We used these systems intermittently between February 1989 through May 1990 and, in July 1990, inter- viewed EPA officials responsible for managing the systems to determine if they agreed with the data problems we identified and whether they planned to resolve them. Our work was conducted in accordance with generally accepted government auditing standards. As the head of a federal agency, you are required by 31 USC. 720 to submit a written statement on actions taken on the recommendations contained in this report to the Senate Committee on Governmental Affairs and the House Committee on Government Operations no later than 60 days after the date of the letter and to the House and Senate Committees on Appropriations with the agency’s first request for appro- priations made more than 60 days after the date of the letter. Page 8 GAO/RCED-90-232 EPA’s Data on Disinfectants B249811 Major contributors to the letter are listed in appendix I. If you have any questions about this letter, please contact me at (202) 275-6111. Sincerely yours, Richard L. Hembra Director, Environmental Protection Issues Page 9 GAO/RCED-99-232 EPA’s Data on Didnfectanti Appendix I Major Contributorsto This Report Peter F. Guerrero, Associate Director Resources, J. Kevin Donohue, Assistant Director Community, and William M. Layden, Assignment Manager Economic Karen Simpson, Evaluator-in-Charge Sarah-Ann Moessbauer, Operations Research Analyst Development Division, Washington, DC. (160055) Page 10 GAO/RCRD-90-232 EPA’s Data on Disinfectanta L . “. . . “I ..-_- ._.... I -.._ -... . . .._. - ._-. . -- - .__. . ..__. -_--_____-~--_--- -- OrcI~~rs uiiiy also t)c* placed by calling (202) 275-6241. ? r
Disinfectants: Concerns Over the Integrity of EPA's Data Bases
Published by the Government Accountability Office on 1990-09-21.
Below is a raw (and likely hideous) rendition of the original report. (PDF)