Disinfectants: Concerns Over the Integrity of EPA's Data Bases

Published by the Government Accountability Office on 1990-09-21.

Below is a raw (and likely hideous) rendition of the original report. (PDF)


svpt ~‘llli~(‘t’ l!b!H)
                          Concerns Over the
                          Integrity of EPA’s
                          Data Bases

                   United States
                   General Accounting Office
                   Washington, DC. 20648

                   Resources, Community, and
                   Economic Development Division

                   R-2408 11

                   September 21, 1990

                   The Honorable William K. Reilly
                   Environmental Protection Agency

                   Dear Mr. Reilly:

                     As you know, the Environmental Protection Agency (EPA) relies on pesti-
                     cide data systems to make a variety of program management, budg-
                     etary, enforcement, and regulatory decisions about pesticides, including
                     disinfectants,’ and for providing information about pesticides to the
                     Congress, industry, and the public. In our August 1990 report,2 we men-
                     tioned our concerns about the accuracy and completeness of disinfectant
                     data in these systems. This report explains these concerns, specifically
                     in terms of three systems: (1) the Pesticide Product Information System
                   ' (PPIS), (2) the Pesticide Document Management System (PDMS), and (3)
                     the Federal Insecticide, Fungicide, and Rodenticide Act and Toxic Sub-
                     stances Control Act Enforcement System (FATES).

                   Although we did not evaluate EPA'S management of these three com-
Results in Brief   puter-based systems, we nonetheless identified several problems with
                   the integrity of the data in these systems, problems that may limit the
                   extent to which the data can support EPA'S disinfectant program and its
                   managers. Specifically, the systems contained inaccurate and/or incom-
                   plete data or were missing data on disinfectants. For example, although
                   1~1sis intended to include essential regulatory data on disinfectant
                   product claims, as much as 60 percent of the data in the system may be
                   inaccurate or incomplete, according to some EPA officials. Furthermore,
                   although intended to capture relevant information, the systems could
                   not be used to identify all types of disinfectants and their product per-
                   formance claims. Although these problems could adversely affect EPA'S
                   regulation of disinfectants, and perhaps the regulation of other pesti-
                   cides, EPA has not fully addressed these problems.

                   ‘As defined by EPA, “disinfectant” refers to only one of several types of antimicrobial pesticides,
                   which, with some exceptions, are substances intended to inhibit or destroy microorganisms (bacteria,
                   fungi, viruses, and spores). However, we use the term “disinfectant” in this report to broadly describe
                   all antimicrobial pesticides intended to protect public health.

                   ‘Disinfectants: EPA Lacks Assurance They Work (GAO/RCED-90-139, Aug. 30,199O).

                   Page 1                                             GAO/RCED-90-232 EPA’s Data on DMnfectanta

             Under the Federal Insecticide, Fungicide, and Rodenticide Act, as
Background   amended (FIFRA), EPA generally must register (license) pesticides,
             including disinfectants, before they may be sold, held for sale, or distrib-
             uted in commerce. Before registering a pesticide, EPA must determine
             that it is effective, when used as directed, without causing an unreason-
             able risk to public health or the environment. To make this determina-
             tion, EPA requires registrants to submit data on health and
             environmental effects and, in the case of disinfectants, on product per-
             formance (efficacy). Because EPA has not evaluated most pesticides used
             today against current testing requirements, the agency is now in the
             process of reregistering older pesticides, including disinfectants, on the
             basis of these requirements.

             Within EPA, the Office of Pesticides and Toxic Substances (OPTS) regu-
             lates disinfectants and other pesticides. EPA developed PPIS, PDMS, and
             FATES for use in performing pesticide registration and FIFRA compliance

             PPIS  is intended to capture basic data on pesticide product registrations,
             including data on product labels, such as the product’s name and regis-
             tration number, the name and address of the product’s registrant, the
             product’s active ingredient(s), the product’s pesticide type, and the uses
             for which the product is registered. The Office of Pesticide Programs
             (OPP) uses data from PPIS for a variety of purposes, including the identi-
             fication of registered pesticide products for which additional data are
             needed before they can be reregistered. In addition, whenever new evi-
             dence suggests that an ingredient contained in a pesticide may cause sig-
             nificant public health or environmental concerns, OPP uses PPIS to
             identify the products that contain the ingredient so that the agency may
             conduct a detailed analysis of the risks and benefits of continued regis-
             tration of such products.

             PDMS  serves as a central archive of the documents (referred to as
             studies) that registrants have submitted to EPA to support pesticide
             registrations. The system contains a catalogue of bibliographic informa-
             tion about each document under a Master Record Identifier (MRID)
             number. This information includes, among other things, the registration
             number and active ingredient(s) of the product to which the document
             pertains and the name and location of the laboratory that generated the
             data cited in the document. The Information Services Branch, OPP, main-
             tains the system. OPP currently uses the system for several purposes, one
             of which is to locate studies on health and environmental effects. These
             studies are needed to make decisions on whether to reregister pesticides.

             Page 2                                GAO/RCED@O-232 EPA’s Data on Disinfectanta

                         On the basis of a recommendation we made in our recent disinfectants
                         report, the Office of Compliance Monitoring (OCM) may also begin using
                         PDMS to identify labs to inspect and studies to audit under its good labo-
                         ratory practices program.

                         FATES  contains data on, among other things, the types and amounts of
                         pesticides, including disinfectants, produced during the current year and
                         the amounts sold or distributed during the previous year. Section 7 of
                         FIFRA and EPA regulations generally require certain establishments to
                         report this information to EPA. Specifically, establishments that produce
                         pesticide products for use in, export from, or import to the United States
                         or that produce substances that will be used as active ingredients in pes-
                         ticides must report this information to the EPA regional office that serves
                         the area where the establishment is located. The Compliance Division,
                         OCM has overall responsibility for managing the system. The states, most
                         of which perform FIFRA compliance activities under cooperative agree-
                         ments with EPA, use information from the system to identify all the loca-
                         tions at which pesticides are produced, so that they can inspect
                         producers for compliance with FIFRA. In addition, OPP uses FATES produc-
                         tion data in risk/benefit assessments of some types of pesticides to sup-
                         plement estimates of the amount of these pesticides used annually.

                         When we attempted to use data from the three systems we found that
SomeKey Data             some key data on disinfectants were inaccurate, incomplete, and/or
Inaccurate, Incomplete   missing. In addition, we found inconsistencies in some data that both
and/or Missing           PPIS and FATES are supposed to contain. Since EPA officials use the data in
                         these three systems for registration and enforcement purposes,
                         problems with the integrity of the data could impair, among other
                         things, EPA'S ability to make sound regulatory decisions about disinfec-
                         tants or EPA'S ability to identify all labs that generate disinfectant effi-
                         cacy studies for inspection. The following examples illustrate the
                         reasons for our concerns.

                         First, PPIS contained inaccurate data on the number of disinfectants reg-
                         istered to kill tuberculosis bacteria. In June 1986, EPA required all regis-
                         trants of tuberculocidal disinfectants to submit new data on the efficacy
                         of their products. By February 1989, registrants of 44 out of 144 disin-
                         fectants subject to the notice had satisfied EPA'S request for data. As for
                         the remaining disinfectants, registrants either deleted their
                         tuberculocidal label claims or EPA suspended or canceled their registra-
                         tions. As of September 1, 1989, however, data files in PPIS still showed

                         Page3                                 GAO/RCED-SO-232EPA'sDataonDMnfectanta
-~ -

       that 130, rather than 44, disinfectants were registered with tuberculo-
       tide claims.

       According to the Chief, Systems Branch, the extent to which pesticide
       data in the system are inaccurate or incomplete is unknown but may be
       in the 60-percent range. Various EPA officials told us that data quality
       problems exist because EPA lacks adequate procedures for ensuring that
       data are accurately coded for and entered into the system and for
       ensuring that the data remain up to date once entered. Because much of
       the data in the system may be inaccurate or incomplete, EPA may not be
       able to identify accurately all disinfectants, or perhaps other pesticides,
       that may require additional data for the agency to assess their health
       and environmental effects, among other things.

       Second, PDMS was missing some data on the disinfectant efficacy studies
       that registrants had submitted to EPA between January 1, 1985, and
       June 26, 1989. Data files on about 30 percent of the disinfectant efficacy
       studies catalogued in the system were missing information on the labo-
       ratory that generated the studies. In addition, we could not identify all
       disinfectant efficacy studies that had been submitted to EPA because, in
       some cases, EPA had catalogued groups of disinfectant efficacy studies
       submitted by registrants, rather than separately cataloguing individual
       studies. According to an official in the Information Services Branch, the
       problems are a result of the relatively low priority EPA has given to auto-
       mating efficacy data on pesticides, including disinfectant efficacy data,
       and the lack of adequate quality control procedures for entering data
       into the system. Until EPA resolves these problems, the agency will not
       be able to meet the goals of its good laboratory practices program by
       identifying and inspecting all labs that generate disinfectant efficacy
       studies, nor will it be able to perform other important tasks.

       Third, an analysis that EPA conducted-matching     disinfectant registra-
       tions from PPISand production data in FATES-indicated that FATESdoes
       not contain production data for some disinfectants. Specifically, FATES
       contained no matching production data for 25 percent of the active dis-
       infectant registrations in PPIS.~As previously noted, Section 7 of FIFRA
       requires that all registered pesticide-producing establishments annually
       report production data. EPAofficials did not know why 25 percent of the
       disinfectants lacked FATESproduction data. EPAofficials suggested that

       3EPAconsidered a pesticide registration active if the registrant opted to pay a fee in 1989 to maintain
       the registration. At our request, EPA matched data from PPIS on disinfectants first registered before
        1987 and considered active in 1989 with 1987 production data from FATE!+--the most recent produc-
       tion data FATFS contained at the time of our review.

       Page 4                                             GAO/RCED-90.232 EPA’s Data on DMnfectants

                   some producers might not produce disinfectants each year. Other pos-
                   sible explanations are that producers are not submitting required pro-
                   duction data to EPA or that EPA'S regional offices are not properly or
                   completely entering all the data that are submitted. As of July 1990, OCM
                   officials managing the data base had not yet determined how much, if
                   any, production data were missing and why data might be missing. To
                   the extent that some data have been missing, EPA may not have provided
                   complete information to the states on pesticide producers that should be
                   inspected, and EPA risk/benefit decisions relying on FATES data may have
                   been impaired.

                   Fourth, although some of the data elements in the systems were
                   intended to contain the same data, there are strong indications that they
                   do not. For example, both PPIS and FATES classify products according to
                   the type of pesticide they are, and both can be used to identify regis-
                   tered disinfectants. However, as noted above, we found that FATES did
                   not contain production information on all of the products that PPIS indi-
                   cated as active disinfectants. In addition, we found that when we
                   matched the pesticides classified as disinfectants in PPIS with production
                   amounts in FATES, we obtained a different value for total annual produc-
                   tion in the disinfectants industry than when we used FATE%,itself to iden-
                   tify the disinfectants. According to the Chief, Compliance Branch, OCM
                   has not attempted to determine whether pesticides are classified consist-
                   ently as disinfectants in both systems. To the extent that these systems
                   do not contain complete and consistent information on registered disin-
                   fectants, the quality of EPA analyses based on the data may be impaired.

                   EPA does not consistently classify and code disinfectants in a way that
Data BasesCannot   completely distinguishes them from other types of antimicrobial pesti-
Identify All       cides in the three systems. In addition, the data bases in these systems
Disinfectants or   do not contain information on all types of disinfectant efficacy claims.
                   Because of these data base limitations, EPA cannot accurately identify all
Claims             registered disinfectants or those disinfectants making various types of
                   efficacy claims.

                   EPA  regulations require registrants to submit efficacy data on disinfec-
                   tants but not on most other types of pesticides, For example, registrants
                   of antimicrobials which target microorganisms that do not cause dis-
                   eases in humans, such as those that target slime-forming or odor-causing
                   bacteria, are not required to submit efficacy data. In addition, EPA'S reg-
                   istration guidelines specify data requirements that vary according to the

                   Page 5                               GAO/RCED-90-232 EPA’s Data on Disinfectants

                     types of efficacy claims a registrant proposes for a disinfectant. How-
                     ever, EPA did not design the coding systems used in PPIS, PDMS, or FATES so
                     that they completely distinguish disinfectants from other types of anti-
                     microbial pesticides. Furthermore, EPA did not design the three data sys-
                     tems so that they could be used to identify disinfectants registered for
                     (1) “limited,” “general,” or “hospital” efficacy; (2) efficacy in hard
                     water or the presence of organic matter; or (3) efficacy on hard surfaces
                     or other types of surfaces.

                     We recognize that the three systems were designed to support different
                     functions. However, EPA relies on the systems to identify disinfectants
                     subject to different registration requirements, risk/benefit assessments,
                     and a variety of other regulatory activities. Because EPA cannot accu-
                     rately identify all registered disinfectants or those disinfectants making
                     various types of efficacy claims, these activities may be impaired. For
                     example, EPA may be unable to completely identify the disinfectant effi-
                     cacy claims that should be retested on the basis of new methods for
                     testing efficacy that are being developed.

                         officials generally agreed with our concerns about the disinfectant
EPA Has Not Fully    EPA
                     data in EPA information systems and told us that they plan to resolve
Addressed Data       some of the concerns through ongoing efforts to replace certain informa-
Integrity Problems   tion in PPIS and examine ways to improve the data in FATES as part of
                     EPA'S efforts to convert this system to a new data-base management
                     system. However, EPA'S plans for the two systems are incomplete and do
                     not address all of our concerns on the integrity of the data in these sys-
                     tems. In addition, EPA is not planning to make any changes to PDMS.

                     As of July 1990, an EPA initiative was underway to replace the pesticide
                     product label information in PPIS with new data from registrants as part
                     of their efforts to develop a new system called the Label Use Informa-
                     tion System, EPA is developing this new system to provide users with
                     automated access to accurate data on pesticide label information,
                     including data on the pesticide’s type, recommendations for use of the
                     pesticide, the pest(s) the pesticide controls, and how the pesticide
                     should be applied. Eventually, the new system will be electronically
                     linked to PPIS. Further, according to EPA officials, EPA has taken several
                     steps to ensure that the label information in the new system is more
                     accurate than that in PPIS. However, according to the Chief, Systems
                     Branch, even when the new system is in place, EPA will lack adequate
                     procedures for ensuring that data are accurately coded for entry into
                     the system, accurately entered into the system, and kept up to date once

                     Page 6                                GAO/RCED-90-232 EPA’s Data on Didnfectant~
              entered. In addition, EPA has been revising the coding system used in PPIS
              to make it, among other things, easier to identify pesticides subject to
              various data requirements in EPA'S registration guidelines; however, as
              of July 1990, the new coding system was incomplete.

              Also as of July 1990, an initiative was underway to convert FATES to a
              new data-base management system and to make other improvements to
              the system by October 1990. According to a planning document, the
              stimulus for the conversion is an agencywide shift to a new type of data
              management system; however, OCM is using the opportunity to make
              other changes to the system. According to the Chief, Compliance
              Branch, EPA agrees with the data integrity problems that we have identi-
              fied with respect to data on disinfectants in the system and will investi-
              gate them further as part of the efforts to implement the new data-base
              management system. However, because OCM is in the initial stages of
              planning improvements to the system, OCM has not reached the point of
              making specific plans for resolving these data problems.

              EPA had not made any plans to address the data problems we identified
              in PDMS as of July 1990. According to the Head, Information Resources
              Development Section, Information Services Branch, EPA does not plan to
              assess the accuracy or completeness of the data on efficacy studies in
              the system or to change the scheme used to code the studies in the
              system until automating efficacy data is made a higher priority in EPA.

              Our work with these three EPA data systems indicated problems with the
Conclusions   integrity of disinfectant data in the systems. We found that the systems
              contained inaccurate and/or incomplete data or were missing data on
              registered disinfectants. In addition, we found that the data could not be
              used to identify all disinfectants or efficacy claims that EPA has regis-
              tered. We did not assess the full extent of the risks from the data integ-
              rity problems we identified. However, these data problems may be
              adversely affecting EPA management and regulatory decisions as well as
              the usefulness of information EPA provides to the Congress, industry,
              and the public on registered disinfectants.

              Although EPA officials generally agreed with the data problems we iden-
              tified, EPA has not yet formulated specific plans to resolve all of them
              and needs to do so. In particular, although EPA officials believe the lack
              of adequate procedures to ensure that data are entered into the systems
              accurately, completely, and consistently and that the data are kept up to
              date is responsible for many of the data problems we identified, EPA has

              Page 7                               GAO/RCED=BO-232EPA’s Data on Meinfectants

                  not taken corrective action. Furthermore, EPA'S lack of adequate proce-
                  dures to enter data into the systems may jeopardize the integrity of data
                  on pesticides other than disinfectants. As a result, EPA needs to deter-
                  mine the extent to which these data integrity problems apply to data on
                  other pesticides, since EPA relies on the data for a variety of program
                  management, budgetary, enforcement, and regulatory decisions.

                  We recommend that you direct OPTSto develop and implement a strategy
Recommendations   for resolving the data integrity problems that we have identified in the
                  data on disinfectants in PPIS, PDMS, and FATES, especially the need to
                  establish procedures to ensure that data are accurately, completely, and
                  consistently entered into automated systems and that the data are kept
                  up to date. We also recommend that you direct OPTS to determine the
                  extent to which the data integrity problems we identified apply to data
                  on pesticides other than disinfectants and take corrective action as

                  As previously noted, we did not evaluate EPA'S management of OPTS
                  information systems or its plans to update and/or improve the systems.
                  Instead, we became concerned about the integrity of the data in PPIS,
                  PDMS, and FATES while using data from the systems for our review of
                  EPA'S disinfectants programs. We used these systems intermittently
                  between February 1989 through May 1990 and, in July 1990, inter-
                  viewed EPA officials responsible for managing the systems to determine
                  if they agreed with the data problems we identified and whether they
                  planned to resolve them. Our work was conducted in accordance with
                  generally accepted government auditing standards.

                  As the head of a federal agency, you are required by 31 USC. 720 to
                  submit a written statement on actions taken on the recommendations
                  contained in this report to the Senate Committee on Governmental
                  Affairs and the House Committee on Government Operations no later
                  than 60 days after the date of the letter and to the House and Senate
                  Committees on Appropriations with the agency’s first request for appro-
                  priations made more than 60 days after the date of the letter.

                  Page 8                               GAO/RCED-90-232   EPA’s Data on Disinfectants

Major contributors to the letter are listed in appendix I. If you have any
questions about this letter, please contact me at (202) 275-6111.

Sincerely yours,

Richard L. Hembra
Director, Environmental Protection

Page 9                               GAO/RCED-99-232   EPA’s Data on Didnfectanti
Appendix I

Major Contributorsto This Report

                        Peter F. Guerrero, Associate Director
Resources,              J. Kevin Donohue, Assistant Director
Community, and          William M. Layden, Assignment Manager
Economic                Karen Simpson, Evaluator-in-Charge
                        Sarah-Ann Moessbauer, Operations Research Analyst
Development Division,
Washington, DC.

(160055)                Page 10                           GAO/RCRD-90-232   EPA’s Data on Disinfectanta
L   .   “.   . . “I ..-_- ._....   I -.._   -...   . . .._.   -   ._-.   .   --   - .__.   . ..__. -_--_____-~--_---                          --

                                                                         OrcI~~rs uiiiy also t)c* placed               by calling   (202)   275-6241.