oversight

Meat and Poultry: Improved Oversight and Training Will Strengthen New Food Safety System

Published by the Government Accountability Office on 1999-12-08.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                 United States General Accounting Office

GAO              Report to Congressional Committees




December 1999
                 MEAT AND POULTRY
                 Improved Oversight
                 and Training Will
                 Strengthen New Food
                 Safety System




GAO/RCED-00-16
      United States
GAO   General Accounting Office
      Washington, D.C. 20548

      Resources, Community, and
      Economic Development Division

      B-283823

      December 8, 1999

      The Honorable Thad Cochran
      Chairman
      The Honorable Herb Kohl
      Ranking Minority Member
      Subcommittee on Agriculture, Rural Development,
        and Related Agencies
      Committee on Appropriations
      United States Senate

      The Honorable Joe Skeen
      Chairman
      The Honorable Marcy Kaptur
      Ranking Minority Member
      Subcommittee on Agriculture, Rural Development,
        Food and Drug Administration,
        and Related Agencies
      Committee on Appropriations
      House of Representatives

      Food contamination may cause an estimated 76 million illnesses, 325,000
      hospitalizations, and 5,000 deaths in the United States each year,
      according to the Centers for Disease Control and Prevention. The U.S.
      Department of Agriculture (USDA) estimates that the costs associated with
      foodborne illnesses are as high as $37 billion annually. To reduce
      foodborne illnesses and improve the safety of meat and poultry products,
      USDA issued regulations in July 1996 requiring that by January 2000 all meat
      and poultry plants adopt a science-based production control system called
      Hazard Analysis and Critical Control Point (HACCP). The HACCP program is
      designed to identify the steps in food production where contamination is
      most likely to occur and then to establish controls that prevent or reduce
      contamination. The HACCP regulations also require that meat and poultry
      slaughter plants regularly test for E-coli bacteria to verify that their
      controls are sufficient to prevent fecal contamination. In addition, USDA’s
      Food Safety and Inspection Service (FSIS) tests for salmonella at plants
      that produce raw and ground meat and poultry products to determine
      whether they are meeting USDA’s salmonella pathogen reduction
      performance standards.

      The mandatory implementation of HACCP systems fundamentally changes
      the government’s approach to ensuring the safety of meat and poultry




      Page 1                   GAO/RCED-00-16 Implementation of New Food Safety System
B-283823




products: It makes industry, not government, primarily responsible for the
safety of these products. Moreover, industry has to implement HACCP
systems that identify and control biological, physical, and chemical
hazards throughout production. HACCP systems are designed to improve
USDA’s traditional inspections, which use sight, touch, and smell to detect
contaminated products. USDA inspectors based at meat and poultry plants
are responsible for overseeing the plants’ implementation of their HACCP
systems.

The HACCP approach, which was recommended by USDA’s National
Advisory Committee on Microbiological Criteria for Foods and endorsed
by the scientific community, is based on seven guiding principles. The
principles center around the identification of biological, physical, and
chemical hazards that are reasonably likely to occur in meat and poultry
plants and the establishment of critical points in the production process
where controls can be applied to prevent, eliminate, or reduce those
hazards.

Concerned about HACCP implementation, the Congress mandated that GAO
(1) determine whether the system adopted by USDA in its regulations is
consistent with the seven HACCP principles endorsed by the Advisory
Committee, (2) evaluate whether the HACCP training program for USDA
inspectors is adequate and science-based, and (3) determine if there is an
adequate dispute resolution (appeals) process between plants and USDA
under the new HACCP inspection system.

As part of our review, we compared the HACCP principles endorsed by the
Advisory Committee with those included in the HACCP regulations and
visited 32 meat and poultry plants (out of about 2,600 that have
implemented HACCP programs) across the country to determine how the
principles were applied in their HACCP plans. We did not evaluate the
plants’ hazard analyses or how well they were implementing their HACCP
plans. The selected plants produce a variety of meat and poultry products
through a number of different processes. To evaluate HACCP training for
USDA inspectors, we reviewed USDA’s HACCP training curriculum,
participated in USDA’s HACCP training program, reviewed and analyzed
evaluations of training conducted by industry and by USDA, and conducted
our own nationwide telephone survey of inspectors most recently trained
in the HACCP system. (App. I presents the results of this survey.) We also
reviewed the regulations concerning appeals and interviewed industry and
inspection personnel to obtain their opinions on how well the appeals
process is working. Appendix II describes our methodology in detail.



Page 2                   GAO/RCED-00-16 Implementation of New Food Safety System
                   B-283823




                   USDA’s HACCP  regulations, along with implementing directives and other
Results in Brief   guidance, are consistent with the seven HACCP principles endorsed by the
                   Advisory Committee. The 32 plants we visited have HACCP plans that are
                   based on the principles; however, some plant managers excluded certain
                   hazards (such as metal contamination) because they believed that existing
                   quality control programs or good manufacturing practices (such as
                   routinely calibrating production equipment) effectively controlled them.
                   To ensure appropriate verification and oversight by USDA inspection
                   personnel, however, the regulations state that plants must identify all
                   hazards reasonably likely to occur in their HACCP plans and control those
                   hazards through their HACCP programs. USDA regulations preclude the use
                   of non-HACCP programs to control hazards that are reasonably likely to
                   occur because the Department’s inspectors focus on ensuring compliance
                   with HACCP regulations and have limited oversight over non-HACCP
                   programs.

                   Generally, inspectors received the training needed to oversee plants’
                   implementation of HACCP programs, although many inspectors responding
                   to our nationwide survey reported that they would benefit from refresher
                   courses. USDA’s training program provided a basic introduction to HACCP’s
                   science-based principles while emphasizing the compliance aspects of the
                   inspectors’ HACCP duties. However, according to our review of USDA’s HACCP
                   training curriculum and the results of our survey, several aspects of the
                   training program need to be clarified and reinforced in order to provide
                   inspectors with the tools they need to conduct consistent and effective
                   inspections of plants’ compliance with HACCP requirements. Many survey
                   responses suggested that the training was unclear about (1) inspectors’
                   authority to request that plants make changes to their HACCP plans, (2) the
                   correct frequency for microbial testing for salmonella, (3) the actions
                   inspectors can take if they become aware of microbial contamination that
                   a plant has identified as a hazard through programs that are outside of its
                   HACCP plan, and (4) situations in which it is appropriate for inspectors to
                   record instances of noncompliance with the HACCP requirements by issuing
                   noncompliance notices (referred to as noncompliance records by FSIS).
                   USDA’s own evaluation of inspectors’ training in 1998 identified areas of
                   training that should be strengthened.

                   USDA’s  dispute resolution process provides industry with an appropriate
                   mechanism to appeal inspectors’ enforcement actions. All of the plant
                   managers we interviewed were generally aware of how the process
                   worked, and many had used it to appeal inspectors’ findings and decisions
                   that, in their opinion, were erroneous. These managers questioned the



                   Page 3                   GAO/RCED-00-16 Implementation of New Food Safety System
             B-283823




             accuracy of the information contained in the documents that inspectors
             prepare when they believe that a plant has failed to comply with one or
             more of the HACCP requirements. For example, managers said that
             inspectors at times document instances of noncompliance by citing
             incorrect regulations or marking the wrong trend indicators. Plant
             managers said that they follow the advice of industry associations, such as
             the National Meat Association, and appeal noncompliance notices that
             appear to be inaccurate. Recurring instances of noncompliance can result
             in an FSIS determination that the plant’s HACCP system failed. However,
             USDA regulations do not explicitly state the number or types of
             noncompliance notices that can result in such a determination. Therefore,
             plant personnel want to ensure that these notices of noncompliance are
             justified and accurately documented. We could not verify the accuracy of
             the plant managers’ observations because USDA’s new automated appeals
             tracking system contains incomplete and inconsistent information. This
             report contains a number of recommendations to the Secretary of
             Agriculture designed to improve the HACCP plans, improve inspector
             training, and ensure the reliability of information on plant appeals.


             For over a decade, scientific studies have highlighted the need for a new
Background   approach to ensure food safety. In 1983 and again in 1987, the National
             Academy of Sciences reported on the need for improved methods that
             focused on the prevention and reduction of microbial pathogens. The
             Academy and other organizations, including GAO, endorsed the HACCP
             approach as an effective tool for preventing and/or reducing hazards in the
             food supply. Following the recommendation of the National Advisory
             Committee on Microbiological Criteria for Foods,1 USDA issued the
             Pathogen Reduction and HACCP rule in July 1996. The rule establishes
             requirements for meat and poultry plants to reduce the occurrence of
             pathogenic microorganisms in their products, reduce the incidence of
             foodborne illness associated with meat and poultry products, and provide
             a new framework for the modernization of the current system of meat and
             poultry inspection. The HACCP rule applies to all federally and
             state-inspected meat and poultry slaughter and processing plants in the
             United States. Table 1 summarizes the implementation schedule for these
             plants. In USDA, FSIS has overall responsibility for overseeing HACCP
             implementation. Within FSIS, the Office of Field Operations oversees 18




             1
             This committee provides scientific advice and recommendations to the Secretaries of Agriculture,
             Commerce, Defense, and Health and Human Services.



             Page 4                           GAO/RCED-00-16 Implementation of New Food Safety System
                                       B-283823




                                       district offices throughout the country that conduct inspection and
                                       enforcement operations.2

Table 1: HACCP Implementation
Schedule for Meat and Poultry Plants   Plant size                           Number of plants                   Implementation date
                                       Large plants (500 employees Approximately 300 plants                    January 1998
                                       or more)
                                       Small plants (between 10 and Approximately 2,300 plants                 January 1999
                                       499 employees)
                                       Very small plants (less than     Approximately 3,300 plants             January 2000
                                       10 employees or annual
                                       sales of less than $2.5 million)

                                       To prepare its inspectors for their new responsibilities under the HACCP
                                       rule, FSIS developed an 8-day HACCP training course. As of February 1999,
                                       about 4,100 inspectors (out of a total of about 4,500 to be trained in the
                                       HACCP program) had participated in the course. Two USDA facilitators—one
                                       representing FSIS management and one representing the inspectors’
                                       union—led the training courses. The courses were offered on a
                                       “just-in-time” basis to coincide with the 3-year HACCP implementation
                                       schedule. The facilitators followed a carefully scripted program, consisting
                                       primarily of videos as a teaching tool, to ensure consistency among the
                                       classes given in different locations.


                                       USDA’s final HACCP rules embody the seven HACCP principles endorsed by
USDA’s Rules Embody                    the Advisory Committee. While the HACCP plans we reviewed during our
HACCP Principles,                      visits to 32 plants were based on these principles, many were at odds with
but Some Plants Rely                   USDA regulations requiring that HACCP plans be self-contained documents.
                                       USDA regulations require that all food safety hazards that are reasonably
on Other Programs to                   likely to occur must be controlled through their HACCP plans so that
Control Hazards                        inspectors can verify that critical limits are being met and that corrective
                                       actions are effective when those limits are exceeded. When hazards are
                                       controlled through non-HACCP programs, USDA’s ability to monitor the
                                       production of safe food is limited.


USDA Regulations Adopt                 As early as 1989, the Advisory Committee endorsed HACCP as an effective
Advisory Committee’s                   and rational approach to ensuring the safety of meat and poultry
HACCP Principles                       products—one that stresses preventing contamination before it occurs
                                       rather than dealing with it after it is detected. The Advisory Committee ’s

                                       2
                                        At the time of our review, FSIS Field Operations had 18 district offices. Subsequently, FSIS
                                       reorganized its field operations into 17 district offices. Throughout this report, however, information
                                       on appeals and survey responses is based on the 18-district office structure.



                                       Page 5                             GAO/RCED-00-16 Implementation of New Food Safety System
                                  B-283823




                                  HACCP Principles and Application Guidelines served as the source
                                  document for USDA’s July 1996 HACCP implementing regulation: Pathogen
                                  Reduction; Hazard Analysis and Critical Control Point (HACCP) Systems.3
                                  Our analysis of USDA regulations and the Advisory Committee’s HACCP
                                  guidelines found little if any deviation from the Advisory Committee’s
                                  seven HACCP principles. Table 2 summarizes these principles.

Table 2: Seven HACCP Principles
                                  Principle                        Description
                                  Principle 1:                     Plants determine the food safety hazards that are
                                  Conduct a hazard analysis.       reasonably likely to occur and identify the preventive
                                                                   measures they will apply to control these hazards.
                                                                   Hazards can be biological, chemical, or physical.
                                  Principle 2:                      Plants identify a point, step, or procedure in a food
                                  Identify critical control points. production process where controls can be applied to
                                                                    prevent, eliminate, or reduce a food safety hazard to an
                                                                    acceptable level.
                                  Principle 3:                     Plants set the maximum or minimum value at which a
                                  Establish critical limits for    biological, chemical, or physical hazard must be
                                  each critical control point.     controlled at each critical control point to prevent,
                                                                   eliminate, or reduce the food safety hazard to an
                                                                   acceptable level.
                                  Principle 4:                     Plants establish monitoring activities that will ensure the
                                  Establish monitoring             process is under control at each critical control point.
                                  requirements.
                                  Principle 5:                     Plants define actions to be taken when monitoring
                                  Establish corrective actions.    discloses a deviation from an established critical limit.
                                  Principle 6:                     Plants are required to maintain documentation of their
                                  Establish record-keeping         hazard analysis and HACCP plan, as well as records
                                  procedures.                      documenting the monitoring of critical control points,
                                                                   critical limits, verification activities, and the handling of
                                                                   processing deviations.
                                  Principle 7:                     Plants establish verification procedures to ensure that
                                  Establish verification           HACCP plans accomplish their intended goal—ensuring
                                  procedures.                      the production of safe products.

                                  The HACCP regulations require plants to address each of the seven
                                  principles during the development of their HACCP plans. A food safety
                                  hazard that is reasonably likely to occur is one for which a meat or poultry
                                  plant would establish controls because the hazard has occurred in the past
                                  or because there is a reasonable possibility that it will occur in the
                                  absence of those controls. Table 3 describes the three types of hazards
                                  that may occur in meat and poultry plants.




                                  3
                                   9 CFR Part 304, et al.



                                  Page 6                          GAO/RCED-00-16 Implementation of New Food Safety System
                                         B-283823




Table 3: Biological, Chemical, and
Physical Hazards                         Type of hazard                     Definition
                                         Biological                         Living organisms that can put human health at risk:
                                                                            bacteria, parasites, protozoa, and viruses. Some of the
                                                                            major organisms that can cause foodborne illness from
                                                                            eating meat and poultry products are salmonella,listeria,
                                                                            campylobacter, and E.coli 0157:H7.
                                         Chemical                           Naturally occurring substances, such as aflatoxins and
                                                                            mycotoxins, and added substances, such as pesticides,
                                                                            fungicides, fertilizers, lubricants, or cleaners.
                                         Physical                           Foreign materials not normally found in a food product
                                                                            that can cause illness or injury, such as glass, metal, and
                                                                            plastic.

                                         Since publishing its HACCP regulations in July 1996, USDA has issued several
                                         clarifications and modifications, including a requirement that all HACCP
                                         plans must contain at least one critical control point and must be
                                         self-contained documents that do not reference good manufacturing
                                         practices as mechanisms for controlling hazards.

                                         In addition to requiring the development of HACCP plans, plants must
                                         comply with the following other requirements of the regulations:

                                     •   Plants must develop and implement written Sanitation Standard Operating
                                         Procedures as a prerequisite to HACCP implementation.4
                                     •   Slaughter plants must regularly test for the presence of E. coli bacteria to
                                         verify the adequacy of their process controls for preventing and removing
                                         fecal contamination and associated bacteria.
                                     •   Plants that produce raw or ground beef, raw or ground chicken, ground
                                         turkey, or raw pork products must meet certain pathogen reduction
                                         performance standards for salmonella. This provision sets targets for
                                         reducing the incidence of salmonella contamination and requires that
                                         products sampled and tested for salmonella not test positive at rates
                                         exceeding the standard for each class of product. For example, the
                                         maximum number of positive tests allowed per 55 hogs tested are 6 (or
                                         8.7 percent), while the maximum number allowed per 51 chickens is 12 (or
                                         20 percent). The salmonella performance standards provide a substantive
                                         basis for judging the effectiveness of HACCP programs, according to USDA.

                                         In its totality, USDA’s HACCP regulatory framework exceeds the
                                         requirements of the seven principles of the HACCP approach as endorsed by
                                         the Advisory Committee.

                                         4
                                          These procedures describe all sanitation procedures that meat and poultry plants conduct before and
                                         during daily operations to prevent direct contamination of their products.



                                         Page 7                            GAO/RCED-00-16 Implementation of New Food Safety System
                             B-283823




HACCP Plans Follow           In our on-site review of 57 HACCP plans at 32 plants, we found that each
Seven Principles, but Some   plan was developed using the seven principles and included at least one
Plants Control Hazards       critical control point for controlling biological hazards.5 Because HACCP
                             plans are proprietary documents, many of the plants did not authorize us
Through Mechanisms           to make copies of their HACCP plans for further analysis.6 However,
Other Than HACCP             according to our analysis of the 28 plans that we were allowed to study in
                             detail, 13 contained statements indicating that a particular food safety
                             hazard was not reasonably likely to occur because it was controlled
                             through good manufacturing practices. Thus, these HACCP plans were not
                             the self-contained documents that USDA has required since January 1998.
                             For example, one plan included a statement that microbial hazards were
                             not reasonably likely to occur during product storage because the storage
                             temperature and condition of the coolers was sufficiently controlled
                             through good manufacturing practices.

                             USDA inspectors do not have the authority to approve HACCP plans. They
                             have only the authority to verify that the plans establish critical control
                             points for controlling the food safety hazards that plants identified during
                             the hazard analysis phase of developing their HACCP plans. Some inspectors
                             told us that they cannot verify other control mechanisms, such as good
                             manufacturing practices, even though they are referenced in the plant’s
                             HACCP plans. When inspectors learn that plants are referring to good
                             manufacturing practices in their HACCP plans, they are instructed to refer
                             these cases to FSIS district management for further review. District
                             managers can then request the plants to reassess their HACCP plans within
                             30 days to bring them into compliance with HACCP requirements. However,
                             USDA directed the plants to reassess only one of the 13 HACCP plans that we
                             had identified as making reference to good manufacturing practices.

                             Furthermore, plant managers representing 12 of the 32 plants we visited
                             told us that they identified and controlled one or more food safety hazards
                             through process control mechanisms other than their HACCP plans. For
                             example, some managers said they have metal detectors to control the
                             incidence of metal contamination in their products; however, they elected
                             not to include those in their HACCP plans. Other managers reported
                             controlling the condition and temperature of the incoming product, the
                             final shipment temperature, and chemical contamination through total
                             quality control programs and good manufacturing practices. One manager
                             told us that his plant had originally included several critical control points

                             5
                              The regulations state that plants may have one or more HACCP plans addressing various production
                             processes or products. Accordingly, some of the 32 plants we visited had more than one HACCP plan.
                             6
                              All 32 plants allowed us to look at their HACCP plans during our visits. In total, we reviewed 57 plans.



                             Page 8                             GAO/RCED-00-16 Implementation of New Food Safety System
B-283823




in its HACCP plan, but because USDA inspectors were finding too many
instances of noncompliance, the control points were moved to the plant’s
overall quality control program. Another manager said that he controls the
temperature of all products awaiting shipment from his plant through
good manufacturing practices because doing so under the plant’s HACCP
plan would require a large amount of time-consuming paperwork and
would require the plant to include the control in its pre-shipment reviews.

Plants may be opting to control some hazards through programs other
than HACCP systems in part because of the National Advisory Committee
and the industry’s support of the use of such an approach. The Advisory
Committee maintains that the production of safe food products requires a
HACCP system to be built on a solid foundation of other control programs,
such as good manufacturing practices, quality control programs, and
standard operating procedures. It further states that such programs are
often proprietary and established and managed separately from a HACCP
plan. Industry supports the use of good manufacturing practices to control
hazards that are reasonably likely to occur. For example, industry
representatives said that in a facility with well-functioning prerequisite
programs that control product temperature, bacteriological hazards are
not likely to occur and thus should be excluded from a HACCP plan. On the
other hand, industry representatives acknowledge that in a facility without
such prerequisite programs or where such programs are not well managed,
the loss of temperature control could allow the growth of microbiological
hazards that can be better managed through a HACCP plan.

USDA recognizes the usefulness of prerequisite programs, such as quality
control programs and good manufacturing practices, in the production of
safe products. However, USDA maintains that to ensure the effective
oversight of HACCP systems, it must be able to verify plants’ compliance
with HACCP regulations, and inspectors cannot verify non-HACCP plans.
Therefore, USDA requires plants to identify and control through their HACCP
plans all hazards reasonably likely to occur. HACCP plans that refer to such
programs as good manufacturing practices limit federal oversight and thus
are not in compliance with the regulations.




Page 9                   GAO/RCED-00-16 Implementation of New Food Safety System
                        B-283823




                        According to USDA inspectors and our review of the training materials,
Inspectors Generally    inspectors generally received the training they needed to verify that meat
Reported Satisfaction   and poultry plants comply with HACCP principles. In addition, their training
With HACCP Training     provided a basic introduction to the scientific principles of the HACCP
                        approach while focusing on the compliance aspects of their duties. Nearly
but Also Identified     two-thirds (65 percent) of the inspectors responding to our survey
Weaknesses              indicated that the training they received adequately prepared them for
                        their HACCP duties; however, as many inspectors also indicated that they
                        would benefit from refresher courses. Survey responses also identified
                        training weaknesses in the following areas: (1) inspectors’ authority to ask
                        for changes to a plant’s HACCP plan, (2) frequency of salmonella sample
                        collections, (3) inspectors’ authority to take action when a plant’s
                        microbial testing program not cited in the plant’s HACCP plan detects
                        contamination, and (4) inspectors’ issuance of noncompliance notices. In
                        its 1998 evaluation of inspection activities during the first phase of HACCP
                        implementation, USDA also identified areas of training that should be
                        strengthened.7


Inspectors’ Training    The HACCP system shifts the responsibility for ensuring the safety of meat
Emphasizes Compliance   and poultry products from USDA to industry. Therefore, HACCP training
                        programs for industry and for USDA inspectors reflect their different roles
                        and responsibilities. The regulations require that plant managers in charge
                        of HACCP operations be trained in the application of the seven HACCP
                        principles, including the development of a HACCP plan. USDA’s role, on the
                        other hand, is to ensure that plants comply with HACCP regulations. Thus,
                        the Department’s inspector training emphasizes compliance rather than
                        the development and implementation of HACCP plans.

                        In 1994, the International HACCP Alliance was formed to help the meat and
                        poultry industry prepare for HACCP implementation by, among other things,
                        standardizing HACCP training for plant personnel.8 The Alliance is not
                        involved in training; rather, it reviews and approves HACCP training
                        curricula before accrediting other organizations’ training programs. The
                        curriculum for industry includes courses on how to recognize the
                        relationship between the HACCP principles and food safety; how to identify
                        and control hazards; how to design control measures to prevent, reduce,
                        or minimize hazards; and how to identify critical control points using valid

                        7
                         Evaluation of Inspection Activities During Phase One of HACCP Implementation, Food Safety and
                        Inspection Service, United States Department of Agriculture, July 1998.
                        8
                         The HACCP Alliance includes over 100 members from industry associations, educational foundations,
                        professional societies, universities, government agencies and related private companies.



                        Page 10                          GAO/RCED-00-16 Implementation of New Food Safety System
                           B-283823




                           scientific criteria. Individuals who complete an Alliance course are said to
                           be HACCP-certified, although there is no requirement for such certification
                           in the HACCP regulations. Nor is there a requirement for industry HACCP
                           personnel to provide proof of such training to USDA inspection personnel.

                           In contrast with industry’s training, USDA’s training of its inspection
                           workforce is in-house, and its program focuses on how inspectors verify
                           plants’ compliance with their HACCP plans. Because the inspectors’
                           responsibilities under the HACCP regulations are to document breakdowns
                           in plant-established controls, their training includes only an introduction
                           to the science-based HACCP principles; it does not stress the scientific basis
                           behind hazard analyses, critical control points, and critical limits. Of the 12
                           separate modules offered in USDA’s HACCP training program, only one
                           covers the science-based HACCP principles.


USDA Inspectors Were       Nearly two-thirds of the respondents to our nationwide survey rated their
Generally Satisfied With   overall preparation for work as HACCP inspectors after training as good or
Training but Identified    excellent, while as many said that they would benefit from refresher
                           courses. Over two-thirds rated their course facilitators as good or
Some Weaknesses            excellent, and more than half said that the training was provided just in
                           time. However, only 28 percent of the respondents said that the real-life
                           examples used during training were very useful.

                           To examine in more detail the strengths and weaknesses of USDA’s training,
                           we asked inspectors to rate their understanding of several specific HACCP
                           concepts addressed in training. Most respondents (81 to 85 percent) rated
                           their understanding of various aspects of the microbial testing required by
                           the HACCP regulations as good or excellent. However, fewer respondents
                           (53 to 59 percent) rated their understanding of how to recognize a HACCP
                           system failure and how HACCP plans are developed as good or excellent.
                           Table 4 summarizes the inspectors’ responses to these questions.




                           Page 11                   GAO/RCED-00-16 Implementation of New Food Safety System
                                 B-283823




Table 4: Understanding HACCP
Concepts Discussed in Training   Understanding of HACCP concepts rated “good” to
                                 “excellent”                                                                    Percent
                                 The proper method for E. coli sampling                                              85
                                 The proper method for salmonella sampling                                           85
                                 The purpose of collecting E. coli and salmonella samples                            84
                                 Inspectors’ responsibilities regarding salmonella sampling                          82
                                 Inspectors’ responsibilities regarding E. coli sampling                             81
                                 The difference between the old inspection system and the
                                 new HACCP system                                                                    72
                                 What it means to verify a HACCP plan                                                63
                                 The difference between the old and the new Performance-
                                 Based Inspection System                                                             63
                                 The changes in standard sanitation operating procedures
                                 inspections                                                                         62
                                 The difference between basic inspections and system
                                 inspections                                                                         61
                                 How to recognize a HACCP system failure                                             59
                                 How plants develop their HACCP plans                                                53
                                 Note: 161 inspectors responded to these questions.



                                 We also asked the inspectors to rate their preparation for performing
                                 specific HACCP-based inspection tasks after their training. Over
                                 three-quarters of the respondents (77 percent) reported that their
                                 preparation for conducting day-to-day HACCP compliance activities was
                                 good to excellent, but fewer (56 percent) said that their preparation for
                                 using the appropriate trend indicators was good to excellent. USDA uses
                                 trend indicators to categorize the specific types of noncompliance
                                 identified by inspectors in HACCP plants. For example, if an inspection
                                 procedure revealed that a plant employee had not initialed and dated an
                                 entry on a record required by the HACCP plan, the inspector would mark the
                                 record-keeping trend indicator on the noncompliance record. USDA also
                                 uses trend indicators to determine whether additional regulatory or
                                 administrative actions should be taken. Table 5 summarizes the inspectors’
                                 responses to the group of questions dealing with preparation for HACCP
                                 duties after training.




                                 Page 12                         GAO/RCED-00-16 Implementation of New Food Safety System
                                            B-283823




Table 5: Preparation for HACCP Duties
After Training                              Preparation for HACCP duties rated as “good” to
                                            “excellent”                                                                                Percent
                                            Day-to-day verification of plant compliance                                                       77
                                            The resources to use if uncertain about HACCP principles                                          77
                                            Taking action in cases of noncompliance                                                           70
                                            Verifying that the HACCP plan complies with requirements                                          69
                                            Handling disputes over noncompliance notices                                                      68
                                            Tracking changes to the HACCP plan                                                                63
                                            Responding to appeals by plants                                                                   61
                                            Overseeing plants’ testing for E. coli                                                            59
                                            Using trend indicators when issuing noncompliance notices                                         56
                                            Note: 161 inspectors responded to these questions.



                                            Inspectors’ responses to another set of questions showed a significant
                                            degree of confusion. We asked inspectors about three specific situations
                                            covered in training that appeared to be unclear or ambiguous in the HACCP
                                            courses we attended. In each case, we asked the inspectors to select the
                                            response closest to what they had learned in training. Furthermore, we
                                            asked the FSIS Technical Services Center to provide us with what it
                                            considers to be the correct response to each of the questions.9 The
                                            following summarizes the inspectors’ responses to the three questions and
                                            the percent that selected the correct answer to each question, according to
                                            the Technical Services Center:

                                        •   Are inspectors allowed to ask for changes to the HACCP plan when they
                                            conduct basic HACCP compliance procedures? According to the Technical
                                            Services Center, the correct response is that inspectors are not allowed to
                                            request plants to make changes to their HACCP plans, even if they believe
                                            changes are necessary. Sixty-one percent of the respondents to our survey
                                            selected the correct response, 21 percent said that inspectors are allowed
                                            to ask for changes, and 17 percent were uncertain. In total, about one-third
                                            of the respondents were uncertain or provided the incorrect response to
                                            this question.
                                        •   How frequently should inspectors collect salmonella samples at the plant
                                            after being notified by their USDA district management office to begin
                                            taking samples? According to the Technical Services Center, the correct
                                            answer is that salmonella samples should be collected every day the


                                            9
                                             In June 1997, FSIS established the Technical Service Center to provide accurate and consistent
                                            information regarding the development and implementation of inspection programs to inspection
                                            personnel and industry.



                                            Page 13                          GAO/RCED-00-16 Implementation of New Food Safety System
    B-283823




    product is produced. Sixty-five percent of the respondents to our survey
    selected the correct response, 12 percent said “as time permits,” and
    20 percent were uncertain. In total, about one-third of the respondents
    provided the incorrect answer to this question or were uncertain.
•   Are inspectors allowed to take action if they become aware of microbial
    contamination identified through testing programs (other than E.coli and
    salmonella) that are not part of the HACCP plans? According to the
    Technical Services Center, the correct answer is that inspectors have the
    authority to take whatever action they deem necessary when they suspect
    product contamination, whether or not the plant has included testing for
    microbiological hazards (other than E-coli and salmonella) in its HACCP
    plan. Thirty-two percent of the respondents selected the correct response,
    44 percent selected the incorrect response, and 21 percent were uncertain.
    Nearly two-thirds (65 percent) of the respondents gave the incorrect
    answer or were uncertain.

    In summary, the responses to these three questions indicate considerable
    confusion about these aspects of an inspector’s HACCP duties. For example,
    only 13 percent of the inspectors selected the responses that the Center
    provided as the correct answers to all three questions, and one-third
    provided the correct response to only one of the three questions.

    There is also confusion concerning how the training program that
    inspectors attended addressed their responsibilities for HACCP compliance.
    Specifically, we asked the inspectors whether they were taught to (1) issue
    a noncompliance record as soon as a noncompliance is detected or
    (2) allow the HACCP system to work first. Recording instances of
    noncompliance with HACCP systems is a very important aspect of the
    inspectors’ duties, and the training courses we attended suggested that
    inspectors should allow the HACCP system to work first. However, the
    Technical Services Center said that there is no definitive answer.10
    Accordingly, the inspectors responses varied—67 percent said they should
    give a HACCP system a chance to work first in a noncompliance situation,
    25 percent said they should issue a noncompliance record immediately
    when noticing a noncompliance with HACCP requirements, and 8 percent
    were either uncertain or had other responses.




    10
      The Technical Services Center told us that there is no definitive response to this question, and other
    FSIS officials said that the answer depends on the type of HACCP inspection being conducted. That is,
    if inspectors conducting a basic verification procedure find an instance of noncompliance, they should
    immediately take action. On the other hand, when conducting a procedure to verify an entire HACCP
    system, inspectors should allow the HACCP system to work first.



    Page 14                            GAO/RCED-00-16 Implementation of New Food Safety System
                        B-283823




                        During our plant visits, inspectors recommended that HACCP training place
                        more emphasis on when and how to prepare noncompliance notices and
                        how to use noncompliance trend indicators. Plant managers also
                        frequently stated that inspectors should do a better job of documenting
                        instances of noncompliance and selecting the appropriate trend indicators
                        for each instance. Finally, a 1998 USDA evaluation of HACCP implementation
                        pointed out that inspection personnel expressed a need for further
                        understanding of how to use trend indicators. Appendix I presents
                        inspectors’ responses to our nationwide survey.


                        The appeals process under the HACCP regulations for plants charged with
Dispute Resolution      noncompliance appears to provide an adequate avenue of redress. The
Process Is Working,     plant managers and inspectors we interviewed generally knew how the
but the Tracking        process worked. However, about half of the plant managers had concerns
                        about the accuracy of the information contained in the inspectors’
System Has Certain      noncompliance notices and said that they tend to appeal noncompliance
Weaknesses              findings they believe are inaccurate. We could not determine the nature
                        and extent of these appeals because FSIS’ Industry Appeals Tracking
                        System lacks complete and consistent data.


Appeals Process Under   Inspectors are required to issue noncompliance notices to plants failing to
HACCP Regulations       comply with HACCP regulations. Multiple, recurring findings of
Appears Adequate        noncompliance without successful interventions to correct a problem can
                        result in additional enforcement actions, such as the suspension of
                        inspection, which could result in slowing production or shutting down the
                        plant.

                        The right to appeal enforcement actions and the process for making such
                        appeals have not changed since the HACCP regulations were implemented,
                        and, as before, the existing mechanism provides plants “due process” as
                        required by law. That is, the plants have an avenue to appeal enforcement
                        actions that they believe are incorrect. The regulations give the plants the
                        right to appeal inspectors’ findings and decisions orally or in writing.11 An
                        appeal must first be submitted to the inspector with the most immediate
                        jurisdiction over the appeal–in most cases, the inspector-in-charge. If an
                        appeal is rejected at this level, the plant can appeal further through the FSIS
                        chain of command—circuit supervisor, district manager, Assistant Deputy
                        Administrator for District Inspection Operations, and Deputy
                        Administrator for the Office of Field Operations. All of the inspectors and

                        11
                          9 CFR 306.5 and 9 CFR 381.35.



                        Page 15                           GAO/RCED-00-16 Implementation of New Food Safety System
                           B-283823




                           plant officials we interviewed were aware of their plants’ right to appeal,
                           and almost all were familiar with the process and how it works. FSIS
                           officials and inspectors said that with the implementation of HACCP
                           requirements, FSIS has made a concerted effort to inform plants of their
                           right to appeal enforcement actions and the process for doing so.

                           While the appeals procedures have not changed under the HACCP
                           regulations, some inspectors and plant officials perceived that the process
                           is more formal now than it used to be. For example, several managers told
                           us that under the old inspection system, plants generally made oral
                           appeals, but now because of FSIS’ preference, plants generally submit
                           written appeals. Similarly, inspectors said they used to explain orally why
                           an appeal was rejected or granted, but now they are encouraged to provide
                           plants with written explanations.


Concerns Over Inaccurate   When issuing noncompliance notices, inspectors are required, among
Noncompliance Notices      other things, to cite the applicable regulation that was violated and to
Result in Plant Appeals    identify the trend indicator that specifies the type of noncompliance
                           observed. Inspectors review noncompliance notices periodically to
                           identify noncompliance trends, such as a plant’s repeatedly failing to
                           monitor a critical control point or repeatedly failing to take effective
                           corrective actions. However, about half of the 32 plant managers we
                           interviewed expressed concerns about the accuracy of the information
                           contained in the noncompliance notices issued to their plants. Some
                           managers also told us that inspectors often issue noncompliance notices
                           for violations that are in the process of being corrected, as outlined in
                           their HACCP corrective action plans, before the corrective action had a
                           chance to work.

                           Accurately prepared noncompliance notices are essential because USDA
                           uses trend data to decide whether additional enforcement action at a plant
                           is necessary. Plant managers are concerned because USDA has not issued
                           guidance on the use of trend data or clarified how many instances of
                           noncompliance would result in a plant’s being shut down. Accordingly, 17
                           plant managers said they appeal inspectors’ noncompliance notices when
                           they contain factual errors, incorrect regulatory citations, improper trend
                           indicators, and/or when they were issued before the plant was allowed
                           adequate time to implement the corrective action. For example, one
                           manager emphasized that his plant had never appealed a noncompliance
                           notice prior to the HACCP regulations, but now he appeals because there is
                           no guidance on the number of noncompliance notices that could result in



                           Page 16                   GAO/RCED-00-16 Implementation of New Food Safety System
                      B-283823




                      USDA’s slowing down production or shutting the plant. Similarly, three
                      other plant managers said that the National Meat Association told them to
                      appeal all noncompliance notices for the aforementioned reasons.


Data in the Appeals   During 1998, USDA established the automated Industry Appeals Tracking
Tracking System Are   System to monitor industry appeals. Inspectors are required to submit
Incomplete and        appeals information to their district offices weekly for compilation into
                      this tracking system. The information is to include the names and
Inconsistent          addresses of the plants making the appeals; the dates the appeals were
                      made, resolved, and closed; the numbers of related noncompliance
                      notices, descriptions of the decisions or actions that a plant is appealing;
                      and explanations of the resolution. FSIS’ instructions for completing the
                      tracking report provide inspectors with an example of the information that
                      should be submitted.

                      While data in the appeals tracking system are incomplete and inconsistent,
                      they still show that plants make use of the appeals system. For example,
                      from June 6, 1998, through September 7, 1999, 234 plants made a total of
                      1,564 appeals. Of the 1,564 appeals, 1,051 were denied, 449 were granted,
                      and 64 were under review. Plants located in 6 of the 18 FSIS
                      districts—Springdale, Des Moines, Jackson, Alameda, Dallas, and
                      Raleigh—filed 1,099 of the appeals. Table 6 summarizes these data.




                      Page 17                  GAO/RCED-00-16 Implementation of New Food Safety System
                                         B-283823




Table 6: Appeals Data From FSIS Industry Appeals Tracking System, June 6, 1998, Through September 7, 1999
                                                     Number of                  Number of    Number of                         Number of
                                        Number of        plants        Total       appeals     appeals                      appeals under
FSIS district            States            plants    appealing      appeals         denied     granted                             review
Springdale             AR, LA, OK               120             27              251             184               64                       3
Des Moines             IA, NE                   140             26              202             131               69                       2
Jackson                AL, MS, TN               157             23              190             144               44                       2
Alameda                CA                       298             23              129             106               22                       1
Dallas                 TX                       181             22              208             118               77                      13
Lawrence               KS, MO                     90            17               78               40              37                       1
Chicago                IL, IN                   224             11               83               54              28                       1
Raleigh                NC, SC                   107             11              119               81              33                       5
Atlanta                FL, GA                   147             10               82               70              12                       0
Minneapolis            MN, MT, ND,
                       SD, WY                     91            11               50               33              17                       0
Beltsvillea            DE, MD, VA                 80              9              46               18                7                     21
Philadelphia           PA                       154               9              25               21                4                      0
Madison                MI, WI                   132               9              25               16                6                      3
Boulder                AZ, CO, NM,
                       NV, UT                     77              7              37               13              14                      10
Albany                 NJ, NY                   233               7              12                7                5                      0
Pickerington           KY, OH, WV               118               5              17               12                5                      0
         b
Boston                 CT, ME, MA,
                       NH, RI,VT                  99              5                6               3                2                      1
        c
Salem                  AK, HI, ID,
                       OR, WA                   107               2                4               3                0                      1
Total                                          2,555           234            1,564           1,051              449                      64
                                         a
                                         Includes the District of Columbia.
                                         b
                                           Subsequent to our review, the Boston District Office was closed because FSIS reorganized its
                                         field office operations.
                                         c
                                             Includes American Samoa and Guam.



                                         When we attempted to analyze the extent to which plants appeal
                                         noncompliance notices they believe to be inaccurate, we found that the
                                         data contained in the Industry Appeals Tracking System were inconsistent
                                         and incomplete. Inspectors and district managers do not always submit
                                         thorough descriptions and explanations of the appeal cases. For example,
                                         some district offices provided only a description of the noncompliance
                                         notice being appealed, but no explanation of the basis for granting or
                                         denying the appeal. Other district offices provided only partial



                                         Page 18                          GAO/RCED-00-16 Implementation of New Food Safety System
              B-283823




              descriptions or did not include any kind of description. Furthermore, in
              comparing the appeals listed on the appeals tracking report with the lists
              of appeals provided by seven of the plants we visited, we found that the
              tracking report excluded 9 of the 64 appeals one plant had made and 7 of
              the 25 appeals another plant had made. In addition, the appeals tracking
              system did not contain any data for five plants, although these plants had
              received written appeals determinations from FSIS.

              These irregularities can be attributed in part to (1) inspectors’ not
              submitting appeals data to their respective district offices for entry into
              the tracking system, (2) inspectors’ time constraints, or (3) FSIS data entry
              errors. For example, an inspector and a circuit supervisor told us that they
              did not submit such information to the district office because they did not
              know they were required to do so. Another inspector said that he did not
              have sufficient time to complete the required data submissions and that
              the instructions were too confusing. Finally, some district offices
              submitted inaccurate entries to FSIS headquarters for entry to the tracking
              system, and some entries were incorrectly entered into the system at the
              headquarters level.


              USDA’s adoption of the HACCP system is designed to prevent or reduce
Conclusions   contamination in meat and poultry plants and thus enhance the safety of
              the products they produce. USDA regulations implementing the HACCP
              system are consistent with the seven HACCP principles endorsed by the
              National Advisory Committee on Microbiological Criteria for Foods but
              also contain regulatory provisions to ensure USDA’s oversight. Although
              USDA does not discourage plants from using quality control or good
              manufacturing practices programs, it expects them to identify and control
              food safety hazards that are reasonably likely to occur thorough their
              HACCP plans. USDA does not allow plants to refer to good manufacturing
              practices in their HACCP plans as mechanisms for controlling such hazards.
              However, some meat and poultry plants continue to rely on non-HACCP
              programs to control these types of hazards. This practice limits the
              consistent implementation of the HACCP system nationwide as well as
              USDA’s oversight of food safety at these plants.


              HACCP training for inspectors was for the most part sufficient to prepare
              them for their new inspection duties. However, weaknesses in the training
              program—such as whether inspectors have the authority to ask for
              changes to a HACCP plan, when they should collect salmonella samples, and




              Page 19                   GAO/RCED-00-16 Implementation of New Food Safety System
                         B-283823




                         when it is appropriate to issue noncompliance notices—affect USDA’s
                         ability to ensure the consistent and effective oversight of the HACCP system.

                         USDA’sprocess for resolving disputes provides industry with an avenue to
                         appeal enforcement actions. However, inconsistent and incomplete data
                         preclude FSIS from effectively analyzing the types of HACCP-related
                         noncompliance actions that are appealed or the extent to which plants
                         appeal inaccurate noncompliance notices.


                         To enable USDA to oversee all aspects of food safety at meat and poultry
Recommendations to       plants, we recommend that the Secretary of Agriculture direct the
the Secretary of         Administrator of FSIS to review all HACCP plans to verify that plants are
Agriculture              identifying and controlling food safety hazards that are reasonably likely
                         to occur through their HACCP programs. This review will ensure that USDA
                         appropriately oversees the HACCP system.

                         To ensure the consistent inspection of HACCP programs, we recommend
                         that the Secretary direct the Administrator of FSIS to provide clarification
                         and additional training for inspectors in the following areas:

                     •   inspectors’ roles, responsibilities, and authorities for reviewing and
                         verifying HACCP plans;
                     •   inspectors’ responsibilities for microbial sampling and the frequency of
                         salmonella testing;
                     •   inspectors’ responsibilities for how and when to file noncompliance
                         notices and how to select the correct trend indicators.

                         To ensure that FSIS has complete and accurate information for analyzing
                         industry appeals of noncompliance actions under the HACCP system, we
                         recommend that the Secretary of Agriculture direct the Administrator of
                         FSIS to (1) issue instructions to FSIS’ district offices clearly stating that
                         inspectors must provide complete, accurate, timely, and consistent
                         appeals data for the automated appeals tracking system and
                         (2) periodically review the accuracy and completeness of the data in the
                         appeals tracking system.


                         We provided USDA with a draft of this report for review and comment. We
Agency Comments          met with USDA officials, including FSIS’ Deputy Administrator, Field
and Our Response         Operations. Overall, USDA concurred with the conclusions and
                         recommendations contained in the report and found it accurate and



                         Page 20                   GAO/RCED-00-16 Implementation of New Food Safety System
B-283823




balanced. USDA also stated that FSIS has recently established a cadre of
HACCP experts who will review HACCP plans to ensure that plants are
identifying and controlling the hazards that are reasonably likely to occur.
USDA also noted that the Administrator of FSIS has recently appointed a
committee to address the role of training and education in FSIS, including
issues related to HACCP training. Beyond these overall observations, USDA
provided minor technical suggestions, which we incorporated into the
report as appropriate.

We conducted our review from February through November 1999 in
accordance with generally accepted government auditing standards.


We will send copies of this report to the congressional committees with
jurisdiction over food safety issues; the Honorable Dan Glickman,
Secretary of Agriculture, the Honorable Thomas Billy, Administrator, Food
Safety and Inspection Service; and other interested parties. We will also
make copies available to others on request.

If you have any questions about this report, please contact me at
(202) 512-5138. Key contributors to this report are listed in appendix III.




Lawrence J. Dyckman
Director, Food and
  Agriculture Issues




Page 21                   GAO/RCED-00-16 Implementation of New Food Safety System
Contents



Letter                                                                                             1


Appendix 1                                                                                        24
                       Telephone Survey                                                           24
Results of Telephone
Survey of Inspectors
Appendix II                                                                                       30

Objectives, Scope,
and Methodology
Appendix III                                                                                      32

GAO Contacts and
Staff
Acknowledgments
Tables                 Table 1: HACCP Implementation Schedule for Meat and Poultry                 5
                        Plants
                       Table 2: Seven HACCP Principles                                             6
                       Table 3: Biological, Chemical, and Physical Hazards                         7
                       Table 4: Understanding HACCP Concepts Discussed in Training                12
                       Table 5: Preparation for HACCP Duties After Training                       13
                       Table 6: Appeals Data from FSIS Industry Appeals Tracking                  18
                        System, June 6, 1998, Through September 7, 1999
                       Table II.1: Sampling Errors Associated with Survey Responses               31




                       Abbreviations

                       FSIS      Food Safety and Inspection Service
                       HACCP     Hazard Analysis and Critical Control Point
                       USDA      U.S. Department of Agriculture


                       Page 22                GAO/RCED-00-16 Implementation of New Food Safety System
Page 23   GAO/RCED-00-16 Implementation of New Food Safety System
Appendix 1

Results of Telephone Survey of Inspectors


                   This appendix presents a copy of the telephone questionnaire used to
                   survey 161 inspectors nationwide and the results of that survey.


                   Hello, my name is _______________. I am with the U.S. General
Telephone Survey   Accounting Office. GAO is an independent agency of the U.S. Congress, and
                   we are not associated with USDA. As part of a study we are conducting on
                   the new HACCP program for meat and poultry plants, we are conducting a
                   telephone survey of inspectors. We are calling inspectors who attended
                   the HACCP training to ask how well that training prepared them for the new
                   program. Your name appears on a nationwide list of inspectors who
                   attended HACCP training for small plant inspectors. Is that correct? (If caller
                   answers no, thank inspector and terminate interview.) This interview will
                   take about twenty minutes depending on your answers.

                   OK. Let me read our introduction. Your name was one of a sample drawn
                   randomly to represent the views of inspectors nationwide on the training
                   program, so it is very important that I follow our interview format. If you
                   have any other comments about the HACCP program, I’d like to hold those
                   comments until we’ve finished the survey questions. At the end of the
                   interview, we can discuss any other issues relating to the HACCP
                   implementation. If there is any question in this interview that you prefer
                   not to answer, just tell me and we’ll skip on to the next question, OK?



                   1. I’d like to start our interview by asking about the   29% October 1998
                   two-week HACCP training you attended for small           26% November 1998
                   plant inspectors. In what month and year did you         24% December 1998
                   complete that HACCP training? N=161                      21% Other months
                   2. Were you able to complete the entire training         96% Entire training
                   program or did you miss 1 day or more of the             3% Missed 1 day or more
                   training? N=161                                          1% Other




                   Page 24                         GAO/RCED-00-16 Implementation of New Food Safety System
                                                 Appendix 1
                                                 Results of Telephone Survey of Inspectors




                                                     Excellent (1)         Good (2)          Fair (3)        Poor (4)    No opinion (5)
3. Training for new programs such as HACCP
is often challenging because the ideas are
new and unfamiliar. This next set of questions
asks you to rate the understanding you
gained for new concepts covered in the
HACCP training. As I read each item, please
tell me how well you would rate your
understanding of that concept just after you
completed your training. Our categories for
these questions are excellent, good, fair,
poor, and no opinion.
a. The major differences between the                            12%               60%             22%               4%               2%
previous inspection system and the new
HACCP inspection system N=161
b. The differences between the old                              11%               52%             26%               9%               3%
Performance Based Inspection
System—PBIS—and the revised one N=161
c. The difference between basic requirements                    14%               47%             33%               6%               1%
and other requirements for
compliance/noncompliance actions. That is,
the difference between 01 and 02 procedures
N=160
d. What it means to verify a HACCP plan                         12%               51%             32%               5%               0%
N=161
e. How to recognize a HACCP system failure                      14%               45%             33%               8%               1%
N=161
f. How plants develop their HACCP plans                         18%               35%             35%              11%               1%
N=161
g. The purpose of collecting E. coli and                        33%               51%              9%               4%               3%
salmonella samples N=160
h. The inspector’s responsibilities regarding                   23%               58%             12%               7%               1%
E. coli sampling N=161
i. The proper method for E. coli sampling                       29%               56%             11%               2%               3%
N=160
j. The inspector’s responsibilities regarding                   24%               58%             13%               4%               2%
salmonella sampling N=161
k. The proper method for salmonella sampling                    29%               56%             12%               2%               1%
N=160
l. The changes in SSOP inspections N=161                        12%               50%             29%               6%               4%




                                                 Page 25                       GAO/RCED-00-16 Implementation of New Food Safety System
                                                  Appendix 1
                                                  Results of Telephone Survey of Inspectors




                                                               Excellent (1)      Good (2)       Fair (3)     Poor (4)    No opinion (5)
4. We talked about how well you understood new
concepts just after the HACCP training. Now we’d like
to talk about the time period when you started using
the HACCP training on the job. We want to know how
well the training prepared you as you started
implementing your HACCP duties in the plant. As I
read a list of items, I’ll ask you to rate your preparation
to do each one. Again, our categories are excellent,
good, fair, poor, and no opinion.
a. What actions I need to take to verify that a HACCP                     14%            55%            26%          5%               1%
plan at my plant complies with requirements N=161
b. What day-to-day activities I should conduct to verify                  13%            64%            16%          6%               1%
compliance with the plant’s HACCP plan N=161
c. What actions I need to take in cases of                                21%            49%            19%         10%               1%
noncompliance N=161
d. How I handle disputes over noncompliance at the                        17%            51%            18%         11%               3%
plant N=161
e. What I need to do if there is an appeal of a                           17%            44%            24%         10%               4%
noncompliance record N=161
f. What I should do to oversee the plant’s testing for E.                 14%            45%            22%          9%              11%
coli N=161
g. How I should use the NR trend indicators to track                      10%            46%            30%         12%               1%
the plant’s HACCP system performance N=161
h. What I need to do to keep up on the plant’s changes                    12%            51%            29%          6%               2%
to the HACCP plan N=161
i. What resources I can use if I am uncertain about                       27%            50%            18%          4%               1%
HACCP compliance N=160




5. Now I’d like for you to think about all aspects of the HACCP program and      9% Excellent
how well the training prepared you, overall, for your work as a HACCP            56% Good
inspector. Again, our categories are excellent, good, fair, poor, and no         27% Fair
opinion. How do you rate your overall preparation for work as a HACCP            7% Poor
inspector? N=161                                                                 1% No opinion
                                                                                 1% Other
6. In this next set of questions, we’d like to ask about some general aspects    58% Just in time
of the HACCP training you attended. First, we’d like to know how you would       17% Too soon
rate the timeliness of your HACCP training—that is, whether the timing was       17% Not soon enough
right for you to conduct your responsibilities as an inspector. Which            8% Cannot say
category best fits your situation—just in time, too soon, not soon enough, or    0% Other (Describe.)
can’t you say? N=161
                                                                                                                             (continued)




                                                  Page 26                       GAO/RCED-00-16 Implementation of New Food Safety System
                                                Appendix 1
                                                Results of Telephone Survey of Inspectors




7. Another aspect of training is how well the facilitators are able to respond     37% Excellent
to questions from the audience. Overall, how would you rate the skill of the       32% Good
training facilitators in answering questions that came up during your training     22% Fair
sessions? Would you rate that as excellent, good, fair, poor, or would you         9% Poor
say no opinion? N=161                                                              0% No opinion
                                                                                   1% Other
8. Training is usually more helpful when “real-life” examples are used to          28% Very useful
practice new concepts. We’d like to ask how useful such real-life examples         48% Moderately useful
were during the training you attended. Would you say the examples used at          19% Not very useful
your training were very useful, moderately useful, not very useful, or would       4% No opinion
you say no opinion?                                                                1% Other (Describe.)
Next we’d like to ask about several situations covered in training where an
inspector must decide what to do. As I ask you about each situation, I’ll
read two ways an inspector could handle it. Then, I’ll ask you which way is
closer to how your training taught you to handle it, even if you decided to do
it differently at your plant. If you don’t remember a situation being covered
in the training, just tell me and we’ll go on to the next item, OK?
9.The first situation concerns whether or not you are allowed to ask for           21% It’s okay for me to ask plant management to make
changes to the plant’s HACCP plan. Here are the two options. First, it’s okay      changes to the HACCP plan
for me to ask plant managementto make changes to the HACCP plan, or,               61% It’s not okay for me to ask for changes to the HACCP
second, It’s not okay for me to ask for changes to the HACCP plan. Which           plan
way is closer to what you learned in training, or would you say that you are       17% Uncertain
uncertain? N=161                                                                   1% Other
10.The next situation concerns sampling after you have been notified to            12% I should take samples as time permits.
begin salmonella testing. Here are the two options. First, I should take           65% . I should take a sample every day the product is
samples as time permits, or, second, I should take a sample every day the          produced.
product is produced. Which way is closer to what you learned in training, or       20% Uncertain
would you say that you are uncertain? N=161                                        3% Other
11. The next situation concerns your decision to prepare a noncompliance           67% I should give the HACCP system a chance to work
record—NR. The first option is. . . I should give the HACCP system a               first without filing an NR in a noncompliance situation
chance to work first without filing an NR in a noncompliance situation, or,        25% I should immediately file an NR in situations of
second, I should immediately file an NR in situations of noncompliance. (As        noncompliance.
necessary:) Which statement is closer to what you learned in training, or          3% Uncertain
would you say you are uncertain? N=161                                             5% Other
12. And, the last situation in this group concerns your role regarding             44% I am not required to take action based on the test
microbial testing other than for E. coli and salmonella. In this situation, you    results unless the HACCP plan includes that type of
find out that the plant is testing for other pathogens and those results           testing
indicate contamination at the plant. The first option is, I am not required to     32% I should take whatever actions I think are necessary,
take action based on the test results unless the HACCP plan includes that          regardless of the HACCP plan
type of testing, or, second, Ishould take whatever actions I think are             21% Uncertain
necessary, regardless of the HACCP plan. (As necessary:) Which statement           3% Other
is closer to what you learned in training, or would you say you are
uncertain? N=161




                                                Page 27                           GAO/RCED-00-16 Implementation of New Food Safety System
                                                  Appendix 1
                                                  Results of Telephone Survey of Inspectors




                                                                                              Somewhat             Not very
                                                                        Very helpful (1)      helpful (2)         helpful (3)   No opinion (4)
13. Our next question is about ways that you might learn more
about the HACCP program. I’ll read a list of items and ask you
whether each one seems very helpful, somewhat helpful, or not
very helpful for learning more about HACCP. OK? The first one
is . . .[read first item]. How helpful would that be for you to learn
more about HACCP? Would you say it sounds very helpful,
somewhat helpful, not very helpful, or would you say you have
no opinion?
A. To attend a classroom, refresher training course N=161                            68%                26%                6%               1%
B. To have a training facilitator visit my plant N=161                               39%                35%               24%               3%
C. To receive a newsletter for inspectors N=161                                      47%                40%               11%               2%
D. To attend training that is given to plant employees N=160                         23%                36%               36%               5%
E. To use a self-paced computer training package N=161                               42%                42%               14%               3%
F. To participate in a computerized forum on the INTERNET
where inspectors share their HACCP experiences N=161                                 44%                42%                9%               4%




G. Any other types of additional training that would be very helpful to you?       24% gave additional comments
N=161
If respondent said “very helpful” to more than one item, ask the following:        Letter of most helpful item:
                                                                                   A 37%
H. You mentioned that ___ [state number] things that might be very helpful         B 17%
to you. That’s ___, ___, and ___. [ Read back “very helpful” answers,              C 11%
including “other.”] Which of those do you think would be the most helpful          D 4%
to you? N=161                                                                      E 9%
                                                                                   F 12%
                                                                                   G 1%
                                                                                   None of the above 9%




                                                  Page 28                         GAO/RCED-00-16 Implementation of New Food Safety System
                                                 Appendix 1
                                                 Results of Telephone Survey of Inspectors




14. Now that we’ve covered several possibilities for additional training, I’d       56% Very important
like to ask you how important, if at all, you think additional training would be    38% Moderately important
for you, personally. Would you say additional training is very important,           6% Not too important
moderately important, not too important for you, or would you say you have          1% No opinion
no opinion? N=160                                                                   0% Other
15. We’ve talked about many aspects of the HACCP training and now we’d 87% gave comments
like to ask a final question about your ideas on improving the training. This 13% gave no comments
time we have no set categories for the answer. Suppose you were the one
in charge of the HACCP training. What would you most like to see changed?
[If respondent does not mention real-life examples use this probe:]
Are there any other real-life examples that you would like to see in the
training. N=161
16. That’s all of the survey questions I have, and I want to thank you for your 66% gave comments
help today. Did you have any questions before we finish or any other
comments you would like to make? N=161




                                                 Page 29                           GAO/RCED-00-16 Implementation of New Food Safety System
Appendix II

Objectives, Scope, and Methodology


              To analyze the degree to which the U.S. Department of Agriculture’s
              (USDA) Food Safety and Inspection Service’s (FSIS) Hazard Analysis and
              Critical Control Point (HACCP) system adheres to the seven HACCP
              principles as defined by the National Advisory Committee on
              Microbiological Criteria for Foods, we met with members of this
              Committee and with officials in FSIS’ Office of Policy. We compared the
              Advisory Committee’s seven principles with FSIS’ HACCP regulations and
              interviewed meat and poultry plant managers and inspectors at 32 plants
              to discuss and observe how the seven principles are being implemented.
              We did not evaluate the plants’ hazard analyses or how well they
              implemented their HACCP plans. We selected plants that together represent
              (1) most of the FSIS meat and poultry processing categories, such as
              slaughter and canning; (2) both the large (over 500 employees) and small
              (10 to 499 employees) plants that implemented HACCP systems as of
              January 25, 1999; and (3) plants on both the East and West coasts and in
              the Midwest. At the plants we visited, we reviewed 57 HACCP plans that
              were made available to us to determine how the seven principles were
              addressed. Finally, we discussed the implementation of HACCP systems
              with FSIS district managers in three locations—Alameda, California;
              Philadelphia, Pennsylvania; and Des Moines, Iowa.

              To examine the adequacy of inspectors’ training, we reviewed USDA’s HACCP
              training curriculum, participated in the 8-day course for HACCP inspectors,
              reviewed and analyzed industry’s and USDA’s previous assessments of
              inspectors’ training, and conducted a nationwide telephone survey of
              inspectors who had most recently completed HACCP training. That group
              consisted of inspectors who were trained prior to the implementation of
              HACCP plans at small plants in January 1999. We identified these inspectors
              by contacting FSIS district offices and obtaining the names of the 1,787
              inspectors who had completed this training. We randomly selected a
              sample of 200 inspectors to represent this population. The sample size was
              designed to provide sampling errors of no more than 5 percent at the
              95-percent confidence level. The actual sampling errors for our survey
              were somewhat higher when the percent of responses to our survey
              questions was between 18 and 82 percent. Table II.1 displays the sampling
              errors associated with selected percentage responses to our survey
              questions.




              Page 30                  GAO/RCED-00-16 Implementation of New Food Safety System
                                         Appendix II
                                         Objectives, Scope, and Methodology




Table II.1: Sampling Errors Associated
With Survey Responses                    Range of reported survey percents (161
                                         interviews)                                                          Sampling error
                                         1 percent                                                                  1 percent
                                         2 to 3 percent                                                             2 percent
                                         4 to 6 percent                                                             3 percent
                                         7 to 10 percent                                                            4 percent
                                         11 to 17 percent                                                           5 percent
                                         18 to 27 percent                                                           6 percent
                                         28 to 72 percent                                                           7 percent
                                         73 to 82 percent                                                           6 percent
                                         83 to 89 percent                                                           5 percent
                                         90 to 93 percent                                                           4 percent
                                         94 to 96 percent                                                           3 percent
                                         97 to 98 percent                                                           2 percent
                                         99 percent                                                                 1 percent

                                         We pretested the telephone survey by contacting five inspectors to ensure
                                         that the (1) questions were understandable, (2) terms used were clear,
                                         (3) survey did not place an undue burden on USDA employees that would
                                         result in a lack of cooperation, and (4) survey was independent and
                                         unbiased. Appropriate changes were incorporated into the final survey on
                                         the basis of our pretesting. We completed interviews with 161 of the 200
                                         inspectors, which provides a response rate of 81 percent. Finally, we
                                         reviewed FSIS’ curriculum and materials for training inspectors and
                                         attended both the abbreviated (1 day) and full-length (8 days) HACCP
                                         training courses. We also interviewed plant managers and FSIS inspectors
                                         at the plants we visited to obtain their views on FSIS’ training.

                                         To determine if there is an adequate HACCP dispute resolution process
                                         available to address noncompliance issues between plants and inspectors,
                                         we reviewed existing and proposed regulations, directives, and guidelines,
                                         and the pre-HACCP dispute resolution and appeals process to determine if
                                         and how the new process differs. To determine how well the current
                                         process is working, we analyzed appeals data collected by FSIS through its
                                         Industry Appeals Tracking System and collected testimonial evidence from
                                         FSIS and plant personnel during our plant visits and through the nationwide
                                         inspector telephone surveys.

                                         We conducted our review between February and November 1999 in
                                         accordance with generally accepted government auditing standards.




                                         Page 31                      GAO/RCED-00-16 Implementation of New Food Safety System
Appendix III

GAO Contacts and Staff Acknowledgments


                  Lawrence J. Dyckman, (202) 512-5138
GAO Contacts      Keith W. Oleson, (415) 904-2000


                  In addition to those named above, Maria Cristina Gobin, George R. Senn,
Acknowledgments   Inez M. Azcona, Katherine Carey, Fran A. Featherston, Carol Herrnstadt
                  Shulman, and Oliver Easterwood made key contributions to this report.




(150127)          Page 32                 GAO/RCED-00-16 Implementation of New Food Safety System
Ordering Information

The first copy of each GAO report and testimony is free.
Additional copies are $2 each. Orders should be sent to the
following address, accompanied by a check or money order
made out to the Superintendent of Documents, when
necessary. VISA and MasterCard credit cards are accepted, also.
Orders for 100 or more copies to be mailed to a single address
are discounted 25 percent.

Orders by mail:

U.S. General Accounting Office
P.O. Box 37050
Washington, DC 20013

or visit:

Room 1100
700 4th St. NW (corner of 4th and G Sts. NW)
U.S. General Accounting Office
Washington, DC

Orders may also be placed by calling (202) 512-6000
or by using fax number (202) 512-6061, or TDD (202) 512-2537.

Each day, GAO issues a list of newly available reports and
testimony. To receive facsimile copies of the daily list or any
list from the past 30 days, please call (202) 512-6000 using a
touchtone phone. A recorded menu will provide information on
how to obtain these lists.

For information on how to access GAO reports on the INTERNET,
send an e-mail message with "info" in the body to:

info@www.gao.gov

or visit GAO’s World Wide Web Home Page at:

http://www.gao.gov




PRINTED ON    RECYCLED PAPER
United States                       Bulk Rate
General Accounting Office      Postage & Fees Paid
Washington, D.C. 20548-0001           GAO
                                 Permit No. G100
Official Business
Penalty for Private Use $300

Address Correction Requested