oversight

FDA's Mammography Inspections: While Some Problems Need Attention, Facility Compliance Is Growing

Published by the Government Accountability Office on 1997-01-27.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                 United States General Accounting Office

GAO              Report to Congressional Committees




January 1997
                 FDA’s MAMMOGRAPHY
                 INSPECTIONS
                 While Some Problems
                 Need Attention, Facility
                 Compliance Is Growing




GAO/HEHS-97-25
      United States
GAO   General Accounting Office
      Washington, D.C. 20548

      Health, Education, and
      Human Services Division

      B-271339

      January 27, 1997

      The Honorable James M. Jeffords
      Chairman
      The Honorable Edward M. Kennedy
      Ranking Minority Member
      Committee on Labor and Human Resources
      United States Senate

      The Honorable Thomas J. Bliley, Jr.
      Chairman
      The Honorable John D. Dingell
      Ranking Minority Member
      Committee on Commerce
      House of Representatives

      Breast cancer is currently the second leading cause of cancer deaths
      among American women. One woman in eight will develop breast cancer
      during her lifetime, and, in 1996 alone, an estimated 44,000 women will
      have died from the disease. If breast cancer is detected early, however, the
      probability that a woman can survive is greater than 90 percent.

      Currently, the most effective technique for early detection of breast cancer
      is mammography, an X-ray procedure that can often locate small tumors
      and abnormalities up to 2 years before they can be detected by touch.
      However, mammography is one of the most technically challenging X-ray
      procedures, and ensuring the quality of mammography services is difficult.
      To address concerns about variations in the quality of mammography
      services provided by the more than 10,000 facilities throughout the United
      States and its territories, the Congress passed the Mammography Quality
      Standards Act of 1992 (MQSA). This act established a number of
      requirements aimed at strengthening quality, including requiring
      accreditation and annual inspection of mammography facilities.

      The act also requires us to report to the Congress on the Food and Drug
      Administration’s (FDA) program for implementing these requirements. Our
      first report, which focused on the accreditation program, found that many
      facilities were upgrading their procedures to meet accreditation
      requirements.1 Since that report was issued, FDA has fully implemented its
      annual inspection program for assessing compliance with these


      1
      Mammography Services: Initial Impact of New Federal Law Has Been Positive (GAO/HEHS-96-17,
      Oct. 27, 1995).



      Page 1                                    GAO/HEHS-97-25 FDA’s Mammography Inspections
                   B-271339




                   requirements. In our ongoing monitoring efforts, we identified several
                   inspection-related issues that we thought important to call to your
                   attention as you assess the act’s results. This report focuses on the extent
                   to which facilities are complying with the new mammography standards,
                   whether FDA’s procedures for evaluating image quality at mammography
                   facilities are adequate, and whether FDA’s monitoring and enforcement
                   process ensures timely correction of mammography deficiencies.

                   Our work is based on an analysis of results of inspections that took place
                   at more than 9,000 mammography facilities nationwide between
                   January 1995 and June 1996. For a better understanding of how
                   inspections and follow-up efforts were being carried out, we also visited
                   eight state offices that contracted with FDA to conduct inspections and
                   three FDA field offices responsible for managing and monitoring
                   inspections.2 We supplemented this information through interviews with
                   officials from FDA and other federal, state, and private organizations
                   involved with the program. Our work was conducted from November 1995
                   through September 1996 in accordance with generally accepted
                   government auditing standards.


                   Our work points to growing compliance by facilities with FDA’s
Results in Brief   mammography standards. FDA’s first annual inspection began in
                   January 1995. By mid-1996, over 9,000 facilities had been inspected, and
                   approximately 1,500 of these had undergone two rounds of inspections.
                   The first time these 1,500 facilities were evaluated, 26 percent had
                   significant violations (violations requiring formal responses to FDA as to
                   the corrective actions taken); the second-year inspection revealed that this
                   figure had dropped to about 10 percent. Also, the percentage of facilities
                   with less significant deviations from quality standards had decreased.
                   While these results are positive, we did note some differences in how
                   inspectors are conducting inspections that, left unaddressed, could lead to
                   inconsistent reporting of violations, thereby limiting FDA’s ability to
                   determine the full effect of the inspection process and to identify the
                   extent of repeat violations.

                   Moreover, our review of FDA’s actions during the first 18 months of its
                   inspection program showed a need for management attention to two
                   additional aspects of the inspection program. First, we noted that FDA’s
                   inspection procedures for an important test of mammography equipment

                   2
                   The eight states were Arizona, California, Georgia, Illinois, Louisiana, Michigan, North Carolina, and
                   Washington. The three FDA field offices were located in the Atlanta, Baltimore, and Seattle areas.



                   Page 2                                         GAO/HEHS-97-25 FDA’s Mammography Inspections
             B-271339




             were inadequate. The way this test, called the phantom image test, was
             conducted was open to variability, which could have resulted in differing
             assessments of how well the equipment functioned. In addition, in those
             instances in which test results showed serious problems with the phantom
             image quality, FDA’s procedures allowed facilities to continue taking
             mammograms without follow-up to evaluate whether their quality was
             actually acceptable. Without such follow-up review, women are not fully
             protected from getting poor mammograms from facilities with potentially
             severe quality problems.

             Second, at the time of our review, FDA also lacked procedures to ensure
             that all violations of standards were both corrected and corrected in a
             timely manner. FDA’s program lacked criteria for defining conditions
             constituting a serious risk to human health, which could delay
             enforcement of compliance and notification to women who may have
             received substandard mammograms. For facilities with less severe but
             persistent violations, FDA’s follow-up efforts could not always ensure
             corrective action was taken. In addition, delays in completing a
             management information system have kept FDA’s compliance staff from
             having complete, up-to-date information about the compliance status of all
             mammography facilities.

             Our report contains several recommendations to the Commissioner of FDA
             to establish procedures, guidance, and training to help ensure timely
             compliance with MQSA standards by facilities. In response to our draft
             report, FDA informed us that they had recently taken actions to address
             these matters.


             The best method known to reduce breast cancer mortality is early
Background   detection. Detection of breast cancer is accomplished through a
             combination of self-examination, physical examination by a physician, and
             mammography. Of these methods, mammography is the single most
             effective tool for detection of early-stage breast cancer.3 The use of
             mammography as a tool for detecting early or potential breast cancer
             continues to increase. The proportion of women aged 50 and older who
             had received mammograms in the previous year increased from 26 percent
             in 1987 to 54 percent in 1993, according to the Centers for Disease Control
             and Prevention. Since 1992, at least 23 million mammograms have been
             performed in the United States annually.

             3
              Mammography, however, is not a perfect tool; according to FDA, even under ideal conditions, 10 to 20
             percent of breast cancers cannot be detected by mammography.



             Page 3                                       GAO/HEHS-97-25 FDA’s Mammography Inspections
B-271339




The consequences of substandard mammograms can be very serious. If
the image shows an abnormality when none exists, a woman may go
through unnecessary and costly follow-up procedures, such as ultrasound
or biopsies. If the image is too poor to show an abnormality that is actually
present, a woman may lose the chance to stop the cancer’s spread.

To help ensure the quality of images and their interpretation, MQSA
required FDA to implement both an accreditation and an inspection process
for mammography facilities. For the accreditation process, FDA established
standards that included requirements for personnel qualifications,
equipment performance, and quality assurance recordkeeping. These
standards were based on those used by the American College of Radiology
(ACR), a private, nonprofit professional association of radiologists, and
have been endorsed by industry and government experts. As of July 1996,
almost 10,000 facilities had been accredited and had received an FDA
certificate to that effect.4

MQSA  inspection authority provides FDA with another means to ensure that
facilities comply with standards on a day-to-day operating basis. While for
the vast majority of facilities accreditation application and review are
accomplished through the mail, all inspections are conducted on site.
During an inspection, MQSA inspectors conduct various equipment tests
and review the facility’s records on personnel qualifications, quality
controls, and quality assurance as well as mammography reports.

FDA, which has contracted with virtually all states and territories to
conduct inspections, began its first annual inspections of the nation’s
mammography facilities in January 1995.5 It established an extensive
program for training inspectors, and as of April 1996, about 220 state and
FDA personnel had become certified to perform MQSA inspections. The
majority of the personnel chosen to become MQSA inspectors had 5 or more
years of prior experience in radiological health. FDA uses its own
inspectors to conduct follow-up inspections, monitor the performance of
state inspectors, and conduct inspections in states that either did not
contract with FDA or lacked enough FDA-certified inspectors to do all the
inspections.6

4
FDA approved ACR and the states of California, Arkansas, and Iowa as official accrediting bodies.
ACR serves as the major accreditation body, responsible for more than 95 percent of the workload.
5
 These contracts address such matters as the number and cost of inspections to be conducted. To
cover these costs, FDA assesses an inspection fee on each facility. FDA has agreements with all states
except New Mexico.
6
 FDA inspectors are also responsible for conducting inspections of federal facilities.



Page 4                                         GAO/HEHS-97-25 FDA’s Mammography Inspections
                              B-271339




                              FDA’s  field offices are responsible for following up on inspection violations
                              and enforcing facility compliance. For the most serious violations, FDA’s
                              field offices issue a warning letter informing the facility of the seriousness
                              of the violation. The facility must begin correcting its problem immediately
                              and report the corrective action taken in writing to FDA within 15 work
                              days of receipt of the letter. In some cases, FDA conducts a follow-up
                              inspection of the facility to ensure that the problem is corrected. If the
                              facility fails to correct a problem, FDA can take other enforcement actions,
                              such as imposing a Directed Plan of Correction; assessing a civil penalty of
                              up to $10,000 per day or per failure; or suspending or revoking a facility’s
                              FDA certificate, which prevents a facility from operating lawfully.



                              First-year inspections of mammography facilities showed that a significant
Compliance With               number of facilities were not in full compliance with mammography
Standards Has                 standards. So far, second-year inspections have shown a considerable
Improved but More             reduction in the proportion of facilities cited for violations—an indication
                              that the inspection process is having positive results. However, inspection
Consistent Reporting          results vary considerably from state to state. It is not clear how much
Is Needed                     these differences reflect actual differences in the levels of quality in
                              mammography facilities and how much they reflect varying approaches to
                              conducting inspections and reporting the results. To gain a true picture of
                              the full effect of the inspection process, more consistent reporting of
                              violations is needed.


Second-Year Inspections       FDA’s automated inspection database contained first-year inspection
Identified Fewer Violations   results for 9,186 facilities as of June 20, 1996.7 Of these, 6,177 showed one
Than First-Year               or more violations of the standards. As table 1 shows, 1,849 facilities (or
                              20 percent) had violations that were serious enough to require the facility
Inspections                   to provide FDA with a formal response as to the corrective actions taken.
                              Of these, 214 facilities had violations that ranked in the most serious (or
                              “level 1”) category, requiring FDA to send the facility a warning letter.




                              7
                               Under their contracts with FDA, many states began inspecting facilities in January 1995, but some did
                              not begin until later. By June 1996, the database did not contain first-year inspections for all of the
                              estimated 10,000 mammography facilities nationwide.



                              Page 5                                        GAO/HEHS-97-25 FDA’s Mammography Inspections
                                             B-271339




Table 1: Distribution of Facilities’ First-Year Inspection Results, by Highest Level of Violation
                                                                                                                                  Facilitiesa
Violation level               Explanation                                                                                     Number          Percent
Level 1                       Level 1 violations are considered the most serious—those that may have a                             214                 2
                              detrimental effect on the quality of mammography services. An example is a
                              facility’s having mammography personnel who do not meet FDA’s minimum
                              qualification standards. Level 1 violations require FDA to issue a warning
                              letter; the facility must begin correcting problems immediately and respond in
                              writing to FDA within 15 work days of receiving the warning letter about the
                              corrective actions taken.
Level 2                       Level 2 violations are considered less significant than level 1 violations but                     1,635                18
                              may compromise the quality of mammography services. An example is a
                              facility’s not having an evaluation of equipment by a medical physicist within
                              the last 14 months. The facility must begin correcting problems immediately
                              and respond in writing to FDA within 30 work days of receiving an inspection
                              report about the corrective actions taken.
Level 3                       Level 3 violations are generally considered minor deviations from MQSA                             4,328                47
                              standards. An example is a facility’s not having records for one or more of the
                              quality control tests. No written response is required.
No findings of violations     The facility meets all standards.                                                                  3,009                33
Total                                                                                                                            9,186                100
                                             a
                                              Facilities could have had more than one level of violation and more than one violation at each
                                             level. In this table, facilities that had multiple violations at more than one level were counted only
                                             once, based on their highest level of violation.



                                             The most serious violations found in these inspections were mainly
                                             personnel related: 88 percent of the level 1 violations were for personnel
                                             who did not fully meet FDA’s qualification standards (see app. I for a
                                             further breakdown of the types of level 1 violations).8 Level 2 violations
                                             involved a greater mix of personnel-related and equipment-related
                                             problems, and the majority of level 3 violations involved missing or
                                             incomplete quality assurance records and test results as well as medical
                                             physicist survey problems.

                                             By June 20, 1996, FDA’s database contained the results of 1,503 second-year
                                             inspections. We compared the results of first-year and second-year
                                             inspections for these 1,503 facilities and found a substantial decrease in all
                                             three categories in the proportion of facilities cited for violations (see fig.
                                             1).




                                             8
                                              These personnel include interpreting physicians, medical physicists, and radiologic technologists.



                                             Page 6                                          GAO/HEHS-97-25 FDA’s Mammography Inspections
                                             B-271339




Figure 1: Comparison of First-Year and
Second-Year Inspection Results               80   Percent of Facilities in Each Category


                                             70


                                             60


                                             50


                                             40


                                             30


                                             20


                                             10


                                              0

                                                      Level 1 Violation     Level 2 Violation    Level 3 Violation



                                                               First-year Inspection

                                                               Second-year Inspection




                                             Another measure of facilities’ improvement in compliance is the extent of
                                             repeat violations, that is, violations identified in the first year’s inspection
                                             that are identified again when the facility is reinspected the following year.
                                             Facilities had a better record in not repeating the more severe violations
                                             than they did with minor findings. More specifically, our analysis of the
                                             1,503 facilities showed the following:

                                         •   None of the 50 facilities whose highest level of violation was at the level 1
                                             category during the first-year inspection repeated one or more of the same
                                             violations in the second inspection.
                                         •   Six percent of the 345 facilities whose highest level of violation was at the
                                             level 2 category during the first-year inspection repeated one or more of
                                             the same violations in the second inspection.
                                         •   Twelve percent of the 669 facilities whose highest level of violation was at
                                             the level 3 category during the first-year inspection repeated one or more
                                             of the same violations in the second inspection.




                                             Page 7                                             GAO/HEHS-97-25 FDA’s Mammography Inspections
                          B-271339




Inconsistent Inspection   Our analysis of inspection results showed considerable state-by-state
Practices Could Lead to   variation in the degree to which facilities were cited for violations of MQSA
Underreporting of         standards.9 For example,14 states cited no facilities for level 1 violations,
                          while 6 states cited 5 to 12 percent of the facilities inspected for level 1
Violations                violations (see app. II for state-by-state results). We were unable to
                          determine the reason for these differences. It may be, for example, that
                          facilities in low-violation states really were much better at complying with
                          standards than facilities in high-violation states. Alternatively, the
                          differences may have been related to variations in the way inspectors
                          conducted their inspections. In the eight states in which we observed
                          inspections, we saw several differences in inspection practices that
                          affected the number of violations reported. The two main differences
                          follow.

                          First, inspectors’ adherence to time limits for resolving problems of
                          missing documents was inconsistent. FDA’s current procedures allow
                          inspectors to delay submitting their inspection reports for 5 to 30 days in
                          order to resolve problems of missing documents. This delay is intended to
                          avoid citing facilities for not having certain records available on site. For
                          example, when a facility claims that its personnel meet MQSA qualification
                          requirements but does not have the required documentation at hand, FDA
                          guidelines instruct inspectors to either delay the transmission of the
                          inspection report or note the “claimed items” in the inspection record.
                          These open items are to be resolved within 30 days, at which time the
                          inspection report is to be finalized. However, we found hundreds of cases
                          in the inspection report database that contained open items longer than 30
                          days—many for over 6 months.10 Several inspectors we interviewed said
                          they were not aware of the 30-day limit for resolving pending items. On the
                          other hand, inspectors in two states we visited said they would not wait
                          more than 5 days under any circumstances before submitting a report that
                          a facility was in violation. Thus, a facility in one state might be reported as
                          being in violation, while a facility with the same problem in another state
                          would not. These differences may have resulted in inconsistent reporting
                          of violations; moreover, these inconsistencies make it difficult to
                          determine the full effect of the inspection process.

                          Second, while FDA’s policy is to cite facilities for all violations even if
                          problems are corrected on the spot, we found that inspectors do not

                          9
                           We based this analysis on the 9,186 first-year inspections because there were more first-year than
                          second-year inspections conducted at the time we performed our analysis and also because the
                          first-year inspections represented a broader distribution among all 50 states.
                          10
                              In November 1996, FDA told us it was in the process of investigating and resolving these open items.



                          Page 8                                          GAO/HEHS-97-25 FDA’s Mammography Inspections
                       B-271339




                       always adhere to this policy. For example, we observed that an inspector
                       did not cite a facility that failed its darkroom fog test11—normally a level 2
                       violation—because the facility immediately corrected the problem.
                       Further, FDA’s procedures instruct inspectors to note on-the-spot
                       corrections in the “remarks” section of the inspection software. We
                       observed two inspections on site that involved on-the-spot corrections, but
                       we did not see these inspectors documenting them in the remarks section.

                       We do not question the merit of giving inspectors time to resolve such
                       problems as missing documents or giving facilities opportunities to correct
                       their problems immediately. However, not documenting violations
                       consistently creates problems in forming an accurate picture of what the
                       inspection process is accomplishing. FDA officials told us that they had
                       begun a program in February 1996 to review inspector performance and
                       that, as of October 31, 1996, 65 percent of all inspectors had been audited.
                       FDA officials expect that, when fully implemented, the audit program will
                       help ensure that policies are consistently applied. We agree that the audit
                       program will help identify some inconsistent inspection practices;
                       however, we believe the inspection results should also be monitored to
                       ensure that open items are resolved in a timely manner and that
                       on-the-spot corrections are identified.


                       Although many factors can affect the quality of mammography images, one
Procedures for         key factor is the condition of mammography equipment.12 We identified a
Assessing Image        need for FDA to clarify the procedures it requires for a major equipment
Quality Need           test that evaluates image quality and to follow up when test results suggest
                       problems with the quality of the images being produced.
Strengthening
Issues in Evaluating   One of the most important aspects of the inspection process is testing
Phantom Image Test     mammography equipment by evaluating what is called a “phantom image.”
Results                In this procedure, the inspector uses the facility’s mammography
                       equipment to take an X-ray image of a plastic block containing 16 test
                       objects. This block is X-rayed as though it were a breast to determine how
                       many of the test objects can be seen on the image.13 The inspector

                       11
                        The darkroom fog test is conducted to determine if excessive light exists, inside or leaking in from
                       outside the darkroom, that could fog mammographic images, thus reducing their quality.
                       12
                        Mammography equipment generally includes the mammography unit (machine) used to produce
                       X-ray images, the processor, and other equipment used in developing images.
                       13
                        The plastic block with a wax insert, which is the average size of a compressed breast, contains 16 test
                       objects—5 embedded micro-calcifications, 6 fibrous structures, and 5 different-sized tumor-like
                       masses that simulate growths that could be cancerous.



                       Page 9                                        GAO/HEHS-97-25 FDA’s Mammography Inspections
    B-271339




    evaluates aspects of the performance of the facility’s imaging system by
    scoring the number of objects that can be seen. We found two questions
    that need to be answered with regard to evaluating phantom images.

•   What is the impact of inconsistent phantom image scoring? FDA’s current
    inspection procedures instruct inspectors to score the phantom images
    under viewing conditions at the facilities. However, differences in
    inspectors’ experience and in facilities’ viewing conditions may influence
    the phantom image scores. For greater uniformity in scoring the images,
    two states we visited go beyond FDA’s standards by having their inspectors
    score phantom images using standardized viewing conditions (that is,
    away from the facility), having two or more persons read the images to
    ensure more consistent scoring, or both.14 FDA officials told us that the
    impact of these variations in procedure on the accuracy of image
    evaluation is unknown and that they are studying the problem.
•   How should large image receptors be evaluated?15 FDA procedures
    currently require that phantom images be checked using the receptor that
    is more commonly used by the facility. Since facilities use small image
    receptors for most mammograms, these receptors are typically tested
    during an inspection. Although facilities may use large image receptors for
    some women, FDA does not require that the large image receptor be tested
    and does not have specific criteria for evaluating the phantom images of
    the large receptor.

    Inconsistent phantom image scoring and lack of standards for evaluating
    large image receptors can affect inspection results, as can be seen in the
    example of a 1995 inspection of a large mobile mammography facility
    headquartered in North Carolina and operating in five states. The facility is
    reported to perform over 20,000 mammograms a year. A state inspector
    cited the facility for multiple problems based on the viewing conditions at
    the facility and images from the small receptor. Although it was not
    required by FDA, the inspector also evaluated the phantom images from the
    large image receptor and noted in the remarks section of the inspection
    report that, for three of four mammography units, these images did not
    pass the review. An FDA inspector conducted a follow-up inspection, also


    14
      One state requires all phantom images to be scored by a group of four reviewers using the
    standardized view boxes in their offices. In the other state, the inspector scores the phantom images
    twice, once using the facility’s view boxes and once using the standardized view box in the office. In
    addition, the inspector also asks the facility’s technologist to score the images to see if the scores are
    the same. If there are discrepancies, the inspector asks another inspector to score the images.
    15
      An image receptor is a medium (screen-film or xerox) that is used by mammography facilities to
    record breast images. The receptor comes in two sizes: small (18 by 24 centimeters) and large (24 by
    30 centimeters). The large image receptor is generally used in imaging large-breasted women.



    Page 10                                         GAO/HEHS-97-25 FDA’s Mammography Inspections
                             B-271339




                             using the viewing conditions at the facility and images from both the small
                             and large image receptors. This inspector cited the facility for many
                             violations related to both the small and large image receptors. Finally, four
                             reviewers at FDA headquarters examined these same images away from the
                             facility and together found fewer violations related to both the small and
                             large image receptors than the state inspector and the FDA inspector had
                             found. The reviewers, however, did confirm the serious violations related
                             to the large image receptor that were found by the state inspector and the
                             FDA inspector.


                             Although this facility was cited for serious violations related to the large
                             image receptor as a result of the follow-up inspection, FDA officials told us
                             that, because of the lack of inspection criteria, imposing strong sanctions
                             on the basis of phantom image failures from the large receptor could
                             prove problematic. According to FDA, standards for testing the large
                             receptor have not yet been developed because the technical issues relating
                             to the receptor have not yet been resolved by the scientific and medical
                             community. We discussed this case with senior FDA officials, who said that
                             they plan to both provide additional training and guidance to minimize the
                             variability in phantom image scoring and study the development of
                             standards for evaluating images from the large image receptor.


Taking Additional Steps to   Another issue raised by the inspections of the facility discussed above is
Better Ascertain the         how to proceed if the phantom image test suggests serious problems with
Seriousness of               image quality. FDA views phantom image failures as early indications of
                             potential problems deserving further investigation. FDA’s procedures allow
Deficiencies                 facilities with serious phantom image failures to continue performing
                             mammograms while FDA investigates and the facility corrects problems.
                             During the course of our work, we heard varying opinions on the risk of
                             allowing facilities with serious phantom image failures to continue doing
                             mammograms. Some people we spoke with believe the risk of patients’
                             getting poor mammograms from facilities with serious phantom image
                             failures is high enough that the facilities should not be allowed to do any
                             mammograms until their problems are corrected and those corrections are
                             verified by a reinspection. Several states, including California, Illinois, and
                             Michigan, have rules empowering inspectors to immediately stop facilities
                             with level 1 phantom image failures from doing additional mammograms.
                             However, others (including FDA officials in charge of the MQSA program) do
                             not believe that such drastic action should be taken on the basis of
                             phantom image test results alone. They assert that phantom image failures




                             Page 11                            GAO/HEHS-97-25 FDA’s Mammography Inspections
B-271339




are an indicator of possible image system problems but are not conclusive
evidence that actual mammograms are faulty.16

At the time of our review, FDA did not have a follow-up system in place for
reviewing the actual mammograms (called “clinical images”) of facilities
with serious phantom image violations to ensure that they were not
producing poor mammograms. However, in the case of the mobile facility
discussed earlier, FDA did ask ACR to conduct two reviews of the clinical
images produced by the facility because of image quality concerns. The
more comprehensive review was conducted in July 1996, subsequent to
our inquiry about FDA’s handling of the case.17 This review selected a total
of 28 sets of images from five units operated by the facility for three
different time frames over a 1-year period. In early September 1996, ACR
completed the review and found most of these clinical images of
unacceptable quality. On the basis of these results, FDA obtained the
facility’s agreement to discontinue performing mammography until its
radiologic technologists and its radiologist obtained additional training
approved by FDA and ACR, which they did the following week. In addition,
at FDA’s request, ACR is planning to review another sample of clinical
images produced by the facility to determine to what extent patients
should be notified of past quality problems at the facility.

This case clearly demonstrates the need for a procedure to review clinical
images when there is sufficient evidence to suggest problems with the
quality of a facility’s mammograms. Without the criteria and process in
place for determining when and how follow-up review of clinical images
should be conducted and patient notification should be carried out, there
is no assurance that patients are protected from the risk of receiving poor
mammograms. FDA officials agreed that there is a need to incorporate a
follow-up clinical image review process. In its proposed final regulation
dated April 3, 1996, FDA has included a provision that specifically provides
FDA with authority to require clinical image review and patient notification
if FDA finds that image quality at a facility has been severely
compromised.18


16
 In support of this position, FDA cited data from ACR’s accreditation program showing that failed
phantom images (level 1 and level 2 phantom image violations under the inspection program) correlate
with failed mammograms about 24 percent of the time.
17
  The other review, conducted in November 1995, involved ACR’s check of one set of clinical images
from each of two units operated by the facility.
18
  FDA implemented MQSA by issuing interim regulations in December 1993. It published the proposed
final regulations for public comment on April 3, 1996, but does not expect to have the final regulations
issued until some time in 1998.



Page 12                                        GAO/HEHS-97-25 FDA’s Mammography Inspections
                             B-271339




                             Although FDA has made progress in bringing facilities into compliance with
FDA’s Monitoring and         mammography standards, it lacks procedures to enforce timely correction
Enforcement Process          of all deficiencies found during inspections. One major problem involves
Does Not Ensure              the need to develop criteria for defining conditions constituting a serious
                             risk to human health and determining when severe sanctions are
Timely Correction of         warranted. Other problems that also merit attention relate to determining
Deficiencies                 whether a stronger approach is needed to resolve repeated level 3
                             violations and establishing an effective information system for follow-up
                             on inspection results. FDA is developing such an information system.


Developing Criteria for      MQSA  provides FDA a broad range of sanctions to impose against
Defining a Serious Risk to   noncomplying facilities, but it emphasizes bringing facilities into
Human Health and             compliance through those sanctions that are less severe, such as imposing
                             a Directed Plan of Correction. FDA has the authority to impose stronger
Determining When Severe      sanctions, such as an immediate suspension of a facility’s FDA certificate, if
Sanctions Are Justified      it determines that the facility’s operation presents a serious risk to human
                             health.19 Since the implementation of MQSA, FDA has never done so.20 We
                             found evidence that FDA needs to define those circumstances in which
                             such actions are warranted.

                             In dealing with the continuing problems at the mobile facility discussed
                             earlier, there was considerable internal debate at FDA about the level of
                             action that should be taken. Inspections of the facility beginning in June
                             1995 had disclosed serious violations. (See app. III for a chronology of key
                             events surrounding the resolution of quality assurance problems at the
                             facility.) Several state and FDA field personnel involved in the case told us
                             they thought the severity of violations warranted an immediate suspension
                             of the facility’s certificate and had made such a recommendation. FDA
                             officials decided against suspending the facility’s certificate because they
                             thought the evidence of health risk was not clear and compelling enough
                             to do so.21 In September 1996, when ACR’s review of clinical images
                             eventually confirmed that the quality of the mammograms was
                             unacceptable, FDA obtained the facility’s agreement to discontinue

                             19
                               42 U.S.C. 263b(i)(2) (1994).
                             20
                              After reviewing a draft of this report, FDA informed us that it had issued a suspension without a
                             hearing to a facility for the first time in September 1996.
                             21
                               FDA stated that there was countervailing evidence that the facility was producing mammograms of
                             acceptable quality. Specifically, in August 1995 and November 1995, two of the five units operated by
                             the facility had passed ACR’s clinical image review as part of the facility’s accreditation process.
                             Further, in November 1995, ACR conducted a review of one set of clinical images from each of two
                             other accredited units. In February 1996, ACR notified the facility and FDA that it found them to be
                             acceptable.



                             Page 13                                       GAO/HEHS-97-25 FDA’s Mammography Inspections
                            B-271339




                            performing mammography until facility personnel received more training.
                            Because of the agreement, FDA did not have to go through the process of
                            imposing an immediate suspension of the facility’s certificate.
                            Nevertheless, this incident points to the need for having criteria in place to
                            impose such a sanction to protect patients, if necessary, from continuing
                            to receive poor mammograms. We believe—and FDA officials agreed—that
                            timely imposition of an appropriate sanction is in part dependent on
                            (1) criteria for when conditions constitute a serious risk to human health,
                            justifying immediate suspension of operations, and (2) a process for
                            discontinuing mammography services until the problems are corrected.


Determining Whether a       Another matter that also merits attention from FDA is whether more
Stronger Approach Is        serious follow-up is needed for facilities with multiple or repeated level 3
Needed to Resolve Level 3   violations. Current policy for facilities whose most serious violations are
                            in the level 3 category requires no reporting on the facility’s part and no
Violations                  follow-up on the part of FDA until the next year’s inspection. However, of
                            the facilities that had gone through two inspections, 18 percent of those
                            whose most serious violation was in the level 3 category during the first
                            year had five or more such violations, and 12 percent repeated one or
                            more of the same violations in the next year. Several state inspectors we
                            interviewed expressed concern that current procedures do not call for
                            stronger action against such facilities. Inspectors from one state told us
                            that their state regulations allow them to impose more serious penalties
                            for recurring level 3 violations. Some inspectors also told us that even
                            though level 3 violations were generally considered less serious, some
                            level 3 violations—such as a facility’s failure to take corrective action
                            when called for in the medical physicist’s survey report—are serious
                            enough that they should be corrected as soon as possible to maintain
                            quality assurance.

                            We did not evaluate the appropriateness of FDA’s classification of the
                            various levels of violations. Because of the concerns expressed by the
                            inspectors and the extent of multiple and repeat violations noted above,
                            however, we believe that FDA should evaluate its classification of level 3
                            violations and the enforcement actions taken on them. If FDA believes
                            these violations are important and need to be corrected, it could raise the
                            violation level for facilities with multiple or repeated violations, which
                            would ensure formal follow-up. However, if FDA views some of these
                            violations as insignificant or having little effect on mammography, it may
                            choose not to classify them as violations.




                            Page 14                            GAO/HEHS-97-25 FDA’s Mammography Inspections
                            B-271339




FDA Field Offices Need a    FDA  generally delegates inspection responsibility through contracts with
Better Information System   states but remains responsible for follow-up and enforcement when
to Manage Inspections       violations are reported. For level 1 violations, FDA’s field offices are
                            responsible for validating inspection results and issuing a warning letter
                            that requires the facility to respond within 15 work days. For level 2
                            violations, no warning letters are sent, but facilities are required to
                            respond in writing within 30 work days of the receipt of an inspection
                            report. Since June 1995, FDA has been working with contractors to develop
                            an automated compliance system that would supply its field offices with
                            computer-based information to manage this compliance effort.
                            Development problems have delayed the system, which is now projected
                            to be operational early in 1997. In the meantime, FDA has been relying on
                            field offices to maintain their own tracking systems.

                            Our reviews at three of FDA’s field offices showed that these interim
                            systems were inadequate. Staff responsible for compliance follow-up had
                            no direct access to inspection databases and were relying either on the
                            state inspectors or on FDA headquarters to send them copies of inspection
                            reports showing level 1 and level 2 violations that needed to be tracked.
                            Staff said that sometimes they did not receive reports from headquarters
                            until 2 to 3 months after the inspections and that state inspectors did not
                            always send reports on level 2 cases. As a result, field office staff often
                            received facility responses on corrective actions taken for level 2
                            violations before they even knew that violations had been cited. None of
                            the three offices maintained case logs or prepared any status reports on
                            their tracking efforts or the timeliness of facility responses.

                            Problems in these makeshift systems have stymied our attempts to
                            determine how quickly and completely violations were being corrected. To
                            determine whether field offices were sending out warning letters in a
                            timely manner and whether facilities were correcting their deficiencies
                            within required time frames, at our request, FDA headquarters in April 1996
                            sent all of its field offices a list of all level 1 and level 2 violations cited in
                            their jurisdictions and asked them to compile data on facility response
                            times for corrective actions. Field offices had difficulty responding with
                            complete information. FDA headquarters had initially told us that these
                            data would be available in early June, but at the time that we completed




                            Page 15                              GAO/HEHS-97-25 FDA’s Mammography Inspections
              B-271339




              our field work, discrepancies still remained unresolved.22 We conducted
              an on-site file review at one FDA field office in August and September 1996
              and found that the office had incomplete documentation for 13 of the 40
              cases with level 2 violations cited between July 1, 1995, and June 20, 1996.
              In one case, documentation was absent altogether.

              We also found problems with the timeliness of follow-up on level 1
              violations. For example, while FDA guidelines require a field office to issue
              a warning letter for a level 1 violation within 15 to 30 business days after
              the inspection, the office we reviewed took up to 132 business days. Also,
              although FDA procedures require a facility to respond within 15 business
              days of receiving the warning letter, in two of the eight level 1 cases that
              we reviewed the facilities did not respond within the required time frame,
              and one case file contained no record of a facility response.

              These findings highlight the importance of completing and implementing
              the automated compliance system as soon as possible. Until field offices
              have ready access to up-to-date information, it will be difficult for them to
              conduct effective follow-up and enforcement for facilities that violate the
              standards.


              The results of the current inspection program of mammography facilities
Conclusions   appear to be generally positive. Establishing this comprehensive
              inspection program has been a substantial effort on FDA’s part and, as
              mammography facilities move into their second year of inspections,
              violations of mammography standards are declining.

              Despite these encouraging results, at the time of our review, we found
              indications that certain aspects of the inspection program needed
              attention. First, to ensure an accurate picture of how many problems were
              found and how well the inspection program was working, violations would
              need to be more consistently recorded. In addition, even though serious
              violations do not occur often, when they do, they have the potential for
              posing a serious health risk to those women affected. To ensure high
              quality mammography, FDA must be vigilant in its efforts to confirm that
              facilities promptly and adequately correct violations. As a result, FDA

              22
                In November 1996, in response to our draft report, FDA provided us the spreadsheet data for the
              period ending March 1996 that we had earlier requested. While the data showed that all level 1 cases
              had been resolved, we noticed that 46 percent of the warning letters had not been issued within FDA’s
              established time frame and, when warning letters were sent, 20 percent of facilities did not respond on
              time. For level 2 cases, 6 percent of 1995 cases still had not been resolved as of November 1996. For
              those cases that had been resolved, 20 percent of the facility responses were not received within the
              required time frame.



              Page 16                                       GAO/HEHS-97-25 FDA’s Mammography Inspections
                          B-271339




                          would need to provide an expeditious means to follow up, including
                          notifying patients, when serious problems affecting image quality were
                          indicated. Finally, improvements would be needed in systems and
                          procedures for monitoring facilities with violations and for ensuring that
                          they corrected deficiencies.


                          We recommend that FDA take action in the following areas:
Recommendations to
the Commissioner of   •   Strengthening the inspection reporting process. To better reflect the
the Food and Drug         extent to which inspections detect compliance problems, FDA needs to
                          monitor its inspection results more closely to ensure that its procedures
Administration            for resolving open items and documenting on-the-spot corrections are
                          consistently followed.
                      •   Strengthening procedures for assessing image quality and protecting
                          patients. To minimize the variability in how phantom images are scored,
                          additional training and guidance should be provided, including guidance
                          for evaluating phantom images using the large image receptor. Also, to
                          minimize patients’ risk of poor quality mammograms, the final
                          implementing regulations should include the criteria and process for
                          requiring follow-up clinical image reviews and, when necessary, patient
                          notification when inspections detect violations, such as serious phantom
                          image failures, that could severely compromise image quality.
                      •   Ensuring that violations are corrected in a timely manner. Several steps
                          are needed here. First, to help ensure that appropriate action is taken
                          when serious problems are discovered, procedures need to be developed
                          for (1) determining when the health risk is serious enough to justify
                          immediate suspension of certification and (2) implementing the
                          suspension. Second, to help ensure better performance from facilities that
                          exhibit lingering, though less serious, deficiencies, the classification and
                          enforcement policy on level 3 violations needs reevaluation to determine if
                          additional follow-up is needed on facilities with multiple and repeated
                          level 3 violations. Third, so that compliance personnel can have access to
                          complete, up-to-date information on violations reported, all necessary
                          steps need to be taken to ensure that the compliance tracking system
                          currently under development is completed as soon as possible.


                          In commenting on a draft of this report, FDA generally agreed with our
Agency Comments           recommendations and cited specific program enhancements and
and Our Evaluation        corrective actions it had recently undertaken. FDA was, however, critical of
                          our draft on several accounts.



                          Page 17                           GAO/HEHS-97-25 FDA’s Mammography Inspections
B-271339




FDA  said that the scope of our work did not address some aspects of MQSA
requirements and that the draft did not adequately reflect many of FDA’s
accomplishments in implementing MQSA. Moreover, FDA believed the report
did not recognize changes FDA had made to improve those aspects of the
inspection program that we had found in need of attention. FDA cited
recent actions it had taken, including (1) establishing procedures and
guidance for clinical image reviews, sanctions for failure to comply with
standards, and procedures for follow-up on repeated level 3 violations;
(2) implementing an inspector audit program that had evaluated
65 percent of inspectors as of October 31, 1996; and (3) making a
commitment to fully implement its automated compliance tracking system
in January 1997. FDA expressed concern that not acknowledging these
actions would create an inaccurate impression that the program was
fraught with problems, which could undermine the public confidence in
mammography.

Concerning the scope of our work, this report is not intended as a vehicle
for commenting on implementation of MQSA as a whole; it deals only with
FDA’s inspection program. However, we think that the report speaks both
to FDA’s accomplishments related to the inspection program and to those
problems that we found—and that FDA has now moved to correct. The
main reason that FDA’s recent actions were not reflected in the original
draft was that they occurred about the same time or, in most cases, after
we had provided FDA the draft for comment. We generally consider FDA’s
subsequent actions and approaches to our recommendations to be
responsive and believe that, if properly implemented, they should
strengthen the inspection program. We recognize FDA’s concern about the
importance of promoting public confidence in mammography, and, in fact,
our recommendations to promote timely compliance with MQSA were made
with that objective in mind.

While we generally concur with FDA’s approaches for addressing our
recommendations, we continue to believe that opportunity exists for FDA
to improve its reporting process. We recognize that FDA has acted to
implement the inspector audit program, but we believe that FDA still needs
to monitor its inspection results to ensure timely follow-up on “open
items” and accurate reporting of on-the-spot corrections. As a result, we
have clarified our recommendation on strengthening the inspection
reporting process accordingly.

FDA also provided technical comments, which we considered and
incorporated where appropriate, and cited several other areas of the



Page 18                          GAO/HEHS-97-25 FDA’s Mammography Inspections
B-271339




report that it thought needed clarification. The full text of FDA’s comments,
accompanied by our response, is contained in appendix IV.

We also received comments from the North Carolina facility that we cited
in the report. The facility stated that our report addressed many of its
concerns with the MQSA program. It also commented that its case
demonstrates the need for an organized approach to evaluation and for all
involved agencies to agree upon an appropriate standard for clinical image
evaluation. The facility asserted that FDA’s process lacks these critical
elements and that the facility was being held to unreasonable standards.
As a result, in October 1996, the facility appealed its Directed Plan of
Correction to FDA. We have updated the chronology of FDA’s enforcement
actions regarding the facility to reflect the facility’s appeal and the
subsequent denial of the appeal by FDA (see app. III).


We are sending copies of this report to the Secretary of Health and Human
Services, the Commissioner of the Food and Drug Administration, the
Director of the Office of Management and Budget, and other interested
parties. We will also make copies available to others upon request.

Please contact me at (202) 512-7119 if you or your staff have any questions.
Major contributors to this report are listed in appendix V.




Bernice Steinhardt
Director, Health Services Quality
  and Public Health Issues




Page 19                             GAO/HEHS-97-25 FDA’s Mammography Inspections
Contents



Letter                                                                                                 1


Appendix I                                                                                            22

Distribution of Level 1
Violations by Type,
First-Year Inspections
Appendix II                                                                                           23

Results of States’
First-Year Inspections
Appendix III                                                                                          25

Chronology of FDA’s
Enforcement Actions
for the North Carolina
Mobile Facility
Appendix IV                                                                                           26
                          GAO Comments                                                                37
Comments From the
Food and Drug
Administration and
Our Evaluation
Appendix V                                                                                            39

GAO Contacts and
Staff
Acknowledgments
Table                     Table 1: Distribution of Facilities’ First-Year Inspection Results,          6
                            by Highest Level of Violation

Figure                    Figure 1: Comparison of First-Year and Second-Year Inspection                7
                            Results




                          Page 20                            GAO/HEHS-97-25 FDA’s Mammography Inspections
Contents




Abbreviations

ACR        American College of Radiology
FDA        Food and Drug Administration
MQSA       Mammography Quality Standards Act of 1992


Page 21                        GAO/HEHS-97-25 FDA’s Mammography Inspections
Appendix I

Distribution of Level 1 Violations by Type,
First-Year Inspections


                                                                                    Violations
                Type of noncompliance                                            Number      Percent
                Personnel do not meet FDA’s qualification standards                  261          88
                Failing phantom image score                                           16           5
                Processor quality control charts not available                        11           4
                No survey conducted by medical physicist                               4           1
                Mammography records improperly maintained or recorded                  3           1
                Self-referred system inadequate or not in place                        2           1
                                                                                                    a
                Radiation dose exceeds limits                                          1
                Total                                                                298         100
                a
                    Less than one-half of 1 percent.




                Page 22                                  GAO/HEHS-97-25 FDA’s Mammography Inspections
Appendix II

Results of States’ First-Year Inspections



                                             Total                                       Percentage
                                       number of      Percentage of facilities with the   of facilities
                                         facilities      most serious violation at:           with no
                State                  inspected       Level 1       Level 2      Level 3 violations
                Alabama                       156             2           15              51           32
                Alaska                          28            0           18              68           14
                Arizona                       155             1           15              27           57
                Arkansas                        84            1           10              40           49
                California                    940             3           22              48           27
                Colorado                      114             3           12              54           31
                Connecticut                   167             2           13              44           42
                Delaware                        25            4           48              24           24
                District of Columbia            24            4           38              29           29
                Florida                       470             2            9              39           51
                Georgia                       270             1           10              35           54
                Hawaii                          43            0           33              30           37
                Idaho                           39            3           15              41           41
                Illinois                      398             1           16              51           32
                Indiana                       223             0           17              56           27
                Iowa                          139             0           23              49           28
                Kansas                          95            1           17              55           27
                Kentucky                      163           12            29              41           18
                Louisiana                     155             0           11              34           55
                Maine                           55            0            7              22           71
                Maryland                      169             1           14              35           50
                Massachusetts                 222             0            7              31           62
                Michigan                      337             1           14              56           29
                Minnesota                     188             5           32              55            7
                Mississippi                     96            0           15              46           40
                Missouri                      174             3           18              52           26
                Montana                         47            0           21              51           28
                Nebraska                        85            1            6              62           31
                Nevada                          61            0            2              46           52
                New Hampshire                   42            0            5              24           71
                New Jersey                    263             4           18              52           27
                New Mexico                      49            0           20              59           20
                New York                      700             3           32              49           16
                North Carolina                216             8           29              47           16
                North Dakota                    36            3           19              47           31
                Ohio                          314             5           18              48           29
                                                                                               (continued)


                Page 23                               GAO/HEHS-97-25 FDA’s Mammography Inspections
Appendix II
Results of States’ First-Year Inspections




                                Total                                          Percentage
                          number of         Percentage of facilities with the   of facilities
                            facilities         most serious violation at:           with no
State                     inspected          Level 1       Level 2      Level 3 violations
Oklahoma                         102                0            9              58        33
Oregon                             98               5           41              41        13
Pennsylvania                     474                2           14              51        33
Rhode Island                       48               2            4              48        46
South Carolina                   114                0           16              48        36
South Dakota                       43               2           12              49        37
Tennessee                        177                8           22              55        15
Texas                            495                1            9              50        40
Utah                               48               4            8              46        42
Vermont                            19               0           10              53        37
Virginia                         228                1            9              47        42
Washington                       175                3           27              44        25
West Virginia                      83               1           16              47        36
Wisconsin                        229                2           19              52        28
Wyoming                            24               0           33              42        25




Page 24                                     GAO/HEHS-97-25 FDA’s Mammography Inspections
Appendix III

Chronology of FDA’s Enforcement Actions
for the North Carolina Mobile Facility


Date           Summary of event
6/28/95        Initial inspection revealed 1 level 1 violation for phantom image failure, 2 level 2 violations for phantom image failure, 1
               level 2 violation for processor quality control problems, and 31 level 3 violations for various other problems.
8/29/95        ACR’s clinical image review for one unit found mammograms acceptable and resulted in ACR accreditation for that unit.
10/10/95       FDA issued its warning letter to the facility for the violations found in June 1995.
10/18/95       The facility responded by submitting new phantom images and a processor quality control chart for review.
11/13/95       FDA notified the facility that its response was inadequate because it did not identify the machine on which the phantom
               images were taken and it did not include proper paperwork for the processor.

               ACR clinical image review for another unit found mammograms acceptable, and ACR accreditation was granted for
               that unit.
11/20/95       ACR, at FDA’s request, performed a clinical image review of one set of mammograms for each of two units.
11/30/95       The facility responded to FDA’s 11/13/95 letter by sending new phantom images and processor quality control charts.
12/8/95        FDA notified the facility that the 11/30/95 response was adequate.
1/16/96        FDA did a follow-up reinspection and found 5 level 1 and 7 level 2 phantom image failures using the large image
               receptor, 11 level 2 phantom image failures using the small image receptor, and numerous other level 2 and level 3
               violations.
2/19/96        ACR notified the facility and FDA that the clinical images reviewed on 11/20/95 were acceptable.
3/19/96        FDA imposed a Directed Plan of Correction requiring the facility to (1) have a medical physicist complete a survey of all
               units within 30 days, (2) correct problems identified in the survey within 15 business days, (3) perform phantom image
               evaluation weekly and submit results to FDA monthly, and (4) perform other quality control tests.
4/17/96        FDA and state officials met with the facility’s management to discuss the Directed Plan of Correction and to review
               progress.
7/1/96         FDA reinspected the facility and found one level 2 violation involving dark room fog and two level 3 violations in other
               areas, but no phantom image failures for either large or small image receptors. FDA directed the facility to select a total
               of 28 sets of clinical images from three time periods between July 1995 and June 1996 for ACR review.
9/3/96         ACR review found most of the clinical images were unacceptable.
9/10/96        FDA imposed an amended Directed Plan of Correction and obtained agreement from the facility to discontinue
               performing mammography with the resident radiologic technologists and interpreting physician until they were
               retrained.
9/17/96        All but one of the facility’s radiologic technologists and the interpreting physician completed training, and a new FDA-
               and ACR-approved technologist was added to the facility’s staff. The facility reopened and reestablished
               mammography services.
9/18/96        FDA notified the facility that ACR would conduct additional clinical image reviews of (1) a sample of clinical images
               after the personnel had resumed performing mammography for about 1 month and (2) all mammograms taken
               between June 6, 1996, and September 9, 1996.
10/11/96       The facility appealed FDA’s amended Directed Plan of Correction.
11/12/96       FDA denied the facility’s appeal.




                                              Page 25                                    GAO/HEHS-97-25 FDA’s Mammography Inspections
Appendix IV

Comments From the Food and Drug
Administration and Our Evaluation

Note: GAO comments
supplementing those in the
report text appear at the
end of this appendix.




                             Page 26   GAO/HEHS-97-25 FDA’s Mammography Inspections
Appendix IV
Comments From the Food and Drug
Administration and Our Evaluation




Page 27                             GAO/HEHS-97-25 FDA’s Mammography Inspections
Appendix IV
Comments From the Food and Drug
Administration and Our Evaluation




Page 28                             GAO/HEHS-97-25 FDA’s Mammography Inspections
Appendix IV
Comments From the Food and Drug
Administration and Our Evaluation




Page 29                             GAO/HEHS-97-25 FDA’s Mammography Inspections
                 Appendix IV
                 Comments From the Food and Drug
                 Administration and Our Evaluation




See comment 1.




                 Page 30                             GAO/HEHS-97-25 FDA’s Mammography Inspections
                        Appendix IV
                        Comments From the Food and Drug
                        Administration and Our Evaluation




See comment 2. Now on
pp. 2 and 8.




See comment 3.




                        Page 31                             GAO/HEHS-97-25 FDA’s Mammography Inspections
                 Appendix IV
                 Comments From the Food and Drug
                 Administration and Our Evaluation




See comment 4.




See comment 5.




                 Page 32                             GAO/HEHS-97-25 FDA’s Mammography Inspections
Appendix IV
Comments From the Food and Drug
Administration and Our Evaluation




Page 33                             GAO/HEHS-97-25 FDA’s Mammography Inspections
Appendix IV
Comments From the Food and Drug
Administration and Our Evaluation




Page 34                             GAO/HEHS-97-25 FDA’s Mammography Inspections
Appendix IV
Comments From the Food and Drug
Administration and Our Evaluation




Page 35                             GAO/HEHS-97-25 FDA’s Mammography Inspections
Appendix IV
Comments From the Food and Drug
Administration and Our Evaluation




Page 36                             GAO/HEHS-97-25 FDA’s Mammography Inspections
               Appendix IV
               Comments From the Food and Drug
               Administration and Our Evaluation




               The following are GAO’s additional comments on the letter received from
               the Food and Drug Administration dated November 18, 1996.



GAO Comments

Comment 1      FDA  commented that the draft report did not discuss the inherent
               limitations of the phantom image test or the lack of scientific consensus
               on a test for the large image receptor. While our draft report correctly
               reflected FDA’s view that the phantom image test is only an indicator of
               image problems, we agreed to add clarifying information to recognize
               limitations suggested by FDA. Similarly, we have added clarification to
               recognize that, according to FDA, developing a standard for the large image
               receptor would require additional scientific testing. While we recognize
               that developing guidance for the large image receptor will take time, FDA is
               in a position to continue to provide leadership in conducting experiments
               and in building a scientific consensus on a particular test method.


Comment 2      FDA commented that our method of presenting aggregate data on the
               extent of all violations detected during the first and second years of
               inspections tended to give too much weight to level 3 violations, which FDA
               characterized as minor. While our report points out that all level 3
               violations are not universally regarded as minor, we agree with FDA that
               aggregating all levels of violations could potentially be misleading. As a
               result, we have eliminated the aggregate totals from our final report.


Comment 3      While FDA acknowledged that there have been some start-up problems
               with the timely follow-up of violations, it asserted that it now has all
               necessary procedures in place to follow up on violations. We believe that
               the lack of an adequate compliance follow-up system has been an ongoing
               problem. Our contacts with FDA field offices, one as recent as late
               September 1996, showed the lack of a systematic approach to follow up on
               previous inspection violations. We agree with FDA, however, that the
               establishment of its automated Compliance Tracking System has
               significant potential to alleviate the problems with follow-up.




               Page 37                             GAO/HEHS-97-25 FDA’s Mammography Inspections
            Appendix IV
            Comments From the Food and Drug
            Administration and Our Evaluation




Comment 4   FDA commented that variation among the states in the number of violations
            reported would be expected because some states had well-developed
            mammography programs before MQSA and, as a result, presumably would
            have had fewer violations than other states. In addition, FDA stated that
            some states may have imposed stricter standards than those provided by
            MQSA. We agree that there could be variation in frequency of violations
            among the states attributable to the states’ pre-MQSA experiences with
            mammography standards. However, the violation data, in our view, are not
            reported in a consistent enough fashion to sustain such analysis of
            variation. Moreover, whether states have higher standards than MQSA
            should not affect violation data if they are correctly reported by the states.
            States that establish and enforce higher standards than MQSA should,
            according to FDA’s own guidance, enforce these standards outside of the
            MQSA process.



Comment 5   FDA also commented that our draft report did not accurately reflect the
            circumstances surrounding FDA’s enforcement in the case of the North
            Carolina facility. We believe our draft report provided an adequate
            summary of the key facts in the North Carolina case sufficient to justify
            our recommendations for additional enforcement procedures, guidance,
            and training. We note that, after reviewing our draft report, FDA took action
            to implement our recommendations. However, since FDA believes that
            additional facts are relevant to the discussion, we have added them to our
            final report. Specifically, we have (1) added a footnote to the body of the
            report to explain more fully how FDA reached its conclusion that it would
            not suspend the facility operation; (2) amended the appendix that contains
            the chronology of events related to the facility; and (3) as explained above,
            added information recognizing the limitations of phantom images and
            clarifying the lack of consensus on available tests for the large image
            receptor.




            Page 38                             GAO/HEHS-97-25 FDA’s Mammography Inspections
Appendix V

GAO Contacts and Staff Acknowledgments


                  Frank Pasquier, Assistant Director, (206) 287-4861
GAO Contacts      Sophia Ku, Evaluator-in-Charge, (206) 287-4888


                  In addition to those named above, the following individuals made
Staff             important contributions to this report: Sarah F. Jaggar, Special Advisor for
Acknowledgments   Health Issues; Susan Lawes, Senior Social Science Analyst; Donna Bulvin,
                  Evaluator; Stan Stenersen, Senior Evaluator; Evan Stoll, Computer
                  Specialist; Craig Winslow and Stefanie Weldon, Senior Attorneys; and Clair
                  Hur, Intern.




(108261)          Page 39                           GAO/HEHS-97-25 FDA’s Mammography Inspections
Ordering Information

The first copy of each GAO report and testimony is free.
Additional copies are $2 each. Orders should be sent to the
following address, accompanied by a check or money order
made out to the Superintendent of Documents, when
necessary. VISA and MasterCard credit cards are accepted, also.
Orders for 100 or more copies to be mailed to a single address
are discounted 25 percent.

Orders by mail:

U.S. General Accounting Office
P.O. Box 6015
Gaithersburg, MD 20884-6015

or visit:

Room 1100
700 4th St. NW (corner of 4th and G Sts. NW)
U.S. General Accounting Office
Washington, DC

Orders may also be placed by calling (202) 512-6000
or by using fax number (301) 258-4066, or TDD (301) 413-0006.

Each day, GAO issues a list of newly available reports and
testimony. To receive facsimile copies of the daily list or any
list from the past 30 days, please call (202) 512-6000 using a
touchtone phone. A recorded menu will provide information on
how to obtain these lists.

For information on how to access GAO reports on the INTERNET,
send an e-mail message with "info" in the body to:

info@www.gao.gov

or visit GAO’s World Wide Web Home Page at:

http://www.gao.gov




PRINTED ON    RECYCLED PAPER
United States                       Bulk Rate
General Accounting Office      Postage & Fees Paid
Washington, D.C. 20548-0001           GAO
                                 Permit No. G100
Official Business
Penalty for Private Use $300

Address Correction Requested