oversight

Dietary Supplements: Review of Health-Related Call Records for Users of Metabolife 356

Published by the Government Accountability Office on 2003-03-31.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

             United States General Accounting Office

GAO          Report to the Chairman, Subcommittee
             on Wellness and Human Rights,
             Committee on Government Reform,
             House of Representatives

March 2003
             DIETARY
             SUPPLEMENTS
             Review of
             Health-Related Call
             Records for Users of
             Metabolife 356




GAO-03-494
             a
                                               March 2003


                                               DIETARY SUPPLEMENTS

                                               Review of Health-Related Call Records for
Highlights of GAO-03-494, a report to the
Chairman, Subcommittee on Wellness             Users of Metabolife 356
and Human Rights, Committee on
Government Reform, House of
Representatives




Dietary supplements containing                 Adverse event reports generally are not sufficient on their own to establish
ephedra, such as Metabolife 356,               that reported problems are caused by the use of a particular product, but can
have been associated with serious              signal potential health problems that deserve investigation. The information
adverse health-related events. In a            in the Metabolife International call records was limited. Call records were
February 28, 2003, announcement,               sometimes difficult to understand, and consumer information was not
the Food and Drug Administration
                                               consistently recorded. In some cases, the evidence for a report of an
(FDA) proposed that dietary
supplements containing ephedra                 adverse event was limited to a single word on the record. Most call records
include a statement on their label             also did not record complete information about potentially relevant items
warning that “Heart attack, stroke,            such as the consumer’s age, sex, weight, and height. Information about both
seizure, and death have been                   the amount of product used and the duration of use was recorded for 60
reported after consumption of                  percent of the call records. Handwritten call records were difficult to read
ephedrine alkaloids.”                          and understand.

GAO was asked to review health-                By GAO’s categorization, 14,684 of the call records contained reports of at
related call records that Metabolife           least one adverse event. GAO found that there were 92 reports of the serious
International—the manufacturer of              adverse events identified in FDA’s proposed label warning—18 reported
Metabolife 356—collected from
                                               heart attacks, 26 reported strokes, 43 reported seizures, and 5 reported
consumers from May 1997 through
July 2002. Most of the records                 deaths. Other types of adverse events identified as serious or potentially
were from calls to a consumer                  serious by FDA in 1997 that were reported in the call records included
phone line the company                         significant elevation in blood pressure, abnormal heart rhythm, loss of
maintained. Metabolife                         consciousness, and systemic rash. Because of the inherent limitations of
International voluntarily provided             adverse event reports and the incomplete nature of these call records, it can
the call records to GAO.                       not be established from the information available to GAO that the adverse
                                               events reported were caused by Metabolife 356.
Specifically, GAO (1) examined the
extent to which consumer                       All of the reviews of Metabolife International call records—one by
information in the call records was            Metabolife International; three by consultants commissioned by Metabolife
comprehensive, interpretable, and
                                               International; one by the minority staff of the Committee on Government
consistently recorded, (2) counted
the number of call records                     Reform, House of Representatives; one by the RAND Corporation; and one
reporting types of adverse events              by GAO—found reports of serious adverse events, although none reported
that FDA had identified in 1997 as             identical results. For the set of adverse events counted by Metabolife
serious or potentially serious, and            International—heart attack, stroke, seizure, death, and cardiac arrest—
(3) compared GAO’s findings to                 GAO’s counts were similar to those of the other reviews. GAO counted 96
those of six other reviews of the              such reports and the counts of the other reviews ranged from 65 to 107.
call records, including one by
Metabolife International.                      In commenting on a draft of this report, FDA discussed the value of reports
                                               of adverse events in helping to understand the causes of such events.




www.gao.gov/cgi-bin/getrpt?GAO-03-494.

To view the full report, including the scope
and methodology, click on the link above.
For more information, contact Marcia Crosse
at (202) 512-7119.
Contents


Letter                                                                                      1
               Results in Brief                                                             5
               Background                                                                   6
               Consumer Information in Call Records Was Limited, Sometimes
                 Difficult to Interpret, and Not Consistently Recorded                      9
               Metabolife International Had Thousands of Call Records Reporting
                 Adverse Events Associated with Metabolife 356                            11
               Findings of Different Reviews of Metabolife International Call
                 Records Vary                                                             14
               Summary                                                                    16
               Agency and Metabolife International Comments and Our
                 Evaluation                                                               16

Appendix I     Scope and Methodology for Categorizing the Call
               Records                                                                    18
               Call Records And Supplementary Information                                 18
               Exclusion of Duplicate and Nonhealth-Related Call Records                  19
               Classification of Records and Data Entry Procedures                        20

Appendix II    Metabolife 356 Label                                                       22



Appendix III   Requirements for Reporting Adverse Events to FDA                           23



Appendix IV    Comments from the Food and Drug Administration                             28



Appendix V     GAO Contact and Staff Acknowledgments                                      30
               GAO Contact                                                                30
               Acknowledgments                                                            30


Tables
               Table 1: Percentage of Call Records in Which Consumer and
                        Response Details Were Recorded                                    10
               Table 2: Metabolife 356 Call Records Reporting Heart Attack,
                        Stroke, Seizure, or Death                                         12


               Page i                         GAO-03-494 Metabolife 356 Adverse Event Reports
Table 3: Metabolife 356 Call Records Reporting Adverse Events
         Described as Serious or Potentially Serious in FDA’s 1997
         Proposed Rule                                                                    13
Table 4: Number of Call Records Containing Reports of Heart
         Attack, Stroke, Seizure, Death, or Cardiac Arrest Reported
         in Reviews of Metabolife International Call Records                              15
Table 5: Requirements for Reporting Adverse Events to FDA                                 23




Abbreviations

ANADA             Abbreviated New Animal Drug Application
ANDA              Abbreviated New Drug Application
FDA               Food and Drug Administration
HHS               Department of Health and Human Services
NADA              New Animal Drug Application
NDA               New Drug Application



This is a work of the U.S. Government and is not subject to copyright protection in the
United States. It may be reproduced and distributed in its entirety without further
permission from GAO. It may contain copyrighted graphics, images or other materials.
Permission from the copyright holder may be necessary should you wish to reproduce
copyrighted materials separately from GAO’s product.




Page ii                                GAO-03-494 Metabolife 356 Adverse Event Reports
United States General Accounting Office
Washington, DC 20548




                                   March 31, 2003

                                   The Honorable Dan Burton
                                   Chairman
                                   Subcommittee on Wellness and Human Rights
                                   Committee on Government Reform
                                   House of Representatives

                                   Dear Mr. Chairman:

                                   Medical experts have expressed concerns about the safety of dietary
                                   supplements containing ephedra or ephedrine alkaloids, which are used by
                                   millions of Americans annually.1 On February 28, 2003, the Food and Drug
                                   Administration (FDA) announced several proposed changes to its
                                   regulation of dietary supplements containing ephedra, including requiring
                                   a product label warning that “Heart attack, stroke, seizure, and death have
                                   been reported after consumption of ephedrine alkaloids.”2 As of
                                   September 27, 2002, FDA had received approximately 1,800 adverse event
                                   reports regarding consumers of dietary supplements containing ephedra.
                                   Of these, 322 concerned Metabolife 356, a weight loss product first
                                   marketed in 1995 by Metabolife International, a large manufacturer of
                                   dietary supplements containing ephedra. Adverse event reports can signal
                                   potential health problems that deserve additional investigation, but, on
                                   their own, generally are not sufficient to establish that the reported
                                   problems were caused by use of the product.

                                   Metabolife International has also received complaints about adverse
                                   health events among users of Metabolife 356.3 Between August and
                                   December 2002, Metabolife International made available to the public



                                   1
                                    It has been estimated that 12 million Americans consumed dietary supplements with
                                   ephedra in 1999 (C. A. Haller and N. L. Benowitz, “Adverse Cardiovascular and Central
                                   Nervous System Events Associated with Dietary Supplements Containing Ephedra
                                   Alkaloids,” The New England Journal of Medicine, vol. 343, no. 25 (2000)).
                                   2
                                    See 68 Fed. Reg. 10417 (Mar. 5, 2003). FDA also announced that it is reopening the
                                   comment period for its June 4, 1997, proposed rule, “Dietary Supplements Containing
                                   Ephedrine Alkaloids” (62 Fed. Reg. 30678).
                                   3
                                    There is no information available about the extent to which reports of particular adverse
                                   events may have been reported to both FDA and Metabolife International.



                                   Page 1                                 GAO-03-494 Metabolife 356 Adverse Event Reports
redacted4 copies of nearly 16,000 pages of documentation that it identified
as containing reports of adverse events among consumers of Metabolife
356.5 These complaints, which were received between May 1997 and July
2002, had not been previously released to FDA. Most of them were records
of calls received through a consumer health information phone line
established by Metabolife International in 1998 and staffed by its nurses.6
Metabolife International officials told us that the phone line was
established to provide information to consumers regarding appropriate
use of Metabolife 356. In letters to the Texas Department of Health and
FDA,7 company officials described the phone line as a “safety monitoring
procedure” for the reporting of medical complaints. The call records
ranged from handwritten notes to printed versions of records that had
been entered into a database developed by Metabolife International. These
call records have been the subject of six previous reviews: one by
Metabolife International,8 three by consultants commissioned by




4
 The redaction consisted primarily of the removal of personal identifying information (such
as names, phone numbers, addresses, and e-mail addresses) to protect consumer privacy.
Although data relevant to the adverse event being reported were not supposed to be
removed, Metabolife International officials noted that such information was occasionally
accidentally removed.
5
 The number of adverse event reports does not equal the pages of documentation because
some pages contained reports of more than one call reporting an adverse health event,
some reports of adverse health events spanned several pages, and some pages included
reports not related to negative health consequences.
6
 In addition to phone calls, some call records were letters and e-mails sent to Metabolife
International.
7
 The letter to FDA is available at http://www.fda.gov/ohrms/dockets/dockets/98n0148/2.htm
(letter from Metabolife International received February 10, 1999) (downloaded March 24,
2003).
8
 Metabolife International has not issued a report on its review of the call records, but
provided to us a list of the calls it believed to report heart attack, stroke, seizure, death, and
cardiac arrest.




Page 2                                    GAO-03-494 Metabolife 356 Adverse Event Reports
Metabolife International,9 one by the minority staff of the Committee on
Government Reform, House of Representatives,10 and one by the RAND
Corporation.11

You asked us to review the content of all health-related call records made
public by Metabolife International. Specifically, you asked us to answer
the following questions. (1) To what extent was consumer information in
the call records comprehensive, interpretable, and consistently recorded?
(2) How many call records reported health-related problems, and how
many of those were serious? (3) How do our counts of reported serious
adverse events compare with those of other reviews for those events
counted by Metabolife International?

In responding to your request, we reviewed all the pages of documentation
voluntarily provided to us by Metabolife International. We did not
independently verify that we received all of the call records held by
Metabolife International. We excluded from our review call records that




9
 Each of the consultants reviewed the first set of approximately 12,700 pages of Metabolife
International records released in August 2002. Steven B. Karch, An Analysis of Metabolife
356 HealthLine Contacts (August 2002) www.ephedrafacts.com/metabolife.html
(downloaded Dec. 12, 2002); Craig A. Molgaard, Epidemiologic Assessment of Health Line
Reports about a Dietary Supplement (August 2002); Ashraf Mozayani, Analysis of
Metabolife 356 Health Line Reports (August 2002). After more pages of call records were
made available, each of the consultants completed updated reviews with these additional
reports. Steven B. Karch, An Analysis of an Additional 3268 HealthLine Records
(Jan. 17, 2003); Craig A. Molgaard, An Analysis of Additional HealthLine Records (Jan. 17,
2003); Ashraf Mozayani, Supplemental Report of Analysis of Metabolife 356 Health Line
Reports (January 2003).
10
 Minority Staff Report, Special Investigations Division, Committee on Government Reform,
House of Representatives, Adverse Event Reports from Metabolife (October 2002).
www.house.gov/reform/min (downloaded Dec. 11, 2002).
11
 Paul Shekelle, Sally C. Morton, Margaret Maglione, and colleagues, Ephedra and
Ephedrine for Weight Loss and Athletic Performance Enhancement: Clinical Efficacy
and Side Effects. Evidence Report/Technology Assessment No. 76 (Prepared by Southern
California Evidence-based Practice Center, RAND, under Contract No. 290-97-0001, Task
Order No. 9). AHRQ Publication No. 03-E022, Rockville, Md: Agency for Healthcare
Research and Quality, February 2003.




Page 3                                 GAO-03-494 Metabolife 356 Adverse Event Reports
duplicated other records.12 To determine the extent to which consumer
information was comprehensive, interpretable, and consistently recorded
in the call records, we recorded information about the adverse event,
demographic information about the individual consumer, and other details
in the call record. Specifically, we recorded background information
similar to that used by FDA in the reporting of adverse events, including
age, sex, weight, height, the amount of Metabolife 356 used, the duration
of use, and whether any medical history was noted in the call record.

To assess how many call records reported health-related problems and
how many of those were serious, we first counted the number of call
records that reported at least one adverse event. Within this set of call
records, we then counted the number of reports of heart attack, stroke,
seizure, and death—the types of serious adverse events identified in FDA’s
proposed label warning. We also counted the number of reports of the
23 other types of adverse events that were described as serious or
potentially serious in FDA’s 1997 proposed rule on dietary supplements
containing ephedrine alkaloids.13 For call records that did not report any of
the above adverse events, we counted the number of records, but did not
count the number of other specific types of adverse events reported.

We classified events in the call records based solely on the words and
phrases therein; we did not attempt to diagnose a consumer’s condition or
to otherwise interpret the information presented.14 We did not apply



12
  Metabolife International officials identified call records they believed were duplicates of
each other. We reviewed the relevant call records to determine which were duplicates. Call
records identified by Metabolife International officials as duplicates were either
photocopies of specific call records, multiple entries of the same call (such as handwritten
notes that were later also entered into the database, creating two pages of call records for
the same call), or multiple calls about the same consumer describing different events. We
considered the first two instances, but not the third, to be duplicates. We did not include in
our review reports that we considered duplicates. We also identified additional call records
that were duplicates and removed them from our review.
13
  FDA’s June 4, 1997, proposed rule identified serious or potentially serious adverse events
associated with the use of ephedra based on a review of the literature and an analysis of
600 adverse event reports that FDA had received by June 7, 1996. See “Dietary Supplements
Containing Ephedrine Alkaloids,” 62 Fed. Reg. 30678. We did not count reports of one of
the events FDA identified, “altered serum enzymes,” because the proposed rule did not
specify threshold values.
14
 We required that certain words be in the call record for it to be counted as a specific type
of event. For example, for a call record to meet the criteria for a stroke, it needed to
specifically include the word “stroke,” not related terms like “stroke-like symptoms.”




Page 4                                  GAO-03-494 Metabolife 356 Adverse Event Reports
                   medical judgment in the process of classifying events and we did not
                   independently verify the accuracy of the information in the records or
                   determine the validity of the claims made in the call records. We also did
                   not attempt to determine whether Metabolife 356 caused the reported
                   adverse events. Our results may either overestimate or underestimate the
                   number and severity of adverse events because the call records generally
                   do not include medical diagnoses made by qualified professionals.15

                   To determine how our counts of reported serious adverse events compare
                   with those of other reviews, we examined the six previous reviews of
                   Metabolife International’s call records. In addition, we interviewed
                   Metabolife International and FDA officials. Appendix I describes our
                   methodology in more detail. We conducted our work from September 2002
                   through March 2003 in accordance with generally accepted government
                   auditing standards.


                   The information in the Metabolife International call records was limited,
Results in Brief   sometimes difficult to understand and interpret, and consumer
                   information was not consistently recorded. In some cases, the evidence for
                   a report of an adverse event was limited to a single word on a call record.
                   In addition, most call records did not record complete information about
                   the consumer’s age, sex, weight, and height. Information about both the
                   amount of product used and duration of use was recorded for 60 percent
                   of the call records. Further, handwritten call records were difficult to read
                   and understand. Different versions of the call records sometimes
                   contained different information about the consumer and the symptoms
                   they reported. Nearly all of the reports of adverse events that contained
                   information about the amount of Metabolife 356 used and duration of use
                   were for consumers who reported following the usage guidelines on the




                   15
                     Our findings may either overestimate or underestimate the number and severity of
                   adverse events. Our findings may overestimate the number of adverse events because we
                   accepted the events as they were reported on the page. For example, if a call record
                   reported a stroke, we counted it as a stroke even though the consumer may not have
                   actually had a stroke. Conversely, our findings may underestimate the number and severity
                   of adverse events because individual adverse events we categorized as other adverse
                   events may collectively suggest a more serious event. For example, we categorized a call
                   record reporting left-side numbness and tingling and left-side face drooping as an other
                   adverse event where a physician or other health professional might have determined that
                   these symptoms actually represented a stroke.




                   Page 5                                GAO-03-494 Metabolife 356 Adverse Event Reports
             product label, not for consumers who reported that they took too much
             Metabolife 356 or used it for too long a period.

             We categorized 14,684 call records from Metabolife International as
             containing reports of at least one adverse event associated with Metabolife
             356. We found that there were 92 reports of the serious adverse events
             identified in FDA’s proposed label warning for dietary supplements
             containing ephedrine alkaloids: 18 reported heart attacks, 26 reported
             strokes, 43 reported seizures, and 5 reported deaths. Among the other
             adverse events reported that were identified as serious or potentially
             serious in FDA’s 1997 proposed rule, we found, for example, 93 reports of
             significant elevation of blood pressure, 31 reports of abnormal heart
             rhythm, 47 reports of loss of consciousness, and
             181 reports of systemic rash. Because of the inherent limitations of
             adverse event reports and the incomplete nature of these call records, we
             cannot establish that the reported adverse events were caused by the use
             of Metabolife 356.

             All of the reviews of the Metabolife International call records, including
             ours, counted reports of serious adverse events, although none of the
             reviews reported identical results. For those adverse events that
             Metabolife International counted—heart attacks, strokes, seizures, deaths,
             and cardiac arrests—our counts of reported events are similar to the
             counts from the other reviews. We counted 96 such reported events.
             Metabolife International counted 78, and the counts of the other reviews
             ranged from 65 to 107.

             In commenting on a draft of this report, FDA discussed the value of
             reports of adverse events in helping to understand the causes of such
             events.


             Metabolife 356, which claims to raise the body’s metabolism and help
Background   dieters lose weight while maintaining high energy levels, contains
             32 ingredients, including ephedra, guarana (an herbal source of caffeine),




             Page 6                          GAO-03-494 Metabolife 356 Adverse Event Reports
bee pollen, and caffeine.16 The product label recommends that adults take
one to two caplets two to three times per day or every 4 hours, not to
exceed eight caplets per day. Warnings on the product label suggest that a
health care professional be consulted by individuals who are using any
other dietary supplement, prescription drug, or over-the-counter drug
containing ephedrine alkaloids or who have, or have a family history of,
any of 11 health conditions, including heart disease, high blood pressure,
diabetes, recurrent headaches, and depression. The label also
recommends that persons should not use the product for more than
12 weeks and that exceeding the recommended amount may cause serious
adverse health effects including heart attack or stroke. Other possible side
effects mentioned on the label include rapid heartbeat, dizziness, severe
headache, and shortness of breath. The complete product label is in
appendix II.

The Dietary Supplement Health and Education Act of 1994 created a
framework for FDA’s regulation of dietary supplements as part of its
oversight of food safety. Dietary supplements are generally marketed
without prior FDA review of their safety and effectiveness.17
Manufacturers of dietary supplements are responsible for ensuring the
safety of the dietary supplements they sell. Therefore, FDA relies on
voluntary reports of adverse events from consumers, health professionals,
and others in its effort to oversee the safety of marketed dietary
supplements.

Although there are no adverse event reporting requirements for
manufacturers of dietary supplements, there are such requirements for
many other products regulated by FDA. Various types of adverse events




16
 According to Metabolife International officials, the only ingredient change since
Metabolife 356 was placed on the market was made in early 2001, when bovine complex
was removed from the product. Some other inactive ingredients may vary by
manufacturing facility. Metabolife International officials told us that the same labels are
used for products sold in all states.
17
  FDA officials reported that the agency conducts a premarket review of safety information
for certain supplements that contain new dietary ingredients.




Page 7                                  GAO-03-494 Metabolife 356 Adverse Event Reports
associated with the use of human drugs and biologics,18 animal drugs,
animal feeds containing animal drugs, medical devices, infant formulas,
and radiation-emitting devices must be reported to FDA. In addition to
dietary supplements, other products regulated by FDA that do not require
adverse event reporting are foods, cosmetics, and color additives. (See
app. III for details about adverse event reporting requirements.)

Voluntary adverse event reporting systems can be valuable tools for
identifying potentially serious health issues that may be associated with
the use of a product and for maintaining ongoing surveillance. FDA has
used adverse event reports to identify issues for further investigation and,
as we previously reported, it has used adverse event reports to help
identify dietary supplements for which evidence of harm existed, and has
issued warnings and alerts for dietary supplements.19 However, by
themselves, adverse event reporting systems generally are not sufficient to
establish that a product caused the reported health problem. As we noted
in 1999, all voluntary surveillance systems, including FDA’s adverse event
reporting system, have certain weaknesses.20 These include
underreporting, reporting biases, difficulties estimating population
exposure, and poor report quality. For example, the Department of Health
and Human Services (HHS) Inspector General reported that a study
commissioned by FDA estimated that FDA receives reports for less than
1 percent of adverse events associated with dietary supplements. 21 In
addition, it is often difficult to rule out other possible explanations for the
event; for example, the event may have been caused by preexisting
medical conditions, or by the concurrent use of prescription drugs, over-




18
  Biologics are any virus, therapeutic serum, toxin, antitoxin, vaccine, blood, blood
component or derivative, allergenic product, or analogous product applicable to the
prevention, treatment, or cure of a human disease or condition. Biological products
include, but are not limited to, bacterial and viral vaccines, human blood and plasma and
their derivatives, and certain products produced by biotechnology, such as interferons and
erythropoietins.
19
 U.S. General Accounting Office, Health Products for Seniors: “Anti-Aging” Products Pose
Potential for Physical and Economic Harm, GAO-01-1129 (Washington, D.C.: Sep. 7,
2001).
20
  U.S. General Accounting Office, Dietary Supplements: Uncertainties in Analyses
Underlying FDA’s Proposed Rule on Ephedrine Alkaloids, GAO/HEHS/GGD-99-90
(Washington, D.C.: July 2, 1999).
21
 HHS Office of Inspector General, Adverse Event Reporting for Dietary Supplements: An
Inadequate Safety Valve, OEI-01-00-00180 (Washington, D.C.: April 2001).




Page 8                                 GAO-03-494 Metabolife 356 Adverse Event Reports
                        the-counter drugs, or other supplements. For these reasons, data from
                        adverse event reports alone cannot be used to determine if the occurrence
                        of a symptom among product users is unusually high.

                        Between August and December 2002, Metabolife International released
                        copies of 15,948 pages of documents that it said contained call records
                        that reported adverse events associated with Metabolife 356 that the
                        company had received from May 1997 through July 2002. Some pages of
                        call records contained information about more than one call while others
                        did not contain reports of adverse events. Some pages were photocopies
                        or duplicates of other pages.


                        The information about reported adverse events in the 14,684 health-related
Consumer                call records we examined was limited. Most of the call records we
Information in Call     reviewed did not completely record demographic or medical history
                        information about the consumer. Information about age, sex, weight,
Records Was Limited,    height, the amount of product used, and the duration of use was frequently
Sometimes Difficult     not recorded. Handwritten call records were difficult to read and interpret.
                        Information was often inconsistent across different versions of the same
to Interpret, and Not   call record.
Consistently
Recorded                The call records contained limited information about reported adverse
                        events and consumers. In some cases the evidence for a report of an
                        adverse event was a single health-related word on the call record, such as
                        “seizure” or “stroke.” In addition, demographic and medical history
                        information was not consistently recorded in the call records. Most of the
                        call records we reviewed did not record information about the consumers’
                        sex, age, weight, or height. Eighty-eight percent of the call records did not
                        record at least one of these variables. In addition, information about the
                        amount of Metabolife 356 used and the duration of use was not recorded
                        in 27 and 33 percent of the call records, respectively. (See table 1.) The
                        absence of this information makes it difficult to assess whether the call
                        records represent a signal of health concerns related to the consumption
                        of Metabolife 356.22




                        22
                          We previously reported that adverse event reports should optimally include demographic
                        data (GAO/HEHS/GGD-99-90). Such information is useful for determining whether or not
                        the adverse events reported would be unexpected in a specific population of users, for
                        example, heart attacks in young adults.




                        Page 9                                GAO-03-494 Metabolife 356 Adverse Event Reports
Table 1: Percentage of Call Records in Which Consumer and Response Details
Were Recorded

                                                                     Percentage of call records
                                                                              with information
    Type of detail recorded                                                          (n=14,684)
    Age                                                                                    42%
    Sexa                                                                                     41
    Weight                                                                                   62
    Height                                                                                   34
    Amount of Metabolife used                                                                73
    Duration of use                                                                          67
    Medical history                                                                          45

Source: GAO.

Note: Analysis of 14,684 health-related call records provided by Metabolife International. Where
information was not recorded, we do not know if Metabolife International did not record information in
the call records or if the caller did not provide the information.
a
 Metabolife International likely has more information about consumers’ sex than we did because in
many cases the company had access to the names of consumers to help make that determination.
Consumers’ names had been removed from the records Metabolife International provided us to
protect consumer privacy.


Both the amount of product used and duration of use were recorded for
60 percent of the calls reporting adverse events. Relatively few of these
records involved consumers who reported taking too much Metabolife 356
or using it for too long a period. Specifically, among call records
containing information on the amount of product used or duration of use,
99 and 91 percent of consumers, respectively, reported using the product
within the guidelines recommended on the label.

The format of the call records varied from brief handwritten notes to typed
notations to printed versions of a form used by Metabolife International. In
general, less information was recorded for the one-third of call records
that were handwritten than all other types of records. For example, calls
recorded on a typed form more frequently recorded additional information
such as recommendations by Metabolife International to discontinue
Metabolife 356 (62 percent) or contact a doctor (54 percent) than did
those on handwritten forms (13 percent and 8 percent, respectively).

Further, it was often difficult to read handwritten call records. We could
not always determine how many calls were reported on a single page since
there was rarely a clear delineation of events. Because handwritten call
records did not follow a template, we were unable to determine if some




Page 10                                     GAO-03-494 Metabolife 356 Adverse Event Reports
                            information was medical history or symptom information, or if a number
                            was a weight, heart rate, or blood pressure.

                            Information in call records was sometimes inconsistent. Where duplicate
                            call records were available, information about consumers and their usage
                            of Metabolife 356 was sometimes presented differently in the different
                            records of the same consumer call. In addition, Metabolife International
                            officials told us that its nurses sometimes used several different terms to
                            document the same type of adverse event.


                            We found that 14,684 of the Metabolife International call records reported
Metabolife                  at least one adverse event. Ninety-two of these were for the serious
International Had           adverse events identified in the proposed label warning for dietary
                            supplements containing ephedra that FDA announced on February 28,
Thousands of Call           2003. Other adverse events reported included significant elevation of
Records Reporting           blood pressure, abnormal heart rhythm, loss of consciousness, and
                            systemic rash. We cannot establish that any of the reported adverse events
Adverse Events              were caused by the use of Metabolife 356.
Associated with
Metabolife 356

Reports of Adverse Events   We counted 92 reports of heart attack, seizure, stroke, or death—the
Identified as Serious in    serious adverse events identified in FDA’s proposed label warning for
FDA’s Proposed Label        dietary supplements containing ephedra (see table 2).23
Warning




                            23
                             See 68 Fed. Reg. 10417 (Mar. 5, 2003).




                            Page 11                                   GAO-03-494 Metabolife 356 Adverse Event Reports
                       Table 2: Metabolife 356 Call Records Reporting Heart Attack, Stroke, Seizure, or
                       Death

                           Type of adverse event                                                                    Numbera
                           Heart attack                                                                                 18
                           Stroke                                                                                       26
                           Seizure                                                                                      43
                           Death                                                                                         5

                       Source: GAO.

                       Note: Analysis of 14,684 health-related call records provided by Metabolife International.
                       a
                       The counts do not represent unique consumers because a single call record may have more than
                       one complaint and because some consumers called the Metabolife health information phone line
                       more than once.




Other Adverse Events   In its 1997 proposed rule on dietary supplements, FDA also identified
                       other types of adverse events as serious or potentially serious. Table 3
                       shows our counts for almost all such events.24 The serious and potentially
                       serious types of adverse events described in FDA’s June 4, 1997, proposed
                       rule were reported to the agency prior to June 7, 1996. FDA officials report
                       that some other types of adverse events not included in the table may be
                       considered serious or potentially serious but had not been reported to
                       FDA during the time period considered by its proposed rule.




                       24
                         We counted all reports of 23 of the 24 other types of adverse events FDA identified as
                       serious or potentially serious in its 1997 proposed rule. We did not count reports of “altered
                       serum enzymes” since the proposed rule did not specify threshold values. The other serious
                       or potentially serious adverse events—coma, myopathies, exfoliative dermatitis, and
                       epididymitis—are not reported in the table because we did not find any reports of them in
                       the call records provided by Metabolife International.




                       Page 12                                     GAO-03-494 Metabolife 356 Adverse Event Reports
Table 3: Metabolife 356 Call Records Reporting Adverse Events Described as
Serious or Potentially Serious in FDA’s 1997 Proposed Rule

    Category of event          Event reported                                                Numbera
    Cardiovascular
                               Chest pain                                                        433
                               Significant elevation in blood pressureb                           93
                               Abnormal heart rhythm (alternative names for
                               this event include dysrhythmia, ventricular
                               tachycardia, ventricular fibrillation, atrial
                                                            c
                               fibrillation, atrial flutter)                                      31
                               Cardiomyopathy                                                      7
                               Cardiac arrest                                                      4
                               Angina                                                              3
    Nervous system
                               Loss of consciousness                                              47
                               Psychosis                                                           7
                               Altered consciousness (including disorientation
                               or confusion)                                                       4
                               Suicidal                                                            3
                               Vestibular (inner ear) disturbance                                  2
                               Severe depression                                                   2
                               Mania                                                               1
    Other
                               Systemic rash                                                     181
                               Urinary infection                                                 110
                               Urinary retention                                                  72
                               Elevations of liver function tests                                 54
                               Prostatitis                                                        24
                               Hepatitis                                                           1

Source: GAO.

Note: Analysis of 14,684 health-related call records provided by Metabolife International.
a
The counts do not represent unique consumers because a single call record may have more than
one complaint and because some consumers called the Metabolife health information phone line
more than once.
b
 We used the MEDLINE Plus Medical Encyclopedia to define significant elevations in blood pressure
as a measurement of greater than 160 millimeters of mercury systolic or 100 millimeters of mercury
diastolic. This count does not include call records that mentioned “high blood pressure” or “elevated
blood pressure” without specifying these levels.
c
 Alternative names for abnormal heart rhythm were determined using the MEDLINE Plus Medical
Encyclopedia (www.nlm.nih.gov/medlineplus/encyclopedia.html) (downloaded December 2002
through February 2003).




Page 13                                     GAO-03-494 Metabolife 356 Adverse Event Reports
                            In addition, the 14,684 call records with health-related reports presented a
                            broad range of types of adverse events. Many of the call records contained
                            reports of jitters, insomnia, hair loss, bruising, menstrual irregularities, and
                            sexual dysfunction, as well as vague references to events such as “side
                            effect” or “felt sick.” Some reported blood in stool, blood in urine, or blood
                            clots. There were also some reports of visits to emergency departments
                            and hospital admissions. Some call records contained reports of diseases
                            such as pulmonary embolus (a blockage of an artery in the lungs), multiple
                            myeloma, and inflammation of heart tissue.


Causal Role of Metabolife   We cannot establish that any of the adverse events reported in the
356 Cannot Be Established   Metabolife International call records were caused by the use of Metabolife
                            356. As we noted earlier, adverse event reports by themselves are
                            generally not sufficient to establish that a health problem was caused by
                            the use of a particular product. For example, for many adverse event
                            reports it is difficult to rule out other possible explanations for the event—
                            the event may have been caused by preexisting medical conditions, or by
                            the concurrent use of prescription drugs, over-the-counter drugs, or other
                            dietary supplements. In addition, the limited information available in the
                            Metabolife International call records means that we cannot confirm that a
                            particular adverse event occurred, much less identify a specific cause for
                            it.


                            All the reviews of the Metabolife International call records, including ours,
Findings of Different       counted reports of serious adverse events. None of the reviews reported
Reviews of Metabolife       identical tabulations of these events. For the set of adverse events that
                            Metabolife International counted—heart attack, stroke, seizure, death, and
International Call          cardiac arrest—our counts are similar to those of the other reviews (see
Records Vary                table 4). In total, we counted 96 such events, Metabolife International
                            counted 78, and the counts of the other reviews ranged from 65 to 107.




                            Page 14                           GAO-03-494 Metabolife 356 Adverse Event Reports
Table 4: Number of Call Records Containing Reports of Heart Attack, Stroke, Seizure, Death, or Cardiac Arrest Reported in
Reviews of Metabolife International Call Records

                                                                                                            Minority Staff,
                                                                                                            Committee on
                                                                                                              Government
                                                                                                         Reform, House of
 Events                     GAO      Metabolife          Karcha       Mozayanib        Molgaardc         Representativesd             RAND
                                                                                                                                              e

                                               f
 Heart attack                 18            16               17             13               13                         20               22
 Stroke                       26            20               24             19               13                         24               31
 Seizure                      43            35               40             52               36                         40               46
 Death                         5              3               2              4                 3                         3                5
                                               f
 Cardiac arrest                4              4               4              5               NC                        NC                 3
 Total                        96            78               87             93               65                         87              107

Source: GAO and others.

                                         Notes: “NC” indicates that these types of events were not counted by these reviews. The counts do
                                         not represent unique consumers because a single call record may have more than one complaint and
                                         because some consumers called the Metabolife health information phone line more than once.
                                         a
                                         Steven B. Karch, An Analysis of Metabolife 356 HealthLine Contacts (August 2002),
                                         www.ephedrafacts.com/metabolife.html (downloaded Dec. 12, 2002), and An Analysis of an
                                         Additional 3268 HealthLine Records (Jan. 17, 2003).
                                         b
                                         Ashraf Mozayani, Analysis of Metabolife 356 Health Line Reports (August 2002), and Supplemental
                                         Report of Analysis of Metabolife 356 Health Line Reports (January 2003).
                                         c
                                          Craig A. Molgaard, Epidemiologic Assessment of Health Line Reports about a Dietary Supplement
                                         (August 2002), and An Analysis of Additional HealthLine Records (Jan. 17, 2003).
                                         d
                                          Minority Staff Report, Special Investigations Division, Committee on Government Reform, House of
                                         Representatives, Adverse Event Reports from Metabolife (October 2002), www.house.gov/reform/min
                                         (downloaded Dec. 11, 2002). This review did not include at least 1,480 pages of call records
                                         Metabolife International later made available to us and other reviews.
                                         e
                                         Paul Shekelle, Sally C. Morton, Margaret Maglione, and colleagues, Ephedra and Ephedrine for
                                         Weight Loss and Athletic Performance Enhancement: Clinical Efficacy and Side Effects. Evidence
                                         Report/Technology Assessment No. 76 (Prepared by Southern California Evidence-based Practice
                                         Center, RAND, under Contract No. 290-97-0001, Task Order No. 9). AHRQ Publication No. 03-E022,
                                         Rockville, Md: Agency for Healthcare Research and Quality, February 2003.
                                         f
                                         Metabolife International provided to us a list of call records it considered to report cardiac events.
                                         Because the other reviews counted heart attacks and cardiac arrests separately, we examined the
                                         events that Metabolife International classified as cardiac events to categorize them as cardiac arrest
                                         or heart attack.


                                         There are several possible reasons for the slightly different counts of
                                         serious adverse events in the different reviews. First, the call records
                                         themselves are often difficult to understand and interpret. Second, not all
                                         of the reviews included the same set of call records, both because some
                                         were completed before all of the Metabolife International call records
                                         were released and because the reviews adopted different procedures for
                                         identifying and discarding duplicate records. Third, the reviews used
                                         different definitions of particular events or established different thresholds



                                         Page 15                                     GAO-03-494 Metabolife 356 Adverse Event Reports
                   for categorizing a particular event. For example, we included reports of
                   “convulsions” in our count of seizures, while some other reviews may not
                   have. Specifically, the counts we report in table 4 for our review and the
                   reviews by Metabolife International and Karch include reports of
                   convulsions, while it is not clear if the other reviewers’ counts did.
                   Similarly, we did not count as a report of a heart attack a call record that
                   reported “heart attack?”, while at least one other review did.


                   The information in the Metabolife International call records was limited,
Summary            sometimes difficult to understand and interpret, and consumer
                   information was not consistently recorded. Most call records contained
                   only limited information about a consumer and the event being reported,
                   and handwritten records were difficult to read and understand. We
                   categorized 14,684 call records from Metabolife International as containing
                   reports of at least one adverse event associated with Metabolife 356. We
                   found that there were 92 reports of the types of serious adverse events
                   identified in FDA’s proposed label warning for dietary supplements
                   containing ephedrine alkaloids. All of the reviews of the Metabolife
                   International call records, including ours, counted reports of serious
                   adverse events, although none of the reviews reported identical results.
                   We counted 96 reports of the types of events counted by Metabolife
                   International—heart attack, stroke, seizure, death, and cardiac arrest—
                   and the counts of the other reviews ranged from 65 to 107.


                   We provided a draft of this report to FDA and Metabolife International for
Agency and         their review. FDA asked us to clarify that it has not conducted its own
Metabolife         review of the Metabolife International call records, that we only reviewed
                   reports of adverse events contained in the Metabolife International call
International      records, and that we did not review other reports of adverse events among
Comments and Our   users of Metabolife 356 that have been received by FDA. In addition, FDA
                   pointed out that, when combined with other information, adverse event
Evaluation         reports can help establish that an adverse event was caused by a particular
                   health product. FDA’s comments are included as appendix IV. FDA also
                   provided technical comments, which we incorporated as appropriate.

                   In its comments, Metabolife International was primarily concerned about
                   our use of the term “adverse events” to describe the health-related
                   complaints that were reported in the call records we reviewed. We believe
                   that our use of the term is accurate and consistent with its use by FDA and
                   others in the field. Metabolife International also wanted us to clarify that,
                   while it did identify some call records as containing references to types of


                   Page 16                          GAO-03-494 Metabolife 356 Adverse Event Reports
specific adverse events that have been categorized as serious by others, it
has not identified any call records as reporting “serious adverse events.”
We have made revisions so as not to imply that Metabolife International
labeled these events as serious adverse events. Metabolife International
also made other comments, which we incorporated as appropriate.


As agreed with your office, unless you publicly announce its contents
earlier, we plan no further distribution of this report until 30 days from the
date of this report. At that time, we will send copies to the Secretary of
HHS, the Commissioner of FDA, and others who are interested. We will
also provide copies to others upon request. In addition, the report will be
available at no charge on GAO’s Web site at http://www.gao.gov.

If you or your staff have any questions, please contact me at (202) 512-
7119. Another contact and major contributors to this report are listed in
appendix V.

Sincerely yours,




Marcia Crosse
Acting Director, Health Care—Public
 Health and Science Issues




Page 17                          GAO-03-494 Metabolife 356 Adverse Event Reports
                   Appendix I: Scope and Methodology for
Appendix I: Scope and Methodology for
                   Categorizing the Call Records



Categorizing the Call Records

                   We reviewed call records and supplementary information voluntarily
                   provided to us by Metabolife International to (1) determine the extent to
                   which information was comprehensive, interpretable, and consistently
                   recorded in the call records, and (2) count the number of call records
                   reporting health-related problems associated with Metabolife 356, and how
                   many of them were serious. During our review we removed duplicate call
                   records and records that did not report health-related events. For each
                   record we recorded demographic information about the individual
                   consumer, other details about the call record and the consumer, and
                   categorized the reported events.


                   From August 2002 through December 2002, Metabolife International
Call Records And   voluntarily provided to us 15,948 pages1 of documentation relating to
Supplementary      reports of adverse events among consumers of Metabolife 356. Most of
                   these records were from calls made to the company’s consumer health
Information        information phone line from May 1997 through July 2, 2002.2 Other records
                   included e-mail messages and letters that had been sent to the company.
                   Nurses on the staff of Metabolife International documented the calls to the
                   consumer HealthLine in a variety of formats. The records included
                   handwritten notes on a page, typed and handwritten letters, forms with
                   handwritten entries, e-mails, and printed versions of records that had been
                   entered into a database developed by Metabolife International. Many kinds
                   of forms were used to record calls, ranging from simple forms with few
                   spaces or check boxes to full-page forms with multiple boxes for
                   consumer and event-related information. Metabolife International officials
                   told us that health complaints that were noted on product return forms
                   that it received were not in the call records provided to us.

                   Metabolife International also provided to us copies of 46 redacted medical
                   records and a list of corresponding call records. After reviewing these
                   records we found 8 that were not associated with other call records. Five




                   1
                    These 15,948 pages contained 14,684 call records that we categorized as reporting adverse
                   events. The number of adverse event reports does not equal the pages of documentation
                   because some pages contained reports of more than one call reporting an adverse health
                   event, some reports of adverse health events spanned several pages, and some pages
                   included reports not related to negative health consequences.
                   2
                    Metabolife International received the call records we reviewed primarily from mid
                   1998 through July 2002, although 12 call records were from 1997 and some were from early
                   1998.




                   Page 18                                 GAO-03-494 Metabolife 356 Adverse Event Reports
                    Appendix I: Scope and Methodology for
                    Categorizing the Call Records




                    of these records contained enough information to determine the nature of
                    the adverse event and were coded in the same way as other call records.
                    The other medical records were used as additional sources of information
                    for documenting the events and consumer information reported in their
                    corresponding records.

                    While most pages of call records contained information about a single call,
                    some included information about multiple calls on the same page, other
                    calls spanned multiple pages, and some did not include any report of
                    adverse events. Records that spanned multiple pages were often letters to
                    the company, some of which were sent with additional information (such
                    as medical bills). Records that did not report an adverse event were either
                    incomplete printouts of other records from the database, product
                    questions, complaints about not losing weight, or reports of consumer
                    satisfaction. As a result, the number of pages of call records that we
                    received from Metabolife International does not correspond to the number
                    of reports of adverse events.

                    The call records and medical records we received were redacted by
                    Metabolife International to remove personal identifying information such
                    as name, phone number, address, fax number, and e-mail address to
                    protect consumer privacy. Metabolife International officials told us that in
                    the process of redacting the records, some relevant adverse event
                    information was also inadvertently removed.


                    Metabolife International officials told us that there were duplicate call
Exclusion of        records in the set of call records they provided to us. Some duplicate
Duplicate and       reports were photocopies of the same call record. In other cases, there
                    were multiple versions of the same call record in different formats.
Nonhealth-Related   Metabolife International officials reported these multiple versions were
Call Records        the result of nurses taking handwritten notes and later entering the same
                    information directly into a database established in September 1999.

                    Metabolife International gave us lists of those call records it believed to be
                    duplicates. Over the course of our review, it identified more than
                    2,200 records for which there were at least one duplicate. Metabolife
                    International officials reported that they identified the duplicates on the
                    basis of the name of the consumer. Duplicates may have included
                    subsequent calls about different events from the same individual. We
                    examined the duplicate call records identified in the lists provided
                    throughout our review by Metabolife International. Because identifying
                    information was removed, we examined the date of the call record,


                    Page 19                                 GAO-03-494 Metabolife 356 Adverse Event Reports
                    Appendix I: Scope and Methodology for
                    Categorizing the Call Records




                    demographic information about the consumer (such as age, height, weight,
                    the amount of the product used, and duration of use), and event details to
                    determine if they were duplicate records. Where this information was the
                    same or similar, we considered the records to be duplicates and excluded
                    the extra records from our review. We did, however, include in our
                    analysis any additional information that appeared on the duplicate
                    records. For example, if one version included height and another weight,
                    we recorded both of these.

                    We agreed with Metabolife International that most of the more than
                    2,200 records it identified as duplicates were, in fact, duplicates. However,
                    we did not exclude records that represented multiple calls from the same
                    consumer for different events if the dates on the call records differed by
                    more than a few days or the symptoms were clearly different. During the
                    course of our review, we also identified duplicates not previously
                    identified by Metabolife International, including photocopied records and
                    records that used identical language in event descriptions. We do not
                    know if all duplicate call records were identified.

                    We also excluded from our analysis records in which there was no health
                    complaint or the health complaint could not be clearly determined. We
                    also excluded call records that reported third-hand knowledge of adverse
                    events (such as a friend of a friend who experienced an adverse event). In
                    addition, we did not count call records that clearly referred to nutrition
                    bars or other ephedra-free products manufactured by Metabolife
                    International. In total, we determined that the 15,948 pages of
                    documentation provided by Metabolife International contained
                    14,684 separate health-related call records.


                    We classified the adverse events reported in each call record and entered
Classification of   the appropriate codes into a database. We classified the reported adverse
Records and Data    events as either one of the events FDA identified as serious in its
                    February 28, 2003, announcement regarding a proposed label warning for
Entry Procedures    dietary supplements containing ephedra (heart attack, stroke, seizure, or
                    death) or as an other adverse event. All serious events reported within a
                    particular call record were counted. Therefore an individual could have
                    reported multiple serious adverse events, though this happened in few
                    records. For other adverse events, we documented whether the call record
                    reported one or more adverse events. We did not count the number of
                    reports for every type of event reported in the record. We did, however,
                    count the number of all but 1 of the 24 other types of adverse events that
                    were described as serious or potentially serious in FDA’s June 4, 1997,


                    Page 20                                 GAO-03-494 Metabolife 356 Adverse Event Reports
Appendix I: Scope and Methodology for
Categorizing the Call Records




proposed rule on dietary supplements containing ephedrine alkaloids.3 The
set of events identified by FDA in the proposed rule is not an exhaustive
list of the adverse events that may be associated with the use of dietary
supplements containing ephedrine alkaloids. FDA officials told us that
some other types of adverse events may be considered serious or
potentially serious but had not yet been reported to FDA during the time
period considered by its proposed rule.

We did not apply medical judgment in the process of identifying and
classifying events. Our classification of events in the call records was
based solely on the words and phrases therein; we did not diagnose a
consumer’s condition or otherwise interpret the information presented.
For example, if a report said “poss. heart attack,” “heart attack symptoms,”
or “heart attack?”, we did not classify it as a heart attack since it was not
clear that a heart attack was reported. Also, while we counted “blood
pressure 210/120” as an instance of significantly elevated blood pressure
because it reported measurements greater than 160 systolic or 100
diastolic, we did not place in the same category call records that reported
only “high blood pressure” because they did not contain the specific
measurements needed for that determination. We used MEDLINE Plus
Medical Encyclopedia definitions to further clarify individual symptoms
related to these categories.4 We also did not attempt to determine whether
Metabolife 356 caused the reported adverse events.




3
 FDA’s June 4, 1997, proposed rule identified serious or potentially serious adverse events
associated with the use of ephedra based on a review of the literature and an analysis of
600 adverse event reports that FDA had received by June 7, 1996. See “Dietary Supplements
Containing Ephedrine Alkaloids,” 62 Fed. Reg. 30678.
4
 www.nlm.nih.gov/medlineplus/encyclopedia.html (downloaded December 2002 through
February 2003).




Page 21                                 GAO-03-494 Metabolife 356 Adverse Event Reports
                                                       Appendix II: Metabolife 356 Label
Appendix II: Metabolife 356 Label




Source: Metabolife International, February 12, 2003.




                                                       Page 22                             GAO-03-494 Metabolife 356 Adverse Event Reports
                                           Appendix III: Requirements for Reporting
Appendix III: Requirements for Reporting   Adverse Events to FDA



Adverse Events to FDA

                                           Adverse events about many types of products regulated by FDA are
                                           required to be reported to the agency. Such products include human drugs,
                                           biologics, animal drugs, animal feeds containing animal drugs, medical
                                           devices, infant formulas, and radiation-emitting devices. There are,
                                           however, no reporting requirements for adverse events associated with
                                           other products regulated by FDA, including food and food additives,
                                           dietary supplements, cosmetics, or color additives. (See table 5.)

Table 5: Requirements for Reporting Adverse Events to FDA

                               Adverse events that must be
Product                        reported to FDA                       Who reports                          When reported
Human drugs (including over-   Serious and unexpected                NDA and ANDA applicants,             As soon as possible but within
the-counter drugs) with        adverse drug experiences from         and any person whose name is         15 calendar days. Nonapplicants
approved New Drug              all sources (domestic and             on the label of an approved          may, instead, submit reports to
                                         b
Application (NDA) or           foreign).                             drug as manufacturer, packer,        applicants within 5 calendar
Abbreviated New Drug                                                 or distributor (“ nonapplicants”).   days.
Application (ANDA)a
                               New information obtained as           Same as above.                       Within 15 calendar days of
                               result of follow-up investigation                                          receipt of new information or as
                               of earlier reports.                                                        requested by FDA.
                                                                                                          Nonapplicants may, instead,
                                                                                                          submit reports to applicants
                                                                                                          within 5 calendar days.
                               Adverse experiences that occur        NDA and ANDA applicants.             At quarterly intervals for the first
                               domestically and that are                                                  3 years after approval and then
                               serious and expected or not                                                annually or at different times
                               serious (expected or                                                       upon written notice by FDA.
                               unexpected).
                               Serious and unexpected                NDA and ANDA applicants and Within 15 calendar days.
                               adverse drug experiences              nonapplicants.
                               described in scientific literature
                               as case reports or as the result
                               of a formal clinical trial, or from
                               or during postmarketing studies
                               where the applicant concludes
                               that there is a reasonable
                               possibility that drug caused
                               reaction.b




                                           Page 23                                    GAO-03-494 Metabolife 356 Adverse Event Reports
                                   Appendix III: Requirements for Reporting
                                   Adverse Events to FDA




                        Adverse events that must be
Product                 reported to FDA                     Who reports                          When reported
Human drugs without     Serious and unexpected              Any person whose name is on          As soon as possible but within
                    c
approved NDAs/ANDAs     adverse drug experiences from       the label as a manufacturer,         15 calendar days; packers and
                        all sources (domestic and           packer, or distributor; and the      distributors may, instead, submit
                        foreign).b                          manufacturer even if its name        reports to manufacturers within
                                                            does not appear on the label,        5 calendar days.
                                                            when it receives adverse drug
                                                            experience reports directly
                                                            from a packer or distributor.

                        Serious and unexpected              Same as above.                       Same as above.
                        adverse drug experiences from
                        a postmarketing study where
                        there is reasonable possibility
                        that drug caused reaction.b
                        New information obtained as         Same as above.                       Within 15 calendar days of
                        result of follow-up investigation                                        obtaining the information or as
                        of 15-day alert reports.                                                 requested by FDA.
            d
Biologics               Serious and unexpected              Licensed manufacturerse and          As soon as possible but no later
                        adverse experiences from all        manufacturers, packers,              than 15 calendar days. Packers,
                        sources described in scientific     distributors, or other               distributors, and other
                        literature, or described in         manufacturing participants           nonlicensees required to report
                        postmarketing clinical studies      whose name appears on the            may submit reports to licensed
                        where there is a reasonable         label.                               manufacturers within 5 calendar
                        possibility product caused                                               days.
                        reaction.b
                        New information obtained as a       Same as above.                       Within 15 days of receipt of new
                        result of follow-up of 15-day                                            information or as requested by
                        alert reports.                                                           FDA. Packers, distributors, and
                                                                                                 other unlicensed firms required
                                                                                                 to report may submit reports to
                                                                                                 licensees within 5 calendar days.
                        Adverse experiences that are        Licensed manufacturers.              At quarterly intervals for the first
                        expected or nonserious.                                                  3 years after license approval
                                                                                                 and then annually or at different
                                                                                                 times upon written notice by
                                                                                                 FDA.
                        Certain reactions associated        Vaccine manufacturers and            Within 7 days of the
                        with administration of vaccines     health care providers.               administration of listed vaccines
                                                                                                                  f
                        listed in 42 U.S.C. §300aa-14.                                           or as specified.
                        Fatality resulting from blood       Collecting facilities in the event   As soon as possible by
                                                   g
                        collection or transfusion.          of donor reaction; facilities        telephone, facsimile, express
                                                            performing compatibility tests       mail, or electronic transmission
                                                            in the event of transfusion          with a written report to follow
                                                            reaction.                            within 7 days.




                                   Page 24                                   GAO-03-494 Metabolife 356 Adverse Event Reports
                                      Appendix III: Requirements for Reporting
                                      Adverse Events to FDA




                           Adverse events that must be
Product                    reported to FDA                     Who reports                       When reported
            h
Animal drugs               Unexpected side effects, injury,    Applicants for New Animal         As soon as possible but within
                           toxicity, sensitivity, reaction;    Drug Application (NADA) or        15 working days of receipt by the
                           unexpected incidence or             Abbreviated New Animal Drug       applicant.
                           severity, or unusual failure to     Application (ANADA), including
                           exhibit expected                    those whose name appears on
                           pharmacological activities.         the labeling as a manufacturer,
                                                               packer, distributor, or who are
                                                               engaged in manufacturing,
                                                               processing, packing, or
                                                               labeling of drug.
                           Mix-up in new animal drug or its Same as above.                       Immediately (generally within
                           labeling with another article,                                        3 days).
                           bacteriological or significant
                           physical or other change or
                           deterioration in the drug, or
                           failure to meet specifications.
Animal feeds bearing or    Mix-up with another drug or its     NADA and ANADA applicants.        Immediately (generally within
containing animal drugsi   labeling with another article;                                        3 days).
                           bacteriological or significant
                           chemical, physical, or other
                           change or deterioration in the
                           drug; or failure to meet
                           specifications.
                           Information concerning              Same as above.                    As soon as possible but within
                           unexpected side effect, injury,                                       15 working days of receipt by the
                           toxicity, sensitivity reaction, any                                   applicant.
                           unexpected incidences or
                           severity, or unusual failure to
                           exhibit expected
                           pharmacological activities.
                j                                                                     k
Medical devices            Device-related deaths or            Device user facilities.           Within 10 work days of receiving
                           serious injuries.                                                     relevant information; annual
                                                                                                 reports must summarize all
                                                                                                 reported events.
                           Device-related deaths or           Importers.                         Within 30 days of becoming
                           serious injuries.                                                     aware of event.
                           Device-related deaths or           Device manufacturers.              Within 30 days of becoming
                                            l
                           serious injuries.                                                     aware of event, or within 5 days
                                                                                                 if the event requires remedial
                                                                                                 action to prevent an
                                                                                                 unreasonable risk of substantial
                                                                                                 harm to the public health or if
                                                                                                 FDA has made a written request.
                           Information that would have had Same as above.                        Within 1 month of receiving
                           to have been reported earlier                                         information.
                           but was unknown or
                           unavailable.




                                      Page 25                                 GAO-03-494 Metabolife 356 Adverse Event Reports
                                             Appendix III: Requirements for Reporting
                                             Adverse Events to FDA




                                 Adverse events that must be
Product                          reported to FDA                            Who reports                       When reported
               m
Infant formula                   Possible causal connection                 Manufacturers.                    Within 15 days, conduct an
                                 between consumption of an                                                    investigation and notify FDA.
                                 infant formula and infant death.
                             n
Radiation-emitting devices       Injurious or potentially injurious         Manufacturers.                    Immediately.
                                 exposure to radiation from
                                 nonmedical electronic
                                           o
                                 products.
Food and food additives          No requirements to report                  Not applicable.                   Not applicable.
                                 adverse events.
Dietary supplements              No requirements to report                  Not applicable.                   Not applicable.
                                 adverse events.
Cosmetics                        No requirements to report                  Not applicable.                   Not applicable.
                                 adverse events.
Color additives                  No requirements to report                  Not applicable.                   Not applicable.
                                 adverse events.

                                             a
                                              21 C.F.R. §§ 314.80, 314.98 (2002). Over-the-counter drugs are subject to FDA’s adverse event
                                             reporting requirements only to the extent they are covered by approved NDAs or ANDAs. On March
                                             14, 2003, FDA published a proposed rule which includes requirements for reporting suspected
                                             adverse events associated with drugs and biological products (“Safety Reporting Requirements for
                                             Human Drug and Biological Products,” 68 Fed. Reg. 12406).
                                             b
                                                 FDA refers to these as 15-day alert reports.
                                             c
                                              21 C.F.R. § 310.305. Adverse events associated with investigational new drugs are required to be
                                             reported under sections 312.32 and 312.33 of Title 21 of the Code of Federal Regulations. Also see
                                             FDA’s proposed rule at 68 Fed. Reg. 12406 (Mar. 14, 2003).
                                             d
                                              21 C.F.R. § 600.80. There are no reporting requirements for manufacturers of whole blood or
                                             components of whole blood. 21 C.F.R. § 600.80(k)(1). Also see FDA’s proposed rule at 68
                                             Fed. Reg. 12406 (Mar. 14, 2003).
                                             e
                                              In vitro diagnostic products are subject to the reporting requirements for devices.
                                             21 C.F.R. § 600.80(k)(2).
                                             f
                                                 42 U.S.C. § 300aa-25(b).
                                             g
                                                 21 C.F.R. § 606.170(b).
                                             h
                                             21 C.F.R. § 510.300. FDA is in the process of redrafting the adverse event reporting rules for
                                             approved animal drugs.
                                             i
                                             21 C.F.R. § 510.301. Certain medicated items incorporated into animal feeds are also subject to the
                                             animal drug reporting requirements. See 21 C.F.R. § 514.80(a)(4).
                                             j
                                             21 C.F.R. pt. 803. Not all medical device adverse events must be reported to FDA; user facilities are
                                             required to report serious injuries to FDA only if the manufacturers are not known. 21 C.F.R. §
                                             803.30(a)(2). Adverse events associated with devices under Investigational Device Exemptions must
                                             be reported and summaries must be included in applications submitted to FDA for premarket
                                             approval. 21 C.F.R. §§ 812.150, 814.20.
                                             k
                                             Device user facilities do not include physician offices, school nurse offices, and employee health
                                             units. 21 C.F.R. § 803.3(f).
                                             l
                                             Manufacturers must also report to FDA if a device has malfunctioned and such malfunction, were it to
                                             recur, would be likely to cause or contribute to a death or serious injury. 21 C.F.R. § 803.50(a)(2).




                                             Page 26                                          GAO-03-494 Metabolife 356 Adverse Event Reports
Appendix III: Requirements for Reporting
Adverse Events to FDA




m
 21 C.F.R. § 106.100(k)(3). Manufacturers must promptly report to FDA knowledge about an infant
formula it has processed and that has left its establishment if the infant formula may be adulterated or
misbranded and that may present a risk to human health. 21 C.F.R. § 106.120(b).
n
21 C.F.R. §1002.20.
o
 21 C.F.R. §§ 1000.3, 1002.20. Nonmedical electronic products include, for example, microwave
ovens and infrared alarm systems. If a product is classified as a medical device, the normal medical
device reporting requirements apply.




Page 27                                     GAO-03-494 Metabolife 356 Adverse Event Reports
             Appendix IV: Comments from the Food and Drug Administration
Appendix IV: Comments from the Food and
Drug Administration




             Page 28                             GAO-03-494 Metabolife 356 Adverse Event Reports
Appendix IV: Comments from the Food and Drug Administration




Page 29                             GAO-03-494 Metabolife 356 Adverse Event Reports
                  Appendix V: GAO Contact and Staff
Appendix V: GAO Contact and Staff
                  Acknowledgments



Acknowledgments

                  Martin T. Gahart, (202) 512-3596
GAO Contact

                  Carolyn Feis Korman, Chad Davenport, Julian Klazkin, and Roseanne
Acknowledgments   Price also made major contributions to this report.




(290236)
                  Page 30                             GAO-03-494 Metabolife 356 Adverse Event Reports
                         The General Accounting Office, the audit, evaluation and investigative arm of
GAO’s Mission            Congress, exists to support Congress in meeting its constitutional responsibilities
                         and to help improve the performance and accountability of the federal
                         government for the American people. GAO examines the use of public funds;
                         evaluates federal programs and policies; and provides analyses,
                         recommendations, and other assistance to help Congress make informed
                         oversight, policy, and funding decisions. GAO’s commitment to good government
                         is reflected in its core values of accountability, integrity, and reliability.


                         The fastest and easiest way to obtain copies of GAO documents at no cost is
Obtaining Copies of      through the Internet. GAO’s Web site (www.gao.gov) contains abstracts and full-
GAO Reports and          text files of current reports and testimony and an expanding archive of older
                         products. The Web site features a search engine to help you locate documents
Testimony                using key words and phrases. You can print these documents in their entirety,
                         including charts and other graphics.
                         Each day, GAO issues a list of newly released reports, testimony, and
                         correspondence. GAO posts this list, known as “Today’s Reports,” on its Web site
                         daily. The list contains links to the full-text document files. To have GAO e-mail
                         this list to you every afternoon, go to www.gao.gov and select “Subscribe to daily
                         E-mail alert for newly released products” under the GAO Reports heading.


Order by Mail or Phone   The first copy of each printed report is free. Additional copies are $2 each. A
                         check or money order should be made out to the Superintendent of Documents.
                         GAO also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a
                         single address are discounted 25 percent. Orders should be sent to:
                         U.S. General Accounting Office
                         441 G Street NW, Room LM
                         Washington, D.C. 20548
                         To order by Phone:     Voice:    (202) 512-6000
                                                TDD:      (202) 512-2537
                                                Fax:      (202) 512-6061


                         Contact:
To Report Fraud,
                         Web site: www.gao.gov/fraudnet/fraudnet.htm
Waste, and Abuse in      E-mail: fraudnet@gao.gov
Federal Programs         Automated answering system: (800) 424-5454 or (202) 512-7470


                         Jeff Nelligan, managing director, NelliganJ@gao.gov (202) 512-4800
Public Affairs           U.S. General Accounting Office, 441 G Street NW, Room 7149
                         Washington, D.C. 20548