oversight

Data Tampering / Sabotage / Fabrication

Published by the National Science Foundation, Office of Inspector General on 1999-09-30.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                              Closeout for M96090025

   In carrying out his research, the subject1 collected samples, treated them in his
laboratory, and then sent them to testing facilities for analysis. In September 1996,
we received a manuscript2 that described the authors' analysis of, and their
unsuccessful efforts to reproduce, the subject's data. We were subsequently
contacted by the complainants,3 who alleged that the subject falsified data, i.e., he
added material to his samples to pre-determine the results the facilities would
obtain when his samples were tested. The complainants further alleged that the
subject had become aware of the authors' re-testing of his samples, and
consequently was attempting to retrieve and destroy his samples from the facilities.
Therefore, we issued a subpoena for, and obtained, the remaining samples from the
authors.
   The subject became aware of the allegation and contacted us, denying he had
tampered with his samples. He also raised many technical issues related both to
the authors' method of analysis and to his own analytical techniques. The subject
argued that this was a scientific dispute over analytical methods and the
interpretation of the resulting data, and not a case of misconduct in science.
    The authors contacted us and indicated there had been a contamination problem
during their testing of some of the subject's samples. They wanted to re-test some
of the samples to verify their earlier results and ensure the complainants' allegation
was supported by the results. The results of their re-testing confirmed their earlier
findings, which they argued supported the complainants' allegation against the
subject.
   Before we could assess whether the allegation constituted possible misconduct,
or would more properly be characterized as a scientific dispute, the subject's
University informed us that it had become aware of the allegation and was taking
steps to investigate it. We discussed the matter with a University administrator4
and agreed to defer our inquiry pending completion of the University's inquiry. The
University asked two independent scientists to learn the subject's methodology and
attempt to reproduce his results. Following its inquiry, the University concluded
there was enough substance to the allegation to proceed to a n investigation, and we
agree to defer our investigation while the University conducted its investigation.
   The University's investigation Committee wanted to test some of the subject's
samples that we had in our possession. We transferred of some samples to a
scientist that the Committee designated. Based on the results of these tests, the
Committee requested an extension to carry out tests on additional samples and to

     (footnote redacted).
   2 (footnote redacted).
   3 (footnote redacted).
   4 (footnote redacted).




                                      Page 1 of 4
                               Closeout f o r M96090025



complete the preparation of the final investigation report. The Committee also met
with the subject to obtain his explanation for the test results and discuss the draft
of their report. The subject responded to their draft and answered their questions
about the test results.
    The University's Adjudicator5 provided us with a cover letter, the report, and
attachments. The Committee assessed the allegation by four main criteria:
(i) analysis of the material in a selection of the subject's samples; (ii) analysis of the
plausibility and replicability of the subject's methodology; (iii) comparison of the
results obtained from the subject's samples with known "control" data; and (iv) the
relative timing of the allegation about his data and the subject's questioning of his
own results.
    Regarding the nature of the material in the subject's samples (criterion i), the
Committee report described the results of the two scientists who worked with the
subject to learn his collection method during the inquiry, as well as the additional
tests it requested. Because of the differences in the methodologies of the two
scientists and the subject, we agree with the Committee's conclusion that their
inability to reproduce the subject's results did not substantiate the allegation. In
the first round of tests the Committee requested, one sample showed the presence of
a n anomalous material (AM). The material was anomalous because it is not known
to be naturally occurring in the location where the sample was collected. Based on
the presence of the AM, the Committee requested another round of tests on samples
specifically chosen such that the AM would not be expected in those samples. Those
results showed that half of the samples contained the AM, while half did not-and
the half that contained the AM were from the same location a s the AM-containing
sample from the first round test. The subject arranged for a test of his samples
from a different laboratory, and those results did not show the presence of the AM.
The subject raised concerns about the chain of custody of the samples and suggested
that some samples may have been inadvertently contaminated. Although the
Committee noted "the contamination explanation cannot be discounted," it
concluded that intentional alteration of some samples by the subject "seems a
slightly more likely explanation." In our view, the evidence of alteration adduced by
the Committee must be considered in light of the substantial chain of custody issues
raised by the subject and the Committee's conclusion (discussed below) that there
was no evidence that the subject's samples were altered so a s to yield particular
results. We therefore, agree with the Committee's conclusion that the evidence does
not support the allegation.
   The Committee analyzed the plausibility and replicability of the subject's
methodology (criterion ii). It noted that the subject's methodology has changed
"drastically" over the years, that his techniques are difficult to master, and two
independent attempts to duplicate the subjects results were unsuccessful.

   5   (footnote redacted).


                                        Page 2 of 4
                              Closeout for M96090025



Although, it thought the examples cited by the subject were not conclusive, it found
some evidence supporting the subject's results. We agree with the Committee's
conclusion that the ambiguities and uncertainties in the subject's methodology do
not support the allegation.
    The Committee evaluated the likelihood that the subject was trying to
manipulate his data to obtain a pre-determined result (criterion iii). A scientists
had provided the Committee with a comparison of the subject's test results with
known "control" results. This scientist's analysis showed remarkable agreement
between the two sets of results, and he suggested that such close agreement could
be obtained only if the subject had manipulated his samples in such a way a s to
match the "control" result. In response, the subject provided the Committee with
his own analysis of his data with the "control" data. The subject's analysis showed
substantially less agreement, and he suggested that the lack of uniformity was
indicative of honest research results. Additionally, the subject pointed out that his
data included samples in which his results were practically impossible to reconcile
with the "control" data, yielding results so obviously wrong they would not have
been intentionally fabricated. The subject also provided the Committee with
scientific reasons why some of his results should show good agreement with the
"control" data and others would not. He explained that some of these problems
were scientific in nature and were of current interest, and debated within, the
relevant literature. The Committee's analysis showed that both the scientist's and
the subject's representations of the data had inaccuracies. The Committee learned
that there was ambiguity associated with the "control" data itself. Ultimately, it
concluded that such an analysis was not especially useful because it should
expected that the subject's method could be expected to work some of the time and,
alternatively, for those results in which the subject's method didn't work, it did not
mean no misconduct took place. We agree with the Committee's conclusion that
there existed convincing scientific explanations for the agreements between the
subject's data and the "control" data.
   With regard to when the subject began to repudiate his results (before or after
the allegation was made--criterion iv), the Committee noted the subject now
disavows his data (stating that it is unreliable due to the presence of different
materials), and there was some evidence that he began to do so before the allegation
was made. Accordingly, we agree with the Committee's conclusion that the relative
timing of the subject's repudiation of his previous research results does not support
the allegation.
   To summarize, the Committee concluded that a preponderance of the all the
evidence did not support the allegation made against the subject. The Adjudicator's
c6ver letter stated that he reviewed the Committee's report and concurred with its


   6   (footnote redacted).


                                      Page 3 of 4
                              Closeout for M96090025



conclusion. He said the University would take reasonable steps to counter the
damage to the subject's professional reputation.
   We found the Committee's report to be objective and thorough, and we concur
with its conclusions concerning the allegation of misconduct in science. Although
the Committee concluded that some samples seemed likely to have been altered by
the subject (criterion i), there was no evidence that the samples were altered to
achieve a pre-determined result (criterion iii)-which in our view (consistent with
the Committee's conclusions) precludes a finding of culpable action necessary to
conclude that any such alterations constituted a serious deviation from accepted
practices. Therefore, this investigation is closed and no further action will be taken
on this case.
    Many of the Committee's efforts went well beyond investigating the allegation of
misconduct, and were similar to those that a scientist carrying out research in this
field would take to verify the subject's reported results. Indeed, the Committee
went so far as to map out future research that could be conducted to test the
reliability of the subject's results. We think any remaining issues are best resolved
by the community in which the subject practices. While members of the subject's
research community may find it disturbing that no one has completely duplicated
the subject's results, we also note that the subject has himself repudiated them and
the methods he used to obtain them. Issues such a s replicability, sample analysis,
reliability of control dates, and the amount and depth of scientific detail in
publications, are at the heart of the scientific enterprise and are-as they should
be-addressed by the scientific community in the ordinary practice of research.

cc: Integrity, IG




                                      Page 4 of 4