NSF Procedures/Errors/Reconsiderations

Published by the National Science Foundation, Office of Inspector General on 1999-09-30.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                                   Closeout for M98120032

    The complainant met with us on October 26, 1998, to discuss information he
received from NSF about the declination of his proposal. Because he had received
two different explanations of how reviewers were chosen to evaluate his proposal,
he alleged that the Program Officer (P0)l had, either on her own or acting on behalf
of her Program Leader (PL),2intentionally biased the review process against his
proposal by selecting reviewers predisposed to decline his proposal. Because we
were unsure if the complainant's concerns were more properly addressed by us or
NSF management, we reviewed the documents the complainant provided, as well a s
others, before deciding to begin our inquiry. Based on our review, we opened a case
on December 2, 1998.
    The complainant submitted a pre-proposal a s the first step in a multi-stage
process involving the evaluation of a pre-proposal and, contingent upon a
satisfactory review, a full proposal and a site visit. The complainant's pre-proposal
ranked well enough (receiving two Excellent ratings among the five individual
ratings) in the review process to make it to the next step (submission of a full
proposal). The complainant's full proposal did not fare as well and was not selected
for the next stage (a site visit). The complainant wondered if the two reviewers who
had rated his pre-proposal as excellent were deliberately not used to evaluate his
full proposal and asked the PO how the proposal reviewers were selected.
   The PO told us she discussed the complainant's question and the selection
process with the PL. The PL sent the complainant a n e-mail stating that all pre-
proposal panelists were asked to review the full proposal, and those who agreed to
review it, did. The PL told us the e-mail was meant to reflect general policy
guidance for the review process, rather than being specific to one proposal.
   The complainant requested a reconsideration3 of his proposal, which was
handled by the designated reconsideration official (AD), who consulted with a senior
Program Officer (SPO).4 After talking to the SPO in the course of the
reconsideration, the PO wrote a diary note explaining her selection of reviewers.
The content of that diary note was consistent with what the AD wrote to the
complainant in response to his request for further explanation about how the
proposal reviewers were chosen. In that letter, the AD told the complainant that
the reviewers were selected to review the full proposals on the basis of availability
and expertise. Regarding the two reviewers who gave the pre-proposal an excellent,

    1 (footnote redacted).
   2  (footnote redacted).
    3 A reconsideration can be requested by a PI when hislher proposal has been declined, and the PI
believes NSF's standard procedure has not been followed or a serious mistake has been made. It is a
independent evaluation carried out by the Assistant Director of the directorate in whlch the proposal
was submitted, and assesses the substantive and procedural decisions related to the proposal.
      (footnote redacted).

                                             Page 1 of 3                                     M98-32
                                   Closeout of M98120032

the AD told the complainant one reviewer was unable to review anyof the full
proposals (Reviewer I), and the other asked to review only one (Reviewer 2). The
AD explained that Reviewer 2's expertise was most relevant to another proposal.
    We spoke to the five pre-proposal reviewers (four of whom also provided reviews
of the invited full proposals as well). All said they were not subject to any pressure
or bias, either from NSF or otherwise, to rate any of the pre-proposals or full
proposals either positively or negatively. All reviewers stated t h a t the evaluation
and ratings they gave all the pre-proposals and full proposals were based on their
own opinions. Regarding availability to review full proposals, Reviewer 1confirmed
that he had told the PO that he was too busy to review any of them. Reviewer 2,
who reviewed only one full proposal, told u s he was asked to review only one full
proposal, but he had not requested to review only one, and he could have reviewed
another if asked. (In fact, none of the panelists indicated t h a t they had any
constraints such t h a t they asked to review only one proposal.) Reviewer 1
understood from the PO that the PO was giving him t h a t proposal to review because
it was in his area of expertise. The remaining pre-proposal reviewers each reviewed
two full proposals.
     We interviewed the PO to ask about the discrepancies among (1)the statements
made i n the PL's e-mail, (2) the PO'S diary note and the AD'S letter, and (3) the
reviewers' accounts. She told us there was a discrepancy between what she told the
PL and what was in the PL's e-mail, i n t h a t her description to the PL was broader
t h a n the e-mail reflected.
    The contradiction between the PO'S statements and the statements of the
reviewers was in whether the reviewers were specifically asked about their
availability. The PO stated t h a t she found out t h a t Reviewer 2 had administrative
duties and t h a t he requested to review only one proposal. We could not resolve the
discrepancy between the reviewers' and PO'S recollections. Regardless, there is no
evidence t h a t she selected the reviewers to obtain a particular result, or that her PL
influenced her selection of reviewers. NSF guidelines for choosing reviewers afford
Program Officers latitude and flexibility in their choice of individual reviewers.5 We
found no evidence that the PO'S actions were inconsistent with the guidelines.
Based on the information we have, a preponderance of the evidence does not
support the allegation. This inquiry is closed and no further action will be taken on
this case.
   We note t h a t following the reconsideration, t h e complainant (through his
attorney) wrote to NSF to complain about the reconsideration and review processes
and to claim bias. NSF's Office of General Counsel (OGC) responded that (1)the

    5 The Proposal a n d Award Manual is a compendium of NSF internal proposal and award policies
and procedures intended for internal NSF staff use. It replaces NSF Circulars, Bulletins, and Staff
Memoranda directly related to the NSF proposal and award system. The relevant section is
"Selection of Ad Hoc ReviewersJ'-122.4.

                                            Page 2 of 3
                                   Closeout of M98120032

next step in NSF's reconsideration process required the complainant's University to
join his request, and it declined to do so6; and (2) the reconsideration and review
processes appeared to have been carried out in agreement with NSF's policies. As
described above, our conclusion about the review of the complainant's proposal is
consistent with OGC's.

cc: Integrity, IG

    6 The President of the complainant's University notified OGC that it accepted NSF's decision on
the complainant's proposal as final.

                                            Page 3 of 3