oversight

OIG Review of Institutions' Implementation of NSF's Responsible Conduct of Research Requirements

Published by the National Science Foundation, Office of Inspector General on 2017-07-25.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

National Science Foundation
Office of Inspector General




   OIG Review of Institutions’ Implementation
   of NSF’s Responsible Conduct of Research
                Requirements




July 25, 2017
OIG Tracking No. PR12030006
Executive Summary
In 2007, Congress passed the America Creating Opportunities to Meaningfully Promote
Excellence in Technology, Education, and Science Act (America COMPETES Act), 1 which among
other things directed NSF to introduce a requirement for awardees to provide adequate training for
undergraduate students, graduate students, and postdoctoral researchers about the Responsible
Conduct of Research (RCR). In 2013, 3 years after NSF’s implementation of the Act, we contacted
53 institutions to learn how they had implemented their own RCR training in response to NSF’s
requirement. We requested and reviewed institutions’ RCR plans and asked a variety of questions
of institutional officials and RCR trainees about the institutional plans, such as who gets trained,
the format of the training, and the training’s content.
This report is divided into two parts. The first answers the key compliance questions related to
NSF’s policy, e.g., did the institutions in our sample have a plan, did they designate a person to
oversee compliance, and can the institutions verify that the necessary people are being trained. The
second contains observations from our fieldwork for NSF’s consideration, including opportunities
to strengthen its RCR policy.


Background
The scientific enterprise is based on a foundation of trust. If the trust is found to have been
misplaced as a result of unethical or unprofessional conduct on the part of scientists, the impact of
that breakdown is not limited to the research community alone — it can undermine the relationship
between science and society as a whole. 2 For this reason, the National Academy of Sciences has
long recognized the importance of training scientists in RCR. Within the Federal Government, the
National Institutes of Health (NIH) implemented a requirement for its awardees to provide RCR
training in 1989; the US Department of Agriculture (USDA) enacted a similar requirement in 2013.
NSF implemented its RCR requirement in 2010. 3
NSF’s RCR requirement grew out of a provision in the America COMPETES Act of 2007, which
directed NSF to require that each institution that applied for financial assistance for science and
engineering research or education describe in its grant proposal a plan to provide appropriate
training and oversight in the responsible and ethical conduct of research to undergraduate students,
graduate students, and postdoctoral researchers (postdocs) participating in the proposed research
project. Given the differing needs of the students and researchers covered by the new requirement,
in report language accompanying the Act, the drafters gave NSF “maximum flexibility in
determining the full range of activities that would constitute appropriate training.” 4 In the
following sentence, the drafters made clear, however, that they expected NSF to promptly develop
and provide written guidelines and/or templates for universities to follow so their compliance could
be verified by all parties.



1
  America Creating Opportunities to Meaningfully Promote Excellence in Technology, Education, and Science Act
(codified as 42 USC 1862o-1)
2
  On Being A Scientist: A Guide to Responsible Conduct in Research: Third Edition (2009), p. ix.
3
  Responsible Conduct of Research (RCR), January 4, 2010,
 http://www.nsf.gov/pubs/policydocs/pappguide/nsf16001/aag_4.jsp#IVB
4
  House Report 110-289, p. 184, 2007

                                                                                                           2
NSF developed its RCR policy through a multi-stage process in which the agency obtained and
considered input from internal working groups; a workshop sponsored through the National
Academies of Sciences, Engineering, and Medicine; and comments from the public through the
Federal Register. When enacted in 2010, the policy established the following requirements for
institutions seeking NSF research funding:
    •   The institutions must have a plan in place to provide appropriate training and oversight in
        the responsible and ethical conduct of research to undergraduates, graduate students, and
        postdocs who will be supported by NSF to conduct research. Institutional certification of
        compliance with this responsibility is required for each proposal submitted to NSF. While
        institutions are not required to include training plans in their proposals, those plans are
        subject to review upon request.
    •   The institutions must designate one or more persons to oversee compliance with the RCR
        training requirement.
    •   The institutions are responsible for verifying that the students and researchers supported
        by NSF to conduct research have received training in the responsible and ethical conduct
        of research.
To date, NSF has not provided written guidelines or templates for universities to follow, as
requested in 2007 by the America COMPETES Act report language.


This review
We conducted this review to examine compliance in a sample of institutions with the three
requirements of the NSF RCR policy and to evaluate whether the lack of guidance and templates
might be having an effect on such compliance. We are also sharing observations we made during
the course of our work with the hope that NSF will use this information to strengthen its
implementation.
To make these assessments, we selected 53 institutions that requested NSF funding in 2013 for
undergraduates, graduate students, and/or postdocs. Our sample consisted of small (fewer than
5,000 students), medium (between 5,000-15,000 students), and large (over 15,000 students)
institutions in 24 different states, and contained public and private colleges and universities as well
as community colleges. At each institution we sought to interview a senior administrator who sets
the tone at the top regarding the importance of RCR, the designated RCR administrator, and
students/postdocs who had completed the required training. Because we spoke to more than nine
institutions, we sought and received OMB approval for our review. 5


Results of Compliance Testing
We found that almost one quarter of the institutions in our sample were not in compliance with
NSF’s RCR requirements at the time they received our engagement letter. Specifically, 23 percent
(11 out of 48) of institutions did not initially have an RCR plan, or, by default, a designated person
to oversee the plan or verifying and tracking that the required participants took the training. While
8 out of the 11 such institutions developed a plan after being contacted by our office, the level of
5
 For additional details on our methodology, see the Appendix—Methodology on page 14 of this report and OMB,
Control number 3145-0227.

                                                                                                         3
noncompliance raises a question as to whether institutions are uniformly and successfully
implementing NSF’s RCR policy. Our specific findings for each requirement are set forth below.


Does the institution have an RCR plan?
About 30 percent (16 out of 53) of the institutions in our sample did not have an RCR plan when
we first contacted them. As part of our fieldwork, we sought to determine why those 16 institutions
did not have an RCR plan. Some of the institutions said they did not have a plan because they only
received education funding from NSF. We found that although the America COMPETES Act
training requirement applies to institutions that apply for funding for science and engineering
research and for education, NSF’s RCR policy applies only to institutions that receive research
funding. 6 Working under the assumption that some grants from the NSF Directorate for Education
and Human Resources would qualify as funding for education (and not research) purposes, 5 of
the 16 institutions that did not have plans were community colleges that informed us they received
only education funding. Under NSF’s policy, such entities were not required to have an RCR plan;
accordingly, we eliminated those entities from our sample, reducing the number of institutions
from 53 to 48. This left us with a total of 11 institutions that did not have plans (2 large and 9
small), which amounts to almost 23 percent of our sample of 48. The fact that there were so many
noncompliant institutions in the group we examined indicates that NSF may have an
implementation problem with this requirement.
The number in and out of compliance for the three sizes of institutions we examined, after
removing the community colleges, is shown in Figure 1.

Figure 1: Institutions with RCR Plans when first contacted by OIG

                                      Institution RCR Plans

                          Large                18                     2
     Institution Size




                        Medium        11            0



                          Small   8                     9



                                       Compliant    Non-Compliant

Source: NSF-OIG generated based on data provided by institutional participants

Of the 11 institutions that did not have an RCR plan when we first contacted them, 8 developed a
plan after receiving our engagement letter. Of the remaining three, one developed a draft plan but

6
    We note that both NIH and USDA require RCR training for recipients of their education grants.

                                                                                                    4
never formally instituted it, one chose not to provide its plan despite the requirement to do so upon
request, and one does not consider its plan a formal RCR plan (the RCR training was determined
on an ad hoc basis). Adding the 8 institutions that developed and shared copies of their plans to
the 37 that originally had plans, a total of 45 institutions in our sample of 48 eventually had an
RCR plan (94 percent).

Has the institution designated a person to oversee compliance with the RCR requirement?
By default, institutions that did not have a plan would not have anyone designated to oversee
compliance with the RCR requirement. Therefore 11 out of 48 institutions were initially not in
compliance with this requirement. We found that each of the 37 institutions that had an RCR plan
at the start of our review had designated an individual to track compliance with the NSF RCR
training requirement. Of the eight institutions that developed a plan after receiving our engagement
letter, seven had designated such an individual. Ultimately, only 1 of the 45 institutions in our
sample that either had a plan at the start of our review or developed one after our review began
had not designated an individual to oversee compliance with this requirement. One institution that
did not have a formal plan told us that because it only occasionally receives NSF research grants,
the Principal Investigator (PI) of the awards it receives performs this function.


Can the institution verify that the necessary people are being trained?
Again, by default, institutions that did not have a plan could not verify that the necessary people
were being trained. Therefore 11 out of 48 were initially not in compliance with this requirement.
Of the 37 institutions that had an RCR plan at the start of our review, 30 told us they had sufficient
tracking in place to ensure that students received required RCR training. Of the eight institutions
that developed a plan after receiving our engagement letter, seven told us they had an adequate
tracking system. At the end of our fieldwork, then, approximately 18 percent (8 out of 45) of the
institutions in our sample that either had a plan at the start of our review or developed one after
our review began could not tell if their students were receiving required RCR training. The fact
that there were so many noncompliant institutions in the group we examined indicates that NSF
may have an implementation problem with this tracking requirement.


Observations made during interviews
During the course of our review, we made many observations about the nature of the RCR training
being provided in response to the America COMPETES Act. We detail those observations below
to provide insights to NSF about how institutions are responding to this requirement. We hope that
NSF will use this information to identify ways it can strengthen implementation of this important
requirement.


1. The institutions we reviewed utilized a wide variety of training approaches and formats.
Some institutions only train NSF-supported participants; others include all students, regardless of
their funding source. Some institutions provide different training based on participants’ status or
discipline, while others provide the same training to all. Some schools provide instruction in-

                                                                                                    5
person and online; others only provide online training. We summarize below the variety of
approaches and formats we found. For purposes of this assessment, if institutions had different
training requirements for undergraduates and graduate students, our results reflect how the
institution trained graduate students. 7


Differences in training provided based on status and/or discipline: We found that some institutions
provide the same RCR training to all students, irrespective of their educational level, i.e.,
undergraduate or graduate student or postdoc, which we characterize as a uniform plan. 8 Other
institutions provide different training to participants based on their educational level, which we
characterize as a differentiated plan. In our sample, excluding those institutions that have only
undergraduates, we found a nearly even split for each approach (52 percent differentiate; 48
percent do not). This result differs somewhat from a recent survey (the Study) 9 that found 69
percent of the 91 entities studied had uniform requirements, while 30 percent used differentiated
requirements based on educational level.
Of the institutions that differentiate and use online training, many did so by requiring graduates
and/or postdocs to take more modules of online training than were required of undergraduates. The
institutions that differentiate and require interactive training typically require undergraduates to
take an online course, but require graduate students to take an in-person course.
A majority of institutions in our sample considered students’/researchers’ disciplines when
deciding how to train them on RCR. Approximately 72 percent of the institutions we sampled told
us they provided RCR training based on participant discipline. The primary method of
differentiation in such cases was the use of different online modules for different disciplines. A
smaller number of respondents indicated that some departments required participants to attend
additional seminars or other interactive training to supplement online training. Several of the larger
research institutions have unique differentiated training, in that the schools within the institution
create customized, interactive training for their respective departments.


Differences in training formats: The institutions we queried also used various formats to provide
the required training. Many provided all their RCR training online. The most frequently cited
benefits of such training were that it could be completed at the participants’ convenience, it
introduced common terms and language, and it provided an easy way to track completion of the
training and compliance with RCR training requirements. However, participants who only took

7
  The rationale for this approach was that undergraduates’ primary responsibility is to take courses and typically
participate in research in the summer, or in limited amounts during the academic year. Graduate students and postdocs,
however, are more involved in research for longer periods of time and have greater flexibility in their schedules.
Furthermore, graduate students and postdocs are more probable current and future participants in the research
community. They have a greater need to be educated about the responsible conduct in, and expectations of, the
community of which they are current or future members.
8
  We use these terms and definitions taken from a recent, scholarly survey of the RCR plans of high-research-activity
institutions. T. Phillips, F. Nestor, G. Beach, and E. Holmes, “America COMPETES at 5 Years: An Analysis of RCR
Training Plans,” presented at the 25th Annual Meeting of the Association for Practical and Professional Ethics,
February 2016, and at the NSF workshop on “Responsible Conduct of Research” hosted by SBE and CCE STEM in
April 2016. We refer to this research as “the Study.”
9
  Ibid.—the Study

                                                                                                                    6
online RCR training told us they found it provided mostly common-sense advice and/or advice
that was repetitive, not applicable to their research, and/or too basic and generalized. Some of the
online-trained participants we interviewed indicated that they did not like the online training
format because they did not have the opportunity to ask questions about what they were learning
or discuss the content being presented. A particularly consistent complaint was the fact that the
online format did not provide them with an opportunity to discuss case studies.
Summary results: In Table 1, we summarize our findings about the different training formats being
utilized in RCR courses students were required to take, as well as the extent to which RCR training
was provided beyond NSF-supported participants. The table reflects definitions we created based
on the courses participants were required to take, not ones students had the option of taking. Some
institutions in our review provided various options for RCR training, but let students choose which
option to take. Awardees that provided various course options often did not track optional
participation in those courses, for example, allowing anyone who was interested in online training
to take it.

Table 1: Required RCR Training Formats and Participants
                                          Required trainee population is Required trainee population is
                                          limited to NSF-supported       not limited to NSF-supported
                                          participants                   participants
 Trainee population is able to
 fulfill the RCR requirement              64% (30/47)                             9% (4/47)
 by only taking online training
 or through document review
 Trainee population receives
 RCR content through                      9% (4/47)                               19% (9/47)
 required interactive training
 (i.e., a course, workshop or
 seminar)
Source: NSF-OIG assessment of participants’ data
Note: Numbers add to greater than 100 percent due to rounding. As previously discussed, there were 45 (out of
48) institutions that eventually had an RCR plan. There were 2 additional institutions that required RCR training,
but did not have a formal plan, bringing the total number of institutions that provided training to 47.

We found that most of the institutions in our sample (34 out of 47, or approximately 73 percent)
that required students to take RCR training allowed trainees to complete all or most of their
required RCR training online. 10 Indeed, only one institution in our sample did not offer any online
training. The majority of institutions that provided online training use CITI 11 (Collaborative
Institutional Training Initiative) (approximately 87 percent), with approximately 24 percent of
those supplementing CITI with other online material, e.g., material from the Department of Health



10
   This result is consistent with the Study, which found that online-only training was considered sufficient in 79 percent
of universities with uniform requirements and 78 percent of universities with differentiated requirements.
11
   https://www.citiprogram.org

                                                                                                                        7
and Human Services Office of Research Integrity’s web page 12 or using another RCR online
training course, such as EPIGEUM. 13
Both our results and the Study’s results found some institutions offered face-to-face training as an
alternative to online training, but allowed participants to choose which to take. Approximately 65
percent of the participants who expressed a preference generally preferred interactive training.
Interestingly, one university offered participants a choice between taking an in-person course and
taking online training and tracked the participants’ choices. That university told us only 5 percent
of the participants choose the in-person course.

2. The lack of guidance from NSF as to what constitutes “appropriate training” means that
NSF cannot guarantee that the instruction provided in response to the RCR training
requirement meets a minimum level of quality. The core of the NSF policy is that institutions
should provide “appropriate training” in the responsible conduct of research to undergraduate
students, graduate students, and postdocs who are directly supported by an NSF award for research.
Because NSF has not defined what constitutes appropriate training, that determination is left up to
each individual institution. As a result, when we examined the training provided by the institutions
we reviewed, we had no basis for concluding that the training provided was insufficient to meet
the RCR training requirement, even though some of the approaches we found did little to ensure
that students and postdocs were being adequately educated about the responsible conduct of
research.
To illustrate, the following are examples of some of the approaches we identified during our review
at one or more institutions:
       •   The institution’s only RCR training requirement is that participants read the Office of
           Research Integrity’s Introduction to the Responsible Conduct of Research.
       •   RCR training occurs at the discretion of the PI. The institution provides no guidance to the
           PI about the purpose, content, or duration of the training, and thus cannot guarantee that
           students receive consistent, acceptable instruction.
       •   Generic standard lab safety, or animal/human subjects, or data/IT security training
           constitutes RCR training.
       •   RCR training is supposed to be incorporated into the curriculum, but the institution cannot
           identify specific RCR topics in any individual classes.
In each of the examples cited above, while the institution claims to be providing training in
response to the RCR requirement, the approaches taken do not appear to be sufficient to ensure
that undergraduate students, graduate students, and postdocs participating on an NSF research
project have a working understanding of the importance of the responsible conduct of research.
The lack of guidance as to what constitutes appropriate RCR training has implications for NSF
and its community that extend beyond the implementation of the COMPETES requirement. NSF
generally requires subjects of research misconduct investigations to take RCR training with an
emphasis in the area in which the misconduct occurred. In the absence of minimum quality
standards for RCR training for NSF students and researchers, neither NSF nor the research


12
     RCR resources tab at https://ori.hhs.gov
13
     https://www.epigeum.com/

                                                                                                     8
misconduct subjects know whether the training available at the subject’s institution is sufficient to
satisfy this requirement.
To ensure the integrity and consistency of the RCR training provided in response to the America
COMPETES requirement, NSF could identify and share with the community minimum standards
for such training. The standards could vary by discipline or by researcher status (undergraduate
student, graduate student, or postdoc).


3. Some institutions are engaged in promising practices or using techniques that are worthy
of being shared with the broader community. As we conducted our review, we attempted to
identify practices and/or techniques that were worthy of being shared with the community at large.
The following are examples of an RCR course we found particularly interesting (the first bullet),
implementation approaches we considered promising or best practices, and feedback from students
on how to enhance their interest in RCR training (the last bullet). We suggest that NSF find ways
to communicate these promising practices and identify and share others with its recipients.
•   Adding stress management to RCR training. While there are many reasons why researchers
    fail to uphold community standards, stress created by “publish or perish” pressures, time
    pressures to finish a project, and pressure to obtain a grant can lead to compromised decisions.
    To address stress resulting from these pressures, one university added a stress management
    class to its RCR training. The RCR administrators told us that that course turned out to be the
    most popular class in the entire RCR program, and students at that university also praised the
    class. In fact, we learned that students at other institutions had also suggested that a stress
    management class be part of the RCR training.
•   Requiring RCR training for all graduate students. Several of the larger research universities
    concluded that the most important group to reach were graduate students. Accordingly, their
    RCR programs require all graduate students to take RCR training, irrespective of whether they
    are funded by NSF or currently participating in research.
•   Involving faculty in RCR training. Several institutions required faculty involvement in RCR
    training. At several of those institutions, participants told us it made a big impact on their
    impression of the importance of the course if they saw their mentors involved.
•   Requiring repeat RCR training on a regular basis. A small number of universities required
    periodic RCR training, typically every three years, either as retraining or as a refresher, which
    is consistent with NIH guidance.
•   Tailoring training to a participant’s educational level and/or discipline. A majority of
    institutions in our review differentiated training based on educational level, and/or discipline.
•   Augmenting training provided during a semester by providing additional sessions on
    either side of the required course. One university provides two required, faculty-taught,
    university-wide “bookend” sessions each semester, before and after required discipline-based,
    university-approved training. These courses serve both to reinforce the discipline-based
    training provided by the individual departments, and to provide students in different disciplines
    the opportunity for standardized, in-person training.
•   Ensuring that participants understand why they are required to take RCR training.
    Many of the participants we interviewed indicated that neither their institution nor their PI

                                                                                                   9
       explained why they were required to take RCR courses. We were told that the students received
       emails from a third party who told them to register and take the online courses. One
       undergraduate told us that she did not know why she was taking the course — she had simply
       applied for a part-time job at the school she attended, not knowing she would be working on
       an NSF-funded research project. Similarly, a graduate student at a research university received
       a notice to take CITI, but did not know why he needed to do so until he spoke with his advisor.


4. No institutions are conducting risk assessments, despite the fact that NSF’s FAQ says that
they should. The America COMPETES Act stated institutions should provide RCR training for
individuals participating in research or education; however, NSF required training only for
participants who are directly supported on research grants. NSF’s FAQ indicates NSF expected
institutions to conduct a risk assessment to best determine student and postdoc participation in its
institutional RCR training plans. 14 According to our interviews, not a single institution carried out
a risk assessment in order to determine who should take what RCR training and when such training
should be given. Rather, they either turned to the guidance provided by other government sources
(like NIH) or created their own definitions. If use of a risk assessment in this context is important
to NSF, it should convey that fact to the institutions it supports and ensure that they build such an
assessment into their processes.


5. Requiring RCR training only for participants supported by NSF can have negative
consequences. In the majority of institutions in our sample (34 out of 47; 72 percent), 15 only the
participants supported by NSF are required to take RCR training. Requiring training only for NSF-
funded students can lead to a situation in which a PI could have several students or postdocs in a
laboratory, all of whom are participating on NSF research, but not all of whom are directly
supported by NSF. If NSF wants the RCR training it requires to have a broader impact, it should
consider requiring all of these participants in NSF research to receive such training.
In the current research environment, it is common for individuals working on the same project to
be funded by multiple sources. For example, while a PI’s primary research may be funded by NSF,
a graduate student or postdoc working on the award may be funded by a private foundation,
university research funds, or another Federal agency. Undergraduate students may either be
participating in funded research or working in a non-research position (animal care or laboratory
ware cleaning) as part of a financial aid package. Nonetheless, all of these individuals, regardless
of the source of their individual funding, must be identified along with each individual’s
contributions 16 in the annual and final reports that NSF requires PIs to file for all research grants.
In a similar manner, NSF should recognize that all participants in NSF-funded research, including
those who are not directly funded by NSF, may “touch” the research data and thus should have
appropriate training to ensure they know how to maintain the integrity of the data.


14
     http://www.nsf.gov/bfa/dias/policy/rcr/rcrfaqs.jsp#3
15
   Note: Numbers add to greater than 100 percent due to rounding. As previously discussed, there were 45 (out
of 48) institutions that eventually had an RCR plan. There were 2 additional institutions that provided RCR
training, but did not have a formal plan, bringing the total number of institutions that provided training to 47.

16
     See, for example, NSF report guidance on research.gov

                                                                                                                    10
6. Although faculty play a critical role in the research enterprise and constitute a significant
percentage of research misconduct subjects, only 15 percent of the plans we reviewed require
faculty to take RCR training. We observed that only 15 percent (7 out of 46—we do not have
data from one university on this specific point) of the institutions currently require faculty
involvement in RCR training, either by requiring all new PIs or those new to NSF funding to take
training, or by requiring those with supported participants to take RCR training. NSF does not
require any faculty involvement with or participation in RCR training, either as a provider or a
recipient. An exception is when a PI is found to have committed research misconduct. In those
cases, NSF generally requires the PI to take RCR training, with a focus on the particular category
of research misconduct that occurred and in an interactive format (e.g., an instructor-led course).
We reviewed NSF’s findings of research misconduct for plagiarism for the last 5 fiscal years to
determine the group from which our subjects were most likely to come — undergraduates,
graduate students, and postdocs or faculty members/PIs. Table 2 shows the distribution of subjects
with respect to whether they would be required to take RCR training under NSF’s policy.


                         Table 2: Subjects of NSF Findings for Plagiarism
                              Plagiarism               Undergraduates,       Faculty/PIs
                                                         Graduates,
                                                          Postdocs
                                                               3                  67
                 Source: NSF findings of research misconduct for plagiarism FY2012-FY2016.


Faculty/PIs were the subjects of 96 percent (67 out of 70) of plagiarism cases in which NSF made
a finding of research misconduct from FY2012-2016. Because subjects are not required to provide
a reason for why they plagiarized, not all cases have such information. In many of the faculty/PI
investigations, however, the subjects argued that the plagiarized material was not actually
plagiarism. In fact, several of the faculty/PIs in those cases, with positions ranging from assistant
to full professor, claimed a faulty understanding of some aspect of proper attribution, with most of
those claiming a faulty understanding also stating they were unaware that using someone else’s
words verbatim required quotation marks.
As our data show, faculty/PIs are overwhelmingly more likely to be subjects of plagiarism cases.
Recent research highlighted by the online blog RetractionWatch 17 studied the effectiveness of
RCR training. That research showed that training specifically for plagiarism seemed to be more
effective than general training on research integrity. 18 This is possibly due to the formulaic
application of quotation, citation, and referencing and the ability to use plagiarism software to



17
       http://retractionwatch.com/2016/04/12/do-interventions-to-reduce-misconduct-actually-work-maybe-not-says-
new-report/
18
   A. Marusic, et al., Cochrane Database of Systemic Reviews, 4 (2016);
DOI: 10.1002/14651858.MR000038.pub2

                                                                                                             11
identify copied text. 19 Therefore, NSF should consider encouraging RCR training, or at least
plagiarism training, for all new faculty or faculty who have not submitted an NSF proposal.


7. There is no requirement that NSF-funded students and/or researchers take RCR training
before they begin work on NSF-funded research. Although NSF requires students and postdocs
receiving funding to complete RCR training, our results showed that at half of the institutions for
which we have the data (22 out of 44—we did not have data for 3 institutions on this specific
point), it was possible for NSF-funded students and researchers to conduct research prior to being
trained. In fact, NSF-funded students were, in some instances, performing research for many
months or even a year before being trained in RCR. 20 Some institutions required the completion
of training concurrent or prior to, or shortly after, initiation of the research or NSF grant award.
To address this problem, NSF should consider implementing guidance regarding training prior to
involvement in NSF-funded research.


8. Every institution that applies for an NSF research award is required to have an RCR plan,
even if it rarely receives such funding. Several of the 11 institutions that did not initially have
RCR plans told us that they did not have such a plan because they received few and/or infrequent
grants from NSF. After examining the number and types of NSF grants the institutions that did not
have plans received from 2010 (when NSF’s RCR policy was initiated) through 2013 (when we
selected our sample), we found that:
    •    2 of the 11 did not have a research grant in the year we selected (FY 2013). These two
         institutions had a combined total of three grants over the 4 years we examined. Each later
         developed an RCR program.
    •    3 of the 11 had at least one research grant, but as a whole received few NSF awards (6
         grants over 4 years). Two of the three ultimately developed an RCR plan.
    •    6 of the 11 all had research grants, including at least one Research for Undergraduate
         Institutions and/or Research Experience for Undergraduates award. These institutions
         generally had more grants than the previous 5, with a combined total of 86 awards over the
         4 years we examined. Four of these six institutions subsequently developed an RCR
         program.
NSF’s policy requires all institutions that apply for research funding, not just those who receive it,
to certify that they have an RCR training program in place for undergraduate students, graduate
students, and postdocs who are supported by NSF. Accordingly, each of the 11 noncompliant
institutions should have had an RCR plan. Based on the foregoing, it appears that 5 of the 11
institutions rarely received research funding from NSF. Given the effort required to create,
implement, and maintain a training plan that may seldom be used, NSF may want to consider
whether there is a funding level or number of grants below which institutions will not be required

19
   See p. 43 of our March 2009 Semiannual Report for a discussion of quotation, citation, and reference in avoiding
plagiarism.
20
   In a recently adjudicated case (A13100087), a graduate student worked on an NSF grant for nearly two semesters
before being trained in RCR; university policy required only that students take it within a year of beginning work on
an NSF-funded project. However, before taking the training, the student wrote about his research contribution in an
NSF annual report attachment and some of the text was plagiarized. NSF made a finding of research misconduct.

                                                                                                                 12
to implement a formal RCR training program, but could be required to provide more informal RCR
training. If NSF decides to pursue such an exception, it should seek Congressional approval to
deviate from the broad requirement of the America COMPETES Act. Any exception should
clearly articulate the level of funding or number of grants that will not require creation of a formal
plan.


Conclusion
While most of the institutions we sampled complied with NSF’s RCR requirements, almost one
quarter of the institutions did not initially do so. In light of that finding and the related observations
we made during the course of review, it appears that NSF’s awardees could benefit from NSF
providing written guidelines or templates for universities to follow, as requested by the Act’s report
language, and from the sharing of best practices with the broader community. We encourage NSF
to take whatever actions it believes will ensure that awardees understand the importance of
providing vibrant RCR instruction to NSF-funded researchers. Such actions will help minimize
the risk of unethical or unprofessional conduct by such individuals and, in so doing, help protect
the relationship between science and society as a whole.




                                                                                                       13
                                       Appendix—Methodology
We identified as our sample population those institutions requesting funding in 2013 for
undergraduate students, graduate students, or post-docs. From those, we randomly selected 200
institutions and judgmentally reduced the sample size to 53 institutions, adjusting based on
institution size and location, resulting in institutions from 24 different states. We used a common
definition for size, with ‘small’ being institutions with less than 5,000 students; ‘medium’ with
student population between 5,000 and 15,000; and institutions with more than 15,000 students
designated as ‘large.’ Our group included public and private colleges and universities, as well as
community colleges. Using these designations, our distribution of the 53 is:

                                     Small         17 (32%)
                                     Medium        13 (25%)
                                     Large         23 (43%)


As discussed in the report, we removed five institutions that did not have RCR programs, but
received only education grants from NSF. Thus, there are 48 institutions in Figure 1, not 53.
We created a questionnaire based on the Federal Sentencing Guidelines Manual, Effective
Compliance and Ethics Program. We determined three groups of respondents would be able to
best address our questions about the different institutions implementation of NSF’s RCR
requirement: a senior administrator (President, Provost, etc.) who sets the tone at the top, the
designated RCR administrator, and trainees who had completed the required training. Because we
were seeking information from more than nine institutions, we were required to have the Office of
Management and Budget (OMB) approve our review plan. OMB approved our plan, and details
of our plan, including the three questionnaires and the engagement letter, are available at the OMB
site. 21
We sent each of the institutions an engagement letter asking for a copy of the institution’s plan and
tracking data, and to arrange for three interviews. The interviews were generally conducted by two
staff members. The responses were then assessed so that we could draw conclusions about patterns.




21
     OMB Control Number 3145-0227.

                                                                                                  14