National Science Foundation Office of Inspector General OIG Review of Institutions’ Implementation of NSF’s Responsible Conduct of Research Requirements July 25, 2017 OIG Tracking No. PR12030006 Executive Summary In 2007, Congress passed the America Creating Opportunities to Meaningfully Promote Excellence in Technology, Education, and Science Act (America COMPETES Act), 1 which among other things directed NSF to introduce a requirement for awardees to provide adequate training for undergraduate students, graduate students, and postdoctoral researchers about the Responsible Conduct of Research (RCR). In 2013, 3 years after NSF’s implementation of the Act, we contacted 53 institutions to learn how they had implemented their own RCR training in response to NSF’s requirement. We requested and reviewed institutions’ RCR plans and asked a variety of questions of institutional officials and RCR trainees about the institutional plans, such as who gets trained, the format of the training, and the training’s content. This report is divided into two parts. The first answers the key compliance questions related to NSF’s policy, e.g., did the institutions in our sample have a plan, did they designate a person to oversee compliance, and can the institutions verify that the necessary people are being trained. The second contains observations from our fieldwork for NSF’s consideration, including opportunities to strengthen its RCR policy. Background The scientific enterprise is based on a foundation of trust. If the trust is found to have been misplaced as a result of unethical or unprofessional conduct on the part of scientists, the impact of that breakdown is not limited to the research community alone — it can undermine the relationship between science and society as a whole. 2 For this reason, the National Academy of Sciences has long recognized the importance of training scientists in RCR. Within the Federal Government, the National Institutes of Health (NIH) implemented a requirement for its awardees to provide RCR training in 1989; the US Department of Agriculture (USDA) enacted a similar requirement in 2013. NSF implemented its RCR requirement in 2010. 3 NSF’s RCR requirement grew out of a provision in the America COMPETES Act of 2007, which directed NSF to require that each institution that applied for financial assistance for science and engineering research or education describe in its grant proposal a plan to provide appropriate training and oversight in the responsible and ethical conduct of research to undergraduate students, graduate students, and postdoctoral researchers (postdocs) participating in the proposed research project. Given the differing needs of the students and researchers covered by the new requirement, in report language accompanying the Act, the drafters gave NSF “maximum flexibility in determining the full range of activities that would constitute appropriate training.” 4 In the following sentence, the drafters made clear, however, that they expected NSF to promptly develop and provide written guidelines and/or templates for universities to follow so their compliance could be verified by all parties. 1 America Creating Opportunities to Meaningfully Promote Excellence in Technology, Education, and Science Act (codified as 42 USC 1862o-1) 2 On Being A Scientist: A Guide to Responsible Conduct in Research: Third Edition (2009), p. ix. 3 Responsible Conduct of Research (RCR), January 4, 2010, http://www.nsf.gov/pubs/policydocs/pappguide/nsf16001/aag_4.jsp#IVB 4 House Report 110-289, p. 184, 2007 2 NSF developed its RCR policy through a multi-stage process in which the agency obtained and considered input from internal working groups; a workshop sponsored through the National Academies of Sciences, Engineering, and Medicine; and comments from the public through the Federal Register. When enacted in 2010, the policy established the following requirements for institutions seeking NSF research funding: • The institutions must have a plan in place to provide appropriate training and oversight in the responsible and ethical conduct of research to undergraduates, graduate students, and postdocs who will be supported by NSF to conduct research. Institutional certification of compliance with this responsibility is required for each proposal submitted to NSF. While institutions are not required to include training plans in their proposals, those plans are subject to review upon request. • The institutions must designate one or more persons to oversee compliance with the RCR training requirement. • The institutions are responsible for verifying that the students and researchers supported by NSF to conduct research have received training in the responsible and ethical conduct of research. To date, NSF has not provided written guidelines or templates for universities to follow, as requested in 2007 by the America COMPETES Act report language. This review We conducted this review to examine compliance in a sample of institutions with the three requirements of the NSF RCR policy and to evaluate whether the lack of guidance and templates might be having an effect on such compliance. We are also sharing observations we made during the course of our work with the hope that NSF will use this information to strengthen its implementation. To make these assessments, we selected 53 institutions that requested NSF funding in 2013 for undergraduates, graduate students, and/or postdocs. Our sample consisted of small (fewer than 5,000 students), medium (between 5,000-15,000 students), and large (over 15,000 students) institutions in 24 different states, and contained public and private colleges and universities as well as community colleges. At each institution we sought to interview a senior administrator who sets the tone at the top regarding the importance of RCR, the designated RCR administrator, and students/postdocs who had completed the required training. Because we spoke to more than nine institutions, we sought and received OMB approval for our review. 5 Results of Compliance Testing We found that almost one quarter of the institutions in our sample were not in compliance with NSF’s RCR requirements at the time they received our engagement letter. Specifically, 23 percent (11 out of 48) of institutions did not initially have an RCR plan, or, by default, a designated person to oversee the plan or verifying and tracking that the required participants took the training. While 8 out of the 11 such institutions developed a plan after being contacted by our office, the level of 5 For additional details on our methodology, see the Appendix—Methodology on page 14 of this report and OMB, Control number 3145-0227. 3 noncompliance raises a question as to whether institutions are uniformly and successfully implementing NSF’s RCR policy. Our specific findings for each requirement are set forth below. Does the institution have an RCR plan? About 30 percent (16 out of 53) of the institutions in our sample did not have an RCR plan when we first contacted them. As part of our fieldwork, we sought to determine why those 16 institutions did not have an RCR plan. Some of the institutions said they did not have a plan because they only received education funding from NSF. We found that although the America COMPETES Act training requirement applies to institutions that apply for funding for science and engineering research and for education, NSF’s RCR policy applies only to institutions that receive research funding. 6 Working under the assumption that some grants from the NSF Directorate for Education and Human Resources would qualify as funding for education (and not research) purposes, 5 of the 16 institutions that did not have plans were community colleges that informed us they received only education funding. Under NSF’s policy, such entities were not required to have an RCR plan; accordingly, we eliminated those entities from our sample, reducing the number of institutions from 53 to 48. This left us with a total of 11 institutions that did not have plans (2 large and 9 small), which amounts to almost 23 percent of our sample of 48. The fact that there were so many noncompliant institutions in the group we examined indicates that NSF may have an implementation problem with this requirement. The number in and out of compliance for the three sizes of institutions we examined, after removing the community colleges, is shown in Figure 1. Figure 1: Institutions with RCR Plans when first contacted by OIG Institution RCR Plans Large 18 2 Institution Size Medium 11 0 Small 8 9 Compliant Non-Compliant Source: NSF-OIG generated based on data provided by institutional participants Of the 11 institutions that did not have an RCR plan when we first contacted them, 8 developed a plan after receiving our engagement letter. Of the remaining three, one developed a draft plan but 6 We note that both NIH and USDA require RCR training for recipients of their education grants. 4 never formally instituted it, one chose not to provide its plan despite the requirement to do so upon request, and one does not consider its plan a formal RCR plan (the RCR training was determined on an ad hoc basis). Adding the 8 institutions that developed and shared copies of their plans to the 37 that originally had plans, a total of 45 institutions in our sample of 48 eventually had an RCR plan (94 percent). Has the institution designated a person to oversee compliance with the RCR requirement? By default, institutions that did not have a plan would not have anyone designated to oversee compliance with the RCR requirement. Therefore 11 out of 48 institutions were initially not in compliance with this requirement. We found that each of the 37 institutions that had an RCR plan at the start of our review had designated an individual to track compliance with the NSF RCR training requirement. Of the eight institutions that developed a plan after receiving our engagement letter, seven had designated such an individual. Ultimately, only 1 of the 45 institutions in our sample that either had a plan at the start of our review or developed one after our review began had not designated an individual to oversee compliance with this requirement. One institution that did not have a formal plan told us that because it only occasionally receives NSF research grants, the Principal Investigator (PI) of the awards it receives performs this function. Can the institution verify that the necessary people are being trained? Again, by default, institutions that did not have a plan could not verify that the necessary people were being trained. Therefore 11 out of 48 were initially not in compliance with this requirement. Of the 37 institutions that had an RCR plan at the start of our review, 30 told us they had sufficient tracking in place to ensure that students received required RCR training. Of the eight institutions that developed a plan after receiving our engagement letter, seven told us they had an adequate tracking system. At the end of our fieldwork, then, approximately 18 percent (8 out of 45) of the institutions in our sample that either had a plan at the start of our review or developed one after our review began could not tell if their students were receiving required RCR training. The fact that there were so many noncompliant institutions in the group we examined indicates that NSF may have an implementation problem with this tracking requirement. Observations made during interviews During the course of our review, we made many observations about the nature of the RCR training being provided in response to the America COMPETES Act. We detail those observations below to provide insights to NSF about how institutions are responding to this requirement. We hope that NSF will use this information to identify ways it can strengthen implementation of this important requirement. 1. The institutions we reviewed utilized a wide variety of training approaches and formats. Some institutions only train NSF-supported participants; others include all students, regardless of their funding source. Some institutions provide different training based on participants’ status or discipline, while others provide the same training to all. Some schools provide instruction in- 5 person and online; others only provide online training. We summarize below the variety of approaches and formats we found. For purposes of this assessment, if institutions had different training requirements for undergraduates and graduate students, our results reflect how the institution trained graduate students. 7 Differences in training provided based on status and/or discipline: We found that some institutions provide the same RCR training to all students, irrespective of their educational level, i.e., undergraduate or graduate student or postdoc, which we characterize as a uniform plan. 8 Other institutions provide different training to participants based on their educational level, which we characterize as a differentiated plan. In our sample, excluding those institutions that have only undergraduates, we found a nearly even split for each approach (52 percent differentiate; 48 percent do not). This result differs somewhat from a recent survey (the Study) 9 that found 69 percent of the 91 entities studied had uniform requirements, while 30 percent used differentiated requirements based on educational level. Of the institutions that differentiate and use online training, many did so by requiring graduates and/or postdocs to take more modules of online training than were required of undergraduates. The institutions that differentiate and require interactive training typically require undergraduates to take an online course, but require graduate students to take an in-person course. A majority of institutions in our sample considered students’/researchers’ disciplines when deciding how to train them on RCR. Approximately 72 percent of the institutions we sampled told us they provided RCR training based on participant discipline. The primary method of differentiation in such cases was the use of different online modules for different disciplines. A smaller number of respondents indicated that some departments required participants to attend additional seminars or other interactive training to supplement online training. Several of the larger research institutions have unique differentiated training, in that the schools within the institution create customized, interactive training for their respective departments. Differences in training formats: The institutions we queried also used various formats to provide the required training. Many provided all their RCR training online. The most frequently cited benefits of such training were that it could be completed at the participants’ convenience, it introduced common terms and language, and it provided an easy way to track completion of the training and compliance with RCR training requirements. However, participants who only took 7 The rationale for this approach was that undergraduates’ primary responsibility is to take courses and typically participate in research in the summer, or in limited amounts during the academic year. Graduate students and postdocs, however, are more involved in research for longer periods of time and have greater flexibility in their schedules. Furthermore, graduate students and postdocs are more probable current and future participants in the research community. They have a greater need to be educated about the responsible conduct in, and expectations of, the community of which they are current or future members. 8 We use these terms and definitions taken from a recent, scholarly survey of the RCR plans of high-research-activity institutions. T. Phillips, F. Nestor, G. Beach, and E. Holmes, “America COMPETES at 5 Years: An Analysis of RCR Training Plans,” presented at the 25th Annual Meeting of the Association for Practical and Professional Ethics, February 2016, and at the NSF workshop on “Responsible Conduct of Research” hosted by SBE and CCE STEM in April 2016. We refer to this research as “the Study.” 9 Ibid.—the Study 6 online RCR training told us they found it provided mostly common-sense advice and/or advice that was repetitive, not applicable to their research, and/or too basic and generalized. Some of the online-trained participants we interviewed indicated that they did not like the online training format because they did not have the opportunity to ask questions about what they were learning or discuss the content being presented. A particularly consistent complaint was the fact that the online format did not provide them with an opportunity to discuss case studies. Summary results: In Table 1, we summarize our findings about the different training formats being utilized in RCR courses students were required to take, as well as the extent to which RCR training was provided beyond NSF-supported participants. The table reflects definitions we created based on the courses participants were required to take, not ones students had the option of taking. Some institutions in our review provided various options for RCR training, but let students choose which option to take. Awardees that provided various course options often did not track optional participation in those courses, for example, allowing anyone who was interested in online training to take it. Table 1: Required RCR Training Formats and Participants Required trainee population is Required trainee population is limited to NSF-supported not limited to NSF-supported participants participants Trainee population is able to fulfill the RCR requirement 64% (30/47) 9% (4/47) by only taking online training or through document review Trainee population receives RCR content through 9% (4/47) 19% (9/47) required interactive training (i.e., a course, workshop or seminar) Source: NSF-OIG assessment of participants’ data Note: Numbers add to greater than 100 percent due to rounding. As previously discussed, there were 45 (out of 48) institutions that eventually had an RCR plan. There were 2 additional institutions that required RCR training, but did not have a formal plan, bringing the total number of institutions that provided training to 47. We found that most of the institutions in our sample (34 out of 47, or approximately 73 percent) that required students to take RCR training allowed trainees to complete all or most of their required RCR training online. 10 Indeed, only one institution in our sample did not offer any online training. The majority of institutions that provided online training use CITI 11 (Collaborative Institutional Training Initiative) (approximately 87 percent), with approximately 24 percent of those supplementing CITI with other online material, e.g., material from the Department of Health 10 This result is consistent with the Study, which found that online-only training was considered sufficient in 79 percent of universities with uniform requirements and 78 percent of universities with differentiated requirements. 11 https://www.citiprogram.org 7 and Human Services Office of Research Integrity’s web page 12 or using another RCR online training course, such as EPIGEUM. 13 Both our results and the Study’s results found some institutions offered face-to-face training as an alternative to online training, but allowed participants to choose which to take. Approximately 65 percent of the participants who expressed a preference generally preferred interactive training. Interestingly, one university offered participants a choice between taking an in-person course and taking online training and tracked the participants’ choices. That university told us only 5 percent of the participants choose the in-person course. 2. The lack of guidance from NSF as to what constitutes “appropriate training” means that NSF cannot guarantee that the instruction provided in response to the RCR training requirement meets a minimum level of quality. The core of the NSF policy is that institutions should provide “appropriate training” in the responsible conduct of research to undergraduate students, graduate students, and postdocs who are directly supported by an NSF award for research. Because NSF has not defined what constitutes appropriate training, that determination is left up to each individual institution. As a result, when we examined the training provided by the institutions we reviewed, we had no basis for concluding that the training provided was insufficient to meet the RCR training requirement, even though some of the approaches we found did little to ensure that students and postdocs were being adequately educated about the responsible conduct of research. To illustrate, the following are examples of some of the approaches we identified during our review at one or more institutions: • The institution’s only RCR training requirement is that participants read the Office of Research Integrity’s Introduction to the Responsible Conduct of Research. • RCR training occurs at the discretion of the PI. The institution provides no guidance to the PI about the purpose, content, or duration of the training, and thus cannot guarantee that students receive consistent, acceptable instruction. • Generic standard lab safety, or animal/human subjects, or data/IT security training constitutes RCR training. • RCR training is supposed to be incorporated into the curriculum, but the institution cannot identify specific RCR topics in any individual classes. In each of the examples cited above, while the institution claims to be providing training in response to the RCR requirement, the approaches taken do not appear to be sufficient to ensure that undergraduate students, graduate students, and postdocs participating on an NSF research project have a working understanding of the importance of the responsible conduct of research. The lack of guidance as to what constitutes appropriate RCR training has implications for NSF and its community that extend beyond the implementation of the COMPETES requirement. NSF generally requires subjects of research misconduct investigations to take RCR training with an emphasis in the area in which the misconduct occurred. In the absence of minimum quality standards for RCR training for NSF students and researchers, neither NSF nor the research 12 RCR resources tab at https://ori.hhs.gov 13 https://www.epigeum.com/ 8 misconduct subjects know whether the training available at the subject’s institution is sufficient to satisfy this requirement. To ensure the integrity and consistency of the RCR training provided in response to the America COMPETES requirement, NSF could identify and share with the community minimum standards for such training. The standards could vary by discipline or by researcher status (undergraduate student, graduate student, or postdoc). 3. Some institutions are engaged in promising practices or using techniques that are worthy of being shared with the broader community. As we conducted our review, we attempted to identify practices and/or techniques that were worthy of being shared with the community at large. The following are examples of an RCR course we found particularly interesting (the first bullet), implementation approaches we considered promising or best practices, and feedback from students on how to enhance their interest in RCR training (the last bullet). We suggest that NSF find ways to communicate these promising practices and identify and share others with its recipients. • Adding stress management to RCR training. While there are many reasons why researchers fail to uphold community standards, stress created by “publish or perish” pressures, time pressures to finish a project, and pressure to obtain a grant can lead to compromised decisions. To address stress resulting from these pressures, one university added a stress management class to its RCR training. The RCR administrators told us that that course turned out to be the most popular class in the entire RCR program, and students at that university also praised the class. In fact, we learned that students at other institutions had also suggested that a stress management class be part of the RCR training. • Requiring RCR training for all graduate students. Several of the larger research universities concluded that the most important group to reach were graduate students. Accordingly, their RCR programs require all graduate students to take RCR training, irrespective of whether they are funded by NSF or currently participating in research. • Involving faculty in RCR training. Several institutions required faculty involvement in RCR training. At several of those institutions, participants told us it made a big impact on their impression of the importance of the course if they saw their mentors involved. • Requiring repeat RCR training on a regular basis. A small number of universities required periodic RCR training, typically every three years, either as retraining or as a refresher, which is consistent with NIH guidance. • Tailoring training to a participant’s educational level and/or discipline. A majority of institutions in our review differentiated training based on educational level, and/or discipline. • Augmenting training provided during a semester by providing additional sessions on either side of the required course. One university provides two required, faculty-taught, university-wide “bookend” sessions each semester, before and after required discipline-based, university-approved training. These courses serve both to reinforce the discipline-based training provided by the individual departments, and to provide students in different disciplines the opportunity for standardized, in-person training. • Ensuring that participants understand why they are required to take RCR training. Many of the participants we interviewed indicated that neither their institution nor their PI 9 explained why they were required to take RCR courses. We were told that the students received emails from a third party who told them to register and take the online courses. One undergraduate told us that she did not know why she was taking the course — she had simply applied for a part-time job at the school she attended, not knowing she would be working on an NSF-funded research project. Similarly, a graduate student at a research university received a notice to take CITI, but did not know why he needed to do so until he spoke with his advisor. 4. No institutions are conducting risk assessments, despite the fact that NSF’s FAQ says that they should. The America COMPETES Act stated institutions should provide RCR training for individuals participating in research or education; however, NSF required training only for participants who are directly supported on research grants. NSF’s FAQ indicates NSF expected institutions to conduct a risk assessment to best determine student and postdoc participation in its institutional RCR training plans. 14 According to our interviews, not a single institution carried out a risk assessment in order to determine who should take what RCR training and when such training should be given. Rather, they either turned to the guidance provided by other government sources (like NIH) or created their own definitions. If use of a risk assessment in this context is important to NSF, it should convey that fact to the institutions it supports and ensure that they build such an assessment into their processes. 5. Requiring RCR training only for participants supported by NSF can have negative consequences. In the majority of institutions in our sample (34 out of 47; 72 percent), 15 only the participants supported by NSF are required to take RCR training. Requiring training only for NSF- funded students can lead to a situation in which a PI could have several students or postdocs in a laboratory, all of whom are participating on NSF research, but not all of whom are directly supported by NSF. If NSF wants the RCR training it requires to have a broader impact, it should consider requiring all of these participants in NSF research to receive such training. In the current research environment, it is common for individuals working on the same project to be funded by multiple sources. For example, while a PI’s primary research may be funded by NSF, a graduate student or postdoc working on the award may be funded by a private foundation, university research funds, or another Federal agency. Undergraduate students may either be participating in funded research or working in a non-research position (animal care or laboratory ware cleaning) as part of a financial aid package. Nonetheless, all of these individuals, regardless of the source of their individual funding, must be identified along with each individual’s contributions 16 in the annual and final reports that NSF requires PIs to file for all research grants. In a similar manner, NSF should recognize that all participants in NSF-funded research, including those who are not directly funded by NSF, may “touch” the research data and thus should have appropriate training to ensure they know how to maintain the integrity of the data. 14 http://www.nsf.gov/bfa/dias/policy/rcr/rcrfaqs.jsp#3 15 Note: Numbers add to greater than 100 percent due to rounding. As previously discussed, there were 45 (out of 48) institutions that eventually had an RCR plan. There were 2 additional institutions that provided RCR training, but did not have a formal plan, bringing the total number of institutions that provided training to 47. 16 See, for example, NSF report guidance on research.gov 10 6. Although faculty play a critical role in the research enterprise and constitute a significant percentage of research misconduct subjects, only 15 percent of the plans we reviewed require faculty to take RCR training. We observed that only 15 percent (7 out of 46—we do not have data from one university on this specific point) of the institutions currently require faculty involvement in RCR training, either by requiring all new PIs or those new to NSF funding to take training, or by requiring those with supported participants to take RCR training. NSF does not require any faculty involvement with or participation in RCR training, either as a provider or a recipient. An exception is when a PI is found to have committed research misconduct. In those cases, NSF generally requires the PI to take RCR training, with a focus on the particular category of research misconduct that occurred and in an interactive format (e.g., an instructor-led course). We reviewed NSF’s findings of research misconduct for plagiarism for the last 5 fiscal years to determine the group from which our subjects were most likely to come — undergraduates, graduate students, and postdocs or faculty members/PIs. Table 2 shows the distribution of subjects with respect to whether they would be required to take RCR training under NSF’s policy. Table 2: Subjects of NSF Findings for Plagiarism Plagiarism Undergraduates, Faculty/PIs Graduates, Postdocs 3 67 Source: NSF findings of research misconduct for plagiarism FY2012-FY2016. Faculty/PIs were the subjects of 96 percent (67 out of 70) of plagiarism cases in which NSF made a finding of research misconduct from FY2012-2016. Because subjects are not required to provide a reason for why they plagiarized, not all cases have such information. In many of the faculty/PI investigations, however, the subjects argued that the plagiarized material was not actually plagiarism. In fact, several of the faculty/PIs in those cases, with positions ranging from assistant to full professor, claimed a faulty understanding of some aspect of proper attribution, with most of those claiming a faulty understanding also stating they were unaware that using someone else’s words verbatim required quotation marks. As our data show, faculty/PIs are overwhelmingly more likely to be subjects of plagiarism cases. Recent research highlighted by the online blog RetractionWatch 17 studied the effectiveness of RCR training. That research showed that training specifically for plagiarism seemed to be more effective than general training on research integrity. 18 This is possibly due to the formulaic application of quotation, citation, and referencing and the ability to use plagiarism software to 17 http://retractionwatch.com/2016/04/12/do-interventions-to-reduce-misconduct-actually-work-maybe-not-says- new-report/ 18 A. Marusic, et al., Cochrane Database of Systemic Reviews, 4 (2016); DOI: 10.1002/14651858.MR000038.pub2 11 identify copied text. 19 Therefore, NSF should consider encouraging RCR training, or at least plagiarism training, for all new faculty or faculty who have not submitted an NSF proposal. 7. There is no requirement that NSF-funded students and/or researchers take RCR training before they begin work on NSF-funded research. Although NSF requires students and postdocs receiving funding to complete RCR training, our results showed that at half of the institutions for which we have the data (22 out of 44—we did not have data for 3 institutions on this specific point), it was possible for NSF-funded students and researchers to conduct research prior to being trained. In fact, NSF-funded students were, in some instances, performing research for many months or even a year before being trained in RCR. 20 Some institutions required the completion of training concurrent or prior to, or shortly after, initiation of the research or NSF grant award. To address this problem, NSF should consider implementing guidance regarding training prior to involvement in NSF-funded research. 8. Every institution that applies for an NSF research award is required to have an RCR plan, even if it rarely receives such funding. Several of the 11 institutions that did not initially have RCR plans told us that they did not have such a plan because they received few and/or infrequent grants from NSF. After examining the number and types of NSF grants the institutions that did not have plans received from 2010 (when NSF’s RCR policy was initiated) through 2013 (when we selected our sample), we found that: • 2 of the 11 did not have a research grant in the year we selected (FY 2013). These two institutions had a combined total of three grants over the 4 years we examined. Each later developed an RCR program. • 3 of the 11 had at least one research grant, but as a whole received few NSF awards (6 grants over 4 years). Two of the three ultimately developed an RCR plan. • 6 of the 11 all had research grants, including at least one Research for Undergraduate Institutions and/or Research Experience for Undergraduates award. These institutions generally had more grants than the previous 5, with a combined total of 86 awards over the 4 years we examined. Four of these six institutions subsequently developed an RCR program. NSF’s policy requires all institutions that apply for research funding, not just those who receive it, to certify that they have an RCR training program in place for undergraduate students, graduate students, and postdocs who are supported by NSF. Accordingly, each of the 11 noncompliant institutions should have had an RCR plan. Based on the foregoing, it appears that 5 of the 11 institutions rarely received research funding from NSF. Given the effort required to create, implement, and maintain a training plan that may seldom be used, NSF may want to consider whether there is a funding level or number of grants below which institutions will not be required 19 See p. 43 of our March 2009 Semiannual Report for a discussion of quotation, citation, and reference in avoiding plagiarism. 20 In a recently adjudicated case (A13100087), a graduate student worked on an NSF grant for nearly two semesters before being trained in RCR; university policy required only that students take it within a year of beginning work on an NSF-funded project. However, before taking the training, the student wrote about his research contribution in an NSF annual report attachment and some of the text was plagiarized. NSF made a finding of research misconduct. 12 to implement a formal RCR training program, but could be required to provide more informal RCR training. If NSF decides to pursue such an exception, it should seek Congressional approval to deviate from the broad requirement of the America COMPETES Act. Any exception should clearly articulate the level of funding or number of grants that will not require creation of a formal plan. Conclusion While most of the institutions we sampled complied with NSF’s RCR requirements, almost one quarter of the institutions did not initially do so. In light of that finding and the related observations we made during the course of review, it appears that NSF’s awardees could benefit from NSF providing written guidelines or templates for universities to follow, as requested by the Act’s report language, and from the sharing of best practices with the broader community. We encourage NSF to take whatever actions it believes will ensure that awardees understand the importance of providing vibrant RCR instruction to NSF-funded researchers. Such actions will help minimize the risk of unethical or unprofessional conduct by such individuals and, in so doing, help protect the relationship between science and society as a whole. 13 Appendix—Methodology We identified as our sample population those institutions requesting funding in 2013 for undergraduate students, graduate students, or post-docs. From those, we randomly selected 200 institutions and judgmentally reduced the sample size to 53 institutions, adjusting based on institution size and location, resulting in institutions from 24 different states. We used a common definition for size, with ‘small’ being institutions with less than 5,000 students; ‘medium’ with student population between 5,000 and 15,000; and institutions with more than 15,000 students designated as ‘large.’ Our group included public and private colleges and universities, as well as community colleges. Using these designations, our distribution of the 53 is: Small 17 (32%) Medium 13 (25%) Large 23 (43%) As discussed in the report, we removed five institutions that did not have RCR programs, but received only education grants from NSF. Thus, there are 48 institutions in Figure 1, not 53. We created a questionnaire based on the Federal Sentencing Guidelines Manual, Effective Compliance and Ethics Program. We determined three groups of respondents would be able to best address our questions about the different institutions implementation of NSF’s RCR requirement: a senior administrator (President, Provost, etc.) who sets the tone at the top, the designated RCR administrator, and trainees who had completed the required training. Because we were seeking information from more than nine institutions, we were required to have the Office of Management and Budget (OMB) approve our review plan. OMB approved our plan, and details of our plan, including the three questionnaires and the engagement letter, are available at the OMB site. 21 We sent each of the institutions an engagement letter asking for a copy of the institution’s plan and tracking data, and to arrange for three interviews. The interviews were generally conducted by two staff members. The responses were then assessed so that we could draw conclusions about patterns. 21 OMB Control Number 3145-0227. 14
OIG Review of Institutions' Implementation of NSF's Responsible Conduct of Research Requirements
Published by the National Science Foundation, Office of Inspector General on 2017-07-25.
Below is a raw (and likely hideous) rendition of the original report. (PDF)