DOCUMENT RESUME 00167 - [A1051996] [Management of a National Science Foundation Office of Enerly Research and Development Policy Grant to the George ashing -on University: Questions, nswers, and Recommendations]. HaD-77-38; B-133183. January 25, 1977. Released January 27, 1977. 9 pp. + enclosure (28 pp.). Report to Sen. Edward . Kennedy, Chairman, Senate Committee on Labor and Public elfare: National Science Foundation Special Subcommittee; by Elmer B. Staats, Comptroller General. Issue Area: Energy (1600). Contact: Human Resources Div. Budget Function: Natural Resources, Environment, and Energy: Energy (305). Organization Concerned: National Science oundation; George Washington Univ. Congressional Relevance: House Committee on Science and Technology; Senate Committee on Labor and Public elfare: National Scienci Foundation Special Subcommittee; Senate committee on Labor and Public Welfare. Authority: National Science Foundation Act of 1950 (42 U.S.C. 1861 et seq.). The National Science Foundation's (SF) Office of Fnergy R and D Policy awarded a 1974 grant to George Washington University for energy policy research. The initial gran' to the unsolicited proposal and a 12-month renewal were awarded for a series of discussion papers on Federal gas and oil. policies and consultation. Findings/Conclusions: Evaluation of the research proposal was not done by qualified personnel, or under established guidelines. Some of the reviewing personnel ad a prior employment relationship with the principal investigator (Dr. illiam A. Johnson). Additional outside funding came from two interest groups of oil arketing companies. The broad grant objective of preparing the papers and aintaining contact with the NSF were met, but their quality o utility were not determined. Recommendations: NSF should issue a policy directive on how outside funding should be considered in a grant award decision to guard against potential conflicts of interest. It should establish formal policy for evaluating research, including peer reviews of proposals, and timely postevaluation; make its research available through the National Technical Information Service; formally evaluate discussion papers; determine the need for formal evaluation of other recent policy research; require a statement of Foundation support and a disclaimer on its publications; and disclose all funding sources of NSF supported research. Grantees should get formal approval for commercia3 publication of research and the Foundation should fix the disposition of moneys from the sale of publications. (DJ) COMPTROLLER GIMNRAL OF THE UNITEDX AT8 _Tw s WAHINGTONl D.C. B UT B-133183 JAr, 2 5 197i The Honor:ble Edward M. Kennedy Chairman, Special Subcommittee on the National Science Foundation Committee on Labor and Public Welfare United States Senate i)ear Mr. Chairman: In accordance with your August 24, 1976, request and later agreements with your office, we reviewed a number of management issues concerning the National Science Founda- tion's award and administration of a September 5, 1974, grant to the George Washington University, Washington, D.C., for energy policy research. Dr. William A. Johnson was principal investigator under the grant. As directed by your office, we obtained the views informally only of Foundation officials on our findings, and their comments are considered in the report. OuL findings are summarized below and dis- cussed in more detail in the enclosure. The initial grant to the university and a 12-month renewal approved on June 30, 1975, were awarded by the Foundation's Office of Energy R&D Policy. At hat time the Office was part of the Foundation Director's office. Its primary mission was to provide the Director, in his role as the President's Science Adviser, with information on energy- reiated matters. The grant resulted from an unsolicited proposal y the university, requesting funds to suppoLt the work of Dr. Johnson. Under the grant, he was to prepare a series of discussion papers on Federal gas and oil policies and to be available for consultation. The proposal was pre- ceded by a preliminary discussion between the Office and Dr. Johnson, in which they agreed on the content of the pro- posal to be submitted. The Office did not have written guidelines for evaluat- ing research proposals. Review methods were determined by the Founiation's program manager evaluating the proposal. The Fourdation traditionally obtains views of outside ex- perts in the subject area of the proposal; however, the Foundation decides whether to fund the proposal. The re- viewers of the initial proposal and the renewal were largely HRD-77-38 B-133183 Office and other Foundation employees. Except for Treasury Department employees, the reviewers selected did not nclude policy decisionmakers or representatives from other overn- ment agencies and private industry with knowledge of and a probable interest in energy policy research. In addition, some reviewers and the initial Foundation program manager had had a prior employment relationship with the principal investigator. A September 1976 Foundation management study which in- cluded the Office's operating procedures stated that the traditional peer review procedures were regarded as inappro- priate for the policy analysis carried out by the Office. This was because the Office usually had to respond quickly to a specific request from a policy decision office, uch as the White House. The study also noted that external peer review, w!hich usually takes 2 to 4 months, was considered inappropriate to meet deadlines that were often stated in days or weeks. Further, according to former Office offi- cials, research proposal reviews were usually handled by Office staff who understood the need for the research and were often the users of it. No evidence in the Foundation's grant records showed that the proposed research was in response to a specific re- quest from outside the Foundation. Apparently the Office was the intended immediate user, although no formal Foundation records show how the research was used. Also, regarding the time frame, both the initial and renewal grants were made for 12-month periods. The reviewers were given the proposals and a rating sheet, but were apparently not given guidelines to help them consider the meritr of the proposals. Officials of several Foundation directorates 1/ said they usually provide general guidelines that ask the reviewer to consider such factors as merit of the proposal, reasonableness of the budget, and qualifications of the project personnel. The reviewers rated the initial proposal and the renewal high. However, the re- viewers raised some questions and the disposition of the questions is not documented in the Foundation's grant records. /rThe Foundation has six major organizational units (called directorates) for carrying out its research activities. 2 B-133183 Management instructions issued and organization changes made since the initial grant and renewal were awarded to the George Washington University should improve the proposal re- view process. These changes include (1) transferring the Office's functions to the Directorate for Scientific, Tech- nological and International Affairs, which will provide addi- tional levels of grant review and a more formal review proc- ess, (2) creating an action review board 1/ within each direc- torate to review decisions on grant awards, including their documentation, and (3) issuing oundation-wide guidelines on peer reviewer selection, including instructions to the re- viewers. Both the initial proposal and the renewal said there would be outside support for the project, but neither iden- tified the source. Apparently none of the reviewers of the initial proposal asked about outside funding; however, a re- viewer of the renewal proposal asked that the source be con- sidered ir making certain that the study was independent. The program manager could not remember how he handled the comment; however, he believed that the fact of outside sup- port was rot a principal concern, because Foundation grantees commonly have other support. Dr. Johnson sid the Foundation did not inquire about the source of his outside funding at the time of the initial proposal or the renewal. The university's records show that for September 1, 1974, to August 31, 1975, te university receive $110,000 to support Dr. Johnson's work. The funds included the Foun- dation's initial grant for $60,000 and a grant by a group of oil marketing companies known as the Independent Oil Marketers' Conference for $50,000. For September 1, 1975, to August 31, 1976, the university received additional fund- ing of $145,000. These funds consisted of the Foundation's renewal for $70,000, 2/ a second grant by the Independent 1/The Directorate for Scientific, Technological and Inter- nat:ional Affairs' action review board was formalized on January 23, 1976. Its functions were to include a review of all proposed awards after recommended approval by the division irector. Board membership includes top manage- ment of t,, directorate and legal and business representa- tives from other Foundation offices. 2/Includes $20,000 transferred from the Treasury Department to the Foundation to help support the Foundation's energy policy research project. 3 B-133183 Oil Marketers' Conference for $35,000, and a grant by another group of marketers known as the Southern Caucus for $40,000. As of January 7, 1977, the Foundation was considering a June 1976 request from the university for an additional $35,000 to support the project for 12 more months. The grants from the marketing groups were given to the university unconditionally to support Dr. Johnson's work. The university decided to use the gants to fulfill the cost- sharing requirements of the Foundation's grants for energy policy research. The Foundation's grants to the university used funds obtained under its fiscal year 1975 appropriation, which required cost-sharing for wards resulting from un- solicited research proposals. On September 10, 1976, the Foundation issued a directive requiring applicants for funding to furnish information on current support and pending proposals with other funding sources. However, the directive does not prescribe guidelines concerning the effect the source of the outside funding should have when the proposal is considered. If the researcher is overcommitted or has a similar proposal under consideration, prudent management should dictate the actions to be taken. The iopact of the funding source can be an issue more diffi- cult to resolve. We recommend that the Foundation's Director issue a policy directive providing guidance on how the source of outside funding should be considered in a gran' award deci- sion. Outside funding from organizations that could be af- fected by the research might not influence the researcher in ,making his study. However, it could lead to conflict-of- interest questions that might reduce the usefulness of the research. We recognize that obtaining experts to do sensi- tive policy research who have not been supported or otherwise affiliated with a concerned industry or organization might be difficult. Thus, the source of outside funding to support a specific research proposal could be largely an academic ques- tion. Nevertheless, the Foundation should be cautious in funding researchers to do sensitive policy research and try to avoid the appearance of a conflict of interest. Improved evaluation of the research proposal and evaluation of the research publications resulting directly from Foundation- supported policy research would also better insure that the research is useful. (See the recommendations below concer,- ing evaluation.) 4 B-133183 The broad grant objectives of preparing series of discussion papers and providing consultation and otherwise maintaining contact with the Pundation were met. Because of inadequate Foundation grant records, we could ret deter- mine whether the specific papers prepared by the researcher under the grant were those agreed to by the Foundation. The quality c the papers had not been formally determined by the Foundation, and the usefulness of the papers to the Office staff, the apparent intended users, is not documented in the Foundation's records. Recently established management controls will provide for improved grant administration, including better documen- tation of proje:t activities and records of papers submitted under a grant. However, pclicies and procedures for formally evaluating policy research supported by the Directorate for Scientific, Technological and International Affair3 do not exist. We believe that decisionmakers, the ultimate users of policy research, should know the quality of information they are receiving and tnat certainly the Foundation should make sure of the quality of information it passes on to them. Documented evaluations of publications resulting directly from Foundation-supported policy research are necessary to implement these quality controls. We recommend that the Directcr of the Foundation: -- Establish formal policies and procedures for the evaluation of policy research, inc'uding peer reviews of the research proposal that provide a balance of points of view, and timely postevaluation by peers of publications resulting directly from Foundation- supported research. -- Require that eluations of Foundation-supported policy research be made available for distribution through the National Technical Information Service. -- Provide for a fo:mal evaluation of the discussion papers prepared under the Foundation's energy policy grants to the university. -- Determine the need for formal evaluation of other recently completed policy research, considering the sensitivity of the subject matter and its distribution and use. 5 8-133183 The grants to the university were subject to Foundation policy requiring that (1) an acknowledgment of its support be made with the publication of material based on or developed under the project and (2) any publications, such as monographs and books, produced under the grant state that the findings and views ae those of the author and do not necessarily re- flect the Foundation's views. The requirements were not consistently applied i the publications issued under the Foundation's grants to the university in that some papers omitted one or both of the statements. Also, where the ac- knowledgment of Foundation support was used, it stated the work was supported in part by the Foundation, or the work was supported in full by the Foundation. The Foundation's policy requirement, as stated in its grant administration manual, provided only suggested state- ments. To provide uniformity the Foundation has recently established a standard format for the statements. Procedures to make certain that the statements appear in publications have not been established. We recommend that the Foundation's Director require that program managers examine publications directly resulting from Foundation-supported policy research before their public release ro check for compliance with the acknowledgment of support and disclaimer pclicy. Questions have beeli raised over the propriety of not dis- closing all sources of funding for publications resulting from Foundation-supported policy research. As of January 7, 1977, the Foundation had not taken a position on this issue. liWe believe cost-sharing obtained from sources that could be affected by the research results should be avoided. Dis- closure of all sources supporting the rsearch would serve notice to users who might believe the source of funding af- fects the outcome of the research. However, it should be recognized that a researcher not receiving direct outside support from interested sources to help do Foundation- supported research might receive payments from interested parties for speeches, consulting, or other services while doing a Foundation-supported project. Further, before being funded by the Foundation, the researcher might have received funding from interested sources for various services, and upon completing a Foundation-supported project, the researcher might again be employed by interested sources. As previously stated, we believe evaluation is the prin- cipal way to determine the quality of Foundation-supported research. Also, policy research publications resulting directly fr.m Foui._ation support could cite a bibliography of the researcher's recent related publications that the 6 B-133183 interested user could obtain to learn about the researcher's prior positions. We recommend that the Director of the Foundation: -- Require that the Foundation's acknowledgement of support policy be revised to require disclosure of all funding sources of Foundation-supported research. -- Determine the feasibility of requiring that policy research publications developed with Foundation support include citations to related research of the principal investigator. The university realized about $15,000 from the sale of publications developed partly under the Foundation's energy policy grants. Publication costs were about $11,000, leaving a $4,000 net income. All costs and receipts were handled through the Independent Oil Marketers' Conference grant, except costs of about $525 charged to the Southern Caucus grant. These grants were used by the university as the required cost-sharing for the Foundation's grants. Nearly all the income came from the sale of a monograph about petition in the oi' industry, which was distributed by com- Dr. Johnson. The related costs were largely for producing the monograph, which was contracted for by Dr. Johnson. The Foundation, in reviewing the university's proposals, apparently determined there was no incomeresearch tial because the discussion papers were to be used by poten- Foundation. Therefore, no specific rovision was made the in the initial grant or the renewal for disposition of income. Foundation income policy provides that normally income used to offset costs otherwise allowable and chargeable be the grant. If estimated total income is more than $10,000,to the income not used as provided for in the grant would returned. If estimated total income is less than $10,000,be the grant may provide the unused income be retained grantee for science or science education purposes. byThe the Foundation also requires prior approval of commercial produc- tion and distribution of books, films, or similar materials developed with its support. The publication and distribution arrangements for publications resulting in part from the Foundation's grants to the university were not formally proved by the Foundation and might not have been approvedap- orally. 7 B-133183 University records show that te Foundation has con- tributed over 50 percent of the grant funds for the univer- sity's energy research project and has been charged with about 50 percent of the total project costs. Since the Foundation pays a large part of the research costs and it might not have had the opportunity to consider the publica- tion and distribution arrangements, it ought to be able to determine the disposition of at least some of the net income resulting from the sale of publications developed with its support. We recommend that the Director of the Foundation: -- Require grant recipients, as a condition of the award, to (1) obtain the Foundation's written approval of commercial publication and distribution arrangements for publications resulting directly from the Founda- tion's support and (2) allow the Foundation to deter- mine the diposition of all income resulting from the sale of sch publications. -- Consult wih the university to provide for dsposition by the Foundation of an equitable share of the net in- come from the sale of publications developed under the Foundation's energy policy grants to the university. -- Specifically provide for the disposition of income in the energy policy grant to the university, if the grant is renewed. We found that payments charged to the Foundation principal investigator's salary were in accordance withforthethe budgets approved by the Foundation for its initial grant and renewal and the percentage of the researcher's time to be devoted to the Foundation's grants. We did not find any evidence in the Foundation's or the university's grant records that Foundation employees served as consultants. Foundation policy prohibits its employees from serving as consultants under its grants but allows other Federal employees to do so. Other Federal employees were used as consultants and their services were approved by their superiors. As you know, section 236 of the Legislative Reorganiza- tion Act of 1970 requires the head of a Federal agency to submit a written statement on actions taken on our recom- mendations to the House and Senate Committees on Government Operations not later than 60 days after the date of the B-133183 report and to the House and Senate Committees on Appropria- tions with the agency's first request for appropriations made more than 60 days after the date of the report. We will be in touch with your office in the near future to arrange for release of the report so that the requirements of section 236 can be set in motion. S yours, Comptroller General of the United States Enclosure 9 ENCLOSURE I ENCLOSURE I MANAGEMENT OF A NATIONAL SCIENCE FOUNDATION OFFICE OF LNERGY R&D POLICY GRANT TO THE GEORGE WASHINGTON UNIVERSITY: QUESTIONS, ANSWERS, AND RECOMMENDATIONS INTRODUCTION On August 24, 1376, the Chairman, Special on the National Science Foundation, Senate Subcommittee and Public Welfare, requested that we Committee on Labor review the procedures used by the Foundation in granting $130,000 to the George Washington University, Washington, D.C., of a series of papers on Federal oil and for the preparation papers were to provide information and gas policies. These analyses to assist tne President's Science Adviser (the Foundation's Director) in recommending appropriate Federal actions in the oil and gas segment of energy policy. Dr. William A. principal investigator under the grant. Johnson was the The Chairman's primary concerns were that: -- The grant application said there would be other sources of support, but the Foundation made no effort mine the sources of outside funding. to deter- -- Subcommittee inquiries showed the outside funding came from elements of the oil industry holding positions on divestiture and other subjectswell-defined searcher's inquiries. of the re- - Among the Federal employees selected by to review the research proposal, two had the Foundation worked under the researcher and another had een his superior. -- The opinion of only one reviewer who was ployee of the Treasury Department or the not an em- Foundation was sought before the grant award. -- The oil industry used the researcher's work in its advertisements urging opposition to divestiture, the work did not disclose all sources of funding. but Pursuant to the Chairman's letter and later agreements with his office, we: 1 ENCLOSURE I ENCLOSURE I -- Examined the policies and procedures used by the Foundation's Office of Energy R&D Policy in awarding the initial grant in 1974 and a 1975 renewal as well as policies and procedures currently used in evaluat- ing research proposals. --deermined if the peer reviewers selected to evaluate the grant proposals included representatives of organizations likely to be interested in the research and examined the guidance given the reviewers and the Foundation's disposition of their comments. -- Determined the extent that the Foundation considered utsiae funding in reviewing the grant proposals and Identified the applicable Foundation policies and procedures. -- Determined whether the research papers prepared under the grants met the grant objectives. -- Ascertained whether the Foundation evaluated the re- search papers. -- Determined the Foundation's planned use of the re- search papers and the general use and distribution of them. -- Checked all research papers prepar:ed under the grants for compliance with Foundation policy concerning ac- knowledgment of support and considered the policy'L adequacy for informing research users of all sources of funding. -- Determined the disposition of income from the sale of research papers prepared under'the grants. -- Determined whether the salary paid the researcher wax in accordance with the approved grant budgets and whether the researcher's use of Federal employees as consultants was in accordance with Foundation require- ments. We examined the Foundation's project records for the grants and the grantee's records at the George Washington University. We also interviewed present and past Foundation officials responsible for administering the grants, the prin- cipal investigator, grant officials at the university, and Treasury Department officials about matters related to its cost sharing on the renewal grant, 2 ENCLOSURE I ENCLOSURE I The Foundation funded the grants under the authority the National Science Foundation Act of 1950 (42 of U.S.C. 1361 et seq.), as amended, which authorizes the Foundation tiate and support (1) basic scientific research, to ini- (2) applied research, and (3) programs to increase research through contracts or other forms of assistance, potential such as grants. The grants were awarded and administered Foundation's Office of Energy R&D Policy. by the The Office was created in the summer of 1973, organization Plan No. 1 abolished the Office of when Re- Science and Technology and the Foundation's Director was additional responsibilities of serving as the assigned the President's Science Adviser. The major purpose of the Office of Energy R&D Policy was to provide the Director, in his Science Adviser, an independent source of advice role as and analysis of energy research and development and other energy-related programs for use by the Executive Office of the President. The Office was also responsible for gathering a wide range of information and advice in energy-related matters hancing the Foundation Director's capability and en- to respond ef- fectively to other requests for policy analyses of energy issues. The Office was part of the Foundation Director's until July 1975, when it began reporting to the office newly established Directorate 1/ for Scientific, Foundation's and International Affairs. Technological In February 1976 its functions were placed in the Division of Policy Research and Analysis within the Directorate and .:he Office was abolished. Funding for the Office and $9 million from its inception in its successor totaled over 197,3 through August 15, 1976. During this period about 123 grants, renewals, contracts were funded in support of research and activities. One of these projects was the "Energy Policy Problems" grants, which grew out of an unsolicited proposal sent to the Founda- tion by the George Washington University. Under the principal investigator, Dr. William A. Johnson, the grants marily to prepare a series of discussion papers was pri- about various energy issues, focusing primarily on Federal oil and gas policies. The initial Foundation grant of $60,000, made September 5, 1974, was to cover from September on 1, 1974,through August 31, 1975. On June 30, 175, the Foundation an additional $70,000 to support the project provided for another 1/The Foundation has six major organizational units (called directorates) for carrying out its research activities. 3 ENCLOSURE I ENCLOSURE I 12-month period beginning September 1, 1975. This amount included $20,000 transferred from the Treasury Department to help support the project. As of January 7, 1977, the Founda- tion was considering a June 19'6 request from the uiiversity for an additional $35,000 to support the project for 12 more months. REVIEW AND EVALUATION OF THE UNIVERSITY'S RESEARCH PROPOSAL "ENERGY POLICY PROBLEMS" There are no Foundation-wide written or formal proce- dures governing the process for evaluating research proposals. Each Foundation directorate has established its own proce- dures to review and evaluate proposals. These procedures vary within each directorate, but they are generally struc- tured around and conform with requirements of the various Foundation policy and instructional directives, such as (1) Office of the Director staff memorandums, (2) Offic of General Counsel bulletins, (3) Foundation circulars, and (4) Foundation important notices, which concern various as- pects of proposal evaluation and award of grants. The review process common to each directorate for unsolicited research proposals generally is as follows. Unsolicited proposals are funding requests sent to the Foundation largely on the proposers' initiative. Such pro- posals are usually submitted in response to general Founda- tion literature or as a result of personal contacts with the Foundation's program officials. Proposals are prepared by the principal investigator, approved by his sponsoring insti- tution, and sent to the Foundation. Program managers respon- sible for particular program areas, such as energy policy, productivity, or computer science, review and evaluate the proposals falling within their areas. Generally, proposal review and evaluation involves the following major steps: (1) initial determination whether the proposal is potentially supportable, (2) formal review by the program manager and others, usually outside peer reviewers, (3) evaluation of the review comments and a determination by the program manager to recommend or decline funding of the proposal. (The Foundation decides whether a proposal will be funded.) The program manager's evaluation and recommenda- tion to fund the proposal are subject to review by the sec- tion head, division director, and the assistant director in charge of the directorate. Grants are made to the sponsor- ing institution to support a specific project proposed by an investigator. 4 ENCLOSURE I ENCLOSURE I All awards must be approved by the Foundation's Director Awards or his designee and the Grants and Cntracts Officer. or at involving expenditures of at least $500,000 in a year least $2 million in total must also be approved by the Na- tional Science Board. Projects are generally funded annually. Renewal proposals requesting additional support usually undergo a similar review process. At the time the university's research grants were awarded, the Office of Energy R&D Policy was not part of a research directorate and did not have written procedures or uniform practices governing the review and evaluation of re- search proposals. The review methods used were selected by the program manager responsible for the subject area of the proposal. For some proposals, the Foundation's traditional peer review system of using outside reviewers was used. For other proposals, where the information was needed to quickly respond to other agency requests for policy information, the traditional system was considered too slow and the proposals were reviewed internally by the Office staff. Origin and contents of initial university proposal According to Dr. Johnscn and the individual who was the Office's Deputy Director in 1974, they had a preliminary discussion in May 1974, after which Dr. Johnson furnished the Office with a description of the research grant he was seeking. By letter dated June 19, 1974, the Office advised Dr. Johnson that, as previously discussed, his grant proposal should provide for consultation and engaging in discussions with Foundation staff, including maintaining contact with the staff about energy-related issues. As stated in the letter,a his primary task would be to prepare a series of papers on wide range of topics, drawing upon his experiences in the Federal Energy Office and the Treasury Department and from his independent research. The letter suggested certain topics for submission to the Office, including: -- Problems of State and local participation in energy resource development. -- Policies essential to achieving reasonable U.S. self- sufficiency in energy by 1980; some problems confront- ing Project Independence. ENCLOSURE I ENCLOSURE I -- Use of incentives to the private sector for developing energy resources. -- Ability of the Federal bureaucracy to conceive a na- tional energy policy. -- Problems in U.S.-Canadian energy relations. The letter stated that other topics could be pursued if the Foundacion agreed to them. The proposal submitted to the Office on July 22, 1974, consisted of 21 pages; 2 pages generally described the work to be performed under the grant and identified five discussion papers to be prepared; 11 pages contained outlines of the five discussion papers; 1 page contained a grant budget; and 7 pages gave the background of the principal investigator. The scope of the proposal essentially restated or refined the contents of the June 19, 1974, preliminary letter from the Office. The proposal provided f a series of discussion papers on a wide range of energy issues, focusing primarily on Federal gas and oil policies. The proposal specifically identified the five toplos stated in the June 19, 1974, pre- liminary letter as topics of papers to be prepared. The pro- posal also called for mutual agreement about papers on other topics to be prepared, and it identified several possible topics not included in the preliminary letter. The proposal also provided for consultation and ongoing contact with the Office about energy-related issues, as provided for in the letter. Review of the initial proposal The Foundation uses various procedures in deciding whether to fund the proposed research project. A widely used procedure involves soliciting written comments by mail from outsiders in academic institutions, government, and in- dustry who are experts in the area covered by the proposal. These experts are the "peers" of the investigator whose pro- posal is being considered, and the solicitation of their views as input in evaluating the proposal is called the mail peer review system. When the initial proposal was reviewed, there were no Foundation-wide criteria or procedures governing the use of mail peer review, the selection of peer reviewers, or the number to be used. Although each directorate used mail peer review to evaluate proposals, decisions as to the use of the system were made by responsible program managers in accordance with the generally accepted practice in each directorate. 6 ENCLOSURE I ENCLOSURE I The FoundaLion's project folder for the grant showed that the Office program manager selected three reviewers to evaluate the initial roposal in July 1974. Two of the re- viewers were Foundation employees--one was from another foundation directorate and had worked for Dr. Johnson at the Federal Energy Office before joining the Foundation and one was an Office of Energy R&D PolicyVistaff member--and the other reviewer was from the academic community. In addition, the program manager responsible for reviewing the proposal had worked under Dr. Johnson at the Treasury Department. When the proposal was submitted and was being processed, he was on loan to the Foundation from Treasury. One of the Foundation employees who acted as reviewers had a masters in electrical engineering and a Ph.D. in plan- ning theory and systems analysis; the other had a masters in geology and a Ph.D. in mineral economics. The program manager had a masters in mineral economics. Since the work was going to be done over a 12-month period, reviewer selection could have been expanded to include & number of private and public organizations that might have been interested in reviewing the proposal. Private organiza- tions that might have been interested include the American Gas Association, the American Petroleum Institute, oil market- ing associations, and individual petroleum companies. On the public side, State and local agencies and a number of Federal agencies concerned with energy matters, such as the Depart- ment of the Interior, Federal Power Commission, and Federal Energy Administrations might have been interested. Content and review of grant renewal proposal The renewal proposal that the university submitted to the Foundation on May 5, 1975, consisted of nine pages: five pages generally described the work to be done, includ- ing summaries of five additional discussion papers to be prepared; one page was a grant budget, and three pages con- tained a progress report of work under the nitial grant. The progress report said that the following five dis- cussion papers had been prepared and submitted to the Foundation: -- "Trends in World Oil Prices and Production" (Oct. 8, 1974). 7 ENCLOSURE I ENCLOSURE I -- "The Case Against Further Regulation of the Major Oil Companies" (Dec. 27, 1974). -- "Federal Energy Policy: A Conflict of Interests" (Jan. 20, 1975). -- "The Naval Petroleum Reserves" (Jan. 27, 1975). -- "Some Comments on the President's Energy Program and Alternatives to It" (Feb. 20, 1975). Several other papers were identified as under preparation: -- "U.S. Price and Allocation Controls and Their Impact on U.S. Energy Shortages." -- "The Two-Tier Price System and the Entitlements Pro- gram." -- "Regulation of Natural Gas at the Wellhead." -- "U.S. Policy Toward Coal." -- "State and Local Participation in Energy Resource Development." -- "U.S.-Canadian Relations on Energy Policy Matters." The proposal stated that a series of discussion papers would be prepared that would be useful to Federal energy policymakers (to be submitted to the Office and the Treasury Department). The papers were to survey policy-oriented research on selected energy topics to assess the merits of the research and to determine what additional research was needed. The papers were also to examine and assess ongoing Foundation-supported studies of various energy policy issues. Topics of these papers included: The Petro-dollar Problem; The Future of OPEC [Organization of the Petroleum Exporting Countries]; The Arab Oil Embargo: Past and Future; Multi- national Companies; and Oil Import Quotas. Also, the pro- posal said papers on other tpics of interest to the Founda- tion and the Treasury Department could be prepared and con- sultant service could be expanded to include Treasury. The Foundation's project folder showed that in May 1975 the Office program manager selected eight reviewers--three from Treasury and five from the Foundation--to evaluate the renewal proposal. Of the six reviewers submitting evaluative comments, five were Foundation employees--two were from other 8 ENCLOSURE I ENCLOSURE I Foundation directorates and had worked under Dr. Johnson at other Federal agencies before joining three were Office staff members--and the Foundation and the other reviewer (a Treasury employee) was a former associate at Treasury. of Dr. Johnson's In addition: -- One Office staff member acting as a reviewer for the renewal proposal had been the program manager for the initial proposal. -- The Office program manager for the renewal proposal had been a reviewer of the initial proposal. -- In approving the award within the Office, the program manager, program director, and deputy signed off on the award were the same director who person. The five Foundation employees who acted the following educational backgrounds: as reviewers had (1) a Ph.D. in nuclear science, (2) a Ph.D. in economics, (3) a masters in mineral economics, (4) a Ph.D. in nuclear physics, in geology and a Ph.D. in mineral economics.and (5) a masters manager had a masters in electrical The program engineering and a Pn.D. in planning theory and systems analysis. Rationale for theproposal review procedures A September 1976 Foundation managemert cluded the Office's operating procedures study that in- peer review procedures were regarded said the traditional as inappropriate for much of the policy analysis carried out by the Office. This was because the analysis was mostly intended to quickly re- spond to specific requests from policy decision offices, such as the White House offices, the Office get, and other Federal agencies. of Management and Bud- The study also said that external peer review--a process which usually takes 2 to 4 months--was considered inappropriate to meet deadlines often stated in days or weeks. Further, according to former Office officials, research proposal handled by Office staff members who reviews were largely understood the need for the research and were often the users of the results. The university's proposal was unsolicited; mitted after Dr. Johnson was told it was sub- by an Office official that the Office funded energy policy research. contains no documentation indicating The project folder that the proposed reseaLch--meetings, consultations, and research papers--was in response to a specific request from any source outside 9 ENCLOSURE I ENCLOSURE I the Foundation. Further, both the initial and renewal grants were made for 12-month periods. The review and evaluation time required to process Dr. Johnson's initial proposal totaled 1 month and 13 days from the date of receipt to the date of award. The review and evaluation time required to process the renewal proposal otaled 1 month and 25 days. Instructions on peer reviewer selection As previously stated, at the time of the initial grant and renewal to the university, there were no Foundation-wide instructions on selecting peer reviewers. On February 13, 1976, the Foundation issued "Guidelines for Program Officers on Selection of Reviewers"--Foundation-wide instructions that provide uniform detailed criteria to guide program managers in selecting peer reviewers. The criteria are aimed at pro- viding a representative selection of peer reviewers who -- can evaluate the principal investigator's ability; -- have broad view of the subject matter; -- can comment on the usefulness of the proposed work; and -- represent the institutional, educational, regional, or other elements of the community that might be affected by the project. Included in the guidelines are instructions which stress the need to provide balance among differing points of view, call attention to the possibility of scientific and personal biases, and caution against involving reviewers in potential conflicts of interest. Guidelines given to reviewers for proposal review Officials of several of the Foundation's directorates said that, when requesting outsiders to review and evaluate research proposals, the Foundation usually sends a copy of the research proposal, a comment or rating sheet, and general guidelines for the reviewer to use in evaluating the proposal. Such guidelines generally request the reviewer to consider such factors as the merit of the proposal, the feasibility and effectiveness of the proposed work, the reasonableness of the project budget, and the qualifications of project per- sonnel. The project folder for the initial proposal and the renewal showed reviewers were given the proposal and a rating 10 ENCLOSURE I ENCLOSURE I sheet. The folder contained no documents indicating that review guidelines were furnished. On February 13, 1976, the Foundation issued "Guidelines on Instructions to Reviewers"--Foundation-wide guidelines that set forth standards and requirements applicable to re- quests for peer reviews, including a requirement that peer reviewers be given criteria for judging the proposal. Disposition of reviewers' comments Reviewerst comments are an important factor considered by the program manager in deciding whether to support a re- search proposal. The program manager assesses reviewers' evaluative comments and recommendations and their impact on the proposal. Generally, any aspects that are counter to Foundation policy or that might hinder the successful com- pletion of the project are identified and resolved before a recommendation regarding funding is made. The proposal rating sheets for the initial and renewal proposals submitted by the university sho% hat, although the proposals were rated excellent and very good, the narrative cuzments of several reviewers included questions about the work to be performed. For example, these reviewers: -- Felt that the study's scope might be too broad to be covered in the proposed 10 to 12 papers (initial pro- posal). -- Believed that several proposed papers might be over- ambitiois in light of the time covered by the grant and cited two papers as examples (initial proposal). -- Suggested that the study be made more comprehensive and identified several additional topics for possible inclusion (renewal). -- Questioned whether the Foundation had verified that parallel work was not already underway in other in- stitutions, thus avoiding unnecessary duplication of effort (renewal). The proposal review worksheets and other documents in the project folder include no evidence indicating the disposi- tion of these questions by the program managers. A comment by one reviewer relates to one of the specific issues we are dealing with in this inquiry--outside funding 11 ENCLOSURE I ENCLOSURE I sources. Both the initial proposal and renewal stated that there would be outside support for the project, but neither identified the source. The initial proposal said that the Foundation's grant would cover about three-fourths of the principal investigator's time and that he would receive sup- port from other sources to cover the rest of his time (a similar statement appeared in the renewal proposal). Apparently none of the reviewers asked about outside funding when considering the initial proposal. In commenting on the renewal proposal, one reviewer--an Office official-- stated: "'* * * My only question concerns the de sup- port from another source (this is mentil,,ed on page 4.) What is it, how is it provided and for what, and does it assuredly provide for independ- ence in study? This question should be clarified before an award is made." Documents in the project folder do not show what the program manager did to resolve the reviewer's questions concerning outside support. The program manager for the renewal pro- posal said he did not remember how he disposed of the re- viewer's comment about outside funding. He said he probably called the principal investigator about his other source of support. However, he said other support was not a principal concern of the Office because Foundation grantees commonly obtained other support. According to Dr. Johnson, at the time of the grants the Foundation did not inquire about the specific ource of his outside funding. The Foundation has since acted to provide for better documentation of the management decisions regarding individ- ual projects. Action review boards 1/ have been established in each directorate (the Office's functions are now part of a directorate) to review awards. One review factor to be con- sidered is whether full and adequate consideration is given to reviewers' comments. Also, in August 1976 Foundation-wide instructions were issued prescribing the minimum levels of l/The Directorate fr Scientific, Technological and Inter- national Affairs' action review board was formalized on January 23, 1976. Its functions were to include a review of all proposed awards after recommended approval by the division director. Board membership includes top manage- ment of the directorate and legal and business representa- tives from other Foundation offices. 12 ENCLOSURE I ENCLOSURE I documentation required for award recommendations. The instructions require that the proposal review worksheet summarize the pertinent factors leading to the program manager's recommendation. The instructions outline several considerations to be included in the summary; for example, the summary is to indicate whether the principal investiga- tor satisfactorily responded to major review criticisms. Disclosure of other funding sources Several of the research directorates have published brochures or guidelines that inform researchers of the requirements and general format for research proposals. One requirement is that the proposal list all current re- search, in addition to the proposed project, to which the principal investigator and other senior research personnel have committed part of their time and indicate whether the salary for the person involved is included in the budgets of the various projects. Such information should include the titles and dates of current grants or contracts, the source of funds, annual budget levels, and the time devoted to each project by each of the senior personnel. Such information is to be provided for all other proposed research which is being considered by, or which will be submitted in the near future to, other possible sponsors, including other Founda- tion programs. The Office had no such requirements when the awards were made to the university. The Foundation's grants to the university were not con- sortium grants, which involve a master grant to one institu- tion to support a project being carried out through a coopera- tive arrangement between the grantee and one or more academic institutions or profit or nonprofit organizations. Rather they were grants that generally provided full funding to the university for accomplishing the grant objectives outlined in the two research proposals, with the understanding that there would be some cost sharing by the university as re- quired for unsolicited research proposals by the Founda- tion's fiscal year 1975 appropriation act, from which the grant funds originated. Grant records at the university show that the Founda- tion was one of three sources giving grants to the university to support Dr. Johnson's research work. The records showed that for September 1, 1974, to August 31, 1975, the univer- sity received $110,000 to support Dr. Johnson's work. The funding consisted of the Foundation's grant for $60,000 and a grant by a group of oil marketing companies known as the Independent Oil Marketers' Conference for $50,000. For 13 ENCLOSURE I ENCLOSURE I September 1, 1975, to August 31, 1976, the university received $145,000 to support Dr. Johnson's work. The funding consisted of the second Foundation grant for $70,000 (which included $20,000 from the Treasury Department), a second grant by the Independent Oil Marketers' Conference for $35,000, and a grant by another group of marketers known as the Southern Caucus for $40,000. The grants from the marketing groups were given unconditionally to support Dr. Johnson's work. The university decided to use the grants to fulfill the cost- sharing requirements of the Foundation's grants for energy policy research. On September 10, 1976, the Foundation's Acting irector issued a directive requiring that all proposal-generation mechanisms used within the Foundation have applicants furnish information on current support and pending applications with other funding sources. The directorates are to require that detailed information on other funding sources and pending research applications be included as part of each research proposal. The directive acknowledges that awareness of other sup- port is an important factor in making awards, avoiding dupli- cate funding of projects, and ascertaining that the proposers have not overcommitted themselves to other projects and duties. However, it does not prescribe guidelines or cri- teria about what effect the source of the outside funding should have in the consideration of a grant prcposal. Conclusions The review process used for both proposals from the university did not use a broad representative selection of peer reviewers. The reviewers selected did not include policy decisionmakers or representatives from several govern- ment agencies and from private industry with knowledge of and a probable interest in energy policy. Office staff members were extensively used as reviewers, and some reviewers and the initial program manager had had a prior employment re- lationship with the principal investigator. In addition, the reviewers were not given criteria to guide their review. The research proposals had a broad charter, the research was to be done over a long period of time, and no evidence indicated that the research was i- response to an urgent request from an outside source. Tnase facts do not appear to fit the Office justification for not using the external peer review system. 14 ENCLSOURE I ENCLOSURE I The reviews of the initial proposal and the renewal were favorable; however, the grant records were not documented to show the disposition of questions raised. Management instructions issued and organization changes m since the grants were awarded should help solve some p. ems we identified in the peer review process. These changes include (1) issuing Foundation-wide guidelines on peer reviewer selection, including instructions to the reviewers, (2) transferring the Office's functions to a research direc- torate, which will provide additional management levels of grant review and a more formal grant review process, and (3) creating an action review board within each directorate to review decisions on grant awards, including the documenta- tion supporting these decisions. The Foundation's directive concerning awareness of other sources of support for principal investigators does not ad- dress the question of what impact such support will have on the Foundation's grant award decision. If the other sources of funding indicate that the researcher is overcommitted or has a similar proposal under consideration elsewhere, prudent management would seemingly dictate the actions to be taken. However, the source of the outside funding can also be an issue to be considered in the grant award decision. This issue is more difficult to resolve and the Foundation has no written policy to help resolve it. Outside funding from organizations that could be affected by the research might not influence the researcher in making his study. However, it could lead to conflict-of-interest questions that might reduce the usefulness of the research. We recognize that obtaining experts to do sensitive policy research who have not been supported or otherwise affiliated with a concerned industry or organization might be difficult. Thus, the source of outside funding to support a specific research proposal could be largely an academic question. Nevertheless, the Foundation should be cautious in funding researchers to do sensitive policy research and try to avoid the appearance of a conflict of interest. Using - broad selection of peer reviewers, including representatives-from concerned public and private organizations to obtain a balance of points of view should be common practice in evaluating re- search proposals. Such a broad selection of peer reviewers can also help to respond to conf ict-of-interest questions that might arise when supporting sensitive policy research. The reviewers could also participate in a peer evaluation of the publications resulting directly from the Foundation- supported policy researcil. (See p. 18.) 15 ENCLOSURE I ENCLOSURE I Recommendation We recommend that the Director of the Foundation issue a policy directive providing guidance on how the source of out- side funding should be considered in a grant award decision. ACCOMPLISHING GRANT OBJECTIVES The objectives of the university's research grant were broad. In addition to preparing a series of discussion papers on a wide range of energy issues focusing primarily on Federal gas and oil policies, the principal investigator was to consult about energy-related issues. The Foundation did not maintain records to show the re- ceipt of papers prepared under the grants. It prepared a list of papers received under the grants for Chairman Kennedy and later prepared a list for us. The lists did not completely agree. Dr. Johnson and one of his assistants said that the following papers were prepared and issued under the Founda- tion's initial grant and renewal to the university. Title Issue date 'Trends in World Oil Prices and Production" Oct. 1974 "The Impact of rice Controls on the Oil Industry" Oct. 1974 "The Case Against Further Regulation of the Major Oil Companies" Dec. 1974 "The Naval Petroleum Reserves" Jan. 1975 "Federal Energy Policy: A Conflict of Interests" Jan. 1975 "Some Comments on the President's Energy Pro- gram and Alternatives to It" Feb. 1975 "Pending Energy Legislation" July 1975 "How Federal Regulations are Putting Independent Oilmen Out of Business" Sept. 1975 "Why U.S. Energy Policy Has Failed" Jan. 1976 "Coal Policy or the Lack Thereof" Mar. 1976 "Competition in the Oil Industry" Apr. 1976 "The International Implications of the Vertical Divestiture of U.S. Oil Companies" May 1976 "The Petrodollar Problem" July 1976 Dr. Johnson said Office officials informally approved the selection of papers that he prepared and the changes from the 10 papers identified in the grants. In October 1976 16 ENCLOSURE I ENCLOSURE I Dr. Johnson advised us of the following disposition of the 10 papers. Subject Disposition Problem of State and loccal par- Not written because it ticipation in energy resource became less of an development issue. Project Independence Not written, but parts appear in other papers. Private sector incentives in Issued as impact of price energy development controls (Oct. 1974), testimony before House Ways and Means Committee (Jan. 1975). National energy policy Issued as why U.S. energy policy failed (Jan. 1976). U.S.-Canadian energy relations Not written, but testimony given before Senate Sub- committee on Budgeting and Management (Nov. 1974). The petrodollar problem Issued (July 1976). Future of OPEC Included in paper on inter- national implications of vertical divestiture of U.S. oil companies (is- sued May 1976). Arab oil embargo In draft (as of Oct. 20, 1976). Multinational companies In draft (as of Oct. 20, 1976). Oil import quotas Not written, became a non- issue, and other studies existed. The Foundation's grant folders contain no documentation to show its approval of the changes. The program manager for the initial grant could not be reached for comment on the changes, and the program manager for the grant renewal agreed that changes Zfom the initial grant and the renewal were ap- proved informally. Concerning the consultation, Foundation grant records show that Dr. Johnson attended several Office staff meetings, participated in a discussion seminar at the Foundation in January 1975 concerning the petroleum industry market struc- ture, and presented the results of some research at the Fed- eral Trade Commission in April 1976. 17 ENCLOSURE I ENCLOSURE I Standardizing project folders In August 1976 the Foundation issued instructions to standardize the format and content of its grant folders. The instructions require that final fiscal and technical reports and other publications resulting from the grant be filed in the grant folder. Evaluation and use of research results Evaluation is the principal means of determining whether grant objectives have been accomplished and of assessing the quality of research products. There are no Foundation-wide requirements that the results of research grants be formally evaluated. Individual research directorates have the lati- tude to determine how research performance and results are evaluated. According to the Office's former Director, the crite- rion for an overall evaluation of the papers submitted by Dr. Johnson would be whether the Office staff could use them. The grant folders neither contain a record of any evaluation of the papers nor show what use was made of them. The pro- gram manager for the grant renewal said that the Office was the intended immediate user and that the papers were cir- culated within the Foundation. Also, copies were given to the Office of Management and Budget, the Treasury Department, and other interested organizations. Dr. Johnson did not main- tain distribution records but said about 7,200 copies of the monograph concerning competition in the oil industry were distributed. On March 8, 1976, the Foundation's Director issued an instruction requiring that each directorate arrange for the dissemination of substantial technical reports, including those involving policy and applied research, through the De- partment of Commerce's National Technical Information Service. According to an official of the Directorate for Scientific, Technological and International Affairs, none of the papers submitted to the Foundation under the university's initial grant and renewal were sent to the Information Service, but the Directorate is now considering doing so. Conclusions The broad grant objectives of preparing a series of discussion papers and consulting and otherwise maintaining contact with the Foundation were met. Because of inadequate Foundation grant records, we cannot determine whether the 18 ENCLOSURE I ENCLOSURE I specific discussion papers prepared by the researcher are those agreed to by the Foundation. The quality of the dis- cussion papers has not been formally determined by the of the papers to the Office Foundation, and the usefulness documented in the staff, the apparent intended users, is not Foundation's records. Management controls, including organization changes, established since the grants were awarded to the university will provide for improved grant administration, including activities and records of better documentation of project policies and pro- papers submitted under a grant. However, research supported by cedures for formally evaluating policy the Directorate for Scientific, Technological and Interna- tional Affairs do not exist. We believe that decisionmakers, the ultimate users of the quality of information they policy research, should know make are receiving and that certainly the Foundation should sure of the quality of information it passes on to them. from Documented evaluations of publications resulting directly to imple- Foundation-supported policy research are necessary ment these quality controls. In the case of Dr.a Johnson's panel which research, a review of the discussion papers by included representatives of the interested public and private organizations might have accomplished this purpose. Recommendations We recommend that the Director of the Foundation: -- Establish formal policies and procedures for the evalu- ation of policy research, including peer reviews of the research proposal that provide a balance of points of view, and a timely postevaluation by peers of publica- tions resulting directly from Foundation-supported research. -- Require that eyaluations of Foundation-supported policy research be made available for distribution through the National Technical Information Service. -- Provide for a formal evaluation of the discussion papers prepared under the Foundation's energy policy grants to the university. -- Determine the need for formal evaluation of other recently ccipleted policy research, considering the sensitivity of the subject matter and its distribu- tion and use. 19 ENCLOSURE I ENCLOSURE I ACKNOWLEDGMENT OF SUPPORT AND DISCLAIMER POLICY The grants to the university were subject to Foundation policy requiring that (1) an acknowledgment of Foundation support be made with the publication of any material based on or developed under the project and (2) any publications, such as monographs and books, produced under the grant state that any opinions, findings, conclusions, or recommendations are those of the author and do not necessarily reflect the Foundation's views. These requirements were not consistently applied to the research papers issued under the Foundation's grants to the universi:y, as shown below. Foundation Disclaimer support of Title acknowledged opinion "Trends in World C Prices and Production" No No "The Impact of Price Controls on the Oil Industry" No No "The Case Against Further Regula- tion of the Major Oil Companies" Yes (partial) Yes "The Naval Petroleum Reserves" Yes (partial) Yes "Federal Energy Policy: A Con- flict o Interests" Yes (partial) Yes "Some Comments on the President's Energy Program and Alternatives to It" Yes (total) No "Pending Energy Legislation" Yes (partial) Yes "How Federal Regulations are Put- ting Independent Oilmen Out of Business" Yes (partial) Yes "Why U.S. Energy Policy Has Failed" Yes (partial) Yes "Coal Policy or the Lack Thereof" Yes (total) Yes "Competition in the Oil Industry" Yes (partial) Yes "The International Implications of the Vertical Divestiture of U.S. Oil Companies" No No "The Petrodollar Problem" Yes (total) Yes Dr. Johnson said that he was not aware of the Foundation's policy when the early papers under the initial grant were is- sued nd that someone from the Foundation informed him later. He did not understand why the paper "The International Impli- cations of the Vertical Divestiture of the U.S. Oil Companies" did not contain the acknowledgment/disclaimer statement. 20 ENCLOSURE I ENCLOSURE I Dr. Johnson said he did not follow any prescribed format in writing the statement and did not check his prior papers for uniformity. Concerning acknowledgment of partial versus total support, he stated that (1) if the papers concerned international issues or issues of interest only to the Foundation, he acknowledged full support, and (2) if the papers were of mutual interest to the Foundation, the other grantors (Oil Marketers' Conference and Southern Caucus), and Treasury, he acknowledged partial Foundation support. For example, the paper Coal Policy or the Lack Thereof" acknowledged that it was written "under a grant from the National Science Foundation," and the paper "Competition in the Oil Industry" acknowledged that "This paper was written, in part, under a grant by the National Science Foundation." The Foundation's policy as stated in its grant adminis- tration manual provided only a suggested format for the re- quired statements. On May 27, 1976, the Foundation's Acting Director established a standard format for the tatements. The acknowledgment/disclaimer statements are required on all publications of the researcher resulting from the grant, but the disclaimer statement may be omitted from scientific ar- ticles or papers published in scientific journals. This is the same requirement that was applicable to the university grants. Procedures for the Foundation's program managers to obtain compliance have not been established. Disclosure of all funding sources Questions have been raised about the propriety of not disclosing all sources of funding for Foundation-supported policy research. Office officials responsible for the energy policy research grants to the university were not concerned with the other sources of funding because they believed they would not influence the outcome of the research. As of Janu- ary 7, 1977, the Foundation had not taken a position on this issue. / Conclusions The Foundation's policy requiring acknowlegment of sup- port and a disclaimer for content was not consistently com- plied with in the publications resulting from the Foundation's energy policy grants to the university. Foundation program managers, as part of their grant-monitoring responsibilities, should be required to examine publications resulting directly from Foundation-supported policy research before their public release to check for compliance with thi policy. 21 ENCLOSURE I ENCLOSURE I As previously recommended, the Foundation needs to issue a policy directive on how the source of outside funding should be considered in the grant award process. (See p. 16.) We believe outside funding from organizations that could be af- fected by the research results should be avoided if possible, because it could easily raise conflict-of-interest questions and reduce the credibility of the research. Disclosure of all sources supporting the research would serve notice to users who might believe the source of funding affects the outcome of the research. However, it should be recognized that a researcher not receiving direct outside support from interested sources to help do Foundation-supported research might receive payments from interested parties for speeches, consulting, or other services while doing a Foundation- supported project. Further, before being funded by the Foundation, the researcher might have received funding from interested sources for various services, and upon completing a Foundation-supported project, the researcher might again be employed by interested sources. As previously stated, we believe evaluation is the prin- cipal way to determine the quality of Foundation-supported policy research. Also, policy research publications directly resulting from Foundation support could cite a bibliography of the researcher's recent related publications that the in- terested user could obtain to learn about the researcher's prior positions. Recommendations We recommend that the Director of the Foundation: -- Require that program managers examine publications resulting directly from Foundation-supported policy research before their public release to check for compliance with the acknowledgment of support and disclaimer policy. ---Require that the Foundation's acknowledgment of sup- port policy be revised to require disclosure of all funding sources of Foundation-supported research. -- Determine the feasibility of requiring that policy research publications developed with Foundation sup- port include citations to related research of the principal investigator. 22 ENCLOSURE I ENCLOSURE I INCOME FROM SALE OF PUBLICATIONS The Foundation defines program income as gross revenues received by a grantee through activities under the grant. Income includes proceeds received by a grantee from the sale of books, monographs, and other material developed or pro- duced with Foundation support. Foundation policy provides that before a grant is awarded, consideration should be given to the grant's income potential. Also, considering the nature of the project, the purpose of Federal support, and the amount and source of ex- pected income, the Foundation should provide for the dis- position of the income in the grant. Foundation policy pro- vides that income would normally be maintained in a separate account and, as much as possible, used to offset costs other- wise allowable and chargeable to the grant, including expenses associated with the income-producing activity. If total in- come is estimated to be more than $10,000, the grant is to provide that income not used as provided for in the grant be returned. If total income is estimated to be less than $10,000, the grant may provide that icome not used be re- tained by the grantee for science or science education purposes. 1/ The Foundation's project folder for the university grants showed that the Foundation determined that, since the discus- sion papers were to be used by the Foundation and not sold for income, there was no income potential. Therefore, no specific provision was made in the initial grant or the re- newal for disposition of income. Grant records at the university showed that: -- Through September 30, 1976, the grantee had realized $14,826.59 in income from the sale of publications, of which $14,076.09 resulted from the sale of the monograph "Competition in the WO1 Industry" and $750.50 was from the sale of reproduced copies of research papers ($685.00 from "Competition n the Oil 1/Currently, the Foundation requires its share of royalties earned on sales of science education material developed with its support to be returned by the grantees for deposit in the Treasury. The requirement resulted from a 1975 agreement with the Chairman, Subcommittee on HUD-Independent Agencies, Senate Appropriations Committee, that the gross royalties (considered a profit) be returned. 23 ENCLOSURE I ENCLOSURE I Industry" and $65.50 from "International Implications of the Vertical Divestiture of U.S. Oil Companies".) -- Publication costs totaled $10,988.81 for the same period. Publication costs of the monograph totaled $10,465.26 and costs of reproducing copies of the research paper "Competition in the Oil Industry" that were sold totaled $523.55. -- Income from total sales exceeded the recorded costs of publication and reproduction by $3,837.78. -- All receipts from sales of publications were credited to the Independent Oil Marketers' Conference grant. -- All related costs, except $523.55 that was charged to the Southern Caucus grant, were charged to the In- dependent Oil Marketers' Conference grant. -- Other than a charge for the registration fee for copy- righting the monograph, no income or related expenses for publications that were sold were processed through the Foundation's grant. The Foundation's grant administration manual, which contains policies and procedures for grantees to follow and which was included as part of the grant to the university, provides under the section on contracts that: "* * * Commercial production and distribution of books, films or similar materials developed with the support of an NSF [National Science Founda- tion] grant may be undertaken only as described in a proposal which results in an NSF grant, or, if not so described, with prior NSF approval. * * *.. Dr. Johnson arranged for the production (printing, art work, etc.) and distribution of the monograph "Competition in the ( il Industry." He also handled arrangements for re- producing and distributing the other research papers. Neither the Foundation's nor the university's grant records document Foundation approval of these activities. Dr. Johnson said he probably mentioned the intended publication of the monograph to the Foundation. The program manager for the renewal said he did not know what was done, but he believes the Foundation would have approved because it encourages publication and distribution of research results. 24 ENCLOSURE I ENCLOSURE I University records show that the Foundation has contributed over 50 percent of the grant funds for the university's energy research prcject. As of August 31, 1976, the university's records of expenditures and encum- brances show that costs have generally been charged to the grants used by the university to support the project in pro- portion to their support. Consequently, the Foundation has been charged with about 50 percent of the total project costs. Conclusions The publication and distribution arrangements for re- search papers resulting from the Foundation's grants to the university were apparently not formally approved (in writing) by the Foundation. It is uncertain whether the arrangements were approved orally. Because of this informality, the Foundation lost control of income generated under the grant. Since the Foundation pays a large part of the research cost, and it might not have had the opportunity to consider the publication and distribution arrangements, it ought to be able to determine the disposition of at least some of the net income resulting from the sale of publications developed with its support. Sound business practices would dictate formal written Foundation approval of commercial publication and distribu- tion arrangements. The Foundation's program managers, through their monitoring practices, should be responsible for making certain that Foundation policies regarding publication and distribution arrangements and income from the sale of publica- tions are enforced. Recommendations We recommend that the Director of the Foundation: -- Require grant recipients, as a condition of the award, to (1) obtain the Foundation's written approval of commercial publication and distribution arrangements for publications resulting directly from the Founda- tion's support and (2) allow the Foundation to deter- mine the disposition of all income resulting from the sale of such publications. -- Consult with the university to providt for disposition by the Foundation of a& equitable shaLe of the net in- come from the sale of publications developed under the Foundation's energy policy grants to the university. 25 ENCLOSURE I ENCLOSURE I -- Specifically provide for the disposition of income in the energy policy grant to the university, if the grant is renewed. PAYMENT FOR SALARIES AND CONSULTANT SERVICES By agreement with the Subcommittee Chairman's office, we reviewed the salary payments to the principal investiga- tor under the Foundation's initial grant and renewal to the university. I For the initial grant period (September 1, 1974, through August 31, 1975), 75 percent of his salary was charged to the Foundation grant and 25 percent to the Independent Oil Mar- keters' Conference grant. For the renewal grant period (September 1, 1975, through August 31, 1976), 71.4 of his salary was charged to the oundation's grant, 15.8 percent to the Southern Caucus grant, and 12.8 percent to the In- dependent Oil Marketers' Conference grant. The salary pay- ments charged to the Foundation grants were in accordance with the budgets approved by the Foundation for its initial grant and renewal and the percentage of the researcher's time to be devoted to the Foundation's grants. By agreement with the Subcommittee Chairman's office, we also determined whether the grantee employed any Founda- tion personnel as consultants under the grant. Foundation policy provides that Federal employees--except Foundation employees--can be employed under its grants as consultants if they obtain their agency's approval and their services are provided outside regular working hours or while on leave from official duty. We did not find any evidence in the Foundation's or the university's grant folders that Founda- tion employees served as consultants under the grant. How- ever, other Federal employees did serve as consultants. One consultant, employed by the Treasury Department as an international economist in its Office of Energy Policy, was paid $360 during the first grant period to help do re- search and collect and interpret data. The Independent Oil Marketers' Conference grant was charged for these services. The Director of Treasury's Office of Energy Policy was employed as a consultant to write a chapter on Federal and State policies affecting the coal industry for inclusion in a planned book on energy policy problems. He was paid $400 during the first grant period and $1,600 during the second grant period, amounts which were charged respectively to the 26 ENCLOSURE I ENCLOSURE I Independent Oil Marketers' Conference grant and the Foundation's renewal grant (which included $20,000 trans- ferred to the Foundation from Treasury to support the grant). Treasury Department regulations concerning outside em- ployment (31 CFR 0.735-38) provide that: "Employees shall not engage in any outside employ- ment or other outside activities, with or without compensation, which (a) interfere with tha effi- cient performance of official duties, (b) might bring discredit n or cause unfavorable and jus- tifiable critic sm of the Government or (c) might reasonably result in a conflict of interest, or an apparent conflict of interest, with official duties and responsibilities. * * *" Further, the regulations (31 CFR 0.735-39) allow employees to accept engagements to speak, write, or teach provided that the information to be used was not obtained as a result of Federal employment, unless (1) the information has been made available to the public or will be made available upon re- quest or (2) a determination is made that using nonpublic information is in the public interest. Before release of a speech or article related to Treasury business, the employee must obtain Department approval. The official approving the employee's participation in the engagement will determine whether the activity will be performed as an official duty or in a private capacity. In the former case, the employee cannot accept outside compensation. In the latter case, the employee can accept outside compensation, but he may not use duty hours or Federal facilities. The Treasury employees obtained approval from their superiors to perform the above-described work as consultar.;s for the university's energy research project. Employee time and attendance records and billings for the consultant serv- ice show that the activities were performed outside of the employees' regularly scheduled Federal working hours. Ac- cording to a Department official, the chapter on the coal industry prepared by the Treasury employee was reviewed by the official who approved the employee's request to write the chapter. Approval was granted to the Treasury employee in Janu- ary 1975 to prepare the chapter in a private capacity. In June 1975, the Department transferred $'20,000 to the Founda- tion in general support of the energy policy grant to the university. The $1,600 payment to the employee charged to 27 ENCLOSURE I ENCLOSURE I the Foundation's grant was for vouchers submitted in December 1975 and May 1976. Treasury Department regulations concerned with outside employment (31 CFR 0.735-38 and 31 CFR 0.735-39) do not specifically prohibit an employee being paid in a private capacity for Department-financed activities. A Treasury official responsible for processing the transfer of funds to the Foundation said she was not aware of the consultant con- tract held by the Treasury employee. She viewed the situa- tion as questionable and said she would have referred it to the General Counsel. The Department's Counsel to the General Counsel on ethics matters advised us that the employee's con- sultant services were provided under contract to the univer- sity as a separate transaction from the Department's cost sharing and should be viewed that way. He added that in view of the total Federal funds for the renewal grant ($50,000 from the Foundation and $20,000 from Treasury), the $1,600 paid the Treasury employee is insignificant. A member of a Sena.or's staff was paid $2,000 during the first grant period to write a paper on the oil import control protram and various proposals to reinstitute import quotas. Payment was charged to the Independent Oil Marketers' Conference grant. About half of the money was earned before the person rsigned from the Senator's staff. Effective September 1, 1975, the person was employed full time by the university as a research associate on the energy policy re- search project. According to the Senator's administrative aide, the em- ployee who performed consulting services for the university's energy research project was allowed and encouraged to do outside consultant work or free lance writing. Further, we were advised that office records showed the employee on leave for 5 of the 8 Federal workdays for which consultant services were billed. Leave records for the other 3 workdays could not be located. 28
Management of a National Science Foundation Office of Energy Research and Development Policy Grant to the George Washington University: Questions, Answers, and Recommendations
Published by the Government Accountability Office on 1977-01-25.
Below is a raw (and likely hideous) rendition of the original report. (PDF)