oversight

Management of a National Science Foundation Office of Energy Research and Development Policy Grant to the George Washington University: Questions, Answers, and Recommendations

Published by the Government Accountability Office on 1977-01-25.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                         DOCUMENT RESUME
00167 - [A1051996]

[Management of a National Science Foundation Office of Enerly
Research and Development Policy Grant to the George ashing -on
University: Questions, nswers, and Recommendations]. HaD-77-38;
B-133183. January 25, 1977. Released January 27, 1977. 9 pp. +
enclosure (28 pp.).

Report to Sen. Edward . Kennedy, Chairman, Senate Committee on
Labor and Public elfare: National Science Foundation Special
Subcommittee; by Elmer B. Staats, Comptroller General.

Issue Area: Energy (1600).
Contact: Human Resources Div.
Budget Function: Natural Resources, Environment, and Energy:
    Energy (305).
Organization Concerned: National Science oundation; George
    Washington Univ.
Congressional Relevance: House Committee on Science and
    Technology; Senate Committee on Labor and Public elfare:
    National Scienci Foundation Special Subcommittee; Senate
    committee on Labor and Public Welfare.
Authority: National Science Foundation Act of 1950 (42 U.S.C.
    1861 et seq.).

          The National Science Foundation's (SF) Office of
Fnergy R and D Policy awarded a 1974 grant to George Washington
University for energy policy research. The initial gran' to the
unsolicited proposal and a 12-month renewal were awarded for a
series of discussion papers on Federal gas and oil. policies and
consultation.   Findings/Conclusions: Evaluation of the research
proposal was not done by qualified personnel, or under
established guidelines. Some of the reviewing personnel ad a
prior employment relationship with the principal investigator
(Dr.   illiam A. Johnson). Additional outside funding came from
two interest groups of oil arketing companies. The broad grant
objective of preparing the papers and aintaining contact with
the NSF were met, but their quality o utility were not
determined. Recommendations: NSF should issue a policy
directive on how outside funding should be considered in a grant
award decision to guard against potential conflicts of interest.
It should establish formal policy for evaluating research,
including peer reviews of proposals, and timely postevaluation;
make its research available through the National Technical
Information Service; formally evaluate discussion papers;
determine the need for formal evaluation of other recent policy
research; require a statement of Foundation support and a
disclaimer on its publications; and disclose all funding sources
of NSF supported research. Grantees should get formal approval
for commercia3 publication of research and the Foundation should
fix the disposition of moneys from the sale of publications.
(DJ)
                      COMPTROLLER GIMNRAL OF THE UNITEDX        AT8
            _Tw   s              WAHINGTONl   D.C.   B        UT




B-133183                                      JAr, 2 5 197i
The Honor:ble Edward M. Kennedy
Chairman, Special Subcommittee on
  the National Science Foundation
Committee on Labor and Public Welfare
United States Senate
i)ear Mr. Chairman:

     In accordance with your August 24, 1976, request and
later agreements with your office, we reviewed a number of
management issues concerning the National Science Founda-
tion's award and administration of a September 5, 1974,
grant to the George Washington University, Washington, D.C.,
for energy policy research. Dr. William A. Johnson was
principal investigator under the grant. As directed by your
office, we obtained the views informally only of Foundation
officials on our findings, and their comments are considered
in the report. OuL findings are summarized below and dis-
cussed in more detail in the enclosure.

     The initial grant to the university and a 12-month
renewal approved on June 30, 1975, were awarded by the
Foundation's Office of Energy R&D Policy.  At hat time the
Office was part of the Foundation Director's office.   Its
primary mission was to provide the Director, in his role as
the President's Science Adviser, with information on energy-
reiated matters. The grant resulted from an unsolicited
proposal y the university, requesting funds to suppoLt the
work of Dr. Johnson. Under the grant, he was to prepare a
series of discussion papers on Federal gas and oil policies
and to be available for consultation.  The proposal was pre-
ceded by a preliminary discussion between the Office and
Dr. Johnson, in which they agreed on the content of the pro-
posal to be submitted.

     The Office did not have written guidelines for evaluat-
ing research proposals. Review methods were determined by
the Founiation's program manager evaluating the proposal.
The Fourdation traditionally obtains views of outside ex-
perts in the subject area of the proposal; however, the
Foundation decides whether to fund the proposal.  The re-
viewers of the initial proposal and the renewal were largely


                                                                      HRD-77-38
B-133183


Office and other Foundation employees.  Except for Treasury
Department employees, the reviewers selected did not nclude
policy decisionmakers or representatives from other overn-
ment agencies and private industry with knowledge of and a
probable interest in energy policy research.  In addition,
some reviewers and the initial Foundation program manager
had had a prior employment relationship with the principal
investigator.

     A September 1976 Foundation management study which in-
cluded the Office's operating procedures stated that the
traditional peer review procedures were regarded as inappro-
priate for the policy analysis carried out by the Office.
This was because the Office usually had to respond quickly
to a specific request from a policy decision office, uch as
the White House. The study also noted that external peer
review, w!hich usually takes 2 to 4 months, was considered
inappropriate to meet deadlines that were often stated in
days or weeks. Further, according to former Office offi-
cials, research proposal reviews were usually handled by
Office staff who understood the need for the research and
were often the users of it.

     No evidence in the Foundation's grant records showed
that the proposed research was in response to a specific re-
quest from outside the Foundation. Apparently the Office was
the intended immediate user, although no formal Foundation
records show how the research was used. Also, regarding the
time frame, both the initial and renewal grants were made for
12-month periods.

     The reviewers were given the proposals and a rating
sheet, but were apparently not given guidelines to help them
consider the meritr of the proposals. Officials of several
Foundation directorates 1/ said they usually provide general
guidelines that ask the reviewer to consider such factors as
merit of the proposal, reasonableness of the budget, and
qualifications of the project personnel.  The reviewers rated
the initial proposal and the renewal high. However, the re-
viewers raised some questions and the disposition of the
questions is not documented in the Foundation's grant records.


 /rThe Foundation has six major organizational units (called
  directorates) for carrying out its research activities.




                              2
B-133183


     Management instructions issued and organization changes
made since the initial grant and renewal were awarded to the
George Washington University should improve the proposal re-
view process. These changes include (1) transferring the
Office's functions to the Directorate for Scientific, Tech-
nological and International Affairs, which will provide addi-
tional levels of grant review and a more formal review proc-
ess, (2) creating an action review board 1/ within each direc-
torate to review decisions on grant awards, including their
documentation, and (3) issuing oundation-wide guidelines on
peer reviewer selection, including instructions to the re-
viewers.
     Both the initial proposal and the renewal said there
would be outside support for the project, but neither iden-
tified the source. Apparently none of the reviewers of the
initial proposal asked about outside funding; however, a re-
viewer of the renewal proposal asked that the source be con-
sidered ir making certain that the study was independent.
The program manager could not remember how he handled the
comment; however, he believed that the fact of outside sup-
port was rot a principal concern, because Foundation grantees
commonly have other support. Dr. Johnson sid the Foundation
did not inquire about the source of his outside funding at
the time of the initial proposal or the renewal.
     The university's records show that for September 1,
1974, to August 31, 1975, te university receive $110,000
to support Dr. Johnson's work. The funds included the Foun-
dation's initial grant for $60,000 and a grant by a group
of oil marketing companies known as the Independent Oil
Marketers' Conference for $50,000. For September 1, 1975,
to August 31, 1976, the university received additional fund-
ing of $145,000. These funds consisted of the Foundation's
renewal for $70,000, 2/ a second grant by the Independent

1/The Directorate for Scientific, Technological and Inter-
  nat:ional Affairs' action review board was formalized on
  January 23, 1976. Its functions were to include a review
  of all proposed awards after recommended approval by the
  division irector. Board membership includes top manage-
  ment of t,, directorate and legal and business representa-
  tives from other Foundation offices.
2/Includes $20,000 transferred from the Treasury Department
  to the Foundation to help support the Foundation's energy
  policy research project.

                              3
B-133183


Oil Marketers' Conference for $35,000, and a grant by another
group of marketers known as the Southern Caucus for $40,000.
As of January 7, 1977, the Foundation was considering a June
1976 request from the university for an additional $35,000
to support the project for 12 more months.
     The grants from the marketing groups were given to the
university unconditionally to support Dr. Johnson's work.
The university decided to use the gants to fulfill the cost-
sharing requirements of the Foundation's grants for energy
policy research. The Foundation's grants to the university
used funds obtained under its fiscal year 1975 appropriation,
which required cost-sharing for wards resulting from un-
solicited research proposals.
     On September 10, 1976, the Foundation issued a directive
requiring applicants for funding to furnish information on
current support and pending proposals with other funding
sources. However, the directive does not prescribe guidelines
concerning the effect the source of the outside funding should
have when the proposal is considered. If the researcher is
overcommitted or has a similar proposal under consideration,
prudent management should dictate the actions to be taken.
The iopact of the funding source can be an issue more diffi-
cult to resolve.
      We recommend that the Foundation's Director issue a
 policy directive providing guidance on how the source of
 outside funding should be considered in a gran' award deci-
 sion. Outside funding from organizations that could be af-
 fected by the research might not influence the researcher in
,making his study. However, it could lead to conflict-of-
 interest questions that might reduce the usefulness of the
 research. We recognize that obtaining experts to do sensi-
 tive policy research who have not been supported or otherwise
 affiliated with a concerned industry or organization might be
difficult. Thus, the source of outside funding to support a
 specific research proposal could be largely an academic ques-
 tion. Nevertheless, the Foundation should be cautious in
 funding researchers to do sensitive policy research and try
to avoid the appearance of a conflict of interest. Improved
evaluation of the research proposal and evaluation of the
research publications resulting directly from Foundation-
supported policy research would also better insure that the
research is useful. (See the recommendations below concer,-
 ing evaluation.)



                              4
B-133183


     The broad grant objectives of preparing   series of
discussion papers and providing consultation and otherwise
maintaining contact with the Pundation were met. Because
of inadequate Foundation grant records, we could ret deter-
mine whether the specific papers prepared by the researcher
under the grant were those agreed to by the Foundation. The
quality c the papers had not been formally determined by
the Foundation, and the usefulness of the papers to the
Office staff, the apparent intended users, is not documented
in the Foundation's records.
     Recently established management controls will provide
for improved grant administration, including better documen-
tation of proje:t activities and records of papers submitted
under a grant. However, pclicies and procedures for formally
evaluating policy research supported by the Directorate for
Scientific, Technological and International Affair3 do not
exist. We believe that decisionmakers, the ultimate users
of policy research, should know the quality of information
they are receiving and tnat certainly the Foundation should
make sure of the quality of information it passes on to them.
Documented evaluations of publications resulting directly
from Foundation-supported policy research are necessary to
implement these quality controls.
    We recommend that the Directcr of the Foundation:
    -- Establish formal policies and procedures for the
       evaluation of policy research, inc'uding peer reviews
       of the research proposal that provide a balance of
       points of view, and timely postevaluation by peers of
       publications resulting directly from Foundation-
       supported research.
    -- Require that eluations of Foundation-supported
       policy research be made available for distribution
       through the National Technical Information Service.
    -- Provide for a fo:mal evaluation of the discussion
       papers prepared under the Foundation's energy policy
       grants to the university.
    -- Determine the need for formal evaluation of other
       recently completed policy research, considering the
       sensitivity of the subject matter and its distribution
       and use.



                             5
8-133183


     The grants to the university were subject to Foundation
policy requiring that (1) an acknowledgment of its support
be made with the publication of material based on or developed
under the project and (2) any publications, such as monographs
and books, produced under the grant state that the findings
and views ae those of the author and do not necessarily re-
flect the Foundation's views. The requirements were not
consistently applied i the publications issued under the
Foundation's grants to the university in that some papers
omitted one or both of the statements. Also, where the ac-
knowledgment of Foundation support was used, it stated the
work was supported in part by the Foundation, or the work
was supported in full by the Foundation.
     The Foundation's policy requirement, as stated in its
grant administration manual, provided only suggested state-
ments. To provide uniformity the Foundation has recently
established a standard format for the statements. Procedures
to make certain that the statements appear in publications
have not been established. We recommend that the Foundation's
Director require that program managers examine publications
directly resulting from Foundation-supported policy research
before their public release ro check for compliance with the
acknowledgment of support and disclaimer pclicy.
     Questions have beeli raised over the propriety of not dis-
closing all sources of funding for publications resulting from
Foundation-supported policy research. As of January 7, 1977,
the Foundation had not taken a position on this issue. liWe
believe cost-sharing obtained from sources that could be
affected by the research results should be avoided. Dis-
closure of all sources supporting the rsearch would serve
notice to users who might believe the source of funding af-
fects the outcome of the research. However, it should be
recognized that a researcher not receiving direct outside
support from interested sources to help do Foundation-
supported research might receive payments from interested
parties for speeches, consulting, or other services while
doing a Foundation-supported project. Further, before being
funded by the Foundation, the researcher might have received
funding from interested sources for various services, and
upon completing a Foundation-supported project, the researcher
might again be employed by interested sources.
     As previously stated, we believe evaluation is the prin-
cipal way to determine the quality of Foundation-supported
research. Also, policy research publications resulting
directly fr.m Foui._ation support could cite a bibliography of
the researcher's recent related publications that the
                              6
 B-133183


interested user could obtain to learn about the researcher's
prior positions.
     We recommend that the Director of the Foundation:
     -- Require that the Foundation's acknowledgement of
        support policy be revised to require disclosure of
        all funding sources of Foundation-supported research.
     -- Determine the feasibility of requiring that policy
        research publications developed with Foundation
        support include citations to related research of the
        principal investigator.
     The university realized about $15,000 from the sale of
publications developed partly under the Foundation's  energy
policy grants. Publication costs were about $11,000, leaving
a $4,000 net income. All costs and receipts were handled
through the Independent Oil Marketers' Conference grant,
except costs of about $525 charged to the Southern Caucus
grant. These grants were used by the university as the
required cost-sharing for the Foundation's grants. Nearly
all the income came from the sale of a monograph about
petition in the oi' industry, which was distributed by com-
Dr. Johnson. The related costs were largely for producing
the monograph, which was contracted for by Dr. Johnson.
      The Foundation, in reviewing the university's
proposals, apparently determined there was no incomeresearch
 tial because the discussion papers were to be used by poten-
Foundation. Therefore, no specific rovision was made the
                                                         in
the initial grant or the renewal for disposition of income.
Foundation income policy provides that normally income
used to offset costs otherwise allowable and chargeable be
the grant. If estimated total income is more than $10,000,to
the income not used as provided for in the grant would
returned. If estimated total income is less than $10,000,be
the grant may provide the unused income be retained
grantee for science or science education purposes. byThe
                                                         the
Foundation also requires prior approval of commercial produc-
tion and distribution of books, films, or similar materials
developed with its support. The publication and distribution
arrangements for publications resulting in part from the
Foundation's grants to the university were not formally
proved by the Foundation and might not have been approvedap-
orally.



                             7
 B-133183


     University records show that te Foundation has con-
tributed over 50 percent of the grant funds for the univer-
sity's energy research project and has been charged with
about 50 percent of the total project costs. Since the
Foundation pays a large part of the research costs and it
might not have had the opportunity to consider the publica-
tion and distribution arrangements, it ought to be able to
determine the disposition of at least some of the net income
resulting from the sale of publications developed with its
support.
     We recommend that the Director of the Foundation:
     -- Require grant recipients, as a condition of the award,
        to (1) obtain the Foundation's written approval of
        commercial publication and distribution arrangements
        for publications resulting directly from the Founda-
        tion's support and (2) allow the Foundation to deter-
       mine the diposition of all income resulting from the
        sale of sch publications.
     -- Consult wih the university to provide for dsposition
        by the Foundation of an equitable share of the net in-
        come from the sale of publications developed under the
        Foundation's energy policy grants to the university.
     -- Specifically provide for the disposition of income in
        the energy policy grant to the university, if the
        grant is renewed.
     We found that payments charged to the Foundation
principal investigator's salary were in accordance withforthethe
budgets approved by the Foundation for its initial grant and
renewal and the percentage of the researcher's time to be
devoted to the Foundation's grants.
     We did not find any evidence in the Foundation's or the
university's grant records that Foundation employees served
as consultants. Foundation policy prohibits its employees
from serving as consultants under its grants but allows other
Federal employees to do so. Other Federal employees were
used as consultants and their services were approved by their
superiors.
     As you know, section 236 of the Legislative Reorganiza-
tion Act of 1970 requires the head of a Federal agency  to
submit a written statement on actions taken on our recom-
mendations to the House and Senate Committees on Government
Operations not later than 60 days after the date of the
B-133183


report and to the House and Senate Committees on Appropria-
tions with the agency's first request for appropriations
made more than 60 days after the date of the report. We
will be in touch with your office in the near future to
arrange for release of the report so that the requirements
of section 236 can be set in motion.
                              S         yours,




                              Comptroller General
                              of the United States

Enclosure




                              9
 ENCLOSURE I
                                                     ENCLOSURE I


         MANAGEMENT OF A NATIONAL SCIENCE FOUNDATION

                 OFFICE OF LNERGY R&D POLICY

          GRANT TO THE GEORGE WASHINGTON UNIVERSITY:

           QUESTIONS, ANSWERS, AND RECOMMENDATIONS

 INTRODUCTION

     On August 24, 1376, the Chairman, Special
on the National Science Foundation, Senate       Subcommittee
and Public Welfare, requested that we       Committee  on Labor
                                       review the procedures
used by the Foundation in granting $130,000
                                              to the George
Washington University, Washington, D.C.,
of a series of papers on Federal oil and for the preparation
papers were to provide information and    gas policies. These
                                        analyses to assist tne
President's Science Adviser (the Foundation's Director) in
recommending appropriate Federal actions in the oil and gas
segment of energy policy. Dr. William A.
principal investigator under the grant.   Johnson was the

     The Chairman's primary concerns were that:
     -- The grant application said there would be
                                                   other sources
        of support, but the Foundation made no effort
        mine the sources of outside funding.           to deter-

     -- Subcommittee inquiries showed the outside
                                                   funding came
        from elements of the oil industry holding
        positions on divestiture and other subjectswell-defined
        searcher's inquiries.                        of the re-

     - Among the Federal employees selected by
       to review the research proposal, two had the Foundation
                                                 worked under
       the researcher and another had een his superior.
     -- The opinion of only one reviewer who was
        ployee of the Treasury Department or the not an em-
                                                 Foundation
        was sought before the grant award.
     -- The oil industry used the researcher's work
                                                    in its
        advertisements urging opposition to divestiture,
        the work did not disclose all sources of funding. but
     Pursuant to the Chairman's letter and later
                                                 agreements
with his office, we:



                              1
ENCLOSURE I                                       ENCLOSURE I



     -- Examined the policies and procedures used by the
        Foundation's Office of Energy R&D Policy in awarding
        the initial grant in 1974 and a 1975 renewal as well
        as policies and procedures currently used in evaluat-
        ing research proposals.

     --deermined if the peer reviewers selected to evaluate
       the grant proposals included representatives of
       organizations likely to be interested in the research
       and examined the guidance given the reviewers and the
       Foundation's disposition of their comments.
     -- Determined the extent that the Foundation considered
         utsiae funding in reviewing the grant proposals and
       Identified the applicable Foundation policies and
        procedures.
     -- Determined whether the research papers prepared under
        the grants met the grant objectives.
     -- Ascertained whether the Foundation evaluated the re-
        search papers.
     -- Determined the Foundation's planned use of the re-
        search papers and the general use and distribution of
        them.
     -- Checked all research papers prepar:ed under the grants
        for compliance with Foundation policy concerning ac-
        knowledgment of support and considered the policy'L
        adequacy for informing research users of all sources
        of funding.
     -- Determined the disposition of income from the sale of
        research papers prepared under'the grants.
     -- Determined whether the salary paid the researcher wax
        in accordance with the approved grant budgets and
        whether the researcher's use of Federal employees as
        consultants was in accordance with Foundation require-
        ments.
     We examined the Foundation's project records for the
grants and the grantee's records at the George Washington
University. We also interviewed present and past Foundation
officials responsible for administering the grants, the prin-
cipal investigator, grant officials at the university, and
Treasury Department officials about matters related to its
cost sharing on the renewal grant,
                              2
 ENCLOSURE I
                                                    ENCLOSURE I


      The Foundation funded the grants under the authority
 the National Science Foundation Act of 1950 (42            of
                                                  U.S.C. 1361
 et seq.), as amended, which authorizes the Foundation
 tiate and support (1) basic scientific research,       to ini-
                                                   (2) applied
 research, and (3) programs to increase research
 through contracts or other forms of assistance, potential
                                                  such as
 grants. The grants were awarded and administered
 Foundation's Office of Energy R&D Policy.          by the

      The Office was created in the summer of 1973,
organization Plan No. 1 abolished the Office of       when Re-
                                                  Science and
Technology and the Foundation's Director was
additional responsibilities of serving as the assigned the
                                               President's
Science Adviser. The major purpose of the Office
                                                    of Energy
R&D Policy was to provide the Director, in his
Science Adviser, an independent source of advice role  as
                                                   and  analysis
of energy research and development and other
                                              energy-related
programs for use by the Executive Office of the
                                                  President.
The Office was also responsible for gathering
                                               a  wide range
of information and advice in energy-related matters
hancing the Foundation Director's capability           and en-
                                              to respond ef-
fectively to other requests for policy analyses
                                                  of energy
issues.

     The Office was part of the Foundation Director's
until July 1975, when it began reporting to the        office
newly established Directorate 1/ for Scientific, Foundation's
and International Affairs.                        Technological
                            In February 1976 its functions
were placed in the Division of Policy Research
                                                and Analysis
within the Directorate and .:he Office was abolished.

      Funding for the Office and
$9 million from its inception in its  successor totaled over
                                  197,3 through August 15,
1976.   During this period about 123 grants, renewals,
contracts were funded in support of research              and
                                               activities.
One of these projects was the "Energy Policy
                                               Problems" grants,
which grew out of an unsolicited proposal sent
                                                  to the Founda-
tion by the George Washington University. Under
the principal investigator, Dr. William A. Johnson, the grants
marily to prepare a series of discussion papers        was pri-
                                                   about various
energy issues, focusing primarily on Federal
                                               oil and gas
policies.   The initial Foundation grant of $60,000, made
September 5, 1974, was to cover from September               on
                                                 1, 1974,through
August 31, 1975. On June 30, 175, the Foundation
an additional $70,000 to support the project          provided
                                               for another

1/The Foundation has six major organizational
                                              units (called
  directorates) for carrying out its research activities.

                              3
ENCLOSURE I                                       ENCLOSURE I



12-month period beginning September 1, 1975. This amount
included $20,000 transferred from the Treasury Department to
help support the project. As of January 7, 1977, the Founda-
tion was considering a June 19'6 request from the uiiversity
for an additional $35,000 to support the project for 12 more
months.

REVIEW AND EVALUATION OF THE
UNIVERSITY'S RESEARCH PROPOSAL
"ENERGY POLICY PROBLEMS"

     There are no Foundation-wide written or formal proce-
dures governing the process for evaluating research proposals.
Each Foundation directorate has established its own proce-
dures to review and evaluate proposals.  These procedures
vary within each directorate, but they are generally struc-
tured around and conform with requirements of the various
Foundation policy and instructional directives, such as
(1) Office of the Director staff memorandums, (2) Offic of
General Counsel bulletins, (3) Foundation circulars, and
(4) Foundation important notices, which concern various as-
pects of proposal evaluation and award of grants. The review
process common to each directorate for unsolicited research
proposals generally is as follows.

     Unsolicited proposals are funding requests sent to the
Foundation largely on the proposers' initiative. Such pro-
posals are usually submitted in response to general Founda-
tion literature or as a result of personal contacts with the
Foundation's program officials.   Proposals are prepared by
the principal investigator, approved by his sponsoring insti-
tution, and sent to the Foundation.   Program managers respon-
sible for particular program areas, such as energy policy,
productivity, or computer science, review and evaluate the
proposals falling within their areas.

     Generally, proposal review and evaluation involves the
following major steps:    (1) initial determination whether the
proposal  is  potentially supportable, (2) formal review by the
program manager and others, usually outside peer reviewers,
(3) evaluation of the review comments and a determination by
the program manager to recommend or decline funding of the
proposal.    (The Foundation decides whether a proposal will
be funded.)    The program manager's evaluation and recommenda-
tion to fund the proposal are subject to review by the sec-
tion head, division director, and the assistant director in
charge of the directorate. Grants are made to the sponsor-
ing institution to support a specific project proposed by an
investigator.

                                 4
                                                 ENCLOSURE I
ENCLOSURE I



     All awards must be approved by the Foundation's Director
                                                        Awards
or his designee and the Grants and Cntracts Officer.
                                                      or  at
involving expenditures of at least $500,000 in a year
least $2 million in total must also be approved by the Na-
tional Science Board. Projects are generally funded annually.
Renewal proposals requesting additional support usually
undergo a similar review process.

     At the time the university's research grants were
awarded, the Office of Energy R&D Policy was not part of a
research directorate and did not have written procedures or
uniform practices governing the review and evaluation of re-
search proposals. The review methods used were selected by
the program manager responsible for the subject area of the
proposal. For some proposals, the Foundation's traditional
peer review system of using outside reviewers was used. For
other proposals, where the information was needed to quickly
respond to other agency requests for policy information, the
traditional system was considered too slow and the proposals
were reviewed internally by the Office staff.
Origin and contents of
initial university proposal
     According to Dr. Johnscn and the individual who was the
Office's Deputy Director in 1974, they had a preliminary
discussion in May 1974, after which Dr. Johnson furnished
the Office with a description of the research grant he was
seeking.
      By letter dated June 19, 1974, the Office advised
 Dr. Johnson that, as previously discussed, his grant proposal
 should provide for consultation and engaging in discussions
 with Foundation staff, including maintaining contact with the
 staff about energy-related issues. As stated in the letter,a
 his primary task would be to prepare a series of papers on
 wide range of topics, drawing upon his experiences in the
 Federal Energy Office and the Treasury Department and from
 his independent research. The letter suggested certain
 topics for submission to the Office, including:
      -- Problems of State and local participation in energy
         resource development.
      -- Policies essential to achieving reasonable U.S. self-
         sufficiency in energy by 1980; some problems confront-
         ing Project Independence.
ENCLOSURE I                                       ENCLOSURE I

     -- Use of incentives to the private sector for developing
        energy resources.

     -- Ability of the Federal bureaucracy to conceive a na-
        tional energy policy.

     -- Problems in U.S.-Canadian energy relations.

The letter stated that other topics could be pursued if the
Foundacion agreed to them.

     The proposal submitted to the Office on July 22, 1974,
consisted of 21 pages; 2 pages generally described the work
to be performed under the grant and identified five discussion
papers to be prepared; 11 pages contained outlines of the
five discussion papers; 1 page contained a grant budget; and
7 pages gave the background of the principal investigator.
The scope of the proposal essentially restated or refined the
contents of the June 19, 1974, preliminary letter from the
Office. The proposal provided f    a series of discussion
papers on a wide range of energy issues, focusing primarily
on Federal gas and oil policies.  The proposal specifically
identified the five toplos stated in the June 19, 1974, pre-
liminary letter as topics of papers to be prepared. The pro-
posal also called for mutual agreement about papers on other
topics to be prepared, and it identified several possible
topics not included in the preliminary letter.  The proposal
also provided for consultation and ongoing contact with the
Office about energy-related issues, as provided for in the
letter.

Review of the initial proposal

     The Foundation uses various procedures in deciding
whether to fund the proposed research project. A widely
used procedure involves soliciting written comments by mail
from outsiders in academic institutions, government, and in-
dustry who are experts in the area covered by the proposal.
These experts are the "peers" of the investigator whose pro-
posal is being considered, and the solicitation of their
views as input in evaluating the proposal is called the mail
peer review system.

     When the initial proposal was reviewed, there were no
Foundation-wide criteria or procedures governing the use of
mail peer review, the selection of peer reviewers, or the
number to be used.  Although each directorate used mail peer
review to evaluate proposals, decisions as to the use of the
system were made by responsible program managers in accordance
with the generally accepted practice in each directorate.

                                 6
 ENCLOSURE I                                         ENCLOSURE I


     The FoundaLion's project folder for the grant showed
that the Office program manager selected three reviewers to
evaluate the initial roposal in July 1974. Two of the re-
viewers were Foundation employees--one was from another
foundation directorate and had worked for Dr. Johnson at
                                                          the
Federal Energy Office before joining the Foundation and one
was an Office of Energy R&D PolicyVistaff member--and the
other reviewer was from the academic community.   In addition,
the program manager responsible for reviewing the proposal
had worked under Dr. Johnson at the Treasury Department.
When the proposal was submitted and was being processed,
                                                          he
was on loan to the Foundation from Treasury.
     One of the Foundation employees who acted as reviewers
had a masters in electrical engineering and a Ph.D. in plan-
ning theory and systems analysis; the other had a masters
                                                          in
geology and a Ph.D. in mineral economics. The program manager
had a masters in mineral economics.

     Since the work was going to be done over a 12-month
period, reviewer selection could have been expanded to include
& number of private and public organizations that might have
been interested in reviewing the proposal.  Private organiza-
tions that might have been interested include the American
Gas Association, the American Petroleum Institute, oil market-
ing associations, and individual petroleum companies. On
                                                           the
public side, State and local agencies and a number of Federal
agencies concerned with energy matters, such as the Depart-
ment of the Interior, Federal Power Commission, and Federal
Energy Administrations might have been interested.

Content and review of
grant renewal proposal

     The renewal proposal that the university submitted to
the Foundation on May 5, 1975, consisted of nine pages:
five pages generally described the work to be done, includ-
ing summaries of five additional discussion papers to be
prepared; one page was a grant budget, and three pages con-
tained a progress report of work under the nitial grant.

     The progress report said that the following five dis-
cussion papers had been prepared and submitted to the
Foundation:

    -- "Trends in World Oil Prices and Production"    (Oct. 8,
       1974).



                             7
ENCLOSURE I                                        ENCLOSURE I


     -- "The Case Against Further Regulation of the Major
        Oil Companies" (Dec. 27, 1974).

     -- "Federal Energy Policy:   A Conflict of Interests"
        (Jan. 20, 1975).

     -- "The Naval Petroleum Reserves" (Jan. 27, 1975).

     -- "Some Comments on the President's Energy Program and
        Alternatives to It" (Feb. 20, 1975).

Several other papers were identified as under preparation:

     -- "U.S. Price and Allocation Controls and Their Impact
        on U.S. Energy Shortages."

     -- "The Two-Tier Price System and the Entitlements Pro-
        gram."

     -- "Regulation of Natural Gas at the Wellhead."

     -- "U.S. Policy Toward Coal."

     -- "State and Local Participation in Energy Resource
        Development."

     -- "U.S.-Canadian Relations on Energy Policy Matters."

     The proposal stated that a series of discussion papers
would be prepared that would be useful to Federal energy
policymakers (to be submitted to the Office and the Treasury
Department).   The papers were to survey policy-oriented
research on selected energy topics to assess the merits of
the research and to determine what additional research was
needed. The papers were also to examine and assess ongoing
Foundation-supported studies of various energy policy issues.
Topics of these papers included:   The Petro-dollar Problem;
The Future of OPEC [Organization of the Petroleum Exporting
Countries]; The Arab Oil Embargo: Past and Future; Multi-
national Companies; and Oil Import Quotas.   Also, the pro-
posal said papers on other tpics of interest to the Founda-
tion and the Treasury Department could be prepared and con-
sultant service could be expanded to include Treasury.

     The Foundation's project folder showed that in May 1975
the Office program manager selected eight reviewers--three
from Treasury and five from the Foundation--to evaluate the
renewal proposal. Of the six reviewers submitting evaluative
comments, five were Foundation employees--two were from other

                              8
 ENCLOSURE I
                                                         ENCLOSURE I


 Foundation directorates and had worked
                                         under Dr. Johnson at
 other Federal agencies before joining
 three were Office staff members--and   the Foundation and
                                       the other reviewer (a
 Treasury employee) was a former associate
 at Treasury.                               of Dr. Johnson's
               In addition:
       -- One Office staff member acting as
                                            a reviewer for the
          renewal proposal had been the program
                                                manager for the
          initial proposal.
       -- The Office program manager for the
                                             renewal proposal
          had been a reviewer of the initial
                                             proposal.
      -- In approving the award within the
                                           Office, the program
         manager, program director, and deputy
         signed off on the award were the same director who
                                               person.
     The five Foundation employees who acted
the following educational backgrounds:        as reviewers had
                                         (1) a Ph.D. in nuclear
science, (2) a Ph.D. in economics,
                                    (3) a masters in mineral
economics, (4) a Ph.D. in nuclear physics,
in geology and a Ph.D. in mineral economics.and (5) a masters
manager had a masters in electrical            The program
                                     engineering and a Pn.D.
in planning theory and systems analysis.

Rationale for theproposal
review procedures

       A September 1976 Foundation managemert
 cluded the Office's operating procedures         study that in-
 peer review procedures were regarded          said  the traditional
                                         as inappropriate for
 much of the policy analysis carried
                                       out by the Office.       This
 was because the analysis was mostly
                                       intended     to quickly  re-
 spond to specific requests from policy
                                            decision   offices,  such
 as the White House offices, the Office
 get, and other Federal agencies.           of Management and Bud-
                                    The study also said that
external peer review--a process which
                                          usually takes 2 to
 4 months--was considered inappropriate
                                            to meet deadlines
often stated in days or weeks.    Further, according to former
Office officials, research proposal
handled by Office staff members who reviews were largely
                                      understood the need for
the research and were often the users
                                          of the results.
      The university's proposal was unsolicited;
mitted after Dr. Johnson was told                      it was sub-
                                    by an Office official that
the Office funded energy policy research.
contains no documentation indicating            The project folder
                                        that the proposed
reseaLch--meetings, consultations,
                                     and research papers--was
in response to a specific request from
                                           any source outside

                                 9
ENCLOSURE I                                        ENCLOSURE I



the Foundation.  Further, both the initial and renewal grants
were made for 12-month periods. The review and evaluation
time required to process Dr. Johnson's initial proposal
totaled 1 month and 13 days from the date of receipt to the
date of award. The review and evaluation time required to
process the renewal proposal otaled 1 month and 25 days.

Instructions on peer reviewer selection

     As previously stated, at the time of the initial grant
and renewal to the university, there were no Foundation-wide
instructions on selecting peer reviewers. On February 13,
1976, the Foundation issued "Guidelines for Program Officers
on Selection of Reviewers"--Foundation-wide instructions that
provide uniform detailed criteria to guide program managers
in selecting peer reviewers. The criteria are aimed at pro-
viding a representative selection of peer reviewers who

     -- can evaluate the principal investigator's ability;

     -- have   broad view of the subject matter;

     -- can comment on the usefulness of the proposed work;
        and

     -- represent the institutional, educational, regional,
        or other elements of the community that might be
        affected by the project.

     Included in the guidelines are instructions which stress
the need to provide balance among differing points of view,
call attention to the possibility of scientific and personal
biases, and caution against involving reviewers in potential
conflicts of interest.

Guidelines given to reviewers
for proposal review

     Officials of several of the Foundation's directorates
said that, when requesting outsiders to review and evaluate
research proposals, the Foundation usually sends a copy of
the research proposal, a comment or rating sheet, and general
guidelines for the reviewer to use in evaluating the proposal.
Such guidelines generally request the reviewer to consider
such factors as the merit of the proposal, the feasibility
and effectiveness of the proposed work, the reasonableness
of the project budget, and the qualifications of project per-
sonnel. The project folder for the initial proposal and the
renewal showed reviewers were given the proposal and a rating

                                10
ENCLOSURE I                                       ENCLOSURE I



sheet. The folder contained no documents indicating that
review guidelines were furnished.

     On February 13, 1976, the Foundation issued "Guidelines
on Instructions to Reviewers"--Foundation-wide guidelines
that set forth standards and requirements applicable to re-
quests for peer reviews, including a requirement that peer
reviewers be given criteria for judging the proposal.

Disposition of reviewers' comments

     Reviewerst comments are an important factor considered
by the program manager in deciding whether to support a re-
search proposal. The program manager assesses reviewers'
evaluative comments and recommendations and their impact on
the proposal. Generally, any aspects that are counter to
Foundation policy or that might hinder the successful com-
pletion of the project are identified and resolved before a
recommendation regarding funding is made.

     The proposal rating sheets for the initial and renewal
proposals submitted by the university sho%  hat, although the
proposals were rated excellent and very good, the narrative
cuzments of several reviewers included questions about the
work to be performed. For example, these reviewers:

     -- Felt that the study's scope might be too broad to be
        covered in the proposed 10 to 12 papers (initial pro-
        posal).

     -- Believed that several proposed papers might be over-
        ambitiois in light of the time covered by the grant
        and cited two papers as examples (initial proposal).

     -- Suggested that the study be made more comprehensive
        and identified several additional topics for possible
        inclusion (renewal).

     -- Questioned whether the Foundation had verified that
        parallel work was not already underway in other in-
        stitutions, thus avoiding unnecessary duplication of
        effort (renewal).

The proposal review worksheets and other documents in the
project folder include no evidence indicating the disposi-
tion of these questions by the program managers.

     A comment by one reviewer relates to one of the specific
issues we are dealing with in this inquiry--outside funding

                             11
ENCLOSURE I                                       ENCLOSURE I



sources.  Both the initial proposal and renewal stated that
there would be outside support for the project, but neither
identified the source. The initial proposal said that the
Foundation's grant would cover about three-fourths of the
principal investigator's time and that he would receive sup-
port from other sources to cover the rest of his time (a
similar statement appeared in the renewal proposal).

     Apparently none of the reviewers asked about outside
funding when considering the initial proposal. In commenting
on the renewal proposal, one reviewer--an Office official--
stated:

     "'* * * My only question concerns the      de sup-
     port from another source (this is mentil,,ed on
     page 4.) What is it, how is it provided and for
     what, and does it assuredly provide for independ-
     ence in study? This question should be clarified
     before an award is made."

Documents in the project folder do not show what the program
manager did to resolve the reviewer's questions concerning
outside support.   The program manager for the renewal pro-
posal said  he did not remember how he disposed of the re-
viewer's comment about outside funding. He said he probably
called the principal investigator about his other source of
support.   However, he said other support was not a principal
concern of the Office because Foundation grantees commonly
obtained other support. According to Dr. Johnson, at the
time of the grants the Foundation did not inquire about the
specific ource of his outside funding.

     The Foundation has since acted to provide for better
documentation of the management decisions regarding individ-
ual projects. Action review boards 1/ have been established
in each directorate (the Office's functions are now part of
a directorate) to review awards.  One review factor to be con-
sidered is whether full and adequate consideration is given
to reviewers' comments. Also,  in August 1976 Foundation-wide
instructions were issued prescribing the minimum levels of


l/The Directorate fr Scientific, Technological and Inter-
  national Affairs' action review board was formalized on
  January 23, 1976.   Its functions were to include a review
  of all proposed  awards after recommended approval by the
  division director.   Board membership includes top manage-
  ment of the directorate and legal and business representa-
  tives from other Foundation offices.

                             12
ENCLOSURE I                                       ENCLOSURE I


documentation required for award recommendations. The
instructions require that the proposal review worksheet
summarize the pertinent factors leading to the program
manager's recommendation. The instructions outline several
considerations to be included in the summary; for example,
the summary is to indicate whether the principal investiga-
tor satisfactorily responded to major review criticisms.

     Disclosure of other funding sources

      Several of the research directorates have published
brochures or guidelines that inform researchers of the
requirements and general format for research proposals.
One requirement is that the proposal list all current re-
search, in addition to the proposed project, to which the
principal investigator and other senior research personnel
have committed part of their time and indicate whether the
salary for the person involved is included in the budgets of
the various projects. Such information should include the
titles and dates of current grants or contracts, the source
of funds, annual budget levels, and the time devoted to each
project by each of the senior personnel.   Such information
is to be provided for all other proposed research which is
being considered by, or which will be submitted in the near
future to, other possible sponsors, including other Founda-
tion programs.   The Office had no such requirements when the
awards were made to the university.

     The Foundation's grants to the university were not con-
sortium grants, which involve a master grant to one institu-
tion to support a project being carried out through a coopera-
tive arrangement between the grantee and one or more academic
institutions or profit or nonprofit organizations. Rather
they were grants that generally provided full funding to the
university for accomplishing the grant objectives outlined
in the two research proposals, with the understanding that
there would be some cost sharing by the university as re-
quired for unsolicited research proposals by the Founda-
tion's fiscal year 1975 appropriation act, from which the
grant funds originated.

     Grant records at the university show that the Founda-
tion was one of three sources giving grants to the university
to support Dr. Johnson's research work. The records showed
that for September 1, 1974, to August 31, 1975, the univer-
sity received $110,000 to support Dr. Johnson's work.   The
funding consisted of the Foundation's grant for $60,000 and
a grant by a group of oil marketing companies known as the
Independent Oil Marketers' Conference for $50,000.  For

                             13
ENCLOSURE I                                      ENCLOSURE I



September 1, 1975, to August 31, 1976, the university received
$145,000 to support Dr. Johnson's work.  The funding consisted
of the second Foundation grant for $70,000 (which included
$20,000 from the Treasury Department), a second grant by the
Independent Oil Marketers' Conference for $35,000, and a
grant by another group of marketers known as the Southern
Caucus for $40,000. The grants from the marketing groups
were given unconditionally to support Dr. Johnson's work.
The university decided to use the grants to fulfill the cost-
sharing requirements of the Foundation's grants for energy
policy research.

     On September 10, 1976, the Foundation's Acting   irector
issued a directive requiring that all proposal-generation
mechanisms used within the Foundation have applicants furnish
information on current support and pending applications with
other funding sources. The directorates are to require that
detailed information on other funding sources and pending
research applications be included as part of each research
proposal.

     The directive acknowledges that awareness of other sup-
port is an important factor in making awards, avoiding dupli-
cate funding of projects, and ascertaining that the proposers
have not overcommitted themselves to other projects and
duties. However, it does not prescribe guidelines or cri-
teria about what effect the source of the outside funding
should have in the consideration of a grant prcposal.

Conclusions

     The review process used for both proposals from the
university did not use a broad representative selection of
peer reviewers.  The reviewers selected did not include
policy decisionmakers or representatives from several govern-
ment agencies and from private industry with knowledge of and
a probable interest in energy policy. Office staff members
were extensively used as reviewers, and some reviewers and
the initial program manager had had a prior employment re-
lationship with the principal investigator.  In addition,
the reviewers were not given criteria to guide their review.

     The research proposals had a broad charter, the research
was to be done over a long period of time, and no evidence
indicated that the research was i- response to an urgent
request from an outside source. Tnase facts do not appear
to fit the Office justification for not using the external
peer review system.


                             14
ENCLSOURE I                                       ENCLOSURE I


     The reviews of the initial proposal and the renewal were
favorable; however, the grant records were not documented to
show the disposition of questions raised.

     Management instructions issued and organization changes
m    since the grants were awarded should help solve some
p.   ems we identified in the peer review process.  These
changes include (1) issuing Foundation-wide guidelines on peer
reviewer selection, including instructions to the reviewers,
(2) transferring the Office's functions to a research direc-
torate, which will provide additional management levels of
grant review and a more formal grant review process, and
(3) creating an action review board within each directorate
to review decisions on grant awards, including the documenta-
tion supporting these decisions.

     The Foundation's directive concerning awareness of other
sources of support for principal investigators does not ad-
dress the question of what impact such support will have on
the Foundation's grant award decision.  If the other sources
of funding indicate that the researcher is overcommitted or
has a similar proposal under consideration elsewhere, prudent
management would seemingly dictate the actions to be taken.
However, the source of the outside funding can also be an
issue to be considered in the grant award decision. This
issue is more difficult to resolve and the Foundation has no
written policy to help resolve it.

      Outside funding from organizations that could be affected
by the research might not influence the researcher in making
his study. However, it could lead to conflict-of-interest
questions that might reduce the usefulness of the research.
We recognize that obtaining experts to do sensitive policy
research who have not been supported or otherwise affiliated
with a concerned industry or organization might be difficult.
Thus, the source of outside funding to support a specific
research proposal could be largely an academic question.
Nevertheless, the Foundation should be cautious in funding
researchers to do sensitive policy research and try to avoid
the appearance of a conflict of interest.   Using - broad
selection of peer reviewers, including representatives-from
concerned public and private organizations to obtain a balance
of points of view should be common practice in evaluating re-
search proposals.   Such a broad selection of peer reviewers
can also help to respond to conf ict-of-interest questions
that might arise when supporting sensitive policy research.
The reviewers could also participate in a peer evaluation of
the publications resulting directly from the Foundation-
supported policy researcil.  (See p. 18.)

                             15
ENCLOSURE I                                        ENCLOSURE I


Recommendation
     We recommend that the Director of the Foundation issue a
policy directive providing guidance on how the source of out-
side funding should be considered in a grant award decision.
ACCOMPLISHING GRANT OBJECTIVES
     The objectives of the university's research grant were
broad. In addition to preparing a series of discussion
papers on a wide range of energy issues focusing primarily
on Federal gas and oil policies, the principal investigator
was to consult about energy-related issues.
     The Foundation did not maintain records to show the re-
ceipt of papers prepared under the grants. It prepared a list
of papers received under the grants for Chairman Kennedy and
later prepared a list for us. The lists did not completely
agree.
     Dr. Johnson and one of his assistants said that the
following papers were prepared and issued under the Founda-
tion's initial grant and renewal to the university.
                 Title                             Issue date
 'Trends in World Oil Prices and Production"       Oct.    1974
"The Impact of rice Controls on the Oil
   Industry"                                       Oct.    1974
"The Case Against Further Regulation of the
   Major Oil Companies"                            Dec.    1974
"The Naval Petroleum Reserves"                     Jan.    1975
"Federal Energy Policy: A Conflict of
   Interests"                                      Jan.    1975
"Some Comments on the President's Energy Pro-
   gram and Alternatives to It"                    Feb.    1975
"Pending Energy Legislation"                       July    1975
"How Federal Regulations are Putting Independent
  Oilmen Out of Business"                          Sept.   1975
"Why U.S. Energy Policy Has Failed"                Jan.    1976
"Coal Policy or the Lack Thereof"                  Mar.    1976
"Competition in the Oil Industry"                  Apr.    1976
"The International Implications of the Vertical
  Divestiture of U.S. Oil Companies"               May     1976
"The Petrodollar Problem"                          July    1976
     Dr. Johnson said Office officials informally approved
the selection of papers that he prepared and the changes from
the 10 papers identified in the grants. In October 1976

                            16
ENCLOSURE I                                       ENCLOSURE I

Dr. Johnson advised us of the following disposition of the
10 papers.

              Subject                     Disposition
Problem of State and loccal par-    Not written because it
  ticipation in energy resource       became less of an
  development                          issue.
Project Independence                Not written, but parts
                                      appear in other papers.
Private sector incentives in        Issued as impact of price
  energy development                  controls (Oct. 1974),
                                      testimony before House
                                      Ways and Means Committee
                                       (Jan. 1975).
National energy policy              Issued as why U.S. energy
                                      policy failed (Jan.
                                      1976).
U.S.-Canadian energy relations      Not written, but testimony
                                      given before Senate Sub-
                                      committee on Budgeting
                                      and Management (Nov.
                                      1974).
The petrodollar problem             Issued (July 1976).
Future of OPEC                      Included in paper on inter-
                                      national implications of
                                      vertical divestiture of
                                      U.S. oil companies (is-
                                      sued May 1976).
Arab oil embargo                    In draft (as of Oct. 20,
                                      1976).
Multinational companies             In draft (as of Oct. 20,
                                      1976).
Oil import quotas                   Not written, became a non-
                                      issue, and other studies
                                      existed.
     The Foundation's grant folders contain no documentation
to show its approval of the changes.  The program manager for
the initial grant could not be reached for comment on the
changes, and the program manager for the grant renewal agreed
that changes Zfom the initial grant and the renewal were ap-
proved informally.

     Concerning the consultation, Foundation grant records
show that Dr. Johnson attended several Office staff meetings,
participated in a discussion seminar at the Foundation in
January 1975 concerning the petroleum industry market struc-
ture, and presented the results of some research at the Fed-
eral Trade Commission in April 1976.

                               17
ENCLOSURE I                                     ENCLOSURE I



Standardizing project folders
     In August 1976 the Foundation issued instructions to
standardize the format and content of its grant folders.
The instructions require that final fiscal and technical
reports and other publications resulting from the grant be
filed in the grant folder.

Evaluation and use of research results
     Evaluation is the principal means of determining whether
grant objectives have been accomplished and of assessing the
quality of research products. There are no Foundation-wide
requirements that the results of research grants be formally
evaluated. Individual research directorates have the lati-
tude to determine how research performance and results are
evaluated.
     According to the Office's former Director, the crite-
rion for an overall evaluation of the papers submitted by
Dr. Johnson would be whether the Office staff could use them.
The grant folders neither contain a record of any evaluation
of the papers nor show what use was made of them. The pro-
gram manager for the grant renewal said that the Office was
the intended immediate user and that the papers were cir-
culated within the Foundation. Also, copies were given to
the Office of Management and Budget, the Treasury Department,
and other interested organizations. Dr. Johnson did not main-
tain distribution records but said about 7,200 copies of the
monograph concerning competition in the oil industry were
distributed.
     On March 8, 1976, the Foundation's Director issued an
instruction requiring that each directorate arrange for the
dissemination of substantial technical reports, including
those involving policy and applied research, through the De-
partment of Commerce's National Technical Information Service.
According to an official of the Directorate for Scientific,
Technological and International Affairs, none of the papers
submitted to the Foundation under the university's initial
grant and renewal were sent to the Information Service, but
the Directorate is now considering doing so.
 Conclusions
      The broad grant objectives of preparing a series of
 discussion papers and consulting and otherwise maintaining
 contact with the Foundation were met. Because of inadequate
 Foundation grant records, we cannot determine whether the
                                18
                                                     ENCLOSURE I
ENCLOSURE I

specific discussion papers prepared by the researcher are
those agreed to by the Foundation. The quality of       the dis-
cussion papers has not been  formally  determined   by  the
                                of the  papers  to  the  Office
Foundation, and the usefulness
                                              documented    in the
staff, the apparent intended users, is not
Foundation's records.
      Management controls, including organization changes,
established since the grants were awarded to the university
will provide for improved grant administration,     including
                                 activities   and  records   of
better documentation of project
                                             policies   and  pro-
papers submitted under a grant. However,  research  supported   by
 cedures for formally evaluating policy
 the Directorate for Scientific, Technological    and  Interna-
 tional Affairs do not exist.
     We believe that decisionmakers, the ultimate    users of
                             the quality  of  information  they
policy research, should know                              make
are receiving and that certainly the  Foundation  should
sure of the quality of information it passes on to them. from
Documented evaluations of publications resulting directly
                                                      to imple-
Foundation-supported policy research are necessary
ment these quality controls. In the case of Dr.a Johnson's
                                                    panel which
research, a review of the discussion papers by
included representatives of the interested   public  and private
organizations might have accomplished  this  purpose.

 Recommendations

      We recommend that the Director of the Foundation:
      -- Establish formal policies and procedures for the evalu-
         ation of policy research, including peer reviews of the
         research proposal that provide a balance of points of
         view, and a timely postevaluation by peers of publica-
         tions resulting directly from Foundation-supported
         research.
      -- Require that eyaluations of Foundation-supported
         policy research be made available for distribution
         through the National Technical Information Service.

       -- Provide for a formal evaluation of the discussion
          papers prepared under the Foundation's energy policy
          grants to the university.

       -- Determine the need for formal evaluation  of other
          recently ccipleted policy research, considering  the
          sensitivity of the subject matter and its  distribu-
          tion and use.
                                 19
ENCLOSURE I                                          ENCLOSURE I

ACKNOWLEDGMENT OF SUPPORT
AND DISCLAIMER POLICY
     The grants to the university were subject to Foundation
policy requiring that (1) an acknowledgment of Foundation
support be made with the publication of any material based
on or developed under the project and (2) any publications,
such as monographs and books, produced under the grant state
that any opinions, findings, conclusions, or recommendations
are those of the author and do not necessarily reflect the
Foundation's views.
     These requirements were not consistently applied to the
research papers issued under the Foundation's grants to the
universi:y, as shown below.
                                       Foundation       Disclaimer
                                        support             of
              Title                   acknowledged        opinion
"Trends in World C   Prices and
  Production"                         No                   No
"The Impact of Price Controls on
  the Oil Industry"                   No                   No
"The Case Against Further Regula-
  tion of the Major Oil Companies"    Yes (partial)        Yes
"The Naval Petroleum Reserves"        Yes (partial)        Yes
"Federal Energy Policy: A Con-
  flict o Interests"                  Yes (partial)        Yes
"Some Comments on the President's
  Energy Program and Alternatives
  to It"                              Yes (total)          No
"Pending Energy Legislation"          Yes (partial)        Yes
"How Federal Regulations are Put-
  ting Independent Oilmen Out of
  Business"                           Yes   (partial)      Yes
"Why U.S. Energy Policy Has Failed"   Yes   (partial)      Yes
"Coal Policy or the Lack Thereof"     Yes   (total)        Yes
"Competition in the Oil Industry"     Yes   (partial)      Yes
"The International Implications of
  the Vertical Divestiture of U.S.
  Oil Companies"                      No                   No
"The Petrodollar Problem"             Yes (total)          Yes
     Dr. Johnson said that he was not aware of the Foundation's
policy when the early papers under the initial grant were is-
sued nd that someone from the Foundation informed him later.
He did not understand why the paper "The International Impli-
cations of the Vertical Divestiture of the U.S. Oil Companies"
did not contain the acknowledgment/disclaimer statement.
                             20
ENCLOSURE I                                      ENCLOSURE I


     Dr. Johnson said he did not follow any prescribed format
in writing the statement and did not check his prior papers
for uniformity. Concerning acknowledgment of partial versus
total support, he stated that (1) if the papers concerned
international issues or issues of interest only to the
Foundation, he acknowledged full support, and (2) if the
papers were of mutual interest to the Foundation, the other
grantors (Oil Marketers' Conference and Southern Caucus),
and Treasury, he acknowledged partial Foundation support.
For example, the paper Coal Policy or the Lack Thereof"
acknowledged that it was written "under a grant from the
National Science Foundation," and the paper "Competition in
the Oil Industry" acknowledged that "This paper was written,
in part, under a grant by the National Science Foundation."
     The Foundation's policy as stated in its grant adminis-
tration manual provided only a suggested format for the re-
quired statements. On May 27, 1976, the Foundation's Acting
Director established a standard format for the tatements.
The acknowledgment/disclaimer statements are required on all
publications of the researcher resulting from the grant, but
the disclaimer statement may be omitted from scientific ar-
ticles or papers published in scientific journals. This is
the same requirement that was applicable to the university
grants. Procedures for the Foundation's program managers to
obtain compliance have not been established.
Disclosure of all funding sources
     Questions have been raised about the propriety of not
disclosing all sources of funding for Foundation-supported
policy research. Office officials responsible for the energy
policy research grants to the university were not concerned
with the other sources of funding because they believed they
would not influence the outcome of the research. As of Janu-
ary 7, 1977, the Foundation had not taken a position on this
issue.
                                                 /
Conclusions
     The Foundation's policy requiring acknowlegment of sup-
port and a disclaimer for content was not consistently com-
plied with in the publications resulting from the Foundation's
energy policy grants to the university. Foundation program
managers, as part of their grant-monitoring responsibilities,
should be required to examine publications resulting directly
from Foundation-supported policy research before their public
release to check for compliance with thi policy.

                             21
ENCLOSURE I                                      ENCLOSURE I



     As previously recommended, the Foundation needs to issue
a policy directive on how the source of outside funding should
be considered in the grant award process.   (See p. 16.)  We
believe outside funding from organizations that  could be af-
fected by the research results should be avoided if possible,
because it could easily raise conflict-of-interest questions
and reduce the credibility of the research. Disclosure of
all sources supporting the research would serve notice to
users who might believe the source of funding affects the
outcome of the research. However, it should be recognized
that a researcher not receiving direct outside support from
interested sources to help do Foundation-supported research
might receive payments from interested parties for speeches,
consulting, or other services while doing a Foundation-
supported project. Further, before being funded by the
Foundation, the researcher might have received funding from
interested sources for various services, and upon completing
a Foundation-supported project, the researcher might again
be employed by interested sources.

     As previously stated, we believe evaluation is the prin-
cipal way to determine the quality of Foundation-supported
policy research. Also, policy research publications directly
resulting from Foundation support could cite a bibliography
of the researcher's recent related publications that the in-
terested user could obtain to learn about the researcher's
prior positions.

Recommendations

     We recommend that the Director of the Foundation:

     -- Require that program managers examine publications
        resulting directly from Foundation-supported policy
        research before their public release to check for
        compliance with the acknowledgment of support and
        disclaimer policy.

     ---Require that the Foundation's acknowledgment of sup-
        port policy be revised to require disclosure of all
        funding sources of Foundation-supported research.

     -- Determine the feasibility of requiring that policy
        research publications developed with Foundation sup-
        port include citations to related research of the
        principal investigator.




                             22
ENCLOSURE I                                         ENCLOSURE I


INCOME FROM SALE OF PUBLICATIONS
     The Foundation defines program income as gross revenues
received by a grantee through activities under the grant.
Income includes proceeds received by a grantee from the sale
of books, monographs, and other material developed or pro-
duced with Foundation support.
     Foundation policy provides that before a grant is
awarded, consideration should be given to the grant's income
potential. Also, considering the nature of the project, the
purpose of Federal support, and the amount and source of ex-
pected income, the Foundation should provide for the dis-
position of the income in the grant. Foundation policy pro-
vides that income would normally be maintained in a separate
account and, as much as possible, used to offset costs other-
wise allowable and chargeable to the grant, including expenses
associated with the income-producing activity. If total in-
come is estimated to be more than $10,000, the grant is to
provide that income not used as provided for in the grant be
returned. If total income is estimated to be less than
$10,000, the grant may provide that icome not used be re-
tained by the grantee for science or science education
purposes. 1/
     The Foundation's project folder for the university grants
showed that the Foundation determined that, since the discus-
sion papers were to be used by the Foundation and not sold
for income, there was no income potential. Therefore, no
specific provision was made in the initial grant or the re-
newal for disposition of income.
     Grant records at the university showed that:
     -- Through September 30, 1976, the grantee had realized
        $14,826.59 in income from the sale of publications,
        of which $14,076.09 resulted from the sale of the
        monograph "Competition in the WO1 Industry" and
        $750.50 was from the sale of reproduced copies of
        research papers ($685.00 from "Competition n the Oil

1/Currently, the Foundation requires its share of royalties
  earned on sales of science education material developed
  with its support to be returned by the grantees for deposit
  in the Treasury. The requirement resulted from a 1975
  agreement with the Chairman, Subcommittee on HUD-Independent
  Agencies, Senate Appropriations Committee, that the gross
  royalties (considered a profit) be returned.
                             23
ENCLOSURE I                                        ENCLOSURE I


         Industry" and $65.50 from "International Implications
         of the Vertical Divestiture of U.S. Oil Companies".)

     -- Publication costs totaled $10,988.81 for the same
        period.  Publication costs of the monograph totaled
        $10,465.26 and costs of reproducing copies of the
        research paper "Competition in the Oil Industry" that
        were sold totaled $523.55.

     -- Income from total sales exceeded the recorded costs
        of publication and reproduction by $3,837.78.

     -- All receipts from sales of publications were credited
        to the Independent Oil Marketers' Conference grant.

     -- All related costs, except $523.55 that was charged to
        the Southern Caucus grant, were charged to the In-
        dependent Oil Marketers' Conference grant.
     -- Other than a charge for the registration fee for copy-
        righting the monograph, no income or related expenses
        for publications that were sold were processed through
        the Foundation's grant.

     The Foundation's grant administration manual, which
contains policies and procedures for grantees to follow and
which was included as part of the grant to the university,
provides under the section on contracts that:

     "* * * Commercial production and distribution of
     books, films or similar materials developed with
     the support of an NSF [National Science Founda-
     tion] grant may be undertaken only as described
     in a proposal which results in an NSF grant, or,
     if not so described, with prior NSF approval.
     *   *   *..



     Dr. Johnson arranged for the production (printing, art
work, etc.) and distribution of the monograph "Competition
in the ( il Industry." He also handled arrangements for re-
producing and distributing the other research papers. Neither
the Foundation's nor the university's grant records document
Foundation approval of these activities. Dr. Johnson said he
probably mentioned the intended publication of the monograph
to the Foundation. The program manager for the renewal said
he did not know what was done, but he believes the Foundation
would have approved because it encourages publication and
distribution of research results.


                              24
ENCLOSURE I                                      ENCLOSURE I


     University records show that the Foundation has
contributed over 50 percent of the grant funds for the
university's energy research prcject. As of August 31,
1976, the university's records of expenditures and encum-
brances show that costs have generally been charged to the
grants used by the university to support the project in pro-
portion to their support. Consequently, the Foundation has
been charged with about 50 percent of the total project costs.
Conclusions
     The publication and distribution arrangements for re-
search papers resulting from the Foundation's grants to the
university were apparently not formally approved (in writing)
by the Foundation. It is uncertain whether the arrangements
were approved orally. Because of this informality, the
Foundation lost control of income generated under the grant.
Since the Foundation pays a large part of the research cost,
and it might not have had the opportunity to consider the
publication and distribution arrangements, it ought to be
able to determine the disposition of at least some of the net
income resulting from the sale of publications developed with
its support.
     Sound business practices would dictate formal written
Foundation approval of commercial publication and distribu-
tion arrangements. The Foundation's program managers, through
their monitoring practices, should be responsible for making
certain that Foundation policies regarding publication and
distribution arrangements and income from the sale of publica-
tions are enforced.
Recommendations
     We recommend that the Director of the Foundation:
     -- Require grant recipients, as a condition of the award,
        to (1) obtain the Foundation's written approval of
        commercial publication and distribution arrangements
        for publications resulting directly from the Founda-
        tion's support and (2) allow the Foundation to deter-
        mine the disposition of all income resulting from the
        sale of such publications.
     -- Consult with the university to providt for disposition
        by the Foundation of a& equitable shaLe of the net in-
        come from the sale of publications developed under the
        Foundation's energy policy grants to the university.

                             25
ENCLOSURE I                                       ENCLOSURE I



     -- Specifically provide for the disposition of income in
        the energy policy grant to the university, if the grant
        is renewed.

PAYMENT FOR SALARIES
AND CONSULTANT SERVICES

     By agreement with the Subcommittee Chairman's office,
we reviewed the salary payments to the principal investiga-
tor under the Foundation's initial grant and renewal to the
university.

   I For the initial grant period (September 1, 1974, through
August 31, 1975), 75 percent of his salary was charged to the
Foundation grant and 25 percent to the Independent Oil Mar-
keters' Conference grant. For the renewal grant period
(September 1, 1975, through August 31, 1976), 71.4 of his
salary was charged to the  oundation's grant, 15.8 percent
to the Southern Caucus grant, and 12.8 percent to the In-
dependent Oil Marketers' Conference grant. The salary pay-
ments charged to the Foundation grants were in accordance
with the budgets approved by the Foundation for its initial
grant and renewal and the percentage of the researcher's
time to be devoted to the Foundation's grants.

     By agreement with the Subcommittee Chairman's office,
we also determined whether the grantee employed any Founda-
tion personnel as consultants under the grant. Foundation
policy provides that Federal employees--except Foundation
employees--can be employed under its grants as consultants
if they obtain their agency's approval and their services
are provided outside regular working hours or while on leave
from official duty. We did not find any evidence in the
Foundation's or the university's grant folders that Founda-
tion employees served as consultants under the grant. How-
ever, other Federal employees did serve as consultants.

     One consultant, employed by the Treasury Department as
an international economist in its Office of Energy Policy,
was paid $360 during the first grant period to help do re-
search and collect and interpret data.  The Independent Oil
Marketers' Conference grant was charged for these services.

     The Director of Treasury's Office of Energy Policy was
employed as a consultant to write a chapter on Federal and
State policies affecting the coal industry for inclusion in
a planned book on energy policy problems. He was paid $400
during the first grant period and $1,600 during the second
grant period, amounts which were charged respectively to the

                             26
ENCLOSURE I                                       ENCLOSURE I


Independent Oil Marketers' Conference grant and the
Foundation's renewal grant (which included $20,000 trans-
ferred to the Foundation from Treasury to support the grant).

     Treasury Department regulations concerning outside em-
ployment (31 CFR 0.735-38) provide that:

     "Employees shall not engage in any outside employ-
     ment or other outside activities, with or without
     compensation, which (a) interfere with tha effi-
     cient performance of official duties, (b) might
     bring discredit n or cause unfavorable and jus-
     tifiable critic sm of the Government or (c) might
     reasonably result in a conflict of interest, or
     an apparent conflict of interest, with official
     duties and responsibilities.   * * *"

Further, the regulations (31 CFR 0.735-39) allow employees
to accept engagements to speak, write, or teach provided that
the information to be used was not obtained as a result of
Federal employment, unless (1) the information has been made
available to the public or will be made available upon re-
quest or (2) a determination is made that using nonpublic
information is in the public interest. Before release of a
speech or article related to Treasury business, the employee
must obtain Department approval.  The official approving the
employee's participation in the engagement will determine
whether the activity will be performed as an official duty
or in a private capacity.  In the former case, the employee
cannot accept outside compensation.  In the latter case, the
employee can accept outside compensation, but he may not use
duty hours or Federal facilities.
     The Treasury employees obtained approval from their
superiors to perform the above-described work as consultar.;s
for the university's energy research project.  Employee time
and attendance records and billings for the consultant serv-
ice show that the activities were performed outside of the
employees' regularly scheduled Federal working hours.   Ac-
cording to a Department official, the chapter on the coal
industry prepared by the Treasury employee was reviewed by
the official who approved the employee's request to write
the chapter.

     Approval was granted to the Treasury employee in Janu-
ary 1975 to prepare the chapter in a private capacity.   In
June 1975, the Department transferred $'20,000 to the Founda-
tion in general support of the energy policy grant to the
university. The $1,600 payment to the employee charged to

                             27
ENCLOSURE I                                      ENCLOSURE I


the Foundation's grant was for vouchers submitted in December
1975 and May 1976.
     Treasury Department regulations concerned with outside
employment (31 CFR 0.735-38 and 31 CFR 0.735-39) do not
specifically prohibit an employee being paid in a private
capacity for Department-financed activities. A Treasury
official responsible for processing the transfer of funds to
the Foundation said she was not aware of the consultant con-
tract held by the Treasury employee. She viewed the situa-
tion as questionable and said she would have referred it to
the General Counsel. The Department's Counsel to the General
Counsel on ethics matters advised us that the employee's con-
sultant services were provided under contract to the univer-
sity as a separate transaction from the Department's cost
sharing and should be viewed that way. He added that in view
of the total Federal funds for the renewal grant ($50,000
from the Foundation and $20,000 from Treasury), the $1,600
paid the Treasury employee is insignificant.
     A member of a Sena.or's staff was paid $2,000 during
the first grant period to write a paper on the oil import
control protram and various proposals to reinstitute import
quotas. Payment was charged to the Independent Oil Marketers'
Conference grant. About half of the money was earned before
the person rsigned from the Senator's staff. Effective
September 1, 1975, the person was employed full time by the
university as a research associate on the energy policy re-
search project.
     According to the Senator's administrative aide, the em-
ployee who performed consulting services for the university's
energy research project was allowed and encouraged to do
outside consultant work or free lance writing. Further, we
were advised that office records showed the employee on leave
for 5 of the 8 Federal workdays for which consultant services
were billed. Leave records for the other 3 workdays could
not be located.




                            28