oversight

Need To Strengthen Management Control Over the Basic Research Program Administered by the Air Force Office of Scientific Research

Published by the Government Accountability Office on 1971-01-29.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                          Illllillllllllll
                                 lllllllllllllllllllll
                                LM095733




BY THE COMPTROLLER GENERAL
OF THE UNITED STATES



                  JAN.29,19?      a
                          CQMPTRQLLER     GENIZRAL    OF      THE      UNITED   STATES
                                        WASHINkTON.    D.C.         20548




        To the      President       of the Senate     and the
    /
   i.   Speaker       of the     House    of Representatives

                This is our report        on the need to strengthen         management
        control    over the basic    research      program   administered           by the
   1    Air Force      Office of Scientific     Research.                f.,2 ; i:,


                   Our     review   was made pursuant    to the Budget                         and Ac-
        counting         Act,   1921 (31 U.S.C. 531, and the Accounting                           and
        Auditing         Act of 1950 (31 U.S.C.   67).

               Copies   of this report   are being   sent to the                            Director,
        Office  of Management       and Budget;  the Secretary                              of Defense;
                                                                                                   # I.~
        and the Secretary      of the Ais Force.
i;;;




                                                                    Comptroller          General
                                                                    of the United        States
I
I
I
I
I
I
I   COMPTROLLERGENERAL'S                        NEED TO STRENGTHENMANAGEMENT CONTROLOVER
I
I   IZEPOKZ'
          TO THE CONGRESS                       THE BASIC RESEARCHPROGRAMADMINISTERED BY
I                                               THE AIR FORCEOFFICE OF SCIENTIFIC RESEARCH
I
I                                               B-170801
I
I
I
I   DIGEST
    --m--m
I
I
I   WHYTHEREVIEWWASMADE
I
I
I         The General Accounting       Office
                                           (GAO) sought to evaluate the practices-
I                                             nf Scienti fic_~~~:~~~~~~~~.~~~~~-~~g,,
I
I                                      ~~L~~~o~~~~~~~~~~~~ect~    -as..Q..AP- most
                                                                              _ -.~,i/ ofxhj,.h were
I          proposed and performed~by_unni~yers..t,y-Xresearchers.       The Office of Scien-
I
           tific-R~~~~~~~"~out             1,200 projects costing about $285 million.
I


    'FINDINGSANDCONCLUSIONS
          ReZevan.ceof research projects
          Federal policy is that the National Science Foundation will provide Fed-
          eral support for general-purpose basic research.    Other Federal agencies
          should support basic research only in areas closely related to their
          missions.   This policy is set forth in an Executive order as well as a
          Department of Defense (DOD) directive.   (See p. 7.)

           Guidelines for applying this policy were not provided, however. The
           Office of Scientific  Research interpreted     the policy broadly when se-
           lecting projects to be funded. Also, it did not prepare written justi-
           fications  showing the basis for supporting the projects.       (See p. 10.)
           Although many of the projects appeared to be closely related to the Air
           Force mission, some--including   the following    examples--did  not.

          Methodology for Analysis of Internal
I         SoeiaZ ‘Movements f$201,860)
           The purpose of this research was to construct and test a workable theory
           of the relationship   between the rise of ideological   movements and the
           changes in socioeconomic systems of selected countries in two different
           cultures,  the western and the oriental.    (See p. 11.)

I         Automatic ErtgZish Sentence
          Analysis ($261,000)
                                                                                                       ,
           The volume and redundancy of natural language records, as well as the
           existence of a number of different  languages in which significant


    Tear Sheet




                                                 1
information   is recorded, create the need for a capability   for automatic         i
processing of this material for such purposes as producing compressed               I
records, extracts of them, indexes to them, and translation      from one to        f
another.    Research results should furnish an understanding of how com-
puters may be used for natural language analysis for such applications              I
as text compression, extracting,   indexing, and translation.      (See p. 15.)     i

Support of research not closely related      to the Air Force mission reduces
the effectiveness  of its basic research     program since fewer dollars are
available for closely related research.

Inmediately after GAO's evaluation the Air Force reviewed al 1 current
projects in the Office of Scientific     Research as part of a DOD-wide
study following    enactment of the 1970 Defense Procurement Authorization
Act.    This act contained a provision forbidding      the use of 1970 funds
for research projects which did not have a direct and apparent relation-
ship to a specific    military  function or operation.     (See p. 22.) The
Deputy Secretary of Defense stated that this provision represented no
change in DOD's longstanding relevancy policy.        Nevertheless,    the Air
Force disqualified    115 Office of Scientific  Research projects--l0      per-
cent of the active projects-- because of insufficient       relevance.    In-
cluded were

  --26 nuclear physics projects ($12 million)  which related to functions
     of the Atomic Energy Commission and the National Aeronautics and
     Space Administration and

  --a project to develop a better rabies vaccine ($56,800).    The Air
     Force review panel concluded that rabies was not a unique military
     problem and that research in this area should be conducted by the
     Communicable Disease Center of the National Institutes  of Health.
     The Center has been engaged in rabies research for over 30 years
     and has awarded grants totaling  $2.3 million over the past 12 years.
     (See p. 23.)

Mavzagement
         procedures
The Office of Scientific   Research was not obtaining maximum benefit from
its basic research program because certain procedures had not been es-
tablished and existing   procedures were not consistently followed.

  --Surveillance     over ongoing research projects was not adequate.        In
     some cases researchers deviated significantly      from negotiated bud-
     gets or the agreed amount of the principal     investigator's     time to be
     devoted to the project,     but the Office of Scientific     Research did
     not recognize the deviation until too late to correct the problem.
     For example, one principal     investigator agreed to devote half his
     time to the %-year project but instead spent only 2 months at the
     outset.     (See pa 30.)




                                 2
             --Procedures for ensuring that all scientific      reports had been re-
                ceived were not consistently    followed.  Scientific   reports on some
               research efforts were not obtained at all, and others were obtained
                only after long delays.     (See p. 36.)

             --Procedures for disseminating        research results were not fully effec-
                tive.     Of the scientific   reports received9 some were not distributed
                to Air Force users or made available to the Defense Documentation
                Center.     The Center is responsible for making research results
                available    to any qualified   Government user.    (see p. 40.)


 l2ECOWNDATIONS
              OR SUGGESTIONS
    >- GAO suggested that DOD issue guidelines        adequately limiting   defense            z!
       funding to projects which were clearly relevant to assigned missions.
       Consideration should be given to the most appropriate agency to support
       the research in view of the missions and research performed by other
       organizations--military     and civilian,    Government and private.     Instruc-
       tions should be issued requiring        the need for each contract to be
       clearly established     in writing.     (See p. 26.)

         GAO also made a number of suggestions          for improving   management proce-
         dures.   (See pp. 34, 39, and 44.)


 AGENCY
      ACTIONSANDUNRESOLVED
                        ISSUES
         DOD said that relevance had always been an established  criterion of
         their research activities  and noted that, even prior to enactment of the
         1970 act, actions were taken to deemphasize certain areas of research
         and, in some cases, to transfer projects to other Government agencies.
         (See p. 26.)

         Subsequent to GAO's fieldwork      and passage of the 1970 act,       the Air Force
         issued regulations requiring:
             --Each research effort    selected for funding to have a direct       and co-
                herent relationship   to an Air Force function or operation.

             --Evaluation  as to whether each project is more appropriate         for the
                Air Force than for some other agency.

             --A written   record of that evaluation.       (See p. 26.)

         Also, the Office of Scientific Research took corrective  actions for im-
         proving management control over research work.   (See pp. 35, 39, and 44.)

         These actions, if properly      implemented,     should improve management of the
         basic research program.

Tear Sheet
    Other DODresearch
    The other military   services and defense agencies conduct similar    re-     1
>   search programs. GAO recommends that DOD review existing     regulations    < :
    of these organizations,    to determine if improvements are needed like       1
    those made by the Air Force.      (See p. 27.)                                ;
                                                                                  I
                                                                                  I
                                                                                             I
MATTERS
      FORCONSIDERATION
                    BY THECONGRESS                                                           I
                                                                                             I
    This report Is being submitted to the Congress because of congressional              1
    concern over the relevance of DOD research to its mission and continu-               ;
    ing congressional  interest in DOD's management of research and develop-             ,
    ment activities.                                                                     I
                                                                                         I
                                                                                         I
                                                                                         I
                                                                                         I
                                                                                         I
                                                                                         I
                                                                                         I
                                                                                          I
                                                                                          I
                                                                                         I
                                                                                         I
                                                                                         I
                                                                                         I
                                                                                         I
                                                                                         I

                                                                                         1
                                                                                         I
                                                                                         I
                                                                                     I
                                                                                     I
                                                                                     I
                                                                                     I
                                                                                     I
                                                                                     I
                                                                                     I
                                                                                     I


                                                                                     I




                                                                                 I
                                                                                 i
                                                                                 I
                                                                                 I
                                                                                 I
                                                                                 I
                                                                                 I
                                                                                 I

                                                                                 I
                                                                                 I
                                                                                 I
                          Contents
                                                              Page

DIGEST                                                          1

CHAPTER

  1       INTRODUCT.ION                                         5

   2      NEED FOR DEFINITIVE GUIDELINES FOR SELECTING
          RESEARCHPROJECTS                                      7
              Governmental policy limits  type of re-
                 search Air Force can sponsor                   7
              Close relevance of little  concern in ear-
                 lier years                                     8
              Relevance to Air Force mission inter-
                 preted broadly                               10
              Air Force review of OSR projects                22
              Internal   audit                                25
              Conclusion                                      25
              Agency action and our evaluation                25
              Recommendation                                  27

          NEED FOR BETTER SURVEILLANCE OVER RESEARCH
          PROJECTS                                             28
              OSR requirements  provide for monitoring         28
             Monitoring   not done consistently    and uni-
                formly                                        29
             Examples of administrative     problems          29
             Conclusions                                      34
             Agency actions and our evaluation                34
          RESEARCHRESULTS NOT OBTAINED OR NOI OB-
          TAINED PROMPTLY                                      36
              Scientific  reports must be submitted to
                OSR promptly                                  36
              Reports not received or delayed unduly          36
              Conclusion                                      38
              Agency actions and our evaluation               39
          NEED FOR MORE EFFECTIVE DISSEMINATION OF REi-
          SEARCHRESULTS                                       40
              Informal dissemination of research re-
                 sults                                        40
                                                                Page

CHAPTER

                Conclusion                                       44
                Agency actions      and our evaluation           44
           SCOPE OF REVIEW                                       46
APPENDIX

       I   Listing   of 46 projects      reviewed                49

  II       Letter  dated June 15, 1970, from the Direc-
              tor of Defense Research and Engineering   to
              the General Accounting Office                      56
III        Principal    officials    of the Department of De-
              fense and the Department of the Air Force
              responsible      for administration  of activi-
              ties discussed in this report                      75
                            ABBREVIATIONS

DOD        Department   of Defense

GAO        General   Accounting     Office

OSR        Air Force Office       of Scientific     Research
COMPTROLLER
          GENERAL'S                  NEED TO STRENGTHENMANAGEMENT CONTROLOVER
REPORT
     TO THECONGRESS                  THE BASIC RESEARCHPROGRAMADMINISTERED BY
                                     THE AIR FORCEOFFICE OF SCIENTIFIC RESEARCH
                                     B-170801


DIGEST
_-----

WHYTHEREVIEWWASMADE
    The General Accounting Office (GAO) sought to evaluate the practices
    followed by the Air Force Office of Scientific      Research in selecting,
    managing, and using the results of research projects,      most of which were
    proposed and performed by university   researchers.     The Office of Scien-
    tific   Research supports about 1,200 projects costing about $285 million.


FINDINGSANDCONCLUSIONS
    ReZezrrmceof research projects
    Federal policy is that the National Science Foundation will provide Fed-
    eral support for general-purpose basic research.    Other Federal agencies
    should support basic research only in areas closely related to their
    missions.   This policy is set forth in an Executive order as well as a
    Department of Defense (DOD) directive.   (See p* 7.)

    Guidelines for applying this policy were not provided, however. The
    Office of Scientific  Research interpreted     the policy broadly when se-
    lecting projects to be funded. Also, it did not prepare written justi-
    fications  showing the basis for supporting the projects.       (See p. 10.)
    Although many of the projects appeared to be closely related to the Air
    Force mission, some--including   the following    examples--did  not.

    MethodoZogyfor Analysis of In-h=na~
    Social Mozrements($201,860)
    The purpose of this research was to construct and test a workable theory
    of the relationship   between the rise of ideological   movements and the
    changes in socioeconomic systems of selected countries in two different
    cultures,  the western and the oriental.    (See p. 11.)

    Automatic EngZ&shSentence
    Analysis ($261,000)
    The volume and redundancy of natural language records3 as well as the
    existence of a number of different  languages in which significant
information   is recorded, create the need for a capability   for automatic
processing of this material for such purposes as producing compressed
records, extracts of them, indexes to them, and translation      from one to
another.    Research results should furnish an understanding of bon com-
puters may be used for natural language analysis for such applications
as text compression, extracting,   indexing, and translation.      (See p. 15.)

Support of research not closely related       to the Air Force mission reduces
the effectiveness  of its basic research      program since fewer dollars are
available for closely related research.

Imnediately after GAO's evaluation      the Air Force reviewed all current
projects in the Office of Scientific       Research as part of a DOD-wide
study following    enactment of the 1970 Defense Procurement Authorization
Act. This act contained a provision forbidding         the use of 1970 funds
for research projects which did not have a direct and apparent relation-
ship to a specific    military  function or operation.     (See p. 22.) The
Deputy Secretary of Defense stated that this provision represented no
change in DOD's longstanding relevancy policy.         Nevertheless,    the Air
Force disqualified    115 Office of Scientific    Research projects--lo     per-
cent of the active projects-- because of insufficient        relevance,    In-
cluded were

  --26 nuclear physics projects ($12 million)  which related to functions
     of the Atomic Energy Commission and the National Aeronautics and
     Space Administration 'and

  --a project to develop a better rabies vaccine ($56,800).    The Air
     Force review panel concluded that rabies was not a unique military
     problem and that research in this area should be conducted by the
     Communicable Disease Center of the National Institutes  of Health.
     The Center has been engaged in rabies research for over 30 years
     and has awarded grants totaling  $2.3 million over the past 12 years.
     (See p. 23.)

Managementprocedures
The Office of Scientific   Research was not obtaining maximum benefit from
its basic research program because certain procedures had not been es-
tablished and existing   procedures were not consistently followed.

  --Surveillance     over ongoing research projects was not adequate.       In
     some cases researchers deviated significantly      from negotiated bud-
     gets or the agreed amount of the principal     investigator's     time to be
     devoted to the project,     but the Office of Scientific     Research did
     not recognize the deviation until too late to correct the problem.
     For example, one principal     investigator agreed to devote half his
     time to the 3.-year project but instead spent only 2 months at the
     outset.     (See p. 30.1
     --Procedures for ensuring that all scientific      reports had been re-
        ceived were not consistently    followed.  Scientific   reports on some
        research efforts were not obtained at all, and others were obtained
        only after long delays.     (See p. 36.)

     --Procedures for disseminating     research results were not fully effec-
        tive.    Of the scientific reports received, some were not distributed
        to Air Force users or made available     to the Defense Documentation
        Center.    The Center is responsible for making research results
        available to any qualified    Government user.   (See p. 40.)


RECOMMENDATIONS
            ORSUGGESTIONS
   GAO suggested that DOD issue guidelines         adequately limiting   defense
   funding to projects which were clearly relevant to assigned missions.
   Consideration   should be given to the most appropriate agency to support
   the research in view of the missions and research performed by other
   organizations --military     and civilian,    Government and private.     Instruc-
   tions should be issued requiring         the need for each contract to be
   clearly established      in writing.     (See p. 26.)

   GAO also made a number of suggestions            for improving   management proce-
   dures.   (See pp. 34, 39, and 44.)


AGENCY
     ACTIONSANDUNR?3SOLVED
                       ISSUES
   DOD said that relevance had always been an established  criterion of
   their research activities  and noted that, even prior to enactment of the
   1970 act, actions were taken to deemphasize certain areas of research
   and, in some cases, to transfer projects to other Government agencies.
   (See p. 26.)

   Subsequent to GAO's fieldwork       and passage of the 1970 act,        the Air   Force
   issued regulations requiring:
     --Each research effort     selected for funding to have a direct          and co-
        herent relationship    to an Air Force function or operation.

     --Evaluation  as to whether each project is more appropriate             for the
        Air Force than for some other agency.

     --A written   record of that     evaluation.       (See p. 26.)

   Also, the Office of Scientific Research took corrective actions for im-
   proving management control over research work. (See pp. 35, 39, and 44.)

   These actions, if properly       implemented,      should improve management of the
   basic research program.
    Other DOD research

    The other military   services and defense agencies conduct similar    re-
    search programs. GAO recommends that DOD review existing     regulations
    of these organizations,    to determine if improvements are needed like
    those made by the Air Force.      (See p. 27.)


MATTERS FOR CONSIDERATIONBY mE CONGRESS

    This report is being submitted to the Congress because of congressional
    concern over the relevance of DODresearch to its mission and continu-
    ing congressional interest  in DOD's management of research and develop-
    ment activities.
                                CHAPTER 1

                              INTRODUCTION

        The General Accounting      Office has made a review of se-
lected management aspects of the basic research program ad-
ministered    by the Air Force Office of Scientific           Research
 (OSR) 9 a component of the Office of Aerospace Research, De-
partment of the Air Force,1          Our inquiries    were directed
primarily    toward evaluating      the effectiveness      of management
policies    and practices    followed     by OSR, relating    to the se-
lection    and monitoring    of research projects       and evaluation
and dissemination      of project     results.    Of approximately
1,200 research efforts,        we selected for review 46 on which,
over the years, OSR had expended or committed about $21
million.

       The purpose of the Office of Aerospace Research is to
manage the conduct and support of basic research essential
to the continued         superiority     of the Air Force operational
capability     and the conduct and support of exploratory           devel-
opment efforts.          The mission of OSR is to support programs
of extramural       (out-of-house)       basic research in domestic and
foreign    universities,       nonprofit     institutions, and industry
through contracts         and grants.      Other Office of Aerospace Re-
search components conduct programs of basic and exploratory
development research in-house and programs through contracts
that closely      support and extend in-house efforts,

        OSR is headed by an executive           director   and is organized
into six scientific        directorates       and one procurement direc-
torate.     The scientific       directorates      are responsible     for
eValuating     and selecting      research proposals,       monitoring     re-
search projects,      and evaluating        and disseminating      project
results.      The procurement directorate           is responsible     for the
negotiation     and administration         of research contracts       and
grants.


1Effective July 1, 1970, the Office of Aerospace Research
 was merged into the Air Force Systems Command; however, the
 mission and operations  of its subordinate,     OSR, which are
 dealt with in this report,   remain essentially   unchanged.

                                     5
       OSR supports research in chemistry,       mathematical    sci-
ences, electronics,    mechanics, energy conversion,        general
physics,   nuclear  physics,  astronomy and astrophysics,       be-
havioral   and social sciences,    and biological     and medical
sciences.

       During the time of our review, OSR was supporting      about
1,200 research grants and contracts      with 391 research insti-
tutions.     A total of about $285 million   had been expended
on or approved for these 1,200 projects       at the time of our
review.     About 75 percent of the research grants and con-
tracts   had been renewed one or more times, and about one
third of the renewed projects     had been supported continu-
ously for 6 years or longer.

       The very nature of contracted       research--the        indefinite-
ness of the required     work and the high degree of uncertainty
about the outcome of the work--involves            unique problems for
Government scientists     and contracting       officers     and makes it
a difficult     area to manage. Despite the difficulties,                a
measure of control     is essential     to protect      the interests       of
the Government and to ensure that maximum benefits                 are de-
rived from the millions      of dollars    spent annually on such
research.     In subsequent chapters of this report,             we de-
scribed the problems that came to our attention               during this
review as well as our pertinent         conclusions       and recommenda-
tions.
       The scope of our review is described on page 46. The
principal     officials     of the Department of Defense and the De-
partment of the Air Force responsible         for the administration
of activities        discussed in this report are listed  in appen-
dix III.




                                      6
                             CHAPTER2

                NEED FOR DEFINITIVB GUIDELINES

                FOR SELECTING RESEARCHPROJECTS

      OSR has been subject to the broad policy statements
which are set forth in Executive Order 10521 and DOD Direc-
tive 3210.1, which limit     defense agencies to the support of
research which is closely      related  to military     needs.  Defin-
itive  guidelines    for use in selecting    research projects     were
not provided,     however, and OSR interpreted      relevance to the
Air Force mission very broadly in selecting          projects  which
it funded.

        In its review of 46 of OSR's active research projects,
GAO noted that the relevancy       of the projects  was generally
not adequately documented and that a number of projects            did
not appear to be closely related        to military needs,      An Air
Force review of OSR's active research projects,          subsequent
to the completion     of our review and in response to the di-
rection    of the Deputy Secretary    of Defense, resulted      in the
disqualification     of 115 projects    because of insufficient
relevance.

GOVEWMENT POLICY LIMITS TYPE OF
RESEARCHAIR FORCE CAN SPONSOR

       Since 1954 there have been restrictions'on        the type of
research which the Air Force and other military           services
can sponsor.      In that year Executive Order 10521 was issued.
Section 4 of that order states that the National           Science
Foundation shall be increasingly      responsible    for providing
support by the Federal Government for general purpose basic
research.     This section of the order goes on to state that
other Federal agencies should continue to support basic re-
search in areas which are closely related         to their missions.
DOD Directive     3210.1, which was issued to implement the Ex-
ecutive order, states that it is the policy of DOD to con-
duct and support a broad basic research program to provide
fundamental knowledge with emphasis on that related            to the
needs of DOD.
        In 1968 the Congress indicated   that DOD should be more
selective     in determining the research efforts  which it spon-
sored.     In House Report 1735, dated July 18, 1968, the House
Committee on Appropriations,     in commenting on the DOD appro-
priation     bill for fiscal year 1969, stated:

     "There is considerable    evidence that the Depart-
     ment has found sufficient     research and develop-
     ment funds available    in past years to contract
     for volumes of studies on a myriad of topics,
     some of which have little     or no apparent applica-
     tion to weapons systems developments or even to
     military  matters.   Some studies continue for
     months and years without reaching conclusions.
     Not enough attention    has been given to evaluating
     the need for such study efforts       and the elimina-
     tion of all studies not absolutely       essential.

     "Among the vast numbers of individual        projects
     funded in R.D.T.&E.     (Research, Development, Test,
     and Evaluation),   there are many desirable       pro-
     grams which are not absolutely     essential     at this
     time.   Such programs can be delayed or postponed
     without  adverse effect    to the Nation's    defense."

CLOSE RELEVANCEOF LITTLE
CONCERNIN EAR_z,_IER
                   YEARS

       We found that,   originally, OSR did not view itself as
being restricted     to sponsoring research projects that were
closely related    to Air Force programs.

     Subsequent to the issuance of Executive Order 10521 in
March 1954, the Chief of the Office of Scientific Research
held the following views:
     I**** Executive Order 10521 *** restricted         agencies
     other than NSF [National       Science Foundation]      to
      'basic research in areas which are closely          related
     to their missions.'   *** To OSR, particularly          to
     [the Chief],   this restriction      meant one thing:       the
     organization   would henceforth      be required   to de-
     fend its program, and its budget, on the basis of



                                    8
     relevance to existing         Air Force requirements,          the
     difficulty    inherent    in intelligently        relating     ba-
     sic research with existing           requirements     notwith-
     standing.     Equally clear to [him: was that OSR's
     progr=b    if it was to continue to be of maximum
     usefulness    to the Air Force, would have to go on
     as before.      The problem, as [heI stated it later,
     was that 'by virtue      of the Executive Order [basic
     research]    could no longer be labeled basic re-
     search' if the Air Force were to retain               a measure
     of control    over it.      In the end, OSR solved the
     problem by a skillful         exercise in semantics. ***
     And, as will be shown, executive            order or no, in
     the face of increasing         opposition    to basic re-
     search by the Bureau of the Budget, OSR would
     have had to resort to some such tactic               sooner or
     later."    (Underscoring       supplied.)

      Thus, according to its history,1  OSR originally  re-
sisted Executive Order 10521 and continued to support basic
research that was not closely   related to Air Force problems.


1A History of the Air Force Office of Scientific                Research,
 Vol. I, Komons, Office of Aerospace Research.
RELEVANCETO AIR FORCE MISSION
INTERF'RXTEDBROADLY

      As a part of our review, we examined into the state-
ments which OSR scientists   had made to show the relevancy
to the Air Force mission of the projects   which were selected
for support.

       We believe that to be relevant        a project    should have a
traceable,    significant,     logical   connection    to the Air Force
mission,   which has been stated as the organizing,            training,
and equipping of forces for the defense of the United States
against air attack;        the achievement of general air superi-
ority;   and the prosecution       of air combat operations       against
the enemy in time of war.

      OSR instructions   for evaluation   of research proposals
were contained in OSR Headquarters      Operating   Instruction
80-4, dated February 10, 1965. This instruction           suggested,
but did not require,   that a determination      be made of the
relevance of proposed projects     to the Air Force mission,       as
shown by the section quoted below.

      "The project     scientist    will,    upon completion       of
      the evaluation      of the proposal,      set forth *** a
      concise,  descriptive      statement of his evaluation
      and recommendation based, where appropriate,                 on
      comments of reviewers.         To be included in this
      section will be pertinent           comments on such factors
      as *** Air Force and DOD relevance,             etc.    The
      foregoing   factors     should not be considered=
      mandatory or limiting."           (Underscoring      supplied.)

       Although the OSR instruction      placed upon the project
scientist    a requirement    for an evaluation    of the relevancy
of the project     to the Air Force mission,      the portion  of the
instruction    which we have underlined      tended to make the re-
quirement permissive       and we found that it was more often
ignored than observed.

        In response to our request for written statements of
project    relevance, we were furnished with copies of project
summaries which had been prepared for annual review


                                     10
purposes.   A listing of the 46 projects,  which includes
the statements of relevance,  is presented in appendix I.
       The breadth of OSR's determinations       of relevancy     is
apparent from some of the statementsmade         to indicate    rele-
vance.    The statements    seemed to closely    relate   some proj-
ects to the Air Force mission.         Some projects,   however,
seemed much farther     removed from its mission.        The remote-
ness of some projects     from areas of direct      Air Force con-
cern is illust?ated     by the justifications      for the following
five projects.

Methodology for Analysis of Internal
Social Movements ($201,861)
       The OSR-prepared     abstract    on this   project    described    it
as follows:
      Y&e purpose of this research is to construct                 and
      test a workable theory of the relationship              between
      the rise of ideological         movements and changes in
      socio-economic        systems of selected countries        in
      two different       cultures,   one western and the other
      oriental.      The program has two partially        over-
      lapping phases:         a. Formulation   of a set of prop-
      ositions    or hypotheses which identify       critical
      variables     in social system change, and which have
      important     effects     on the nature and direction        of
      ideological      movements and b. the testing        of hy-
      potheses on the basis of inventory          of social move-
      ments in these two widely divergent          social sys-
      tems, China and Germany, through case studies.
       Secondary data from domestic and foreign            libraries
      and information        centers will be made available          in
      a computer based data file and combined with the
      results    of case studies a comparative       analysis        of
      the two social systems.          With the techniques       being
      developed and tested it will be possible             to take


1 Subsequent  to the GAO review, the Air Force reviewed all
 current    OSR projects.    The second and third described proj-
 ects expired prior to the Air Force review, and the-other
 three were found by the Air Force to have insufficient
 relevance.     (See p0 22,)
     advantage of computers to analyze more complete
     data than could otherwise   be manageable.        Results
     will be useful for environmental      and political
     analysis   and forecasting by Air Force systems
     planners."    (Underlining supplied.)

     It  is pertinent   to note that the abstract     did not es-
tablish  that the research was related      to the Air Force mis-
sion or to specific    Air Force functions.     Also, it did not
show that the research had been requested by any particular
user of such information.      Rather, it stated that the re-
sults would be useful to "Air Force systems planners."

Criteria for the Design of Mew
Forms of Organization  ($176,774)

       The OSR-prepared     abstract        on this    project   described   it
as follows:

     'The overall        objective    of this study is to develop
     criteria      from research for the design of new forms
     of organization          or the planned change of existing
     organizations,         in order to effectively        accomplish
     different       organizational     goals.     Phase I specifies
     preliminary       design criteria       on the basis of an
     analysis      of prior experiences         in organizational     de-
     sign.       Phase II is testing       the effectiveness       of
     these design criteria           by applying them to ongoing
     organizational        design activities       and to modify the
     criteria,       as appropriate.       Phase III will      synthed.
     size and publish the results             of this investigation
     in a Handbook of Organizational              Design, appropri-
     ate for the use of managers in innovative                 forms
     of organization          in government and private        in-
     stitutions.         This handbook will be especially
     useful to Air Force and other military               super-

     nizational     change and adaptation             to new condi-
     tions."      (Underlining supplied.)

       It is again pertinent  to note that the Air Force did
not state what organizational      problems it had which needed
resolving   through this research.      The research seemed to


                                       12
have been oriented         very broadly to apply to any form of or-
ganization      within    or outside the Government, as shown by
the statement that the results            would be published      in a hand-
book appropriate        for use in Goverlnment and private          institu-
tions.      Further,    the  results   ware   obtained   by  observing       re-
organizations        in Government, industrial,        and educational
institutions,        to analyze th, p usefulness     of different      prin-
ciples and tools for organizational             design,     This informa-
tion was to be used to develop criteria              for the design of
research organizations,          educational    organizations,      and op-
erational     or production      organizations.

Socio-Cultural      Aspects
of Development      ($82,920)

       The OS-prepared       abstract        on this   project   described   it
as follows:

      s'This investigator       will make a comparative          anal-
      ysis of six developing         countries     on three conti-
      nents, with regard to the influence             on attitudes,
      values, and behavior of a wide variety               of social
      factors    and institutions.         Data have already been
      collected    through interviews         in each of these
      countries    of a sample of 1,000 subjects,             including
      -workers 9 peasants,      and students.       The contribu-
      tion of each of the following             to making people
      hold more modern attitudes          will be assessed:
      Education,     social origins,       urban experience,        ex-
      posure to mass-communication,            and factory      work.
      The completed interviews          touched on almost every
      major aspect of value judpents               and attitudes
      toward family size f participation            in voluntary
      organizations,      etc.     The analysis will       show the
      relative    contribution      which each ty@ of back-
      around experience makes toward modernization                  of
      zttitudes,     both when it is acting alone, and in
      combination with other influences.

      This research will make a comparative          analysis
      of data already collected,       from six countries     on
      three continents,     on the important    social and in-
      stitutional   factors    which influence   people in de-
      veloping countries     to have more modern attitudes

                                        23
     and act on them.     These comparisons may indicate
     where a country is ahead or lagging behind in its
     development.   This knowledge will be very useful
     in guiding programs for military       aid and policy
     planning."   (Underlining   supplied.)

OSR subsequently  acknowledged the remoteness of this project
from the Air Force mission when the following    statement was
made to the contractor   in explaining the discontinuance  of
further Air Force support.

     "For each task submitted    for review we must dem-
     onstrate  relevance   to the Air Force mission,     pro-
     posed Air Force use of the results       and, also,
     issue a guarantee that the study will not invite
     attack from Congress, the press or the general
     public.   We did not undertake to support your
     work with these objectives      in mind, and I doubt
     that [we] would claim that what you are doing is
     directly  and uniquely   applicable    to the Air Force."

Preparation of a Manuscript             for a
Reference Book on Laboratory             himal
Diseases ($82,120)

       The OSR-prepared      abstract        on this   project   described   it
as follows:

     "Comprehensive information            of laboratory         animal
     diseases is not readily           available     to those who
     need it.     A lot of information           exists but little
     has been published          - that which has been pub-
     lished is scattered          throughout     the literature,
     frequently     in obscure journals.            The purpose of
     this project     is to get experts in the various
     facets of laboratory          animal diseases to review
     the literature       in their area and to add to it
     any unpublished        information      either    in,their
     possession or of which they are aware.                     All of
     this information        will then be compiled into a
     reference    book organized by etiologic               agent.
     This book will aid in the diagnosis,                 prevention
     and treatment      of spontaneous illnesses             of re-
     search animals.          It will also be useful in

                                        14
      evaluating      experimental    results    by helping to dif-
      ferentiate     between spontaneous and experimentally
      induced lesians.         Its long-term     value -will be to
      point up areas where more research is needed.
      The results      will be relevant       to the extensive
      use of animals        in Air Force research."        (Under-
      lining     supplied.)

It should be noted that the               abstract    did not establish  that
the research was relevant   to              the -Air Force mission or on
what basis it was concluded               that support of this general-
purgo~~ type of project   was             appropriate    for the Air Force.

Automatic    English Sentence
Analysis    ($264,,001)

      The OSR-prepared         abstract      on this    project     described     it
as follows:

      "The volume and redundancy of natural                    language
      records,      as well as the existence            of a number of
      different      languages in which significant                informa-
      tion is recorded,         create the need for a capabil-
      ity for automatic         processing      of this material
      for such purposes as producing compressed record,
      extracts      of them, indexes to them, and transla-
      tion from one language to another.                   This effort
      is studyirr, 0 the structures           of several natural          lan,-
      tP-%es 9    the   way   words   are   used    to  express      meaning,
      and the relationships           that exist between words
      and within       strings    of words for the purpose of
      establishing        rules comprehensible          to machines for
      the automatic manipulation              of natural       language
      test.     In prior research,          a machine-verified
      procedure was developed for analyzing                    natural
      language sentences and theories                 pertaining       to
      syntax and semantics were formulated                    and tested.
      Proposed research will extend the procedure to
      incorporate       dictionary      look-up and idiomatic
      phrases.       The data base will be enlarged and the
      procedure applied to coherent sequences of sen-
      tences.      Attempts will be made to specify a gram-
      mar of interpropositional             relations       that can be
      used for automatic          selection     of sentence
      interpretations       in the analysis    of text.     Research
      results     should furnish   an understanding      of how
      computers may be used for natural           language anal-
      ysis for such applications        as text compression,
      extracting,      indexing,  and translation."        (Under-
      lining    supplied.)

Here, also, the abstract      neither   stated that the research
was relevant     to the Air Force mission nor did it attempt to
explain the appropriateness       of Air Force sponsorship     of the
project.     Also, the project    was classified    by OSR as pio-
neering research which OSR described,          in part, as that for
which the usefulness     of the results     could not be forecast
with any degree of certainty.




                                   16
Pioneering    research

        Not only has OSR broadly interpreted   relevance    in se-
lecting    the type of subjects   that are remote to the Air
Force mission,     but it has also broadly interpreted    rele-
vance in heavily      engaging in what it terms sFpioneeringl'
research,

      OSR stated that, under research of this type, investi-
gators are free to pursue their research without undue con-
cern for the usefulness  of their new knowledge.

       Examples of pioneering     research are provided by the
projects   entitled  "Socio-Cultural      Aspects of Development"
and *'Automatic English Sentence Analysis,O' which were dis-
cussed on pages 13 and 15. Of the 46 projects           selected      for
our review, 24 were classified        by OSR as pioneering      (see
app. I for the identification        of these 24 pioneering      proj-
ects).    On the basis of statements made by OSR officicals,
we concluded that about half of the OSR projects           that were
active during 1968 and 1969 were of the pioneering            type.

       The other two types of projects are classified as sup-
porting   and connecting, which are described by OSR as fol-
lows:

      --Supporting     research is aimed at acquiring         knowledge
         or understanding        needed by development groups to
         extend capabilities         beyond the limits    of present
         technology,      Initiation     of this type of research is
         based on a high degree of awareness of Air Force re-
         search needs by both the project          scientist    and the
         investigator.

      --Connecting      research consists   of research projects    in
         scientific     areas which are of interest    to the Air
         Force but which are largely       supported by other agen-
         cies.      By spending relatively    small amounts, the Air
         Force is able to capitalize       on the much larger    expen-
         ditures     of others.

      Of the 22 projects   in our sample that were classified
as other than pioneering,    13 were classified  as supporting,
two were classified    as connecting, and the other seven were


                                    17
classified  as combinations  of the three classifications.
It  would appear that, of the three classifications,       the
supporting  classification  would be most clearly     relevant
to the Air Force mission.

Testimony    of Air    Force officials

       In June 1969 hearings before a subcommittee of the
House Committee on Appropriations,            officials       of the Air
Force acknowledged that OSR had interpreted                 the Air Force
mission very broadly.          With specific      reference     to an QSR-
supported project      entitled      %tudy of Cognitive         and Effec-
tive Attitudes,DF    the Assistant       Secretary      of the Air Force
(Research and Development) told the House subcommittee that
this was not the type of research that the Air Force should
be putting    its resources       into.   At the same hearings,          with
respect to behavioral         sciences studies in foreign           countries,
the Air Force Deputy Chief of Staff (Research and Develop-
ment) stated,     Y agree with [the subcommittee]               that too
many of these studies have not been relevant                  in the past,
*** B**

National    Science    Foundation

       Some of the projects     which were undertaken by OSR were
of a general nature which would have been supported by the
National   Science Foundation had funds been available,         We
observed two instances      (one of which was not part of our
sample of 46 projects)      in which the researcher   had first
applied to the National       Science Foundation and, having been
turned down because of lack of funds, applied subsequently
to QSR and received support.

       In one instance the National Science Foundation advised
OSR in September 1964 that it had declined a proposal en==
titled   #measuring the Gamma Ray Spectrum of Supernova Rem-
nants'@ because:

      NJ;* our stringent   budgetary situation      during the
      present fiscal   year makes it impossible      for us to
      take on another balloon research project         in addi-
      tion to Stratoscope   II at Princeton    University.
      *** we have told [the principal     investigator]
      in person that it is simply lack of funds,
      and not the absence of scientific merit.            We ad-
      vised him to apply to AFOSR and NASA.lC

       In another instance the National      Science Foundation
informed OSR, with respect to another research proposal en-
titled   *Theoretical  Nuclear Astrophysics,'*     that:

      O-j;** we have told [the principal investigator]
      that lack of money does not permit us to support
      this new program, and that he might apply to
      AFOSR or to the ARC."

       OSR records showed that it had accepted both of           these
proposals    and that both research efforts   were still        continu-
ing at the time of our review.      A total of $602,800         has been
obligated    for the former project  and $99,500 for the         latter.

         In commenting on our report,        the Air Force stated that
by including     these two eases we were implying that,           if
the National      Science Foundation declined to fund a scien-
tifically     acceptable   effort,     the effort    must be inherently
irrelevant     and therefore      unacceptable    from the Air Force
point of view.       We did not intend such an implication           but
included these cases only to illustrate              that research of a
general nature and of general interest              was brought within
the purview of relevance to the Air Force under the Air
Force's broad interpretation           of relevance,

Relevance   based upon 5-year     plan

        In discussing   the general lack of documentation    estab-
lishing    the relevance    to the Air Force mission of the 46
projects     included in our review, OSR officials    advised us
that the relevance of each project      was established    in rela-
tion to the Office of Aerospace Research 5-year plan.

       We were advised that Headquarters,  United States Air
Force, had a plan for the accomplishment    of the Air Force
mission and that part of this plan spelled out the need
for research and development;    this is amplified   by the Of-
fice of Aerospace Research 5-year plan which delineates
specific   areas of research which are relevant    to the Air
Force-mission.



                                  19
       We were not given access to the .5-year plan for the
stated reason that it contained financial         data for future
years.     Nevertheless,     we were provided with copies of those
sections of the plan to which the 46 projects          were stated
to relate.     An illustration     of the type of information     in-
cluded in the plan is shown below for that portion           of the
plan involving      the project   entitled  YMethodology for Anal-
ysis of Internal       Social Movements,$' discussed on p8 If.

     "Strategic     Forecasting      and Planning      Studies

     These studies should involve interdisciplinary
     efforts     by sociologists,        political     scientists,
     economists,        and systems specialists         in explor-
     atory studies of methods for assembling and han-
     dling complexdataon          foreign     social systems and
     changing world conditions.               The use of computer
     simulation       techniques and dynamic models of soci-
     eties and nations to improve the reliability                     and
     accelerate       the projection      of strategic       and his-
     toric processes:, the specification               of the most
     likely    future outcomes, and the description                 of
     probable future military            environments      will be en-
     couraged through the support of research in con-
     tributory      fields   of science.         The results     of these
     studies should be presented in forms suitable
     for a variety        of military     uses, including        planning,
     training,      and future operations,           Guidance con-
     tained in Air Force planning and technical                    objec-
     tives documents for the period 1967-1982 empha-
     sizestheimportance           of studies relevant          to the
     military     role in dealing with technological,
     political      and economic aspects of strategic               in-
     terest.

     DrSuch studies will include appraisal          of world sit-
     uations likely     to affect    the level,    and type of
     intensity   of warfare and military        strategies     for
     succeeding years; assessment of the capacity of
     U.S. science and technology to contribute             to sys-
     tems development and the organizational             and force
     structure   required to meet future needs; forecasts
     of the capabilities      required by the Air Force in
     maintaining    readiness for engagement in the full
      range of military     involvements;   and capabilities
      required    to support friends    and deal with threat
      and conflict.

      glComputer-facilitated      studies of critical   prob-
      lems will be encouraged where adequate data are
      available   for making analyses and forecasts;        where
      data on foreign       areas are limited,  other types of
      simulation    studies will be undertaken.OR

       We believe that each individual         contract  or grant
should be sutiported by a statement which s&s forth the
basis on which it was concluded that the expenditure              of
Air Force funds was warranted.          Most of these projects       in-
volve expenditures      of many thousands of dollars       of &ppro-
priated   funds.     It seems completely     reasonable,   in the
light   of practices    normally followed      in the expenditure      of
Government funds, that a full        disclosure     be made of the
need for the expenditure.

       The Air Force informed us that,   to improve the documen-
tation   of Air Force needs, a new section would be added to
the 5-year plan.




                                   21
Aa FORCE,
        RFXIEWOF OSRYROJECTS
       Section 203 of the 1970 Defense Procurement Authoriza-
tion Act, Public Law 95-121, approved November 19, 1969,
required    that funds authorized     by that act be used only for
support of projects    having a direct      and apparent relation-
ship to a specific    military    function   or operation.

       On December 2, 1969, the Deputy Secretary             of Defense
issued a memorandum to the heads of the military               services
and defense agencies,           stating  that section 203 was, in ef-
fect,    reiterative       of the legal principles     and longstanding
policies      which had governed and would continue to govern
the use of Defense appropriations             for research and develop-
ment activities.           He requested a review of all current        re-
search projects,          to ensure that the longstanding      departmen-
tal policy,        requiring    relevance to military    missions,    had
been and was being applied explicitly              in every case.

        In implementing      the Deputy Secretary    of Defense's in-
struction,     the Air Force, subsequent to the completion          of
the GAO review, reviewed all OSR projects            by using three
levels of review by panels or teams.            The panel members were
designated     from the Office of Aerospace Research Headquar-
ters and field      organizations,     as well as from Headquarters,
U.S. Air Force,       and  from  Air Force  Systems Command. The
panel members were to classify          each project   into one of
the following      categories,

      1. Work units     considered    relevant.

      2. Work units     considered    relevant      but inadequately
         justified.

      3. Work units     considered    irrelevant.

        As a result    of the Air Force's top management review,
115 OSR research projects        were disqualified because of in-
sufficient     relevance.

      A substantial number of those projects   were disquali-
fied because they were more appropriate    for support by
other agencies,  as discussed in the following   examples.


                                     22
Air Force nuclear      physics
research program

        The Air Force disqualified   26 of the 27 projects    in
the nuclear physics program.       About $12 million   had been
expended on these 26 projects.       Of this amount, about $2.3
million    had been expended since 1953 on one project--Solar
Flare Phenomena and Electromagnetic       and Particle  Environment
of Space Radiation    Hazards.

       Historically,        the Air Force had been involved in re-
search that would develop new nuclear energy and propulsion
sources.      As stated in the Air Force publication             entitled
"Air Force Research Objectives             1969," the nuclear physics
program was to focus on three areas of importance:                   cosmic
rays j  nuclear      structures,     and high-energy    physics.     The
publication       pointed out that the study of cosmic rays was
important      to the prediction        of the severity   and duration     of
radio communications           blackouts.

        The Deputy for Laboratories     advised us that, after re-
viewing the Air Force's nuclear physics program, he believed
that the type of research that the Air Force had been pur-
suing was not directly     related   to Air Force functions,       For
example, although limited       areas of nuclear physics research,
such as the effect     of cosmic rays on communications,        were
related    to Air Force functions,     other areas of cosmic ray
study for schemes of power generation         were not directly     re-
lated and were properly      the function    of the Atomic Energy
Commission and the National Aeronautics         and Space Adminis-
tration.

Air Force rabies problems--research  on
better prevention  in Southeast Asia
and other epidemic areas

      The research project was related  to the development of
a better  rabies vaccine.  The Air Force Office of Aerospace
Research had sponsored this research since March 1969 in
the amount of $56,801.

       The project  was disqualified  during the section 203
review, because the review panel felt that rabies was not a
unique military    problem and that research in this area


                                      23
should be conducted by the Communicable        Disease   Center    of
the National Institutes of Health.

     We learned that the National  Institutes   of Health had
been engaged in rabies research for over 30 years and had
awarded 29 rabies research grants totaling    $2.3 million
over the past 12 years.

Aerospace medical aspects      of
rapid diagnosis   of disease
and detection   of pathogens

       The purpose of this research      project  was to develop
techniques    for the early detection     and identification   of
viruses   and other microorganisms.

      It had been sponsored by the Office of Aerospace            Re-
search for 5 years in the amount of $372,201,

        According to information      provided by the Office of
Aerospace Research, the review panel decided that the need
for these clinical        techniques was not unique to the Air
Force, since it would use the techniques          in the same way as
would a civilian       hospital.    It was felt that research in
clinical     techniques was the responsibility      of the National
Institutes      of Health.




                                    24
INTERNAL AUDIT

      OSR did not have an internal      audit staff,     and Air
Force auditors     had not reviewed OSR in 3 years.        The Execu-
tive Director    informed us that internal     control    over OSR's
program was exercised     through discussions    at staff meetings,
continual   review by supervisors,     and presentations      at the
annual OSR program review.       We found, however, that such re-
views did not include independent checks of performance.

      We were advised in May 1969 that the financial             manage-
ment staff    of the Office of Aerospace Research, OSR's par-
ent organization,        had started   an internal   review of certain
aspects of management designed to appraise the top manage-
ment regarding      compliance with established       procedures   for
processing    scientific      reports.   Because this review had
been recently     instituted,      we did not evaluate its effective-
ness.    An Office of Aerospace Research official           stated that,
as additional     staff became available,        such reviews would be
expanded to cover other areas of activities.

      The measures taken by the Office of Aerospace Research
should, if properly   carried out, help overcome the problems
discussed in this report and should ensure more effective
performance.

CONCLUSION

       OSR interpreted    relevance to the Air Force mission very
broadly in selecting      research projects    which it funded.      We
believe that this situation         is due to the fact that neither
DOD nor Headquarters,       Air Force, provided definitive      guide-
lines for OSR to use in selecting         the research projects     to
be funded.      Further,   statements   of need for the research
projects    did not adequately establish      the basis for funding
individual     research projects,     most of which involved the
expenditure      of many thousands of dollars.

AGENCYACTION AND OUR EVALUATION

      At the conclusion   of our fieldwork,    we informally
brought our findings    and observations    to the attention    of
OSR. In April 1970 we formally       brought our findings     and
proposals for corrective    action to the attention      of the


                                   25
Secretary    of Defense.     We suggested that DOD issue defini-
tive guidelines     which would adequately limit       the defense
agencies and military       services to funding projects      which
were clearly    relevant    to their missions.      These guidelines
should also require      that, in evaluating     the need for fund-
ing relevant    research proposals,     consideration     be given to
the most appropriate       agency to support the research in view
of the missions and research performed by other Government
and by private     organizations.

       We also suggested that DOD issue definitive           instruc-
tions which would require      that the military      need for each
contract    and grant be fully     disclosed   and clearly     estab-
lished in writing     and that these justifications        be made a
part of the official     files   relating    to the contract      or
grant.

       The Director      of Defense Research and Engineering         re-
plied to our proposals by letter             dated June 15, 1970,
(See app. II.)       DOD said that relevance had always been an
established    criterion      of their research activities       and
noted that,    even prior to enactment of the 1970 act, actions
were taken to deemphasize certain              areas of research and, in
some cases, to transfer          projects    to other Government agen-
cies.    This reply did not directly            comment on our proposals
for corrective      actions by DOD, We were advised, however,
of the following       corrective     actions which were taken by the
Office of Aerospace Research subsequent to the discussion
of our tentative       findings     and the passage of section 203,

       The Office of Aerospace Research issued a regulation
establishing     a policy on the criteria         to be used in the se-
lection    and evaluation       of research efforts    and revised an
existing    regulation     to incorporate     this new criteria    in the
proposal evaluation        procedures.     These regulations     now re-
quire that research proposals be evaluated for relevancy
and Air Force need and, more specifically,             that each re-
search effort      selected for procurement have a direct          and
coherent relationship         to an Air Force function      or operation
through a technology problem area.             These regulations     also
require    an evaluation      and determination     as to whether each
research proposal would be more appropriate              for the Air
Force than for some other agency.



                                   26
       In response to our suggestion on documenting the mili-
tary need for research efforts,       we were also advised that
the revised regulation       of the Office of Aerospace Research
requires    that an official    form for documenting the evalua-
tion of relevancy    and need be placed in appropriate     files.
An instruction    was also issued for the preparation     of this
form.

      We believe that the actions taken by the Air Force in
establishing    evaluation    criteria    and documentation     require-
ments should, if properly         implemented,   improve the manage-
ment of the basic research program administered              by OSR. We
plan to review the implementation          and effectiveness      of the
Air Force criteria      and requirements      at a later date.

RRCOMMFNDATION

      In view     of the similarities      in the research programs
of the military       services and defense agencies, we recommend
that DOD take      appropriate     measures to ensure that each of
the military      services    and defense agencies has consistent
and adequate      evaluation    criteria   and management controls.




                                  27
     NEED
     -_- -- FOR    BETTER SURVEILLANCE
             -_-------    --_l__--~-_  OVER RESMCH             PROJECTS

       Under OSR's system of management, project             scientists
are delegated the responsibility          for monitoring      the work on
OSR-sponsored research projects.            One of the objectives         of
this monitoring     is to ensure that the research group does
the work it agreed to perform and does not deviate unduly
from agreed objectives        and negotiated     budgets.     Our review
of 46 projects     disclosed    that OSR's informal       monitoring      pro-
cedures were not consistently         and uniformly     followed,       and we
believe more effective       monitoring     could have prevented cer-
tain administrative       problems and major deviations          from
agreed objectives      and budgets.

OSR REQUIREMENTSPROVIDE FOR MONITORING

        OSR operating  requirements     provided for visits       to the
project    site by the OSR scientist       responsible    for the proj-
ect and charged the scientist        with responsibility       for seeing
whether the agreed amount of scientific            effort  was being de-
voted to the work OSR was sponsoring.            Inherent    in OSR in-
structions     was the requirement    that major deviations         from
agreed objectives     and negotiated     budgets be acceptable         to,
and approved by, OSR.

       The only written         guidance on how this monitoring           was to
be performed,      other than the general requirements              cited
above, was contained in OSR Headquarters                 Operating    Instruc-
tion 80-1, dated November 13, 1964.                 It set forth OSR policy
which stated that the amount of time proposed to be devoted
to a research project           by scientific      personnel was one of the
principal    determinative        factors     in the selection     of proposals
to be sponsored.         The instruction        provided for OSR scientists
and contracting       officers,      during each visit      to research
sites,    to inquire whether the proposed amount of scientific
effort    was being devoted to the projects.

      Beyond this instruction,    QSR scientists  were not pro-
vided with written   guidance on how monitoring    was to be per-
formed or on documentation     of the monitor's  observations   and
of the agreements he reached with the research group.


                                        28
      Generally,     good monitoring       requires     periodic   visits    to
the project     site, written    postvisit       reports on the results
thereof,    and analysis of technical           and nontechnical      reports
submitted by the contractors         and grantees to inform Govern-
ment scientists      and contracting       officers     on the status and
progress of research work.          Good monitoring         also includes
systematic    consideration     of the work being done to ensure
that the contractor       or grantee is complying with the terms
of the agreement and the recording              of deviations    in writing
for resolution.

MONITORING NOT DONE
CONSISTENTLY AND UNIFORMLY

      We found that,    to some extent,   OSR scientists      complied
with the informal    standards by visiting    domestic contractors
at least once a year and by obtaining       annual reports on the
progress of the research projects.        These procedures,      how-
ever, were not consistently     and uniformly    followed.      We ob-
served cases where some or all of the monitoring           standards
were not met. For example, we found that OSR scientists              had
not obtained progress reports      for the most recent period of
support from four of the 46 research efforts          we reviewed,
because a progress report had not been requested.

        Also, we found that, during the year prior to our review,
four of the 36 domestic projects              we reviewed had not been
visited.        Several of these projects         involved problems in
meeting grant and contract           obligations      with respect to com-
pleting     the research and submitting           scientific      reports to OSR.
Moreover, we found that generally               OSR did not require       its
project     scientists     to make written       postvisit     reports   on mat-
ters observed or agreements made. Contract                    and grant files
maintained       by OSR scientists      for 26 of the 35 projects           visited
in the last year prior          to our review contained no written               rec-
ord of discussions         held or agreements made during the project
site visits.         Several scientists       stated that they follow the
practice      of orally    reporting    significant       matters observed
during site visits.

EXAMPLESOF ADMINISTRATIVE PROBLEMS

         The four.projects     below illustrate   the need for          addi-
tional     practice    of consistent   and uniform monitoring.            Had

                                        29
monitoring  procedures been more consistently     followed,    we
believe that problems and deviations    from contract     agree-
ments could have been disclosed,   providing   greater assurance
that research funds were being used effectively.

Aabraic
     -------. Them _-of--- Machines
        In 1965 OSR awarded a l-year contract        for $25,644.
This contract     was negotiated     on the basis that the principal
investigator     would devote half of his time to this research,
for which he was to receive $7,500.           OSR extended the con-
tract in 1966 and 1967. In each case, the principal                 inves-
tigator    proposed to devote 6 months of his time at a cost of
$8,000.      The contract  involved over the 3-year period a to-
tal of $79,000 of which $23,500 was proposed and negotiated
specifically     for the effort     of the principal     investigator
for 18 months of his time.          Upon completion    of the contract,
however, the principal       investigator   advised OSR that he had
devoted only 2 months at the outset of the research and that
16 other persons had been working at various times on this
research.

       The current OSR project         monitor stated that he had
never met these persons and did not know their qualifications.
OSR project     files   contained no indication         that during the
life of the contract         OSR scientists      had detected that the
principal    investigator       did not live up to his proposed bud-
gets.     OSR officials      told us they became aware in June 1967
that the principal        investigator      had too many other research
commitments and notified           the principal    investigator   in Au-
gust 1967 that OSR probably would not renew the present con-
tract.

        The OSR project   scientist    currently      responsible     for mon-
itoring     this effort  advised us that he had not made any
visits    to the contractor,     although the contractor          was lo-
cated in Washington,      D.C., and that the project           had not been
visited     by OSR personnel since June 1965. He told us that
visits    to the research site in this case were not considered
necessary because the principal          investigator      had visited      OSR
on several occasions and also had submitted progress reports.
It appears that these visits         and progress reports         alone are
not adequate to keep the project           scientists    properly     informed.


                                      30
Had visits   to the research site been made and had inquiries
been made as to whether the proposed amount of scientific
effort   was being devoted to the project,   as required  by OSR
Headquarters   Operating Instruction   80-1, we think that the
contractor's   deviation  from the agreement with OSR would
have been detected much earlier.

Cosmic Radiation      of Extremely      High Energy

        The files     on this project   contained no evidence that
significant      deviations,     made by the grantee from the nego-
tiated    budget, had been made known to OSR or were done with
the knowledge and approval of the project           scientist     and the
contracting      officer.     OSR had supported this research proj-
ect since 1964 and had expended or authorized            $458,000
thereon.      The initial     grant for $178,000 was solely for the
purchase of equipment to help build 100 stations              for observ-
ing cosmic ray showers.

      Prior to the award of the grant, the grantee specifi-
cally agreed to adhere to the negotiated       budget and agreed
to bear all labor costs associated with the construction        of
the stations.    We were informed,  however, that only 34 of
the proposed 100 stations    were built with funds provided un-
der this grant and that a portion      of funds was used for pay-
ment of salaries   of researchers.

       Although OSR scientists       had visited    the research site
in July 1965, November 1966, and May 1967, they could not
explain the reasons for the deviation.             The contracting  of-
ficer    responsible    for this grant at the time of our review
stated that he could find no evidence that the grantee had
requested and that OSR had given approval for deviation             from
the negotiated       budget.   After we brought this matter to the
attention     of OSR's officials,      several letters   were sent to
the grantee requesting        explanations    of the use of grant
funds.

        OSR officials      told   us that they were aware of this de-
viation    from the agreed        budget through the periodic   fiscal
reports    submitted by the        grantee and that acceptance of
these reports       indicated     OSR approval of this change.     We
noted, however, that the           fiscal  and progress reports pre-
pared by the grantee did           not explain the reasons for

                                       31
deviating     from the agreed budget, as required        by OSR grant
provisions.       Moreover, there was no evidence in project
files   that, prior      to our review, OSR had investigated       this
matter,despite       the fact that OSR personnel visited      the
grantee--located        at a distance of some 10,000 miles from
Washington,      D.C .--during    the time the deviation   occurred.
There were no postvisit         reports prepared for the visits       in
1966and 1967.

      The Air Force advised us that,       despite deviation   from
the budget, responsible       program managers were satisfied     that
the overall   objectives    for which the DOD research money was
expended were met.       The fact remains, however, that the Air
Force did not get 66 of the 100 stations         that were proposed
to be built  with these funds and that this matter was never
brought up for consideration        at top management levels de-
spite three monitoring      visits.

        To the extent that the OSR monitoring             activities    of this
project    did not result     in necessary information            being ob-
tained during project       visits     and being appropriately         docu-
mented to provide a basis for management and procurement ac-
tions,    OSR resources--research          funds, time of project       scien-
tists,    and the travel    funds involved        in the three visits--
were not being efficiently         utilized.        We believe that sys-
tematic considerations       of the work being done during visits
and that recording      deviations       in writing    for resolution
would have assured more prompt consideration                of the devia-
tions in this case.




                                      32
Theoretical   Research   in Astrophysics

       OSR supported this project      from June 1960 through Sep-
tember 15, 1967, at a cost of $274,000.         According to proj-
ect files    and OSR officials,     progress and postvisit   re-
ports were not prepared.        A memorandum to the files   showed
that OSRss monitor evaluated the progress of the research
in January 1966 and concluded that the research objectives
were so general that it was difficult        to determine what was
to be done or whether anything had been accomplished.           He
stated:

       "'The proposals received from the university          for this
      effort    have in general been unsatisfactory        ** since
      the documents are not complete.          For instance,     the
      proposal dated 15 January 1964, did not provide a
      review of previous work with citation          of papers pub-
      lished.     The work to be pursued in the continuation
      of the support for the next two years was described
      only in a general way. The document states              'Work
      under the current grant has been supporting            graduate
       students and research associates        working in astro-
      physical    subjects.'    The last status report describes
      the subjects that are at the moment under investiga-
      tion.     The detailed  projects   will,   of course, change
      depending on developments in the subjects           and on
       student and faculty    interest."

      The project   monitor informed us that OSR subsequently
declined to further    support the project  because of the
broadness of the research work and the tight      budget.

       We believe that the $274,000 might have been used more
effectively     if there had been systematic    evaluation  of prog-
ress during the visits     and use of postvisit     reports to in-
form OSR managers of problems occuring under the contract.

Theoretical  and Experimental   Studies in
Magnetohydrodynamics   and Plasma Physics

      OSR awarded a 5-year grant for $66,667 to a university
in 1963 to provide two-thirds   of the salary for a professor
of space sciences to help the university    establish a de-
partment of aerospace sciences.    The research proposal listed


                                   33
several research objectives     to be accomplished.     We found,
however, that OSR had not received any fiscal        and scientific
reports  in the 5-year period the grant was in existence           and
that progress reports had not been submitted        subsequent to
December 1966. A letter     submitted by the grantee in May
1968--4 months before the grant was to expire--was         accepted
by OSR in 1969 as a final    scientific  report required under
the terms of the grant.

      We found also that OSR project  monitors had not visited
the grantee,   located in Washington, D.C., for a 3-year pe-
riod and had not followed   normal monitoring   procedures with
respect to this grant.    Under these circumstances    there was
a lack of assurance that the research objectives      were being
achieved.

        OSR officials advised us in October 1969 that the "con-
tribution    from the grant was important    and considerable"
and that,    in their opinion,  the grant was a success.       The
research group headed by the principal       investigator   had
produced a number of scientific     reports.

CONCLUSIONS

       We believe that OSR can provide greater assurance that
deviations    will be detected promptly by establishing                  firm
requirements     for periodic      progress reports          from researchers;
for mandatory visits,         at least annually,           to researchprojects
for purposes of ascertaining            scientific      progress and possi-
ble problems;      and for written        reports     on observations      made
during such visits,        with particular         attention     to deviations
from agreed objectives         and negotiated         budgets and problems
in meeting established         contractual       obligations,       We believe
that written     monitoring     requirements,         if carried    out con-
sistently    and uniformly,      would enhance management control
over the research grants and contracts,                 would facilitate       ap-
propriate    administrative       action,     and would provide better
protection    of the interest       of the Government.

AGENCYACTIONS AND OUR EVALUATION

      To improve the effectiveness     in maintaining    surveil-
lance over research projects,      we suggested that OSR require
contractors   and grantees to submit periodic      progress reports.


                                        34
We proposed also that OSR issue guidelines             to its project
scientists    establishing        a minimum interval   between visits
to research project        sites,    specifying   the aspects that
should be evaluated during each visit             to a grantee and con-
tractor    and requiring      that all visits     be appropriately    doc-
umented to cover such matters as significant             observations
made, results     of discussions         held, and an evaluation     of
progress to provfde a basis for management and procurement
actions,

        Commenting on our suggestions,       the Air Force informed
us in June 1970 that the OSR had issued instructions             in
April 1970 requiring        contractors   and grantees to submit an
annual report containing         a statement on research progress,
listing    of publications,      and contemplated   major deviations
in program and budget.          The Air Force also informed us that
appropriate    procedures have been issued in February 1970
requiring    a minimum of one visit       to the research project
site each year.        The procedures also contain guidelines        for
information    to be obtained on each visit        and a requirement
for trip reports       documenting the results     of these reviews.

      We believe that these measures, if properly   implemented
and sustained,   will enhance management control  over the OSR
research program.




                                    35
                              CHAPTER4
                              ---

                  RESEARCHRESULTS
                               --. NOT OBTAINED
                    OR NOT OBTAINED PRQIXE'TLY

       QSR project     scientists    are repuired    to ensure timely
receipt   of the scientific       reports produced under the re-
search that OSR sponsors.          In the 46 projects     we examined,
we found that,     in 22 projects,       all published reports had not
been received and that long delays ensued between comple-
tion of the reports and their           submission to QSR. In 23
cases,   OSR  scientists      had not  made  appropriate   checks to see
whether all reports had been received.

SCIENTIFIC REPORTSMUST BE
SUBMITTED To OSR PROMPTLY

       Because OSR recognizes       that scientific   reports are its
primary products,     its policy is that such reports must be
submitted    to OSR promptly      so they can be appropriately    uti-
lized to reduce the leadtime between discovery            of new kns~l-
edge and its application          to the advancement of aerospace
technology    and to the maintenance of Air Force operational
superiority.      The procedures which OSR had issued to imple-
ment this policy provided that grantees and contractors
submit scientific     reports     to QSR in the required number of
copies immediately      following    publication.   OSR project
scientists    were responsible      for assuring that all scientific
reports produced under OSR-sponsored research had been re-
ceived.

       To ensure that all such reports had been received and
processed promptly,    OSR Headquarters  Operating Instruction
80-3, dated September17, 1966, required      project scientists
to check listings    of research results  that the grantees and
contractors   were expected to submit.

REPORTSNOT RECEIVED OR DELAYED UNDULY

       We found that in 23 of the 46 projects    we reviewed,
OSR scientists    had not always checked the listings     of re-
ports produced to see whether all of the reports had been
received by OSR. In eight instances OSR's files         were not
complete,    and we collld not make a conclusive   determination
                                  36
as to whether all reports  issued by the researcher  had been
received by OSR. In 22 other instances,   however, we found
that one or more of the reports either  had been received by
OSR only after a long delay or had not been received at all,

     According to the Air Force, the reports which had not
been received represented  about 7 percent of the documents
that should have been included in OSR files.

        For example, upon completion            of the research effort
entitled    DBPolitical    Development and Modernization           in Is-
lamic Countries:        Military      Planning,*@ supported by OSR for
 $64,140, the principal          investigator     reported   that 11 articles
and monographs had been published during the period 1964
through June 1968, OSR records showed that only one report,
published in 1964, had been received by OSR as of February
1969. The cognizant OSR scientist                advised us he was aware
that several papers had bean written                but that he would ask
the principal     investigator        for copies of all reports       pub-
lished when the final         scientific      report was received by OSR.
Records show that, during February through June 1969, OSR
received and sent to the Defense Documentation                  Center 10 re-
ports that had been published              in the period 1964 through
1967.     Some examples of the reports            that were delinquent
follow:

                                             Year of          Date of
               Title                       publication   receipt  by OSR

Ideological    Foundations of
  Egyptian-Arab    Nationalism                  1964         Feb. 1969
Egypt:    The Integrative    Revo-
  lution                                        1964         Feb. 1969
The Proofs of Islam:       Religion
  and Politics    in Iran                       1965         Apr. 1969
Political    Change in Lebanon                  1966         June 1969
The Tragedy of Syria                            1967         May 1969

       OSR officials  told us that the principal   investigator
had not sent manuscripts    of the 10 delinquent   reports    to
OSR, although OSR procedures required      that a manuscript
copy be furnished    at the time the report was sent to the
publisher.



                                      37
       In another case involving      a research project        entitled
9DAtomic Ghemistry,O' supported since 1961 at a cost of
$492,000, scientific       reports published during 1967 were not
furnished    to OSR for purposes of appropriate         distribution
until    December 1968. OSR officials       advised us that these
reports were delinquent        because of administrative       problems
encountered by the grantee during a half-year            absence of
the principal    investigator.

        OSR officials      acknowledged the fact that there were
some delays in receiving          reports because of known built-in
publication      backlogs of up to 2 years.          They pointed out,
however, that they considered direct             contact of interested
scientists     at professional      meetings,    seminars, and confer-
ences as the primary method of disseminating              research re-
sults.     For   this   reason  they   do not  regard   delays   in receiv-
ing published       reports   as a severe impediment to the dissem-
ination    of scientific      knowledge.     They stated that,     if OSR
project    monitors recognized        a subject or an idea that was of
concern to an Air Force laboratory,            the monitors could
immediatelysendmanuscript           copies to such users.

      OSR scientists     did not know, however, whether all prin-
cipal investigators      supported by OSR periodically       present
their achievements in professional        meetings and seminars.
Moreover, as stated on page 36 of this report,            we found
that OSR did not always receive manuscripts          of reports     and,
in the examples described in this section,         the OSR scien-
tists   could not or did not forward manuscript         copies of the
delinquent    research reports    to other Air Force scientists
and laboratories.      In view of the high cost of producing
research results,     prompt receipt    of all scientific      reports
by appropriate     Government recipients    is necessary to real-
ize maximum return on the investment.

CONCLUSION

        When scientific  reports    are not received promptly or
when delays in obtaining        them are experienced,   research
results    cannot be effectively     evaluated and disseminated      by
OSR, and the return on these results         may not be fully    real-
ized or its realization      may be delayed.     Although dissem-
ination    of scientific   research results    may take place



                                   38
through the less formal methods at professional       meetings and
symposia, we believe this should not be a reason for OSR
scientists  to permit such delays in receiving     reports as de-
scribed in this chapter.    We believe that OSR procedures
established   to ensure prompt receipt   of the results    of re-
search sponsored are generally    sound and well conceived,
What is needed is more vigorous    implementation   of these
procedures.

AGENCYACTIONS AND OUR EVALUATION

       To improve the effectiveness       in achieving maximum return
on the millions     of dollars    invested annually in basic re-
search, we suggested that OSR place greater           emphasis on im-
plementing   existing   procedures with respect to obtaining
promptly all scientific       reports produced under the research
projects   supported.

        The Air Force informed us in June 1970 that OSR proce-
dures had been augmented in April 1970 to include a review
of listings     of publications    in the annual progress reports,
Also, an OSR official        had been given responsibility  for mon-
itoring    the compliance with this procedure.

         We believe that these measures, if carried    out consis-
tently     and uniformly, will ensure prompt receipt    of the re-
sults     of research sponsored by OSR.




                                 39
                                    CHARTER 5

                NEED FOR MORE EFFECTIVE DISSEMINATION

                             OF RESEARCHRESULTS

        The results     of research projects      sponsored by the Air
Force are to be promptly disseminated             to those responsible
for developing Air Force weapons and technology               as well as
to other interested         governmental   organizations.      Making re-
search results       available     to such persons and groups is nec-
essary if the fruits         of research are to be used effectively.
We found, however, many instances           in which published      re-
search results       either    were not directly    made available      to
interested    parties      or were made available      only after   long
delays.     OSR did not make prompt dissemination            of these
published    results     because its operating      procedures were not
effective    enough to ensure that research results            were uni-
formly disseminated         on a timely basis.

INFORMAL DISSEMINATION
OF RESEARCHRESULTS

        The requirements         for dissemination       of research results
obtained by OSR were contained in Air Force Regulation                        80-4,
dated April        11, 1966, and Regulation          23-18, dated Janu-
ary 31, 1968.         These regulations         required    that the results
of Air Force-sponsored             research be effectively          disseminated
to those responsible           for the development of improved aero-
space technology,          weapons, equipment,         and operations.        Air
Force Regulation         80-4 and the Office         of Aerospace Research
command policy provided that publication                  of results      in sci-
entific      journals    be encouraged and required             that copies of
findings       and results     be furnished      promptly     to the Defense
Documentation        Center so that the data would be available                   for
secondary distribution             to all eligible      users.      Office of
Aerospace Research policy              also provided for review of exist-
ing distribution         lists     for currency so that appropriate            ini-
tial    distribution       of reports      could be held to a minimum.

      As discussed in chapter 4, OSR could not have accom-
plished  effective  distribution of research results  in a num-
ber of instances   because it did not obtain research results


                                         40
promptly.    In addition,     we found that, even when results
were obtained by OSR, they had not always been disseminated
promptly or in an appropriate        manner.     In this respect,      in
23 of the 46 projects     we examined, OSR project          scientists
had not distributed     the scientific     reports obtained directly
to selected potential      prime users within      the Air Force or in
other agencies,     Also, reports      in certain    instances     were not
sent to the Defense Documentation Center for secondary dis-
tribution  to other interested       users as required       by regula-
tions.

        For example, the final        report received in February 1968
and certain     manuscripts     of journal     articles   produced in
September and December 1968, under a project               entitled   "Qua-
dratically    Nonlinear     Differential      Systems," were not sent to
selected Air Force users or to the Defense Documentation
Center even though this project            had cost the Government
$76,000.     The Air Force advised us in June 1970 that the
results    of this project      had been presented at a scientific
meeting in January 1969 and an article               had been published
in March 1970. We do not believe that this could be con-
sidered adequate and effective            dissemination    of research re-
sults to potential      Air Force and other governmental            users.

       In another example, the cognizant OSR scientist              ap-
proved reports produced under a project           entitled     "Studies
of Psychotomimetics I' for distribution       to the Defense Docu-
mentation Center, but he did not send reports              directly    to
selected potential     users in the Air Force or in other agen-
cies.    He stated that he would not consider such distribu-
tion unless the results      involved a major new discovery             and
felt that making the results        available   through the Defense
Documentation    Center was adequate distribution.             A total of
$29,000 had been expended on this effort.

      OSR records showed that about $8.8 million      in Govern-
ment funds were spent to obtain the scientific      reports,   pro-
duced by the 23 projects,   that were not distributed      to se-
lected interested  parties.

      In our review of OSR's operating       instruction,     we found
one reason why selective  initial    distribution        may not have
been made. OSR Headquarters      Operating     Instruction    80-3,
dated September 12, 1966, dealing with dissemination            of

                                     41
research results,    stated     that    each division   chief   and proj-
ect scientist   would:

      "'Take appropriate    steps to exploit     ideas appear-
      ing in manuscripts     by referring    them to other
      interested   agencies or to the AFOSR Assistant
      Executive   Director,    Research Communication
      (SRCC)."

        This instruction       did not specifically      provide for se-
lective     distribution      of results  or notification      of the exis-
tence of such results          to Vsthose responsible     for the devel-
opment of improved aerospace technology“               as specified   by Of-
fice of Aerospace Research policy,            and we found that OSR did
not maintain listings          of selected potential      users for all
projects.        Consequently,     OSRPs operating    procedures would
not have produced results           in consonance with the information
supplied for the record by the Director              of Defense Research
and Engineering         to the Senate Foreign Relations        Committee,1
in which it was stated:
                                                          .
      "There is prompt initial    distribution       of research
      reports to those individuals       and offices    concerned
      with the subject matter and to all to whom the in-
      formation contained could be immediately          useful."

       Since the basic research program supported by OSR is to
be based on the Air Force's needs as identified                 by Air Force
scientists    from various      inputs provided by all Air Force re-
search and development and operational              elements,    the results
should be made available--         in addition    to other methods of
dissemination--     directly    to Air Force elements most inter-
ested in such research if the research is to produce tan-
gible results     for the Air'Force.         Accordingly,     it would seem
that many of the benefits          of the research might be lost if
prompt distribution        was not made to those Air Force groups
having most interest         in the subject matter covered by the
research.


1Hearing before the Committee on Foreign Relations,  U.S.
 Senate, on "Defense Department Sponsored Foreign Affairs
 Research," May 9, 1968.


                                       42
       The need for such selective          distribution       has been rec-
ognized by some OSR project          scientists.        For example, the
scientist     monitoring   a project     entitled      "Feasibility     Study
of the Fluid Transpiration        Arc" maintained         a list    of recip-
ients who were to receive scientific              reports produced.
Among the recipients       were the Air Force Cambridge Research
Laboratories,      Aerospace Research Laboratories,              and other
laboratories.

      We recognize that other methods of disseminating        re-
search results --scientific   meetings,   use of abstracts,     in-
dexes, and the like--are    available   and, in many instances,
are used to supplement the more formal distribution         chan-
nels.    In view of the amount of money spent on support of
basic research each year9 however, we believe that informal
methods of dissemination    should not supplant the more formal
methods.

        We found that the management at OSR did not recognize
a need for systematic         distribution      of research results       to
selected potential        users in the Air Force.            The Executive
Director     advised us that the information            network in the sci-
entific     and technical     community was sufficient          and that in-
formation was disseminated            very rapidly     within   the scienti-
fic community through scientific             journals,       He expressed the
belief    that the existing        flow of information        could not be
accelerated      to any significant        degree.

        Also, we found that OSR project        scientists     believed
that results     of basic research were quickly           and effectively
disseminated     through the information       channels available
among peers in the scientific          community.     For this reason,
they generally       regard a formal method of distribution            as un-
necessary.      Some scientists     have indicated      that publication
in scientific      journals   is sufficient    to assure adequate dis-
tribution     of results    of basic research.

       In discussing  utilization  of selected scientific   re-
ports with several. users who had requested these documents
through the Defense Documentation     Center data bank, however,
we were advised that, owing to the vast number of scientific
journals   and the volume of reports published therein,     it was
not possible    to keep up with all that was being published.
This view was shared also by several OSR scientists       who


                                      43
stated that the volume of scientific  literature had grown to
such proportions that one could not read it all.

       The inadequacy of current methods of dissemination      was
pointed out in a May 1968 report of the Subcommittee on Sci-
ence and Technology to the Senate Committee on Small Busi-
ness, which stated that the transfer     of technical   knowledge
occurred slowly and sporadically    without   some deliberate   ef-
fort to accelerate    its flow.  Thus9 it would seem that the
need for channeling of research results     to selected poten-
tial users was great.

CONCLUSION

        We believe that the effective        use of research results
will be enhanced by making scientific           reports directly
available     to selected Air Force research organizations           re-
sponsible     for developing Air Force weapons and technology.
It would seem that OSR scientists,           who selected the re-
search projects      based presumably on the research needs of
the Air Force, were also in the best position             to determine
which Air Force organizations         had the most interest      in the
results    of such research and should receive copies of sci-
entific    reports,    as contemplated    in the Office of Aerospace
Research command policy on published documents.              Although
other less formal methods may also be used in the dissemi-
nation process, we believe that OSR procedures,             with respect
to dissemination      of scientific    results,    should be expanded
to require      that results   of research be distributed       on a
systematic      but selective   basis to designated potential
users.

AGENCYACTION AND OUR EVALUATION

         To improve the effectiveness          in achieving maximum re-
turn     on dollars     invested in research,       we suggested that OSR
issue     instructions      requiring    OSR scientists    to make the re-
sults     of research projects        promptly   available     to those Air
Force     organizations      having the most interest        in such re-
sults.

        The Air Force informed us in June 1970 that, subsequent
to our review9 appropriate      procedures had been issued re-
quiring    distribution  of scientific    results to selected

                                      44
personnel and organizations.      In addition,    OSR has desig-
nated the use of specific    forms for rapid dissemination       of
information with current developmental       potential.

       We believe that these actions,    if properly  implemented
and sustained,    will enhance the effective    use of research
results.




                                                                      i


                                 45
                                -CHAPTER
                                    ---  6

                            SCOPE OF REVIEW

      Our examination      of the management of the basic            re-
search program administered           by OSR included a review of the
laws and policies        which govern the conduct and support of
basic research by Federal agencies.              The review was directed
toward examining the effectiveness             of management controls
within    the OSR and the Air Force Office of Aerospace Re-
search, including        procedures,    records,     and internal    reviews
relating     to the evaluation       and selection     of research pro-
posals,     surveillance     over research projects,         and evaluation
and utilization        of scientific    reports    resulting    from re-
search projects        supported.     We did not review the terms of
the contracts        and grants for compliance with Government
procurement regulations.

        In our review we inquired       into the circumstances        sur-
rounding the selection,       surveillance,      and utilization      of
results    of 46 of the approximately        1,200 research grants and
contracts    administered   by OSR during the period of our exam-
ination.     Our selection    included projects       from each of the
six OSR divisions      with an emphasis on behavioral            and social
science projects.       Our inquiry     into the 46 contracts        and
grants included a review of OSR files            regarding     these con-
tracts    and grants and discussions        with involved personnel
of OSR.

       Our review was conducted        at OSR and at the Air Force
Office of Aerospace Research,          both located in Arlington,
Virginia.




                                     46
APPENDIXES




 47
                                                    IJSTING          OF 46 PROJECTS REVIEWED


                                                                                            Scientific         area
                                                                                               related       to        General    Woe
                           Title       of pro_tect                                         Air Force        mission     of proie'ci

DIRECTORATE OF LIFE SCIENCES:
     1. Preparation   of Manuscript    for a Reference
           Rook on Laboratory    Animal Diseases                                               Microbiology             Connecting




      2. Studies      of Psychotomimetics                                                      Pharmacology             Supporting


      3. Anatamical         and Physiological                 Studies      on the
           Nervous         System                                                              Physiology               Supporting



      4. Physiology          of Cerebrospinal                Fluid      in Pri-
           mates                                                                               Pharmacology             Supporting

      5. Genesis of Action                Potentials           in Excitable
           Tissue                                                                              BFonics                  Supporting



      6. Political         Development        and Modernization                   in           History,       law,
            Islamic        Countries--Military         Planning                                political        sci-
                                                                                               ence                      Supporting




      7. Influence    of Campus Environment                           on Student
             Commitment to the USAF ROTC                                                       sociology                Supporting




      8. Methodology            for     Analysis       of    Internal       Social
           Movements                                                                           sociology                 Supporting



      9. Perception            of Dynamic          Stimuli                                     Psychology                Supporting



     10.   Criteria      for       the Design         of New Forms of Or-
              ganization                                                                        sociology                Supporting


     11.   Sociocultural              Aspects      of Development                               Social      Psy-
                                                                                                chology                  Pioneering
                                                                                                                 APPENDIXI
                                                                                                                    Page 1

               Status of
    Funds     project   In                          Statement      of relevancy    of project
authorized     November                                         to Air Force mission
for protect       196$                                          as presented    by OSR

                             This book will         aid in the diagnosis,           prevention,      and treatment
 8 82,120      Active        of spontaneous         illnesses      of research      animals.      It will    also be
                             useful     in evaluating         experimental      results    by helping      to dif-
                             ferentiate       between spontaneous           and experimentally        induced      le-
                             sions.       Its long-term        value will      be to point      up areas where
                             mare research        is needed.          The results     will   be relevant     to the
                             extensive       use of animals         in Air Force research.
   28,920      Completed     This       study is    exploratory      in this     context      and relates       to mili-
                             tary       interests    in these      matters.

  151,111      Completed     These studies          are relevant   to Air Force studies    of the effect
                             of the multiple           stresses  of the aerospace  environment    on the
                             body.
                             All of these topics            are related    to the bodily            response    to
   17,605      Completed     stressful  conditions            and adjustment    to hostile            environments.


  109,050      Completed     The results are important  for                understanding      how biological
                             membranes work and how nerves                 respond     and are of interest              to
                             engineers.
                             The proposed research         will     be an investigation          of the relation
                             between national      integration         and political        development.       These
   64,140      Completed     studies   bear directly       on the role of the U.S. military                  in pro-
                             viding  military    advice,        assistance,      and advanced training          in
                             U.S. military    schools      for the indigenous           military     leaders    and
                             on the problems     of long-range          military     planning.
                             The purpose of this research                   is to investigate           the relation-
  147,558      Completed     ship between organizational                  attributes       of the Air Force Re-
                             serve Officers'           Training      Corps at a university              and the pat-
                             terns of recruitment,              retention        and leadership         attainment      in
                             the cadet population.                Comparison        of students        who enter the
                             Corps and those who do not, of cadets who differ                              in duration
                             of Corps career           and cadets who differ             in level       of leadership
                             attainment        will    be utiIized        to discover        longitudinal        processes
                             of commitment of the Corps, as these are affected                                by its in-
                             ternal     social      structure      and its external            organizational        en-
                             vironment.
                             With the techniques    being developed    and tested       it will be pos-
  201,861      Active        sible  to take advantage    of computers   to analyze        more complete
                             data than would otherwise     be manageable.       Results     will be use-
                             ful for environmental     and political   analysis     and forecasting
                             by Air Force systems planners.
  222,380      Completed     The results      of this investigation     will be applicable     to a wide
                             variety   of situations     in which the operator     is called     upon to
                             respond    in some fashion      to the velocity  and acceleration       of
                             visual  stimuli.
                             This handbook           will  be especially    useful     to the Air Force and
  176,774      Active        other military           supervisory    staffs   engaging    in the planning of
                             organizational           change and adaptation        to new conditions.
                             This knowledge           will   be very useful        in   guiding     programs      for
   82,920      Completed     military        aid    and policy   planning.




                                                                    49
                                                 LISTING       OF 46 PROJECTS REVIEWED (continued)


                                                                                     Scientific  area
                                                                                         related to         General   type
                            Title      of project                                   Air Force mission        of project

DIRECTORATE OF INFORMATION SCIENCES (continued):
     12. Automatic English Sentence Analysis                                          Linguistics            Pioneering



      13.   hrolutionary            and Multilevel            Information
               Processing           Networks                                          Bionics               Pioneering


      14.   Contributions         to an Understanding                   of the
              Language         Communication   Process                                Linguistics           Pioneering




      15.   Languages         for    Automatic        Programming           Tools     Linguistics           Pioneering



      16.   Cybernetic         Model     of Learning           and Perception         Bionics               Pioneering



      17.   Selection        of Variables            in Clustering           and      Documentation,
               Pattern       Recognition                                              information
                                                                                      technology            Pioneering

      18,. Problems         in Machine       Problem          Solving                 Linguistics           Pioneering


DIRECTORATE OF CHEMICAL SCIENCES:
     19. Bibliographic      File of Abstracts                      on Photo-          Inorganic             Supporting,
            graphic    Science                                                        chemistry             Connecting


      20.   Analysis  of Electron                Scattering         from Atoms        Physical              Pioneering,
              and Molecules                                                           chemistry             supporting
      21.   Flame Spectrometry              and Gas Chromatographic                   Physical              Pioneering,
               Detectors                                                              chemistry             supporting,
                                                                                                            connecting



      22. Atomic         Chemistry                                                    Organic       chem-
                                                                                      istry                 Pioneering
DIRECTORATE OF ENGINEERING SCIENCES:
     23. Selected    Problems     in the Theory                    of Elec-           Wave propaga-
            tromagnetic     Radiation                                                 tion                  Pioneering
                                                                                                           APPENDIX I
                                                                                                               Page 2



                Status of
                project   in                         Stateiient      of relevancy    of project
authorized       November                                         to Air Force mission
for proj ect_        1969                                         as presented    by OSR


 $261,001         Active       Research     results   should furnish            an understanding         of how com-
                               puters may be used for natural                 language      analysis      for such ap-
                               plications      as text compression,             extracting,       indexing,    and
                               translation.

  162,621         Active       The research        is expected    to disclose       methods of       improving  the
                               information       processing    capabilities       of computers        for automatic
                               pattern     recognition      and circuit     design.
                               Research results     will     be examined for their          applicability      to
   41,500         Active       the problem of partial         understanding      as it arises         in human
                               communication.      This research        has direct     relevance        to the so-
                               lution of practical       communication       problems      as well as to the
                               need for effective       natural     language    processors       in man-man and
                               man-machine    systems.
   20,600         Active       The research  is expected              to add to an understanding     of funda-
                               mentals of the pragmatics                of language, i.e.,   those relation-
                               ships that exist   between             signs and symbols and those who pro-
                               duce and use them.
   93,778         Completed    This work is basic to the development        of intelligent      machines
                               that can be of use in Air Force command and control             systems
                               and in operational     environments    where control     and decision
                               processes  are required      with a minimum of human intervention.
                               Research  results     are expected   to show how to reduce the amount
                               of computation    required    for the discrimination     of large num-
  159,291         Active       bers of objects     for such applications      as target   recognition
                               end weather forecasting.
  259,620         Active       This research     is clarifying         the language     requirements    for de-
                               scribing   problems      and methods of solution           toward formulation
                               of maximally     efficient      specifications       for machine problem
                               solvers.
                               A system was set up 12 years ago under OSR sponsorship    which
  106,859         Active       proved quite useful  to the Air Force.   This new system will
                               extend the former system from 1958 through    1967 and be compat-
                               ible with it.
                               Use will     be made of techniques    developed             in this    research     for
  312,032         Active       studying     aerospace vehicular   hardware.
                               The development     of the flame fluorescence           spectrometric   analy-
                               sis method by the principal          investigator      has made this new
  389,091         Active       technique   a candidate     to replace       emission   spectroscopy  in the
                               rapidly   expanding    Spectrometric       Oil Analysis     Program (SOAP) of
                               the Air Porte as applied        to the maintenance        of aircraft   en-
                               gines.
                               This research will   open up a completely                  new area of chemistry
  492,276         Active       and should make available   new materials                  for Air Force applica-
                               tions .
                               This research         provides  highly    relevant      fundamental     knowledge
  549,074         Active       supporting      Air    Force efforts    in electromagnetic          pulse,   turbu-
                               lence,     antenna,     and propagation       problems.
                                                LISTING     OF 46 PROJECTS REVIEWED (continued)


                                                                                  Scientific        area
                                                                                     related      to           G~;erz           V:
                          Title      of protect                                  Air Force       mission             P      1
DIRECTORATE OF ENGINEERING SCIENCES (continued):
     24. Basic Research  in Electronics                                           Components                    Pioneering




     25.   Mechanics       and Thermodynamics               Research              Solid     mechanics           Pioneering



     26. Feasibility              Study    of   the Fluid         Transpira-
            tion Arc                                                              Fluid     mechanics           Pioneering
     27.   Stability       and Transition            of Laminar
              Boundary      Layer                                                 Fluid     mechanics           Supporting

     28.   Solid     Propellant           Ignition      Studies                   Combustion        and
                                                                                  ignition                      Supporting

     29. Collision         Cross Sections            for    Momentum              Plasma physics,
            Transfer        in Plasmas                                            conversion      tech-
                                                                                  niques,    fusion de-
                                                                                  vices                         Supporting




     30.   Experimental            Aerodynamics                                   Fluid   mechanics;
                                                                                  thermodynamics                Supporting
     31.   Theoretical    and Experimental    Studies in
             Magnetohydrodynamics      and Plasma Physics                         Plasma       physics          CoMecting


DIRECTORATE OF MATHEMATICAL SCIENCES:
     32. Quadratically Nonlinear Differential                             Sys-
           tems                                                                   Mathematics                   Pioneering

     33.   Conformal       and Variational            Methods                     Mathematics            and
                                                                                  statistics                   Pioneering



     34. Basic Research    in Ordinary   and Partial     Dif-                    Mathematics             and
            ferential  Equations    and Special   Functions                      statistics                    Pioneering


     35. Algebraic         Theory         of Machines                             Applied       mathemat-
                                                                                  ics                          Pioneering
                                                                                                               APPENDIX I
                                                                                                                   Page 3



               Status of
    Funds     project   in                      .Statement      of relevancy    of project
authorized     November                                      to Air Force mission
for project        1969                                      as presented    by OSR

$1,625,000     Active        This research      is relevant      to a significant         number of published
                             technical   objectives.         Special     emphasis    is placed   on technical
                             domains of bionics,        lasers,      molecular    electronics,    electromag-
                             netic propagation,        and plasmas.         This work has special       rele-
                             vance to data storage         systems.
   151,244     Active        The results     of this research   will   lead           to new, simpler      methods
                             of analyzing      the mechanical  and thermal              stresses   of flight    ve-
                             hicles  and will      suggest ways of building              more reliable      and ef-
                             ficient   structures.
                             The feasibility       of applying      the results        of this work to more
   646,635     Active        realistic      and practical     solar    simulators        is being studied.
                             This effort      is relevant   to the phenomenon of transition      from
 1,392,833     Active        smooth air     flow to turbulent     air flow along a flight   vehicle's
                             surface.
                             This study will increase    understanding               of ignition     processes
    351,209    Completed     and lead to improved   design or scaling                criteria    for solid     pro-
                             pellant  igniters and rockets.
                             Plasma constituent            interaction      forces represent        one of the ba-
                             sic parameters         required       to predict     the performance        and charac-
                             teristics       of plasma devices,          such as propulsion          accelerators,
    390,437    Completed     electrical       power generators,          and controlled       fusion     energy
                             sources.        Present    data in this area is uncertain                and conflict-
                             ing.       This program seeks to improve                the accuracy     of this area.
                             Results       of this work were published              in the AIAA Journal          and
                             Physics of Fluids.
                             This research  is relevant            to the design of future              high   super-
    163,880    Active        sonic and hypersonic    Air          Force flight  vehicles.
                             The objective     of this project     is to carry on fundamental,                     the-
     66,667    Completed     oretical,     and experimental    work in magnetohydrodynsmics                     and
                             plasma physics.

                             Emphasis is on developing             new and better    methods for finding
     76,347    Completed     solutions  to differential            equations  generated    by advanced Air
                             Force systems development.
                             This research      is fundamental     to the better       understanding        of
    631,630    Completed     differential      equations,     both ordinary    and partial,        particu-
                             larly     those types of equations       that arise     in the study of
                             aerodynsmics,      electrodynamics,      control   theory,     and communica-
                             tion theory.
                             This research    will  contribute   significantly       to our fundamental
   411,900     Active        knowledge    in those areas of technology         where the Air Force has
                             a demanding need, especially      in commni cation         theory, diffrac-
                             tion theory,    and radar.
                             The objective       is the decomposition         of     these systems         to their
     79,384    Completed     irreducible      structure  to provide         better      understanding        and more
                             efficient     design and utilization.




                                                                     53
                                             LISTING      OF 46 PROJECTS REVIEWED (continued)


                                                                                     Scientific  area
                                                                                         related to             General   type
                          Title      of project                                     Air Force mission            of project

DIRECTORATE OF MATHEMATICAL SCIENCES (continued):
     36. Research In Algebraic Theory of Machines                                    Mathematics        and
                                                                                     statistics                  Pioneering




DIRECTORATE OF PHYSICAL SCIENCES:
     37. Low-Level Earth Motion                                                      Seismology                  Pioneering,
                                                                                                                 supporting


     38. Theoretical  Investigations   in Quantum
           Field Theory and Elementary    Particle
           Physics                                                                   Particle       physics      Pioneering
     39.   Theoretical            Research    in Astrophysics                        Electromagnetics
                                                                                     and electron
                                                                                     physics                     Pioneering



     40.   Radioactivity  of Some Unusual                      Celestial                                         Pioneering,
             Energy Sources                                                          Particle       physics      supporting

     41.   Relaxation        Mechanisms         and Gas Laser              Media     Wave propagation            Pioneering



     42.   Experimental            Low Energy        Nuclear      Physics            Particle       physics      Pioneering,
                                                                                                                 connecting
     43.   Electron       and Nuclear         Spin     Interactions            in    Solid      state   phys-
              Solids                                                                 ics                         Pioneering


     44.   Operation        of Accelerator-Storage                    Ring    to     Solid      state   phys-    Pioneering,
             Provide        Ultraviolet     Radiation                                its                         supporting
     45.   Electronic        Properties         of Solids                            Solid      state   phys-
                                                                                     iCS                         Pioneering




     46.   Cosmic Radiation             of Extremely           High     Energy       Particle       physics      Pioneering


                 Total
                                                                                                                   APPENDIXI
                                                                                                                      Page 4


                  Status of
    Funds        project   in                         Statement      of relevancy    of project
  authorized      November                                        to Air Force mission
for project           1969                                        ai presented    by OSR

                                This theory      is applicable         to finite     state systems.        Itwill
$       63,609    Completed     provide   a representation          of discrete        input   sequential     ma-
                                chines,   e.g.,     digital      computers,      which will      permit   more effi-
                                cient utilization           of present     machines      and guidelines      for fu-
                                ture machines.         It has applications           to systems theory,           biol-
                                ogy, molecular       chemistry,        programming,       and many areas of in-
                                terest  to modern science and aerospace                   technology.

                                This research   is directed     to a study of the manner in which
    2,449,942     Completed     the earth responds     to low-level    energy sources.      The objective
                                is to measure and understand        the earth's   responses    to these
                                sources.

                                This work is essential               to an understanding           of the fundamental
                                properties     of matter.            Such an understanding            is needed before
       96,000     Active        further    breakthroughs           in weapons technology            can be achieved.
                                This research     involves a study of magnetohydrodynamics.            with
                                emphasis on the role of mole&lar           and ato&c     hy&ogen     in as-
      294,300     Completed     tronomical    space and on bodies     in the solar      systems,    and the
                                motion of interstellar     and interplanetary        grains.     The results
                                will   assist  in revising  current     theories   concerning     these ob-
                                jects.
                                These       flights  provide     useful    information      on the background
      602,795     Active        fluxes        of gamma rays in the        environment      where many future          Air
                                Force       systems will     operate.
      520,013     Active        The research     will allow gas lasers        to be developed     with im-
                                proved efficiencies       and power and operating       at wavelengths
                                from the infrared     to the ultraviolet         to be used in communica-
                                tions,  radars ,,and illumination        applications.
                                The work provides  th fundamental                 basis      for    future   Air   Force
    4,241,203     Active        special weapons and energetics.
                                The goal is improved  understanding      of electronic    properties                        of
    1,370,409     Active        known materials  and prediction     of new materials     with proper-
                                ties of use in future   Air Force    detection     and communication
                                devices.
                                This       understanding   will  facilitate         the design of radiation
       326,814    Active        sensors        over a large portion       of    the electromagnetic    spectrum.
                                The electronic      energy states       in solids      and their      response       to
    1.042.639
      .    _      Active        magnetic    and strain     fields    are studied      using     intense      synchro-
                                tron radiation      to aid in the development             of-new solid         state
                                materials.      Charge transport        by means of tunneling             through
                                barriers    and between impurities           is studied      at cryogenic         tem-
                                peratures     to explain     contact    phenomena in solid           state devices
                                and to increase      understanding        of transport       processes       in met-
                                als, semiconductors,         and insulators.
      458.300     Active        These investigations      are already yielding    much new knowledge
                                on the structure     of our universe,   the environment    in space,
                                and the properties     of matter at extremely     high energies.
$21,503,192
APPEND.IX        I I


                           DIRECTOR      OF DEFENSE       RESEARCH           AND ENGINEERING
                                               WASH1NGTOM.        D   C   20301




Honorable       Charles     M. Bailey
Director,      Defense      Division
United    States    General      Accounting              Office                                     15 JUN 1970
Washington,        D. C. 20548

Dear    Mr.      Bailey:

This letter       and its attachments           provide   comments,        recommendations,
and actions        taken    concerning       the GAO Draft      Report     entitled   ‘“Need                      to
Strengthen        Management         Control     Over the Basic       Research      Program
Administered           by the Air Force         Office  of Scientific    Research.       If

The following        comments             are made relative        to the GAO recommendation
that DOD issue        definitive           criteria  which    will ensure   that DOD funds                         only
research     projects       which         are relevant     to the DOD mission:

     (1) Relevance         has always           been an established                 criterion       of the
    legal     principles       and long-standing               RDT&E          policies        which     have
    governed          and will continue            to govern      the use of Defense                  appro-
    priations         for RDT&E          activities.         Secretary           Packard’s          memorandum
    to all DOD components                   concerning         Section        203 of PL 91-121 pointed
    out that those          provisions          were,      in effect,        reiterative          of the DOD’S
    policy      concerning         relevancy,          but, to assure             full compliance,
    directed        that reviews,           documentation,             and supplemental                directives
    be accomplished             to ensure          full compliance             with Section          203.       This
    action      has been completed                for each 6.1 Research                    and 6.2 Exploratory
    Development           project       at the work          unit level,           About      15,000      tasks
    were      reviewed,        of which         about 450 (3%) were                 found      to be marginally
    relevant       under     a stringent          application        of the wording              in Section        203
    which      requires       a ‘“direct        and apparent          relationship            to a military
    function       or operation.         I*

    (2) During    the past few years,           policies     concerning        specific    areas
    of research     have been modified.              It should    be noted that prior          to
    the enactment      of Section       203 of PL 91-121 actions             were taken to
    de-emphasize,        withdraw,         and in some cases transfer              to other
    government      agencies       projects    or particular        scientific      endeavors.




                                                             56
                                                                                                    APPENDIX II
                                                                                                         Page 2

    For example,          we have de-emphasized           the social     sciences.        We
    have withdrawn          from    certain areas    of nuclear      physics,      general
    mathematics,          and some areas       of chemistry.         We would      hope to
    transfer     applicable      programs    to AEC     and NSF.

    (3) The DOD Directive     3210.1,   subject:     “Administration                          and Support
    of Basic Research     by the DOD” was changed            (3 April                    1970) to imple-
    ment a new definition    of Defense     Research      (Attachment                        #l).

    (4) The DOD recognizes            the scientific       contributions         of other    agencies
    like NASA,      AEC,     HEW,      and NSF .via special             committees,       publication
    exchange,     proposal     review     lists,     personal       contact,      etc.     The
    contributions      of the industrial         base of the United          States    are also
    recognized.

    (5) There     are many          technologies         which    are “absolutely         essential”
    to insure     a strong       national       defense.       Therefore,        it is essential
    that we engage        in research           to advance       and support        those technologies.
    Although     research         results      are sometimes           negative,      the lack of a
    desired   finding      after      thorough       study may often be very             important
    to a new weapons           system        concept.

The Department        of the          Air Force     review    of the Report    is attached.      It
specifically   addresses              each of the findings       and recommendations,         and
describes    management               actions   taken    as corrective    measures      where
appropriate.

It should       be noted that Section            203 of Public          Law 91-121,        requiring        each
work      unit to have a direct          and apparent           relationship         to a specific        military
function,       was passed         more    than a year        after      the beginning         of the GAO
review,        and several        years   after     the initiation         of some of the 46 contracts
examined         during     the review.         In fact,     only 24 of those           46 contracts          were
still   active     at the time       of the 203 review,             and 18 or 75% of that group
were      found to meet         the requirements           of Section         203.     In light      of the fact
that 93% of 1,780            OAR work        units passed          the stringent        Section       203 review,
the findings         of the GAO investigation             are not truly           representative           of the
present       AFOSR       program.

AFOSRls      program      will    continue    to evolve    as the emphasis     on Air Force
requirements        changes.       Their    management        procedures   have been modified
in response      to your     recommendations,           and in accordance    with the laws
and policies      under   which     they must     operate.

                                                                         S


Attachments
                                                                  4 WJ
     DOD Directive           3210.1
     AF Report




                                                          57
APPENDIX II
     page 3




           DEPARTMENT       OF    THE    AIR   FORCE    RESPONSE

                                  TO

     GAO   DRAFT   REPORT    TO    THE       CONGRESS   OF   THE   U.   S.




     “NEED  TO STRENGTHEN      MANAGEMENT        CONTROL            OVER
     THE BASIC  RESEXRCH     PROGRAM    ADMINISTERED               BY THE
     AIR FORCE  OFFICE    OF SCIENTIFIC   RESEARCH”




                                        58
                                                                                 d’PEND.IX I I
                                                                                      Page 4


                           CONTENTS




SECTION   I     OVERVIEW


SECTION   II    RESPONSE            TO RECOMMENDATIONS


SECTION   III   COMMENTS             ON FINDINGS            AND       CONCLUSIONS


                A.   1;ntroduction

                B.   Refinitive       Criteria        for   Selecting       Research          Projects

                C.   Surveillance         over    Research            Projects

                D.   Obtaining        Research         Results       and   Obtaining          Them
                     Promptly

                E.   Effective        Dissemination              of Research        Results




                                          59
APPENDIX II
    Page 5



 The QLO upon their                     rcvicw          cf various         AFOSR corltrncts,          ::tai.cs    four    basic

 prc-ml Cf!S:

    --      -ikit      the ~:-anil~~cmcnt of contract                      rcse0ch       i nvolvcs     unique      problems

            which           make it     a difficult              area to manage

    --      that       AFOSR intorpretcd                  relevance         to the Air       Force mission          very broadly

            in selectin. ., ,rojects
                            3.1                          which      it    funded

    --      that       it     lacked         the procedures              to carry     out certain       functions         effectively

    -- that            cstablis:led            procedures           were not consistently              followed


 AFOSR’ s pro&ram and management have been in constant                                               evolution      responding

 to the chenSinS climate                          for     research         supported      by the DOD, decreasing

 budgets,            and increasingly                   stringent         requirements       for     documentinS         mission

 application.


 14anagement actions                    have been taken which meet or surpass                             many of the

 recom..endations                 of the GAO report.                     OAR has implemented            changes in their

 command regulation                     reflectinS           the criteria            of Section       203.       AFOSR has

 revised            their      Headquarters              Operatin S Instructions              and policies          to conform

 to the GAO report.                          These are detailed              in Section       II     llResponse to

 Recommendations."                      It     must be noted that               the requirement           to document

 this    relevance              to show a direct                 and apparent          relationship          to a specific

 military            function          or operation           was initiated            under Section          203 of Public        Law

 91-121.             Many of the research                    efforts        examined by the GAO were initiated                     one

 or more years                 prior         to their      review        and this      could not be expected               to reflect

 current        uolicies           and procedure-s.


                                                 [See GAOnote,                 p,     74.1




                                                                            60
                                                                                          APPENDIX1.1
                                                                                              Page 6
Section I - O'mN~~~ cant ' d




!Phc contracts     that wme extensively             reviewed were preselected           by a GAO

review team and were not randomly chosen.                    Eleven of the 46 cdntracts              reviewed
were in the Social and Behavioral                 Sciences, an area that had already been
cut drastically       by the Air Force.            In addition,       four more were in the Ruclear
Physics area, which has been completely doletcd fran the air                            Force research
pmgram.       The non-random selection             and the constraints        of the contract             discipline
emphasis do not permit credible                 extrapolation       to the total     AFGSRprogram.
These and related matters are ascussed                    in detail     in Section IfI       "COr;Jncntson
Findings and Conclusions.ti

It is also informative          to set thE GAO6mple IL -cxmtext w%th respect to the
requirements of Section 203.              Twenty-four        of the 46 GAOselected contracts
are still      active and 18 of these were found to meet the requirements                            of
Section 203.        The remaining 22 efforts.were               not subjected to 203 criteria
because they were complete.            All 46 contracts             were submitted.to       bDR.%E:
                                                                                                 for
review and p@y six were deemed questionable.                         Of the six,      four have been
terminated;       one will   terminate      on June 30, 1970, and the last                one will        teminate
on September 3S, 1970.           By contrast,         about 93 percent of all           GARwork 1~?its
 (about ~800) passed the stringent                 requirements of Section 203.

We have found that GAOsecomendations                     were most helpful. in revictring             our
procedurds and in developing             better      operating methods.            The net effect          o: tiis

 rcvicw will      bc to ;trcnLgtht:a     ILir     E’crcc research      n;enaptnont.      To a larfr - e>::~:i:t,.
 actions to effect                              1 have alrcslly
                          thr?sqz irqrzw-~*,..L~~.t~                     been nccorzplizhed.


                                                        61
APPENDIX II
    Page 7

  Section'11       -- RIZWNSE TO 1WOKKiQ;DATIOI'~S


        1.     GAO Recom-nendation

               "GAO is recommending that DOD issue definitive                criteria
               to the dcfcnse agencies and military             services which will
               adequately limit these agencies to funding projects                   which
               are clearly     relevant to -- as well as essential            to the
               accomplishment of -- their assigned missions.                 This criteria
               should also require that in evaluating               need and essentiality
               consideration      be given to the mission of and research per-
               formed by other organizations.             GAO is also recommending
               that definitive      instructions     be issued by DOD which require
               that the need for each contract and grant be fully disclosed
               and clearly     established     in writing   and that these justifica-
               tions be made a part of the official             files relating       to the
               contract or grant."


               Action

               On December         18, 1969,AFOSR issued                    instructions        on

               evaluation         and doclmmentation              of that     evaluation        in the

               cfficial      files.        These instructions                (Atch 1) specifically

               include      relevance         criteria       and consideration              of the

               research      supported         by other          agencies.       OAR Regulation

               500-37,      dated February           9, 1970 (Atch 2), provides                      OAR

               policy     on the criteria            to be used in the selection                      and

               evaluation         of all      research      efforts         conducted       or supported

               by OAR. OAR Regulation                    80-1,     revised     February        2, 1970
               (Atch 3) sets          forth     OAR procedure           for evaluating,

               processing,         documenting           and reporting          the status       of
               proposals         for research.            Para l&b(l) of that              regulation

               states     that     OAR Form 43A "Proposal                   Control     and Evalua-

               tion,"     the official          OAR form for documenting                    the technical




                                                                 62
                                                                                                                                APPENDIX II
                                                                                                                                    Page 8

          evaluation              of an unsolicited                            proposal              and     Paragraph                5a

          requires            that         the      OAR             Form           43A       become          a part          of the

          procurement                  case              file.

2.    GAO         Recommendation

     “That        contractors                    and grantees                      be required              to submit
     periodic         progress                   reports.   ‘I

     Action

     The previous                 informal                  methods                of obtaining              periodic           progress

     reports          have        been           formalized                   as a contractual                     (grant)          require-

     ment       for    an annual                  report            to include               progress,               publications,

     bibliography,                 and where                      appropriate,                   contemplated                 major

     deviations          in program                         and budget                (Atch         4).

3.    GAO         Recommendation

     “Guidelines              be issued                    to project       scientists      (a) establishing
     a timetable    for visiting                              projects,       (b) providing      for the data
     to be obtained    on each                              visit,      and (c) requiring       adequate
     documentation                   of each                visit.    I’

     Action

     1.    OAR        Regulation                   500-38                “Visits          to Contractors’*                   was      issued

          on February                  9,        1970            (Atch       5).          This      regulation            establishes

          a minimum                  of one              ‘site      visit          each      year         by the       scientific

          monitor            or      his     designated                      representative.                       It provides             for

          the information                         to be obtained                      on each visit                and       established

          the requirement                          for       trip         reports           being         prepared            and     placed




                                                                            63
APPENDIX II
    Page 9

                     in the        project           file.

              2.      AFOSR               interim            instruction                 “Documentation                   of

                     Significant                 Events           or     Changes            to a Work            Unit”           (Atch         6)

                      requires             the      documentation                       in the    technical              project           file

                     of trip        reports               and      reports              of significance             obtained               from

                     other         sources.

         4.    GAO           Recommendation

              “That    greater     emphasis      be placed     on procedures                                                 for obtaining
              promptly       all reports    produced     under    research                                               projects     sup-
              ported    by AFOSR.        ”

              Action

              Attachment                   4,     AFOSR                Interim          Instruction           on Progress                      Reports,

              requires             the          submission                of annual           progress            reports               with        a

              bibliography                      of publications                   produced            under      the       research

              effort.            The        requirement                    in AFOSR              HOI      80-3      to         review          the

              Final          Scientific             Report              bibliography              has     been         augmented

              (Atch          7) to include                   a review             of the      bibliography                in the           annual

              progress              reports            to insure                 that     appropriate            publications

              have       been          received.              *



  *These      two actions,         along with monitoring                                        for compliance          by the AFOSR
  Assistant        Executive       Director,        Research                                Operations,       will     emphasize
  through      line management              the requirement                                    to obtain   promptly         all
  appropriate         publications        and will     provide                               the Executive         Director      with
  managerial         information         concerning        any                             deviations.




                                                                                   64
                                                                                                                                         APPENDIXII
                                                                                                                                            Page 10
          5.    GAO         Recommendation
                                -.-__

               “That      project       scic;ntists      be required                          to make    the results                        of
               research         projects        promptly     available                          to those   Air Force
               organizations           having       the most interest                             in such results.”

               Action

               AFOSR             has      been        informally             complying               continually              with      this
t   Y . r .,
               recommendation                         through         many        well-established                     informal

               mechanisms.                       OAR       Supplement                1 to AFR           SO-44         dated          December

               12,      1969      (Atch          8) formally             requires           primary            distribution                  lists.

               This        has     been          further         implemented                by AFOSR                letter           “Timely

               Dissemination                     of    Research              Results”        ‘with      two        attachments                   .{Atch

               9).       The      Air       Force          Form          111 Coupling            Program               provides                  a

               formal            mechanism                 of insuring            that      appropriate               level

               individuals                within        AFSC         are      aware         of new        discoveries                  or

               research                results        having         possible            application            to on-going

               technology                programs.                 The       AFOSR          Technology                Alert           is to

               advise          key       technology              users        in any        organization                on a timely

               basis        of development                      potentials           created           by AFOSR-supported

               research                investigations               in advance              of their          scientific              pub-

               lication.




                                                                               6.5
APPENDIXJI
   Page 11

     Section     III - COMMENTS                 ON FINDINGS              AND CONCLUSIONS



          A.     Introduction

                 The preceding               section      listed      management           actions      taken

          during       the two-year           period      during       which     the GAO conducted

          its review          of AFOSR,          as well       as actions        taken      subsequent             to

          that     review      .   It should      be noted that many                of these        actions

          were       taken     in response         to the changing             climate      for research

          supported          by the DOD.           This      changing         climate      is reflected

          in decreasing            budgets       and increasingly              stringent        requirements

          for documenting                 relevance.

                 Relevance            has always          been a foremost               consideration              for

          research           proposal        selection.        It must        be noted however                that

          the    requirement              to document         this     relevance         and further          to

          show       its direct         and apparent          relationship          to a specific

          military         function       or operation         was initiated            under    Section           203

          of PL 91-121.               Many     of the research            efforts       examined        by the

          GAO wore            initiated      one or more              years    prior     to the period              of

          the review          . Consequently,              the files      on these        efforts      could

          not    reflect       current       policies      and procedures.                Of course,

          none of these            efforts     were       initiated      under      the criteria        of

          Section       203.




                                                             66
                                                                                                    APPENDIX.I,1
                                                                                                       Page 12
B.     Definitive         Criteria       for Selecting           Research       Projects

        While       it is true       that past AFOSR             directives         did not

require      detailed         documentation            on the work          unit basis           of the

relevance           of selected        research        efforts     (work      units),          relevance

to projects          has always          been a consideration                in the selection.

In the past,          this    documentation            has been used primarily                      by

AFOSR        scientific         managers        who were          themselves            competent

to translate          the individual           work     unit to the larger              project

grouping,           to the planning         guidance        and to Air        Force        needs.

The scrutiny            now being         given individual           WJ rk efforts             by

external         agencies,        and the       requirements           of Section          203 which

introduced           the requirement            to demonstrate             direct       and apparent

relationship          to a function         or operation           which      can be understood

by lra reasonable               and prudent         man, ‘I necessitates                that      our

documentation                be more      complete.          Accordingly,           directives             to

this    effect      have been issued            and have been implemented                           (See

Section      II).

        On page        20, under         “National         Science     Foundation,              It two

efforts      are cited         as supported           by AFOSR        after     being          declined

by NSF on the basis                  of lack    of funds.         The implication                 is that

if NSF declines               to fund a scientifically             acceptable           effort,

which      is then      picked       up by AFOSR,            that the effort            must        perforce

be inherently           unacceptable           from     an Air       Force      relevancy            point

of view.         The Air         Force     cannot      agree      that a test       for relevancy

is whether           NSF would           or could have supported                the same

research,.




                                                      67
APPENDIXII
   Page 13
             With     respect        to the        46 cases              studied     by GAO and the

     Section        203 review           directed          by the Deputy              Secretary          of Defense,

     of the      24 still       active     and reviewed,                    18 were        found    to meet            the

     requirements             of Section           203 even though                  they were           selected         for

     support        under       a less         stringent            definition      of relevancy           than

     Section        203.      The remaining                    22 efforts          were      not reviewed

     because        they were            completed.                  It is informative             to set the GAO

     sample         in context.           About         93 percent               of all OAR work               units

     (about      1,800)      passed        the stringent                 203 review.

             The     process          of selection              of research           proposals          by AFOSR

     has been modified                 by the new requirements                             of Section          203.

     None of the 46 research                        efforts          selected       by the GAO            for review

     were      initially      funded           under     the 203 requirements                      for clearly

     demonstrating              a direct          and apparent                relationship         to Air        Force

     operations            and functions.               The full             repercussions              of 203 upon

     the AFOSR              research           program              can not yet be assessed,                     but any

     discussion            today     of relevance               of research            proposals          can only

     be made         meaningfully               in terms             of 203.       In response           to the far-

     reaching         effects       of 203,         especially              the extensive          documentation

     required        to implement                the law,            selection       criteria       have been

     modified,         subsequent              to the GAO review,                     by two OAR regulations

     and one AFOSR                 Directive           (see         pp- II-1      to II-3).        [Soa pp., 63 and 64.1

             A portion           of AFOSR              Operating             Instruction         80-4     of

     February         10, 1965 is quoted                      (p.     12) with the permissive                     clause

     regarding         the requirement                  for documenting                    Air   Force         and DOD

     relevance.             This      instruction              was superceded                 on December                18,

     1969,     and the permissive                      clause          removed.            While    relevance                to




                                                                       68
                                                                                                      APPENDIXJJ
                                                                                                                 Page 14

Air      Force     mission        has always           been a required                criterion,           its

documentation              was not required                to be reflected            in tlz       contract

file.      The GAO Report                 notes that         relevance         statements            existed

for the work          units      in summaries               prepared         for annual            review

purposes.

         Pages      13 through       17 list         five projects           cited     as showing

remoteness           from       areas      of direct        Air      Force      concern,           quoting

from      the project          abstracts.            It should       be noted        that at the time

the efforts         were       selected      and initially           funded,         they were         clearly

related      to the relevance               documented            at the Project             level     as

well     as in the OAR            Plan.      These         in turn      were     worked         out in

conjunction         with      the Air       Force      using      agencies           and the OAR

Defense          Research        Sciences          Subelement          Panels.          It is to be

noted that the column                “Statement             of relevancy             of project        to

Air      Force     Mission        as presented              by AFOSR”           in Appendix            I

of the GAO report                is from       the individual            work        unit documentation

and not from           the Project           or Plan.          It was not written               as the

prime       statement          of relevancy           but was pointed                out to the GAO

investigators           merely       as the link           from      the work         unit

documentation              to the OAR Plan.

         Three      of these      five     efforts      were      allowed        to lapse          as a

result      of management               decisions          ending      the foreign           area

programs,           and cutting           back the         behavioral          sciences        program.

The fourth          effort,      which      will     be completed             in June 1970,            was

a one-time          effort,      not subject          to renewal.              The fifth       effort,

while      judged     to be relevant               under     203 criteria,            failed       to meet

the competition               for the reduced              funding     available.

         Under      “Testimony            of Air      Force       Officials,         I’ p. 19, the
APPENDYLX TI
    Page 15
       report            111c,lltionc          “:;tudy          of Cognitive                   and      1Sffccl       i-,-e Attitudes.               I’

       (cu~~cluct          cd by Dr.                 Hcnri          Tajfel         of Oxford               Univcrsii.y).                   The

       slatemcnts                   by the          IXS         R&D       and       Assistant              Sccrctary                of the

      Air       l?orce          (!?&I$           cited          on pages            19 and           20 refer            to the        decision

       by the        Air        Force            to withdraw                 Irom         foreign           area           studies           and

       from         social            science             research              and      not     to the           relevance              of this

       specific             effort.

                A        discussion                  of the        OAR       Five         Year          Plan       (the      technical

       sections             of which                are    available               to the        public         as OAR              Research

       Objectives)                    begins           on page            20 of the            GAO         report.               GAO         is of

       the      opinion             that       each       effort          should         be accompanied                          by a suitable

       relevancy               statement                  (p.      23).         We       concur.               The        next       edition          of

       the      Plan         will          contain         a new          section            on Air         Force           needs.             The

       Plan       will        document                 these        needs          and       interpret             the      problem

       statements                   into       research               opportunities                  for       meeting           those            needs.

       OAR          has      established                  an interaction                     activity           of seven            task          groups

      to      prepare               the     problem-oriented                          part       of the         Plan        (see       Atch          1,

       this       section).

                 The        GAO            report         cites        (p.      10) the         House           Committee                    on

      Appropriations,                          Report             1735.


                                                          [See GAO note,                             p. 74,1

                                                    The      term          essentiality                 is subject               to wide

       interpretation                      and       should         be avoided.                      We believe              that      all

       research               which           we      support             is both         relevant              and       essential            to

      the       sllcrrssful                 accomulishmcnt                         of the        Air       Force           mission             in
      1GAO footnote:
                                                    House Hearings on the                                      Department                    of
                                                    Defense appropriations                                        for 1970,
                                                    pages 436 and 439.
                                                                                                   APPENDIXII
                                                                                                      Page 16
the     future.       Thus we apply              judgements           as to essentiality

within      the context             of relevance,           and our documentation                   of

relevancy          implicitly          addresses           essentiality.

C.      Surveillance            of Research             Efforts

           On-site         visits     are desirable,               but other      well      established

and successful              methods           of monitoring           research          efforts     were

minimized           by the GAO report.                       The report           contends        that

monitoring           was inadequate               because          four    out of the 36

domestic          projects          had not had on-site               visits      in the year

prior      to the review.               However,           these      four     investigators             had

been contacted              by their          project     monitors           at least      once    during

that     period      through         visits      at AFOSR           or by face-to-face              dis-

cussions          held at scientific              meetings.           In addition,          the GAO

was concerned               that     in four      out of the 46 cases,                  written     pro-

gress       reports        had not been requested.                        The project         monitor

must      prepare          a progress           report      on each work            unit     each year.

It had been left to his determination                               whether       or not he needed

a written         report       from      the principal             investigator          or whether

his personal             contacts       with      the investigator             and the scientific

publications           received         from       the effort        provided       adequate

information           for this        progress           report.          New procedures             now

require         annual       site visits         and annual          contractor/grantee

progress          reports.           (See Section           II)

      Further       discussion          is appropriate              concerning           two of the

four     projects         singled       out by the GAO report.
APPENDIXII
   Page 17
      Cosmic        Radiation         at Extremely            High Energy              (page 31)

           There      was       a deviation        from     the terms           of the grant             of the

      type described             by the GAO investigators,                      namely            for the

      budget       negotiated.          In view      of the lack          of precedent              for an

      effort      involving       innovation        of equipment              on this          scale,       the

      departure        was necessary              to optimize           the expenditure                  of the

      funds.        However,          responsible           program           managers            were

      satisfied       that the over-all             objectives         for which             the DOD

      research        money        was expended             were       met.

      Theoretical           Research           in Astrophysics            (page        32)

           We are unable             to account          for the statement                “. . . . it was

      difficult      to determine             what had been done or whether                              anything

      had been accomplished.                    . . . ‘I The facts            are that forty-two

      reports       and publications              were      received          under      this      effort         and

      are on file        at AFOSR.              Copies      were       sent to DDC.                In addition

      one book (25 copies                received         by AFOSR)            resulted           from       this

      support        entitled      The Nature            of Time       edited      by Thomas                Gold,

      Cornell       University          Press,       1967.       All     reports         and publications

      listed      in the final        report      were      submitted          in accordance                 with

    . the requirements               of the research             agreements.

           With      regard       to the statement            in the GAO report                    ‘I.. . .funds

      had been used to support                    graduate          students.         . . .”     it should          be

      noted       that Graduate          students         participated          in this         effort      as research

      workers.          This       is a general          practice        for most          research               efforts

      at universities.




                                                                 12
                                                                                                       APPENDZ XI
                                                                                                          Page 18
D.       OBTAINING            RESEARCH             RESULTS              AND OBTAINING
                                  THEM           PROMPTLY

     While        the GAO report            does not state              specifically       which

documents            were     not obtained          and placed           in AFOSR         and DDC

files,     an internal         review       of documents            associated           with       the 46

work       efforts      in the GAO report             revealed           that less than             seven

percent        of the documents              which     would       appropriately             have been

included        in the AFOSR             files    and DDC had not been received.

Yet,      these      46 efforts        have produced            more       than 500 publications

which      have been included                in AFOSR           files     and DDC.

     In many         cases,       the missing         reports           concerned         were       pur-

posely       excluded         from     the files      even though           they are included

in the bibliography               of the final        report.           Examples          are theses

and brief         presentations           or papers         for specific          meetings,           the

contents        of which       are contained           in other          publications.              It should

also      be noted       that copies         of individual         papers         included          in pro-

ceedings          are    seldom      available        separately.

E.       Effective       Dissemination             of Research            Results

     Addressing             the specific          case discussed            by the GAO:

Quadratically            Nonlinear         Differential          Systems          (page      43).      As

stated,       the final       report      was a “four           page administrative

summary”,               it was not sent to Air              Force         users     or to the Defense

Documentation               Center       precisely        because         it was an administrative

and not a technical               document.           DDC will            not accept         and file       an

“administrative”               report.




                                                       73
APPENDIX II
    Page 19
         Two       of the    referenced              manuscripts              were     incorporated

  into     a single         document          which         was presented              at the principal

  annual          meeting         of the American                Mathematical              Society      in New

  Orleans           in January,            1969.       It has been             published        in the Journal

  of Differential             Equations,              v.    7, n. 2, pp.             251-273,        March    1970.

  As soon as reprints                      of this     article         are     received         by AFOSR,

  they     will      be submitted            to DDC.             The     results        in the other

  manuscripts               are    still     being         extended          and “polished”          before

  formal          verbal      or written           presentation.


  GAO note:                 Deleted comments pertain to material
                            presented in the draft report which has
                            been revised or which has not been in-
                            cluded in the final report.
                            The DOD Directive 3210.1 and the Air
                            Force directives   attached to the DOD
                            letter have not been included in this
                            appendix because of their length and
                            because they are not essential to ob-
                            taining an understanding of the report
                            and corrective   actions being taken.




                                                                  74
                                                    APPENDIX III
                                                         Page 1

                   PRINCIPAL OFFICIALS

            OF THE DEPARTMENTOF DEFENSEAND

             THE DEPARTMENTOF THE AIR FORCE

      RESPONSIBLE FOR ADMINISTRATION OF ACTIVITIES

                DISCUSSED IN THIS REPORT


                                          Tenure of office
                                          From             -To
                  DEPARTMENTOF DEFENSE

SECRETARYOF DEFENSE:
   Melvin R. -Laird                Jan.      1969     Present
   Clark M. Clifford               Mar.      1968     Jan.    1969
   Robert S. McNamara              Jan.      1961     Mar.    1968

DEPUTY SECRETARYOF DEFENSE:
   David M. Packard                Jan.      1969     Present
   Paul H. Nitze                   July      1967     Jan.    1969
   Cyrus R. Vance                  Jan.      1964     June 1967

DIRECTOR OF DEFENSERESEARCHAND
  ENGINEERING:
    Dr. John S. Foster, Jr.        Oct.      1965     Present
    Dr. Harold Brown               bY        1961     Sept. 1965


               DEPARTMENTOF THE AIR FORCE

SECRETARYOF THE AIR FORCE:
   Robert C. Seamans, Jr.          Jan.      1969     Present
   Harold Brown                    Ott a     1965     Jan.    1969
   Eugene M. Zuckert               Jan.      1961     Sept. 1965

ASSISTANT SECRETARYOF THE AIR
  FORCE (RESEARCHAND DEVE~P-
  MENT):
    Grant L. Hansen                Mar.      1969     Present
    Dr. Alexander H. Flax          July      1963     Mar. 1969
APPENDIX III
     Page 2

                     PRINCIPAL OFFICIALS

               OF THE DEPARTMENTOF DEFENSEAND

                THE DEPARTMENTOF THE AIR FORCE

       RESPONSIBLE FOR ADMINISTRATION OF ACTIVITIES

                  DISCUSSED IN THIS REPORT (continued)


                                           Tenure of office
                                           From             To


                 DEPARTMENTOF THE AIR FORCE (continued)

COMMANDER,OFFICE OF AEROSPACE
  RESEARCH:
    Brig. Gen. H. W. Eddy             Aug.    1969    June           1970
    Brig. Gen. Leo Kiley              Jan.    1968    July           1969
    Brig. Gen. Ernest Pinson          Oct.    1965    Ott *          1967
    Maj. Gen. Don Ostrander           Sept.   1962    Oct.           1965
    Maj. Gen. Daniel E. Hooks         Apr.    1961    June           1962

EXECUTIVE DIRECTOR, OFFICE OF
  SCIENTIFIC RESEARCH:
    Dr, William J. Price              Sept.   1963    Present




                                                      U.S.   GAO   Wash.,   D.c.

                             76