oversight

Opportunities To Improve Administration of the Research Program at the National Radio Astronomy Observatory

Published by the Government Accountability Office on 1971-06-15.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

         0
     ;
     CK
         m



 0
l-    a
     xz
z
.-
111
     l-


-z
!
1


                                             UNITED STATES GENERAL ACGOUNTINGQFFICE
     :                                                         Wa’AS#INGTON,       D.C.   20548


         CIVIL   DIVISION



                        B-133338

                        Dear     Dr.   McElroy:

                                This is our report               on opportunities      to improve       administration
iL
.                      of the research     program                at the National    Radio     Astronomy         Observatory
 Y
                       by the National     Science              Foundation      and Associated       Universities,        Incor-
                       porated.


1                                 In this report         we have recognized    your comments    on a draft                        of
I                      this     report   and the        actions taken or contemplated    by the Foundation.

                                 Your     attention     is invited      to section     236 of the Legislative        Reorgani-
1                       zation    Act of 1970 which          requires       that you submit      written    statements       of
                       the action       taken with respect           to the recommendations.             The statements
                       are to be sent to the House                 and Senate       Committees     on Government          Op-
                       erations      not later      than 60 days after           the date of this report       and to the
                       House      and Senate        Committees         on Appropriations        in connection      with the
“i                     first    request      for appropriations           submitted      by your agency       more    than
                       60 days after          the date of this report.

                                 We shall         appreciate       being       advised    of the    actions    taken     by the    Foun-
                       dation     on these        matters.

                                 Copies    of this report      are being    sent to the Director,                      Office    of Man-
                       agement       and Budget;      the Director,     National     Radio     Astronomy                    Observa-
                       tory;    the President       of Associated      Universities,       Incorporated;                    and to
                       appropriate       committees       and subcommittees          of the Congress.

                                                                           Sincerely       yours,




                                                                           Director,       Civil    Division

                       The Honorable        William      D. McElroy
                       Director,   National       Science   Foundation




                                                           50 TH ANNIVERS.4RY             1921 - 1971
    GENERALACCOUNTINGOFFICE                         OPPORTUNITIESTO IMPROVE
    REPORTTO THE DIRECTOR,                          ADMINISTRATION OF THE RESEARCH
    NATIONAL SCIENCE FOUNDATION                     PROGRAMAT THE NATIONAL RADIO
                                                    ASTRONOMYOBSERVATORY
                                                    National Science Foundation
                                                    B-133338                    ,
                                                                             d

    DIGEST
    __----

    WHYTHE REVIEW WASMADE

            The National Science Foundation (NSF) is the principal      Federal agency
            responsible for supporting ground-based radio astronomy.        The National
            Radio Astronomy Observatory, with its main observing facilities---
            Green Bank;-+Jest Virginia,' was established    as a center where astronomers
            of the Nation could perform specialized      research and obtain the use
            of large expensive radio telescopes not otherwise available.

            The Observatory is operated for NSF by Associated Universities,    Incor-
            porated, under a cost-reimbursable  contract.   Federal funds provided
            for the construction  and operation of the Observatory totaled about
            $62.1 million  through December 31, 1970.

             Because of the expenditure of substantial    funds under the contract,   the
             General Accounting Office (GAO) has examined into the policies,      proce-
             duress and practices for administering    the research program at the Ob-
             servatory.


    FIUDINGS AND CONCLUSIONS

            AZZocation   of telescope   observing   time
I

             The Observatory's     policy is to allocate observing time for research
             studies on the basis of the scientific       merits of the proposed research
             and of the availability      of the telescopes.    Both staff and visiting
             scientists  desiring to use the telescopes are required to submit writ-
             ten proposals describing the research, the time required, and the equip3-
             ment needed. The Observatory approves the proposals on the basis of
             evaluations  by independent referees and Observatory scientists.
             (See p. 17.)

             The Observatory has, over the years, made improvements in its evalua-
             tion procedures.   Further improvements could be made in the procedures
             for review by independent referees by

    Tear
    -      Sheet
         .___




i
                                                                                 f
   --establishing   a pool of referees to obtain a wider range of views
      and more expert opinions in highly specialized  areas of research          ?'
      and to expedite the review process (see p. 18);

  --soliciting   referees'  evaluations of updated requests, which may
     involve significant   amounts of additional  observing time or other
     changes in the scope of ongoing research to ensure that the addi-
     tional time requested is commensurate with the scientific    merits
     of the research (see p. 19); and
  --obtaining   referees'    pertinent narrative comments on the scientific
     merits of proposed research projects to make the ratings more use-
     ful in allocating    telescope observing time (see p. 20).

Also, improvements could be made in the approval and scheduling of re-
search studies by maintaining     formal and complete records, including
original    rating data by referees.     Such records, in GAO's opinion, are
necessary to document the approval and evaluation of research studies
involving    the use of costly federally    owned facilities and would be of
assistance in the order1 management and in surveillance        of observatory
activities.       (See p. 23. 1

Evaihation     of Zeveh    of research   efforts

The contractor  has established    two review committees to assist it in de-     i
termining the appropriateness     of the Observatory's   concentration of re-
search efforts  and the quality of the research program. The Observatory
maintains basic records of the monthly use of each telescope and pre-
pares narrative  reports on research activities      but does not summarize
the observing time devoted to individual     projects or significant   re-       i
search areas.   Such data should assist management and the review com-            0
mittees in their evaluation     of the adequacy of levels of research ef-        I
                                                                                 #
forts.   The Observatory has begun to maintain summaries by individual           I 1
users for one of its telescopes.

Although this additional     information    should be helpful in the review
of research activities,    it should be supplemented by appropriate sum-
maries of observing time used on all telescope systems to enable an
effective  overall evaluation of the direction       of research efforts.
Such supplemented information      could be helpful to the Director of the
Observatory in exercising     his responsibility    for maintaining  research
programs of the highest quality       and for being alert to needed changes
in research emphasis.     (See p. 28.)

Use of   telescopes   ~LJ visitors   ano? staff

The Observatory's  policy provides that visitors   be allocated 60     percent
or more of the observing time on the telescope systems and that        the re-
mainder of time be for use by the resident staff.     GAO's review     of the
manner in which this policy has been carried out raises certain        ques-
tions regarding the allocation   of observing time between resident      staff
and visitors  because:
                    --The Observatory classifies    its temporary employees as visitors
                       and on this basis considers that visitors    have used about 54 per-
                       cent of observing time during the Observatory's    11 years  of opera-
                       tions through fiscal year 1969. However, if temporary employees
                       are classified  as staff,  which GAO believes to be a more appropriate
                       classification,  visitors'  use has averaged only 34 percent of total
                       observing time during this period.    (See p. 30-I

                    --The average telescope time used by each visitor         during the ll-year
                       period has declined significantly     compared with    the average time
                       used by each staff observer.      (See pm 30.)

                 In GAO's opinion the present policies  and practices followed in al-
                 locating telescope time may not be adequate to ensure that the Observa-
                 tory fully serves its mission as a national center primarily  for the
                 benefit of visiting  scientists.


         RECOiWMENDATIONS
                     OR SUGGESTIONS
                 In accordance with the above findings,      GAO is making recommendations
                 designed to improve the system for allocating      telescope observing
                 time and for evaluating  levels of research efforts.        (See pp. 23 and
                 29.) GAO is further recommending that NSF, in cooperation with the
                 contractor,  undertake a study of visitors'     use of the Observatory's
                 telescopes to determine what action 9 if any,is needed to ensure that
                 the Observatory fully serves its mission.       (See p. 41.)
 I
I_

i        AGENCY
              ACTIONSANDUNRESOLVED
                                ISSUES
‘1
                 NSF agreed in general with GAO's recommendations for improving the
                 procedures for review of research by the independent referees and
                 pointed out that the Observatory had already increased the number of
                 independent referees.   (See p. 24.)

                 NSF stated that, although opinions differed   as to what constituted for-
                 ma1 and complete records on the approval and scheduling of research
                 studies, opportunities  for improvement in this area may exist.

                 NSF stated that the potential  usefulness of additional  information   for
                 evaluation of levels of research efforts merited consideration     and that
                 it would explore with the Observatory the possibility   for accumulating
                 such data.
     I
     I           NSF indicated that the Observatory's         classification     of its temporary
     I           staff as visitors      was appropriate and did not consider that a study of
     I
     I           visitor  participation      as suggested by GAO was necessary.         In support of
     I           this view, NSF explained that the Observatory's             permanent staff supported
     I           projects requiring      long-term research whereas temporary staff generally
     I
     I           supported projects requiring        a more limited     amount of time and that it
     I
     I
     I   Tear
         -----   Sheet

                                                     3
                                                                                    i
    was not intended that visitors  function   as staff   or even as a complement   :
    to the Observatory's  permanent staff.
                                                                                    I
    GAO noted that, although temporary staff may generally be concerned
    with projects requiring a more limited amount of time, several l-year
    appointments to temporary staff were extended to 2 years and that tem-
    porary staff,   in some cases ) complemented the permanent staff in its
    research efforts.     Therefore, in GAO's view, it does not seem appropri-              :
    ate to classify   all salaried temporary staff as visitors  for purposes                I
    of determining compliance with the 60 to 40 user policy.




                                                                                        I




i
                              Contents
                                                                   Page

    DIGEST                                                           1

    CHAPTER

      1       INTRODUCTION                                           5

      2       OPERATIONS OF THE OBSERVATORY                          7
1                Brief description        of radio astronomy         7
                 Functions and achievements                          a
                 Location    and facilities                        10
                 Administrative     and financial     operations   15

      3       PROCEDURESFOR ALLOCATING TELESCOPEOBSERV-
              ING TIME                                             17
                  Review by independent referees                   17
                       Establishing    a pool of referees          ia
                       Updated research proposals       not
                          reviewed by referees                      19
                       Rating information     requested
                          from referees   being expanded            20
                  Approval and scheduling       of research
                     studies                                        22
                       No formal approval record maintained         23
                  Recommendations to the Director        of NSF     23
                       Agency comments                              24

      4       EVALUATION OF LEVELS OF RESEARCHEFFORTS               26
                  Potential   for improved information on
                    research performed at NRA0                      27
                       Recommendation to the Director  of
                          NSF                                       29

      5       USE OF TELESCOPESBY VISITORS AND STAFF                30
                  Intended use of NRA0 telescopes                   30
                  Actual use of NRA0 telescopes                     32
                       Limited use of 36-foot      telescope        32
                       Use by staff   and visitors                  33
                       Classification   of temporary staff
                          as visitors                               36
                       Decrease in average visitor        use of
                          telescopes                                37
                                                                              1


CHAPTER                                                                Page

                    Proposal,     agency comments,           and our
                       evaluation                                       39
                Recommendation to the Director               of NSF     41

      6    SCOPE OF REVIEW                                              42

APPENDIX                                                                          I
                                                                                  /
      I    Letter     dated December 18, 1970, from the Di-                        :
              rector,     National Science Foundation, to
              the General Accounting Office                            45

                              ABBREVIATIONS

AUI        Associated     Universities,       Incorporated

GAO        General    Accounting     Office

NRA0       National     Radio Astronomy       Observatory

NSF        National     Science    Foundation
\   GEflERAL
           ACCOUNTINGOFFICE                   OPPORTUNITIESTO IMPROVE
    REPORTTO THEDIRECTOR,                     ADMINISTRATION OF THE RESEARCH
\   NATIONALSCIENCEFOUiiiL;ATi-ON             PROGRAMAT THE NATIONAL RADIO
                                              ASTRONOMYOBSERVATORY
                                              National Science Foundation                  w"
                                                                               i'        a@
                                                                                          c
                                              B-133338                     r ;
                                                                      . _^ Jf,    L ‘$ .a

    DIGEST
    ------                                                                         I


    WHYTHEREUEWWASIv&DE
        The National Science Foundation (NSF) is the principal    Federal agency
        responsible for supporting ground-based radio astronomy.      The National
        Radio Astronomy Observatory, with its main observing facilities     at
        Green Bank, West Virginia,  wa>established    as a center where astronomers
        of the Nation could perform specialized    research and obtain the use
        of large expensive radio telescopes not otherwise available.

        The Observatory is operated for NSF by Associated Universities,   Incor-
        porated, under a cost-reimbursable  contract.  Federal funds provided
        for the construction  and operation of the Observatory totaled about
        $62.1 million  through December 31, 1970.

        Because of the expenditure of substantial    funds under the contract,  the
        General Accounting Office (GAO) has examined into the policies,     proce-
        dures, and practices  for administering   the research program at the Ob-
        servatory.


    FINDINGSANDCOIVCLUSIONS
        AZZocation of teZescope   observing   time

        The Observatory's    policy is to allocate     observing time for research
        studies on the basis of the scientific      merits of the proposed research
        and of the availability     of the telescopes.     Both staff and visiting
        scientists  desiring to use the telescopes are required to submit writ-
        ten proposals describing     the research, the time required, and the equip-
        ment needed. The Observatory approves the proposals on the basis of
        evaluations  by independent referees and Observatory scientists.
        (See p. 17.)

         The Observatory has, over the years, made improvements in its evalua-
         tion procedures.   Further improvements could be made in the procedures
         for review by independent referees by
   --establishing    a pacal of referees to obtain a wider range of views
      and more expert opinions in highly specialized    areas of research
      and to expedit- R the review process (see pa 18);

   --soliciting   referees" evaluations  of updated requests3 which may
      involve significant   amounts of additional  observing time or other
      changes in the scope of ongoing research to ensure that the addi-
      tional time requested is commensurate with the scientific    merits
      of the research (see p* 19); and

   --obtaining   referees'    pertinent narrative comments on the scientific
      merits of proposed research projects to make the ratings more use-
      ful in allocating    telescope observing time (see p. 2D)*

Also, improvements could be made in the approval and scheduling of rt-
search studies by maintaining      formal and complete records9 including
original    rating data by referees.      Such records9 in GAO's opinion, are
necessary to document the approval and evaluation of research studies
involving     the use of costly federally    owned facilities and would be of
assistance in the orderly management and in surveillance        of observatory
activities.       (See p. 23.)

Evaluation        of      levela    of research         efforts

The contractor  has established   two review committees to assist it in de-
termining the appropriateness    of the Observatory's    concentration of re-
search efforts  and the quality of the research program. The Observatory
maintains basic records of the month'ly use of each telescope and pre-
pares narrative  reports on research activities      but does not summarize
the observing time devoted to individual     projects or significant   re-
search areas.   Such data should assist management and the review com-
mittees in their evaluation of the adequacy of levels of research ef-
forts.   The Observatory has begun to maintain summaries by individual
users for one of its telescopes.

Although this additional     information  should be helpful in the review
of research activities,    it should be supplemented by appropriate sum-
maries of observing time used on al? telescope systems to enable an
effective  overall evaluation of the direction      of research efforts.
Such supplemented information      could be helpful to the Director of the
Observatory in exercising     his responsibility   for maintaining  research
programs of the highest quality and for being alert to needed changes
in research emphasis.     (See p. 28.)

Use of       teZescopes
              --               by visitors        and    staff

The Observatory's  policy provides that visitors   be allocated 60     percent
or more of the observing time on the telescope systems and that        the re-
mainder of time be for use by the resident staff.     GAO"s review     of the
manner in which this policy has been carried out raises certain        ques-
tions regarding the allocation   of observing time between resident      staff
and visitors  because:
      --The Observatory classifies    its temporary employees as visitors
         and on this basis considers that visitors    have used about 54 per-
         cent of observing time during the Observatory's     11 years of opera-
         tions through fiscal year 1969. However, if temporary employees
         are classified  as staff,  which GAO believes to be a more appropriate
         classification,  visitors'  use has averaged only 34 percent of total
         observing time during this period.     (See pa 30.)
      --The average telescope time used by each visitor         during the ll-year
         period has declined significantly     compared with    the average time
         used by each staff observer.      (See p. 30.)

    In GAO's opinion the present policies  and practices followed in al-
    locating telescope time may not be adeQuate to ensure that the Observa-
    tory fully serves its mission as a national center primarily  for the
    benefit of visiting  scientists.


RECOMMENDATIONS
              OR SUGGESTIOX5
    In accordance with the above findings,      GAO is making recommendations
    designed to improve the system for allocating      telescope observing
    time and for evaluating  ?evels of research efforts.        (See pp. 23 and
    29.) GAO is further recommending that NSF, in cooperation with the
    contractorg  undertake a study of visitors'    use of the Observatory's
    telescopes to determine what action, if any,is needed to ensure that
    the Observatory fully serves its mission.       (See p. 41.)


AGENCYACTIONS AND UNRESOLVEDISSUES

    NSF agreed    in general with GAO's recommendations for improving the
    procedures    for review of research by the independent referees and
    pointed out     that the Observatory had already increased the number of
    independent     referees.  (See p. 24.)

    NSF stated that, although opinions differed   as to what constituted for-
    mal and complete records on the approval and scheduling of research
    studies, opportunities  for improvement in this area may exist.

    NSF stated that the potential  usefulness of additional  information   for
    evaluation of levels of research efforts  merited consideration    and that
    it would explore with the Observatory the possibility   for accumulating
    such data.

    NSF indicated    that the Observatory's     classification     of its temporary
    staff as visitors      was appropriate and did not consider that a study of
    visitor  participation      as suggested by GAOwas necessary.         In support of
    this view, NSF explained that the Observatory's            permanent staff supported
    projects requiring      long-tewn research whereas temporary staff generally
    supported projects requiring       a more limited     amount of time and that it


                                       3
was not intended     that  visitors              function     as staff      or even      as a complement       r
to the Observatory's      permanent              staff.

GAO noted that,            although      temporary        staff    may generally       be concerned
with projects          requiring        a more limited          amount of time,        several    l-year
appointments         to temporary          staff     were extended          to 2 years    and that     tern-       '
porary    staff,       in some cases , complemented                   the permanent     staff   in its              I
research     efforts.           Therefore,         in GAO's view,         it does not seem appropri-
ate to classify           all salaried           temporary      staff    as visitors      for purposes
of determining           compliance        with the 60 to 40 user policy.




                                             4
          The ~~v~~~~e~%-0wn~~ Nationa% Wadis Astrsnomy Observa-
                Cl?, which has its principal      observing      faei%i%ies
                lsank, west Virginia,     is (operated for %he Nat%onal
    Science FQun ation under a. cost-reimbursable            contract     by
    Associated     Universities   B Incorporated.   (ATJI), a nmprcafit
    corpsration      formed by nine northeastern      universities.        We
    reviewed the golieies,        prQeedu%es, and psactiees         for (I>
    af%ocating     telescope    observing time, (2) evaluating         IevePs
    of researc& efforts,        and (31 dividing   use of the telescopes
    between visitors       and staff,

             NSF was es%ablished by the NatisnaK Sck-xe               Foundation
    Act of 1950 as an independent Federal agency TV support ba-
    sic research and educa%iQn in the sciences and ts promote
    the in%erchange of seientifie             information     among the w~rld~s
    scientists.          In carrying    cmt its res snsibilities        under the
    act, NSF suppsr%s research in various branches of astronmnyy,
    including       radio astronomy,      a rehative'%_y new discipline         which
    requires      the use of speciaILized c0stl.y equipment,             NSF sup-
    port consists         of providing    grants to educational.      institu-
    tions for individual.          research projects      and of providing,         for
    the use of all interested            scientists,     specialized    research
    facilities,        inc'%uding those of ??EzAoand three other           natdQnal
    sbservatories.          For the 5 fiscal       years 1966 through 1970,
    EJSFsupport of research in astronomy to%aled about
    $Iu3,5 rni'p%iQln, induding          about $37.5 RRi~~iQl-l fQr radio as-
    %~T~I-IQITIYe

           In November P 65 NSF was designated. the principal      Fed-
    eral agency responsible   for support of ground-based radio
    astronomy by CtiQJ2 of the Office Of Science and Technology,
I   Ekecutive   Off ee of the President,     and by agreement among
    the Federa agencies       porting    research in the sciences,

          The need for a natisna% radio astssnomy observatory
    was first  discussed a% an international  conference 8n. radio
    aStT33nQ~~    hE?'Ed   iI   bkShing%QlX,   De&,   ChXil'lg   19%   Which    kd
    %Q a study9 wdertaken      by AU1 under a NSF grant of $85,000,
    to detem~ime %he feasibility     of establishing  such a research

                                               5
center *     After completion   and acceptance of the feasibility
study9 NSF entered into a cost-reimbursable         contract    with
MI,    effective     in November 1956, to organize,   construct,
operate,     and maintain an observatory    for research in radio
astronomy,       Expenditures  under this contract   amounted to
$47.3 million      through June 30, 31968,

      A second contract  effective in July 1968 provides for
the operation  of NRA0 through June 1973. For the 2-l/Z-year
period through December 31, 9970, NSF had allocated
$14,8 million  to this contract,
 \

                                    --CMAPTER2

 \                      OPERATIONS OF TI% OBSERVATORY
iI
     BRIEF DESCRIPTION OF RADIO ASTROMOMY

            Radio astronomy is a relatively       new branch of astronomy
     that observes and analyzes radio signals received from out-
     side the earthIs    atmosphere.      The purpose of studying     these
     signals is to discover and explain new scientific          facts
     about the universe,      Although the results      of such research
     may have ultimate    practical    value, the main impact is, at
     present,    limited to increasing     basic scientific   knowledge.

             Just as optical      astronomers use light waves in making
     their    studies,     radio astronomers use radio waves which are
     collected      and measured by radio telescopes.         These tele-
     scopes can detect many varieties           of astronomical     objects
     which are not visible         to the optical    astronomers.      The typi-
     cal radio telescope         consists   of a hfghly sensitive      radio re-
     ceiver connected to an antenna system that can be pointed
     to different       parts of the sky.      The antenna system consists
     of a large reflector         surface which reflects     the radio waves
     to specialized        equipment mounted at a focal point above the
     reflector,        The radio signals are then carried         to the re-
     ceiver,     an  electronic    device  which  amplifies,    detects,    and
     measures the intensity          of the radio signals received.

            From the receiver9   the signals are processed and re-
     corded by computers as digital       data on tapes or cards.      The
     signals are then further      reduced, plotted,    and analyzed by
     electronic    digital  computers for further    study by interested
     radio astronomers.

          A diagram (furnished   by NRAO) illustrating            the function-
     ing of a typical. radio telescope  follows.
     ANTENNA         I   RAQIO      RECEIVER      I    RECORDING     EQUIPMENT
                     I                            I



FUNCTIONS AND ACHIEVEMENTS

       NRA0 was established     as a national       center where the as-
tronomers of the Nation could perform specialized                      research
and could obtain the use of large expensive radio telescopes
not otherwise     available   at individual     universities             and re-
search organizations.       It is NRAQ's    policy         that   all.    tele-
scopes and support facilities        be available         without       charge
to scientists     and students   from any institution.                 NRA0 is to
promote the utilization       of knowledge in astronomy by appro-
priate     means and to disseminate     and publish          scientific         infor-
mation developed in the course of research work performed at
N-MO.

       NRA0 officials    told us that NRA0 had made or had as-
sisted in about 200 major accomplishments         in radio astronomy,
Most of these involved       new scientific knowledge, although
some involved      the development and/or application     of new in-
.stcy:ments and techniques     to study the universe.     Following

                                          8
    are some notable    examples of achievements       cited   by NRA0 of-
    ficials.

         1. Discovery    of interstellar         formaldehyde.    Formaldehyde
            is the first    complex molecule ever detected           in space
            and its widespread distribution             is expected to pro-
            vide important     information         about the chemical pro-
            cesses of galaxies,        i.e.,     large bodies of gas, dust,
            stars and their companions held together              by their
            mutual gravitational         attraction,      such as the Milky
            Way9 which includes the earth.

         2. Detection   and measurement of the magnetic field  of
            the Milky Way, the study of which is considered    es-
            sential   to the understanding of how stars are formed
            in the Milky Way.

         3. Development and application       of very long base-line
            interferometry.       Under this technique     two or more
            telescopes , placed at long--possibly        intercontinen-
            tal--distances,      study the same radio source and
            achieve very precise measurements.         In addition,
            this technique can be used to study irregularities
            in the earthIs     rotation,   continental   drifts,    and
            temperature     expansion amd contraction      of the earth's
            surface.
         4. First use of autoeorrelation        receivers.     This type
            of receiver9    although designed and built        elsewhere,
            was first    used extensively    for radio astronomy at
            FJR.AO. The large number of receiving          channels built
            into this receiver--as       many as 413--has significantly
            reduced the time necessary to carry out certain            re-
            search projects.




.
TX)CATION m       FACIIJTIES

       NM0 has its        principal     observing     site   at Green Bark,
West Virginia,    and has additional       facilities           at Tucson,
Arizona.     Its administrative   and scientific               headquarters      are
located in Charlottesville,      Virginia,

      At Green Bank, the observing            site covers 2,700 acres.
This site was selected because it is isolated                 from man-made
radio interference.        Mountains     rise    to  more  than   4,000 feet
in multiple    folds in all directions           shielding    t?ze observa-
tory from extraneous       radio signals and against high winds
which might damage the instruments,                Major radio telescopes
at the Green Bank site        include a 300-foot-diameter            tele-
scope movable in a north-south           plane2 a 140-foot-diameter
fully  steerable    telescope,      and an interferometer         consisting
of three 85-foot-diameter         telescopes,        (See  photographs     on
ppO 11, 12, and 13 furnished          by NRAO.)

       At Tucson, a 36-foot-diameter     highly precise telescope
is located at the Kitt Peak National        Observatory,   a national
researchcenterfor     optical   astronomy sponsored and supported
by NSF, This site     was selected because the operation        of
this instrument   requires    a high, dry location     to reduce ab-
sorption   of the signals by water vapor in the atmosphere.

       The original      plans for NRA0 called for all administra-
tive   and scientific       functions to be performed at the Green
Bank site.       Due to the remokness           of the area,       NRAQ found
it difficult       to hire and retain high-quality               personnel.
Therefore,      beginning in 1966 NRA0 moved most of its major
administrative        offices,     library,    analytic     computers, and
scientific      staff     to the campus of the University             of Virginia
in Charlottesville,          Virginia.       The   Charlottesville       center
was built     and is owned by the University               of Virginia      and is
occupied by NRA0 under a lease agreement providing                       for rent
payments during the first              5 years intended to cover the
University's       cost of construction          including     interest,     total-
ing about $702,000.            In addition,      NRA0 pays for building
maintenance and utilities;              these payments will continue
after    the initial       5-year period.       Because of additional
space requirements,          NRA0 has proposed an expansion to the
Charlottesville        center at a cost of about $1 million.                   The
arrangements       for ownership and financing             of the expanded

                                        10
”   .




        11
     :_-
               .__
‘.         ,         1

                 1
‘3NlCilifl6’    lO~lN03       ~313WOa33831NI             3Hl   NJ 3lOSN03       3NO   WOkId     Cl31lOtllNO3              SI W31ShS    SlHl    ‘3Nll3SVS     V 3NOlV    319VlaOd         38V   H3IHM
       JO OMl    ‘S3dOX3131         OIQV’tl    lOOd-SE     319Vt1331S       338Hl     S3SfI   ‘NOIl-lIW        9’l$lrlOEit’      30   IS03    V IV   llln9   ‘W31SAS   ki313W0~3d~31NI           3Hl




                                                                                                                                                                       ..
building    would be similar  to those for the existing  build-
ing.     NSF has not yet approved these expansion plans.

       In connection     with the exploration         of new types of re-
search equipment in radio astronomy,              NRA0 has made studies
for the design of a very large array radio telescope                  and a
213-foot-diameter       high-precision       antenna.     The costs in-
curred by NRA0 in conducting           these studies through fiscal
year 1970,    exclusive      of  staff    salaries,    totaled  about
$939,000 for the array and about $118,000 for the high-
precision    antenna.

      The array, proposed for construction         in the Southwest-
ern United States, would consist      of 27 antennas arranged on
a Y-shaped configuration    of railroad    tracks.      Each arm of
the "Yrl would be 13 miles long,      The  cost    of developing  and
constructing    the array has been estimated       by NRA0 at $60 mil-
lion.    The high-precision  antenna was in the preliminary
planning stages and was estimated       to cost about $10 mil-
lion.

      An advisory panel convened by NSF to make recommenda-
tions and establish    priorities  on proposals   for design and
construction   of large radio telescopes    favorably  recommended
these two projects    in its report of August 1969.

       In its fiscal   year 1971 budget presentation, NSF stated
that design work on the two systems would continue with ex-
isting    funds and that no funds had been requested for the
const,ruction.




                                    14
A.D/IINISTRATIVE Am FINAMCrn              OPERATIONS

        As of December 31, 1970, NSF had provided about
$62.1    million  for the financial.support       of Nl?AO's construc-
tion.      As of June 30, 1970, expenditures       of NSF contract
funds    totaled  about $58.3 million.       This amount included
$28.2    million  for land, buildings,      and equipment,    of which
$20.2    million  was for the construction       of observing   facili-
ties.

       Under the terms of the operating        contract,    AU1 is paid
a management fee which is intended to defray AUI's normal
operating    expenses associated     with its corporate      being but
are not directly     related   to the work under the contract.
AU1 accumulates the portion       of the fees not required        for
operating   expenses as a corporate       reserve.      The fees  paid
to AU1 from inception       of the contract    through June 30, 1970,
totaled   $1.3 million.      Fees were paid at the following         an-
nual rates.

                    Fiscal     year                Annual      fee

                   1957                              $ 40,000
                   1958                                45,600
                   1959                                55,400
                   1960                                60,000
                   1961 and 1962                       65,000
                   1963-67                            125,000
                   1968-70                            100,000

      The annual funding             Qf N&io'S   operating      expenses   during
the last 5 fiscal   years            follows.

                    Fiscal                         AlllOl.lM
                                                 (millions)

                     1966                          $ 2.8
                     1967                            3.5
                     1968                            3.8
                     1.969                           4.0
                     1970                            4.3

                             Total                 $18,4

                             Average               $37
                                                    --I
                                                    ---
                                          15
       About 60 percent of the operating   expenses were for
salaries     and related benefits of NRA0 scientific and ad-
ministrative     staff.

      The following  table shows NRAQ's total   staff as of
June 30, 1970, at the three locations.     Temporary staff  in-
cludes 50 students who were participating     under NRAO's sum-
mer student program.                                              i
                              Permanent     Temporary     Total

Charlottesville, Virginia         69            37         106
Green Bank, West Virginia        151            41         192
Tucson, Arizona                    7            -3          10

    Total




                                                                      I



                                                                  i
                                   cm?J?TER3

         PROCEDURESFOR ALLOCATING TELESCOPEOBSERVING TIME

           It is NR.AO*s policy to allocate        telescope observing
    time for research studies on the basis of the scientific
    merits of the proposed research and of the availability                  of
    the telescopes.        Both visitors    and staff    desiring    to use
    the telescopes      are required     to submit written      proposals    de-
    scribing    the research to be undertaken,         the observing      time
    requested,     and the type of equipment needed.            NRA0 has de-
    veloped a system of approving these requests on the basis
    of evaluations      by both independent expert reviewers           and
    scientists     on the staff    of NRAO.

           NWAOhas, over the years, made several improvements in
    its review and evaluation    procedures to properly   implement
,   its stated policy.    Our review showed opportunities     for
\   certain   additional improvements.
\   REVIEW BY INDEPENDENT REFEREES
\
           Prior to 1966, the review of proposals   for research
    was handled by the NRA0 management without     advice of outside
    experts.     In 3une 1966 the NRA0 Visiting  Committee1 recom-
    mended an improved review procedure as follows:

           "It was obvious *** that problems are beginning
           to arise in the matter of scheduling        and project
           priority.     *** There was unanimous agreement
           among members of the Committee that a schedul-
           ing committee with outside members should be
           set up. It is recommended that this committee
           be advisors      to the Director  on science and that
           it have regularly       scheduled meetings.    Final
           authority    and responsibility     for scheduling   and
           project   priority     should remain with the Direc-
           tor.*'


    1
     A review      committee established by AU1 to evaluate            the
      scientific     activities  of NRAO. (See p. 26.)


                                          17
        Subsequently,    in 1966 NRA0 established     a system of
sending requests      for observing   time to independent referees
who were noted astronomers        connected with universities        ac-
tive in the field      of radio astronomy.      The referees    serve
for an unspecified       period and without    compensation.     Ini-
tially     NRA0 appointed three referees     from different     uni-
versities;     one has served continuously      since inception      and
two have been replaced,

       The referees    furnish  their   evaluations    independently
of each other through the mail on rating            sheets supplied by
NRAO. The referees        do not meet as a group or individually
with the Director      of NRA0 in their     capacity   as referees.
Therefore    they do not function     as part of a scheduling        com-
mittee   as was recommended by the NRA0 Visiting           Committee,
but the Visiting     Committee has expressed its general approval
of the referee     system as satisfying      its 1966 recommendation.

       Our review indicated    that improvements could be made
in the referee     system by (1) establishing       a pool of referees
to obtain a wider range of views,         (2) soliciting      referees'
evaluations    of updated requests     involving    significant       addi-
tional   observing   time, and (3) expanding the rating            informa-
tion being requested      from referees.

Establishing     a pool   of referees

       We believe  that increasing     the number of referees would
permit NRA0 greater     flexibility    in assigning research pro-
posals for evaluation      and obtaining   expert advice in spe-
cialized   areas of research.

       During fiscal      years 1968 and 1969, NRA0 was using only
two referees     to evaluate      the majority    of research proposals
instead of obtaining        the evaluation      by three independent
reviewers     as originally      contemplated.     We were informed that
appropriate     comments could not be obtained             from the third
referee    during this period.         A replacement      appointed during
fiscal   year 1970 made it possible          to resume the general
practice    of soliciting      the views of three referees.           The use
of a minimum of three referees           appears to be desirable         to
obtain a consensus when the views of two referees                  concerning
the merits of a proposal significantly             differ.      We noted
several    such cases.
      Alsa, the use of a larger number of referees        appears to
be desirable   since the field    of radio astronomy includes
several specialized      areas of research--such   as solar studies
and galactic   studies--and    NFW could refer   research proposals
to those reviewers     most expert in the particular     area,

      An additional      advantage of establishing    a pool of ref-
erees is that it would be possible        to expedite   the review
process,  particularly       in the event that the number of re-
search proposals       should increase beyond the present annual
volume of about 100 as a result        of a larger   demand for the
use of the observing       facilities  as expected by NRAO.

      In response to our suggestion     that NRA0 consider      ex-
panding the number of referees     used, NRA0 officials     told us
that on a few occasions proposals     had been submitted      to
other outside referees   when the present referees      were not
expert in the proposed research area, but they agreed that
this practice   could be expanded.

Updated research proposals
not reviewed by referees

       Requests to update an initial       proposal   to do research
generally   are not referred      for review and evaluation     by the
referees,   even though significant       changes may be involved     in
the research method employed, the equipment used, or the ob-
serving time required.       Initial   proposals    may be updated for
one of the following    reasons,

      1. New developments in the research         area occur between
         the date of the initial proposal         and the start  of
         the project.

      2. Similar    proposals     are received     from more than one
         researcher     requiring     coordination     or other changes
         of the individual        proposals.

      3. The project  cannot     be completed    within   the initially
         approved time.

     For example, of the 28 projects     carried  out on the 300-
foot telescope   during fiscal   year 1969, eight involved   up-
dated requests,   some of which were for significant     amounts
of additional   observing  time,
       One of these requests    called for an increase in total
observing    time from 120 to about 1,800 hours.         The referees
were asked to comment only on the original          request for 120
hours, and in their    evaluation     they expressed some reserva-
tions as to its merit,      NRAQ approved and scheduled most of
the updated request,    without    further  consultation    with the
referees.

        We believe  that updated requests       should be submitted
to the referees     to ensure that the additional         time requested
is commensurate with the scientific           merits of the research.
In our discussion      of this with NRA0 officials,         we were told
that it was NRAO's intended policy          to submit updated requests
to the referees     when significant     additional    time was re-
quested.      Since this policy    apparently     was not followed,    the
officials     agreed that attention    should be given to its im-
plementation.

Rating information    requested
from referees    being expanded

       The NRA0 Visiting      Committee commented in its May 1969
report   that "The system for outside review of proposals      is
considered    excellent,     but further effort should be made to
ensure responsive       reviewers."

        For research proposals       evaluated by referees         prior     to
fiscal    year 1969, rating     information       requested    from referees
was limited     to an overall    project     rating,    designated      "good,f'
"'average,"    or "poor9" and to optional           comments on the merits
of the project.       Referees were not requested           to assign pri-
orities     and to evaluate   the length of the observing             time re-
quested.      The extent of narrative        comments furnished         varied
from proposal to proposal,         and'such comments were only very
brief    and general in nature.        For example, for the 28 proj-
ects carried     out on the 360-foot       telescope     during fiscal
year 1969 when NRA0 was using only two referees,we                  found:




                                       20
                                                       Number
                                                     of projects

      No comments made by either.referee                    8
      Comments made by one referee only                    14
      Comments made by both referees                       -6
           Total                                           n28
        In April 1969 NRA0 added a new category designated
"priority"     on the rating    sheet which the referee may choose
in the event that a proposal should be given preferential
treatment    displacing,     if necessary,   other proposed projects.
In October 1969 NRA0 further        revised the rating    sheet to in-
clude the referee's      evaluation    of the observing   time that
should be approved for the project.

       We believe that these revisions        in the rating     sheet will
make the referee's      evaluation    more meaningful   and useful to
the Director     and the staff     of NRA0 but that consideration
should also be given to obtaining,         in all cases, pertinent
narrative    comments on the scientific      merits of proposed re-
search projects     to supplement the basic rating        data.
APPROVA'I,AND SCHFLDUI,I?!GOF RESEARCHSTUDIES
                           ___--I--~-_

        Research proposals          in most instances       are approved and
scheduled on a tentative             basis several months in advance of
the date the project           is to be performed.          All proposals       are
first    referred      to the NRA0 Director.          The proposals        are then
forwarded      (1) to designated         staff   scientists      according    to
the telescope        for which they have been assigned responsibil-
ity,    (2) to the independent referees,              and (3) to several
NRA0 technical         divisions     which will be concerned with the
technical      requirements       of the project.        After    the referees'
evaluations       and any comments by the technical               divisions    have
been received       and considered,         the projects      are tentatively
entered on monthly schedules which set forth?                     for each tele-
scope,     the  dates     of  all  proposed    projects     to  be   performed
during the month.            At that time, informal         notice is given
to the interested          astronomer     that his project        has been ten-
tatively      approved.

       Changes in the tentative            schedules are made from time
 to time, as considered         necessary,      to accommodate competing
proposals,     the availability        of supporting      equipment, or any
other developments affecting             the use of the telescopes,
Final approval of the allocation              of telescope     time is made
at the monthly meeting of the scheduling                committee composed
of the Director      and key scientific          and technical     officials
of MUJO. The scheduling           committee,      after   considering      the
availability     of facilities       and equipment,       in particular        re-
ceivers,     computers,    and other electronic         instruments,       de-
cides on a final       monthly schedule for each telescope.                  Such
schedules are circulated          within     NRA0 and are furnished          to
the astronomers      whose projects         have been approved.

       In the process of approving and scheduling            individual
projects,    the dates and length of time of a project's              per-
formance may be adjusted        several times and may be spread
over several months.        As a result,    the observing      time actu-
ally scheduled for a project         has varied significantly         in
many instances       from the time requested.       For example, of the
28 projects     carried   out on the 300-foot      telescope     during fis-
cal year 1969, one project        was scheduled for the same period
of time that was requested,        15 projects    were scheduled for
more time than requested--the         scheduled hours ranging up to
426 percent of requested hours--and           seven projects      were

                                         22
    scheduled for less time than requested--ranging    between 89
    and 13 percent of requested  hours.   For the remaining   five
    projects,  the record was not clear as to the number of hours
i   requested.
\
          We were told that some projects   had been granted more
    time than requested because vacant time had occurred on the
    telescopes   and it had been determined  that these projects
    could productively   use the additional  time.

    No formal    approval    record   maintained

            NRAOvs procedures        for approving and scheduling       research
    projects    do not require         the maintenance of a formal record
    showing, for each proposal,             the telescope    observing  time re-
    quested, NRAQ's approval or disapproval               of the request,    the
    total    observing     time authorized,       and the reasons for any
    significant      differences       between the time requested      and the
    time authorized.          Also, the ratings      received   from the refer-
    ees are not made a part of the permanent record of reviewing
    research proposals.           To protect    the anonymity of the refer-
    ees 9 the adjective        ratings     and any narrative    comments are
    abstracted     and transcribed         onto the proposal    document and
    the original      rating     sheets are discarded.

           In our opinion,       a formal and complete record,           including
    the original       rating   sheets submitted        by the referees,     is nec-
    essary to document the evaluation             and approval of research
    studies     to be undertaken      on NRAO's telescopes        because these
    are important       management decisions        involving    the use of
    costly    federally      owned research    facilities.       Also, we believe
    that a formally         documented approval is needed to assist             the
    Director      and his staff     in theorderlymanagement          of NRAQ and
    to assist      the review committees       appointed by AU1 in their
    surveillance       of NRAO's activities.

    RECOMI'BNDATIONSTO THE DIRECTOR OF NSF

             We recommend that,     to ensure that NRAOJs telescope    fa-
    cilities      are allocated   in the most effective    manner and for
    the most deserving research         studies,  NSF bring to the atten-
    tion of AU1 the several opportunities          which exist--as  dis-
    cussed in this chapter --for        improving NRAO's system of using
    independent referees        in the evaluation   of proposed research
    studies.       We recommend also that NSF bring to AUI's attention
                                          23
                                                                                     /
                                                                                         i



       The Director,      NSF, in a letter      dated December 18, 1970,
advisd     us that NRA0 had taken steps to improve the imple-
mentation     of the referee     system.     He stated that NRA0 now
received    reviews from three regular referees,             that appropri-
ate members of the resident          staff   provided    expert advice
whenever there were differences            of opinion among the refer-
ees) and that NRA0 had two additional             referees     for solar
proposals 9 an area that is outside the field              of two of the
three regular      referees.     He also pointed out that,          although
1JSFT*j,lSopposed to the establishment           of a large pool of ref-
eree:, ) it wo-uld consider with NRA0 the desirability               of ap-
propriate     additions    to the pool of referees,        particularly
in speclalrzed       areas of research such as had been done for
proposals     involving    solar studies.

         The Director    agreed that updated proposals            requesting
sfgi:ificant      amounts of additional      observing      time should be
rcvjewcd by the referees.            He noted that NRA0 followed            the
policy of resubmitting         to referees     updated proposals          in cases
where the scope of the work had significantly                  changed.      He
pointed out, however, that a decision              to resubmit was one
that required       scientific    judgment in each case and that,              if
the judgment of the referees           regarding     a particular       request
was known through prior          comments of the referees,           the up-
dated proposal may not be resubmitted.                Because our review
showed that this policy was not always followed                   even in a
case where i.t was known that the referees              had reservations
concerning      the merits of the initial        proposal,      we believe
that this matter is in need of further              attention.

             the Director
       .Al. so )              agreed that narrative    comments by
reviewers   on the scientific      merits of proposed research proj-
ects were valuable     supplements to the basic rating      data.
He stated   that, although NRA0 considered       the comments by ref-
erees to be adequate, this did not preclude         the consideration
of med~-~sto encourage expansion of such data..

       With respect     to our recommendation concerning    the need
for   marP adequate     documentation    of NIUO"s management actions
                                      24
regarding     the evaluation   and approval of research proposals,
the Director     stated that,    although opinions differed    as to
what constituted      formal and complete records,     he agreed that
opportunities     for improvement in this area may exist.        Al-
though he pointed out that W'maintained            a record on such
management actions,       he agreed further,   however, that there
was a need for improvement in documentation.

        In elaborating       on the Director's        statement,   agency of-
ficials    informed us that NRAO's record of these actions did
not consist       of a single consolidated          record providing       com-
plete information         related      to the pertinent      management ac-
tions but that most of this type of information                   could be de-
veloped from various records,               documents, and memorandums
which were kept at NRAO. The Director                   advised us that NSF
would review with AU1 and NRA0 appropriate                   means of accom-
plishing     the report recommendation for more adequate docu-
mentation,      including      retention     of the referees"     original
rating    sheets.




                                       25
           AIJI   has   established   two   review   ccmmittecs   to   assist   its
     president   and board of trustees    in their    task of detelrxraining
     the appropriateness     of NRAD6s ccmcent:ration     of research ef-
     fsrts   and the quality   of its research program.

            One of the review co~~Cttees is the Tr~:stces Comittee
     for lYRA.Qwhich is composed of three members sf the board and
     the president    serving ex officio,         The AUP -pl.ic~ nanua'b
     provides that one of the major areas of interest             to the cop11-
     mitcee be an "evaBuaticsn of the present and prospective             bal-
ii   9nce of effort     a171ongthe fields    of research concentration      of
     the resident    staff.P"    This corrarnittee is reqla:ired to report
     to the board annually on its findings.            F$e have been told
     that the committee presents its findings           infon~ClE>i- to the
     ful.2 board of trustees      without preparing     a fcr~~al. record of
     its report e

              The other review cosnnaittee is the NRlKl Visiting      Comnmit-
     t:ec? fnade up of one AUP trustee    and six distinguished       scien-
     tists     selected  from other public and private    institutions,
     most of whose are radio astronomers.        This committee is ex-
     pected to snake an expert examination       of the scientific       ac-
     tivities      at NRA0 and, in its annual. report thereon, make rec-
     ommendations with respect to the research program> including
     whether efforts       in any area should be increased,     diminished,
     or discontinued.

             In our review of the Visiting        Co1?Pn?ittee's reports for
     the last 5 years, we noted that on several occasions the
     comittee     had comended NRA0 for both the quantity             and quaI-
     ity of the scientific        output and that only cne specific          rec-
     smendation      had been made, The comittee@s            report of May
     1966 proposed that increased use be made of the %40-foot
     telescope    in the investigation       of the variability      of radio
     sources which, according to NRAO, was subssquentky accom-
     @shed.       The cornntittee, however, apparently         was not fully
     satisfied    with the information       presented for its review         of
     NIUOBs research activities,         since it requested in its 1968
     report that a listing        of aI1 scientific    prograsns in progress

                                             26
     be submitted to members of the committee in advance of their
     meeting to help them plan the most efficient     coverage of
     NRAOes activities.  Such a list   was submitted,   as requested,
     for the committee's review starting   in 1969.

     POTENTIAL FOR IMPROVED INFORMATION ON
     RESEARCHPERFORMEDAT NRA0

           Our review of the type of information        at NRA0 available
     to evaluate the direction      of its research efforts      indicated
     a potential    f or certain improvements which, we believe,
     would be useful to the review committees.          NRA0 maintains
     basic records of the monthly use of each telescope           and pre-
     pares narrative     reports on research activities      on a quar-
     terly  basis and an annual basis but does not summarize the
     total  hours of telescope    time spent on individual      projects
     and does not accwnulate them by significant         research area,

            We proposed to NRA0 officials       the desirability        of a
     cumulative     record,   possibly  on an annual, basis,       of the ob-
     serving hours spent by al.1 astronomers          classified     by indi-
     vidual research area considered        significant       by NRA0 manage-
     ment.    For example, the 28 projects        that were carried       out
     during fiscal      year 1969 on the 300-foot       telescope,    which
     used a total      of 7,669 observing hours, could be classified
     according    to the research categories
                                          -         used in NRAO's annual
     reports    as follows:

!’                                                                                                        Number of    Hour5
                                                                                                           protects   -used     Percent
                                               Research        area

                 Pulsars--Searches            for and studies      of astronomical
                    objects     which      emit e strong,     rapidly    pulsating
                    radio    sienel                                                                               9   3,240        42.25
                 Neutral      hyiiroaen       lines--Studies          of neutral         hydro-
                     $Z.Sn .WIS of B SDeCified             wevelennth        in the Flilkv
                     way &d other ~alexies                                                                        7   1,732        22.58
                 Source surveys--Survey                 to identify        sources        of ra-
                     dio signals          in the universe          for the purposes               of
                     establishing          maps, catelogues,            end mmber counts
                     of them                                                                                      4   1,458        19.01
                  Ionized      hydrogen       regions      and recombination              lines--
                     Studies      of regions         aramd     very     bright      stars      where
                     hydrogen       gas is electrically              &rg;d                                        2       641       8.36
                 liydroxyl      molecules--Studies             of radio        signels        emitted
                     by molecules           composed of one atom of hydrogen                        end
                     one atcm of oxygen                                                                           1       359       4.66
                 Suoerncwa        remnants--Studies             of the gases remaining
                     frcm a star         which has exploded                                                       2       138       1.80
                  Spectral      studies       and time variations--Studies                     of
                     variable       radio      murces      and their       physical         prop-
                    erties                                                                                        2        96
                 Exterior      galaxies--Studies               of the physical      proper-
                    ties     end characteristics               of galaxies     beyond the
                    Milky     way                                                                             1       3

                                                                                                              2       7.669



                                                                                 27
      We obtained the above information        by analyzing      the
monthky summary of telescope utilization          and by correlating
the number of hours used on specific        projects    during the en-
tire fiscal   year 1969 with the description         of significant
research activities      contained   in NRAO"s annual report for
the year.    In our analysis,      we were assisted    by NRA.8 person-
nel who generally     agreed with the proposed classification
of telescope   projects.

      NRA0 officials     agreed with us that additional     informa-
tion on research projects       would be useful. and informed us
that,   since completion     of our field work, NRA0 had begun
maintaining    summaries of the observing time of each user of
the 14%foot     telescope    and that these summaries would be
made available     to the review committees.     These officials
also told us that similar       summaries would be *maintained for
the other telescopes      when increased demand for their use
would make such records desirable,

       NRA0 officials further    pointed out that the field of
radio astronomy was still     relatively  limited in scope so
that balance among research areas could be achieved without
additional   formal use records.

        We believe that the additional     information    that is be-
ing accumulated by NRA0 on the use of the 140-foot            telescope
will be helpful      in the review of research activities        but
that,    to enable an effective    overall  evaluation    of the di-
rection    of research efforts,    it should be supplemented by
appropriate      summaries of observing time ,used on all telescope
systems, according to research projects         and significant
areas of research.

      We believe allso that such supplemented information
could be helpful    to the Director    of NRA0 in exercising his
responsibility   for maintaining    research programs of the
highest quality    and for being alert to needed changes in re-
search emphasis.

      The NSF Assistant   Director      for Administration,  with
whom we discussed this matter,        told us that a summary record
showing Eeve!ls of research efforts        at IRAQ would provide use-
ful information   for NSF's astronomy support program and that
NSF believed such an expansion of NRAOcs program records
would be feasible    and desirable.
                                    24:
Recommendation    to the Director     of NSF

        We recommend that the Director,    NSF, bring to        the atten-
tion of the board of trustees       of AU1 the desirability         of im-
proving the information     available   on the levels of        research
efforts    at NRA0 and that the board require     NRA0 to      make such
improvements to its records as may be most useful             to AU1 re-
view committees and to Nl?AQmanagement.



      The Bisector,   NSF, informed    us that:

      "NRA0 management had reservations        about the need
      to expand the present information        systems, be-
      lieving     that scientific   management should seek
      to foster      and encourage the very best proposals
      for telescope      time that are submitted    to it by
      scientists,      but it should rarely  attempt to man-
      age the scheduling by research area."'

He also advised us that NRAO's management also believed
that,   as long as the visitors     and staff    that used the facil-
ities   were first-rate,   the proposals     submitted would indi-
cate where the interesting      research areas lay and that the
proportion    of time spent observing      in various research areas
should be determined through the process of trying             to pro-
mote research in specific      areas,     The Director    stated that
NSF agreed in general with this philosophy;            however, NSF
believed that the potential       usefulness   of additional      infor-
mation merited consideration.         He stated also that NSF would
explore with NRA0 the possibility         of accumulating     data in
such a manner that the time and effort         involved would not
outweigh its usefulness,
                                      CHAPTER5

               USE OF TEIESCOIPESBP VISITORS               STAFF

       NRA0 was established      as a national    research center to
be used primarily     by visiting    scientists,
establishment,    both NRA8 and NSF have publi
several sccasions the policy that visitors            will be allocated
60 percent or more of the observing           time on      OBs tele-
scope systems    and that the remainder of the time will be for
use by the resident     staff.

        Our review of the manner in which this policy has been
carried    out raises certain   questions   regarding  the alloca-
tion of observing     time between resident     staff and visitors
because:
      --   .NRAO classifies     its     temporary employees as visitors
           and on this basis considers that visitors              have used
           about 54 percent of observing time during NRAO's                   I



           11 years of operations            through fiscal   year 1969.
           However, if temporary employees are classified               as
           staff--which        we believe to be a more appropriate
           classification--visitorss             use has averaged only 34
           percent of total          observing time during the ll-year
           period.

      --The average telescope    time used by each visitor   dur-             A
         ing the ll-year  period has declined  significantly
         compared with the average time used by each staff
         observer.

     Also,      there has been only minimal use of the 36-foot
telescope     located at Mitt Peak National  Observatory   because
of problems connected with atmospheric      conditions   which have
limited    actual observing time in 1969 to less than 12 per-
cent of total       available time. We have been informed that
remedial actions have been taken by NRA0 to increase the
usefulness     of this telescope.

JKI'ENDED USE
            --._OF
       _-____       NRAQ TELESCOPES
                -_-~.-   .---,----
     In the original          plan for    establishing    NRAO, prepared
in 1956, AUI pointed          out that    the support    of a national
research center by NSF seemed to offer         the only way whereby
radio astronomers      in this country would have access to large
telescopes.      In response to questions    raised in the fiscal
year 1958 Senate appropriation       hearings regarding   the pur-
pose of the new observatory,      the Director    of NSF pointed out
that AU1 was only the constructing        and managing agent for
NRA.0 and thatitsinstruments      were for the use of all astron-
omers of the country.

       In 1959 the NRA0 Director         explained   NRAQ's user   policy
as follows:

     I'*** We believe that about 60% of the research
     activity     at the NRA0 should be by visitors9         and
     40% by the permanent staff of the Observatory.
     This ratio of visitor        to staff activity     is based
     on the experience       of other institutions      with sim-
     ilar aims.        It is felt that a higher ratio       of
     visitor    activity    does not allow a staff      of suffi-
     cient size and diversity        to maintain productive
     staff    research and still     provide necessary ser-
     vices to visitors,       while a lower degree of visitor
     activity     would not be consistent       with the principal
     objective     of the Observatory     as an institution       for
     all scientists,

     The GO/40 ratio     cannot of course be rigidly   ad-
     hered to on an hour by hour basis in the schedul-
     ing of observing    time and other activities   at the
     Observatory.     It is, however, a basic premise that
     is used as a guide in determining     the size and
     nature of the Observatory     staff."

       The policy that visitors     be allocated  QQ percent or
more of the observing    time has been stated in such publica-
tions as NRAO's annual reports        and in NSFBs budget justifi-
cations   to the Congress.     Certain other public statements by
NSF and NRA0 have referred      to a policy of allocating    about
70 percent of telescope     observing    time to scientists  not on
the NRAQ staff,




                                    31
ACTUAL USE OF NRA0 TELESCOPES
I--
      In contrast     with optical    telescopes    which generally
can be used during nighttime         only, most radio telescopes      can
be used for some type of observation           by both day and night.
Our analysis    of actual usage of NRAOVs four major telescope
systems for fiscal       year 1969 showed that,      with the excep-
tion of the 36-foot       telescope   located at Kitt Peak National
Observatory,    the systems were used for observing most of
the time after     allowing    for testing,    maintenance,  bad
weather,   and similar     factors   as follows:

                                 Percent of telescope         time used
                         Interfer-      300-     140-          36-        Aver-
                           ometer       foot     foot          foot       2s
Actual observing              74.08         87.54     78.58    11.52      62,93
'resting                      14.94          1.76      2.73     7.47       6.72
Time not sched-
   uled                         .82            .84     1.08    57.32      15.01
Maintenance                    4.62          5.51      5.98       -        4.03
Equipment changes               .85          2.46      3.39     1.00       1.93
Lost time                      4.69          1.89      8.24    22.69       9.38

     Total                100.00        100,00       100.00   100.00   100.00

Limited      use of 36-foot     telescope

       The 36-foot   telescope  located in Tucson, Arizona,      was
constructed    at a cost of about $1 million     and was placed in
operation   in April    1967.  This is a highly precise instru-
ment specially     designed to study radio signals of millimeter
lengths which can not be studied on other NRA0 telescopes.
Although NRA0 had intended that the telescope would be fully
used for both daytime and nighttime      observing,    in fiscal
year 1969 actual observing      time was limited    to about 11.5
percent of total     available  time.

        We have been informed by NRA0 officials           that the utili-
zation of the 36-foot       telescope    has been low because the re-
flector    surface has deformed due to the sun's heat, which
has limited     the instrument's     effective      use to nighttime   ob-
serving,    and because the millimeter          radio waves have been
absorbed by water vapor in certain            rainy seasons, which has
    further  limited      the telescopess        use to periods     of dry
    weather,

           At the time of our field       review,     NIL40 technicians        were
    studying means to increase the utilization                of the 36-foot
    telescope.     NRA0 utilization      records showed that during the
    period January to June 1970 the 36-foot             telescope       was used
    for observing      about 50 percent of the total            available    time.
    NSF advised us that it had reviewed the observing                   time of
    the 36-foot     telescope    for the months of September, October,
    and November 1970 to determine current             utilization.         The
    review showed that actual observing            time, as a percentage
    of maximum possible       observing    time in a 24-hour period,            was
    72 percent for September, 53 percent for October,                   and 61 per-
    cent for November, of which use by visitors                 averaged 56 per-
    cent,   and  that   NSF  expected   that   actual    observing      time would
    continue at a comparable rate in the months ahead,

            The Head of the NRA0 Tucson Division              advised us that
    the 36-foot      telescope was an experimental            instrument      and ini-
    tially     its performance      capabilities       were not fully     under-
    stood.       He stated that added experience had provided a better
    understanding       of the functional         and technical     capabilities
    of the 36-foot        telescope which had made possible            reasonably
    accurate predictions          of the effect      that deformities        of the
    reflector      surface had on the results           of the scientific        data
    collected.       This,,   according     to  the  Head  of   the  NRA0   Tucson
    Division,      was the principal        reason that permitted        increased
    use of the telescope,          particularly      for daytime observing.

i   Use by staff       and visitors

           The following    table shows the use of NRAO"s four major
    telescopes     during the 11 fiscal  years,  1959 through 1969,
    by the three classes of users:       permanent staff,   temporary
    staff,   and visitors.




                                            33
                        Percent    of telescope    observing     time used
                       Interfer-     300-       140-        36-        Aver-
                         ometer      foot       foot        foot       siz!2



      Permanent          63.96        30.79        39.14       54.88        46.28
      Temporary          19.63        25.21        11.49        6.39        19.61

                         83.59        56.00        50.63       61.27        65.89     I

Visitors                 16.41        44.00       49.37        38.73        34.11

      Total             100.00       100.00      100.00       100.00      100.00
                                                                          ___-
        Depending on the classification              of temporary staff--
FThFch NRA0 considers          to be visitor,       whereas we believe that
the temporary staff           should be more appropriately             classified
as staff      (discussed      on p. 36.)--different          percentages      re-
sult when seeking to determine compliance with the 60 to 40
ratio     established      for visitor-staff        usage.      According to
MULOBs classification,           the 60-percent        goal for visitor         use
was attained       for the 300-foot         and the 140-foot        telescopes
but not for the interferometer               (36.04 percent)        and the 36-
foot telescope         (45.12 percent);       overall,     visitor     usage was
53,72 percent.          By excluding      temporary staff        from visitors,
the 60-percent        goal was not met for any of the four systems;
visitors!      overall     use of all telescopes         was 34.11 percent.

       During this ll-year    period,  the visitor  participation
varied considerably     for individual   years and for individual
telescopes,   as shown in the following      table.




                                        34
    Percent   of Telescope      Observing     Time Used by Visitors
                                                           (note a>

Fiscal          Interfer-       300-          140-         36-        Aver-
 year             ometer        foot          foot       foot         as
 1959            40,18                                                40.18
 1960            15,62                                                15.62
 1961               .14                                                  .14
 1962            45.77                                                45.77
 1963            18.29           39108                                28.58
 1964                            29.27                                21.99
 1965             1.27           34.86                                22.11
 1966            15.87           56.89        39135        -          36.57
 1967           100.00           72.59        49.16        -          59.36
 1968             8,93           36,16        63.06      61.72        37.11
 1969            11.99           51,97        44.98      14.67        36.36

Average                          44,oo        49.37      38.73        34.11

aExcluding    temporary     staff.

        We discussed with NRA0 the reasons for the wide varia-
tions in the visitor      participation      shown above.   We were
told that the interferometer          had the lowest overall   visitor
utilization     primarily   because the use of this system re-
quired the scientist      to remain at NRA0 for considerable
lengths of time and thereby made it more difficult            for visi-
tors to use this system.          The low rate of visitor    use of the
interferometer      in 1960 and 1961, which at that time con-
sisted of only one telescope,          was attributed   to the lack of
demand because NRA0 was relatively           new.

      NRA0 stated that it believed that statistics                 for visi-
tor usage of the interferometer          in 1964, 1965, and 1967, of
the 300-foot    telescope      in 1967, and of the 36-foot           telescope
are not meaningful.        It was NRAO's view that these statistics
were affected     by the fact that the total            available    observing
time on these instruments         during those years was signifi-
cantly reduced below normal amounts because of major altera-
tions or additions      of equipment to the telescope             systems.
The limited   available      observing time coupled with NRAO's
policy to allot     available     time to the most deserving projects,
whether visitor     or staff,     has resulted,       in NRAO's opinion,
in the wide variances        in visitor    participation.
                                         35
Classification
._
 .--                 of temporary      staff    as visitors
                                         l_l_-l__...-,
        The temporary staff   %ncBudes research associates,     xi-
entists    on leave from their home institutions,      and students.
Temporary staff     employed by NRA0 in June 1969 were receiving
salaries    at annual rates ranging from about $4,300 to
ST.4g400 D The number of temporary staff and the percent of
tzheir observing time on all telescopes       during fiscal   year
",949 were as follows:

                                                              Percent of
                                              Number           time used
                                                               ------
          Research associates                    %2              13 I.51
          Scientists                              5               9.71
          Students                              13
                                                -                 4.96



       According to AU1 policies,   research associates   are ap-
pointed for 1 year and their appointments     are normally re-
newable for an additional     year.  About one third of the re-
search associates   later accepted positions   on the permanent
staff,

        We have been told by NRA0 that temporary staff are con-
sidered visitors         because they are permitted           to work on re-
search work of their          own choosing and because their research
work benefits         the scientific       community as a whole rather
than NRA0 alone.          We have been told also that research asso-
ciates 9 during their         l-or Z-year stay at NHAO, are gaining
valuable     knowledge and training           that will benefit         the out-     ,
side community when they leave NRA0 and join another insti-
tution.      Other salaried        visitors,     according to NXAO offi-
cials,     may be on leave from a particular              university,      perform
work at NRAO, and then return to the university;                      thus they
primarily      benefit    the outside community.           In summary NRA0
officials      believed that their classification              of temporary
staff     as visitors     was proper and that temporary staff               should
be inckladed in the 60 percent of observing time intended for
use by visitors.

     The Kitt Peak National Observatory, which is also oper-
ated by a contractor for NSF, has a 60 to 40 telescope usage

                                        36
policy   similar   to that of JSRAQ. We have been told by a Kitt
Peak official    that all observers who are on the Kitt Peak
payrofl 9 whether for a short term OF a long term, are con-
sidered part of the staff,     whereas all observers not on the
payroll. are considered visitors.
Decrease        in average          visitor         use of telescopes
       The number of visitors       using the NRAQ telescopes has
increased   substantially     over the years; however, the amount
of observing time used by them has not increased proportion-
ately.    As a result,    the average use time by visiting    ob-
servers has declined      significantly.
        Our observation    is based on an analysis   of the utiliza-
tion of the four telescope        systems from the time of their
installation     through fiscal    year 1969.  In our analysis     we
considered visitors      to be persons not on NRAQ's payroll       and
classified    temporary employees as staff,
     The number of visiting  scientists  has increased from
two in 1959 to 66 in 1969 and for the past 3 years has ex-
ceeded the number of staff  observers,  as shown below,
NUMBER
70




60




50
                       STAFF



40               q     VISITORS




30




20




10



 0
         1959     60           61    62       63            64       65   66   67   68
                                                   F ISCA   L YEAR


                                                       37
       The   average number of telescope      hours ,used by staff
observers     has varied considerably     during the 1959-69 period.
However 9    since 1962 the number sf hours used by visitors       has
steadily     declined,  as shown  in  the  following   table,

                                     Average number of
                                telescope   hours used by
                Fiscal   year   §taff members Visi%ors

                     1959             68               68
                     1960            344              106
                     1961            840                7
                     1962            350              983
                     1963            604              580
                     1964            311              468
                     1965            458              368
                     1966            525              335
                     1967            201              178
                     1968            406              149
                     1969            287              122



      We discussed our observations        regarding      the declining
trend in visitors'     use of telescopes     with NRA0 officials.
The officials    expressed their    belief    that observing time had
declined for both staff      and visitors,      primarily     as a result
of increased pressure for observing time and a tendency for
larger research groups to collaborate           on the average re-
search project,     However, in response to our proposal,             they
agreed that a study would be useful regarding              visitors'   use
of the telescopes     and that NRA0 would undertake           such a study.

       In addition,    they informed us that NRA0 had begun send-
 ing explanatory    information   on the use of NIUOps facilities
and on how to apply for observing time to all potential
'users of NRAO's telescopes.




                                    38
Proposal,   agency comments,         and
our evaluation

      In a draft of this report,    we indicated       that NRA0 may
not be fully    serving its mission as a national         research cen-
ter primarily    for the benefit  of visiting     scientists      and
proposed that a study be undertaken       of visitors'      use of
PJRUXl"stelessopes,

       In commenting 'on our proposal,        the Director,     NSF, stated
that he did not believe that such a study was necessary at
this time.     He noted that,      as a matter of policy,       NSF, NRAO,
and AU1 had striven       for a 60 to 40 ratio     in the allocation
of observing    time between visitors       and permanent staff,         re-
spectively,    and that the ratio was but an operating            goal
which NSF, NRAO, and AU1 all felt had been both satisfacto-
rily   and substantially       met, He explained     that, as a result
of information     developed during regular       monitoring     by NSF
of NRA0 activities       which included the visitor        program and
telescope usage, NSF believed that NRA0 was fulfilling                 tts
mission as a national        research center.

       He agreed, however, that during the ll-year                period
1959-69 the average telescope           use time for visitors,         com-
pared to average use time for staff,               had declfned,      He at-
tributed     the decline to a number of reasons, such as (1) the
use of receivers        with lower noise temperatures           (background
noise) that permit the same observing program to be completed
in much less time than previously             and (2) the teaming up of
observers on an observing program due to increased pressure
for telescope       time and increased complexity           of observing
preparations       and procedures.      The differential        decline of
telescope      time for each user,      with    the  result   that   permanent
staff    users are assigned more telescope             time  on the average,
according      to the Director,      is caused principally         by perma-
nent staff       undertaki.ng  long-term     programs, whereas visitors
tend to work on problems having shorter                observational     times,

      Regarding     classification       of users, the Director       has in-
formed us that      NSF believes      that NRAQ"s classification        of
temporary staff       as visitors     is appropriate  because:

      1, NRAO's permanent staff            appointments    take into con-
         sideration the staffing            required    to provide the
            iaeeded services  to visitors             and the requirements               to
             support research projects           of    lOTlg--   t2TlTl   COiltiIW.ity   o

        2. Visitors    from other institutions   and students,
           whether they are paid by NRA0 or their       own institu-
           tions,   are generally   concerned with research related
           to specific     projects for which the collection     of
           data at NRA3 may take only a limited      amount of time.

       3. The l-year      appointments     for research associates   do
          not allow sufficient         time for them to contribute     ef-
          fectively     toward assisting       other visitors in using
          the facilities.

       4. It is not intended that the visitors ftinction  as
          staff  or even as a complement to NRAO"s permanent
          staff.

      Although temporary          staff   may generally   be concerned
with research related    to        projects   requiring   only a limited
amount of time, we noted           that several l-year     appointments
to 'temporary staff  were         extended to 2 years and that,       in
some cases, the principal            research effort    of temporary    staff
complemented the work of           MUD's permanent staff,

        Moreover, WI's        policy manual and NRAO"s Appointment
Policy for the Scientific             Staff,      which have been reviewed
and approved by NSF, disi~-inguish between salaried                    and non-
salaried    visitors.       These    yol.icl'.es provide      that salaried       ap-
pointments     to visitors,       which      itlclude scientists     classified
by NRA0 as temporary staff,              be given to individuals         who take
leave from their       home institutions            primarily    to assist    in
advancing NRAQ's own pragram.                 Nonsalaried     appointments      to
visitors,     according     to these policies,           are made to scientists
whs come to NRA0 primarily             to carry out i;heir own research
proj ects e

      Therefore?    in our view, it would not seem appropriate
to classify,     in all cases, snli3ried tempe,rary staff         as vis-
itors  for purposes of deterlzining        compliance with the 60 to
40 user policy,      Furthermore,    salaried     appointments    made by
NRA0 to temporary staff      whose prj.mary effort       is to carry out
their  own research projects      rather    i;j;ln to assist   in advanc-
ing NRAOss own program appears to be ir,consistent             with the
appointment   policiesof AU1 and NRA0 and may indicate    the
need for NSF to examine into NRAO"s practices   of hiring    tem-
porary staff,

RIXOMMENDATIONTO THE DIRECTOR CF NSF

      We recommend that the Director,         in cooperation   with the
board of trustees   of AUI, make the necessary arrangements
for undertaking   a study of visitors'        use of NRA09 telescopes
and of NRAO's practices      of hiring     temporary staff   to deter-
mine what action,   if any, is needed to ensure that NRA0 fully
serves its mission as a national         research center primarily
for the benefit   of visiting     scientists.




                                   41
                              CHAPTER 6

                           SCOPE OF REVIEW

        Our review was directed       toward an evaluation     of the
policies,     procedures,    and practices     followed   by NSF and AU1
in the administration        of NRAO. Our review was conducted at
NSF headquarters       in Washington,    D.C,; NIUOss a inistrative
headquarters      at Charlottesville,      Virginia;    and NRAOss oper-
ational    site at Green Bank, West Virginia.

     We reviewed pertinent  contracts,    files,  and other rec-
ords of NSF and NRA0 for fiscal   years '8959-69.    APso, we
had discussions  with NSF and NRA0 officials     concerning var-
ious aspects of the NRA0 operation.




                                                                           \




                                  32
         APPENDIX
        I_---




/




i
    A




            43
                                                                                                           APPENDIX I


                                         NATIONAL             SCIENCE         IFOUNDATION
                                                    OFFICE      OF THE DIRECTOR
                                                       WASHINGTON,        D.C.    20550


                                                                                  DEC 18 1970


      Mr. Lloyd G. Smith
      Associate     Director,     Civil   Division
      United    States    General    Accounting                  Office
      Washington,      D. C. 20548

      Dear    Mr.     Smith:

      This is in response                to your letter            dated October         15, 1970, requestina
      comments on the GAO Draft                    Report       entitled       "Opportunities           to Improve
      Administration            of the Research             Program at the National                  Radio
      Astronomy        Observatory,          Green Bank, Nest              Virginia."           The report
      has been reviewed              by representatives                of the Foundation,             Associated
      Universities.           Inc.,      and NMO.          As a result          of this       review,       questions
      have been raised              with    respect       to some of the information                    presented
      and exceptions            have been taken to some of the conclusions                                 and
      recommendations             of the General            Accounting         Office.        The exceptions
      center      around      the     fundamental         difficulty         in evaluation           of research
      management         and related          questions         which     do not readily           lend themselves
      to letter        responses         such as this.              Therefore,        we believe         that before
      a final       report      is issued        it might be in the best interest                          of GAO and
      the National          Science        Foundation          to discuss        the draft       report       and this
      letter reply          in more detail            concerning          areas in which            there     does not
      appear      to be sufficient             understandinq.             [See GAO note 1, pa 50.1
-I1
      _Procedures
        - _.-----         for Allocatinm
                         ____  __~.._ - -.--.'.-- Telescope         Observ-i-n;--T-&T:e
                                                   . - _ - _ _ A-- _.__ -.-               - (Chapter
                                                                                        _-._-- -- -----~. 3)

      Review
      _-            by_____
                         Independent
                                --..-_         Referees
                                         _- .----                (Panes -~-----
                                                  - -.... -----.-CL-       10-14) --      [17   to   211

      GAO suggested         that  improvements    could be made in tk          referee      system
      by (1) establishing          a pool of referees        to obtain    a wider     range of
  I   views,    (2) solicitin::       referees'   evaluations       of updated     requests
      involving     significant       additional    observing:    time,   and (3) expandin?
      the rating      information       bein? requested      from referees.

      Concerning      the recommendation          to establish         a pool of referees,               :XXJ
      has taken steps         to improve      the conditions         existin::         in fiscal      years
      1968 and 1969 as described              in the repot-t.          For esample,           IG?L40 now
      consfstently       receives     reviews     from three       regular         referees,       and
      ‘appropriate      members of the resident           staff      provide         expert     advise
      whenever     there    are differences         of opinion       amon        the referees         whu
      are, and should         be, in an advisory         capacity.             In ad;liticn,       l:RAG
now has two additional               referees      for solar        proposals,       an area that      is
outside      the field        of two of the three            regular      referees.       There is,
in our opinion,           a distinct        advantage      in keepin:         the number of referees
relatively       small,       since    the reviewers         are better         able to advise       MRAO
when they are able to see nearly                     all,    rather     than a small        fraction
of the proposals.              Accordingly,        we are opposed           to the establishment
of a large       pool     of referees.          However,       we will      consider    with NRA0
the desirability            of appropriate         additions        to the pool of reEerees,
particularly         in specialized          areas of research            such as has been done
for proposals          involving       solar    studies.

We agree that updated        proposals     requesting      significant        airlounts   of
additional     observing   time should       be reviewed      by the referees.            NRA0
follows    the policy    of resubmitting        to referees       updated     proposals
in cases where the scope of the wor k has significantly                       changed.       The
decision     to resubmit   is one that       requires    scientific        judgment     in each
case, and if the judgment          of the referees       regarding        a particular
request    is known through      prior   comsxents    of the referees,            the update{;
proposal     may not be resubmitted.

We agree that narrative           comments by reviewers         on the scientific                     merits
of proposed    research      projects       are valuable   supplements      to the                basic
ratinr:  data.    Although      XRAO considers        the comments     by referees                  to be
adequate    at present,      this     does not preclude      the consideration                    of
means to encourage        expansion       of such data.

Approval      and   Schedulin? A .-____
                                   of Research
                                        - _ ____ -.-- Studies
                                                      -- -- -.- .-- (Panes
                                                                     - - .v--- -15-17)
                                                                                 - --- --   [22   and 231

GAO suggested       the need for more adequate                   documentation       of MAO's
management     actions    regarding       the evaluation               and approval    of research
proposals.      In particular,         examples    were          cited    of adjustments     in the
observing    time scheduled         for individual             projects,       and the absence     of
the original      rating    sheets     submitted    by         referees.

Although     opinions      differ     as to what constitutes            “formal       and complete
records,"      we agree that opportunities              for improvement           in this        area
may exist.        However,      NP,AO does maintain        a record       showing,        for each
pronosal,      the telescope        time requested,        MRAO's approval            or disapproval
of the request,        and the amount of observing               time granted,            including
the dates of telescope            use.    Also,    data from the referees'                  original
rating    sheets    are transcribed        on the proposal.            We agree         that      there
                                                                                                               A
is need for improvement             in documentation.          We will       review       with AU1
and NRA0 appropriate            means of accomplishing           the report         recommendation
for more adequate          documentation,       including      retention        of the referees'
original     ratins    sheets.




                                                      46
                                                                                 APPENDIXI

        Evaluation
        -._-         of Levels   of Research
                                        _____Efforts   - (Chgter
                                               I_~__-.---            4) (Pages
                                                              ----------        18-21)
                                                                            - ----- --- [26 to 291
        GAO suggested that the information        available on the levels of research
        efforts     at NRA0 should be improved, and recommended that NRA0 "make
        such improvements to its records as may be most useful to AUI review
        committees and NRA0 management." GAO expressed the belief             that
        expanded information       could be useful in connection with possible
        changes in research emphasis.         NRA0 management has reservations
        about the need to expand the present information         systems, believing
        that scientific      management should seek to foster and encourage the
        very best proposals for telescope time that are submitted to it by
        scientists,     but it should rarely attempt to manage the scheduling
        by research area. NRAO's management also believes that, as long as
        the visitors      and staff that use the facilities   are first-rate,      the
        proposals submitted will indicate where the interesting           research
        areas lie, and the proportion of time spent observing in various
        research areas should be determined through the process of selecting
        the best proposals rather than through the process of trying to
        promote research in specific        areas. We agree in general with this
        philosophy;     however, we believe that the potential     usefulness of
        additional     information   merits consideration,  and we will explore
        with NRA0 the possibility       of accumulating data in such a manner that
        the time and effort involved will not outweigh its usefulness.

        Use of Telescopes
        --                   by Visitors   and ____
                                               StaffI_----.
                                                        - (Ch*er~._ 5)------
                                                                          - (Pages-------
                                                                                     22-23) [30]

        The report notes that NRA0 was established          as a national research
        center to be used primarily     by visiting     scientists.     It also was
        noted that NRAO, AUI, and NSF, as a matter of policy,             have striven
        for 60/40 ratio between visitors       and resident staff,      respectively,
        in scheduling observing time.       The GAO questions the allocation          of
        observing time between resident staff and visitors            because (1) NRA0
‘t      classifies    its temporary employees as visitors         and, consequently,
        this increases the percentage of observing time allocated             to visitors
        and (2) the average telescope time used per visitor            during an 11-year
        period has declined significantly       as compared to average time used
    I   per staff observer.     The GAO also observed that there has been only
        minimal    use of the 36-foot telescope     located at the Kitt Peak ?!ational
        Observatory because of problems connected with atmospheric conditions.

I       It was recommended that the Director of NSF, in cooperation with the
        Board of Trustees of AUI, make the necessary arran:;ements for under-
I       taking a study of visitors'  use of NRAO's telescopes to determine
        what action, if any, is needed so that NRA0 will fully serve its
        mission as a national research center primarily   for the benefit of
        visiting  scientists.




                                                   47
The Foundation        agrees   that as a matter     of policy  NSF, NRAO, and AU1
have striven       for a 60/40 ratio       in the allocation    of observing     time
between   visitors       and resident    staff,  respectively.     The ratio,      as I
am sure GAO realizes,          is but an operating      goal and one that NSF,
NPAO, and AU1 all feel           has been both satisfactorily      and substantially
met.

As indicated             by the Foundation's                 letter      to the GAO dated               September        4,
1970,      the Foundation             believes         that NRAO's classification                       of temporary
staff      as visitors            is appropriate.                NRAO's permanent              staff      appointments
take into          consideration            the staffing            required       to provide           the needed
services         to visitors,           and the requirements                  to support           research        projects
of long-term             continuity.            Visitors         from other        institutions            and students,
whether        they are paid by NRA0 or their                          own institutions,               are generally
concerned         with research             related        to specific          projects         for which         the
collection           of data at NM0 may take only a limited                                  amount of time.
Although         both groups          perform        productive          research,         it is not intended
that     the visitors            function        as staff          or even as a complement                   to NRAO's
permanent          staff.        We also consider                it appropriate            to classify           research
associates           as visitors          since      their       one-year       appointments           do not allow
sufficient           time for them to contribute                       effectively           toward       assisting
other      visitors         in using        the facilities.                For these reasons               we reaffirm
our previous             statement        that     the present           classification              of visitors         is
appropriate            and should        be continued.

As a matter        of interest,          some radio        observatories          have resident
requirements         for visitors.           The visiting          observers       spend several
weeks,     and even months,            becoming      acquainted        with     the radio       telescopes,
electronics,         and data processing             facilities        of the observatory.                For
example,     visitors        using     the new Westerbork             array     in the Netherlands
are required         to spend a minimum of two months in residence                              at the
facility.        This policy         is also pursued            for certain        observing        programs
at the 210' Parkes            telescope        in Australia.           NRA0 has not placed              such
requirements        on its users,          but it has become obvious                  that     users must
become familiar          with their        instruments          if they are going to enjoy
productive       observing        runs.      For first-time           NRA0 users,         this    frequently
means spending          weeks in Green Bank and Charlottesville.                             As larger,
more complex instruments                 come into use NRA0 may find                  it necessary
to impose minimum residence                  requirements          on visitors.

With respect        to the decline         during       the 11-year       period     1959-1969   in
the average       telescope       use time for visitors,               compared      to average    use
time for staff,         we agree that           the average       telescope       time per user
has declined.          The decline       itself       is attributable          to a number of
reasons,     examples      of which are (1) the use of receivers                       with lower
noise    temperatures        that    permit       the same observing           program     to be




                                                                   48
                                                                  APPENDIX I

completed in much less time than previously,        and (2) teaminl: up of
observers on an observing program due to increased pressure for
telescope time and increased complexity of observing preparations
and procedures.       The differential   decline of telescope time ner
user,  with   the   result that   permanent staff users are assigned more
telescope time on the average, is caused principally         by permanent
staff undertaking long term synoptic and survey observational
programs.     University   users, which make up the bulk of visitors,
tend to preferentially       work on astrophysical  problems having shorter
observational     times.

Concerning the utilization      of the 36-foot telescope which is located
at Kitt Peak National Observatory, the telescope is relatively       new
and it is well established      that a new major instrument does not go
into full use for many months due to de-bugging and calibration
requirements.     In the past, extensive daytime usage has not been
possible due to thermal distortions      caused by telescope exposure
to full sunlight.     These  limitations   have been practically overcome
and daytime usage is expected to expand significantly.

We have reviewed the observing time of the 36-foot telescope for
the months of September, October, and November 1970 to determine
current utilization.   Actual observing time, as a percentage of
maximum possible observing time in a 24-hour period, was 72%, 53%,
and 61%, respectively,  of which use by visitors averaged 56%. It
is expected that actual observing time will continue at a comparable
rate in the months ahead.

In response to the recommendation to undertake a study of visitors'
use of XRAO's telescopes to determine what action, if any, is needed
so that NRA0 will fully serve its mission as a national research
center primarily    for the benefit of visiting    scientists,     we do not
consider that such a study is necessary at this time.            Under our
present organization,     we have an Office of National Centers and
Facilities  under the Assistant Director      for National and International
Programs which, together with the Astronomy Section under the
Assistant Director     for Research, monitors activities       at NRAO. Also,
each year NRA0 presents a program review to the Foundation's            senior
staff at which time activities      such as the visitor    program and
telescope usage are subject to review.        As a result of the infor-
mation developed from the review activities       indicated,     we believe
that NM0 is fulfilling      its mission as a national research center.



                         [See GAO note       2, p. 50.1




                                      49
APPENDIX    I




                               [See GAO note        2.1

We appreciate the opportunity    to comment on the draft report.    As
indicated,   representatives  of the Foundation, AUI, and NRA0 will
be pleased to meet with you to discuss the draft report or any
question you may have concerning this letter.
                                    Sincerely     yours,




                                       Director

Enclosure

GAO notes:
  1. The views expressed during discussion         held with NSF
     officials       on the draft of this report and NSF's letter
     reply have been considered        in the preparation   of our
     final     report.

  2. Deleted comments refer to material     contained           in draft
     report but omitted from final  report.




                                                                U.S.   GAO,   Wash.,   D.C.
                                       50                                                     i