oversight

Criteria Needed for Measuring Technical Assistance Results and Contractor Performance

Published by the Government Accountability Office on 1971-10-20.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

Criteria Needed For Measuring
Technical Assistance Results
And Contractor Performance                Et-160789




Agency for International   Development
Department   of State




UNITED STATES
GENERAL ACCOUNTlNG                       OFFICE
                                        UNITED STATES GENERAL ACCOUNTING OFFICE
                                                   WASHINGTON.      D.C.    20548


iNTERNATIONAL      OlVlSION




         B-l     60789



          Dear     Dr.        Hannah:      “-i-;

                 This report   highlights     some of the accomplishments          and problems
         observed    during our review       of the Agency’s    use of contractors      on tech-
         nical assistance    projects     in Kenya, Tanzania,     and Uganda.       These obser-
         vations   are summarized        in the digest included    in this report.

                A draft of this report  was submitted    to your Agency for comment.
         The response,    included  as an appendix,   was considered   in the prepara-
         tion of this report.

                   In the draft         we proposed     that AID:

                   1. Take appropriate       action to require  in actual practice       that the
                      project  planning    and programming      documentation       and the tech-
                      nical service    contracts    contain more specific     delineations      of
                      scope of work, goals, and objectives        and quantitative      indicators
                      by which progress       may be evaluated.

                   2. Institute         a procedure     for   measuring       on a continuing      basis

                         (a) the seriousness     and causes of contract                staffing  problems,
                             including   the identification     of contractors             who fail to pro-
                             vide staff on a timely       basis and

                         (b) the full significance       of the nonavailability          for    training   of
                             host country     nationals.

                 We have been advised that, after our fieldwork,        the Agency has
        revised    its technical    assistance    evaluation system and that “AID has
        expended a sizeable       amount of effort in reorganizing,     planning,      and es-
        tablishing    documentation       systems   in order to correct deficiencies.”
        We do not know what effect these changes have had in actual practice.




                                               50 TH ANNIVERSARY           1921-1971
           ’ B-160739




                    We recommend     that you obtain an independent     evaluation of the
            current   practices  and procedures    and determine   the extent to which
            the types of weaknesses      pointed up in this report  have been overcome
            in actual practice.

                    Your attention   is invited   to section 236 of the Legislative          Reorga-
           nization    Act of 1970 which requires         that you submit written       statements
           of the action taken with respect          to the recommendation.       The statements
           are to be sent to the House and Senate Committees                on Government        Op-
           erations    not later than 60 days after the date of this report            and to the
           House and Senate Committees            on Appropriations      in connection      with the
           first   request  for appropriations        submitted   by your Agency more than
           60 days after the date of this report.            We would appreciate      receiving
/;.   --   copies of all statements       submitted.
1,
                  Copies of this report       are being sent today to the above committees;
           the Subcommittee     on Foreign       Operations,     Senate Committee         on Appropri-
           ations; the Foreign     Operations     and Government          Information     Subcommit-
           tee, House Committee       on Government          Operations;      the Secretary    of
           State; and the Director,     Office of Management             and Budget.

                   We wish to acknowledge         the cooperation        extended    to our   represen-
           tatives  during the review.

                                                             Sincerely      yours,




                                                             Oye V. Stovall
                                                             Director

           The Honorable      John A. Hannah
           Administrator,     Agency for
             International    Development



                                                         2
;       . GENEML ACCOUNTING OFFICE                       CRITERIA NEEDED FOR MEASURING TECHNICAL
I         REFORT-TO THE ADMINISTRATOR                    ASSISTANCE RESULTS AND CONTRACTOR
I
I         AGENCY FOR INTERNATIONAL                       PERFORMANCE
I         DE.VELOFMENT                                   Agency for International  Development
I                                                        Department of State B-160789
I
I
I        DIGEST
         ------
I
I
I        WHY THE REVIEW WAS MADE
I
I
I              Contracts   are an important   instrument      of the Agency for International
I              Development (AID) in providing       technical    assistance.    As of December 31,
I
I              1970, AID had 1,191 contracts       outstanding     totaling  about $679 million
I              for technical   assistance   in 58 countries.
I
I
I              Because of the dollar     amounts expended and the extensive      use of this
I              means of providing    technical  assistance,     the General Accounting  Office
I              (GAO) examined AID’s use of technical        service  contracts for 10 agricul-
I              tural  projects  in Kenya, Tanzania,     and Uganda.
I                 ---*--.   -- -

               In carrying       out technical  assistance      projects,    AID contracts    with educa-
               tional   institutions      and qualified    firms,    associations,     and individuals.
               U.S. universities        have been among AID’s primary sources.


         FINDINGS AND CONCLUSIONS

                Neither AID’s project     planning    and programming documents nor the contracts
                themselves  provided   specific    targets,   goals,   or objectives.       AID therefore
                could not evaluate    progress    and contractor     performance    objectively.

                Notwithstanding      the absence of objective   measurement criteria,  some positive
                and significant      results  have been achieved through AID’s technical   service
                contracting.       For example:

                   --AID was instrumental      in introducing    vocational    agricultural     education
                      in Kenya's secondary     schools.     (See p. 16.)
    I              --AID assisted     the Government of Uganda to build        and expand a strong and
    I
    I                 independent   cooperative     movement, particularly     in the dairy industry
    I                 and in livestock    ranching.     (See p. 16.)
    I
    I           Various problems,  however, have hindered the effectiveness      of AID’s efforts.
    I
    I
                There were delays in recruiting  and placing  contract    employees in-country
    I           and some employees appeared to lack desired qualifications.         (See p. 19.)
    I
    I
    I           Other problems limiting     the effectiveness      of technical   assistance     were re-
    I
    I
                lated to host country    actions.      For example, preparing     host country      na-
    I           tionals  as replacements   for contract      employees is an important       objective
    I           of the projects.    This objective      was not fully    achieved because host
    I

    I
    I
    I
    I
          Tear Sheet                                                           -m-.20,1971
    countries had not provided enough qualified             local   nationals   for   training:               .
    (See p. 21.)  Other problems were:                                                            r
                                                                                                          L
       --The objectives   agreed upon for        one project  were not achieved because               .
          the host government unilaterally         changed the project.    (See p. 26.)

       --Projects    in one country were delayed because of the host government's
          delays in issuing    temporary employment passes which were necessary for
          AID contract   employees to work in that country.    (See p. 27.)

       --AID had difficulty      in evaluating       the performance    of its contract
          employees at the executive          level and in ensuring     that they were used to
          fill  only essential     positions,      because of a lack    of information   on
          the activities    of these contract        employees within     the host government.
          (See p. 27.)

    AID has worked, especially     since 1967, to improve its technical        assis-
    tance evaluation    system and to overcome other problems hindering         the ef-
    fectiveness   of its efforts.     (See p. 31.)   AID said that,  since GAO's
    fieldwork,  several    other changes have been made, including     further     im-
    provements in the technical      assistance evaluation   system.


RECOMMENDATION
    The Administrator,   AID, should obtain an independent    evaluation of AID's
    current   practices and procedures  and determine  the extent to which the                                    I
    types of weaknesses pointed    up in this report  have been overcome in actual
    practice.


AGENCYACTIONSAND UNRESOLVED
                          ISSUES
    The Assistant     Administrator   for Africa    recognized     the need to ensure that
    the requirements     for measurement criteria       were incorporated        into both the
    project   planning   and programming documents and the technical               service  con-
    tracts.    GAO was advised that improved techniques            and procedures       were being
    utilized   for new projects.      The Assistant     Administrator       agreed with GAO
    on the need to deal with the problems of contract              staffing     and inadequate
    numbers of local nationals      for training.

    The extent to which these        matters   have been corrected       in actual    practice
    has not been determined.




                                                                                                                  I

                                                                                                                  I
                         Contents
                                                             Page

DIGEST                                                         1

CHAPTER
  1       INTRODUCTION                                         3

  2       MEASUREMENT  OF CONTRACTORPERFORMANCE  AND
          PROJECTRESULTS                                       6
             Higher Agricultural  Education project            7
             Crop and Livestock  Development Extension
               project                                        11
             Range Development project                        15
             Other projects                                   16

  3       DIFFICULTIES IN FILLING CONTRACTPOSITIONS           19

  4       PROBLEMSOF HOST COUNTRYSUPPORT                      21
             Training of host country nationals               21
                  Not enough candidates selected for
                      training                                22
                   International      transportation
                      expenses                                24
                   Dissolution      of the University of
                      East Africa                             24
                  Assignment of trained participants          25
             Unilateral      decisions by host government
               ministries                                     26
             Temporary employment passes to contract
               employees                                      27
             Use of contract        employees in host
               country positions                              27

  5       ACTIONS TO IMPROVETECHNICAL ASSISTANCE
          EFFECTIVENESS                                       31

  6       CONCLUSIONSAND RECOMMENDATION                       41
             Recommendation to the Administrator,      AID    42

  7       SUMMARY
                OF AID COMMENTS                               44
APPENDIX                                                   Pane
      I    Letter dated May 7, 1971, from the Auditor
             General, Agency for International  Develop-
             ment, to the General Accounting Office         47
                            ABBREVIATIONS
AID       Agency for   International      Development
GAO       General Accounting     Office
 GENER$L ACCOUNTING OFFICE                      CRITERIA NEEDED FOR MEASURING
                                                                            TECHNICAL
*REPORT 70 THE ADMINISTRATOR                    ASSISTANCERESULTS AND CONTRACTOR
 AGENCY FOR INTERNATIONAL                       PERFORMANCE
 DEVELOPMENT                                    Agency for International  Development
                                                Department of State B-160789


 DIGEST
-_--_-

WHY THE REVIEW WAS M4DE

     Contracts   are an important   instrument      of the Agency for International
     Development (AID) in providing       technical    assistance.    As of December 31,
     1970, AID had 1,191 contracts       outstanding     totaling  about $679 million
     for technical   assistance   in 58 countries.

     Because of the dollar         amounts expended and the extensive      use of this
     means of providing        technical  assistance,     the General Accounting  Office
      (GAO) examined      AID's use of technical      service  contracts for 10 agricul-
      tural projects      in Kenya, Tanzania,     and Uganda.

      In carrying       out technical  assistance      projects,    AID contracts with educa-
      tional   institutions      and qualified    firms,    associations,    and individuals.
      U.S. universities        have been among AID's primary sources.


FINDINGS AND CONCLUSIONS

      Neither AID's project   planning    and programming documents nor the contracts
      themselves  provided  specific   targets,   goals,   or objectives.       AID therefore
      could not evaluate progress and contractor         performance    objectively.

      Notwithstanding       the absence of objective   measurement criteria,  some positive
      and significant       results  have been achieved through AID’s technical   service
      contracting.        For example:

          --AID was instrumental      in introducing    vocational   agricultural   education
             in Kenya's   secondary   schools.     (See p. 16.)

          --AID assisted     the Government of Uganda to build       and expand a strong and
             independent   cooperative     movement, particularly    in the,dairy industry
             and in livestock    ranching.     (See p. 16.)

     Various problems,  however, have hindered the effectiveness     of AID’s efforts.
     There were delays in recruiting  and placing contract    employees in-country
     and some employees appeared to lack desired qualifications.       (See p. 19.)

     Other     problems    limiting
                                 the effectiveness      of technical   assistance     were re-
      lated to host country   actions.      For example, preparing     host country      na-
      tionals as replacements   for contract      employees is an important       objective
      of the projects.   This objective      was not fully    achieved because host



                                                   1
    countries  had not provided  enough qualified           local   nationals   for   training.
    (See p. 21.)   Other problems were:                                                        ,
                                                                                                      1)
      --The objectives   agreed upon for        one project  were not achieved because
         the host government unilaterally         changed the project.    (See p. 26.)               *_

      --Projects    in one country were delayed because of the host government's
         delays in issuing    temporary employment passes which were necessary for
         AID contract   employees to work in that country.    (See p. 27.)

      --AID had difficulty      in evaluating       the performance     of its contract
         employees at the executive          level and in ensuring      that they were used to
         fill  only essential     positions,      because of a lack     of information   on
         the activities    of these contract        employees within      the host government.
         (See p. 27.)

    AID has worked, especially      since 1967, to improve its technical        assis-
    tance evaluation     system and to overcome other problems hindering         the ef-
    fectiveness    of its efforts.     (See p. 31.)   AID said that,  since GAO's
    fieldwork,  several     other changes have been made, including     further     im-
    provements   in the technical     assistance evaluation   system.


RECOMMENDATION
    The Administrator,   AID, should obtain an independent   evaluation of AID's
    current   practices and procedures and determine  the extent to which the
    types of weaknesses pointed up in this report    have been overcome in actual
    practice.


AGENCYACTIONSAND UNRESOLVED
                          ISSUES
    The Assistant     Administrator   for Africa    recognized     the need to ensure that
    the requirements     for measurement criteria       were incorporated        into both the
    project   planning   and programming documents and the technical               service  con-
    tracts.    GAO was advised that improved techniques            and procedures       were being
    utilized   for new projects.      The Assistant     Administrator       agreed with GAO
    on the need to deal with the problems of contract              staffing     and inadequate
    numbers of local nationals      for training.

    The extent to which these       matters    have been corrected       in actual    practice
    has not been determined.




                                                2
                                 CHAPTER1

                               INTRODUCTION

       Technical assistance        to less developed countries            covers
the full range of planning,            analytical,       educational,    advi-
sory, and other activities           traditionally         devoted to the en-
hancement of human resources and institutional                    development.
The Agency for International            Development provides technical
assistance,    under the foreign aid program, through technical
service contracts       with educational         institutions      and techni-
cally qualified      firms, associations,           and individuals.        In
many instances such contracts            make up a significant          part of
technical    assistance     projects.
      As of December 31, 1970, AID had 1,191 technical         ser-
vice contracts   totaling  about $679 million    for work in 58
countries.    About 133 of these contracts,    totaling   about
$105 million,   were for technical  assistance     to 19 African
countries.

       Because of the millions        of dollars   of foreign aid
funds expended by AID through technical            service contracting,
the General Accounting Office examined- into AID's use of
technical   service contracts       to implement technical       assis-
tance projects.     Our review was complicated           by the large
number of small dollar amount contracts,            the diversity     of
specific   contract   purpose, and the wide geographic disper-
sion of contract work. After considering              these factors we
reviewed 10 projects,       primarily    for education in the agri-
cultural   sector, in the east African countries            of Kenya,
Tanzania and Uganda. AID's assistance             to these countries
which constitute    the East African Community is heavily con-
centrated   in agriculture,      education,     and the development of
community infrastructure        and services.
       The projects   reviewed   and AID's    expenditures    were as
follows:                                                                        .

                                             AID expenditures    as
                                              of March 31, 1971
                                                              Contract
                                         Project               portion
                                                   (000 omitted)

Kenya:
    Crop and Livestock      Extension    $ 2,313                   $     319
    Higher Agricultural      Educa-
       tion                                  1,480                  1,200
    Nairobi Veterinary      Faculty          1,839                  1,047
    Range Development                           900                    314
    Vocational   Agriculture     Edu-
       cation                                  857                       686
Tanzania:
    Agricultural   College                   1,340                        953
    Public Service Staff Support             1,167                     1,107
Uganda:
    Agricultural   Education                 2,239                  1,674
            11     Cooperatives              2,552                     988
    Makerere Agriculture      Faculty        1,199                  1,079

          Total                          $15,886                   $9,367
     AID started these projects in 1964 or earlier   and some
of them are not expected to be completed until   1974.

      We observed some accomplishments from the projects            and
several problems which hampered contract           and project effec-
tiveness.      The problems identified     in our review appear to
be symptomatic of problems that will require AID's continu-
ing attention.       Our report highlights     some of the accom-
plishments     of these particular    projects    and some of AID's
problems in implementing technical         assistance   projects  and
in using technical      service contracts.

       Project documentation at AID/Washington and at AID's
missions in east Africa was reviewed, and relevant matters
were discussed with AID, contractor,     and host country offi-
cials.     Our work in the three east African countries   was

                                   4
         .


I




        done during April through July 1970, and subsequent       inqui-
    *   ries were made at AID/Washington.

              A draft     of our proposed report was submitted to AID
        for comment     on March 5, 1971. Comments of the Assistant
        Administrator      for the Bureau of Africa were received on
        May 7, 1971,      and have been recognized,  as appropriate, in
        the body of     this report.    A copy of the complete comments
        is included     as the appendix.
                                  CHAPTER2

               MEASUREMENT
                         OF CONTRACTOR
                                     PERFORMANCE

                           AND PROJECTRESULTS

       Clearly defined project and contract              targets,     goals,
and objectives      are essential     to measuring anticipated            proj-
ect results      and evaluating    contractor     performance,         Even
though positive      results were apparent from the technical                 as-
sistance projects       in east Africa,      these results        could not
be objectively-evaluated        nor related to anticipated             results
because of the absence, in the project planning and program-
ming documentation,        of predetermined      targets,     goals, and
objectives     i,n measurable terms.        Just as the project results
could not be\,objectively        evaluated,     neither could the con-
tractors'     performances be objectively          evaluated because the
technical     service contracts      were written      in the same general
terms as the project        documents.
        Our general impression was that some positive                 results
had been achieved through the use of technical                   service con-.
tracts on the 10 projects            that we reviewed.        For example,
AID was instrumental,           through a contract with a U.S. univer-
sity,    in the    introduction      of vocational     agriculture      in
Kenya's secondary schools.              AID also assisted the Government
of Uganda, by providing            advisors under technical        service X
contracts,     to make considerable           progress in building        and
expanding a strong and independent agricultural                   cooperative
movement. There appeared to be positive                 results    from the
projects,     but these results         could not be compared with the
anticipated      results     because of the absence of quantitative
criteria    in the supporting          project documents.
       The project planning and programming documents, for the
most part, were general; they lacked specifics       as to the
targets,    goals, and overall objectives    of the particular
projects.     Therefore there were few bench marks with which
to compare project      results or to measure progress.      These
factors    severely limit the objective   evaluation   of the ef-
fectiveness    with which foreign aid funds are being used to
provide technical      assistance.



                                        6
     * The failure    to clearly  specify in the project documents
the plan of action and anticipated        accomplishments adversely
affected   the implementing technical      service contracts,    The
contracts    contained almost identical     statements of overall
goals and objectives      as those in the project documents.      As
the purpose of these contracts       was to provide technical    as-
sistance,    the lack of measurement criteria      raised the ques-
tion of what the contractors       were to accomplish by their
advice and assistance.

       Our observations     as to some of the positive tangible
results   and some of the measurement problems are highlighted
in the following     sections.

HIGHER AGRICULTURALEDUCATIONPROJECT

      The objective   of this project,    as recorded in the agree-
ment between AID and the Government of Kenya on May 29, 1962,
was to assist Egerton Agricultural       College at Njoro, Kenya,
to strengthen    its resources and capabilities     as an agricul-
tural college providing     training   at the diploma level.

      Specific project activity   goals were listed in three
areas of activity   and were stated in general terms.    These
stated goals were to assist the college in:

            "1. Planning and implementing educational         and
     administrative      policies     aimed at improving in-
     struction    standards;      selecting  fields  of emphasis
     necessary in meeting Kenya's expanding training
     needs relating      to agriculture;     and expanding the
     training    operations     of this college,
           "2. Strengthening       existing     faculty    resources
     and developing     potential     faculty    members to meet
     increasing   training     responsibilities         as the Col-
     lege broadens its curriculum           and expands its stu-
     dent enrollment.

           "3. Obtaining      certain    commodity items nec-
     essary in carrying       out its    education and train-
     ing responsibilities."



                                     7
       The objective   of the technical      service contract,   which
was signed with a U.S. university        on September 25, 1962, was .
the same as the project      objective;    that is, to provide tech-
nical advice and assistance        to Egerton Agricultural     College
to strengthen     its resources and capabilities      as an agricul-
tural college providing      training   at the diploma level.       The
following   examples are illustrative       of the seven advice and
assistance   services to be provided by the contractor.

             "A. Advice and assistance     in the training
      of persons in animal husbandry,        This will in-
      clude advice and assistance      in the development
      and conduct of improved courses in this field,
      Courses will include those given to students tak-
      ing the regular training     provided by the College
      and refresher    courses for field agricultural      ex-
      tension personnel.         6
            "B. Advice and assistance  to the College
      in developing and conducting courses emphasizing
      improved agricultural  extension techniques and
      methods which can be applied in the Cooperating
      Country."

      The original  contract was superseded on June 30, 1965,
by a new contract.    Although it has been amended several
times, the contract   objectives  and the services to be pro-
vided have not been made more specific.     The two services
quoted above from the superseded contract     are also illustra-
tive of the services to be provided under the new contract.

       Both the project     documents and the contracts        lacked
specific   time targets,      anticipated    project   accomplishments
to meet specific       time targets,     or the anticipated    project
completion    situations.      Lacking also were criteria        by which
to determine overall       attainment     of the objective   of increas-
ing agricultural      productivity.

       AID stated that the overall          performance of the higher
agricultural      education project was satisfactory,          even though
anticipated      project    results were not quantified.         An ap-
praisal    report on the project prepared in August 1969 stated
that "Anticipated        results are difficult      to quantify,     Proj-
ect costs are investments          in trained manpower for the agri-
cultural     sector of Kenya."
                                     8
   .
     The results   of this project cannot be compared with
those anticipated   when the project was started, but there
have been positive   results.
        This project helped the college change from a 2-year
institution      to an institution       offering  3-year diploma
courses.      It improved teaching techniques and practical
training     and introduced      the U.S. system of student evalua-
tion.      Contract    team members     developed specialized     fields    of
study leading to diplomas in agricultural              engineering,     range
management, and agricultural            education.    Enrollment in-
creased from about 150 in 1962 to over 600 in 1970, The
college is now reportedly           able to provide qualified       persons
for positions       within the Government of Kenya and for govern-
ment positions       in other African countries.

       From our review of the records,     it appears that, during
the early years of the contract,      there was a lack of direc-
tion or specific    project purpose.    We believe that the prep-
aration and use of a work plan would have been an aid in
more quickly defining      the course of action to be followed by
the contractor.

        The original     contract    for this project required the
contractor     to develop a detailed        work pian within 120 days
after arrival      in-country,      subject to subsequent revisions
as considered necessary by AID or the contractor.                After more
than 1 year in Kenya, the contractor             had not furnished    work
plans.      There was still      no work plan for the project       in the
records at the time of our review.              The lack of such a nec-
essary planning and implementation            document inhibits     the
identification       of problems and makes it relatively         impossible
to monitor progress,          especially   for projects  such as this
where specific       targets,    goals, and objectives    have not been
previously     identified.




                                       9
April 1970. Construction of a new laboratory for the agriculture engineering department
at Egerton College. The building was designed by the AID contract team at substantial
savings to the college. Further savings were achieved by the use of student and local labor.




                                             10
CROPAND LIVESTOCK DEVELOPMENT
EXTENSIONPROJECT
       The Crop and Livestock         Development project was started
about 1955, but AID advisors did not participate                until     1962.
The project      consisted of several subprojects,          with the objec-
tive of assisting        the Government of Kenya to move farmers
into the cash economy and to increase and improve farm pro-
ductivity.       A project agreement with the Government of Kenya
on May 9, 1966, for continuation             of the agriculture       extension
element of the project         specified     that AID would provide three
contract rural development area advisors.               These advisors
were to advise the provincial            agriculture  officers     and their
staffs     in fulfilling    the National Agriculture         Development
Plan as it applied to the provinces to which they were as-
signed.

      The three advisors were provided to the Government of
Kenya under an AID contract    of August 1966 with a nonprofit
organization.   The contract   objective   was for the contractor
to provide the Government with technical      advice and assis-
tance in its efforts    to move the African farmer into the cash
economy and to increase and improve farm productivity       by:

      --Improving  the effectiveness            of the African   farmer
         through advanced technical            advisor services.

      --Establishing      sound agricultural    institutions that were
         interrelated     and directed    toward social and economic
         development.
      --Utilizing   the pastoral            areas more efficiently    to in==.
         crease their contribution             to the overall   economy.

      --Developing      land resources        and better   methods of culti-
         vation.

      m-Promoting greater utilization of the inland lakes and
        the sea as sources of food and export earnings.

      The contract advisors'   primary responsibilities       were to
assist in fulfilling   the National Agricultural       Development
Plan as it applied in the provinces where they were assigned.


                                       11
These responsibilities           were stated to entail working with,
advising,  and assisting          Kenyan officials  in six areas, such            *
as:
      "Working with the Provincial   Agricultural     Officer
      in developing a more efficient   overall    rural de-
      velopment program and obtaining    the cooperation
      of all associated government agencies in the
      province on the implementation   of such program."
                *            *              *          *


      "Advising and assisting     the district officials
      and agricultural    workers on methods and tech-
      niques of implementing improved agricultural
      practice   demonstrations   such as the growing of
      maize, sugar cane, sisal, peanuts, forage crops
      and livestock    improvement."

        Positive    tangible  results were achieved by one of the
advisors in irrigation         activities      and in pollination        by use
of bees.       The  advisor  planned      and  promoted  an   irrigation
scheme at one village        in the Coast Province of Kenya whereby
two rice crops a year could be produced, using water pumped
from the Tana River.         The village       council agreed to assume
responsibility        for the project,      to furnish    funds to develop
the fields,       and to provide money for the maintenance and op-
eration of the pump and engine supplied by AID under a self-
help program.        The advisor supervised surveying the fields,
installing       the pump and engine, preparing         the seed beds and
plant nursery,       and transplanting        the rice shoots to the
fields.
        Without irrigation,      the farmers had harvested the
equivalent     of one and a half crops during the previous
5-year period and the community had been frequently             on famine
relief.      The advisor estimated that the irrigation        system
now provided the capability          to harvest 10 crops in a 5-year
period.      The success of the project encouraged two other
villages     on the Tana River to take steps to establish          irriga-
tion systems, with the advisor acting only as a consultant
to the Government of Kenya's district           agriculture  officer.
Thus the first      irrigation    project   demonstrated that increased
crop production       was possible and provided the African farmer

                                       12
                                    ---.




                                               Credit AID rural development area advisor

September 1968. Just moments after the engine was started for the first time, villagers in
the Oda settlement express their pleasure with the pump, supplied by AID, which they
helped to install and which will bring water from the nearby Tana River to their rice fields.
This irrigation project was planned, organized, and implemented by the AID agricultural
advisor assigned to the area’s provincial agriculture office.

with an incentive           to raise his standard   of living                  by   chang-
ing and improving           his methods of farming.

       Soon after his arrival   the advisor observed that the
production   of mangoes, cashew nuts, and other crops requiring
pollination    was very low, considering   the size and condition
of the trees.     After a study revealed that few bees lived in
the area, the advisor embarked on a program to establish
beekeeping as an active project.
      Hives, equipment,             and bees have been exhibited at local
fairs and agricultural              shows to impress the local people with

                                               13
the importance of beekeeping.     Lectures on modern beekeeping
methods have been given at mleetings of young farmer groups,                .
at farmer-training  centers'  classes, and at seminars for
secondary school science teachers.

       The advisor,     who currently     has 35 swarms of bees at
various locations       in the coastal area, stated that the pro-
duction of certain crops in these areas had increased,              Cer-
tain government officials         feel the work to date justifies
further     study, and the head of the Animal Husbandry Division
of the Kenya Ministry         of Agriculture    has expressed an inter-
est in establishing        an apiculture     section within the Minis-
try.    A third country has discussed with the Government the
possibility      of providing    funds and training     for the new sec-
tion.
      A June 1969 AID appraisal report stated that overall
performance of the project had been satisfactory     in advancing
the project   goals. The August 1969 appraisal   report,  how-
ever, stated:
      "No reliable    data are available      to indicate   spe-
      cifically    the extent to which the agricultural
      extension service has contributed          to the increase
      in crop production,       but it is fairly    evident that
      if the projected      increase in output is achieved,
      a very substantial       credit must go to the agricul-
       tural extension    service for its part in this ef-
      fort."    (underscoring      supplied.)

      Thus, while this project manifested       some positive      tangi-
ble results,   it also lacked specifics     in the project and con-
tract documents by which to gauge project progress,           contrac-
tor performance and attainment       of the higher overall     objec-
tive of increasing    farm productivity    and moving the farmer
into the cash economy. We believe that this project empha-
sizes the problem and the need for more definitive          criteria
for evaluating   project  effectiveness    and contractor     perfor-
mance.




                                   14
.
    RANGEDEVELOPMENT
                   PROJECT

          The general objective    of the Range Development project
    was to assist the Government of Kenya to develop its vast
    range resources,    a high priority  in Kenya's development plan.
    The stated specific    goals comprised four general areas in
    which technical   advice and assistance   were to be provided.

           Among the AID contributions    were three range management
    advisors.      The goals and duties of these advisors were iden-
    tified    in the implementing technical   service contract only
    as general areas in which the advisors were to provide ad-
    vice and assistance.      Examples of these goals and duties
    were:
          "Assist in the development of the Range Management
          Division  of the Ministry   [of Agriculture     and An-
          imal Husbandry] and provide technical       advice to
          the staff of this Division.

          "Provide technical   advice and assistance       in estab-
          lishing  local grazing associations.*'
              *           *            *           *           *


          "Assist in developing a grassland range manage-
          ment and bush control program that will provide
          the necessary range area for increasing the live-
          stock production  program."

            One of the advisors said that there was very little           re-
    semblance between the job described to him by the organiza-
    tion with which AID contracted       and the job he was actually
    performing.      Another  advisor  said  that his instructions      were
    simply to assist the Government of Kenya range officer.               The
    third advisor said that the job description            given to him was
    vague and incomplete.       As a result the advisors had diffi-
    culties    in determining   the specific    activities     toward which
    to direct their efforts.
          The scope of services in the contract was cited by the
    advisors as being particularly  unrealistic.    The advisors
    were to assist in establishing   about 2,000 ranch units and
    grazing schemes in three provinces    in Kenya. This number

                                       15
was believed to be unrealistic      in view of the small number
that had been established,      the poor prospects for establish-  .
ing a very large number in the future,       and the lack of plans
for establishing      a larger number. As of April 1970, 60 ranch
units had been organized.       The work plan for the coming
2-year period called for establishing       an additional  102
units.     If 162 ranch units are organized by April 1972, the
anticipated    project completion date, only 8 percent of the
total projected     ranch units will have been established.
‘OTHER PROJECTS

       There have been positive     tangible   results    on other
projects    even though measurement of project        progress and ef-
fectiveness    was a problem.     For example, through the Voca-
tional Agriculture     Education project,     AID inaugurated      the
teaching of agriculture      in the Kenya secondary schools in
1960 with a pilot project at a school in the Western Prov-
ince.    The apparent success of the pilot project          led to the
award of a university     contract under which instructors          were
provided for six additional       schools.    The project has been
expanded into a program to include 25 schools--seven             spon-
sored by the United States, 13 by the World Bank, and five by
the Government of Kenya. The Government of Kenya plans to
expand this program into additional         schools as funds and
teachers become avaiable and to introduce a vocational              agri-
cultural    course into the primary schools.
       Since 1963 AID has provided,       through the Agricultural
Cooperatives   project,     from three to six contract      staff mem-
bers to assist the Government of Uganda to build and expand
a strong independent cooperative         movement. Project activi-
ties were directed      primarily    to improving management skills
through training    within the cooperatives       and the responsible
government ministry,       assisting   new cooperative   groups in
working with new and diversified         crops, and establishing    re-
lations with credit and marketing organizations.

       The movement has shown considerable      progress.    Since
1963 the number of cooperatives       has increased from 1,700 to
2,500 and membership has increased about 60 percent.           The
project   documents indicate    that cooperatives    are now widely
accepted as important     instruments   in teaching better business,
better farming, and better living at the country's         local so-
cial level.
                                   16
      The dairy industry and livestock     ranches are two areas
where the cooperative   movement has shown progress.       In 1965
there were three cooperative    dairy groups in Uganda market-
ing 150 gallons of milk daily.      In contrast,   19 groups were
marketing 4.,000 gallons of milk daily in 1969. These groups
vary in size from small collection     centers processing    50 to
100 gallons of pasteurized   milk daily,    to one large coopera-
tive processing  over 700 gallons of pasteurized      milk daily.

       This large cooperative  was organized in 1966 and con-
tinues to grow and prosper.       It delivers   fresh milk to all
principal   markets in the district.       Income of the coopera-
tive has averaged over $10,000 a year since it was organized
4 years ago and has been used to construct         a new dairy plant
and to purchase new pasteurization       equipment.    The coopera-
tive serves as a model and demonstrates what can be achieved
by milk producers who own and operate their own businesses.

      The 20 cooperative  livestock    ranches that have been
organized since 1965 in Uganda have increased their cattle
from 6,000 head valued at $350,000 during 1965 to over
12,000 head valued at $850,000 during 1969. On these ranches
were placed 2,400 head of imported purebred females.          Plans
have been made to establish     another 25 cooperative    ranches
during the next 2 years.     The capital    required to finance
these ranches will be supplied by the new ranch members and
by credit obtained through local commercial banks.

       Thus there have been apparent positive       tangible     results
from this project as with other projects,         but the same prob-
lems continue as to measurement of progress toward goals and
overall objectives.      The project  documents and the technical
service contracts    contain very few quantitative        indicators
by which to measure progress or intermediate          targets and
goals or effectiveness      of AID's contribution     to the cooper-
ative effort.




                                  17
       April 1970. The vocational agriculture teacher at Njoro Secondary School, Njoro,
Kenya, admires the young bean crop under cultivation by the students. A substantial por-
tion of the food needed for about 300 boarding students is grown under the vocational ag-
riculture program on the 16-acre school farm.




                                           18
                              CHAPTER3

         DIFFICULTIES IN FILLING CONTRACTPOSITIONS
      Various contract     staffing  problems were common to the
10 projects    and related contracts     that we reviewed.    The
primary problem was delay in placing contract         employees on
the job in-country.       Other, and perhaps related,    problems
were employee turnover and employee qualifications.           The
direct effect of these problems was not readily         apparent in
all instances;    however, it appeared that they had contrib-
uted to delays in achieving project goals and to extensions
of project completion dates.
       An example of delay in placing contract     employees in-
country is found at the Crop and Livestock      project    in Kenya.
On this project   the contractor  did not provide the required
three rural development advisors until      5 months, 6 months,
and 2-l/2 years after the contract     had been signed in 1966.
Another example is found at the Makerere Agriculture         Faculty
project   in Uganda where the contractor's    chief of party ar-
rived 6-l/2 months after the position      was authorized,    and
two of three staff members needed by June 1966 to prepare
for the academic year arrived in June 1967.

       These delays in placing contract    employees on the job
may be due to a combination    of reasons.    Some of the appar-
ent reasons were delays by the contractor       in recruiting the
required employees and delays by a host government in issuing
temporary work permits which were necessary for contract
employees to work in that country.

       AID contracts     usually require contract        employees to
serve at least a 2-year tour.           A contract    technician   needs
about 6 months to become acclimated overseas, according to
AID, and probably another year to demonstrate real, visible
progress in work objectives.           Thus the effectiveness      of con-
tract employees was limited,         particularly     during the early
phases of project      operations,    because they were new on the
job.    Another   factor    limiting   the effectiveness      of contract
 employees was the absence of an overlap of tours between
departing    and arriving      employees to provide continuity        of
 experienced employees on the projects.


                                     19
      On some projects     it appeared that the employees pro-               *
vi&d by the contractor       had less than the desired qualifi-'
cations.    It was not readily     apparent whether this resulted        -
from the absence of specified       minimal education and experi-
ence requirements    in the contracts     or whether employees with
the desired qualifications      could not be obtained.
        AID planning and programming documents have indicated
for example, that certain positions          should be filled   by se-
nior staff members from a given university's          faculty,   but
the contracts      listed only the title    of the positions    to be
filled,    such as lecturer     in animal husbandry, and did not
prescribe    minimal qualifications      for a college lecturer     in
this field.

       Many persons recruited   by a contractor     for teaching
positions   at the agricultural    colleges    in Tanzania and Uganda
had never taught at the college level.          Some had been only
recently   graduated from college,      and others had been teaching
in high schools or working in state agricultural          extension
programs or at research stations.
       The contract  university      for the Agricultural   College
project   in Tanzania employed a chief of party whose main
duties consisted    of college administration        and development,
but whose professional      experience had been limited       to U.S.
county and regional     agricultural     extension work.




                                   20
    1




,       .



                                          CHAPTER4

                           PROBLEMSOF HOST COUNTRYSUPPORT

                  Various difficulties        associated with host country ac-
            tions and project      support have hindered progress on the AID
            technical   assistance     projects   in east Africa.     The most se-
            rious problem was the failure         to provide for training    an
            adequate number of host country nationals           to replace AID
            contract employees and to carry on and expand the institu-
            tions and techniques initially          sponsored under foreign as-
            sistance.

                   Other problems included the adverse effect on AID-
            financed projects     of a host government's unilateral    deci-
            sions, delays by a host government in issuing the employment
            passes required before contract      employees could work in-
            country,    the lack of assurance that a host government used
            AID-financed    executive employees to fill   only essential     posi-
            tions, and the lack of information      on the work performance
            of the AID-financed      executive employees.

            TRAINING OF HOST COUNTRYNATIONALS
                    A principal purpose of technical      assistance    is to pre-
            pare the people of the host country to carry on and expand
            the institutions    and techniques initially       sponsored under
            foreign assistance.      We noted several problems limiting         the
            achievement of this purpose on the PO projects            in Kenya,
            Tanzania, and Uganda, The primary problem was the nonavail-
            ability    to the AID projects   of qualified     nationals   to be
            trained.

                  Other contributing    problems were a prolonged disagree-
            ment between AID and the three governments over payment of
            expenses for the trainees a transportation         to the United
            States and a reduction     in the number of possible       trainees
            for specific  projects    because of the dissolution       of the re-
            gional University     of East Africa.     Another factor limiting
            the achievement of specific      project   objectives    was the as-
            signment of trained nationals       to projects    other than those
            for which they were trained.



                                               21
                                                                         .
Not enough candidates     selected        for   training

      Prior to the agreements with the host governments for
these projects,    neither the missions nor the host govern-
ments determined that an adequate number of participants
would be available     for training and for replacing AID con-
tract employees.     Only after many of the projects  were well
under way was it apparent that not enough participants     would
be trained within the period required by the project     sched-
ules.

      The Vocational      Agriculture   Education project     in Kenya
is one example of the participant          training  problem.    When
this project was started in 1964, AID stated that it would
be necessary to train locally         an adequate number of teachers
to support future expansion of the program.             In spite of
this recognition     of need, training       plans were not made until
late in 1968 when the AID Mission in Kenya received a re-
quest from the Government of Kenya for a crash program to
train 30 agriculturalists         as secondary teachers.

       The program called for the training    of two groups of 15
students each for 22 weeks to meet the expanding vocational
agricultural    education program.   Training was to start in
January 1969 for the first     group and in June 1969 for the
second group.

      Shortly before the training   program was to start,      the
Government of Kenya advised the Mission that the Government
could provide only 10 candidates for the January class.           In
commenting on the first    group of potential   teachers,   a U.S.
contract   staff member identified  the following    problems.
     --A week after the scheduled starting   date for the
        training  program, less than half of the required num-
        ber of students were present.
     --After     5 weeks of training, only nine students were
        enrolled,    three of whom had no formal agriculture
        training.

     --Because of the late arrivals   of students and the dif-
        ferences in their backgrounds and training,  the course
        content had to be modified and the quality  of train-
        ing was lowered.
                                     22
       --Because       the students  had been selected    without    their
          consent,       they had a poor attitude   toward learning.

        A decision       was made to cancel        the second class,         even
though the Government             of Kenya advised       that sufficient        can-
didates     were available.            These reductions       in the teacher
training     program contributed            to the July 1970 shortage          of
eight    teachers      for the 25 secondary          schools    in the voca-
tional     agriculture        program.      One school had no agriculture
teacher,     and the course was being conducted                 by other teach-
ers.     At seven schools           the workload     reportedly       was too
heavy for the assigned              teachers.     These vacancies        were ex-
pected to exist         until     the December 1970 graduates            from Eger-
ton Agricultural          College      became available       to fill    the posi-
tions.

         The inadequacy         of the number of participants                  available
for training       on AID's Public           Service      Staff     Support project
in Tanzania      is discussed         in a following           section     of this re-
port on the use of contract                staff     in host country           positions.
On another      project       in Tanzania,       the Agricultural           College
project,     an AID evaluation           report      of January        1970 states
that participant          training      has proceeded          at a slow pace, due
mainly     to the continuing          unavailability           of qualified         Tanza-
nian candidates         for training.           This report         states     further
that the slow pace of this               training        has resulted        in inade-
quate staffing        at Morogoro College              in Tanzania        and has con-
stituted     a delay in achieving             the target         of training        for
staffing     the college,

        Nonavailability         of local        nationals     or participants         for
training      was a problem        in Uganda as it was in Kenya and Tan-
zania.       For example,       one of the goals of the Makerere                   Agri-
culture      Faculty    project      in Kampala,         Uganda was to train          re-
placements       for the eight AID contract                staff   members assigned
to Makerere.          The contract        is scheduled        to terminate       in 1971.
Even though the training               program was conceived            in 1966 and a
number of years is required                 to provide      the necessary        educa-
tion and on-the-job           training        for a replacement,          only three
participants         had been identified             and were receiving        training
as of 1970.




                                             23
International     transportation    expenses

       In 1968 AID asked the three east African governments
to pay the transportation       costs for sending their partici-
pants to the United States for training.           In AID's view the
payment of transportation       costs would be a concrete demon-
stration   of self-help    by the east African governments, would
be an encouragement for more careful selection          of candidates
and better utilization      of their skills   after training,    and
would be consistent     with cost-sharing    policies   in other geo-
graphical   areas.

        Tanzania and Uganda agreed in 1969 to pay a portion of
the transportation     costs.    One participant's   training was
delayed more than 6 months before the agreement was nego-
tiated,    and another participant's     training  was delayed sev-
eral months because Uganda did not adhere to the terms of
the agreement,

       Kenya did not sign an agreement with AID to pay trans-
portation   expenses of participants    until June 1970. During
the intervening    Z-year impasse about the payment of trans-
portation   expenses in Kenya, only 21 participants      were nomi-
nated for the about 80 positions     available  for training   in
the United States.      Training for these 21 was delayed from
6 months to 1 year.

Dissolution     of the University   of East Africa
       In the early 1960's, the three east African governments
agreed to consolidate         the administration    of the higher edu-
cational   institutions       in their respective    countries  under a
single regional       University    of East Africa.
        Under this unified    university   system, AID has been pro-
viding technical     assistance     to the Veterinary   College in
Nairobi,    Kenya, and the Makerere Faculty of Agriculture          in
Kampala, Uganda-- the only institutions         in east Africa offer-
ing degrees in veterinary        sciences and agriculture,     respec-
tively.     Citizens  of the entire area may serve as faculty
members or may be admitted as students at these colleges.

       Recently the governments decided to dissolve  the Univer-
sity   of East Africa.  Kenya is establishing  its own

                                    24
                     . -,
       .                    L

 .
     r'agricultural      degree program, and Uganda is considering           es-
    . tablishing      its own veterinary      degree program.     For that rea-
       son the administrators        of the Veterinary     College and the
       Makerere Faculty of Agriculture          are reluctant    to nominate
       any but citizens       of their respective     countries    for advanced
       training     in the United States under the participant          training
0: . portion of the technical          assistance   agreements.

           The administrators   believe that, when the unified           sys-
     tems have been dissolved,     the participants      will be under
     strong pressures to take positions        on faculties     in their
     own countries   after their American university         training.
     Thus the student base from which to select qualified              candi-
     dates for advanced degree training        in the United States has
     been narrowed greatly.

            There are other problems of host country support, in
     addition     to the difficulties     encountered in securing partic-
     ipants    for the eventual replacement of AID-financed            staff
     members. For example, officials           at the veterinary    and agri-
     cultural     schools are concerned about the loss of present
     faculty members under the separate-university            concept.     The
     Makerere Faculty of Agriculture         already has lost one member,
     and the Veterinary       College expects to lose six or seven of
     its present staff of 53 members if a separate veterinary
     faculty    is established      in Uganda.
     Assinnment    of trained    participants

            Several participants     trained in the United States on
     educational    assistance   agreements were not assigned to teach-
     ing positions    when they returned.       AID documents show that
     the loss of these participants         from agreed teaching positions
     has been a contributing      factor in the failure     of certain
     projects    to meet their scheduled completion dates.

            For example, AID's Higher Agricultural       Education proj-
     ect in Kenya included the training       of east Africans    for the
     faculty   at Egerton College.    By May 1970, 21 of 22 partici-
     pants that had been sent to the United States for advanced
     training   had returned but only 12 of the 21 were assigned
     to the Egerton faculty.      The diversion    of trained partici-
     pants to other government positions        was cited by AID auditors
     as a contributing     factor in the need to extend the project
     beyond its scheduled completion date.
                                      25
TJNIWrERAL DECISIONS BY
HOST GJVERNMENT MINISTRIES

       A unilateral     decision by the Ministry     of Agriculture
in Uganda seriously         impaired the success of AID's Higher
Agricultural       Education project    in that country.    The primary
purpose of the project was to meet the estimated manpower
needs of Uganda's agricultural          extension service by provid-
ing assistance       to two agriculture     colleges in Uganda for
training     junior and senior extension workers.

        The agreed educational      goals of the project were to
train two senior officers         in a 3-year-diploma     course for
every three junior assistants         trained in a Z-year-
certificate     course.     Even though there is only a l-year dif-
ference in education,        there is considerable     difference   be-
tween the junior and senior positions.             The junior assis-
tants work more closely with farmers at the local level and
are assigned less territory         to work.     The senior assistants
are considered to be administrative          and they supervise one
or more junior      assistants.     The salaries    of the senior as-
sistants    are about twice those of the junior assistants.

      Apparently    wanting to Africanize     positions    more rapidly,
the Ministry    unilaterally     decided to eliminate     the certifi-
cate program from these schools and to offer the diploma
course only.     Staffing     in the extension service then became
seriously    out of balance, and there were nearly as many
senior officers     in the field as junior assistants.         On the
basis of its manpower survey published subsequently,            how-
ever, the Government of Uganda recognized its shortage of
junior extension workers and reintroduced           the junior certif-
icate course in the schools.          In the meantime several years
were lost in attaining       the project objectives.
      The Mission director      informed us that assistance had
been continued     in this case because the Mission believed
that the Ugandan economy could absorb all levels of manpower
in the agricultural      field.




                                  26
. TEMPORARYEMPLOYMENTPASSES
  TO CONTRACTEMPLOYEES

       Until issued a pass a foreigner may not work in Kenya.
 The Government of Kenya has held up work on some AID proj-
 ects by delays in issuing employment passes for contract   em-
 ployees.
        In one case an AID contractor    lost a candidate for a
 position   on a Kenyan contract when the candidate cancelled
 his agreement after waiting,     unsuccessfully,   for 5 months
 for a pass to work in that country.        This, in turn, created
 additional   delays to recruit   another candidate for the posi-
 tion and to obtain an employment pass for him. There was a
 4-month delay in obtaining     an employment pass for the new
 candidate.    Consequently a total of over 2 years elapsed be-
 fore the contractor   was able to place an employee on the
 job.
       The Government of Kenya delayed issuing a temporary em-
 ployment pass for 5 months for another contract   employee and
 thus delayed the starting   of work on organizing courses for
 a specialized  field of agricultural  education.1

      The AID Mission on occasion      has contacted    the Govern-
 ment of Kenya regarding delays.

 USE OF CONTRACTEMPLOYEESIN
 HOST COUNTRYPOSITIONS

         A type of technical    assistance is the financing     of op-
 erational      employees at the executive level to fill     respon-
 sible positions      within the host government.     Under this
 plan, the host government pays the basic national          salary and
 AID supplements it for a total salary comparable to U.S.
 standards.       The Public Service Staff Support project      in Tan-
 zania is of this type.       AID has had some problems in con-
 trolling     this project,  particularly  in ensuring that AID's

 1
  In a comment dated May 5, 1971, AID stated that "There have
  been no instances of delays cited since the GAO team visited
  Nairobi."


                                  27
contract employees are used to fill      only essential    posi-      .
tions,  in getting nationals   trained   and assigned to replace
contract employees,and in obtaining      information    on the work
performance of contract    employees.

       The project   agreement, dated May 1964, between AID and
the host country     provided,  in part, the following project
description.

     "The purpose of this project     is to assist the
     United Republic of Tanganyika and Zanzibar
      (Tanzania) to meet its short-term      high level
     manpower requirements   in order that the essen-
     tial services and projects     as articulated    in
     the Five Year Plan can be met. There currently
     exists a number of active programs which will
     narrow if not close the high-level       manpower gap
     by the end of the Five Year Plan period.         Until
     this gap is closed, however, a severe shortage
     of trained personnel will continue to exist in
     both the senior administrative     and professional
     areas.

     "In order to assist in the rapid upgrading or
     training  of local personnel for those positions
     supplied through this agreement, it is explic-
     itly understood that at least one citizen       of
     the United Republic will be identified     and,
     where possible,   trained to take over the job be-
     ing done by the U.S.A.I.D.     supplied expatriate
     person at the earliest    possible time."
       By June 1970 AID had provided 28 contract    executive
employees for the Tanzanian Government under the Public Ser-
vice Staff Support project.       Nine of these, or about 32 per-
cent of the total,   returned to the United States after serv-
ing their tours of generally      2 years each and without having
trained or having been replaced by a Tanzanian.        Another
six of these contract    executives were in their second 2-year
tour before local nationals      were designated as replacements.

      In one case a Tanzanian had been selected for on-the-
job training   to qualify  as a replacement but had not yet be-
gun this training   even though the contract  executive had
.
    .
        served 1 year of his second tour.    In other cases Tanzanians
        being trained by their contract   counterparts    were not, for
        various reasons, expected to be qualified      as replacements
        for the contract employees by the end of the contract       em-
        ployees' tours of duty.

                Because of the need to give more attention          to identi-
        fying and training        host country employees to replace AID op-
        erational     employees, AID issued a circular       in April 1969,
        calling    attention     to existing  guidance issued in January
        1968. This circular         stated that each host government should
        be able to develop a plan, prior to project            consideration
        and approval,      setting forth in specific     detail   the measures
        to ensure development of replacement          employees within a care-
        fully estimated time period.

              The Public Service Staff Support project was terminated
        as of June 30, 1969, except for those contract employees
        then authorized.    Contract employees to be provided after
        July 1, 1969, were to be incorporated        into separate project
        agreements by fields     of activity.     According   to AID this
        would enable a project manager to determine the relevance
        of requests for recruitments        and to malce certain    that (1)
        each approved request was related to U.S. project             objectives,
         (2) each approved request had a high priority           in the Tanza-
        nian development plan, and (3) a qualified          Tanzanian replace-
        ment could be identified     and trained.

                AID employees in Tanzania told us that, under the new
        procedure,    the Mission had adopted the practice              of sending
        a Mission official      to visit    the requesting      offices    of the
        Tanzanian Government.         The Mission official       ascertained     the
        significance     of the position      by interviewing      Tanzanian Cov-
        ernment officials      and by examining organizational            charts and
        other documents.       We  were  told   that    the Mission,     when  nec-
        essary, assisted the requesting           office to prepare an ade-
        quate position     description.

              Since AID contract   employees on this program are hold-
        ing executive positions    in the Government of the host coun-
        try, there is an understandable     limit OR AID's monitoring
        of the employees' activities.      Nevertheless    AID recognizes
        that it must have some channel of information        on U,S,-
        financed employees to appraise contract       performance in

                                            29
individual  cases and to appraise         the results     and problems     *
of the program generally.

     This need for information  was met only partially in
Tanzania by an agreement in 1967 between the Mission and the
Government, requiring  contract executive employees to sub-
mit a tour-end report through the Tanzanian Government to
the Mission after 18 months of their Z-year tours.

        AID internal   auditors     have reported on the inadequacy
of this single tour-end report from contract                employees as a
source of information         on the employees' activities          and per-
formances.       The auditors    expressed    their   belief   that   18
months after starting        a 2-year tour was too late for an ef-
fective    action by AID. Mission officials            say that they try
to fill    the information      gap by discussions      with Tanzanian
Government officials        on the activities       of contract     execu-
tives.




                                    30
                               CHAPTER5

                        ACTIONS TO IMPROVE

              TECHNICAL ASSISTANCEEFFECTIVENESS

        The problems noted during our review are of long stand-
ing, and AID has worked toward alleviating             these problems
and improving the effectiveness        of its technical       assistance.
Recent actions have been taken in two major areas--improving
the project     programming and monitoring       system and implement-
ing, on a pilot basis, a new contractual           arrangement with
U.S. universities.      AID expects that the impact of the ac-
tions in both areas will result primarily            in better plan-
ning, programming,     and monitoring    of   technical     assistance
projects    and of implementing contracts.         AID will need, how-
ever, to monitor continuously       technical     assistance     projects
to ensure that its new regulations         are implemented effec-
tively.

      In 1967 AID revised the instructions     for planning,     pro-
gramming, and monitoring  its technical    assistance   projects
to improve the depth and quality    of the planning and evalu-
ative processes.
        The revised instructions      for preparing the project
planning and authorization         document stated that the docu-
ment's relationship       to the process of evaluation       must be
carefully    considered-- that it must be in terms which would
facilitate     subsequent evaluation.      The instructions       further
emphasized that the planning document should show what the
situation    will be upon termination      of the project,      compared
with the beginning,       and when termination     will occur.       Thus
considerable      emphasis was placed upon clearly       identifying
during the planning process the targets,           goals, and objec-
tives in quantifiable       terms.

      A mission project   implementing plan was to be prepared,
showing in greater detail the relationship    between the
planned application    of resources and the project results.

       In 1968 AID initiated a project appraisal system that
required a comparison of actions taken with actions planned


                                    31
and the identification         of unsatisfactory      contractor     perfor-
mance. Effective        evaluation   under this system was dependent'
on the preciseness       of program goals and of objectives            in the
planning documents and contract           specifications.        AID had
difficulty,     therefore,     in making effective        appraisals   of
projects    started prior to about 1967 or 1968, which gener-
ally were initiated        without specific      and meaningful targets,
goals, or objectives.

      Even though AID has revised its instructions,          AID has
not always provided adequate measurement criteria          for re-
cently initiated  projects,   that is, projects    started after
the planning and programming instructions      were revised in
1967 and after the adoption of the project       appraisal     system
in 1968. This was the situation      for new projects      in Tanza-
nia, which, for the most part, replaced the Public Service
Staff Support project    and the Agricultural    College project
which were part of the 10 basic projects      that we reviewed.

      The Public Service Staff Support project   for providing
operational    employees at the executive level was terminated
in 1969 and was replaced by two other projects,     an Agricul-
ture Materials    and Services project and a Management and
Engineering    Services project.

       The Agriculture    Materials    and Services project was in-
tended to assist the Ministry         of Agriculture,      Food and Co-
operatives,    in developing     the capability     for planning,   ad-
ministering,     implementing,     and supporting     agriculture  de-
velopment activities.        This project was approved by AID/
Washington even though the project documents contained no
indication   of what would represent completion of the project
other than the provision       of the specified       inputs.

       Prior to approving the Management and Engineering           Ser-
vices project,     AID/Washington questioned the general nature
of the project     goals.   In this case AID/Washington pointed
out the importance of precisely        defining   the quality    and
quantity    of proposed assistance     and, if feasible,      of speci-
fying a timetable      for completion.     The project was approved,
however, after the Mission in Tanzania advised Washington
of the difficulty      in expressing precise goals and targets
for the project.


                                     32
       The Agricultural    College project     for providing   assis-
tance to Morogoro Agricultural        College was terminated     in
1969, but assistance      was continuing    to be provided through
an amendment to the Education Materials           and Services proj-
ect.    After reviewing documentation       for the Education Mate-
rials and Services project,        AID/Washington stated that the
overall    project objectives    were not clear.      Although the
project objectives      were not clarified     by the Mission, AID/
Washington subsequently       approved the project.

      Thus, although the need for clear and specific     goals
and objectives    in quantifiable terms had been recognized
and emphasized in the AID regulations,     these things were not
made a part of these three projects,    just as they were not
made part of the 10 projects which were reviewed by us and
which were initiated    prior to the 1967 and 1968 revisions
to the programming instructions.

       In 1969 AID contracted    for a review of the technical
assistance   project appraisal     system which it initiated     in
1968. Although this study was intended to concentrate            on
the project appraisal     system, the project planning,      program-
ming, and implementation     were to be evaluated to the extent
that they related to the appraisal       system.

      The contractor reviewed the project  appraisal   system on
the basis of a representative   sample of 63 projects.     Find-
ings on the need for more specifics   and evaluative   indica-
tors in the project planning and programming, similar      to
those found by us, were reported by the contractor.       He
stated that

      ,'*** critical    problems confronting          AID project
      evaluation     do not arise from imperfections           in
      the PAR [project      appraisal     report]     document and
      system.     Rather, they stem from inadequate proj-
      ect planning and deficiencies           in applying the
      concept of project management. USAID [Mission]
      managers lack the logical         framework required to
      define TA [technical       assistance]       projects   in
      terms that permit objective          verification      of suc-
      cess.     In the absence of such a framework, useful
      evaluation--that      is, evaluation        that produces


                                     33
      new project plans with        increased confidence     in
      successful  completion--      is enormously difficult,
      if not impossible.tt

     The contractor    reported that improvements in evaluation
depended upon resolution      of the following three basic prob-
lems hampering AID projects.

      "(1)   The purpose of Technical Assistance   projects
             rarely are defined sharply,   and the connec-
             tion between a project and its higher goals
             is almost never clearly  understood by proj-
             ect staff;

      "(2)   USAID mission staff are not asked to accept
             explicit  responsibility   for achieving proj-
             ect success, as success is highly dependent
             upon actions of others--thus,    there is rarely
             a clear sense of management responsibility;

      "(3)   Lacking both the orientation             that should be
             provided by clearcut        plans and sharply de-
             fined management responsibilities,               and the
             methodology appropriate         to a well-defined
             experimental     situation,     the USAID evaluator
             has found evaluation        a difficult       task, and
             has found it particularly           difficult     to
             translate    evaluation     results      back into
             better plans and better projects."

       The contractor    stated that projects       had not been clearly
related     to sector goals.    Consequently     there  appears to be
widespread lack of understanding          as to why projects       are be-
ing undertaken--     how they  relate   to  U.S.  country    objectives
and programming goals.        Lacking such insight      into broader
objectives,     mission employees find it difficult          to intelli-
gently   replan their projects,       and  the ability    to   replan is
the key to effectiveness.

       The contractor's    report also discusses an end-of-
project   status,   that is, an anticipation      of what should re-
sult from the project,       stated to facilitate    objective  veri-
fication   that the project purpose has been achieved.          The
importance of objectively        verifying  achievement of overall

                                     34
project purpose, in addition       to measuring    specific    project
outputs, is stressed.

      The contractor  indicates     that   his view supports that
already in the following     provision     of the AID programming
regulations.

      It 'It is of prime importance,    both to the project
      review and approval process and to the ultimate
      project  evaluation   process, that anticipated    re-
      sults of the project    be made clear.    The concept
      and specification   for "completior2'  of the proj-
      ect *** should be stated with maximum precision.'             I1

       The contractor   recommended that maximum       precision    in-
clude stating how completion of the project,           that is,
achievement of overall      project purpose, will      be verified
objectively    and that the means of verification         of project
completion be independent of the measurements           of project
outputs.

       The contractor's      recommended revised evaluation       system
was built    around what he termed 'Ia logical       framework for
evaluation,"     that is, showing specifically       the relationship
between inputs,      outputs,    project purpose, and sector or
programming goal and providing          objective criteria     in the
project   planning and programming documentation           for measuring
projects'    outputs and degree of achievement of projects'
purposes.

       On the basis of the contractor's        study and recommenda-
tions reported in July 1970, AID revised its technical               as-
sistance evaluation     system.    Although we have not made a de-
tailed   analysis of the revised system, it readily           is appar-
ent that the emphasis of the revised system is on the need
for more specifics    in technical     assistance    planning and
programming and on building      of evaluative      criteria    into the
project design--   needs  which  were   indicated    in   the projects
that we reviewed and those reviewed by the AID contractor.
In this respect the AID evaluation         handbook which was is-
sued as part of the revised system states under the caption
of implementation    plans:



                                   35
              "As life-of-project        documents, PROPS[non-             '
      capital    project     papers] deal with general strat-
      egy rather than detailed           tactics    and schedules.
      The same is generally          true of loan papers, al-
      though some of them may contain considerable                de-
      tail.     In  either    case,    detailed    plans of action
      are needed.       These plans also provide the bench-
      marks for meaningful          evaluation    of two important
      aspects-- effectiveness          and efficiency.

              "For non-capital       projects,    the Joint Proj-
      ect Implementation         Plan (PIP) is prepared in the
      early stages of the project            (see M.O. 1025.2).
      It sets out the work schedule and certain out-
      put indicators,       as well as such key inputs as
      personnel,     participant      and commodity require-
      ments.      The progress of a project          toward its
      established     targets can be measured against
      these output indicators           in a quantitative       manner.
      Some projects,       such   as  those    of an   advisory   or
      institution     building      nature, do not readily        lend
      themselves to quantitative           measures.       However,
      even in these cases, it should be possible to
      provide some definable          steps or forms of be-'
      havior which are verifiable            evidence of achieve-
      ment,"

       If this renewed emphasis results--in             actual practice--
in better project planning and programming, including                 the
building     of evaluative     criteria    into the project design,
then AID should be in a position             to contract    for more spe-
cific    items of work with a clearer delineation              of the con-
tractorBs     responsibilities       which, in turn, would permit a
more objective      evaluation      of contractor    performance.

      In commenting on our draft          report,    AID representatives
stated that:

      "This installation       process has gone forward sys-
      tematically     since the autumn of 1970. Four re-
      gional training      conferences of three days each
      were held for Evaluation        Officers,   Program
      Officers    and Deputy Directors       of U.S.A.I.D.
      Missions.     These conferences have been followed

                                     36
      by one-week training      visits    to each Mission by
      teams from Washington to work with Mission per-
       sonnel in the actual evaluation         of two projects
      per Mission.      The evaluations      have resulted   in
      action decisions      to improve implementation      or to
      modify forward plans.        Frequently,    the evalua-
      tions have also been a vehicle for communication
      with the contractor      and the host government which
      has clarified     targets and responsibilities.          In
      order to assist Missions in the conduct of rigor-
      ous evaluations,      AID has issued worksheets and
      guidelines    for project managers and an Evaluation
      Handbook for Evaluation        Officers.    Regular train-
      ing in Washington is also being set up to help
      overcome the effects of personnel turnover,"

       The impact of better planning,             programming, and moni-
toring required by the revised system should be felt in all
areas of project        implementation.         The annual evaluation    pro-
cess should draw management attention                 to most of the prob-
lems disclosed        by our review.        The performance analysis     seg-
ment of the evaluation          specifically      covers the implementing
agent's performance,         including      timely recruiting    of U.S.
contract    employees and training            and utilization   of a local
staff;    participant     training      is evaluated as a separate item,
including     participant      availability      and employment after
training;     various factors relating           to host country support
are evaluated as separate items.

     The implementing agents for technical    assistance      proj-
ects are, to a large extent, U.S. educational     institutions--
U.S. universities    being the implementing agents for six of
the LO projects   that we reviewed.
      A joint committee composed of members of the National
Association   of State Universities     and Land Grant Colleges
and AID recommended measures, in its January 1970 report,
to improve operating     arrangements between AID and universi-
ties for long-term institution      building  projects,     At the
heart of the report is the emphasis that the U.S. univer-
sity, AID, and the host country jointly       should define goals
and operational   targets in clear, realistic,       and measurable
terms, as well as accomplishment strategy,        and jointly


                                     37
should perform annual evaluations   of progress toward pro-,                    '
gram goals,   AID is testing  the committee's  proposals on a               .
pilot basis.

       The committee report envisioned    several phases in the
project planning,    having the joint participation       of all con-
cerned, before undertaking     a long-term institution      building
processe    Under one phase, reconnaissance     and assessment9 a
U.S, university    under contract with AID develops and dis-
cusses the university's    views on the validity,      feasibility,
and best shape of the project.

        If AID decides to continue with the project,         it then
extends the contract       with the U.S. university    for the proj-
ect refinement      phase.    During this phase the U.S. university
works with the host institution         to define goals and opera-
tional     targets in clear, realistic     and measurable terms; to
develop a time-phase work plan, at least for the early part
of the project;      to develop plans for evaluation,      including
criteria;      and to develop a budget proposal relating        incre-
ments of financing      to increments of the work program,
       After these phases AID negotiates   a long-term operating
agreement with the U.S. university.      An important aspect of
the agreement is a provision   for annual joint reviews of
progress toward achieving program goals.       The report indi-
cated that, to a large extent, success of the new arrange-
ment would depend upon continuing     emphasis on program eval-
uation.

     The comnittee report places considerable    emphasis on
the efforts  that should be made to obtain qualified    employ-
ees on a timely basis and to train participants     and phase
them into the project.
        The proposed new operating arrangement          between AID and
U.S. universities        is primarily   for projects    for long-term
institution     building    under certain   specified    conditions   and
thus is not applicable         to all AID contracts     with U.S. uni-
versities.

     Even though the new arrangement is primarily      for long-
term institution   building,    many of the principles are appli-
cable to contractual     arrangements for other purposes.    This

                                    38
would be especially  so for the preliminary  planning phases
of defining  goals and targets in measurable terms and the
annual joint program reviews.
        In another effort    to improve its management of techni-
cal assistance     projects,    AID sent a circular      in December
1968 to its missions calling        attention     to a wide range of
problems in implementing projects.            This circular      called
attention    to how AID's technical      assistance     project     instruc-
tions could aid in the technical         assistance     process; but it
emphasized that these instructions          became meaningful only
when the AID staff was alert to recognizing             existing      and po-
tential    problems, was perceptive       in ascertaining      their
causes, was imaginative        and resourceful      in designing solu-
tions, and was objective        and firm in taking indicated           ac-
tion.
     Thus, as desirable          as new procedures      and instructions
may be, AID recognizes         that their ultimate       effectiveness      de-
pends upon the actions         of AID employees.

         In commenting on our draft         report,   AID representatives
stated     that:

         "AID also recognizes      that keeping' its personnel
         alert and effective     requires    continuing     efforts
         by management at all levels.         Thus the December
         1968 circular    mentioned above has been followed
         by several other circulars        about ways to improve
         project management. The teams which visited
         Missions to help install       the new program evalua-
         tion system also conducted training          sessions on
         other aspects of project management besides
         evaluation.     A Handbook for Project Managers has
         been issued and management training          seminars are
         being given on a regular basis for field project
         managers.     Project Management has been on the
         agenda of AID's regional       Mission Director       con-
         ferences.     Despite such continuing       efforts,     how-
         ever, it must be recognized that technical              as-
         sistance personnel,     either contract      or direct-
         hire, will usually have to overcome local diffi-
         culties    in the environment of the host country.


                                       39
The difficulties        which these countries face in   '
operating     effectively,    are among the most im-
portant reasons why they need technical        assis-
tance."
                               CHARTER6

                 CONCLUSIONSAND RECOMMENDATION

       The 10 projects     that we reviewed were initiated         prior
to the emphasis in 1967 on building          evaluative   indicators
into the technical      assistance    project planning and program-
ming and clearly      specifying    what the anticipated     results     of
the projects     should be. Because of the absence of quantified
goals and objectives        in the planning and programming docu-
ments, AID was not in a position          to make an effective      evalu-
ation of project results         or to compare actual results       with
anticipated    results,      The same weaknesses were found in the
technical    service contracts.       Thus AID was not in a position
to adequately evaluate the contractors9           performances in pro-
viding technical      assistance,

      Similar findings  were reported as a result            of an AID
contractor's   review of AID's project     evaluation        system for
63 projects   selected on a representative      basis.

       AID has .worked through the years to improve the effec-
tiveness of its technical      assistance,     including   the revising
of its technical    assistance   project appraisal       system and its
pilot test of new working arrangements with U.S. universities
for long-term institution      building    projects.

       The greatest       impact of these actions,       either direct or
indirect,     appears to be in better planning and programming
through clearer identification            of project    output, project
purpose, and project relationship             to higher programming or
sector goals and through building             into the project     design
evaluative       criteria   for progress measurement and project          re-
planning.        These actions are necessary prerequisites           to more
effective     use of technical       service contracts      by a more spe-
cific    identification       of contractors9     scopes of work, tar-
gets, and overall         contract objectives      and by better eval-
uation of contractors9          performances in providing       technical
advice and assistance.

       Our review, however, of the actions AID took on three
projects   after incorporation of the foregoing management



                                   41
improvements in its instructions    showed that, for those *
projects  AID had not put the requirements    of its instruc-            *
tions into practice.

       AID has indicated,   as well as has the AID contractor,
that it is feasible     and practicable     to build into the.proj-
ects means to evaluate progress and to compare actual re-
sults with anticipated     results.     This is illustrated    by the
following   statement in AID's evaluation       handbook published
in October 1970 and reprinted       in February 1971. The project
implementation    plan

     I'*** sets out the work schedule and certain out-
     put indicators,        as well as such key inputs as
     personnel,     participant     and commodity require-
     ments.      The progress of a project          toward its es-
     tablished     targets can be measured against these
     output indicators         in a quantitative       manner.
     Some projects,       such   as those   of   an   advisory or
     institution      building    nature, do not readily
     lend themselves to quantitative             measures.
     However, even in these cases, it should be pos-
     sible to provide some definable             steps or forms
     of behavior which are verifiable             evidence of
     achievement."

       As AID has indicated that an even greater proportion  of
projects   are to be implemented with contractors,  the need
now appears to be that of ensuring that the output indica-
tors are put into actual practice.

RECOMMENDATION
             TO THE ADMINISTRATOR, AID
      In our draft report     submitted    to AID for    comment we
proposed that AID:

     1. Take appropriate   action to require in actual prac-
        tice that the project planning and programming doc-
        umentation and the technical       service contracts      con-
        tain more specific    delineations     of scope of work,
        goals, and objectives     and quantitative     indicators
        by which progress may be evaluated.



                                  42
   . 2. Institute   a procedure    for measuring   on a continuing
        basis

         (a) the seriousness and causes of contract  staffing
             problems, including  the identification of con-
             tractors  who fail to provide staff on a timely
             basis and

         (b) the full   significance  of the nonavailability         for
             training   of host country nationals.

      We have been advised that, since our fieldwork,        AID has
revised its technical    assistance   evaluation   system and that
"AID has expended a sizeable amount of effort        in reorganiz-
ing, planning,  and establishing     documentation   systems in or-
der to correct deficiencies."       We do not know what effect
these changes have had in actual practice.

      We recommend that the Administrator obtain an indepen-
dent evaluation  of the current practices and procedures and
determine the extent to which the types of weaknesses
pointed up in this report have been overcome in actual prac-
tice.




                                  43
                                 CHAPTER7

                        SUMMARYOF AID COMMENTS
        AID's comments on this report (see app.) were directed
primarily      to the subjects covered by the proposals in the
draft report.        AID indicated   concurrence on our proposal
that there was a need to require--in           actual practice--the
use of more specifics        and evaluative    indicators    in the
project     planning and programming and in the implementing
technical      service contracts.     AID indicated      also that the
proposal relating       to contract   staffing    and to training   of
host country nationals        was acceptable.

       Some elaboration was made upon the inherent difficul-
ties   in overcoming the problems.  AID also highlighted
       --the difficulty    of clearly identifying the effect of
          AID-financed  inputs and outputs because the AID in-
          puts may be only a small portion of the total ef-
          fort;
       --the current    AID actions in the process            of installing
          the revised   project appraisal system;
       --the efforts  by AID management in keeping employees
          alert to ways of making technical assistance more
          effective;

       --the difficulty     in, but the importance           of, providing
          contract   employees on a timely basis;            and

       --the situation      regarding        availability   of host country
          nationals   for   training.




                                        44
APPENDIX




  45
                                                                                    APPENDIX I



                             DEPARTMENT            OF STATE
                    AGENCY   FOR   INTERNATIONAL         DEVELOPMENT
                               WASHINGTON.     DC.    20523




                                                                           MAY 7 1971




Mr.  Oye V. Stovall
Director
International   Division
United States General Accounting Office
Washington, D. C.       20548
Dear Mr. Stovall:
We have completed our review of the draft report titled   "Effective-
ness of Technical Service Contracts in Providing U.S. Technical
Assistance to East Africa" which was transmitted with your letter
of March 5, 1971. Enclosed is a memorandumto the Auditor General
from Dr. Samuel C. Adams Jr., Assistant Administrator   of the Bureau
for Africa, which constitutes  the Agency's consolidated response to
this report.

                                                              Sincerely    yours,


                                                              Edward F. Tennant         //
                                                              Auditor     General       -
Enclosure:   a/s




GAO note:    Page numbers and related   references  in this appendix have
             been changed to correspond    to the pages of this report.




                                              47
           APPENDIX         1
            UNITED       STATES    GOVERNMENT


                                  randum
TO         :AG,   Mr. Edward F. Tennant
                                                                                   DATE:           MAY 5 1971
                                                   &
FROM       : AA/AJ?R, Samuel C. Adams, Jr.&
SUBJECT:    GAO Draft Report llEffectiveness     of Technical Service                 Contracts      in
            Providing    U.S. Technical Assistance   to East Africa"
            Introduction

            Subject draft report         concludes that ten technical            service contracts
            reviewed in East Africa         do not contain precise          statements       of
            objectives      and goals nor criteria       by which progress          toward objectives
            and goals may be measured;            the applicable      AID project        planning    and
            programming documents are equally deficient                 in these respects.           The
            report     also notes briefly      that despite the aforementioned               deficiencies
            creditable     progress was made under some of the contracts,                    e.g.,
            successful      introduction     of Vocational    Agriculture        courses in Kenyan
            secondary schools and strengthening             dairying      and livestock       ranching
            cooperatives        in Uganda.    Other points discussed were occasional
            slowness of contractors         to staff    teams; one host country             (Kenya) was
            sometimes slow in issuing work permits;               qualified      participant
            candidates      are not always available        in desired numbers or on time;
            and host governments         sometimes change priorities           in the middle of a
            project.

            GAO recommended that         AID -

                  - Take appropriate      action to require that planning and programming
                    documents, and technical        service contracts    as well, contain
                    specific  delineation       of objectives and criteria    by which progress
                    may be evaluated.

                  .=. Institute    procedures    to ascertain       on a continuing        basis

                     - The seriousness       of contract      staffing   problems including
                       identification       of contractors       who fail to provide    staff           on
                       a timely basis.

                     - The full    significance      of the non-availability           for   training
                       of participant       candidates.

            Z--I this memorandum we give         primary     attention   to the subjects           covered
            in the recommendations.




                           Bay U.S. Savings BOA Rcg&dy on the Payroll Savings Phn

                                                             48
                                                                                APPENDIX I




The draft      report places strong emphasis on the inadequacy of AID
project     planning and programmin g documentation        with respect   to state-
ments of project        objectives   and establishment    of bench marks by which
implementation        of projects  may be measured.      Contracts  for technical
 services     supplied by universities      or other types of organizations
 suffer   from the same inadequacies.         AD agrees that this makes it
difficult      for GAO to evaluate     the effectiveness    of technical    service
contracts      in East Africa.

Having recognized     this problem quite some time ago, AD has expended
a sizable amount of effort         in reorganizing,        planning,       and establishing
documentation    systems in order to correct             deficiencies.         In 1967, a new
system was initiated      for preparing      life-of-project           proposals     (PROP),
annual implementation      plans (PIP), and project             appraisal      reviews (PAR).
More emphasis was placed upon evaluation,                specification       of objectives
and measurable progress       indicators.        Continuing      modification       of these
interrelated   systems has resulted        in a new PAR format and till                soon
produce a new PROP format.

It should be noted that GAO studied ten technical                   service contracts
which were in effect          before the above mentioned procedures               were
instituted,        and that attempting       to change on-going contracts           over to
new procedures        is very difficult.         Projects  recently     initiated      and
future      projects    till  incorporate      new, improved techniques         and should
produce substantial          progress     in this area0

The ten projects       reviewed are typical          in that they indicate      the
difficulty      of clearly     identifying       the effect     of AD-financed    inputs
and outputs.       The primary reason is that AD inputs form only a small
portion    of the total       effort     often including      inputs by other donors,
which goes into implementation               of a given project.        As an example, AD
financed     six professionals         out of a total       of fifty   working at Edgerton
College in Kenya; therefore,              it is impossible        to accurately  establish
what portion      of the college's          overall  improvement was due to efforts
of the AD-financed          team.      Evaluation     can assess total progress and
can make a judgement that AD contributed                    to that progress.

Occasionally,    AD inputs make up the sole external          means employed to
achieve specific    targets.     Even in these instances,      there may be
problems of definition       and evaluation     because the process of achieving
social change is never as neatly          contrdazble  as a laboratory   experiment.




                                               49
APPENDIX I




In none of the ten cases surveyed in East Africa                    were AID inputs the
sole means, Although additional                efforts   will result       in improvement,
it is not possible        to assure completely         satisfactory        results   in
attributing    causality       in cases of joint ventures           such as technical
assistance.      It is &suggested that the draft report                recognize     this
fact by including        on page 1, before the fifth             complete paragraph,
a new paragraph to the effect             that "GAO recognized,          in the course of
its examination,       that there are often real difficulties                  in stating
anticipated    results      since AID inputs often form only a small part
of the total     input to a project;           and results     of this partial       input
are usually    dependent upon the performance                of elements of the other
inputs.     Nevertheless,        whenever possible,        precise    statements     of
objectives    and input/output         relationships       should be established.11

Although the discrepancies        noted on Page 6 are valid,          it should also
be noted that AID has made a diligent          effort     to make contract     and
other programming documents more ,precise.             It is suggested that an
additional   sentence be       added to the top of page 7 to the effect
that llAID's noteworthy     efforts    in improving project        planning   and
prw amming documentation         should be continued        to successful   completion
and careful    attention  given to achieving        similar    improvement of
technical   service contracts."

In addition   it is suggested that           a new paragraph be inserted            on
page 36 after   the third paragraph           to the effect  that:
     YChis installation        process has gone forward            systematically
     since the autumn of 1970.           Four regional        training       conferences
    of three days each were held for Evaluation                    Officers,       Program
    Officers    and Deputy Directors         of U.S.A.I.D.         Missions.         These
    conferences      have been followed        by one-week training            visits
    to each Mission by teams from Washington to work with Mission
    personnel     in the actual evaluation          of two projects           per Mission.
    The evaluations       have resulted      in action decisions             to improve
    implementation       or to modi@ forward plans.                Frequently,        the
    evaluations      have also been a vehicle          for communication            with
    the contractor       and the host government which has clarified
    targets    and responsibilities.           In order to assist Missions
    in the conduct of rigorous           evaluations,      AID has issued
    worksheets      and guidelines     for project      managers and an
    Evaluation     Handbook for Evaluation          Officers.        Regular training
    in Washington is also being set up to help overcome the effects
    of personnel      turnover.1f




                                            50
                                                                                    APPENDIX I




At the end of page         39 a new paragraph           is suggested     to the effect        that:

     "AID also recognizes        that keeping its personnel            alert      and
     effective    requires    continuing     efforts     by management at all levels,
     Thus the December 1968 circular            mentioned above has been followed
     by several other circulars          about ways to improve project                manage-
     ment.     The teams which visited        FEssions to help install              the new
     program evaluation       system also conducted training               sessions on
     other aspects of project         management besides evaluation.                  A
     Handbook for Froject        Managers has been issued and management
     training    seminars are being given on a regular basis for field
     project    managers.     Project Management has been on the agenda
     of AID's regional       Mission Director        conferences.       Despite such
     continuing     efforts,   however, it must be recognized                that
     technical    assistance     personnel,     either    contract     or direct-hire,
     will usually      have to overcome local difficulties               in the
     environment     of the host country.           The difficulties         which these
     countries    face in operating       effectively,        are among the most
     important    reasons why they need technical              assistance.11

The second two part recommendation               on page 43 of the Report is accept-
able.     The ability     of contractors       to supply qualified            staff     in timely
fashion is of crucial        importance      to many AID projects             now. It will
become even more so as increasing              proportions       of AID-financed            projects
are implemented by contractors.              At present,       difficulties         in staffing
are dealt with on a case-by-case;              and AID officials           involved       with a
given contract       exert appropriate       effort     to urge the contractor                to
speedier and better        performance     in providing        the required          staff.
AID also uses past performance           in earlier        contracts       in selection          of
contractors    for new jobs.                                                   ,
One difficulty        which contractors      cannot always surmount is that AID
sometimes asks them to undertake             jobs on short notice.           This is
especially     difficult      when professors      and highly qualified        educational
administrators        are to be recruited.         A lead time of a least a year
may be required         in order to recruit       well qualified     people.       Such
people usually make arrangements             for employment during the next
academic year by March.           If, therefore,       a contracting      university
signs a contract          with AID after March, it is sometimes impossible               to
recruit    an individual       to arrive before September of the following
year.     Moreover, unavoidable         accidents     occur to upset contractor
efforts    to place individuals         at fixed dates.        Contractors     and AID
must cantend with the fact that individuals                 and their families        some-




                                                   51
APPENDIX I




times change their minds about foreign      service                   at the last       moment
after  having initially expressed interest.        It                 is doubtful       that all
serious lapses can be avoided.

As for the problem of Government of Kenya delays in issuing work
permits,     contributing   to additional     delay in effective   project   staffing,
it appears that the situation         has improved as a result     of USAID/Kenya's
representation       to the GOK. There have been no instances         of delays
cited since the GAO team visited          Nairobi.    It is suggested therefore
that the Draft Report be changed as follows:

      On page     28 the last       sentence     should     be deleted,       lSee GAo note-1

      On page 29 delete the second and third                    sentences;      and substitute
      the following:   [See GAO note.]

            "It would appear that these contacts   have been effective.
            No delays have been reported  since the visit   of the GAO
            team in the spring of 1970.fr

 The second part of the recommendation                    calling     for "...ascertaining
 on a continuing         basis,...     (b) the full        significance        of the non-
 availability       for training       of host country nationals               . . . ..%s also
acceptable.         Over the years we feel that performance                      in this respect
has improved and it can be expected to improve still                            further.       Based
on our experience,           we can now plan project              inputs,    including      partici-
pant training,         more realistically.            In the past, there has been a
tendency toward optimism and high hopes that participants                                would be
found.        We should not assume from this that there will be no further
difficulty        in this regard.         AID does not and cannot control                  the pro-
cess.       As for the host governments,              they face real difficulties                in
finding       appropriate      candidates    for training;           and when they find them,
they must arrange for the candidates                    to be released from current
employment to obtain the further                 training       planned in a project.              These
problems are being sorted out as the LDCls experience                              and give
increasingly        close attention        to priorities.            Although the problem of
shortages       is yet to be resolved,           LDC education          system are turning           out
increasing        numbers of graduates         to fill       skilled      manpower needs.


Decontrol     upon issuance        of the final         GAO report.


GAO note:       The suggested changes to pages 28 and 29 have been
                made in the third and fourth  paragraphs on page 27
                of this report.




                                                   52                                  U.S GAO.   Wash.,D.C.
i   . ..   _.        ..-:.




                Copies  of this report are available      from the
                U. S. General   Accounting   Off ice, Room 6417,
                441 G Street, N W., Washington,     D.C., 20548.

                Copies    are provided     without        charge to Mem-
                bers of Congress,        congressional              committee
                staff members,    Government         officials,        members
                of the press,    college     libraries,         faculty   mem-
                bers and students.        The price to the general
                public   is $1.00 a copy.        Orders should be ac-
                companied    by cash or check.