oversight

Test and Evaluation: The Director, Operational Test and Evaluation's Controls Over Contractors

Published by the Government Accountability Office on 1990-12-21.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

“.. ._I ~ I^. ..-__-,-   _._-....-. ------.---.-.
1)(q*(~1111)(~1’ l!)!)O

                                                             TEST AND
                                                             EVALUATION
                                                             The Director,
                                                             Operational Test and
                                                             Evaluation’s Controls
                                                             Over Contractors

                                                                                                   ,-



                                                                                                        Illllllll~l~ll
                                                                                                                I
                                                                                                            142969




                                                                                      RELEASED
                                                         IUBTIUCTED --Not      to be released outside the
                                                         General Accotmtlng Offlce unless speclfhIly
                                                         approved by the Office of Congressional
                                                         Relations.


  (;hO/NSIAl)-!)I-(;0                               .-
I   __,. “. ..“.   _, . -. .l-- ___..._ _.,-...- -_.. ll...-l__
               United States
               General Accounting Office
               Washington, D.C. 20648

               National Security and
               International Affairs Division

               B-240028

               December 21,199O

               The Honorable David H. Pryor
               Chairman, Subcommittee on Federal
                 Services, Post Office, and Civil Service
               Committee on Governmental Affairs
               United States Senate

               Dear Senator Pryor:

               The Office of the Director, Operational Test and Evaluation (DONE),
               uses contractor assistance to support its oversight responsibility for
               operational test and evaluation (UT&E) of major weapon systems.
               Recently, congressional members have expressed concern that some of
               DOME’S contractors could have conflicts of interest; that is, they could
               assess the operational testing of the same weapon systems that they had
               participated in developing.

               In response to your request, we are providing information on                          DOT&E’S    use
               of contractors. Our objectives were to

             . describe the nature and extent of DOT&E’S management controls over con-
               tractor support, including measures to address possible conflicts of
               interest under omnibus contracts;’
             . provide our views on DOT&E’S use of Federally Funded Research and
               Development Centers,2 particularly the Institute for Defense Analyses;3
               and
             . determine W&E’S use of the Institute during fiscal years 1987-89 and
               identify any possible conflicts of interest and the Institute’s controls to
               avoid such conflicts.


               Concerned that or&~ was not receiving sufficient emphasis and indepen-
Background     dent oversight, the Congress established DOT&E as an independent test

               ‘Omnibus contracts contain a general statement of work to be performed by contractors. The con-
               tracting agency develops specific work orders or task&s for contractors based on this general work
               statement.
               %derally Funded Research and Development Centers are privately operated but publicly funded
               under long-term contracts with federal sponsoring agencies.
               3The Institute for Defense Analyses, a Center established in 1966, primarily assists the Office of the
               Secretary of Defense, the Joint Chiefs of Staff, and the defense agencies. The Institute provides
               studies, analyses, computer software, models, and other technical or analytical support for policy and
               program planning and management by its sponsors. It does not work for private companies, foreign
               governments, or the military departments.



               Page 1                                                           GAO/NSIAD-9180       OTflrE Controls
B-240028




organization in the Office of the Secretary of Defense (0s~). The Con-
gress believed that an organization with a vested interest in a weapon
system’s development should not evaluate the operational testing on
which production decisions are based. Rather, Congress intended that
the OT&E field tests be assessed by an independent test office to deter-
mine whether weapon systems can actually perform their intended
mission.

Because DOT&E lacks a large in-house staff, it obtains contractor assis-
tance to support its oversight of operational testing. Of its $34.8 million
total budget for fiscal years 1987-89, about 90 percent, or $31.2 million,
was for contractor assistance. Approximately $12.5 million was obli-
gated to the Institute for Defense Analyses, $0.2 million to another
center, and $18.5 million to private companies. Of the latter amount,
about $2.1 million was provided to private contractors awarded omnibus
contracts beginning in March 1989.

DUIXE’Smission is to provide independent assessments of major defense
systems. However, the ability to fulfill the independence mission was
questioned in a hearing held on DOT&E’S use of contractors and consul-
tants by the Senate Committee on Governmental Affairs, Subcommittee
on Federal Services, Post Office, and Civil Service. The primary concern
centered on possible conflicts of interest for private contractors. In
response to this concern, a DOT&E official told us DOT&E discontinued
using private contractors in March 1990. Since that time, DOT&E has used
only the Institute for or&~ assistance because it believes that the Insti-
tute, which does not develop or produce weapon systems, is less likely to
have a conflict of interest.

Although DOT&E relies on other defense agencies4to award contracts, the
Congress has indicated that it holds DCT&E ultimately responsible for the
consequences, such as conflicts of interest, of any contractor participa-
tion in or&~ activities.h DOT&E is also expected to follow the govern-
mentwide Federal Acquisition Regulation and applicable Office of
Management and Budget (OMB) guidance on contract management and

4Due to the limited dollar value of DOT&E’s contracting support, the Defense Test and Evaluation
Support Agency and Defense Supply Services-Washington award its contracts.
“See the Conference Report on the DOD Authorization Act for Fiscal Years 1990 and 1991. HRep. No.
101-331, Page 600 (1989). The act added section 2399 to 10 USC., and states that the Director is not
to contract with any person for advice or assistance in the test and evaluation of a system in which
the person participated in the development, production, or testing for a military department or
defense agency (or for another defense contractor). In addition, the legislation states that a contractor
that has participated in the development, production, or testing of a system for a military department
or defense agency is not to he involved (in any way) in the establishment of criteria for data collec-
tion, performance assessments,or evaluation activities for operational testing.



Page 2                                                             GAO/NSIAD-9180       OF&E Controls
                   B-240028




                   conflicts of interest regarding the use of contractors. For example, OMB
                   Policy Letter 89-1 states that the responsibility for identifying and
                   preventing potential conflicts of interest in government contracts is
                   shared between the contracting officer and the requester of the service.


                   DUI’&E’Spolicies and procedures for managing contracts for W&E support
Results in Brief   comply with existing guidelines. Moreover, DOT&E took measures that
                   were consistent with regulations requiring agencies to guard against
                   contractors’ conflicts of interest. For example, DUNE evaluated the
                   omnibus contracts for possible conflicts of interest and inserted lan-
                   guage in contracts requiring the contractors to disclose any conflicts.

                   Because a federal agency has considerable latitude in placing work with
                   Federally Funded Research and Development Centers, DOT&E'S use of the
                   Institute for CYI%E support is permissible under federal regulations. Fur-
                   ther, in our opinion, DCYRQE'Suse of the Institute is appropriate because
                   the Institute is less likely than private contractors to have conflicts of
                   interest.

                   Of the $12.5 million obligated by DONE to the Institute during fiscal
                   years 1987-89, about $2 million was used for consultant and subcon-
                   tractor support. The Institute’s policy is not to employ consultants on
                   tasks that could result in a conflict of interest. However, based on avail-
                   able information, we identified three consultants that were involved in
                   defense programs and later made operational test assessments for the
                   Institute on the same programs. We have no basis to conclude that the
                   consultants’ prior work affected their ability to provide objective and
                   impartial advice. These situations raise questions, however, about the
                   effectiveness of the Institute’s controls over consultants because it did
                   not, periodically update its conflict-of-interest reviews of consultants or
                   disclose to DOT&E possible conflicts so that they could be addressed. Even
                   though the Institute’s contract and Federal Acquisition Regulation do
                   not require disclosure of a consultant’s prior work, we believe such dis-
                   closure would be consistent with the regulation requiiring disclosure of a
                   center’s affairs.

                   The Institute’s work for OSD organizations responsible for system acqui-
                   sition and development testing raises questions regarding its ability to
                   be fully objective in performing operational testing work for DUNE. We
                   found four instances in which the Institute’s objectivity may have been
                   impaired due to its acquisition and development work. Although the
                   Federal Acquisition Regulation does not require the disclosure of such



                   Page 3                                          GAO/NSIAD-91-60   UT&E Controls
                 B-249028




                 circumstances, we believe that full disclosure to                  DOT&E   would have been
                 consistent with the spirit of the regulation.

                 A detailed discussion of these matters is included in appendix I.


                 The Institute is not now required to disclose to D~II‘&E possible conflicts
Conclusion and   of interest arising from (1) its work for OSD organizations responsible for
Recommendation   system acquisitions and development testing or (2) its use of consul-         4
                 tants. Nonetheless, we believe that such disclosure to DOT&E would be
                 consistent with the Federal Acquisition Regulation requiring disclosure
                 of the Institute’s affairs. Accordingly, we recommend that DOT&E require
                 the Institute to disclose possible conflicts of interest to it for resolution
                 because DCYIXE is ultimately responsible for the consequences of any con-
                 tractor participation in Or&E activities.


                 Our fieldwork was performed at various sites in the Washington, DC.,
Scope and        metropolitan area, including the offices of DOT&E and the Institute.
Methodology
                 We reviewed applicable legislation, regulations, policy letters, and
                 DOME’S contract management guidelines and interviewed DOT&E officials
                 about their controls over contractor support. Since the request focused
                 on DOME’S use of omnibus contracts to supplement its in-house capabili-
                 ties, we focused on DOT&E’S conflict-of-interest controls over those con-
                 tracts. We did not evaluate conflict-of-interest controls over other
                 private contractors.

                 Our views on DOT&E’S use of the Institute are, in large measure, based on
                 past reports that dealt with establishing and using Federally Funded
                 Research and Development Centers. However, to identify any potential
                 conflicts of interest at the Institute, we reviewed all 10 task orders spon-
                 sored by the Deputy Director Defense Research and Engineering (Test
                 and Evaluation) during fiscal years 1987-89 and information on 113 out
                 of 145 task orders sponsored by the Under Secretary of Defense for
                 Acquisition during fiscal years 1988-89.6 Our objective was to determine
                 which of these task orders included systems that were also included in
                 all the 42 task orders sponsored by DOT&E. We then interviewed officials
                 regarding the scope of selected task orders and, where necessary,
                 reviewed the Institute’s work products.



                 “At the time of our review, 32 task orders were not available.



                 Page 4                                                           GAO/NSIADBlSO   CYl’&E Controls
B-249028




We reviewed DONE'S contracts, taskings, and funding documents con-
cerning its relationship with the Institute. We also obtained lists of the
Institute’s consultants and subcontractors and reviewed available data
on organizational business ties and controls in place to address conflicts
of interest. The term “conflict of interest” means any situation in which
a person or organization is unable or potentially unable to render impar-
tial advice because of prior work, other activities, or relationships. We
did not evaluate whether the consultants or subcontractors had finan-
cial interests that could result in conflicts of interest.

We performed this review between September 1989 and September 1990
in accordance with generally accepted government auditing standards.
We did not obtain written agency comments. However, the views of
agency officials were sought during the course of our work and are
incorporated where appropriate.


As requested, we plan no further distribution of this report until 30
days after its issue date, unless you publicly announce its contents ear-
lier. At that time, we will send copies to the Secretaries of Defense, the
Navy, the Army, and the Air Force and to interested congressional com-
mittees. Copies will also be made available to others on request.

Please contact me at (202) 275-8400 if you or your staff have any ques-
tions concerning this report. Major contributors to this report are listed
in appendix II.


Sincerely yours,




Paul F. Math
Director, Research, Development,
  Acquisition, and Procurement Issues




Page 6                                          GAO/NSIAD-91-59   OWbE Controls
Contents


Letter                                                                                                 1

Appendix I                                                                                         8
DCYlYkE’Use
          s of          DOT&E’s Controls Over Contractor Support
                        Appropriateness of DOT&E’s Use of the Institute
                                                                                                   8
                                                                                                  10
Private Contractors     The Institute’s Work for DCYI?&E                                          10
and the Institute for
Defense Analyses
Appendix II                                                                                       17
Major Contributors to
This Report




                        Abbreviations

                        Dm&E     Office of the Director, Operational Test and Evaluation
                        OMB      Office of Management and Budget
                        OSD      Office of the Secretary of Defense
                        CT&E     operational test and evaluation


                        Page 6                                        GAO/N&ID-91-50   OT&E Controls
Page 7   GAO/NSIAJ%91-60   Ul’&E Controls
Appendix I

DOT&E’s Use of Private Contractms md the
Instituk for Defense Analyses

                               In obtaining assistance and advisory services, DOT&E uses the required
DCYIY&E’Controls
          s                    controls over both contract management and conflicts of interest, as dis-
Over Contractor                cussed below.
support

Management Controls Meet       DCr&E’s  contract management control system is consistent with proce-
OMB Requirements               dures prescribed by “OMB Circular No. A-120.“1 The circular requires,
                               among other things, that (1) written approval for advisory and assis-
                               tance services be required at a level above the organization sponsoring
                               the activity, (2) requirements be appropriate and fully justified in
                               writing, (3) work statements be specific and complete, (4) contracts be
                               competitively awarded, (5) work be properly administered and moni-
                               tored, (6) work be evaluated when completed, and (7) written reports on
                               advisory and assistance services be obtained when needed. As summa-
                               rized below and on succeeding pages, we found that DOT&E’S contract
                               management process meets these requirements.

                         To initiate contractor assistance, a DOT&E staff member and the DCYME
                           l


                         Program Analyst, who is responsible for DOT&E’S overall funding, draft a
                         task order defining the scope and need for contractor support. The
                         Deputy Director, Resources and Administration, responsible for DOlYkE’S
                         contractor assistance, reviews the work statement and decides whether
                         to proceed with the work request. For all work requests, a review board
                         consisting of DOLCE’S deputy directors decides whether to contract for
                         the proposed work,
                       . DOME’S Program Analyst maintains monthly financial statements that
                         track obligations by task order. The financial statements are designed to
                         ensure that DOT&E does not over obligate funds.
                       . The larger contractors provide m&E monthly status reports which it
                         evaluates and uses to track contract status and costs.
                       l When a work request is reported as having been completed, the con-
                         tracting officer’s technical representative consults with the DOT&E staff
                         member who initiated the work and reviews the monthly status reports
                         to determine whether the work has actually been completed. Also, the
                         OSD Director of Contractor Assistance and Advisory Services helps to
                         ensure that the proper process is followed in determining whether the
                         contractor support was adequate.
                       . To document the use of contractor assistance, the DORUE Program Ana-
                         lyst and the contracting officer’s technical representative maintain log

                               ‘Guidelines for the Use of Advisory and Assistance Services (Jan. 4,1988).



                               Page 8                                                           GAO/N&W-91-60   CYl%E Controls
                          Appendix I
                          lXYl’&E’s Use of private Contractor    and the
                          Institute for DePenae Analyses


                                                                                                                               .




                          books consisting of task orders and supporting records for the larger
                          contracts.


Controls Established by   Before it began obtaining ONE support exclusively from the Institute,
                          DCYl’&E had controls in place to guard against possible conflicts of interest
DOF&E Adhered to          on omnibus contracts. These controls adhered to the Federal Acquisition
Regulations               Regulation, which states that agencies should prevent conflicts of
                          interest that could result in an unfair competitive advantage to a con-
                          tractor or impair a contractor’s objectivity in performing contract work.
                          Therefore, as required by the regulation, DOT&Eevaluated the omnibus
                          contracts to (1) identify and evaluate potential conflicts of interest early
                          in the acquisition process and (2) avoid, neutralize, or mitigate signifi-
                          cant conflicts before contract award.2

                          DOT&E  further attempted to prevent conflicts of interest by inserting a
                          provision in the contracts requiring contractors to self-disclose potential
                          conflicts and eliminate them. For example, contractors were required to
                          evaluate whether the work to be done for W&E conflicted with work
                          being done for other organizations. In addition, a DOT&E official said that
                          based on past experiences, DOT&E was generally aware of a contractor’s
                          business ties and has tried to avoid giving contractors work that could
                          result in a conflict of interest. For example, one contractor was not given
                          work on space systems because the contractor’s prior business had the
                          potential of creating a conflict.

                          In June 1989, the Senate Committee on Governmental Affairs, Subcom-
                          mittee on Federal Services, Post Office, and Civil Service, held a hearing
                          on D(JT&E’S use of contractors and consultants. The Subcommittee
                          released a report which found that private contractors’ work for the
                          Department of Defense raised concerns of both direct and indirect con-
                          flicts of interest. Direct conflicts of interest resulted from the omnibus
                          contractors monitoring and assessing operational tests of weapon sys-
                          tems that they participated in developing. In addition, indirect conflicts
                          of interest could result from their relationships with other private con-
                          tractors that developed weapon systems that the contractors would
                          later assist DOT&E in evaluating. Because DfYl%E discontinued this practice



                          ‘According to the Federal Acquisition Regulation, each contracting situation should be examined on
                          the basis of its particular facts and the nature of the proposed contract. The exercise of common
                          sense, good judgment, and sound discretion is required in deciding whether a significant potential
                          conflict exists and, if it does, in developing an appropriate means for resolving it.



                          Page 9                                                          GAO/NSJAD914IO      Ol%E Controls
                       Appendix I
                       DUME’s Use of Prlvate Contractors    and the
                       Instltnte for Defense Analyses




                       in March 1990, we did not evaluate whether its use of these private con-
                       tractors would result in conflicts of interest. Instead, we focused our
                       attention on DOT&E’S sole use of the Institute for obtaining Ol%E support.


                              now uses only the Institute for OWE support. DOT&E believes that
Appropriateness of     DCT&E
                       this practice is appropriate because the Institute is thought to be less
DCJIY&E’Use
          s of the     likely than private contractors to have a conflict of interest concerning
Institute              particular weapon systems. We agree. Under its charter, the Institute
                       has no commercial interests in developing weapon systems and may not
                       work for the services that could develop and use those systems.

                       In addition, ~&E’S use of the Institute is consistent with govern-
                       mentwide regulations issued as Policy Letter 84-1 by the Office of Fed-
                       eral Procurement Policy and later incorporated into the Federal
                       Acquisition Regulation. The regulation states that Federally Funded
                       Research and Development Centers must meet some special long-term
                       research or development need that cannot be met as effectively by
                       existing in-house or contractor resources, In March 1988, we reported
                       that centers are effective because of their expertise and independence
                       and because they have a special relationship, including the sharing of
                       information, with their sponsoring agency.3

                       Also, the policy letter specifies that a center’s purpose, mission, and gen-
                       eral scope of effort should be stated clearly enough to differentiate
                       between work that should be done by the center and that which should
                       be done by private contractors. The policy letter, however, does not
                       state how these mission statements should reflect such differentiation.
                       According to the Institute’s mission statement, its purpose is to promote
                       national security, public welfare, and advancement of scientific
                       learning. Its scope of effort typically covers analyses, evaluations, and
                       reports, including the examination of the relative effectiveness of alter-
                       native national security measures. Thus, under federal regulations and
                       the Institute’s mission statement, DOT&E has considerable latitude in
                       placing work with the Institute.


                       The Institute’s relationship with DOWE is governed by Policy Letter 84-1
The Institute’s Work   and the Federal Acquisition Regulation, which set out Federal policy for
for DOT&EY             establishing and using Federally Funded Research and Development

                       3Competition: Issues on Establishing and Using Federally Funded Research and Development Centers
                       (GAO/NSIAD -88 - 22 , Mar. 1988).



                       Page 10                                                       GAO/NSIAD-91-60    Ol’&E Controls
                                 Appendix I
                                 DW&E’s Urre of Private cOntractora       and the
                                 Institute for Defense Analyeee




                                 Centers. The Federal Acquisition Regulation reinforces Policy Letter
                                 84-1 by stating that a center is required to conduct its business in a
                                 manner befitting its special relationship with the government, operate in
                                 the public interest with objectivity and independence, be free from orga-
                                 nizational conflicts of interest, and have full disclosure of its affairs to
                                 the sponsoring agency. The Institute has controls to ensure that

                             . its consultants and subcontractors4 do not have previous or current
                               employment that could result in conflicts of interest and
                             l its work for OSDorganizations responsible for weapon systems acquisi-
                               tion and development testing does not conflict with its operational
                               testing work for DOT&E.

                                 As discussed below, we question whether the Institute is completely
                                 effective in administering these controls. In addition, we believe the con-
                                 trols can be improved by having the Institute disclose possible conflicts
                                 to WT&E for resolution because DOT&Eis ultimately responsible for the
                                 consequences of any contractor participation in or&~ activities. Such dis-
                                 closure to DGME would be consistent with the Federal Acquisition
                                 Regulation.


The Institute’s Controls         Of the $12.5 million obligated by DW&E to the Institute during fiscal
Over Consultant/                 years 1987-89, about $2 million was paid to 51 consultants and 2 sub-
                                 contractors. (The Institute’s use of consultants and subcontractors as a
Subcontractor Conflicts of       percentage of its work for DOME declined from 24.4 to 12.3 percent over
Interest Have Weaknesses         this period.) Based on available information, we believe these consul-
                                 tants/subcontractors did not have other current employment that could
                                 cause conflicts of interest. However, the Institute did not periodically
                                 update its conflict-of-interest reviews of consultants or disclose to DOT&E
                                 any prior employment of the consultants that might have involved
                                 potential conflicts. Further, the Institute did not document its subcon-
                                 tractor conflict-of-interest reviews.

ConsultantControls               The Institute’s policy is not to employ consultants on jobs that could
                                 result in a conflict of interest. For example, a consultant would not be
                                 allowed to do or&~ work on a weapon system if the consultant was also
                                 working for the developer of the same weapon system. To carry out this
                                 policy, the Institute requires its consultants to sign an agreement stating

                                 4According to an Institute official, a consultant is an individual who agrees to work for the Institute
                                 as needed at an agreed-upon hourly rate. A subcontractor is an incorporated entity that works a
                                 specified number of hours for the Institute.



                                 Page 11                                                             GAO/NSIAD-9180       Ol’&E Controls
Appendix I
LKYI’& E ’s Use of F’rkate Contra&m     and the
lnntitnte for Defense Analyses




that they will (1) avoid any activities that cause conflicts of interest and
(2) disclose possible conflicts so that the Institute may exclude them
from such work. Before hiring consultants, the Institute asks about their
backgrounds to determine whether any of their past or present business
activities could cause a conflict. However, the Institute does not periodi-
cally update information about its consultants’ business ties or other
employment after they are hired; therefore, a new business tie could
arise and create a potential conflict of interest, and the Institute might
not know about it.

Of the 61 consultants hired by the Institute, 2 had played a role in over-
seeing the services’space programs before they did operational test
assessments on such programs for DOT&E. The Institute was aware of
these consultants’ prior work experiences but did not disclose this infor-
mation to DCJ’IJ~ZEbecause it was satisfied that no conflict of interest was
created, One consultant previously worked for the Air Force in a high-
level policy position involving space systems and later planned and
reviewed the Institute’s OT&E assessments of various Air Force space
systems. The other consultant headed the Naval Space System Division
in the Office of the Chief of Naval Operations and later evaluated the
Institute’s CJI’&E assessments of a Navy weapon system that was to pro-
ceed into full-rate production. While we have no basis to conclude that
the consultants’ prior work affected their ability to provide objective
and impartial advice, these situations nonetheless raise questions about
potential conflicts of interest and the performance of the Institute in
administering pertinent controls. Considering the consultants’ prior gov-
ernment positions and their work for the Institute, we believe the Insti-
tute should have disclosed these situations to DOT&E for resolution.
Although the Institute’s contract6 and the Federal Acquisition Regula-
tion do not specifically require the disclosure of a consultant’s past and
present employment, we further believe that such disclosure would have
been consistent with the regulation requiring disclosure of a center’s
affairs to the sponsoring agency.

In another situation, a consultant performed work on the LHX heli-
copter for a private company prior to performing OT&E related work on
this system. The work for the private company included evaluating the
requirement for the helicopter as well as a general developmental


“The Institute’s contract states that it shall submit quarterly the names of all consultants employed
during the period, a short statement of the matters on which the consultant’s advice or service was
needed, the daily rate of compensation, and the period for which service was required.



Page 12                                                           GAO/NSIAD-9180       (Yl’& E Controls
                             Appendix I
                             DUT&E’s Use of Private Contractors   and the
                             hwt1tut.e for Defense. Analyses




                             approach. The private company later prepared a proposal for a con-
                             tractor that was awarded a development contract. The Institute was
                             aware of this consultant’s prior work experiences but did not disclose
                             this information to DOT&Ebecause it was satisfied that no conflict of
                             interest was created. Although we have no basis to conclude that the
                             consultant’s prior work for the private company affected the con-
                             sultant’s ability to provide objective and impartial advice to the Insti-
                             tute, this situation again demonstrates a need for disclosure of such
                             potential conflicts to D(JT&E.For reasons similar to the above situations,
                             disclosure of this matter to DOT&Efor resolution would have been consis-
                             tent with the regulation.

                             An Institute official told us that more attention will be given to con-
                             sultant’s work in relation to specific areas of prior employment.

SubcontractorControls        Although we were advised that the Institute also assessed whether
                             hiring a subcontractor would cause a conflict of interest, it did not docu-
                             ment such assessments. Despite this shortcoming, the Institute’s two
                             subcontractors did not appear to have past or present employment that
                             would affect their objectivity.

                             We reviewed the outside ties of the subcontractor that received over
                             $852,000, or about 99 percent of the funds that the Institute paid to
                             subcontractors from 1987 to 1989. Although the subcontractor worked
                             for both the Institute and the Air Force in the chemical warfare area, its
                             work for the two organizations differed. The subcontractor performed
                             policy analyses and studies for the Institute as part of its DCYIXEwork
                             regarding how and when the United States should retaliate if the Soviets
                             were to use chemical weapons. For the Air Force, the subcontractor
                             assessed chemical contamination avoidance and decontamination proce-
                             dures and chemical defenses against a Soviet chemical threat.


The Institute’s Work for     The Institute’s work for the OSDorganizations responsible for system
Other OSD Organizations      acquisition and development testing raises questions regarding its
                             ability to be fully objective in performing operational testing work for
Raises Questions About Its   W&E. In three instances, we found that the Institute performed acquisi-
Objectivity                  tion and operational testing work on the same weapon systems. In one
                             other instance, the Institute performed similar develobment and opera-
                             tional testing work on the same system. Even though the Federal Acqui-
                             sition Regulation does not require the disclosure of such instances, we
                             believe disclosure to DCWE for resolution would have been consistent
                             with the regulation.


                             Page 13                                         GAO/NSIAD9180   CJT&E Controls
Appendix I
DUl’&E’s Use of Private Contractors     and the
Institute for Defense Analyses




In fiscal years 1988-89, the Under Secretary of Defense for Acquisition,
who is responsible for the weapon system acquisition policy, including
development testing, sponsored task orders for the Institute. In
reviewing information on 113 of these task orders and all 42 task orders
sponsored by DOT&E, we identified 3 sets of task orders relating to both
acquisition and operational testing tasks for the same weapon systems.
The Institute’s work for the Under Secretary called for assessing alter-
native candidates to perform a specified mission, while the work for
DOT&E required creation of methodologies for operational test planning
and assessment of test results. Although the Institute personnel per-
forming acquisition work do not perform operational testing work on the
same weapon systems, we believe the Institute’s work in assessing these
candidates should be disclosed to DOT&E for possible resolution,

For example, in one case, the Institute’s work for the Under Secretary
required the identification of capabilities and the appropriateness of
various aircraft in satisfying the Army’s scout, attack, and assault mis-
sions. This analysis included assessing the technical and operational
advantages and disadvantages of various competing candidates,
including modified Apache helicopters, various tilt rotor aircraft, and
light helicopter weapon systems. For DOME, the Institute will assess
operational test functions on the light helicopter weapon system ulti-
mately selected, including the review of test plans, monitoring of
ongoing testing, and making recommendations on the test program. We
believe that the Institute’s work in developing acquisition options for
the Under Secretary raises questions about its ability to be fully objec-
tive in assessing operational test matters for the system eventually
selected.

We also compared the 42 task orders sponsored by DOT&E with the 10
task orders sponsored by the Deputy Director Defense Research and
Engineering (Test and Evaluation), OSD’S development test organization.
The Institute’s personnel worked for both the Deputy Director and DW&E
on nine weapon systems.6 A potential conflict generally was not created
because the types of work were different. For example, the Institute’s
work on the Army Tactical Missile System for the development test
organization identified a generic methodology to be used by the develop-
ment tester in evaluating weapon systems, while the Institute’s work for


“During the course of our evaluation, the Institute’s personnel working in its Operational Evaluation
Division performed development and operational testing related work. They were not organization-
ally separated from one another.



Page 14                                                           GAO/NSIAJh9180       m&E    Controls
Appendix I
IXJI%E’s Use of Private Contractors   and the
Institute for Defense Analyses




DOME evaluated the adequacy of the actual operational testing on that
specific system.

As another example, the Institute’s work on the Advanced Medium
Range Air-to-Air Missile for the Director, Live Fire Testing, examined an
approach for conducting live fire testing. Live fire testing is used to
determine the physical vulnerability of selected U.S. aircraft and armor
systems to enemy weapons, and the lethality of U.S. weapons against
selected enemy aircraft and armor systems. On the other hand, the Insti-
tute’s work for DOBE was fundamentally different because it evaluated
the adequacy of the operational testing on that specific missile by
assisting in the evaluation of the missile’s operational effectiveness and
suitability in a realistic environment. In our view, the Institute’s work
for the Director, Live Fire Testing, does not conflict with its work for
m&E because the nature and types of work were very different.
Although live fire testing complements operational testing, this special-
ized form of testing focuses on a system’s vulnerability and lethality.

In one case, however, involving the Forward Army Air Defense weapon
system, the Institute’s personnel working on development and opera-
tional testing performed essentially the same tasks. The tasks included
identifying issues, objectives, and threats that should be addressed in
testing; reviewing and monitoring the Army’s plans and preparations for
testing; observing the conduct of the tests; and assisting in the analyses
of all test phases. In our view, performance of work by the Institute in
the development test phase could impair the Institute’s ability to be
completely objective in performing essentially the same work regarding
operational testing. Nevertheless, the Institute’s development testing
work ended in 1987 because OSDbelieved DCII'&Ewas the most appro-
priate sponsor for the task.

In response to our preliminary findings, an Institute official stated that
its Operational Evaluation Division will not undertake tasks related to
systems and programs over which D~&E has oversight responsibilities
for any sponsor other than DO&E and the Director of Live Fire Testing.
With the exception of tasks relating to live fire testing, the Institute will
not perform development testing work for the Deputy Director Defense
Research and Engineering (Test and Evaluation). Additionally, no Insti-
tute research staff member or consultant may be assigned to an opera-
tional test and evaluation task for D~&E if that staff member or
consultant, subsequent to the establishment of the Office of DOT&E,has
participated in the evaluation or analysis of that system as part of an



Page 15                                           GAO/NSIAD-91-60   oT&E Controls
DCW&E’e Uee of Private Contractmu   and the
Institute for Defense Analyses




Institute task for any sponsor other than     LKY%E    and the Director of Live
Fire Testing.




Page 16                                               GAO/NSIAD9180   CW&E Controls
.
Appendix II

Major Contributors to This Report


                        Michael E. Motley, Associate Director
National Security and   Lester C. Farrington, Assistant Director
International Affairs
Division,     Washington,
                        Charles D. Groves, Evaluator-in-Charge
                        Carlos E. Hazera, Evaluator
                        James A. Driggins, Evaluator
DC.                     Thomas E. Mills, Adviser


                        William T. Woods, Assistant General Counsel
Office of General       Raymond J. Wyrsch, Senior Attorney
Counsel




(296229)                Page 17                                       GAO/N&ID-9180   Ol’&E Controls
‘J’t~lt~ptlont~ 20%275-624 1
------