oversight

Special Studies and Evaluations Issue Area Plan: Fiscal Years 1998-2000

Published by the Government Accountability Office on 1997-12-01.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                United States General Accounting Office

GAO             National Security and International
                Affairs Division



December 1997   Special Studies and
                Evaluations Issue Area
                Plan
                Fiscal Years 1998-2000




GAOIIAP-98-2          /    t69      0700 I1
Foreword


           As the auditing and evaluating arm of Congress, the General Accounting
           Office is charged with following the federal dollar wherever it goes.
           Reflecting stringent standards of objectivity and independence, GAO'S
           audits, evaluations, and investigations promote a more efficient and
           cost-effective government; expose waste, fraud, abuse, and
           mismanagement in federal programs; help Congress target budget
           reductions; assess financial information management; and alert Congress
           to developing trends that may have significant fiscal or budgetary
           consequences. In fulfilling its responsibilities, GAO performs original
           research and uses hundreds of databases or creates its own when
           information is unavailable elsewhere.

           To ensure that GAO'S resources are directed toward the most important
           issues facing Congress, each of GAO's 33 issue areas develops a strategic
           plan that describes the significance of the issues it addresses, its
           objectives, and the focus of its work. Each issue area relies heavily on
           input from congressional committees, agency officials, and subject-matter
           experts in developing its strategic plan.

           The Special Studies and Evaluations (SSE) issue area in the National
           Security and International Affairs Division addresses public policy
           questions that are best answered using applied social science research and
           analytical methods. SSE will undertake studies that involve complex
           measurement problems, extensive quantitative and qualitative data
           analyses, and other methodological research challenges. Planned studies
           will be conducted across a number of substantive areas, including the
           measurement and assessment of government investments in defense
           research and technology, the quality of life in the military services, the
           impacts associated with changes in defense infrastructure, the outcomes
           of foreign assistance programs, and the effectiveness of nontraditional
           roles and missions conducted by the Department of Defense.

           SSE'sproposed plan of work is structured around the following three
           issues:

           * complex methodological evaluations,

           * data quality and measurement, and

           * cross-cutting issues.




           Page 1                                                         GAOIIAF-98-2
Foreword




In the pages that follow, we describe our key planned work with respect to
these issues.

Because events may significantly affect even the best of plans, GAO'S
planning process allows for updates and the flexibility to respond quickly
to emerging issues. If you have any questions or suggestions about this
plan, please call me at (202) 512-3092.




Kwai-Cheung Chan
Director
Special Studies and Evaluation Issues




Page 2                                                          GAOAIAP-98-2
    Contents


    Foreword                                 1
    Table I: Key Issues                      4


    Table II: Planned                        6
    Major Work
    Table III: GAO                           7
    Contacts




I




                          Page 3   GAOALAP-98-2
Table I: Key Issues



Issue                                                  Significance
Complex Methodological Evaluations:                    The demand by Congress for empirical information on program
What are the intended and unintended effects of key    conditions, impacts, and outcomes has grown considerably in recent
programs, policies, and investments?                   years as attention has focused on ways to reduce agency funding,
                                                       increase cost savings, and improve the efficiency and effectiveness of
                                                       programs and operations. Many claims have been made by policy
                                                       advocates and interest groups about the failures and/or successes of
                                                       current programs and policies, but few are supported by convincing data
                                                       and analyses. Studies using complex evaluation approaches and
                                                       extensive data analyses are needed to determine whether programs
                                                       (1) are achieving the results they were designed to accomplish, (2) are
                                                       cost-effective, and (3) are having unintended effects (both positive and
                                                       negative).
Data Quality and Measurement:                          Several issues of congressional concern highlight the need to examine the
Are data and measures used in policy decision-making    quality of information used for policymaking and the methods used in
valid and reliable?                                    generating such information. Increasingly, congressional decisionmakers
                                                       faced with tough legislative choices are hampered by incomplete,
                                                        inconsistent, and faulty data about the performance and effects of
                                                        programs and policies. It is important to evaluate the validity, reliability,
                                                       and utility of data and measures that are currently available (e.g., tracking
                                                        research and development investments and assessing quality-of-life
                                                        conditions), the methods used to develop such information (e.g., risk
                                                        assessment methods for estimating health and safety effects of weapons
                                                        disposal), and the development or adoption of new methods for obtaining
                                                        information (e.g., approaches to monitor illnesses that may result from
                                                        overseas deployments).
Cross-Cutting Issues:                                   With increased frequency, Congress has asked policy questions that
How do conditions, attributes, and outcomes compare     require looking broadly across several different programs and functions.
across different programs and activities?               There is interest, for example, in examining the costs and effectiveness of
                                                        technologies that are common to many types of weapon systems.
                                                        Similarly, there is a need to examine the uses of technology and assess its
                                                        trade-offs and impacts in terms of force structure, training, and readiness.
                                                        Efforts to improve military effectiveness have also drawn attention to many
                                                        quality-of-life issues, such as housing, family life conditions, and
                                                        compensation, that collectively need to be evaluated. Furthermore, as
                                                        efforts continue toward consolidating international assistance programs,
                                                        there is an interest in determining what approaches work best to
                                                        strengthen agencies' evaluation capabilities and lead to more effective
                                                        program outcomes.




                                             Page 4                                                                     GAO/IAP-98-2
                                               Table 1: Key Issues




Objectives                                                  Focus of work
 * Use evaluation research methods to assess the quality, * Cost-effectiveness of key technology investments (i.e., stealth
effectiveness, and efficiency of key programs and policies. technology, ballistic missile defense, global positioning system).
* Examine the factors that significantly influence program     * Analysis of programs and activities that address quality-of-life issues.
outcomes.
                                                          * Evaluation of impacts and outcomes of selected foreign assistance
 * Determine whether alternative approaches are needed to programs.
improve program results.
                                                          * Evaluation of military interventions to achieve foreign economic
                                                          stabilization and development and nation building.


* Analyze existing data on program conditions, trends, and * Analyses of Department of Defense (DOD) research and development
characteristics.                                           investments by geographic location, service/agency, and function.
* Identify and develop measures to evaluate programs,          * Assessments of quality-of-life measures and conditions over time.
policies, and investments.
                                                               * Technical review of DOD's methods for assessing the risks of military
* Examine the appropriateness of methods used to               activities that may affect the health and safety of nearby populations
produce policymaking information.                              (e.g., weapons testing and disposal).
* Identify cost-effective alternatives for improving data,     - Evaluation of the validity and reliability of measures and methods for
measures, and methods.                                         determining foreign assistance program impacts.


* Evaluate programs intended to provide coordinated            * Evaluation capabilities at U.S. and multilateral foreign assistance
responses to fulfill military roles and missions.              agencies.
* Identify and assess different programmatic approaches        * Cost-effectiveness and implications of using DOD resources versus
(i.e., lessons learned) that improve efficiency and            civilian sector resources.
effectiveness.
                                                               * Analyses of costs and benefits of military technologies and impacts on
                                                               force structure, training, and readiness.




                                               Page 5                                                                         GAO/IAP-98-2
Table II: Planned Major Work



Issue                    Planned major job starts
Complex Methodological   *Review critical technology investment and development of ballistic missile defense.
Evaluations
                         *Evaluate effectiveness of DOD's technology investments in chemical and biological warfare detection
                         and prevention (ongoing).

                         *Evaluate the trade-offs of chemcial weapons disposal methods in terms of health and safety risks,
                         technological effectiveness, and costs.
                         *Determine basis for the different services' rotation policies and assess their impacts on the quality of
                         life of enlisted personnel.

Data Quality and         *Examine and analyze DOD's research, development, test, and evaluation budget and investments.
Measurement
                         *Evaluate the nature and extent of family-life related costs and difficulties in the military and their
                         impacts on retention.
                         *Analyze estimates of the prevalence of cancer and other chronic illnesses among Gulf War veterans
                         (ongoing).

                         *Assess selected outcomes of defense spending in states (ongoing).
                         *Evaluate methods and criteria for real property maintenance costs and needs across the services
                         (ongoing).

Cross-Cutting Issues     *Examine methodological approaches bilateral and multilateral agencies use to evaluate foreign
                         assistance program impacts.
                         *Assess cost-effectiveness and outcomes of using DOD resources in international drug control
                         programs.

                         *Evaluate objectives and outcomes of recent U.S. military interventions such as those in Somalia,
                         Panama, and Haiti.




                                        Page 6                                                                            GAOAIAP-98-2
Table III: GAO Contacts


Director              Kwai-Cheung Chan   (202) 512-3092

Assistant Directors   John Oppenheim
                      Sushil Sharma




                      Page 7                              GAO/IAP-98-2
Ordering Information

The first copy of each GAO report and testimony is free.
Additional copies are $2 each. Orders should be sent to the
following address, accompanied by a check or money order
made out to the Superintendent of Documents, when
necessary. VISA and MasterCard credit cards are accepted, also.
Orders for 100 or more copies to be mailed to a single address
are discounted 25 percent.

Orders by mail:

U.S. General Accounting Office
P.O. Box 37050
Washington, DC 20013

or visit:
Room 1100
700 4th St. NW (corner of 4th and G Sts. NW)
U.S. General Accounting Office
Washington, DC
Orders may also be placed by calling (202) 512-6000
or by using fax number (202) 512-6061, or TDD (202) 512-2537.

Each day, GAO issues a list of newly available reports and
testimony. To receive facsimile copies of the daily list or any
list from the past 30 days, please call (202) 512-6000 using a
touchtone phone. A recorded menu will provide information on
how to obtain these lists.
For information on how to access GAO reports on the INTERNET,
send an e-mail message with "info" in the body to:

info@www.gao.gov

or visit GAO's World Wide Web Home Page at:
http://www.gao.gov




PRINTED ON     RECYCLED PAPER
United States                       Bulk Rate
General Accounting Office                Fees Paid
Washington, D.C. 20548-0001    Postage & Fees Paid
Official Business               Permit No. G100
Penalty for Private Use $300

Address Correction Requested