United States General Accounting Office GAO National Security and International Affairs Division December 1997 Special Studies and Evaluations Issue Area Plan Fiscal Years 1998-2000 GAOIIAP-98-2 / t69 0700 I1 Foreword As the auditing and evaluating arm of Congress, the General Accounting Office is charged with following the federal dollar wherever it goes. Reflecting stringent standards of objectivity and independence, GAO'S audits, evaluations, and investigations promote a more efficient and cost-effective government; expose waste, fraud, abuse, and mismanagement in federal programs; help Congress target budget reductions; assess financial information management; and alert Congress to developing trends that may have significant fiscal or budgetary consequences. In fulfilling its responsibilities, GAO performs original research and uses hundreds of databases or creates its own when information is unavailable elsewhere. To ensure that GAO'S resources are directed toward the most important issues facing Congress, each of GAO's 33 issue areas develops a strategic plan that describes the significance of the issues it addresses, its objectives, and the focus of its work. Each issue area relies heavily on input from congressional committees, agency officials, and subject-matter experts in developing its strategic plan. The Special Studies and Evaluations (SSE) issue area in the National Security and International Affairs Division addresses public policy questions that are best answered using applied social science research and analytical methods. SSE will undertake studies that involve complex measurement problems, extensive quantitative and qualitative data analyses, and other methodological research challenges. Planned studies will be conducted across a number of substantive areas, including the measurement and assessment of government investments in defense research and technology, the quality of life in the military services, the impacts associated with changes in defense infrastructure, the outcomes of foreign assistance programs, and the effectiveness of nontraditional roles and missions conducted by the Department of Defense. SSE'sproposed plan of work is structured around the following three issues: * complex methodological evaluations, * data quality and measurement, and * cross-cutting issues. Page 1 GAOIIAF-98-2 Foreword In the pages that follow, we describe our key planned work with respect to these issues. Because events may significantly affect even the best of plans, GAO'S planning process allows for updates and the flexibility to respond quickly to emerging issues. If you have any questions or suggestions about this plan, please call me at (202) 512-3092. Kwai-Cheung Chan Director Special Studies and Evaluation Issues Page 2 GAOAIAP-98-2 Contents Foreword 1 Table I: Key Issues 4 Table II: Planned 6 Major Work Table III: GAO 7 Contacts I Page 3 GAOALAP-98-2 Table I: Key Issues Issue Significance Complex Methodological Evaluations: The demand by Congress for empirical information on program What are the intended and unintended effects of key conditions, impacts, and outcomes has grown considerably in recent programs, policies, and investments? years as attention has focused on ways to reduce agency funding, increase cost savings, and improve the efficiency and effectiveness of programs and operations. Many claims have been made by policy advocates and interest groups about the failures and/or successes of current programs and policies, but few are supported by convincing data and analyses. Studies using complex evaluation approaches and extensive data analyses are needed to determine whether programs (1) are achieving the results they were designed to accomplish, (2) are cost-effective, and (3) are having unintended effects (both positive and negative). Data Quality and Measurement: Several issues of congressional concern highlight the need to examine the Are data and measures used in policy decision-making quality of information used for policymaking and the methods used in valid and reliable? generating such information. Increasingly, congressional decisionmakers faced with tough legislative choices are hampered by incomplete, inconsistent, and faulty data about the performance and effects of programs and policies. It is important to evaluate the validity, reliability, and utility of data and measures that are currently available (e.g., tracking research and development investments and assessing quality-of-life conditions), the methods used to develop such information (e.g., risk assessment methods for estimating health and safety effects of weapons disposal), and the development or adoption of new methods for obtaining information (e.g., approaches to monitor illnesses that may result from overseas deployments). Cross-Cutting Issues: With increased frequency, Congress has asked policy questions that How do conditions, attributes, and outcomes compare require looking broadly across several different programs and functions. across different programs and activities? There is interest, for example, in examining the costs and effectiveness of technologies that are common to many types of weapon systems. Similarly, there is a need to examine the uses of technology and assess its trade-offs and impacts in terms of force structure, training, and readiness. Efforts to improve military effectiveness have also drawn attention to many quality-of-life issues, such as housing, family life conditions, and compensation, that collectively need to be evaluated. Furthermore, as efforts continue toward consolidating international assistance programs, there is an interest in determining what approaches work best to strengthen agencies' evaluation capabilities and lead to more effective program outcomes. Page 4 GAO/IAP-98-2 Table 1: Key Issues Objectives Focus of work * Use evaluation research methods to assess the quality, * Cost-effectiveness of key technology investments (i.e., stealth effectiveness, and efficiency of key programs and policies. technology, ballistic missile defense, global positioning system). * Examine the factors that significantly influence program * Analysis of programs and activities that address quality-of-life issues. outcomes. * Evaluation of impacts and outcomes of selected foreign assistance * Determine whether alternative approaches are needed to programs. improve program results. * Evaluation of military interventions to achieve foreign economic stabilization and development and nation building. * Analyze existing data on program conditions, trends, and * Analyses of Department of Defense (DOD) research and development characteristics. investments by geographic location, service/agency, and function. * Identify and develop measures to evaluate programs, * Assessments of quality-of-life measures and conditions over time. policies, and investments. * Technical review of DOD's methods for assessing the risks of military * Examine the appropriateness of methods used to activities that may affect the health and safety of nearby populations produce policymaking information. (e.g., weapons testing and disposal). * Identify cost-effective alternatives for improving data, - Evaluation of the validity and reliability of measures and methods for measures, and methods. determining foreign assistance program impacts. * Evaluate programs intended to provide coordinated * Evaluation capabilities at U.S. and multilateral foreign assistance responses to fulfill military roles and missions. agencies. * Identify and assess different programmatic approaches * Cost-effectiveness and implications of using DOD resources versus (i.e., lessons learned) that improve efficiency and civilian sector resources. effectiveness. * Analyses of costs and benefits of military technologies and impacts on force structure, training, and readiness. Page 5 GAO/IAP-98-2 Table II: Planned Major Work Issue Planned major job starts Complex Methodological *Review critical technology investment and development of ballistic missile defense. Evaluations *Evaluate effectiveness of DOD's technology investments in chemical and biological warfare detection and prevention (ongoing). *Evaluate the trade-offs of chemcial weapons disposal methods in terms of health and safety risks, technological effectiveness, and costs. *Determine basis for the different services' rotation policies and assess their impacts on the quality of life of enlisted personnel. Data Quality and *Examine and analyze DOD's research, development, test, and evaluation budget and investments. Measurement *Evaluate the nature and extent of family-life related costs and difficulties in the military and their impacts on retention. *Analyze estimates of the prevalence of cancer and other chronic illnesses among Gulf War veterans (ongoing). *Assess selected outcomes of defense spending in states (ongoing). *Evaluate methods and criteria for real property maintenance costs and needs across the services (ongoing). Cross-Cutting Issues *Examine methodological approaches bilateral and multilateral agencies use to evaluate foreign assistance program impacts. *Assess cost-effectiveness and outcomes of using DOD resources in international drug control programs. *Evaluate objectives and outcomes of recent U.S. military interventions such as those in Somalia, Panama, and Haiti. Page 6 GAOAIAP-98-2 Table III: GAO Contacts Director Kwai-Cheung Chan (202) 512-3092 Assistant Directors John Oppenheim Sushil Sharma Page 7 GAO/IAP-98-2 Ordering Information The first copy of each GAO report and testimony is free. Additional copies are $2 each. Orders should be sent to the following address, accompanied by a check or money order made out to the Superintendent of Documents, when necessary. VISA and MasterCard credit cards are accepted, also. Orders for 100 or more copies to be mailed to a single address are discounted 25 percent. Orders by mail: U.S. General Accounting Office P.O. Box 37050 Washington, DC 20013 or visit: Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders may also be placed by calling (202) 512-6000 or by using fax number (202) 512-6061, or TDD (202) 512-2537. Each day, GAO issues a list of newly available reports and testimony. To receive facsimile copies of the daily list or any list from the past 30 days, please call (202) 512-6000 using a touchtone phone. A recorded menu will provide information on how to obtain these lists. For information on how to access GAO reports on the INTERNET, send an e-mail message with "info" in the body to: firstname.lastname@example.org or visit GAO's World Wide Web Home Page at: http://www.gao.gov PRINTED ON RECYCLED PAPER United States Bulk Rate General Accounting Office Fees Paid Washington, D.C. 20548-0001 Postage & Fees Paid Official Business Permit No. G100 Penalty for Private Use $300 Address Correction Requested
Special Studies and Evaluations Issue Area Plan: Fiscal Years 1998-2000
Published by the Government Accountability Office on 1997-12-01.
Below is a raw (and likely hideous) rendition of the original report. (PDF)