May 2003 HIGHLIGHTS OF A GAO FORUM Key National Indicators: Highlights of GAO-03-672SP Assessing the Nation’s Position and Progress The nation confronts profound Developing Key National Indicators for the United States Is Important challenges resulting from a variety While there are a variety of indicator efforts in the United States, there is no of factors, including changing generally accepted, comprehensive indicator system for the nation as a whole. security threats, dramatic shifts in There was broad agreement that the issue of developing key national indicators demographic patterns, the is important for taking a more comprehensive view of the nation’s position and multidimensional processes of progress, both on an absolute and relative basis. Several models were discussed globalization, and the accelerating that offer lessons for developing a national indicator system, including indicator pace of technological change. systems on aging, children, economics, and health. The purpose of These are all coming together in an measurement, the process of deciding what to measure, and determining era of diminishing public audiences are as critical as choosing what and how to measure. resources. The nation’s leaders and concerned A Broad Range of Information Areas Are Considered Significant citizens require better knowledge The range of information areas considered important was broad, covering the of what is happening and where we economy, society, and the environment. Participants agreed that a first step is to are going to support improved assemble “core” indicators from existing data. A straw proposal for such an public choices. The United States indicator set—USA Series 0.5—was presented as a starting point for building could potentially benefit from what might eventually be a broadly supported USA Series 1.0 indicator set. developing a set of key national Series 0.5 included 11 key information areas: community, crime, ecology, indicators to help assess our education, governance, health, the macroeconomy, security, social support, nation’s position and progress. sustainability, and transparency. In reacting to Series 0.5, participants suggested numerous refinements and identified 4 additional information areas: On February 27, 2003, GAO, in communications, diversity, individual values, and socioeconomic mobility. cooperation with the National Academies, hosted a forum on key A Rich History of Indicator Systems Warrants Collective Research national indicators. The purpose of There is a long history of efforts throughout the world by leading democracies to the forum was to have a rich and develop and sustain indicator systems. A distinction was made between meaningful dialogue on whether comprehensive and specialized efforts that focus on a topic or issue. Research and how to develop a set of key on what can be learned from past and present systems is essential to deriving national indicators for the United useful implications for a possible United States system. A multitude of efforts States. are currently under way in other democracies (e.g., Australia and Canada) as well as in the United States at the national, regional, state, and local levels. The forum brought together a Despite this activity, there appear to be few common sources of broad research diverse group of national leaders to to facilitate knowledge sharing on comprehensive indicator efforts. discuss the following: • How are the world’s leading A United States Initiative Must Build on Past Lessons and Current Efforts democracies measuring national performance? Developing a U.S. indicator system requires applying lessons from past efforts • What might the United States do and engaging with many existing ones. A United States system must be flexible to improve its approach and and evolve to respond to societal change and incorporate diverse perspectives. why? An informal national coordinating committee of institutions in the public and • What are important areas to private sectors was constituted to begin organizing a U.S. initiative. It serves as measure in assessing U.S. an initial means to facilitate dialogue, expand participation, plan work and national performance? secure financing. As of May 7, 2003, the committee included the American • How might new U.S. approaches Association of Universities, The Conference Board, the Council for Excellence in be led and implemented? Government, GAO, the International City/County Management Association, The www.gao.gov/cgi-bin/getrpt?GAO-03-672SP. National Academies, the National Association of Asian American Professionals, the Office of Management and Budget and the White House Council on To view the full report, click on the link above. Environmental Quality. For more information, contact Christopher Hoenig, Director, Strategic Issues, at (202) 512-6779 or email@example.com. Contents Letters 1 Comptroller General of the United States Introductory Letter 1 President of the Institute of Medicine, The National Academies Introductory Letter 4 Forum 6 Background 6 Forum Summary 11 How Are the World’s Leading Democracies Measuring National Performance? 14 What Might the United States Do to Improve Its Approach and Why? 24 What Are Important Areas to Measure in Assessing U.S. National Performance? 34 How Might New U.S. Approaches Be Led and Implemented? 38 Appendixes Appendix I: Forum Participants 42 Appendix II: Illustrative Indicators by Information Area for USA Series 0.5 46 Appendix III: Selected Bibliography on Indicator Systems 49 Appendix IV: Selected Web Sites on Indicator Systems 60 Tables Table 1: Illustrative Comprehensive Indicator Systems 16 Table 2: Illustrative Specialized Indicator Systems 17 Table 3: Selected Highlights of Indicator Traditions in the United States during the 20th Century 22 Table 4: Structure of USA Series 0.5 30 Table 5: Proposed Evolutionary Process for a National Indicator System (Includes Illustrative Information Areas) 39 Figures Figure 1: Life Expectancy at Birth and at Age 65 31 Figure 2: U.S. Emissions of Greenhouse Gases, Based on Global Warming Potential, 1990-2000 (in Million Metric Tons of Carbon Equivalent) 32 Page i GAO-03-672SP Key National Indicators Forum Contents This is a work of the U.S. Government and is not subject to copyright protection in the United States. It may be reproduced and distributed in its entirety without further permission from GAO. It may contain copyrighted graphics, images or other materials. Permission from the copyright holder may be necessary should you wish to reproduce copyrighted materials separately from GAO’s product. Page ii GAO-03-672SP Key National Indicators Forum Leter Comptroller General of On February 27, 2003, the U.S. General Accounting Office (GAO) in cooperation with the National Academies convened the Forum on Key the United States National Indicators for the United States. This report summarizes the Introductory Letter research, points of view, and commitments that the event produced. We were pleased to have the National Academies as a partner in this event. They have demonstrated the ability not only to conduct quality research but also to help professional communities reflect on and build consensus around the operational definition of indicator sets, in key areas, such as communities, ecology, education, health and transportation. Although the forum took place in Washington, D.C. the event was not merely about the federal government or the public sector. It addressed issues about indicators for the nation at all levels, from the community to the country as a whole. Those who attended came in the spirit of a national endeavor that rose above personal, institutional, or sectoral interests. Because the United States is diverse, so were the participants. Gathered together were national leaders and experts who could articulate the concerns and perspectives of businesses, government, the media, foundations, and nonprofits as well as the scientific, statistical, and educational communities—a group representing both the users and producers of public information. Essentially, the broad impetus for the new level of dialogue at the forum comes from two sources. First, that more and better public information may be needed to effectively resolve current and future national challenges. Second, that the laboratories of democracy in our country and around the world are engaged in hundreds of efforts to develop indicator systems, many of which are comprehensive in nature. It is a logical extension to consider a comprehensive indicator system for the United States that would help assess the nation’s overall position and progress. There is a strong implication here. To be a leading democracy in the information age means producing objective, independent, scientifically grounded, and widely shared quality information on where we are and where we are going, on both an absolute and relative basis, including comparisons to other nations. Such information must be useful to the public, professionals, and leaders at all levels of our society. Page 1 GAO-03-672SP Key National Indicators Forum The founders of our nation knew this critical issue needed ongoing attention as it grew and evolved. President George Washington, in his first annual message to Congress on January 8, 1790, said, “Knowledge is in every country the surest basis of public happiness. In one in which the measures of government receive their impressions so immediately from the sense of the community as in ours it is proportionably essential.” Since that time, there has been a long history—checkered by success and failure—of attempts to create sources of information that would inform our public dialogues and serve as a context for governance and civic choices. Developing a comprehensive, independent, quality resource of key indicators for a nation as large, complex, and diverse as the United States is a daunting task. If it is to be done, we must work hard and work together to avoid the mistakes of the past and take advantage of new opportunities that have emerged in the 21st century. One lesson shows the need for patience, persistence, and attention to democratic process. There is an important role for the federal government, and in particular the U.S. Congress, to help catalyze an effort to develop and sustain a national indicator system. A fully operational set of credible measures of our progress and prospects will take years to develop, require broad involvement of American society, and involve substantial resource commitments. And yet many believe the benefits, in terms of more facts, broader consensus, and better choices, will far outweigh the costs. Our objective in convening the forum was to stimulate a dialogue that might encourage a collective commitment from several leading institutions to validate the need and begin organizing themselves to take action. While this objective has been met, it should be stressed that this forum is merely the start of a new stage of our country’s long journey of increasing self- awareness and sense of collective accountability. We are pleased to help contribute to this effort and look forward to working with the extraordinary group of committed parties and the many who are Page 2 GAO-03-672SP Key National Indicators Forum continually joining the effort to develop options and approaches that will be of truly lasting value to the American people. David M. Walker Comptroller General of the United States Page 3 GAO-03-672SP Key National Indicators Forum President of the On behalf of the National Academies, let me add my deep appreciation to all who participated in and facilitated this forum. The level of participation Institute of Medicine, in this important event speaks volumes about a topic that is critically The National important to our nation and ripe for our attention. It has been a pleasure for the National Academies to be a part of this promising, important, and Academies timely venture. Introductory Letter It is fitting that the National Academies -- the Institute of Medicine, the National Academy of Sciences and the National Academy of Engineering, together with our operating arm, the National Research Council – cooperated in facilitating this effort. We represent a body that has provided advice on scientific issues that affect public policy decisions since 1863. Over that long period, the Academies have been able to contribute to public discussion and understanding on many issues of national significance. In an important way, our collaboration with the General Accounting Office on this issue is a continuation of the contribution we seek to make to the formulation of sound public decisions based on sound scientific evidence. This collaboration with GAO on the development of key national performance indicators is one of a number of projects we have undertaken under an agreement implemented in 2001. We were pleased to have initially convened a panel of experts representing various disciplines to share experiences and views on the use of indicators, then, when the panel suggested this forum, to support GAO in bringing it together. I am enthusiastic about the possibilities and the promise of this forum. Public policy in many areas, including medicine, is stronger because of the existence of indicators of performance. Indicators help our nation focus on the key issues confronting us. We can be proud of the tremendous efforts that have been made in the scientific community to develop them. As those of us in the public and private sectors jointly consider the next steps to take, benefiting from this very useful report of the forum prepared by GAO, I hope we keep in mind our ultimate objectives. I will be thinking about the enlightening discussion of the issues with a question in mind: Page 4 GAO-03-672SP Key National Indicators Forum “WHY DO WE WANT TO KNOW THIS”? This important question gets us started and helps us frame all of the other questions we must ponder. Dr. Harvey V. Fineberg President, Institute of Medicine The National Academies Page 5 GAO-03-672SP Key National Indicators Forum Forum Background The pace and character of change is having a profound impact on the United States government, the nation itself, and its position in the global community. Changing security, economic, demographic, technological, and other trends have, in some cases, exacerbated economic, social, and environmental tensions. These trends have created new challenges and opportunities both within the United States and throughout the world. In just one example, the United States faces a huge and growing long-range fiscal imbalance due primarily to known demographic trends, rising health care costs, and other factors. Policymakers must reconcile the gap between projected revenues and expenditures in order to exercise fiduciary and stewardship responsibilities to the nation. A large and growing amount of activity is taking place, throughout the United States and around the world, to develop comprehensive indicator systems to inform the democratic process. New facts, insights, and approaches are being developed. Understanding and interpreting these efforts is vital to the process of setting direction and measuring progress— on both an absolute and a relative basis—as a context for governance. Although other leading democratic nations have developed key national indicator systems, the United States has not done so. While there are numerous indicator systems that are national in scale, such systems focus on specialized or specific topics such as health care or education. The question is, should we develop a comprehensive, national system that focuses on major elements of society—economical, social, and environmental areas? A set of key national indicators can help to assess the overall position and progress of our nation in key areas, frame strategic issues, support public choices and enhance accountability. It could help improve evaluations of how well the nation is addressing and resolving key issues and concerns. National indicators built on the foundation of information from our federal statistical system (i.e., official statistics), administrative records, as well as a variety of private sources could provide a unique, fact-based assessment of the state of the nation. The dramatic changes, challenges and increasing interdependencies affecting the nation demand new and more cross-sector and cross-border responses. Such responses could benefit from more integrated information resources to support informed public debate and decisions within and among different levels of government and society. For example, in Page 6 GAO-03-672SP Key National Indicators Forum Forum homeland security, what indicators will accurately reflect national preparedness? In health care, how will we assess the health and well-being of our population? How can we best measure success in education? Is the most useful information available to fully assess our degree of economic and social progress? Are we in fact moving ahead and in the right direction in key areas? How do we compare to other nations? The stakes are high, including considerations regarding scarce public resources, creating jobs, stimulating future industries, maintaining global competitive edge, enhancing security, sustaining the environment, and promoting quality of life. Developing a key national indicator system goes beyond any one sector (i.e., public, private, or nonprofit). It requires designing and executing a process whereby the diverse elements of society can participate in formulating key questions and choosing indicators in a way that increases consensus on the facts over time. It also involves complex issues ranging from fostering agreement on specific indicators to choosing the mechanisms for sharing reliable information used in public planning, decision making and accountability. Furthermore, indicators in the national system should be outcome-oriented, in addition to measuring resources and capabilities. They should measure position and progress on not only an absolute but also a relative basis, including comparing the United States to other nations. They should not be seen as being the nation’s goals or priorities, but rather a more sophisticated base of facts with which to make more informed decisions. To discuss the issues involved in developing a set of key indicators to be included in a national system for the United States, GAO, in cooperation with the National Academies, convened the Forum on Key National Indicators on February 27, 2003, in Washington, D.C. The forum was an attempt to bring more valuable facts to bear on decision making by the public and its leaders. The forum was not intended to decide issues, set priorities, or determine resource allocations—which are the province of the nation’s duly chosen representatives. The purpose of the forum was to have a rich and meaningful discussion on whether and how to develop a key national indicator system for the United States by focusing on four key questions: • How are the world’s leading democracies measuring national performance? Page 7 GAO-03-672SP Key National Indicators Forum Forum • What might the United States do to improve its approach and why? • What are important areas to measure in assessing U.S. national performance? • How might new U.S. approaches be led and implemented? GAO and the National Academies designed this venture to bring together a multidisciplinary, multisector group of producers and users of public information with a wide variety of perspectives. The invited participants were national leaders and experts from the business, education, foundation, government, labor, media, minority, scientific, and statistics communities. Invitations were also extended to chairmen and ranking minority members of relevant congressional committees. (See app. I for a list of participants.) Comptroller General David M. Walker comoderated the forum with the Honorable Thomas Sawyer, former Congressman from Ohio.1 As agreed by the participants, the purpose of the discussion was to engage in an open, not for attribution dialogue. However, one participant is identified in the report because this individual provided a presentation that was critical to the forum’s discussion. Other than this one individual, this report summarizes the collective discussion and does not necessarily represent the views of any individual participant, GAO,or the National Academies. 1 Mr. Sawyer served in the U.S. House of Representatives from 1987 to 2002 and chaired the Subcommittee on Census, Statistics, and Postal Personnel, Committee on Post Office and Civil Service, in the 101st, 102nd, and 103rd Congresses. Prior to his congressional service, Mr. Sawyer served as Mayor of the City of Akron, Ohio, and as a Representative in the Ohio House of Representatives. Page 8 GAO-03-672SP Key National Indicators Forum Forum In addition to summarizing the forum participants’ collective discussion, this report highlights the research conducted in preparation for the forum and follow-on discussions with participants. Developing and preparing for the forum was an intensive 6-month effort. GAO staff, led by Christopher Hoenig, Director, Strategic Issues, researched indicator systems, conducted a series of interviews with producers and users of information, and wrote background papers on the history and state of the practice of indicator systems. A preparatory planning meeting and subsequent conversations were held with representatives of the National Academies to help frame the questions and objectives of the forum. Experts identified by the National Academies also prepared background papers for the forum. Also, GAO, in cooperation with the National Academies, commissioned Dr. Martha Farnsworth Riche2 to independently develop a straw proposal of a key national indicator system to facilitate discussion among the forum’s participants. A selected bibliography on indicator systems is included in appendix III, and selected Web sites on indicator systems are included in appendix IV of this report. Limitations and The dialogue as summarized in this report should be interpreted in the Qualifications Concerning context of five key limitations and qualifications. the Forum First, the forum was only an initial step in a possible long-term, evolving effort to develop and sustain a key national indicator system. Its purpose was to begin a dialogue on an extremely complex topic. Although many leaders, institutions and points of view were represented, many more will need to be involved—as follow-up efforts proceed—to start representing the extraordinary diversity of knowledge and opinion in our nation. This is especially true when it comes to choosing aspects of U.S. society for which it is important to develop indicators. Additionally, the involvement of the federal government, and particularly Congress, will be crucial. Second, even though GAO, in cooperation with the National Academies, conducted preliminary research and heard from national experts in their fields, a day’s conversation cannot represent the current state of the practice in this vast arena. More thought, discussion, and research must be 2 Dr. Riche served as Director of the U.S. Census Bureau from 1994 to 1998. Prior to being appointed Director, she was a founding editor of American Demographics, Director of Policy Studies for the Population Reference Bureau, and an economist with the U.S. Bureau of Labor Statistics. Dr. Riche is currently a Principal with Farnsworth Riche Associates. Page 9 GAO-03-672SP Key National Indicators Forum Forum done to develop greater agreement on what we really know, what needs to be done, and how to do it. Third, several presentations were made regarding (1) the lessons learned from other indicator efforts, (2) a proposal for a draft version of an indicator system for the United States, and (3) a potential organizational model in the areas of children and aging that could be replicated in other topical areas (e.g., public safety and governance). These presentations represented individual opinions, not a broad consensus or any formal endorsement by the cosponsoring or participating institutions. More collaborative work must be done to move from these starting points toward more definitive accomplishments. Fourth, any key national indicator system that would be developed as a result of follow-on efforts to the forum would, of necessity, build on the vast amount of current information already available, from the federal statistical system, the nonprofit and commercial sectors, and the many efforts currently operating below the national level. Many state, regional, and local governments and nonprofits working either in partnership or alone have developed and are using indicator systems. Yet at the same time, working on existing data alone would limit the opportunity to raise new questions and issues and develop new information sources. Fifth, because of the extraordinary diversity and quality efforts in specialized or topical information areas (e.g., education and health care) throughout the United States, this forum generally concentrated on bringing together generalists who could help think through how to organize a more comprehensive approach. As a result, a large number of leading edge individuals, institutions, and networks involved in specialized efforts could not be included for reasons of scope. This is an important limitation of the forum. Any successful effort to develop a national system must find a process and structure for including both specialized and comprehensive approaches. It must also build on and aid current efforts as well as developing new lines of effort. This has yet to be done and will require broad involvement of those specialized organizations that recognize the potential for mutual gain in such an effort. Page 10 GAO-03-672SP Key National Indicators Forum Forum Forum Summary The five key limitations and qualifications described earlier provide contextual boundaries. Nevertheless, the forum provided a rich dialogue on indicator systems and participants produced strong messages on each of the four questions. Those messages are highlighted below. Developing Key National While there is no generally accepted comprehensive, integrated indicator Indicators for the United system at the national level, a wide variety of indicator systems exist in the United States. However, these indicator systems either focus on States is Important specialized or topical areas, such as health or education, or focus on a regional, state, or community level. There was broad agreement among the forum’s participants that the issue of developing a key national indicator system is important but that further work needs to be done on what needs to change and why. • A straw proposal for a comprehensive indicator set—called USA Series 0.5—was presented at the forum, and participants acknowledged it to be a good starting point for building what might eventually be a broadly supported USA Series 1.0 indicator set. • Several possible models were discussed that could offer useful lessons for developing a national indicator system—including the leading economic indicators as well as indicator systems on health, children, and aging. • A broad range of issues were discussed that would need to be addressed to develop a useful key indicator system—including the need to define purpose and audience; the need for public outreach, sophisticated communications, and technology; and the importance of data availability and quality. A Broad Range of While the range of information areas that participants considered Information Areas Are important about the U.S. was extremely broad, there was little argument that an expedient first step is to try and assemble a set of “core” indicators Considered Significant from existing data to include within a national system. However, there was also significant enthusiasm about: (1) refining information areas included in the straw proposal and (2) identifying additional information areas. The term “information area” refers to a body of knowledge including existing data, questions, and ongoing research—that is meaningful in understanding U.S. society. Page 11 GAO-03-672SP Key National Indicators Forum Forum • A core group of information areas that could serve as a starting point for an evolving system—USA Series 0.5—was discussed. Independently developed by Dr. Riche, USA Series 0.5 included 11 information areas: community, crime, ecology, education, governance, health, the macroeconomy, security, social support, sustainability, and transparency. • To move to a USA Series 1.0, participants identified refinements to a majority of the USA Series 0.5 information areas. For example, participants thought the governance information area needed to include indicators on civic engagement. They also proposed the addition of 4 information areas: communications, diversity, individual values, and socioeconomic mobility. However, this list of information areas was not considered exhaustive since it was a first attempt to identify specific information areas to be included in a national indicator system. A Rich History of Indicator There is a long history of efforts around the world by leading democracies Systems Warrants Collective to develop and sustain indicator systems. However, no generally accepted, comprehensive approach yet exists in a society as large and diversified in Research its system of governance as the United States. Research on what can be learned from past and present systems would be essential to deriving useful implications for a possible United States system. • A multitude of specialized and comprehensive efforts are ongoing in the United States at the national, regional, state, and local levels as well as in other democracies—such as Australia and Canada. For example, within the United States, there is an indicator system to nationally assess the well-being of children and the Federal Interagency Forum on Child and Family Statistics annually reports on the data from this system. • Despite the activity, there appear to be few common sources of comprehensive research or communities of practice, either nationally or globally, to facilitate knowledge sharing. Furthermore, there are limitations in inferring lessons from countries of different size, diversity, and political-economic structures than the United States. • However, some lessons have already been learned. Clearly the purpose of measurement, the process of deciding what to measure, and determining who will truly benefit from the data are as critical as what to measure and how to define specific indicators and technical methods. Page 12 GAO-03-672SP Key National Indicators Forum Forum A United States Initiative Participants agreed that developing a key national indicator system would Must Build on Past Lessons require a combination of applying the essential lessons from past efforts and determining how to engage constructively with the many efforts and Current Efforts currently under way. • Any United States system must be flexible and evolve to allow for the rapid rate of change in our society, the complexity of the endeavor, and the wide variety of perspectives that will need to be reflected. • An effort to develop a key national indicator system must not supplant nor compete with the many existing efforts under way in the areas identified by participants but should build on them. • A comprehensive system for the United States must be appropriately focused, have a definable audience, be independent, pay attention to quality issues, and be adequately funded both in terms of its development and sustainability. • After the forum, an informal national coordinating committee of public and private sector institutions was constituted to begin organizing a national initiative and serve as the temporary means of facilitating dialogue, work and financing. Because this effort is in its early stages, the following list should not be misinterpreted as being complete or exclusive. It simply shows the institutions that, to date, have volunteered: the American Association of Universities (AAU), The Conference Board, the Council for Excellence in Government, GAO, the International City/County Management Association (ICMA), The National Academies, the National Association of Asian American Professionals, the Office of Management and Budget (OMB), and the White House Council on Environmental Quality. The committee’s first meeting will take place in the spring of 2003 in Washington, D.C. Page 13 GAO-03-672SP Key National Indicators Forum Forum How Are the World’s Because of the broad scope of this question, most of the material in this section represents preparatory research that was provided to participants Leading Democracies prior to the forum as background and context. A summary of this research Measuring National was presented briefly at the beginning of the forum. Performance? Many leading democracies around the world as well as major international institutions are involved in efforts to develop specialized and comprehensive indicator systems of societal performance. Specialized indicator systems focus on specific topics or information areas, such as health, education, or children while comprehensive indicator systems focus on several information areas, generally within the broader categories of economic, social, and environmental arenas. Additionally, a multitude of both specialized and comprehensive indicator systems are going on in the United States at the local, state, regional, and national levels. Some of these systems have been in place for decades and some have emerged in only the last few years. Some involved in these systems attribute the level of activity to the enabling possibilities created by data integration and presentation technologies (e.g., the World Wide Web). Others mention the increasing demand for cross-sector, cross-border responses to fiscal and other challenges that require new, integrated sources of data as well as new types of information. There appear to be few common sources of broad research on comprehensive systems (either on a national or global level) in a position to facilitate knowledge sharing. In contrast, there are numerous communities of practice dedicated to individual specialized systems. Forum discussion of indicator systems by other countries and the United States focused on the state of the practice of current indicator systems, observations on indicator systems, and past efforts in the United States to develop national indicator systems. State of the Practice of GAO found that a key aspect of the current state of the practice involves Current Indicator Systems comprehensive and specialized (i.e., topical) categories of indicator systems representing a wide range of maturities from formative to advanced. These systems also vary in the number of indicators, ranging from 19 to over 400. Several democracies, such as Canada and Australia, use comprehensive indicator systems and focus on information areas such as economic Page 14 GAO-03-672SP Key National Indicators Forum Forum opportunities and innovation, the strength and safety of communities, national wealth, and national income. Within these information areas are indicators ranging from real national net wealth per capita and real disposable income per capita to life expectancy at birth and literacy. Several states and communities within the United States, such as the State of Minnesota and the metropolitan area of Boston, also use comprehensive indicator systems. These indicator systems focus on information areas such as public safety, housing, and community and democracy and include indicators ranging from growth in gross state product and unemployment rate to volunteer time and prenatal care. Comprehensive indicator systems have two primary characteristics. One characteristic is creating an overall picture of how a community (or region, nation, etc.) is doing. The second characteristic is showing the interconnectedness of various key information areas, such as the interrelationship between economic development and environmental impact. Through both these characteristics, a comprehensive indicator system allows for a deeper understanding of what is really happening in a society and significantly broadens the availability of that knowledge. Different entities take an individualized approach to grouping together key specialized information areas. For example, Australia’s system includes biodiversity, crime, economic disadvantage and inequality, education and training, health, land, national income, national wealth, social attachment, water, and work. Table 1 provides details on several illustrative examples of comprehensive indicator systems regarding who reports the data, sources of the data, their purpose, the first year a system’s data were reported, and frequency of reporting updates. The table also identifies the scale of the system (i.e., national, regional, or local) that refers to the primary focus of the information being reported. However, larger scale efforts (e.g., national) can in some cases be cumulative, including state and/or local data. Page 15 GAO-03-672SP Key National Indicators Forum Forum Table 1: Illustrative Comprehensive Indicator Systems Year report Frequency of Indicator system Reported by Data sources Scale Purpose first issued report updates Canada’s Treasury Board Canadian National To provide information 1994 Yearly Performance 2002 of Canada government on trends in well-being agencies and the and to make Organisation for comparisons Economic Co- internationally. operation and Development Measuring Australian Australian National To provide information 2002 Annual updates Australia’s Progress Bureau of government on national progress in planned 2002 Statistics agencies economic, social, and environmental areas. Minnesota Minnesota Federal agencies, State To assess progress 1991 Periodicallyb Milestones 2002 Planninga Minnesota state toward achieving 19 agencies, and state goals in four universities areas: increasing the health and well-being of Minnesotans, enhancing community and democracy in the state, protecting the environment, and improving government. Achieving the Oregon Federal agencies State To provide information 1989 Biennially Oregon Shines Progress and Oregon state on the economic, Vision: The 2001 Boardc agencies social, and Benchmark environmental health Performance of the state in relation Report to its goals. The Wisdom of Our The Boston Federal agencies, Local To provide information 2000 Biennially Choices: Boston’s Foundation Massachusetts on the health and well- Indicators of state agencies, being of Boston, its Progress, Change Boston city neighborhoods, and and Sustainability agencies, the region as a whole. 2000 universities, and community-based organizations Source: GAO. a Minnesota Planning is a state agency created by the Minnesota legislature in 1991. b Updated in 1993, 1996, 1998, and 2002. c The Oregon Progress Board was created by the legislature in 1989 to develop and implement a state strategic plan. Page 16 GAO-03-672SP Key National Indicators Forum Forum Specialized indicator systems focus on specific subjects or topical areas, such as health status, the environment, the status of children, and aging on multinational, national, or local scales. Table 2 provides details on several illustrative examples of specialized indicator systems. Table 2: Illustrative Specialized Indicator Systems Year report Frequency of Indicator system Reported by Data sources Scale Purpose first issued report updates America’s Federal Interagency Federal agencies National To provide 1997 Annually Children: Key Forum on Child and information on the National Indicators Family Statistics well-being of of Well-Being 2002 children. Healthy People Department of Federal agencies National To provide indicators 1979 Once a decadea 2010 Health and Human of progress on a Services variety of health policy objectives. Kids Count 2002 The Annie E. Casey Federal agencies National To track the well- 1990 Annually Data Book Foundation being of youth. Older Americans Federal Interagency Federal agencies National To track the health 2000 Every 3 to 5 2000: Key Forum on Aging and well-being of years Indicators of Well- Related Statistics Americans aged 65 Being and over. The State of the The H. John Heinz Federal and state National To provide 2002 Annually; next Nation’s III Center for agencies, private information on the full addition in Ecosystems: Science, Economics organizations, and state of the 2007 Measuring the and the universities ecosystems of the Lands, Waters, and Environment United States. Living Resources of the United States The State of the UNICEF United Nations’ Multinational To present 1980 Annually World’s Children agencies, national information on the 2003 governments, and economic and social the World Bank well-being of children worldwide. The World Health World Health United Nations’ Multinational To measure the 1995 Annually Report 2002 Organization agencies, national amount of disease, governments, and disability, and health the Organisation that can be attributed for Economic Co- to certain risks and to operation and calculate how much Development of the burden is Statistics preventable. Source: GAO a Originally, published in 1979 as Healthy People: The Surgeon General’s Report, updated in 1980 as Promoting Health/Preventing Disease: Objectives for the Nation and in 1990 as Healthy People 2000: National Health Promotion and Disease Prevention Objectives. Page 17 GAO-03-672SP Key National Indicators Forum Forum Regardless of whether they are comprehensive or specialized, indicator systems vary in terms of data sources and organizations that report on these systems. As shown in tables 1 and 2, indicator systems can include data from a variety of sources such as the federal government, local government, and nongovernmental organizations. For example, Minnesota Milestones3 uses data from three primary sources: federal agencies, such as the U.S. Bureau of the Census and the U.S. Energy Information Administration; state agencies such as Minnesota’s departments of Revenue and Children, Families and Learning; and educational institutions, such as the University of Minnesota. Information on some systems are reported through government agencies, others through private organizations, and some use a combination of both. America’s Children4 and Measuring Australia’s Progress5 were both produced by government agencies. A private foundation with the extensive participation of government agencies produced The State of the Nation’s Ecosystems6 while another private foundation produced Kids Count.7 A private foundation, the city of Boston, and the Metropolitan Planning Council jointly produced The Wisdom of Our Choices.8 Healthy People 20109 was produced through a public-private partnership between federal 3 Minnesota Planning, Minnesota Milestones 2002 (St. Paul, Minn.: Minnesota Planning, 2002). http://www.mnplan.state.mn.us/mm/index.html (downloaded Jan. 2003). 4 Federal Interagency Forum on Child and Family Statistics, America’s Children: Key National Indicators of Well-Being 2002 (Washington, D.C.: Government Printing Office, 2002). http://childstats.gov (downloaded Jan. 2003). 5 Australian Bureau of Statistics, Measuring Australia’s Progress 2002: Indicators of Australia’s Progress http://www.abs.gov.au/Ausstats/abs%40.nsf/94713ad445ff1425ca25682000192af2/b66ebefc05 cdf265ca256bdc001223ec!OpenDocument (downloaded Jan. 2003). 6 The State of the Nation’s Ecosystems: Measuring the Lands, Waters, and Living Resources of the United States (Cambridge, U.K.: Cambridge University Press, 2002). http://www.heinzctr.org/ecosystems/index.htm (downloaded Jan. 2003). 7 Annie E. Casey Foundation, Kids Count Data Book 2002 (Baltimore, Md.: 2002). http://www.aecf.org/kidscount/kc2002/ (downloaded Jan. 2002). 8 The Boston Foundation, The Wisdom of Our Choices: Boston’s Indicators of Progress, Change and Sustainability 2000, http://www.tbf.org/boston/index.html (downloaded Feb. 2003). 9 Healthy People 2010: Understanding and Improving Health (Washington, D.C.: U.S. Department of Health and Human Services: 2000), http://www.healthypeople.gov/ (downloaded Dec. 2002). Page 18 GAO-03-672SP Key National Indicators Forum Forum agencies, local communities, and professional and trade associations from the health care field. One characteristic that many indicator systems share is that collaboration among various groups was important to their creation. Sometimes the cooperation was across government agencies and sometimes among nongovernmental organizations and government agencies. For example, Older Americans 200010 was produced by a coalition of nine federal agencies11 and supplemented by substantial contributions from three other federal agencies.12 Also, The Wisdom of Our Choices is another example of collaboration among various organizations. This indicator system is maintained and reported on by the Boston Foundation, the City of Boston, and the Metropolitan Area Planning Council with the assistance of local businesses, educational institutions, and community-based organizations. Observations on Indicator Four primary observations on indicator systems emerged based on forum Systems discussions and related research. The first observation concerns the purposes of indicator systems. Generally, there are, at a minimum, three broad purposes for indicator systems that are not mutually exclusive. These three purposes are as follows: • Accelerate learning: This type of indicator system contributes to scientific understanding as well as enhances the awareness, insight, and foresight provided to leadership and the public. • Assess position and progress: This type of indicator system involves a broad, constituent-focused aim and requires a generally accepted 10 Federal Interagency Forum on Aging Related Statistics, Older Americans 2000: Key Indicators of Well Being (Washington, D.C.: 2000), http://www.agingstats.gov/chartbook2000/default.htm (downloaded Jan. 2003). 11 These nine agencies are the Administration on Aging, the Bureau of Labor Statistics, the U.S. Bureau of the Census, the Department of Health and Human Services’ Office of the Assistant Secretary for Planning and Evaluation, the Health Care Financing Administration, the National Center for Health Statistics, the National Institute on Aging, OMB, and the Social Security Administration. 12 The three agencies are the Department of Agriculture’s Center for Nutrition Policy and Promotion, the Department of Justice’s Bureau of Justice Statistics, and the Department of Transportation’s National Highway Traffic Safety Administration. Page 19 GAO-03-672SP Key National Indicators Forum Forum common vision and comprehensive framework that helps uncover especially challenging problems and beneficial opportunities. • Measure performance: This type of indicator system is specifically intended to determine to what degree institutions or projects are successful and are producing appropriate benefits for the resources they use. The second observation was that, at the national level, social and environmental indicators have not received as much attention as other types of indicators. Specifically, traditional economic and business indicators have dominated indicator efforts. As an example, while there is a monthly report on economic indicators and estimates of Gross Domestic Product are released quarterly, reports on indicators of important social topics, such as teenage depression and suicides, are relatively more scarce and less frequent. Additionally, economic indicators give a limited view of how the country is doing. For instance, Gross Domestic Product, one traditional economic indicator, does not capture broad quality of life issues. The third observation was that many indicator systems focus mainly on objective measures as opposed to indicators that reflect the subjective perceptions of the public. For example, 29 of the 307 indicators included in France’s indicator report appear to be subjective measures. The United Kingdom’s indicator report has over 100 indicators, 3 of which are subjective measures. While both types of measures are derived using scientific methods, indicators that reflect subjective perceptions are viewed as important to include along with objective measures to provide an evaluation of the state of a city or a nation that takes account of diverse public points of view. The fourth observation was that criteria have been developed to help frame the design of national indicator systems. Specifically, several countries followed the so-called “Bellagio Principles” in developing their overall indicator systems.13 These 10 principles are that assessment of progress (1) are guided by a clear vision and goals, (2) review the whole system as well as its parts and recognition of the interaction among the parts, 13 These principles were developed as guidelines for the whole process—choice and design of indicators, their interpretation, and communication of results—to measure and assess progress toward sustainable development. They were developed in 1996 at an international meeting of measurement practitioners and researchers at the Rockefeller Foundation’s Study and Conference Center in Bellagio, Italy. Page 20 GAO-03-672SP Key National Indicators Forum Forum (3) consider equity and disparity within the current population and over generations, (4) have adequate scope, (5) have practical focus, (6) involves openness, (7) have effective communication, (8) involve broad participation, (9) be an ongoing assessment, and (10) provide institutional capacity. The Development of GAO found that the development of national indicators in the United States Indicators in the United over the last 75 years has followed three fairly discrete trajectories focusing on economic, social, and environmental issues. Major concerns States facing the nation provided the impetus for each of these trajectories and led to three indicator traditions. • The Great Depression and World War II put a host of economic indicators in wide currency. • The Great Society and civil rights movements enhanced efforts to fashion a wider body of social indicators. • The emergence of the environmental movement brought indicators to measure air and water quality. Solely for the purpose of illustration, table 3 selectively identifies highlights of these three indicator traditions during the 20th century. It is worth pointing out that the inherent strength of the current United States system is its diversity and flexibility. There are numerous specialized and comprehensive indicator systems, driven by either executive or grassroots leadership, in the public and private sectors that have shaped the variety of available information in our society. This table is not exhaustive, nor can it do justice to the diversity of those efforts. However, these highlights do demonstrate three—among possible others—recognizable traditions in the development of the United States’ indicator systems. Page 21 GAO-03-672SP Key National Indicators Forum Forum Table 3: Selected Highlights of Indicator Traditions in the United States during the 20th Century Tradition Illustrative examples Economic indicators National Income and Product Accounts were initially formulated to account for flow of commodities and services during World War II. They provide a base for key economic indicators such as Gross Domestic Product. Business Cycle Indicators, created in the 1930s by the National Bureau of Economic Research, have been compiled by the Conference Board since 1995. The Conference Board determines the specific data series included in the composite leading, coincident, and lagging indicators such as stock prices, employment, and change in consumer prices for services, respectively. The Employment Act of 1946a committed the federal government to the goals of full employment and economic stability. The act created the Council of Economic Advisors that, in 1947, released the first Economic Report to the President. Social indicators The U.S. Department of Labor, Children’s Bureau’sb Handbook of Federal Statistics of Children,c published in 1913, attempted to bring together “scattered” federal data and other information on children’s welfare. The Handbook was an early effort to develop indicators for consistent monitoring of children and health. A proposed bill called the Full Opportunity and Social Accounting Actd was first introduced in 1967. Although, the bill was never passed, it called for an annual social report from the President to the Congress and helped focus a national dialogue on social indicators. The Department of Health, Education and Welfare published a report, in 1969, on social indicators called Toward a Social Report.e The report was prepared at the direction of President Johnson who sought “ways to improve the nation’s ability to chart its social progress.” In 1973, the federal statistical agencies published a report on social indicators. Subsequent reports on social indicators were published in 1976 and 1980. Environmental indicators The National Environmental Policy Actf (NEPA) was signed into law on January 1, 1970, and required federal agencies to assess the impacts of their decisions on the natural environment. While NEPA did not establish any specific indicators, it does require that federal agencies assess the environmental effects of major federal actions significantly affecting the environment. NEPA also established the Council on Environmental Quality to advise the President on environmental matters and to annually report on the state of the environment. During the same year, the Environmental Protection Agency—an independent agency to establish and enforce federal air standards and water pollution control laws and to monitor the environment—was created. The Clean Air Act of 1970g was passed that year as well. These initiatives focused national attention on indicators of environmental quality. The Endangered Species Act of 1973h suggests indicators of species viability, such as size and geographical distribution of species’ populations and their habitats. These indicators can be used as the basis for avoiding the extinction of species. Source: GAO. a Pub. L. 79-304, Feb. 20, 1946. b The Children’s Bureau, created in 1912, is now located within the U.S. Department of Health and Human Services’ Administration for Children and Families, Administration on Children, Youth and Families. It is responsible for assisting states in delivering child welfare services. c U.S. Department of Labor, Children’s Bureau, Handbook of Federal Statistics of Children (Washington, D.C.: Government Printing Office, 1913). Page 22 GAO-03-672SP Key National Indicators Forum Forum d 90th Congress, S.843. e U.S. Department of Health, Education and Welfare (HEW) (Washington, D.C.: U.S. Government Printing Office, 1969). f 42 U.S.C. §§ 4321 – 4347. g 42 U.S.C. §§ 7401 – 7671q. h 16 U.S.C. §§ 1531 – 1544. While much of the development of national indicators in the United States has focused on specific economic, social, or environmental concerns, the importance of interrelationships among these dimensions is growing. For example, economists are working to develop new measures of economic performance that take into account various social and environmental costs. While initial interest in social indicators began as a challenge to the centrality of economic indicators in policy discussions, the focus of the social indicator tradition expanded through the development of frameworks to integrate economic as well as social indicators. Striving to understand the impact of human society on the environment involves focusing on the interrelationships among economic, social, and environmental processes. Page 23 GAO-03-672SP Key National Indicators Forum Forum What Might the United After reviewing research on how the world’s democracies are tackling indicator development, the bulk of the forum’s discussion turned to States Do to Improve implications and issues for the United States. Its Approach and Why? The participants generally agreed that an improved, more comprehensive approach to assessing the nation’s position and progress should be developed. They noted that such an approach should cover a wide variety of information areas—ranging from the macroeconomy and social support to education and health. In addition to identifying a variety of ideas for improving the nation’s approach, the Federal Interagency Forum on Child and Family Statistics was discussed as a potential process/structure model to emulate. A straw proposal of a indicator system for the United States was presented, leading to much fruitful dialogue on what can be done with existing data (e.g., on economics and children) as well as what should be done to develop new information in important, but neglected areas (e.g., personal and national security and socioeconomic mobility). Ideas to Improve the United The participants agreed that a more comprehensive system for measuring States’ Approach national performance could be beneficial. They also recognized that the process of generating indicators for a national system is as important as the specific indicators that would be identified and measured as a result of the process. Hence, the process should be as inclusive as possible, and this inclusiveness should show itself from the very beginning of any effort. For example, state and local governments should have significant roles and responsibilities in helping to develop and implement national indicators, in part because the federal government has devolved responsibilities for many social issues to state and local governments. Several additional ideas for improving the United States’ approach to measuring national progress were raised by participants. These ideas fell into four broad categories: (1) key questions for framing the agenda for a new system, (2) public outreach, (3) communication and dissemination, and (4) key data issues. Key questions for the future agenda: Participants proposed a variety of questions to help frame an agenda for the possible development of a key national indicator system for the United States. These questions included the following: Page 24 GAO-03-672SP Key National Indicators Forum Forum • What is the purpose and value of the national indicator system to be developed? In particular, what do we need to do differently, why, and what net risk-adjusted benefits might the system achieve? • Who are the audiences (e.g., general public, educators, policymakers, and professionals), and how will they benefit? • What would a broadly accessible and useful collection of key national indicators look like? • How would the indicator system be designed, developed, implemented, operated, used, improved, and communicated? In particular, how will the need to build short-term momentum be balanced against the need for longer-term persistence and perspective on the initiative? • What data exist to serve as a foundation for a national effort? Are there important data gaps, and what is the quality of the available data? • What is an appropriate standard for progress, and what are the potential unintended consequences or behavior changes from efforts to demonstrate progress? • What is the response system and how does it work when an indicator increases or decreases? Is there a response system in place to make use of national indicators in everyday life? • What are the experiences of other countries regarding unintended consequences of meeting performance measures? • Are there examples of how national indicators have been used to inform decision making? • How much time and how many resources will a national effort require? How will those resources be allocated to alternative uses, such as making existing indicators more widely available and usable by broader audiences versus building existing institutional capacity to produce more and better indicators? • Do the nation’s leading institutions (e.g., governmental, commercial, and nonprofit organizations) have the capacity to carry out this effort? Page 25 GAO-03-672SP Key National Indicators Forum Forum Public outreach: Developing an indicator system requires extensive outreach to targeted audiences. Such audiences could include some or all of the following: the general public, public leaders, the media, educational institutions, scientific and professional communities, and public interest groups. To be useful, indicators must have consistent form and be clear, easy to digest, user-friendly, and timely. The data also need to be provided in the appropriate context rather than merely presented in freestanding charts or tables. And there must be extensive attention paid to the processes, not only of audience understanding but assent to the importance of the information areas and indicators eventually chosen. Communication and dissemination: Communicating and disseminating information is essential to sustaining interest in any indicator system. However, this can involve significant time and resources. The media will play a key role in communication and dissemination. A carefully thought out approach to working with the media will be essential for any degree of success. Some organizations, such as the Annie E. Casey Foundation, commit significant financial resources to communicating and marketing indicator information. Another issue raised was a strategic question of how the “marginal dollar” of a key indicator effort should really be spent. For instance, in some cases it may be more valuable to communicate existing information for broader impact than to develop new sources of information. Key data issues: The federal statistical system and federal programs produce much data, and they are relied on by the nation. The data are widely accepted because they are “official.” The data are produced using generally accepted practices and principles and are based on sound statistical methodologies for the purposes for which the data were produced. There is also a substantial amount of data produced by the private sector, of which an important component is viewed as proprietary, not public. Hence, two key data issues are quality and availability. In terms of quality, since there is a known quality of official statistics and sometimes an unknown quality of private statistics, how can this variation in knowledge of data quality be addressed so both can be used for a national indicator system? In terms of availability, if certain proprietary information were essential for assessing the position and progress of the nation, how would those data be made more widely available? There needs to be a collective effort to address both of these data issues if a national indicator system is to be successful. Page 26 GAO-03-672SP Key National Indicators Forum Forum A Process/Structure Model Several existing models could be used as reference points for designing a for Developing a National United States indicator system. Two current examples cited were the federal interagency forums on (1) child and family statistics and (2) aging- Indicator System related statistics. A more detailed discussion centered on the Federal Interagency Forum on Child and Family Statistics. This was presented as an example from the United States’ federal statistical system of a process and organizational structure for developing indicators within a specialized area. It should be noted that, at this stage, little discussion of these models concerned the crucial issue of funding, which will be vital to elucidate in order to make any practical progress on applying their lessons to a national effort. In 1994, seven agencies joined together to create the Federal Interagency Forum on Child and Family Statistics (Interagency Forum). Three years later the Interagency Forum was formally established by an Executive Order and directed to develop priorities for collecting enhanced data on children and youth, improve the reporting and dissemination of information on the status of children to the policy community and the general public, and produce more complete data on children at the state and local levels. The Interagency Forum now has participants from 20 agencies as well as partners in private research organizations. It holds several public hearings with agency members each year to discuss key issues and ideas. Annually, the Interagency Forum produces a report called America’s Children: Key National Indicators of Well-Being. The 2002 report is the group’s sixth annual report to the nation on the condition of children in America. The Interagency Forum’s report presents 24 key indicators on important aspects of children’s lives, including their economic security, health, behavior and social environment, and education. It also presents eight contextual measures that describe changes in the characteristics of the population as well as in children’s family settings and living arrangements. The Interagency Forum chose the indicators through careful examination of available data. Data were drawn from national surveys and vital records. Input on which indicators to choose was also sought from the federal policy-making community, foundations, academic researchers, and state and local children’s service providers. The implication of this discussion was that the Interagency Forum could be studied and replicated as a model for other information areas for the United States’ new approach. Page 27 GAO-03-672SP Key National Indicators Forum Forum A Straw Proposal for a New Commissioned by GAO, in cooperation with the National Academies, Dr. National Indicator System Riche produced an independent straw proposal for a key national indicator system called USA Series 0.5. USA Series 0.5 served as a starting point for the forum’s participants to discuss the framework of a key national indicator system and, in the future, move to the next version of an indicator system. Moving to an initial version of a national indicator system, identified as USA Series 1.0, would involve formal and institutional consensus, audience input, and would be the first step toward an evolving key national indicator system. Dr. Riche developed a group of draft principles for developing a key national indicator system. These principles included the following: • The set of indicators is about the nation, not just the government: Defining key national indicators goes beyond any one sector or level of government. • If the set of indicators is about the nation, it must incorporate the nation’s components. Local, state, regional and federal governments, as well as private for-profit and not-for-profit sectors should work to coordinate and integrate their own efforts into a national perspective. • If the set of indicators is national and intended to drive decision making, it must be comprehensive. It should be comprehensive, not just specialized and it must integrate the links and interactions between component measures. • If the set of indicators is to be useful, the information must be targeted and trusted. The set of indicators should be selected based on specific criteria. These criteria might include the significance, objectivity, accuracy, scope, timeliness, accessibility, clarity, efficiency, comparability, and contextual sophistication of a set of indicators. • If the set of indicators is to be credible, it must be both science-based and understandable. The set of indicators should help formulate questions about what knowledge is needed so sensible scientific statements can be made and a framework on key areas of research and investigation can be developed. • If the set of indicators is to be used to monitor progress, the public must be both involved and included. This principle implies a need for polling and related research to define what Americans want for their country. Page 28 GAO-03-672SP Key National Indicators Forum Forum • If the set of indicators is to have staying power, it must acknowledge the reality of resource constraints and the corresponding need for judgment and compromise. A true national effort will need to be based on some type of public/private partnership. No one sector of society can “own” the effort. Additionally, three basic types of indicator approaches were described since the United States’ current approach for measuring performance includes all three approaches. These three approaches are as follows: • Composite indicators: This approach combines information from several different indicators into a single composite number. An example of this approach is the United Nations’ annual Human Development Index. The composite approach is a tool for communicating directional progress to a large audience, especially in a comparative context. However, composite numbers require a consensus on weighting the different indicators that is hard to achieve. • A unified, balance sheet of indicators: This approach uses an accounting framework and presents data in a unified system of accounts. Under this approach, the indicators are both gathered and presented within a coherent hierarchical system. Most countries have a similar set of economic accounts, such as the United States’ National Income and Product Accounts, that are linked at a certain level of detail by the United Nations-sponsored System of National Accounts. • A suite of indicators: This approach groups information areas and key indicators together. Through use of a suite approach, the links between the information areas can be discussed even though not every information area needs to be fully developed as a measure. What the proposal is: USA Series 0.5 specifically addresses areas in which national performance might be measured. It includes a suite of indicators that has been prominent in past efforts in this country. It also includes indicators from other countries whose economic and social systems are, in some respects, comparable to our own. The proposal includes information areas that are developing as well as those that are advanced. What the proposal is not: USA Series 0.5 is not systems based because it lacks a firm mission statement and conceptual framework and does not depend on any particular structure. It does not have an identified audience. USA Series 0.5 does not presume to be complete in terms of including the Page 29 GAO-03-672SP Key National Indicators Forum Forum many information areas that might likely be incorporated into later versions. It does not attempt to propose indicators for new or “formative” information areas that have, by definition, large knowledge gaps because they are taking shape based on new questions being asked about our nation and our world. Overview of the proposal: USA Series 0.5 consists of three broad categories—economic, social, and environmental. Within these three categories are a variety of information areas that are classified, in Dr. Riche’s opinion, by how well data associated with a specific information area are developed. The three classifications of the development stages of information areas are (1) advanced, (2) developing, and (3) formative. This development construct is not tested and there is no consensus as to which information areas are most advanced. The specific structure of the proposal—including 11 specialized information areas—is shown in table 4 below. The proposal also included specific indicators for the advanced and developing information areas, such as crime, ecology, education, and governance. (See app. II.) There were no specific indicators chosen for the information areas that were considered to be in the formative stage (e.g., sustainability, transparency, and security). Table 4: Structure of USA Series 0.5 Information area development stage Economic Social Environmental Advanced Macroeconomy Education Health Crime Social Support Developing Community Ecology Governance Formative Sustainability Sustainability Sustainability Transparency Transparency Security Source: Dr. Martha Farnsworth Riche, The United States of America Developing Key National Indicators. (Paper presented at the forum.) In principle, advanced information areas have a great deal of reliable data and relatively broad public and scientific consensus as to their importance and method of production. Figure 1 shows data for life expectancy, which illustrates the characteristics of an advanced information area. Specifically, life expectancy data are based on well-organized bodies of Page 30 GAO-03-672SP Key National Indicators Forum Forum reliable data and there is a high degree of scientific and political consensus on their significance. Figure 1: Life Expectancy at Birth and at Age 65 90 Years of life 80 70 60 50 40 30 20 10 0 1900 1920 1950 1960 1970 1980 1990 2000 Year At birth At age 65 Source: National Center for Health Statistics. Note: From 1900 to 1950, no data were available for years of life at age 65. In contrast, developing information areas have a higher proportion of indicators that are evolving and lack a broad technical or public consensus about significance. Greenhouse gas emissions, as shown in figure 2, is an example of a developing information area. While these emissions have gained increasing currency in debate and policy making, they are based on estimates of component gases and there is some scientific and public uncertainty about their importance and possible implications. Page 31 GAO-03-672SP Key National Indicators Forum Forum Figure 2: U.S. Emissions of Greenhouse Gases, Based on Global Warming Potential, 1990-2000 (in Million Metric Tons of Carbon Equivalent) 2000 Million metric tons 1500 1000 500 0 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 Year Carbon dioxide Methane Nitrous oxide Hydrofluorocarbons, perfluorocarbons, and sulfur hexafluoride Total Source: Energy Information Administration. Formative information areas are new areas, on issues of potential significance, for which data may not exist. They need pilot projects to outline suitable information databases. An example of a formative information area would be a measure of public confidence in personal security. While there are some data on elements of public confidence in personal security, a broad consensus on the definition of personal security does not exist. Reactions to the Proposal While the participants were receptive to the proposal as a starting point, it prompted several reactions. One reaction was that attention should be paid to the process of developing the next version. While people acknowledged the expediency and practicality of starting with existing data, most participants wanted to work on filling in the gaps between versions 0.5 and 1.0. A participant suggested that the process that produced the “Healthy People 2010” initiative could be a possible model of the effort. The “Healthy People 2010” development process was described Page 32 GAO-03-672SP Key National Indicators Forum Forum by some as having been exhaustive. It had several advantages, including an organized approach to automation and a human capital infrastructure at the federal, state, and local levels. Another reaction, related to the one above, was to suggest that it is important to have a mechanism or process that would allow for creating new indicators and/or revising existing indicators. This mechanism or process would need to capture the public’s changing concerns and other changes nationally and internationally. While it would be a challenge to do, one suggestion to address the challenge was for the system to have a set of “regular” indicators that will remain meaningful over time and a set of “special” indicators that apply in specific situations. A third reaction focused on the need to disaggregate data, which participants considered an important but challenging task. Although aggregated information at the national level needs to be presented, the data also need to be disaggregated into specific categories that are relevant to localities and the public. This capability will also allow localities to relate to and understand how they fit into a larger picture. Health indicators were provided as an example. It is useful to have national data on health. But the information is even more useful if it provides information about health in a specific city or neighborhood. Also, careful consideration should be given as to how data are disaggregated, since there can be degradation of quality due to smaller sample sizes. Applying the lenses of age, race, gender, and geographical location to indicators facilitates identifying trends among specific groups that are masked in aggregated data. Page 33 GAO-03-672SP Key National Indicators Forum Forum What Are Important While participants generally agreed that the straw proposal’s 11 information areas were important to measure, they thought a majority of Areas to Measure in the areas needed refinement and enhancement. (For the specific Assessing U.S. National indicators defined in the proposal, please see app. II) Also, 4 additional information areas were identified as candidates for including in the Performance? proposal. The list of information areas that needed to be refined or added to the proposal was not intended to be exhaustive. Furthermore, participants did not discuss how many indicators might be included under each area. Instead, the proposal was meant to represent a good place to start to build a more comprehensive national indicator system. Information Areas That Participants identified seven information areas included in the 0.5 proposal Need Further Refinement that need to be further refined. These areas are community, education, governance, macroeconomy, security, social support, and sustainability. Community Discussion on this information area focused on refining the area to include the concept of civic engagement. Civic engagement was described as the connectedness of individuals to society and to each other. Further, it involves social capital such as participation in social and cultural organizations, public service, volunteering, and voting. Some participants viewed civic engagement as an undermeasured and underappreciated area, while others feel there is already a good deal of information available. This suggests, as may be the case in other areas, the value of undertaking systematic inventories of existing data. While some participants thought civic engagement should be included within the community information area, others thought it should be within the governance information area. Education Discussion on this information area focused on the need to include both the means by which individuals can attain personal enrichment and improved quality of life and, at the societal level, the extent to which society is served by the educational system. Governance Some participants viewed this information area as being not as well thought out as others. It should incorporate the concepts of public trust and its relationship to institutions. It may also need to cover the effectiveness of governance (e.g., political participation and corruption). Within the concept of public trust, there was some discussion about the Page 34 GAO-03-672SP Key National Indicators Forum Forum need to measure elements of freedom or patriotism, security and civil liberties. The discussion of public trust and institutions encompassed both corporations and government at all levels. Additionally, this concept relates to the responsiveness of institutions to the perceived needs of the public. Macroeconomy Discussion on this information area focused on two broad topics: (1) the nation’s role in the world and (2) its economic well-being (e.g., competitive advantage). Measuring the nation’s role in the world involves portraying and tracking the nation as a member of the world community. This includes such issues as foreign aid by government and private organizations, United States military presence overseas, anti-American sentiments, and economic and cultural globalization. The concept of competitive advantage involves measuring key economic areas, such as technology; innovation; the mobility and flexibility of the nation’s labor force (e.g., geographic, career, and social class mobility); the ability to attract, retain, and develop good people; education; and trade. It was thought that measuring the nation’s competitive advantage should involve national, regional, state, and community levels. Security Participants identified this information area as needing focus on personal, community, national, and international security. This area was identified as a new area of focus for the nation since the terrorist attacks of September 11, 2001, and other terrorist threats. The area could be approached as a subjective measure of individuals’ sense of security and as an objective measure of the extent to which important infrastructure systems are secure. There were different views as to whether personal security should be included within the security information area. Some participants thought personal security was already included in the crime information area of the proposal. Others thought that personal security should be included within the security information area since it was viewed as an expanded notion of public safety given the new focus on the issue, especially safety from terrorist threats. Social Support Participants identified this information area as needing to include both the well-being of children and the well-being of the elderly—those classified by society as dependents. Ever increasing attention is being given to the importance of investment in all aspects of child development (e.g., the No Child Left Behind Act of 2001). The well-being of the elderly, a complex but Page 35 GAO-03-672SP Key National Indicators Forum Forum pressing issue given the well known aging of the baby boomers, is also receiving increased attention. Sustainability Participants viewed this information area as one area that does not fall clearly within a distinct information area. It involves several information areas and encompasses the proposal’s three broad information categories-- economic, social, and environmental. Sustainability involves the concept of leaving a legacy for future generations as well as the notion of environmental and social capital and liabilities. Information Areas to Add to Participants identified four areas that should be included in the proposal: the Proposal communications, diversity, individual values, and socioeconomic mobility. Communications Discussion on this area involved the various forms of media industry—such as television, radio, newspapers, and the Internet—and how to determine the impact on, and accountability of, the industry. One possible measure suggested involved the degree of citizen access to various media sources or types of communication (e.g., the Internet). Diversity Participants saw this area as one area that goes beyond the concept of fairness to encompass the pluralistic nature of our society. Diversity, in all its forms (e.g., population, culture, and points of view), was seen as an asset that is consistent with the founding principles of the nation and vital to its health. It was agreed that this area would be possible, but very difficult, to measure. Individual Values This area cuts across other areas such as community and quality of life. However, because the concept of individual values does cut across other areas, no agreement was reached as to whether the area should stand alone or be included under several other areas. It was generally agreed though that individual values could be measured. Socioeconomic Mobility Some participants defined this area as access to opportunities such as education and jobs. However, when measuring access to jobs there needs to be a distinction between access to good jobs and bad jobs. For others, this concept had a broader meaning relating to how rapidly individuals in Page 36 GAO-03-672SP Key National Indicators Forum Forum the United States are moving from one standard of living to another. This area was seen as being rooted in basic American values and as an essential component of the “American dream.” Page 37 GAO-03-672SP Key National Indicators Forum Forum How Might New U.S. The last segment of the forum was devoted to discussing how to take meaningful action on such a challenging idea. Recognizing that any effort Approaches Be Led pertaining to such a large, diverse nation would of necessity be long term, and Implemented? the participants’ discussion focused on how to develop, sustain, and fund an effort. Development Information areas tend to develop in an evolutionary fashion, over relatively long periods, and in some definable stages as they reach greater degrees of consensus and transparency. There was wide acknowledgment that any United States indicator system, because of the rapidly changing nature of our society, would by definition be evolving. In other words, there would be many successive “versions,” each one improving on the one before it. Table 5 illustrates the possible evolution of a United States indicator system by showing how, over time, additional information areas can be added and developed in a cumulative fashion. Specifically, Series 1.0-2.0 would contain not only new information areas (e.g., energy and citizenship) but also the information areas from earlier versions (such as education and community in Series 0.5). This table, shown as it was presented at the forum, includes some but not all areas mentioned by the participants as well as some used by other nations. Page 38 GAO-03-672SP Key National Indicators Forum Forum Table 5: Proposed Evolutionary Process for a National Indicator System (Includes Illustrative Information Areas) Information area Series 1.0-2.0 Series N.0 development stage Series 0.5 (cumulative) (cumulative) Advanced Crime/public safety Energy Community Education Labor and Ecology Health employment Governance Macroeconomy Microeconomy Social support Resource allocation Special populations Developing Community Citizenship Families Ecology Competitiveness Innovation Governance Infrastructure Justice Wealth/prosperity Knowledge Well-being/happiness Formative Security Accountability Arts and Culture Sustainability Cities Civility Transparency Diversity Freedom Equity Mobility Globalization Opportunity Neighborhoods Trust Values Source: Dr. Martha Farnsworth Riche, The United States of America Developing Key National Indicators. (Paper presented at the forum.) There was discussion about the scope of a “comprehensive” indicator set. Participants observed that a comprehensive set of indicators would include both indicators that are fairly advanced in their development as well as indicators that are new and thus require substantial development. It would include indicators at all levels, in all sectors, and in all disciplines. Furthermore, it would focus on both areas where much is already known as well as areas needing further research. This would help create a learning agenda for developing and sustaining indicators. Sustaining and Funding The discussion of sustaining and funding the effort to develop a key national indicator system focused on the need for forming a public/private partnership. Such a partnership would need a structure that provided a broad, independent, flexible, and responsive base for the effort. Participants expressed the view that unless the effort is a partnership between public and private entities, it will not be sustained. Participants agreed that creating a national coordinating committee and a variety of task forces was the best governance mechanism with which to Page 39 GAO-03-672SP Key National Indicators Forum Forum initiate a national indicators initiative. Such an initiative would include all forum participants and representatives from any other institutions interested in the effort would be invited to participate. As of May 7, 2003, the organizations that have agreed to participate are: AAU, The Conference Board, the Council for Excellence in Government, GAO, ICMA, The National Academies, the National Association of Asian American Professionals, OMB, and the White House Council on Environmental Quality. The consensus was that the national coordinating committee should be viewed in a facilitative and catalyst role as opposed to a day-to-day leadership role. To help promote this role of the committee, it was agreed that an individual committee member would chair the committee for one meeting and then the chairmanship would rotate to another member. The first meeting of the national coordinating committee will be held in the spring of 2003 at the offices of the Council for Excellence in Government and be chaired by the Council’s President. The national coordinating committee’s most important tasks will probably involve providing effective communications, creating a strategy, and obtaining resources. Specifically, the committee will need to focus on articulating the national indicator initiative’s purpose, organizing process and structure, agreeing on governing principles, identifying additional participants, and developing short- and long-term plans. One participant noted, and the rest agreed, that while they voted to move forward with this effort, there were many conceptions of what “this effort” is. Participants noted that objectives need to be clarified and agreement reached on how to proceed. They agreed that developing a charter and set of principles for the initiative would be addressed at the first meeting of the coordinating committee. Committee task forces could focus on a variety of issues such as conducting research, creating short- and long-term action plans, capitalizing operations, identifying possible organizational models to sustain the effort in the long term, and investigating communications and technology solutions. To be successful, people with different skill sets will need to be brought into the effort. Specifically, experts in communication and technology, not just experts in data and indicators, need to be involved. The exact number and functions of all the task forces have yet to be decided. Page 40 GAO-03-672SP Key National Indicators Forum Forum The funding arrangement for this effort was cited as a tremendously important issue since it will require a substantial amount of time and resources to start and sustain a true national effort. A brief discussion on the topic concluded that a range of possible funding alternatives, including private sources and federal funds, need to be studied. Page 41 GAO-03-672SP Key National Indicators Forum Appendix I Forum Participants AA ppp ep ned n x id e x Iis Derek Bok The 300th Anniversary University Professor and President Emeritus, Harvard University Donald Borut Executive Director, National League of Cities Charles Bowsher Former Comptroller General of the United States, U.S. General Accounting Office Heinrich Brungger Director, Statistics Division, The United Nations Economic Commission for Europe Philip M. Burgess President, National Academy of Public Administration Lala Camerer Deputy Director, Global Access Division, Center for Public Integrity Richard Cavanagh President, The Conference Board E. William Colglazier Executive Officer, The National Academies Rita Colwell Director, National Science Foundation Michael Delli Carpini Director, Public Policy Program, The Pew Charitable Trusts Gene Dodaro Chief Operating Officer, U.S. General Accounting Office Cynthia Fagnoni Managing Director, Education, Workforce, and Income Security, U.S. General Accounting Office Scott Farrow Chief Economist, U.S. General Accounting Office Harvey Fineberg President, Institute of Medicine, The National Academies William Galston Professor and Director, University of Maryland Institute for Philosophy and Public Policy Gaston Gianni Vice-Chair, President’s Council on Integrity and Efficiency and Inspector General, Federal Deposit Insurance Corporation Page 42 GAO-03-672SP Key National Indicators Forum Appendix I Forum Participants John Graham Administrator, Office of Information and Regulatory Affairs, Office of Management and Budget Robert Groves Director, Survey Research Center, University of Michigan Judith Gueron President, Manpower Demonstration Research Corporation Hermann Habermann Deputy Director, U.S. Bureau of the Census Nils Hasselmo President, Association of American Universities Harry Hatry Director, Public Management Program, The Urban Institute Theodore Heintz White House Council on Environmental Quality Christopher Hoenig Director, Strategic Issues, U.S. General Accounting Office Helen Hsing Director-Special Projects, Office of the Comptroller General, U.S. General Accounting Office Robert Kaplan Professor, Harvard Business School Nancy Kingsbury Managing Director, Applied Research and Methods, U.S. General Accounting Office Andrew Kohut Director, Pew Research Center for the People and the Press John Koskinen Deputy Mayor/City Administrator, District of Columbia Bill Kovach Chairman, Committee of Concerned Journalists Risa Lavizzo-Mourey President and CEO, The Robert Wood Johnson Foundation Rosemary Marcuss Deputy Director, Bureau of Economic Analysis Sylvia Mathews Chief Operating Officer and Executive Director, The Bill & Melinda Gates Foundation Page 43 GAO-03-672SP Key National Indicators Forum Appendix I Forum Participants Thomas McCool Managing Director, Financial Markets and Community Investment, U.S. General Accounting Office Patricia McGinnis President and CEO, The Council for Excellence in Government Sara Melendez Professor of Nonprofit Management, The George Washington University and Former President, The Independent Sector Alex Michalos Director, Institute for Social Research and Evaluation, University of Northern British Columbia Marc Miringoff Founder and Director, The Fordham Institute for Innovation in Social Policy Daniel Mulhollan Director, Congressional Research Service Janet Norwood Former Commissioner, Bureau of Labor Statistics Robert O’Neill Executive Director, International City/County Management Association Robert Parker Chief Statistician, U.S. General Accounting Office Victor Rezendes Managing Director, Strategic Issues, U.S. General Accounting Office Martha Farnsworth Riche Former Director, U.S. Bureau of the Census Dorothy Ridings President and CEO, Council on Foundations John Rolph Chairman, Committee on National Statistics, The National Academies Thomas Sawyer Former Representative, State of Ohio, 14th District William Scanlon Managing Director, Health Care, U.S. General Accounting Office Max Singer Board Member and Senior Fellow, Hudson Institute Pete Smith President and CEO, Private Sector Council Page 44 GAO-03-672SP Key National Indicators Forum Appendix I Forum Participants Edward Sondik Director, National Center for Health Statistics, Department of Health and Human Services Edward Spar Executive Director, Council of Professional Associations on Federal Statistics Jeffrey Steinhoff Managing Director, Financial Management and Assurance, U.S. General Accounting Office F. Michael Taylor President, National Association of Local Government Auditors Michael Teitelbaum Program Director, Alfred P. Sloan Foundation Dennis Trewin Chief Statistician, Australia Kathleen Utgoff Commissioner, Bureau of Labor Statistics David M. Walker Comptroller General of the United States, U.S. General Accounting Office Katherine Wallman Chief Statistician of the United States, U.S. Office of Management and Budget Andrew White Director, Committee on National Statistics, The National Academies Jacquelyn Williams-Bridgers Managing Director, Strategic Planning and External Liaison, U.S. General Accounting Office Vincent Yee National President, National Association of Asian American Professionals Randall Yim Managing Director, Homeland Security and Justice, U.S. General Accounting Office Page 45 GAO-03-672SP Key National Indicators Forum Appendix II Illustrative Indicators by Information Area for USA Series 0.5 1 Appendx Ii Community Information Area1 • Rate of volunteering, through nonprofit or charitable organizations • Youth rates of volunteering • Charitable contributions as a percentage of incomes • Attendance at events and institutions that address the national heritage (such as monuments, historical sites, and national parks) • Attendance at performing arts, by categories • Participation in organized sports • Voting rates • Reported hate crimes • Allocation of free time • Homelessness Crime/Public Safety Information Area • Crime victimization rates (by subgroups such as age, sex, and race/ethnic origin) • National crime rate • Violent crime rate • Property crime rate • Incarceration (as percentage of population, by age rates and by race/ ethnic origin) • Deaths due to transportation accidents • Deaths due to fires • Proportion of jail inmates who committed offense to get money for drugs • Percentage of working age population providing protective services • Percentage of population afraid to walk alone after dark Ecology Information Area • Level of nitrogen oxide as a percentage of acceptable levels • Level of sulfur oxide as a percentage of acceptable levels • Level of carbon dioxide as a percentage of acceptable levels • Per capita water consumption • Some measure of water quality, for example, percentage of population with access to safe drinking water 1 The indicators and information areas for USA Series 0.5 were proposed by Dr. Martha Farnsworth Riche to facilitate a dialogue among the forum’s participants. Page 46 GAO-03-672SP Key National Indicators Forum Appendix II Illustrative Indicators by Information Area for USA Series 0.5 • Change in status of species at risk of loss • Protected areas as a proportion of vulnerable areas • Emissions of greenhouse gases per capita • Net greenhouse gas emissions per Gross Domestic Product • Reduction of emissions of toxic substances Economic Information Area • Real Gross Domestic Product (GDP) • Real GDP per employed person • Labor force participation • Unemployment • Expenditures on Research and Development as a share of GDP • Real disposable income per capita • Median household net worth • Composition of wage rates (good jobs/bad jobs) • Poverty • Home ownership Education Information Area • Percentage of the population aged 25 and over that has completed postsecondary education • National Assessment of Educational Progress (NAEP) or other measure of literacy equivalent to high school graduation • Percentage of the population aged 15 to 29 that is neither enrolled in nor has completed high school • Enrollment in science and engineering (National Science Foundation) • Mathematics test scores (NAEP) • Percentage of population with computer literacy and computer access • Safe schools • Gap in attainment by race and ethnic origin and other relevant factors (e.g., disability) • Adult education participation/access Governance Information Area • Proportion of high elected offices (Congress, mayors, governors, etc.) held by women, minorities, etc. • Proportion of high-appointed offices held by women, minorities, etc. • Information about the “legal enforcement of constitutional guarantees of civil liberties” Page 47 GAO-03-672SP Key National Indicators Forum Appendix II Illustrative Indicators by Information Area for USA Series 0.5 • Civil rights: Enforcement data? Prevalence of complaints? • Successful management of the voting franchise—for example, proportion of ballots that are disqualified • Some measure of tax expenditures that reflects how effective the government is in taking care of the citizenry • Some measure of how well government agencies are providing fair access to public services and utilities • Some measure of how the law treats/does not treat Americans equally • Some measure of the existence of an effective safety net • Proportion of residents who believe that the nation is “on the right track” Health Information Area • Overweight and obesity • Life expectancy—at birth, at different policy-relevant ages • Health/active life expectancy • Infant/child/youth mortality (i.e., successful survival to adulthood) • Disability limitations—as represented by inability to perform normal activities of daily living • Physical activity • Tobacco use • Substance abuse • Immunization • A measure of access to health care—availability, affordability, etc., for example, personal expenditures for health care as a percentage of per capita income Social Support Information Area • Elderly living alone and in poverty • Proportion of elderly for whom Social Security is more than a “floor” • Older Americans who are involuntarily unemployed • Housing costs as a percentage of income for older Americans • Percentage of older Americans unable to perform certain physical functions • Proportion of children receiving child care, by source • Proportion of children whose diet is “poor” • Proportion of youth ages 16 to 19 neither enrolled in school or working • Adolescent birth rate • Family reading to young children Page 48 GAO-03-672SP Key National Indicators Forum Appendix III Selected Bibliography on Indicator Systems Appendx iI Comprehensive Indicator Abbott, Robert M., Scott D. Johnson, and Tracy Dieckhoner. Embedding Systems Sustainability in the Business of City Government: An Opportunity for Seattle (Vancouver, Canada: Abbott Strategies, ND). http://www.abbottstrategies.com/Papers/pdf/embedsustain.pdf. (downloaded May 2003). Embedding Sustainability in the Business of City Government discusses ways to create a framework for integrating issues of sustainability into decision making in Seattle city government. Australian Bureau of Statistics. Measuring Australia’s Progress 2002:Indicators of Australia’s Progress (Canberra, Australia: 2003). http://www.abs.gov.au. (downloaded May 2003). Measuring Australia’s Progress uses a discussion of human capital, social capital, natural capital, and financial capital indicators to asses the extent to which Australia has progressed. Bok, Derek. The State of the Nation. Cambridge, Mass.: Harvard University Press, 1996. The State of the Nation examines the areas of economic prosperity, quality of life, equality of opportunity, personal security, and societal values, and compares the progress made in these areas with progress made in other countries. The Boston Foundation. The Wisdom of Our Choices: Boston’s Indicators of Progress, Change and Sustainability 2000. (Boston, Mass.: 2000). http://www.tbf.org/boston/boston-L1.asp. (downloaded May 2003). The Wisdom of Our Choices provides indicators of civic involvement, the economy, education, public health, and other measures of well- being. Chang, Ping. State of the Region 2002: Measuring Progress in the 21st Century (Los Angeles, Calif.: Southern California Association of Governments, 2002). http://www.scag.ca.gov/publications/index.htm (downloaded May 2003). State of the Region 2002 assesses southern California’s performance with respect to three overall goals: raise the standard of living, enhance the quality of life, and foster equal access to resources. Page 49 GAO-03-672SP Key National Indicators Forum Appendix III Selected Bibliography on Indicator Systems Chicago Metropolis 2020. 2002 Metropolis Index (Chicago, Ill.: 2002). http://www.chicagometropolis2020.org/10_5.htm. (downloaded May 2003). The 2002 Metropolis Index is intended to give residents of the region benchmarks to assess how the region is doing, and to help them consider what must be done to sustain the region’s status as a globally competitive region. Committee on Geography, Committee on Identifying Data Needs for Place- Based Decision Making. Community and Quality of Life: Data Needs for Informed Decision Making (Washington, D.C.: National Academy Press, 2002). http://www.nap.edu/catalog/10262.html. (downloaded May 2003). Community and Quality of Life examines the concept of livable communities, the selection of livability indicators, data needs, and measurement and analysis issues related to the indicators. The Conference Board of Canada. Performance and Potential 2001-2002. Ottawa, Canada: 2002. Performance and Potential 2001-2002 updates the top 40 performance indicators and demonstrates that overall performance remains average among the six countries used to benchmark Canada’s performance. Global Reporting Initiative. 2002 Sustainability Reporting Guidelines (Boston, Mass.: 2002). http://www.globalreporting.org/guidelines/2002.asp. (downloaded May 2003). The 2002 Sustainability Reporting Guidelines organizes “sustainability reporting” in terms of economic, environmental, and social performance (also know as the triple bottom line). Maine Economic Growth Council. Measures of Growth 2002 (Augusta, Maine: 2002). http://mdf.org/megc/growth02/. (downloaded May 2003). Measures of Growth 2002 provides the results of 60 indicators in the areas of the economy, community, and the environment. Page 50 GAO-03-672SP Key National Indicators Forum Appendix III Selected Bibliography on Indicator Systems Miringoff, Marc and Marque-Luisa Miringoff. The Social Health of the Nation: How America Is Really Doing. New York, N.Y.: Oxford University Press, 1999. The Social Health of the Nation presents a variety of indicators of social well-being over several decades. Minnesota Planning. Minnesota Milestones 2002 (St. Paul, Minn.: 2002). http://www.mnplan.state.mn.us/mm/index.html. (downloaded May 2003). Minnesota Milestones 2002 reports on 70 progress indicators to determine whether the state is achieving 19 publicly determined goals in the areas of people, community and democracy, economy, and environment. The National Audit Office, United Kingdom, Good Practice in Performance Reporting in Executive Agencies and Non-Departmental Public Bodies (London, England: Stationery office, 2000). http://www.nao.gov.uk/publications/nao_reports/9900272.pdf (downloaded May 2003). Good Practice in Performance Reporting in Executive Agencies and Non-Departmental Public Bodies discusses good practices in government performance reporting to ensure transparent, accountable, and efficient government services. New York City Department of City Planning. 2000/2001 Report on Social Indicators. (New York, N.Y.: 2001). http://www.nyc.gov/html/dcp/html/pub/socind00.html (downloaded May 2003). 2000/2001 Report on Social Indicators is a compendium of data on the economic, social, physical, and environmental health of the city. The data are compiled from city, state, and federal sources and summarized on either a calendar or fiscal year basis. Page 51 GAO-03-672SP Key National Indicators Forum Appendix III Selected Bibliography on Indicator Systems The Oregon Progress Board. Achieving the Oregon Shines Vision: The 2001 Benchmark Performance Report (Salem, Oreg.: March 2001). http://www.econ.state.or.us/opb/2001report/reporthome.htm. (downloaded May 2003). Achieving the Oregon Shines Vision is a report on the comprehensive effort to describe progress Oregonians have made in achieving their year 2000 targets for 90 benchmarks. President of the Treasury Board. Canada’s Performance 2002 (Ottawa, Canada: 2002). http://www.tbs-sct.gc.ca/report/govrev/02/cp-rc_e.asp. (downloaded May 2003). Canada’s Performance 2002 reports on the quality of life of Canadians in such areas as economic opportunity, health, the environment, and the strength and safety of communities. Steering Committee Review of Commonwealth/State Services, Australia. Report on Government Services 2001 (Melbourne, Australia: 2001). http://www.pc.gov.au/gsp/2001/index.html. (downloaded May 2003). Report on Government Services 2001 details the performance of government service provision in Australia in education, health, justice, emergency management, community services, and housing. United Nations General Assembly. Implementation of the United Nations Millennium Declaration: Follow up to the Outcome of the Millennium Summit A/57/270. New York, N.Y.: 2002. Implementation of the United Nations Millennium Declaration: Follow up to the Outcome of the Millennium Summit details the progress that the United Nations has made on its millennium development goals, which are (1) halve extreme poverty and hunger, (2) achieve universal primary education, (3) empower women and promote equality between women and men, (4) reduce under five mortality by two-thirds, (5) reduce maternal mortality by three- quarters, (6) reverse the spread of disease especially AIDS/HIV and malaria, (7) ensure environmental sustainability, and (8) create a global partnership for development with targets for aid, trade, and debt relief. Page 52 GAO-03-672SP Key National Indicators Forum Appendix III Selected Bibliography on Indicator Systems United Nations Population Fund. State of the World Population 2002:People, Poverty and Possibilities (New York, N.Y.: 2002). http://www.unfpa.org/swp/2002/english/ch1/index.htm. (downloaded May 2003). State of the World Population 2002 provides a variety of demographic and economic data about people in various regions of the world as well as some data on individual nations. University at Buffalo Institute for Local Governance and Regional Growth, State of the Region Progress Report 2000 (Buffalo, N.Y.: 2000). http://regional-institute.buffalo.edu/sotr/repo/repo00/default.html. (downloaded May 2003). State of the Region Progress Report 2000 offers a first update of the 1999 baseline report with two components--one focused on the data- driven performance measures, the other a second look at the opportunities and challenges that will shape Buffalo-Niagara's progress into the new century. Specialized Indicator Systems Annie E. Casey Foundation. Kids Count Data Book 2002 (Baltimore, Md: 2002). http://www.aecf.org/kidscount/kc2002/. (downloaded May 2002). Kids Count provides information about the physical health, mental health, economic well-being, and educational achievements of children in the United States. Data are available nationwide and for each state. Chrvala, Carole A. and Roger J. Bulger, Eds. Leading Health Indicators for Healthy People 2010: Final Report. Committee on Leading Health Indicators for Healthy People 2010, Division of Health Promotion and Disease Prevention, Institute of Medicine. Washington, DC: National Academy Press, 1999. Leading Indicators for Healthy People 2010 describes the efforts of the Committee on Leading Health Indicators to develop leading health indicator sets that could focus on health and social issues and evoke a response and action from the general public and the traditional audiences for the Healthy People report series. Page 53 GAO-03-672SP Key National Indicators Forum Appendix III Selected Bibliography on Indicator Systems Committee to Evaluate Indicators for Monitoring Aquatic and Terrestrial Environments, Board on Environmental Studies and Toxicology, Water Science and Technology Board, Commission on Geosciences, Environment, and Resources, National Research Council. Ecological Indicators for the Nation. Washington, D.C.: National Academy Press, 2000. Ecological Indicators for the Nation suggests criteria for selecting useful ecological indicators, provides methods for integrating complex ecological information in indicators that are useful, proposes indicators that would meet these criteria, examines the state of data that would be used to calculate these indicators, and offers guidance on communicating and storing ecological indicators. Committee on Measuring and Improving Infrastructure Performance, Board on Infrastructure and the Constructed Environment, Commission on Engineering and Technical Systems, National Research Council. Measuring and Improving Infrastructure Performance (Washington, DC: National Academy Press, 1995). http://www.nap.edu/catalog/4929.html. (downloaded May 2003). Measuring and Improving Infrastructure Performance addresses the issue of measuring the efficiency with which our infrastructure allows people and goods to move, provides adequate safe drinking water, provides energy, removes waste, and so on, which is crucial to being able to manage the assets that our infrastructure represents. Cooper, Ronald S. and Stephen A. Merrill, Eds. Industrial Research and Innovation Indicators: Report of a Workshop. Board on Science, Technology, and Economic Policy, National Research Council (Washington, D.C.: National Academy Press, 1997). http://www.nap.edu/catalog/5976.html. (downloaded May 2003). Industrial Research and Innovation Indicators is the report of a workshop held to discuss methods of improving the measurement, data collection and analysis of indicators on industrial research and innovation, as well as examining ways in which this information could be integrated into measures of firm and national performance. Page 54 GAO-03-672SP Key National Indicators Forum Appendix III Selected Bibliography on Indicator Systems Council of Economic Advisors, Executive Office of the President. The Economic Report of the President (Washington, D.C.: Government Printing Office, 2003). http://w3.access.gpo.gov/usbudget/fy2004/pdf/2003_erp.pdf. (downloaded May 2003). The Economic Report of the President is a discussion of selected economic issues and tables of economic data prepared by the Council of Economic Advisors. The Conference Board. Business Cycle Indicators (New York, N.Y.: 2002). http://www.tcb-indicators.org/Us/LatestReleases/index.cfm. (downloaded May 2003). Business Cycle Indicators provides monthly economic indicators for the United States, such as the leading economic indicators, the coincident indicators, and the lagging indicators. Federal Interagency Forum on Aging Related Statistics. Older Americans 2000: Key Indicators of Well Being (Washington, D.C.: 2000). http://www.agingstats.gov/chartbook2000/default.htm. (downloaded May 2003). Older Americans: 2000 contains statistics regarding the population, economics, health status, health risks and behaviors, and health care of older United States Citizens. Federal Interagency Forum on Child and Family Statistics. America’s Children: Key National Indicators of Well-Being 2002 (Washington, D.C.: Government Printing Office, 2002). http://childstats.gov. (downloaded May 2003). American’s Children provides 24 key indicators on the well-being of children in the areas of economic security, health, behavior and social environment, and education. Page 55 GAO-03-672SP Key National Indicators Forum Appendix III Selected Bibliography on Indicator Systems United States Department of Health and Human Services. Healthy People 2010: Understanding and Improving Health. (Washington, D.C.: 2000). http://www.healthypeople.gov/. (downloaded May 2002). Healthy People 2010 provides a comprehensive set of disease prevention and health promotion objectives for the United States to achieve by 2010. The H. John Heinz III Center for Science, Economics, and the Environment. The State of the Nation’s Ecosystems: Measuring the Lands, Waters, and Living Resources of the United States (Cambridge, U.K.: Cambridge University Press, 2002). http://www.heinzctr.org/ecosystems/index.htm. (downloaded May 2003). The State of the Nation’s Ecosystem is a blueprint for periodic reporting on the condition and use of ecosystems in the United States. Mislevy, Robert J. and Kaeli T. Knowles, Eds. Performance Assessments for Adult Education: Exploring the Measurement Issues Report of a Workshop. Committee for the Workshop on Alternatives for Assessing Adult Education and Literacy Programs, Board on Testing and Assessment, Center for Education, Division of Behavioral and Social Sciences and Education, National Research Council. (Washington, DC: National Academy Press, 2002). http://www.nap.edu/catalog/10366.html. (downloaded May 2003). Performance Assessments for Adult Education examines a variety of ways of measuring learning gains in adult basic education classes in light of the requirement that states evaluate adult students' progress as mandated by the Workforce Investment Act. Nordhaus, William D. and Edward C. Kokkelenberg, Eds. Nature's Numbers: Expanding the National Economic Accounts to Include the Environment. Panel on Integrated Environmental and Economic Accounting, Committee on National Statistics, Commission and Behavioral and Social Sciences and Education, National Research Council (Washington, DC: National Academy Press, 1999). http://www.nap.edu/catalog/6374.html. (downloaded May 2003). Nature's Numbers examines the issues surrounding the question of broadening the U.S. National Income and Product Accounts to include Page 56 GAO-03-672SP Key National Indicators Forum Appendix III Selected Bibliography on Indicator Systems activities that relate to natural resources and the environment to provide a more comprehensive picture of the economy. Norwood, Janet and Jamie Casey, Eds. Key Transportation Indicators: Summary of a Workshop. Committee on National Statistics, Division of Behavioral and Social Sciences and Education, National Research Council. Washington, DC: National Academy Press, 2002. Key Transportation Indicators discusses efforts to review current transportation indicators and issues associated with their uses as well considering what kinds of additional indicators are need. Pellegrino, James W., Lee R. Jones, and Karen J. Mitchell, Eds. Grading the Nation's Report Card: Evaluating NAEP and Transforming the Assessment of Educational Progress. Committee on the Evaluation of National and State Assessment, Commission on Behavioral and Social Sciences and Education, National Research Council. (Washington, D.C.: National Academy Press, 1999). http://www.nap.edu/catalog/6296.html. (downloaded May 2003). Grading the Nation’s Report Card describes the National Assessment of Educational Progress' national assessment, the state assessment program, the student performance standards, and the extent to which the results are reasonable, valid, and informative to the public. Schultze, Charles L. and Christopher Mackie, Eds. At What Price? Conceptualizing and Measuring Cost-of Living and Price Indexes. Panel on Conceptual, Measurement, and Other Statistical Issues in Developing Cost-of-Living Indexes, Committee on National Statistics, Division of Behavioral and Social Sciences and Education, National Research Council (Washington, D.C.: National Academy Press, 2002). http://www.nap.edu/catalog/10131.html. (downloaded May 2003). At What Price? discusses the conceptual, measurement, statistical, and data issues in the development of cost-of-living indexes and assesses the appropriate use of such indexes as for indexing federal programs and other purposes. Starke, Linda, ed. State of the World 2002: Special World Summit Edition (W.W. Norton and Co.: 2002). http://www.worldwatch.org/pubs/sow/2002/. (downloaded May 2003). Page 57 GAO-03-672SP Key National Indicators Forum Appendix III Selected Bibliography on Indicator Systems State of the World 2002 provides information on a variety of issues in sustainable development, such as climate change, farming, and toxic chemicals. UNICEF. The State of the World’s Children 2003 (Oxford, N.Y.: Oxford University Press, 2002). http://www.unicef.org/pubsgen/sowc03/index.html. (downloaded May 2003). The State of the World’s Children 2003 contains a comprehensive set of economic and social indicators on the well-being of children worldwide. The World Health Organization. World Health Report 2002 (Geneva: 2002). http://www.who.int/whr/. (downloaded May 2003). World Health Report 2002 measures the amount of disease, disability, and health that can be attributed to certain risks and calculates how much of the burden is preventable. Background Berry, David, Patrice Flynn, and Theodore Heintz. “Sustainability and Quality of Life Indicators: Toward the Integration of Economic, Social and Environmental Measures,” Indicators: The Journal of Social Health vol. 1, no. 4 (Fall 2002). “Sustainability and Quality of Life Indicators” provides discussion of approaches to integrate social, economic, and environmental indicators and expand the scope of our national data system. Caplow, Theodore, Louis Hicks, and Ben J. Wattenberg. The First Measured Century. Washington, D.C.: AEI Press, 2001. The First Measured Century describes how using statistics to measure social conditions gained importance throughout the United States from 1900 through 2000. Gross, Betram M. Social Intelligence for America's Future: Explorations in Societal Problems. Boston, Mass: Allyn and Bacon, Inc., 1969. Social Intelligence for America’s Future is part of a “trial run” social report ranging from learning and health to crime and the arts. It discusses information methodology and the use of data to guide public policy. Page 58 GAO-03-672SP Key National Indicators Forum Appendix III Selected Bibliography on Indicator Systems Michalos, Alex C. Observations on Key National Performance Indicators. Paper presented at the Forum on Key National Indicators, Washington, D.C., February 2003. Observations on Key National Performance Indicators discusses several integrated performance systems on the national, international, and state levels. Miringoff, Marc, Marque-Luisa Miringoff, and Sandra Opdycke. Social Indicators: What We Need To Make Them Count. Paper presented at the Forum on Key National Indicators, Washington, D.C., February 2003.. Social Indicators addresses the need for social indicators to enhance consideration of policy issues. Riche, Martha Farnsworth. The United States of America Developing Key National Indicators. Paper presented at the Forum on Key National Indicators, Washington, D.C., February 2003.. The United States of America Developing Key National Indicators offers a framework to assess indicators and provides a preliminary draft of what an indicator set might look like for the United States. Slater, Courtenay M. and Martin H. David, Eds. Measuring the Government Sector of the U.S. Economic Accounts. Committee on National Statistics, Commission on Behavioral and Social Science and Education, National Research Council (Washington, DC: National Academy Press, 1998). http://www.nap.edu/catalog/6208.html. (downloaded May 2003) Measuring the Government Sector of the U.S. Economic Accounts summarizes the discussion on and makes recommendations regarding the way the government sector is presented in U.S. economics accounts and how it could be brought into line with the International System of Accounts, which would allow for better cross-national comparisons. U.S. Department of Health, Education, and Welfare, Toward a Social Report. Washington, D.C.: U.S. Government Printing Office, 1969. Toward a Social Report discusses how social reporting can improve the nation’s ability to chart its social progress and to promote more informed policy decisions. Page 59 GAO-03-672SP Key National Indicators Forum Appendix IV Selected Web Sites on Indicator Systems Appendx iIV Multinational European System of Social Indicators http://www.social-science- gesis.de/en/social_monitoring/social_indicators/EU_Reporting/eusi.htm Global Reporting Initiative - Sustainability Reporting Guidelines http://www.globalreporting.org/guidelines/2002.asp Measurement and Indicators for Sustainable Development - IISDnet http://www.iisd.org/measure/default.asp Organisation for Economic Co-operation and Development Worldwide Statistical Sources http://cs3-hq.oecd.org/scripts/stats/source/index.htm The International Society for Quality-of-Life Studies http://www.cob.vt.edu/market/isqols/ United Nations High Commissioner for Refugees - The State of The World’s Refugees http://www.unhcr.ch/pubs/sowr2000/sowr2000toc.htm United Nations Human Development Report 2002 http://www.undp.org/hdr2002/ United Nations Millennium Development Goals http://www.un.org/millenniumgoals/index.html United Nations Population Fund - State of World Population 2002 http://www.unfpa.org/swp/swpmain.htm United Nations Statistics Division - Social Indicators http://unstats.un.org/unsd/demographic/social/default.htm United Nations Statistics Division http://unstats.un.org/unsd/ World Bank’s Millenium Development Goals http://www.developmentgoals.org/ Worldwatch Institute State of the World 2002 http://www.worldwatch.org/pubs/sow/2002/ Page 60 GAO-03-672SP Key National Indicators Forum Appendix IV Selected Web Sites on Indicator Systems National Australian Bureau of Statistics - Measuring Australia’s Progress http://www.abs.gov.au/ Canadian Council on Social Indicators http://www.ccsd.ca/soc_ind.html Conference Board of Canada http://www.conferenceboard.ca/ FedStats Home Page http://www.fedstats.gov/ Heinz Center - The State of the Nation’s Ecosystems http://www.heinzctr.org/ecosystems/index.htm Interagency Working Group on Sustainable Development Indicators http://www.sdi.gov/ Redefining Progress http://www.redefiningprogress.org/ Treasury Board of Canada - Societal Indicators http://www.tbs-sct.gc.ca/rma/eppi-ibdrp/SI/si_e.htm STAT-USA - State of the Nation http://www.stat-usa.gov/econtest.nsf United Kingdom Government Sustainable Development Indicators http://www.sustainable-development.gov.uk/indicators/ United Kingdom National Audit Office http://www.nao.gov.uk/ University of Toronto Performance Indicators for Governance http://www.utoronto.ca/provost/perf98 University of Washington Human Services Policy Center http://www.hspc.org/ White House - Latest Federal Government Statistics http://www.whitehouse.gov/news/fsbr.html Page 61 GAO-03-672SP Key National Indicators Forum Appendix IV Selected Web Sites on Indicator Systems Regional or Multistate List of Performance Indicators for the Buffalo-Niagara Region http://www.regional-institute.buffalo.edu/sotr/repo/indi.html Northeast Midwest Institute Home Page http://www.nemw.org/ Northwest Area Foundation Indicator Web site http://www.indicators.nwaf.org/ Regional Research Institute, West Virginia University http://www.rri.wvu.edu/ Worcester Regional Research Bureau http://www.wmrb.org/CCPM State Living with the Future in Mind - New Jersey’s 1999 Sustainable State Report http://www.njfuture.org/HTMLSrc/SSR/index.html Maine Marks for Children, Families and Communities http://www.mainemarks.org/ Minnesota Planning Home Page http://www.mnplan.state.mn.us/ Oregon Shines - Oregon Progress Board http://www.econ.state.or.us/opb/index.htm Public Policy Institute of California http://www.ppic.org/ Vermont Agency of Human Services http://www.ahs.state.vt.us/ Local Burlington Legacy Indicators Project http://maps.vcgi.org/burlingtonlegacy/ Center for Schools and Communities - Lemoyne, Pennsylvania http://www.center-school.org/ Chicago Metropolis 2020 http://www.chicagometropolis2020.org/report.htm Page 62 GAO-03-672SP Key National Indicators Forum Appendix IV Selected Web Sites on Indicator Systems City and Borough of Sitka, Alaska http://www.cityofsitka.com/ Healthy Anchorage Indicators http://www.indicators.ak.org/ New York City Department of City Planning - Social Indicators http://www.nyc.gov/html/dcp/html/pub/socind00.html Portland Mulnomah Progress Board http://www.p-m-benchmarks.org/tblcnts.html San Diego’s Regional Planning Agency http://www.sandag.cog.ca.us/ Southern California Association of Governments http://www.scag.ca.gov/ Sustainable Community Roundtable, Olympia, Washington http://www.olywa.net/roundtable/ Sustainable Seattle http://www.sustainableseattle.org/ The Baltimore Neighborhood Indicators Alliance http://www.bnia.org/about_bnia_main.html The Boston Foundation http://www.tbf.org/ The Planning Council, Norfolk, Virginia http://www.theplanningcouncil.org/ Specialized Efforts Children America’s Children 2002 - Key National Indicators of Well-Being 2002 http://www.childstats.gov/americaschildren/ Child Trends DataBank http://www.childtrendsdatabank.org/about.htm Page 63 GAO-03-672SP Key National Indicators Forum Appendix IV Selected Web Sites on Indicator Systems Children First for Oregon http://www.cffo.org/ KIDS COUNT - Benchmarks of Child Well-Being http://www.aecf.org/kidscount United Nations - The State of the World’s Children 2000 http://www.unicef.org/sowc00/ Economy Federal Reserve Bank of Philadelphia http://www.phil.frb.org/ Index of the Massachusetts Innovation Economy http://www.mtpc.org/2001index/about.htm International Association for Research in Income and Wealth http://www.iariw.org/ Maine Economic Growth Council http://www.mdf.org/megc Norwegian Ministry of Finance http://www.odin.dep.no/fin/engelsk/ West Virginia Bureau of Employment Programs - Labor Market Information http://www.state.wv.us/bep/lmi/ Education California’s Public Schools Accountability Act http://www.cde.ca.gov/psaa/ National Assessment of Educational Progress -- The Nation’s Report Card http://nces.ed.gov/nationsreportcard/about/ White House Social Statistics Briefing Room - Education http://www.whitehouse.gov/fsbr/education.html Page 64 GAO-03-672SP Key National Indicators Forum Appendix IV Selected Web Sites on Indicator Systems Elderly Federal Interagency Forum on Aging-Related Statistics http://www.agingstats.gov/ Administration on Aging - Performance Outcomes Measures Project http://www.gpra.net/ Environment Environmental Protection Agency Biological Indicators of Watershed Health http://www.epa.gov/bioindicators Food and Agriculture Organization of the United Nations - State of the World’s Forests http://www.fao.org/forestry/FO/SOFO/sofo-e.stm Northwest Environment Watch - Home Page http://www.northwestwatch.org/pubs_index.html Washington Governor’s Salmon Recovery Office http://www.governor.wa.gov/gsro/default.htm World Association of Nuclear Operators Performance Indicators http://www.wano.org.uk/ Health Leading Health Indicators - Healthy People 2010 http://www.health.gov/healthypeople/LHI/lhiwhat.htm Maryland’s Drug Early Warning System http://www.dewsonline.org/ Pan American Health Organization http://www.paho.org/ Partnerships for Networked Consumer Health Information Conferences http://odphp.osophs.dhhs.gov/confrnce/ Page 65 GAO-03-672SP Key National Indicators Forum Appendix IV Selected Web Sites on Indicator Systems The Quality Indicator Project - Association of Maryland Hospitals and Health Systems http://www.qiproject.org/ White House Social Statistics Briefing Room - Health http://www.whitehouse.gov/fsbr/health.html Background or Reference Flynn Research - Measuring Contributions to Society http://www.flynnresearch.com/products.htm From Revolution to Reconstruction - Information on U.S. Presidents http://odur.let.rug.nl/~usa/P/ NonProfit Pathfinder - Measuring the Impact of the Independent Sector http://www.independentsector.org/pathfinder/impact/indepsec_res/biblio.h tml The Social Indicators Survey Center - Columbia University http://www.siscenter.org/ (450175) Page 66 GAO-03-672SP Key National Indicators Forum GAO’s Mission The General Accounting Office, the audit, evaluation and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO’s commitment to good government is reflected in its core values of accountability, integrity, and reliability. Obtaining Copies of The fastest and easiest way to obtain copies of GAO documents at no cost is through the Internet. GAO’s Web site (www.gao.gov) contains abstracts and full- GAO Reports and text files of current reports and testimony and an expanding archive of older products. The Web site features a search engine to help you locate documents Testimony using key words and phrases. You can print these documents in their entirety, including charts and other graphics. Each day, GAO issues a list of newly released reports, testimony, and correspondence. GAO posts this list, known as “Today’s Reports,” on its Web site daily. The list contains links to the full-text document files. To have GAO e-mail this list to you every afternoon, go to www.gao.gov and select “Subscribe to GAO Mailing Lists” under “Order GAO Products” heading. Order by Mail or Phone The first copy of each printed report is free. Additional copies are $2 each. A check or money order should be made out to the Superintendent of Documents. GAO also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a single address are discounted 25 percent. Orders should be sent to: U.S. General Accounting Office 441 G Street NW, Room LM Washington, D.C. 20548 To order by Phone: Voice: (202) 512-6000 TDD: (202) 512-2537 Fax: (202) 512-6061 To Report Fraud, Contact: Web site: www.gao.gov/fraudnet/fraudnet.htm Waste, and Abuse in E-mail: firstname.lastname@example.org Federal Programs Automated answering system: (800) 424-5454 or (202) 512-7470 Public Affairs Jeff Nelligan, Managing Director, NelliganJ@gao.gov (202) 512-4800 U.S. General Accounting Office, 441 G Street NW, Room 7149 Washington, D.C. 20548
Forum on Key National Indicators: Assessing the Nation's Position and Progress
Published by the Government Accountability Office on 2003-05-01.
Below is a raw (and likely hideous) rendition of the original report. (PDF)