oversight

Forum on Key National Indicators: Assessing the Nation's Position and Progress

Published by the Government Accountability Office on 2003-05-01.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                                                     May 2003


                                                     HIGHLIGHTS OF A GAO FORUM

                                                     Key National Indicators:
Highlights of GAO-03-672SP                           Assessing the Nation’s
                                                     Position and Progress



The nation confronts profound                        Developing Key National Indicators for the United States Is Important
challenges resulting from a variety                  While there are a variety of indicator efforts in the United States, there is no
of factors, including changing                       generally accepted, comprehensive indicator system for the nation as a whole.
security threats, dramatic shifts in                 There was broad agreement that the issue of developing key national indicators
demographic patterns, the                            is important for taking a more comprehensive view of the nation’s position and
multidimensional processes of                        progress, both on an absolute and relative basis. Several models were discussed
globalization, and the accelerating                  that offer lessons for developing a national indicator system, including indicator
pace of technological change.                        systems on aging, children, economics, and health. The purpose of
These are all coming together in an                  measurement, the process of deciding what to measure, and determining
era of diminishing public                            audiences are as critical as choosing what and how to measure.
resources.

The nation’s leaders and concerned                   A Broad Range of Information Areas Are Considered Significant
citizens require better knowledge                    The range of information areas considered important was broad, covering the
of what is happening and where we                    economy, society, and the environment. Participants agreed that a first step is to
are going to support improved                        assemble “core” indicators from existing data. A straw proposal for such an
public choices. The United States                    indicator set—USA Series 0.5—was presented as a starting point for building
could potentially benefit from                       what might eventually be a broadly supported USA Series 1.0 indicator set.
developing a set of key national                     Series 0.5 included 11 key information areas: community, crime, ecology,
indicators to help assess our                        education, governance, health, the macroeconomy, security, social support,
nation’s position and progress.                      sustainability, and transparency. In reacting to Series 0.5, participants suggested
                                                     numerous refinements and identified 4 additional information areas:
On February 27, 2003, GAO, in                        communications, diversity, individual values, and socioeconomic mobility.
cooperation with the National
Academies, hosted a forum on key                     A Rich History of Indicator Systems Warrants Collective Research
national indicators. The purpose of                  There is a long history of efforts throughout the world by leading democracies to
the forum was to have a rich and                     develop and sustain indicator systems. A distinction was made between
meaningful dialogue on whether                       comprehensive and specialized efforts that focus on a topic or issue. Research
and how to develop a set of key                      on what can be learned from past and present systems is essential to deriving
national indicators for the United                   useful implications for a possible United States system. A multitude of efforts
States.                                              are currently under way in other democracies (e.g., Australia and Canada) as
                                                     well as in the United States at the national, regional, state, and local levels.
The forum brought together a                         Despite this activity, there appear to be few common sources of broad research
diverse group of national leaders to                 to facilitate knowledge sharing on comprehensive indicator efforts.
discuss the following:
• How are the world’s leading
                                                     A United States Initiative Must Build on Past Lessons and Current Efforts
   democracies measuring national
   performance?                                      Developing a U.S. indicator system requires applying lessons from past efforts
• What might the United States do
                                                     and engaging with many existing ones. A United States system must be flexible
   to improve its approach and                       and evolve to respond to societal change and incorporate diverse perspectives.
   why?                                              An informal national coordinating committee of institutions in the public and
• What are important areas to                        private sectors was constituted to begin organizing a U.S. initiative. It serves as
   measure in assessing U.S.                         an initial means to facilitate dialogue, expand participation, plan work and
   national performance?                             secure financing. As of May 7, 2003, the committee included the American
• How might new U.S. approaches                      Association of Universities, The Conference Board, the Council for Excellence in
   be led and implemented?                           Government, GAO, the International City/County Management Association, The
 www.gao.gov/cgi-bin/getrpt?GAO-03-672SP.            National Academies, the National Association of Asian American Professionals,
                                                     the Office of Management and Budget and the White House Council on
 To view the full report, click on the link above.   Environmental Quality.
 For more information, contact Christopher
 Hoenig, Director, Strategic Issues, at
 (202) 512-6779 or hoenigc@gao.gov.
Contents



Letters                                                                                                    1
                             Comptroller General of the United States Introductory Letter                  1
                             President of the Institute of Medicine, The National Academies
                               Introductory Letter                                                         4


Forum                                                                                                      6
                             Background                                                                    6
                             Forum Summary                                                                11
                             How Are the World’s Leading Democracies Measuring National
                               Performance?                                                               14
                             What Might the United States Do to Improve Its Approach and
                               Why?                                                                       24
                             What Are Important Areas to Measure in Assessing U.S. National
                               Performance?                                                               34
                             How Might New U.S. Approaches Be Led and Implemented?                        38


Appendixes
              Appendix I:    Forum Participants                                                           42
             Appendix II:    Illustrative Indicators by Information Area for USA
                             Series 0.5                                                                   46
             Appendix III:   Selected Bibliography on Indicator Systems                                   49
             Appendix IV:    Selected Web Sites on Indicator Systems                                      60


Tables                       Table 1: Illustrative Comprehensive Indicator Systems                        16
                             Table 2: Illustrative Specialized Indicator Systems                          17
                             Table 3: Selected Highlights of Indicator Traditions in the United
                                      States during the 20th Century                                      22
                             Table 4: Structure of USA Series 0.5                                         30
                             Table 5: Proposed Evolutionary Process for a National Indicator
                                      System (Includes Illustrative Information Areas)                    39


Figures                      Figure 1: Life Expectancy at Birth and at Age 65                             31
                             Figure 2: U.S. Emissions of Greenhouse Gases, Based on Global
                                       Warming Potential, 1990-2000 (in Million Metric Tons of
                                       Carbon Equivalent)                                                 32




                             Page i                                GAO-03-672SP Key National Indicators Forum
Contents




 This is a work of the U.S. Government and is not subject to copyright protection in the
 United States. It may be reproduced and distributed in its entirety without further
 permission from GAO. It may contain copyrighted graphics, images or other materials.
 Permission from the copyright holder may be necessary should you wish to reproduce
 copyrighted materials separately from GAO’s product.




Page ii                                      GAO-03-672SP Key National Indicators Forum
                                                                                                       Leter




Comptroller General of   On February 27, 2003, the U.S. General Accounting Office (GAO) in
                         cooperation with the National Academies convened the Forum on Key
the United States        National Indicators for the United States. This report summarizes the
Introductory Letter      research, points of view, and commitments that the event produced.

                         We were pleased to have the National Academies as a partner in this event.
                         They have demonstrated the ability not only to conduct quality research
                         but also to help professional communities reflect on and build consensus
                         around the operational definition of indicator sets, in key areas, such as
                         communities, ecology, education, health and transportation.

                         Although the forum took place in Washington, D.C. the event was not
                         merely about the federal government or the public sector. It addressed
                         issues about indicators for the nation at all levels, from the community to
                         the country as a whole. Those who attended came in the spirit of a national
                         endeavor that rose above personal, institutional, or sectoral interests.

                         Because the United States is diverse, so were the participants. Gathered
                         together were national leaders and experts who could articulate the
                         concerns and perspectives of businesses, government, the media,
                         foundations, and nonprofits as well as the scientific, statistical, and
                         educational communities—a group representing both the users and
                         producers of public information.

                         Essentially, the broad impetus for the new level of dialogue at the forum
                         comes from two sources. First, that more and better public information
                         may be needed to effectively resolve current and future national
                         challenges. Second, that the laboratories of democracy in our country and
                         around the world are engaged in hundreds of efforts to develop indicator
                         systems, many of which are comprehensive in nature. It is a logical
                         extension to consider a comprehensive indicator system for the United
                         States that would help assess the nation’s overall position and progress.

                         There is a strong implication here. To be a leading democracy in the
                         information age means producing objective, independent, scientifically
                         grounded, and widely shared quality information on where we are and
                         where we are going, on both an absolute and relative basis, including
                         comparisons to other nations. Such information must be useful to the
                         public, professionals, and leaders at all levels of our society.




                         Page 1                               GAO-03-672SP Key National Indicators Forum
The founders of our nation knew this critical issue needed ongoing
attention as it grew and evolved. President George Washington, in his first
annual message to Congress on January 8, 1790, said, “Knowledge is in
every country the surest basis of public happiness. In one in which the
measures of government receive their impressions so immediately from the
sense of the community as in ours it is proportionably essential.”

Since that time, there has been a long history—checkered by success and
failure—of attempts to create sources of information that would inform our
public dialogues and serve as a context for governance and civic choices.
Developing a comprehensive, independent, quality resource of key
indicators for a nation as large, complex, and diverse as the United States is
a daunting task. If it is to be done, we must work hard and work together
to avoid the mistakes of the past and take advantage of new opportunities
that have emerged in the 21st century.

One lesson shows the need for patience, persistence, and attention to
democratic process. There is an important role for the federal government,
and in particular the U.S. Congress, to help catalyze an effort to develop
and sustain a national indicator system. A fully operational set of credible
measures of our progress and prospects will take years to develop, require
broad involvement of American society, and involve substantial resource
commitments. And yet many believe the benefits, in terms of more facts,
broader consensus, and better choices, will far outweigh the costs.

Our objective in convening the forum was to stimulate a dialogue that
might encourage a collective commitment from several leading institutions
to validate the need and begin organizing themselves to take action. While
this objective has been met, it should be stressed that this forum is merely
the start of a new stage of our country’s long journey of increasing self-
awareness and sense of collective accountability. We are pleased to help
contribute to this effort and look forward to working with the
extraordinary group of committed parties and the many who are




Page 2                                 GAO-03-672SP Key National Indicators Forum
continually joining the effort to develop options and approaches that will
be of truly lasting value to the American people.




David M. Walker
Comptroller General
of the United States




Page 3                                GAO-03-672SP Key National Indicators Forum
President of the         On behalf of the National Academies, let me add my deep appreciation to
                         all who participated in and facilitated this forum. The level of participation
Institute of Medicine,   in this important event speaks volumes about a topic that is critically
The National             important to our nation and ripe for our attention. It has been a pleasure
                         for the National Academies to be a part of this promising, important, and
Academies                timely venture.
Introductory Letter
                         It is fitting that the National Academies -- the Institute of Medicine, the
                         National Academy of Sciences and the National Academy of Engineering,
                         together with our operating arm, the National Research Council –
                         cooperated in facilitating this effort. We represent a body that has provided
                         advice on scientific issues that affect public policy decisions since 1863.
                         Over that long period, the Academies have been able to contribute to public
                         discussion and understanding on many issues of national significance. In
                         an important way, our collaboration with the General Accounting Office on
                         this issue is a continuation of the contribution we seek to make to the
                         formulation of sound public decisions based on sound scientific evidence.

                         This collaboration with GAO on the development of key national
                         performance indicators is one of a number of projects we have undertaken
                         under an agreement implemented in 2001. We were pleased to have
                         initially convened a panel of experts representing various disciplines to
                         share experiences and views on the use of indicators, then, when the panel
                         suggested this forum, to support GAO in bringing it together.

                         I am enthusiastic about the possibilities and the promise of this forum.
                         Public policy in many areas, including medicine, is stronger because of the
                         existence of indicators of performance. Indicators help our nation focus
                         on the key issues confronting us. We can be proud of the tremendous
                         efforts that have been made in the scientific community to develop them.

                         As those of us in the public and private sectors jointly consider the next
                         steps to take, benefiting from this very useful report of the forum prepared
                         by GAO, I hope we keep in mind our ultimate objectives. I will be thinking
                         about the enlightening discussion of the issues with a question in mind:




                         Page 4                                 GAO-03-672SP Key National Indicators Forum
“WHY DO WE WANT TO KNOW THIS”? This important question gets us
started and helps us frame all of the other questions we must ponder.




Dr. Harvey V. Fineberg
President, Institute of Medicine
The National Academies




Page 5                             GAO-03-672SP Key National Indicators Forum
Forum



Background   The pace and character of change is having a profound impact on the
             United States government, the nation itself, and its position in the global
             community. Changing security, economic, demographic, technological, and
             other trends have, in some cases, exacerbated economic, social, and
             environmental tensions. These trends have created new challenges and
             opportunities both within the United States and throughout the world. In
             just one example, the United States faces a huge and growing long-range
             fiscal imbalance due primarily to known demographic trends, rising health
             care costs, and other factors. Policymakers must reconcile the gap
             between projected revenues and expenditures in order to exercise
             fiduciary and stewardship responsibilities to the nation.

             A large and growing amount of activity is taking place, throughout the
             United States and around the world, to develop comprehensive indicator
             systems to inform the democratic process. New facts, insights, and
             approaches are being developed. Understanding and interpreting these
             efforts is vital to the process of setting direction and measuring progress—
             on both an absolute and a relative basis—as a context for governance.

             Although other leading democratic nations have developed key national
             indicator systems, the United States has not done so. While there are
             numerous indicator systems that are national in scale, such systems focus
             on specialized or specific topics such as health care or education. The
             question is, should we develop a comprehensive, national system that
             focuses on major elements of society—economical, social, and
             environmental areas?

             A set of key national indicators can help to assess the overall position and
             progress of our nation in key areas, frame strategic issues, support public
             choices and enhance accountability. It could help improve evaluations of
             how well the nation is addressing and resolving key issues and concerns.
             National indicators built on the foundation of information from our federal
             statistical system (i.e., official statistics), administrative records, as well as
             a variety of private sources could provide a unique, fact-based assessment
             of the state of the nation.

             The dramatic changes, challenges and increasing interdependencies
             affecting the nation demand new and more cross-sector and cross-border
             responses. Such responses could benefit from more integrated information
             resources to support informed public debate and decisions within and
             among different levels of government and society. For example, in



             Page 6                                   GAO-03-672SP Key National Indicators Forum
Forum




homeland security, what indicators will accurately reflect national
preparedness? In health care, how will we assess the health and well-being
of our population? How can we best measure success in education? Is the
most useful information available to fully assess our degree of economic
and social progress? Are we in fact moving ahead and in the right direction
in key areas? How do we compare to other nations? The stakes are high,
including considerations regarding scarce public resources, creating jobs,
stimulating future industries, maintaining global competitive edge,
enhancing security, sustaining the environment, and promoting quality of
life.

Developing a key national indicator system goes beyond any one sector
(i.e., public, private, or nonprofit). It requires designing and executing a
process whereby the diverse elements of society can participate in
formulating key questions and choosing indicators in a way that increases
consensus on the facts over time. It also involves complex issues ranging
from fostering agreement on specific indicators to choosing the
mechanisms for sharing reliable information used in public planning,
decision making and accountability. Furthermore, indicators in the
national system should be outcome-oriented, in addition to measuring
resources and capabilities. They should measure position and progress on
not only an absolute but also a relative basis, including comparing the
United States to other nations. They should not be seen as being the
nation’s goals or priorities, but rather a more sophisticated base of facts
with which to make more informed decisions.

To discuss the issues involved in developing a set of key indicators to be
included in a national system for the United States, GAO, in cooperation
with the National Academies, convened the Forum on Key National
Indicators on February 27, 2003, in Washington, D.C. The forum was an
attempt to bring more valuable facts to bear on decision making by the
public and its leaders. The forum was not intended to decide issues, set
priorities, or determine resource allocations—which are the province of
the nation’s duly chosen representatives.

The purpose of the forum was to have a rich and meaningful discussion on
whether and how to develop a key national indicator system for the United
States by focusing on four key questions:

• How are the world’s leading democracies measuring national
  performance?




Page 7                                GAO-03-672SP Key National Indicators Forum
Forum




• What might the United States do to improve its approach and why?

• What are important areas to measure in assessing U.S. national
  performance?

• How might new U.S. approaches be led and implemented?

GAO and the National Academies designed this venture to bring together a
multidisciplinary, multisector group of producers and users of public
information with a wide variety of perspectives. The invited participants
were national leaders and experts from the business, education,
foundation, government, labor, media, minority, scientific, and statistics
communities. Invitations were also extended to chairmen and ranking
minority members of relevant congressional committees. (See app. I for a
list of participants.) Comptroller General David M. Walker comoderated
the forum with the Honorable Thomas Sawyer, former Congressman from
Ohio.1

As agreed by the participants, the purpose of the discussion was to engage
in an open, not for attribution dialogue. However, one participant is
identified in the report because this individual provided a presentation that
was critical to the forum’s discussion. Other than this one individual, this
report summarizes the collective discussion and does not necessarily
represent the views of any individual participant, GAO,or the National
Academies.




1
  Mr. Sawyer served in the U.S. House of Representatives from 1987 to 2002 and chaired the
Subcommittee on Census, Statistics, and Postal Personnel, Committee on Post Office and
Civil Service, in the 101st, 102nd, and 103rd Congresses. Prior to his congressional service,
Mr. Sawyer served as Mayor of the City of Akron, Ohio, and as a Representative in the Ohio
House of Representatives.




Page 8                                        GAO-03-672SP Key National Indicators Forum
                            Forum




                            In addition to summarizing the forum participants’ collective discussion,
                            this report highlights the research conducted in preparation for the forum
                            and follow-on discussions with participants. Developing and preparing for
                            the forum was an intensive 6-month effort. GAO staff, led by Christopher
                            Hoenig, Director, Strategic Issues, researched indicator systems,
                            conducted a series of interviews with producers and users of information,
                            and wrote background papers on the history and state of the practice of
                            indicator systems. A preparatory planning meeting and subsequent
                            conversations were held with representatives of the National Academies to
                            help frame the questions and objectives of the forum. Experts identified by
                            the National Academies also prepared background papers for the forum.
                            Also, GAO, in cooperation with the National Academies, commissioned Dr.
                            Martha Farnsworth Riche2 to independently develop a straw proposal of a
                            key national indicator system to facilitate discussion among the forum’s
                            participants. A selected bibliography on indicator systems is included in
                            appendix III, and selected Web sites on indicator systems are included in
                            appendix IV of this report.



Limitations and             The dialogue as summarized in this report should be interpreted in the
Qualifications Concerning   context of five key limitations and qualifications.
the Forum
                            First, the forum was only an initial step in a possible long-term, evolving
                            effort to develop and sustain a key national indicator system. Its purpose
                            was to begin a dialogue on an extremely complex topic. Although many
                            leaders, institutions and points of view were represented, many more will
                            need to be involved—as follow-up efforts proceed—to start representing
                            the extraordinary diversity of knowledge and opinion in our nation. This is
                            especially true when it comes to choosing aspects of U.S. society for which
                            it is important to develop indicators. Additionally, the involvement of the
                            federal government, and particularly Congress, will be crucial.

                            Second, even though GAO, in cooperation with the National Academies,
                            conducted preliminary research and heard from national experts in their
                            fields, a day’s conversation cannot represent the current state of the
                            practice in this vast arena. More thought, discussion, and research must be


                            2
                              Dr. Riche served as Director of the U.S. Census Bureau from 1994 to 1998. Prior to being
                            appointed Director, she was a founding editor of American Demographics, Director of
                            Policy Studies for the Population Reference Bureau, and an economist with the U.S. Bureau
                            of Labor Statistics. Dr. Riche is currently a Principal with Farnsworth Riche Associates.




                            Page 9                                       GAO-03-672SP Key National Indicators Forum
Forum




done to develop greater agreement on what we really know, what needs to
be done, and how to do it.

Third, several presentations were made regarding (1) the lessons learned
from other indicator efforts, (2) a proposal for a draft version of an
indicator system for the United States, and (3) a potential organizational
model in the areas of children and aging that could be replicated in other
topical areas (e.g., public safety and governance). These presentations
represented individual opinions, not a broad consensus or any formal
endorsement by the cosponsoring or participating institutions. More
collaborative work must be done to move from these starting points toward
more definitive accomplishments.

Fourth, any key national indicator system that would be developed as a
result of follow-on efforts to the forum would, of necessity, build on the
vast amount of current information already available, from the federal
statistical system, the nonprofit and commercial sectors, and the many
efforts currently operating below the national level. Many state, regional,
and local governments and nonprofits working either in partnership or
alone have developed and are using indicator systems. Yet at the same
time, working on existing data alone would limit the opportunity to raise
new questions and issues and develop new information sources.

Fifth, because of the extraordinary diversity and quality efforts in
specialized or topical information areas (e.g., education and health care)
throughout the United States, this forum generally concentrated on
bringing together generalists who could help think through how to organize
a more comprehensive approach. As a result, a large number of leading
edge individuals, institutions, and networks involved in specialized efforts
could not be included for reasons of scope. This is an important limitation
of the forum. Any successful effort to develop a national system must find a
process and structure for including both specialized and comprehensive
approaches. It must also build on and aid current efforts as well as
developing new lines of effort. This has yet to be done and will require
broad involvement of those specialized organizations that recognize the
potential for mutual gain in such an effort.




Page 10                               GAO-03-672SP Key National Indicators Forum
                            Forum




Forum Summary               The five key limitations and qualifications described earlier provide
                            contextual boundaries. Nevertheless, the forum provided a rich dialogue on
                            indicator systems and participants produced strong messages on each of
                            the four questions. Those messages are highlighted below.



Developing Key National     While there is no generally accepted comprehensive, integrated indicator
Indicators for the United   system at the national level, a wide variety of indicator systems exist in the
                            United States. However, these indicator systems either focus on
States is Important
                            specialized or topical areas, such as health or education, or focus on a
                            regional, state, or community level. There was broad agreement among the
                            forum’s participants that the issue of developing a key national indicator
                            system is important but that further work needs to be done on what needs
                            to change and why.

                            • A straw proposal for a comprehensive indicator set—called USA Series
                              0.5—was presented at the forum, and participants acknowledged it to be
                              a good starting point for building what might eventually be a broadly
                              supported USA Series 1.0 indicator set.

                            • Several possible models were discussed that could offer useful lessons
                              for developing a national indicator system—including the leading
                              economic indicators as well as indicator systems on health, children,
                              and aging.

                            • A broad range of issues were discussed that would need to be addressed
                              to develop a useful key indicator system—including the need to define
                              purpose and audience; the need for public outreach, sophisticated
                              communications, and technology; and the importance of data
                              availability and quality.



A Broad Range of            While the range of information areas that participants considered
Information Areas Are       important about the U.S. was extremely broad, there was little argument
                            that an expedient first step is to try and assemble a set of “core” indicators
Considered Significant
                            from existing data to include within a national system. However, there was
                            also significant enthusiasm about: (1) refining information areas included
                            in the straw proposal and (2) identifying additional information areas. The
                            term “information area” refers to a body of knowledge including existing
                            data, questions, and ongoing research—that is meaningful in understanding
                            U.S. society.



                            Page 11                                GAO-03-672SP Key National Indicators Forum
                              Forum




                              • A core group of information areas that could serve as a starting point for
                                an evolving system—USA Series 0.5—was discussed. Independently
                                developed by Dr. Riche, USA Series 0.5 included 11 information areas:
                                community, crime, ecology, education, governance, health, the
                                macroeconomy, security, social support, sustainability, and
                                transparency.

                              • To move to a USA Series 1.0, participants identified refinements to a
                                majority of the USA Series 0.5 information areas. For example,
                                participants thought the governance information area needed to include
                                indicators on civic engagement. They also proposed the addition of 4
                                information areas: communications, diversity, individual values, and
                                socioeconomic mobility. However, this list of information areas was not
                                considered exhaustive since it was a first attempt to identify specific
                                information areas to be included in a national indicator system.



A Rich History of Indicator   There is a long history of efforts around the world by leading democracies
Systems Warrants Collective   to develop and sustain indicator systems. However, no generally accepted,
                              comprehensive approach yet exists in a society as large and diversified in
Research                      its system of governance as the United States. Research on what can be
                              learned from past and present systems would be essential to deriving
                              useful implications for a possible United States system.

                              • A multitude of specialized and comprehensive efforts are ongoing in the
                                United States at the national, regional, state, and local levels as well as
                                in other democracies—such as Australia and Canada. For example,
                                within the United States, there is an indicator system to nationally
                                assess the well-being of children and the Federal Interagency Forum on
                                Child and Family Statistics annually reports on the data from this
                                system.

                              • Despite the activity, there appear to be few common sources of
                                comprehensive research or communities of practice, either nationally or
                                globally, to facilitate knowledge sharing. Furthermore, there are
                                limitations in inferring lessons from countries of different size, diversity,
                                and political-economic structures than the United States.

                              • However, some lessons have already been learned. Clearly the purpose
                                of measurement, the process of deciding what to measure, and
                                determining who will truly benefit from the data are as critical as what
                                to measure and how to define specific indicators and technical methods.



                              Page 12                                GAO-03-672SP Key National Indicators Forum
                             Forum




A United States Initiative   Participants agreed that developing a key national indicator system would
Must Build on Past Lessons   require a combination of applying the essential lessons from past efforts
                             and determining how to engage constructively with the many efforts
and Current Efforts          currently under way.

                             • Any United States system must be flexible and evolve to allow for the
                               rapid rate of change in our society, the complexity of the endeavor, and
                               the wide variety of perspectives that will need to be reflected.

                             • An effort to develop a key national indicator system must not supplant
                               nor compete with the many existing efforts under way in the areas
                               identified by participants but should build on them.

                             • A comprehensive system for the United States must be appropriately
                               focused, have a definable audience, be independent, pay attention to
                               quality issues, and be adequately funded both in terms of its
                               development and sustainability.

                             • After the forum, an informal national coordinating committee of public
                               and private sector institutions was constituted to begin organizing a
                               national initiative and serve as the temporary means of facilitating
                               dialogue, work and financing. Because this effort is in its early stages,
                               the following list should not be misinterpreted as being complete or
                               exclusive. It simply shows the institutions that, to date, have
                               volunteered: the American Association of Universities (AAU), The
                               Conference Board, the Council for Excellence in Government, GAO, the
                               International City/County Management Association (ICMA), The
                               National Academies, the National Association of Asian American
                               Professionals, the Office of Management and Budget (OMB), and the
                               White House Council on Environmental Quality. The committee’s first
                               meeting will take place in the spring of 2003 in Washington, D.C.




                             Page 13                              GAO-03-672SP Key National Indicators Forum
                            Forum




How Are the World’s         Because of the broad scope of this question, most of the material in this
                            section represents preparatory research that was provided to participants
Leading Democracies         prior to the forum as background and context. A summary of this research
Measuring National          was presented briefly at the beginning of the forum.
Performance?                Many leading democracies around the world as well as major international
                            institutions are involved in efforts to develop specialized and
                            comprehensive indicator systems of societal performance. Specialized
                            indicator systems focus on specific topics or information areas, such as
                            health, education, or children while comprehensive indicator systems
                            focus on several information areas, generally within the broader categories
                            of economic, social, and environmental arenas. Additionally, a multitude of
                            both specialized and comprehensive indicator systems are going on in the
                            United States at the local, state, regional, and national levels. Some of
                            these systems have been in place for decades and some have emerged in
                            only the last few years.

                            Some involved in these systems attribute the level of activity to the
                            enabling possibilities created by data integration and presentation
                            technologies (e.g., the World Wide Web). Others mention the increasing
                            demand for cross-sector, cross-border responses to fiscal and other
                            challenges that require new, integrated sources of data as well as new types
                            of information. There appear to be few common sources of broad research
                            on comprehensive systems (either on a national or global level) in a
                            position to facilitate knowledge sharing. In contrast, there are numerous
                            communities of practice dedicated to individual specialized systems.

                            Forum discussion of indicator systems by other countries and the United
                            States focused on the state of the practice of current indicator systems,
                            observations on indicator systems, and past efforts in the United States to
                            develop national indicator systems.



State of the Practice of    GAO found that a key aspect of the current state of the practice involves
Current Indicator Systems   comprehensive and specialized (i.e., topical) categories of indicator
                            systems representing a wide range of maturities from formative to
                            advanced. These systems also vary in the number of indicators, ranging
                            from 19 to over 400.

                            Several democracies, such as Canada and Australia, use comprehensive
                            indicator systems and focus on information areas such as economic



                            Page 14                               GAO-03-672SP Key National Indicators Forum
Forum




opportunities and innovation, the strength and safety of communities,
national wealth, and national income. Within these information areas are
indicators ranging from real national net wealth per capita and real
disposable income per capita to life expectancy at birth and literacy.

Several states and communities within the United States, such as the State
of Minnesota and the metropolitan area of Boston, also use comprehensive
indicator systems. These indicator systems focus on information areas
such as public safety, housing, and community and democracy and include
indicators ranging from growth in gross state product and unemployment
rate to volunteer time and prenatal care.

Comprehensive indicator systems have two primary characteristics. One
characteristic is creating an overall picture of how a community (or region,
nation, etc.) is doing. The second characteristic is showing the
interconnectedness of various key information areas, such as the
interrelationship between economic development and environmental
impact. Through both these characteristics, a comprehensive indicator
system allows for a deeper understanding of what is really happening in a
society and significantly broadens the availability of that knowledge.
Different entities take an individualized approach to grouping together key
specialized information areas. For example, Australia’s system includes
biodiversity, crime, economic disadvantage and inequality, education and
training, health, land, national income, national wealth, social attachment,
water, and work.

Table 1 provides details on several illustrative examples of comprehensive
indicator systems regarding who reports the data, sources of the data, their
purpose, the first year a system’s data were reported, and frequency of
reporting updates. The table also identifies the scale of the system (i.e.,
national, regional, or local) that refers to the primary focus of the
information being reported. However, larger scale efforts (e.g., national)
can in some cases be cumulative, including state and/or local data.




Page 15                               GAO-03-672SP Key National Indicators Forum
                                           Forum




Table 1: Illustrative Comprehensive Indicator Systems

                                                                                                           Year report         Frequency of
Indicator system     Reported by    Data sources            Scale               Purpose                    first issued        report updates
Canada’s             Treasury Board Canadian                National            To provide information 1994                    Yearly
Performance 2002     of Canada      government                                  on trends in well-being
                                    agencies and the                            and to make
                                    Organisation for                            comparisons
                                    Economic Co-                                internationally.
                                    operation and
                                    Development
Measuring            Australian     Australian              National            To provide information 2002                    Annual updates
Australia’s Progress Bureau of      government                                  on national progress in                        planned
2002                 Statistics     agencies                                    economic, social, and
                                                                                environmental areas.
Minnesota            Minnesota      Federal agencies,       State               To assess progress         1991                Periodicallyb
Milestones 2002      Planninga      Minnesota state                             toward achieving 19
                                    agencies, and                               state goals in four
                                    universities                                areas: increasing the
                                                                                health and well-being
                                                                                of Minnesotans,
                                                                                enhancing community
                                                                                and democracy in the
                                                                                state, protecting the
                                                                                environment, and
                                                                                improving
                                                                                government.
Achieving the        Oregon         Federal agencies        State               To provide information 1989                    Biennially
Oregon Shines        Progress       and Oregon state                            on the economic,
Vision: The 2001     Boardc         agencies                                    social, and
Benchmark                                                                       environmental health
Performance                                                                     of the state in relation
Report                                                                          to its goals.
The Wisdom of Our The Boston        Federal agencies,       Local               To provide information 2000                    Biennially
Choices: Boston’s  Foundation       Massachusetts                               on the health and well-
Indicators of                       state agencies,                             being of Boston, its
Progress, Change                    Boston city                                 neighborhoods, and
and Sustainability                  agencies,                                   the region as a whole.
2000                                universities, and
                                    community-based
                                    organizations
Source: GAO.
                                           a
                                               Minnesota Planning is a state agency created by the Minnesota legislature in 1991.
                                           b
                                               Updated in 1993, 1996, 1998, and 2002.
                                           c
                                            The Oregon Progress Board was created by the legislature in 1989 to develop and implement a state
                                           strategic plan.




                                           Page 16                                              GAO-03-672SP Key National Indicators Forum
                                              Forum




                                              Specialized indicator systems focus on specific subjects or topical areas,
                                              such as health status, the environment, the status of children, and aging on
                                              multinational, national, or local scales. Table 2 provides details on several
                                              illustrative examples of specialized indicator systems.



Table 2: Illustrative Specialized Indicator Systems

                                                                                                          Year report       Frequency of
Indicator system      Reported by          Data sources        Scale             Purpose                  first issued      report updates
America’s           Federal Interagency Federal agencies       National          To provide               1997              Annually
Children: Key       Forum on Child and                                           information on the
National Indicators Family Statistics                                            well-being of
of Well-Being 2002                                                               children.
Healthy People        Department of        Federal agencies    National          To provide indicators    1979              Once a decadea
2010                  Health and Human                                           of progress on a
                      Services                                                   variety of health
                                                                                 policy objectives.
Kids Count 2002       The Annie E. Casey Federal agencies      National          To track the well-       1990              Annually
Data Book             Foundation                                                 being of youth.
Older Americans       Federal Interagency Federal agencies     National          To track the health      2000              Every 3 to 5
2000: Key             Forum on Aging                                             and well-being of                          years
Indicators of Well-   Related Statistics                                         Americans aged 65
Being                                                                            and over.
The State of the      The H. John Heinz    Federal and state National            To provide               2002              Annually; next
Nation’s              III Center for       agencies, private                     information on the                         full addition in
Ecosystems:           Science, Economics   organizations, and                    state of the                               2007
Measuring the         and the              universities                          ecosystems of the
Lands, Waters, and    Environment                                                United States.
Living Resources
of the United
States
The State of the      UNICEF               United Nations’    Multinational      To present             1980                Annually
World’s Children                           agencies, national                    information on the
2003                                       governments, and                      economic and social
                                           the World Bank                        well-being of children
                                                                                 worldwide.
The World Health      World Health         United Nations’    Multinational      To measure the          1995               Annually
Report 2002           Organization         agencies, national                    amount of disease,
                                           governments, and                      disability, and health
                                           the Organisation                      that can be attributed
                                           for Economic Co-                      to certain risks and to
                                           operation and                         calculate how much
                                           Development                           of the burden is
                                           Statistics                            preventable.
Source: GAO
                                              a
                                               Originally, published in 1979 as Healthy People: The Surgeon General’s Report, updated in 1980 as
                                              Promoting Health/Preventing Disease: Objectives for the Nation and in 1990 as Healthy People 2000:
                                              National Health Promotion and Disease Prevention Objectives.



                                              Page 17                                          GAO-03-672SP Key National Indicators Forum
Forum




Regardless of whether they are comprehensive or specialized, indicator
systems vary in terms of data sources and organizations that report on
these systems. As shown in tables 1 and 2, indicator systems can include
data from a variety of sources such as the federal government, local
government, and nongovernmental organizations. For example, Minnesota
Milestones3 uses data from three primary sources: federal agencies, such
as the U.S. Bureau of the Census and the U.S. Energy Information
Administration; state agencies such as Minnesota’s departments of
Revenue and Children, Families and Learning; and educational institutions,
such as the University of Minnesota.

Information on some systems are reported through government agencies,
others through private organizations, and some use a combination of both.
America’s Children4 and Measuring Australia’s Progress5 were both
produced by government agencies. A private foundation with the extensive
participation of government agencies produced The State of the Nation’s
Ecosystems6 while another private foundation produced Kids Count.7 A
private foundation, the city of Boston, and the Metropolitan Planning
Council jointly produced The Wisdom of Our Choices.8 Healthy People
20109 was produced through a public-private partnership between federal

3
  Minnesota Planning, Minnesota Milestones 2002 (St. Paul, Minn.: Minnesota Planning,
2002). http://www.mnplan.state.mn.us/mm/index.html (downloaded Jan. 2003).
4
  Federal Interagency Forum on Child and Family Statistics, America’s Children: Key
National Indicators of Well-Being 2002 (Washington, D.C.: Government Printing Office,
2002). http://childstats.gov (downloaded Jan. 2003).
5
  Australian Bureau of Statistics, Measuring Australia’s Progress 2002: Indicators of
Australia’s Progress
http://www.abs.gov.au/Ausstats/abs%40.nsf/94713ad445ff1425ca25682000192af2/b66ebefc05
cdf265ca256bdc001223ec!OpenDocument (downloaded Jan. 2003).
6
 The State of the Nation’s Ecosystems: Measuring the Lands, Waters, and Living
Resources of the United States (Cambridge, U.K.: Cambridge University Press, 2002).
http://www.heinzctr.org/ecosystems/index.htm (downloaded Jan. 2003).
7
 Annie E. Casey Foundation, Kids Count Data Book 2002 (Baltimore, Md.: 2002).
http://www.aecf.org/kidscount/kc2002/ (downloaded Jan. 2002).
8
  The Boston Foundation, The Wisdom of Our Choices: Boston’s Indicators of Progress,
Change and Sustainability 2000, http://www.tbf.org/boston/index.html (downloaded Feb.
2003).
9
  Healthy People 2010: Understanding and Improving Health (Washington, D.C.: U.S.
Department of Health and Human Services: 2000), http://www.healthypeople.gov/
(downloaded Dec. 2002).




Page 18                                     GAO-03-672SP Key National Indicators Forum
                            Forum




                            agencies, local communities, and professional and trade associations from
                            the health care field.

                            One characteristic that many indicator systems share is that collaboration
                            among various groups was important to their creation. Sometimes the
                            cooperation was across government agencies and sometimes among
                            nongovernmental organizations and government agencies. For example,
                            Older Americans 200010 was produced by a coalition of nine federal
                            agencies11 and supplemented by substantial contributions from three other
                            federal agencies.12 Also, The Wisdom of Our Choices is another example of
                            collaboration among various organizations. This indicator system is
                            maintained and reported on by the Boston Foundation, the City of Boston,
                            and the Metropolitan Area Planning Council with the assistance of local
                            businesses, educational institutions, and community-based organizations.



Observations on Indicator   Four primary observations on indicator systems emerged based on forum
Systems                     discussions and related research. The first observation concerns the
                            purposes of indicator systems. Generally, there are, at a minimum, three
                            broad purposes for indicator systems that are not mutually exclusive.
                            These three purposes are as follows:

                            • Accelerate learning: This type of indicator system contributes to
                              scientific understanding as well as enhances the awareness, insight, and
                              foresight provided to leadership and the public.

                            • Assess position and progress: This type of indicator system involves a
                              broad, constituent-focused aim and requires a generally accepted



                            10
                              Federal Interagency Forum on Aging Related Statistics, Older Americans 2000: Key
                            Indicators of Well Being (Washington, D.C.: 2000),
                            http://www.agingstats.gov/chartbook2000/default.htm (downloaded Jan. 2003).
                            11
                              These nine agencies are the Administration on Aging, the Bureau of Labor Statistics, the
                            U.S. Bureau of the Census, the Department of Health and Human Services’ Office of the
                            Assistant Secretary for Planning and Evaluation, the Health Care Financing Administration,
                            the National Center for Health Statistics, the National Institute on Aging, OMB, and the
                            Social Security Administration.
                            12
                              The three agencies are the Department of Agriculture’s Center for Nutrition Policy and
                            Promotion, the Department of Justice’s Bureau of Justice Statistics, and the Department of
                            Transportation’s National Highway Traffic Safety Administration.




                            Page 19                                      GAO-03-672SP Key National Indicators Forum
Forum




     common vision and comprehensive framework that helps uncover
     especially challenging problems and beneficial opportunities.

• Measure performance: This type of indicator system is specifically
  intended to determine to what degree institutions or projects are
  successful and are producing appropriate benefits for the resources
  they use.

The second observation was that, at the national level, social and
environmental indicators have not received as much attention as other
types of indicators. Specifically, traditional economic and business
indicators have dominated indicator efforts. As an example, while there is
a monthly report on economic indicators and estimates of Gross Domestic
Product are released quarterly, reports on indicators of important social
topics, such as teenage depression and suicides, are relatively more scarce
and less frequent. Additionally, economic indicators give a limited view of
how the country is doing. For instance, Gross Domestic Product, one
traditional economic indicator, does not capture broad quality of life issues.

The third observation was that many indicator systems focus mainly on
objective measures as opposed to indicators that reflect the subjective
perceptions of the public. For example, 29 of the 307 indicators included in
France’s indicator report appear to be subjective measures. The United
Kingdom’s indicator report has over 100 indicators, 3 of which are
subjective measures. While both types of measures are derived using
scientific methods, indicators that reflect subjective perceptions are
viewed as important to include along with objective measures to provide an
evaluation of the state of a city or a nation that takes account of diverse
public points of view.

The fourth observation was that criteria have been developed to help frame
the design of national indicator systems. Specifically, several countries
followed the so-called “Bellagio Principles” in developing their overall
indicator systems.13 These 10 principles are that assessment of progress
(1) are guided by a clear vision and goals, (2) review the whole system as
well as its parts and recognition of the interaction among the parts,

13
  These principles were developed as guidelines for the whole process—choice and design
of indicators, their interpretation, and communication of results—to measure and assess
progress toward sustainable development. They were developed in 1996 at an international
meeting of measurement practitioners and researchers at the Rockefeller Foundation’s
Study and Conference Center in Bellagio, Italy.




Page 20                                     GAO-03-672SP Key National Indicators Forum
                           Forum




                           (3) consider equity and disparity within the current population and over
                           generations, (4) have adequate scope, (5) have practical focus, (6) involves
                           openness, (7) have effective communication, (8) involve broad
                           participation, (9) be an ongoing assessment, and (10) provide institutional
                           capacity.



The Development of         GAO found that the development of national indicators in the United States
Indicators in the United   over the last 75 years has followed three fairly discrete trajectories
                           focusing on economic, social, and environmental issues. Major concerns
States                     facing the nation provided the impetus for each of these trajectories and
                           led to three indicator traditions.

                           • The Great Depression and World War II put a host of economic
                             indicators in wide currency.

                           • The Great Society and civil rights movements enhanced efforts to
                             fashion a wider body of social indicators.

                           • The emergence of the environmental movement brought indicators to
                             measure air and water quality.

                           Solely for the purpose of illustration, table 3 selectively identifies highlights
                           of these three indicator traditions during the 20th century. It is worth
                           pointing out that the inherent strength of the current United States system
                           is its diversity and flexibility. There are numerous specialized and
                           comprehensive indicator systems, driven by either executive or grassroots
                           leadership, in the public and private sectors that have shaped the variety of
                           available information in our society. This table is not exhaustive, nor can it
                           do justice to the diversity of those efforts. However, these highlights do
                           demonstrate three—among possible others—recognizable traditions in the
                           development of the United States’ indicator systems.




                           Page 21                                 GAO-03-672SP Key National Indicators Forum
                                             Forum




Table 3: Selected Highlights of Indicator Traditions in the United States during the 20th Century

Tradition                   Illustrative examples
Economic indicators         National Income and Product Accounts were initially formulated to account for flow of commodities and
                            services during World War II. They provide a base for key economic indicators such as Gross Domestic
                            Product.

                            Business Cycle Indicators, created in the 1930s by the National Bureau of Economic Research, have been
                            compiled by the Conference Board since 1995. The Conference Board determines the specific data series
                            included in the composite leading, coincident, and lagging indicators such as stock prices, employment,
                            and change in consumer prices for services, respectively.

                            The Employment Act of 1946a committed the federal government to the goals of full employment and
                            economic stability. The act created the Council of Economic Advisors that, in 1947, released the first
                            Economic Report to the President.
Social indicators           The U.S. Department of Labor, Children’s Bureau’sb Handbook of Federal Statistics of Children,c published
                            in 1913, attempted to bring together “scattered” federal data and other information on children’s welfare.
                            The Handbook was an early effort to develop indicators for consistent monitoring of children and health.

                            A proposed bill called the Full Opportunity and Social Accounting Actd was first introduced in 1967.
                            Although, the bill was never passed, it called for an annual social report from the President to the Congress
                            and helped focus a national dialogue on social indicators.

                            The Department of Health, Education and Welfare published a report, in 1969, on social indicators called
                            Toward a Social Report.e The report was prepared at the direction of President Johnson who sought “ways
                            to improve the nation’s ability to chart its social progress.” In 1973, the federal statistical agencies
                            published a report on social indicators. Subsequent reports on social indicators were published in 1976
                            and 1980.
Environmental indicators    The National Environmental Policy Actf (NEPA) was signed into law on January 1, 1970, and required
                            federal agencies to assess the impacts of their decisions on the natural environment. While NEPA did not
                            establish any specific indicators, it does require that federal agencies assess the environmental effects of
                            major federal actions significantly affecting the environment. NEPA also established the Council on
                            Environmental Quality to advise the President on environmental matters and to annually report on the
                            state of the environment.

                            During the same year, the Environmental Protection Agency—an independent agency to establish and
                            enforce federal air standards and water pollution control laws and to monitor the environment—was
                            created. The Clean Air Act of 1970g was passed that year as well. These initiatives focused national
                            attention on indicators of environmental quality.

                            The Endangered Species Act of 1973h suggests indicators of species viability, such as size and
                            geographical distribution of species’ populations and their habitats. These indicators can be used as the
                            basis for avoiding the extinction of species.
Source: GAO.
                                             a
                                              Pub. L. 79-304, Feb. 20, 1946.
                                             b
                                              The Children’s Bureau, created in 1912, is now located within the U.S. Department of Health and
                                             Human Services’ Administration for Children and Families, Administration on Children, Youth and
                                             Families. It is responsible for assisting states in delivering child welfare services.
                                             c
                                               U.S. Department of Labor, Children’s Bureau, Handbook of Federal Statistics of Children
                                             (Washington, D.C.: Government Printing Office, 1913).




                                             Page 22                                            GAO-03-672SP Key National Indicators Forum
Forum




d
    90th Congress, S.843.
e
U.S. Department of Health, Education and Welfare (HEW) (Washington, D.C.: U.S. Government
Printing Office, 1969).
f
42 U.S.C. §§ 4321 – 4347.
g
    42 U.S.C. §§ 7401 – 7671q.
h
    16 U.S.C. §§ 1531 – 1544.


While much of the development of national indicators in the United States
has focused on specific economic, social, or environmental concerns, the
importance of interrelationships among these dimensions is growing. For
example, economists are working to develop new measures of economic
performance that take into account various social and environmental costs.
While initial interest in social indicators began as a challenge to the
centrality of economic indicators in policy discussions, the focus of the
social indicator tradition expanded through the development of
frameworks to integrate economic as well as social indicators. Striving to
understand the impact of human society on the environment involves
focusing on the interrelationships among economic, social, and
environmental processes.




Page 23                                        GAO-03-672SP Key National Indicators Forum
                              Forum




What Might the United         After reviewing research on how the world’s democracies are tackling
                              indicator development, the bulk of the forum’s discussion turned to
States Do to Improve          implications and issues for the United States.
Its Approach and Why?
                              The participants generally agreed that an improved, more comprehensive
                              approach to assessing the nation’s position and progress should be
                              developed. They noted that such an approach should cover a wide variety
                              of information areas—ranging from the macroeconomy and social support
                              to education and health. In addition to identifying a variety of ideas for
                              improving the nation’s approach, the Federal Interagency Forum on Child
                              and Family Statistics was discussed as a potential process/structure model
                              to emulate. A straw proposal of a indicator system for the United States
                              was presented, leading to much fruitful dialogue on what can be done with
                              existing data (e.g., on economics and children) as well as what should be
                              done to develop new information in important, but neglected areas (e.g.,
                              personal and national security and socioeconomic mobility).



Ideas to Improve the United   The participants agreed that a more comprehensive system for measuring
States’ Approach              national performance could be beneficial. They also recognized that the
                              process of generating indicators for a national system is as important as the
                              specific indicators that would be identified and measured as a result of the
                              process. Hence, the process should be as inclusive as possible, and this
                              inclusiveness should show itself from the very beginning of any effort. For
                              example, state and local governments should have significant roles and
                              responsibilities in helping to develop and implement national indicators, in
                              part because the federal government has devolved responsibilities for
                              many social issues to state and local governments.

                              Several additional ideas for improving the United States’ approach to
                              measuring national progress were raised by participants. These ideas fell
                              into four broad categories: (1) key questions for framing the agenda for a
                              new system, (2) public outreach, (3) communication and dissemination,
                              and (4) key data issues.

                              Key questions for the future agenda: Participants proposed a variety of
                              questions to help frame an agenda for the possible development of a key
                              national indicator system for the United States. These questions included
                              the following:




                              Page 24                               GAO-03-672SP Key National Indicators Forum
Forum




• What is the purpose and value of the national indicator system to be
  developed? In particular, what do we need to do differently, why, and
  what net risk-adjusted benefits might the system achieve?

• Who are the audiences (e.g., general public, educators, policymakers,
  and professionals), and how will they benefit?

• What would a broadly accessible and useful collection of key national
  indicators look like?

• How would the indicator system be designed, developed, implemented,
  operated, used, improved, and communicated? In particular, how will
  the need to build short-term momentum be balanced against the need
  for longer-term persistence and perspective on the initiative?

• What data exist to serve as a foundation for a national effort? Are there
  important data gaps, and what is the quality of the available data?

• What is an appropriate standard for progress, and what are the potential
  unintended consequences or behavior changes from efforts to
  demonstrate progress?

• What is the response system and how does it work when an indicator
  increases or decreases? Is there a response system in place to make use
  of national indicators in everyday life?

• What are the experiences of other countries regarding unintended
  consequences of meeting performance measures?

• Are there examples of how national indicators have been used to inform
  decision making?

• How much time and how many resources will a national effort require?
  How will those resources be allocated to alternative uses, such as
  making existing indicators more widely available and usable by broader
  audiences versus building existing institutional capacity to produce
  more and better indicators?

• Do the nation’s leading institutions (e.g., governmental, commercial, and
  nonprofit organizations) have the capacity to carry out this effort?




Page 25                              GAO-03-672SP Key National Indicators Forum
Forum




Public outreach: Developing an indicator system requires extensive
outreach to targeted audiences. Such audiences could include some or all
of the following: the general public, public leaders, the media, educational
institutions, scientific and professional communities, and public interest
groups. To be useful, indicators must have consistent form and be clear,
easy to digest, user-friendly, and timely. The data also need to be provided
in the appropriate context rather than merely presented in freestanding
charts or tables. And there must be extensive attention paid to the
processes, not only of audience understanding but assent to the
importance of the information areas and indicators eventually chosen.

Communication and dissemination: Communicating and disseminating
information is essential to sustaining interest in any indicator system.
However, this can involve significant time and resources. The media will
play a key role in communication and dissemination. A carefully thought
out approach to working with the media will be essential for any degree of
success. Some organizations, such as the Annie E. Casey Foundation,
commit significant financial resources to communicating and marketing
indicator information. Another issue raised was a strategic question of
how the “marginal dollar” of a key indicator effort should really be spent.
For instance, in some cases it may be more valuable to communicate
existing information for broader impact than to develop new sources of
information.

Key data issues: The federal statistical system and federal programs
produce much data, and they are relied on by the nation. The data are
widely accepted because they are “official.” The data are produced using
generally accepted practices and principles and are based on sound
statistical methodologies for the purposes for which the data were
produced. There is also a substantial amount of data produced by the
private sector, of which an important component is viewed as proprietary,
not public. Hence, two key data issues are quality and availability. In terms
of quality, since there is a known quality of official statistics and sometimes
an unknown quality of private statistics, how can this variation in
knowledge of data quality be addressed so both can be used for a national
indicator system? In terms of availability, if certain proprietary information
were essential for assessing the position and progress of the nation, how
would those data be made more widely available? There needs to be a
collective effort to address both of these data issues if a national indicator
system is to be successful.




Page 26                                GAO-03-672SP Key National Indicators Forum
                            Forum




A Process/Structure Model   Several existing models could be used as reference points for designing a
for Developing a National   United States indicator system. Two current examples cited were the
                            federal interagency forums on (1) child and family statistics and (2) aging-
Indicator System            related statistics. A more detailed discussion centered on the Federal
                            Interagency Forum on Child and Family Statistics. This was presented as
                            an example from the United States’ federal statistical system of a process
                            and organizational structure for developing indicators within a specialized
                            area. It should be noted that, at this stage, little discussion of these models
                            concerned the crucial issue of funding, which will be vital to elucidate in
                            order to make any practical progress on applying their lessons to a national
                            effort.

                            In 1994, seven agencies joined together to create the Federal Interagency
                            Forum on Child and Family Statistics (Interagency Forum). Three years
                            later the Interagency Forum was formally established by an Executive
                            Order and directed to develop priorities for collecting enhanced data on
                            children and youth, improve the reporting and dissemination of
                            information on the status of children to the policy community and the
                            general public, and produce more complete data on children at the state
                            and local levels. The Interagency Forum now has participants from 20
                            agencies as well as partners in private research organizations. It holds
                            several public hearings with agency members each year to discuss key
                            issues and ideas.

                            Annually, the Interagency Forum produces a report called America’s
                            Children: Key National Indicators of Well-Being. The 2002 report is the
                            group’s sixth annual report to the nation on the condition of children in
                            America. The Interagency Forum’s report presents 24 key indicators on
                            important aspects of children’s lives, including their economic security,
                            health, behavior and social environment, and education. It also presents
                            eight contextual measures that describe changes in the characteristics of
                            the population as well as in children’s family settings and living
                            arrangements.

                            The Interagency Forum chose the indicators through careful examination
                            of available data. Data were drawn from national surveys and vital records.
                            Input on which indicators to choose was also sought from the federal
                            policy-making community, foundations, academic researchers, and state
                            and local children’s service providers. The implication of this discussion
                            was that the Interagency Forum could be studied and replicated as a model
                            for other information areas for the United States’ new approach.




                            Page 27                                GAO-03-672SP Key National Indicators Forum
                             Forum




A Straw Proposal for a New   Commissioned by GAO, in cooperation with the National Academies, Dr.
National Indicator System    Riche produced an independent straw proposal for a key national indicator
                             system called USA Series 0.5. USA Series 0.5 served as a starting point for
                             the forum’s participants to discuss the framework of a key national
                             indicator system and, in the future, move to the next version of an indicator
                             system. Moving to an initial version of a national indicator system,
                             identified as USA Series 1.0, would involve formal and institutional
                             consensus, audience input, and would be the first step toward an evolving
                             key national indicator system.

                             Dr. Riche developed a group of draft principles for developing a key
                             national indicator system. These principles included the following:

                             • The set of indicators is about the nation, not just the government:
                               Defining key national indicators goes beyond any one sector or level of
                               government.

                             • If the set of indicators is about the nation, it must incorporate the
                               nation’s components. Local, state, regional and federal governments, as
                               well as private for-profit and not-for-profit sectors should work to
                               coordinate and integrate their own efforts into a national perspective.

                             • If the set of indicators is national and intended to drive decision making,
                               it must be comprehensive. It should be comprehensive, not just
                               specialized and it must integrate the links and interactions between
                               component measures.

                             • If the set of indicators is to be useful, the information must be targeted
                               and trusted. The set of indicators should be selected based on specific
                               criteria. These criteria might include the significance, objectivity,
                               accuracy, scope, timeliness, accessibility, clarity, efficiency,
                               comparability, and contextual sophistication of a set of indicators.

                             • If the set of indicators is to be credible, it must be both science-based
                               and understandable. The set of indicators should help formulate
                               questions about what knowledge is needed so sensible scientific
                               statements can be made and a framework on key areas of research and
                               investigation can be developed.

                             • If the set of indicators is to be used to monitor progress, the public must
                               be both involved and included. This principle implies a need for polling
                               and related research to define what Americans want for their country.


                             Page 28                               GAO-03-672SP Key National Indicators Forum
Forum




• If the set of indicators is to have staying power, it must acknowledge the
  reality of resource constraints and the corresponding need for judgment
  and compromise. A true national effort will need to be based on some
  type of public/private partnership. No one sector of society can “own”
  the effort.

Additionally, three basic types of indicator approaches were described
since the United States’ current approach for measuring performance
includes all three approaches. These three approaches are as follows:

• Composite indicators: This approach combines information from
  several different indicators into a single composite number. An
  example of this approach is the United Nations’ annual Human
  Development Index. The composite approach is a tool for
  communicating directional progress to a large audience, especially in a
  comparative context. However, composite numbers require a
  consensus on weighting the different indicators that is hard to achieve.

• A unified, balance sheet of indicators: This approach uses an
  accounting framework and presents data in a unified system of
  accounts. Under this approach, the indicators are both gathered and
  presented within a coherent hierarchical system. Most countries have a
  similar set of economic accounts, such as the United States’ National
  Income and Product Accounts, that are linked at a certain level of detail
  by the United Nations-sponsored System of National Accounts.

• A suite of indicators: This approach groups information areas and key
  indicators together. Through use of a suite approach, the links between
  the information areas can be discussed even though not every
  information area needs to be fully developed as a measure.

What the proposal is: USA Series 0.5 specifically addresses areas in which
national performance might be measured. It includes a suite of indicators
that has been prominent in past efforts in this country. It also includes
indicators from other countries whose economic and social systems are, in
some respects, comparable to our own. The proposal includes information
areas that are developing as well as those that are advanced.

What the proposal is not: USA Series 0.5 is not systems based because it
lacks a firm mission statement and conceptual framework and does not
depend on any particular structure. It does not have an identified audience.
USA Series 0.5 does not presume to be complete in terms of including the



Page 29                               GAO-03-672SP Key National Indicators Forum
Forum




many information areas that might likely be incorporated into later
versions. It does not attempt to propose indicators for new or “formative”
information areas that have, by definition, large knowledge gaps because
they are taking shape based on new questions being asked about our nation
and our world.

Overview of the proposal: USA Series 0.5 consists of three broad
categories—economic, social, and environmental. Within these three
categories are a variety of information areas that are classified, in Dr.
Riche’s opinion, by how well data associated with a specific information
area are developed. The three classifications of the development stages of
information areas are (1) advanced, (2) developing, and (3) formative. This
development construct is not tested and there is no consensus as to which
information areas are most advanced. The specific structure of the
proposal—including 11 specialized information areas—is shown in table 4
below. The proposal also included specific indicators for the advanced and
developing information areas, such as crime, ecology, education, and
governance. (See app. II.) There were no specific indicators chosen for the
information areas that were considered to be in the formative stage (e.g.,
sustainability, transparency, and security).



Table 4: Structure of USA Series 0.5

Information area
development stage Economic                                       Social                           Environmental
Advanced                        Macroeconomy                     Education
                                                                 Health
                                                                 Crime
                                                                 Social Support
Developing                                                       Community                        Ecology
                                                                 Governance
Formative                       Sustainability                   Sustainability                   Sustainability
                                Transparency                     Transparency
                                                                 Security
Source: Dr. Martha Farnsworth Riche, The United States of America Developing Key National Indicators. (Paper presented at the
forum.)


In principle, advanced information areas have a great deal of reliable data
and relatively broad public and scientific consensus as to their importance
and method of production. Figure 1 shows data for life expectancy, which
illustrates the characteristics of an advanced information area.
Specifically, life expectancy data are based on well-organized bodies of



Page 30                                                          GAO-03-672SP Key National Indicators Forum
Forum




reliable data and there is a high degree of scientific and political consensus
on their significance.



Figure 1: Life Expectancy at Birth and at Age 65
 90 Years of life

 80

 70

 60

 50

 40

 30

 20

 10

  0
      1900             1920              1950    1960       1970      1980       1990       2000
      Year
                At birth
                At age 65
Source: National Center for Health Statistics.

Note: From 1900 to 1950, no data were available for years of life at age 65.


In contrast, developing information areas have a higher proportion of
indicators that are evolving and lack a broad technical or public consensus
about significance. Greenhouse gas emissions, as shown in figure 2, is an
example of a developing information area. While these emissions have
gained increasing currency in debate and policy making, they are based on
estimates of component gases and there is some scientific and public
uncertainty about their importance and possible implications.




Page 31                                                 GAO-03-672SP Key National Indicators Forum
                                                               Forum




Figure 2: U.S. Emissions of Greenhouse Gases, Based on Global Warming Potential, 1990-2000 (in Million Metric Tons of Carbon
Equivalent)
2000 Million metric tons




1500




1000




  500




    0
        1990             1991                1992       1993             1994       1995   1996       1997      1998        1999       2000
          Year
                    Carbon dioxide
                    Methane
                    Nitrous oxide
                    Hydrofluorocarbons, perfluorocarbons, and sulfur hexafluoride
                    Total
Source: Energy Information Administration.


                                                               Formative information areas are new areas, on issues of potential
                                                               significance, for which data may not exist. They need pilot projects to
                                                               outline suitable information databases. An example of a formative
                                                               information area would be a measure of public confidence in personal
                                                               security. While there are some data on elements of public confidence in
                                                               personal security, a broad consensus on the definition of personal security
                                                               does not exist.



Reactions to the Proposal                                      While the participants were receptive to the proposal as a starting point, it
                                                               prompted several reactions. One reaction was that attention should be
                                                               paid to the process of developing the next version. While people
                                                               acknowledged the expediency and practicality of starting with existing
                                                               data, most participants wanted to work on filling in the gaps between
                                                               versions 0.5 and 1.0. A participant suggested that the process that
                                                               produced the “Healthy People 2010” initiative could be a possible model of
                                                               the effort. The “Healthy People 2010” development process was described



                                                               Page 32                               GAO-03-672SP Key National Indicators Forum
Forum




by some as having been exhaustive. It had several advantages, including an
organized approach to automation and a human capital infrastructure at
the federal, state, and local levels.

Another reaction, related to the one above, was to suggest that it is
important to have a mechanism or process that would allow for creating
new indicators and/or revising existing indicators. This mechanism or
process would need to capture the public’s changing concerns and other
changes nationally and internationally. While it would be a challenge to do,
one suggestion to address the challenge was for the system to have a set of
“regular” indicators that will remain meaningful over time and a set of
“special” indicators that apply in specific situations.

A third reaction focused on the need to disaggregate data, which
participants considered an important but challenging task. Although
aggregated information at the national level needs to be presented, the data
also need to be disaggregated into specific categories that are relevant to
localities and the public. This capability will also allow localities to relate
to and understand how they fit into a larger picture. Health indicators were
provided as an example. It is useful to have national data on health. But
the information is even more useful if it provides information about health
in a specific city or neighborhood. Also, careful consideration should be
given as to how data are disaggregated, since there can be degradation of
quality due to smaller sample sizes. Applying the lenses of age, race,
gender, and geographical location to indicators facilitates identifying
trends among specific groups that are masked in aggregated data.




Page 33                                GAO-03-672SP Key National Indicators Forum
                          Forum




What Are Important        While participants generally agreed that the straw proposal’s 11
                          information areas were important to measure, they thought a majority of
Areas to Measure in       the areas needed refinement and enhancement. (For the specific
Assessing U.S. National   indicators defined in the proposal, please see app. II) Also, 4 additional
                          information areas were identified as candidates for including in the
Performance?              proposal. The list of information areas that needed to be refined or added
                          to the proposal was not intended to be exhaustive. Furthermore,
                          participants did not discuss how many indicators might be included under
                          each area. Instead, the proposal was meant to represent a good place to
                          start to build a more comprehensive national indicator system.



Information Areas That    Participants identified seven information areas included in the 0.5 proposal
Need Further Refinement   that need to be further refined. These areas are community, education,
                          governance, macroeconomy, security, social support, and sustainability.

                          Community
                          Discussion on this information area focused on refining the area to include
                          the concept of civic engagement. Civic engagement was described as the
                          connectedness of individuals to society and to each other. Further, it
                          involves social capital such as participation in social and cultural
                          organizations, public service, volunteering, and voting. Some participants
                          viewed civic engagement as an undermeasured and underappreciated area,
                          while others feel there is already a good deal of information available. This
                          suggests, as may be the case in other areas, the value of undertaking
                          systematic inventories of existing data. While some participants thought
                          civic engagement should be included within the community information
                          area, others thought it should be within the governance information area.

                          Education
                          Discussion on this information area focused on the need to include both
                          the means by which individuals can attain personal enrichment and
                          improved quality of life and, at the societal level, the extent to which
                          society is served by the educational system.

                          Governance
                          Some participants viewed this information area as being not as well
                          thought out as others. It should incorporate the concepts of public trust
                          and its relationship to institutions. It may also need to cover the
                          effectiveness of governance (e.g., political participation and corruption).
                          Within the concept of public trust, there was some discussion about the



                          Page 34                               GAO-03-672SP Key National Indicators Forum
Forum




need to measure elements of freedom or patriotism, security and civil
liberties. The discussion of public trust and institutions encompassed both
corporations and government at all levels. Additionally, this concept
relates to the responsiveness of institutions to the perceived needs of the
public.

Macroeconomy
Discussion on this information area focused on two broad topics: (1) the
nation’s role in the world and (2) its economic well-being (e.g., competitive
advantage). Measuring the nation’s role in the world involves portraying
and tracking the nation as a member of the world community. This
includes such issues as foreign aid by government and private
organizations, United States military presence overseas, anti-American
sentiments, and economic and cultural globalization. The concept of
competitive advantage involves measuring key economic areas, such as
technology; innovation; the mobility and flexibility of the nation’s labor
force (e.g., geographic, career, and social class mobility); the ability to
attract, retain, and develop good people; education; and trade. It was
thought that measuring the nation’s competitive advantage should involve
national, regional, state, and community levels.

Security
Participants identified this information area as needing focus on personal,
community, national, and international security. This area was identified as
a new area of focus for the nation since the terrorist attacks of September
11, 2001, and other terrorist threats. The area could be approached as a
subjective measure of individuals’ sense of security and as an objective
measure of the extent to which important infrastructure systems are
secure. There were different views as to whether personal security should
be included within the security information area. Some participants
thought personal security was already included in the crime information
area of the proposal. Others thought that personal security should be
included within the security information area since it was viewed as an
expanded notion of public safety given the new focus on the issue,
especially safety from terrorist threats.

Social Support
Participants identified this information area as needing to include both the
well-being of children and the well-being of the elderly—those classified by
society as dependents. Ever increasing attention is being given to the
importance of investment in all aspects of child development (e.g., the No
Child Left Behind Act of 2001). The well-being of the elderly, a complex but



Page 35                               GAO-03-672SP Key National Indicators Forum
                              Forum




                              pressing issue given the well known aging of the baby boomers, is also
                              receiving increased attention.

                              Sustainability
                              Participants viewed this information area as one area that does not fall
                              clearly within a distinct information area. It involves several information
                              areas and encompasses the proposal’s three broad information categories--
                              economic, social, and environmental. Sustainability involves the concept
                              of leaving a legacy for future generations as well as the notion of
                              environmental and social capital and liabilities.



Information Areas to Add to   Participants identified four areas that should be included in the proposal:
the Proposal                  communications, diversity, individual values, and socioeconomic mobility.

                              Communications
                              Discussion on this area involved the various forms of media industry—such
                              as television, radio, newspapers, and the Internet—and how to determine
                              the impact on, and accountability of, the industry. One possible measure
                              suggested involved the degree of citizen access to various media sources or
                              types of communication (e.g., the Internet).

                              Diversity
                              Participants saw this area as one area that goes beyond the concept of
                              fairness to encompass the pluralistic nature of our society. Diversity, in all
                              its forms (e.g., population, culture, and points of view), was seen as an
                              asset that is consistent with the founding principles of the nation and vital
                              to its health. It was agreed that this area would be possible, but very
                              difficult, to measure.

                              Individual Values
                              This area cuts across other areas such as community and quality of life.
                              However, because the concept of individual values does cut across other
                              areas, no agreement was reached as to whether the area should stand alone
                              or be included under several other areas. It was generally agreed though
                              that individual values could be measured.

                              Socioeconomic Mobility
                              Some participants defined this area as access to opportunities such as
                              education and jobs. However, when measuring access to jobs there needs
                              to be a distinction between access to good jobs and bad jobs. For others,
                              this concept had a broader meaning relating to how rapidly individuals in



                              Page 36                                GAO-03-672SP Key National Indicators Forum
Forum




the United States are moving from one standard of living to another. This
area was seen as being rooted in basic American values and as an essential
component of the “American dream.”




Page 37                              GAO-03-672SP Key National Indicators Forum
                     Forum




How Might New U.S.   The last segment of the forum was devoted to discussing how to take
                     meaningful action on such a challenging idea. Recognizing that any effort
Approaches Be Led    pertaining to such a large, diverse nation would of necessity be long term,
and Implemented?     the participants’ discussion focused on how to develop, sustain, and fund
                     an effort.



Development          Information areas tend to develop in an evolutionary fashion, over
                     relatively long periods, and in some definable stages as they reach greater
                     degrees of consensus and transparency. There was wide acknowledgment
                     that any United States indicator system, because of the rapidly changing
                     nature of our society, would by definition be evolving. In other words,
                     there would be many successive “versions,” each one improving on the one
                     before it. Table 5 illustrates the possible evolution of a United States
                     indicator system by showing how, over time, additional information areas
                     can be added and developed in a cumulative fashion. Specifically, Series
                     1.0-2.0 would contain not only new information areas (e.g., energy and
                     citizenship) but also the information areas from earlier versions (such as
                     education and community in Series 0.5). This table, shown as it was
                     presented at the forum, includes some but not all areas mentioned by the
                     participants as well as some used by other nations.




                     Page 38                               GAO-03-672SP Key National Indicators Forum
                         Forum




                         Table 5: Proposed Evolutionary Process for a National Indicator System (Includes
                         Illustrative Information Areas)

                         Information area                                                    Series 1.0-2.0                     Series N.0
                         development stage                  Series 0.5                       (cumulative)                       (cumulative)
                         Advanced                           Crime/public safety              Energy                             Community
                                                            Education                        Labor and                          Ecology
                                                            Health                            employment                        Governance
                                                            Macroeconomy                     Microeconomy
                                                            Social support                   Resource allocation
                                                                                             Special populations
                         Developing                         Community                        Citizenship                        Families
                                                            Ecology                          Competitiveness                    Innovation
                                                            Governance                       Infrastructure                     Justice
                                                                                             Wealth/prosperity                  Knowledge
                                                                                             Well-being/happiness
                         Formative                          Security                         Accountability                     Arts and Culture
                                                            Sustainability                   Cities                             Civility
                                                            Transparency                     Diversity                          Freedom
                                                                                             Equity                             Mobility
                                                                                             Globalization                      Opportunity
                                                                                             Neighborhoods                      Trust
                                                                                                                                Values
                         Source: Dr. Martha Farnsworth Riche, The United States of America Developing Key National Indicators. (Paper presented at the
                         forum.)


                         There was discussion about the scope of a “comprehensive” indicator set.
                         Participants observed that a comprehensive set of indicators would include
                         both indicators that are fairly advanced in their development as well as
                         indicators that are new and thus require substantial development. It would
                         include indicators at all levels, in all sectors, and in all disciplines.
                         Furthermore, it would focus on both areas where much is already known
                         as well as areas needing further research. This would help create a
                         learning agenda for developing and sustaining indicators.



Sustaining and Funding   The discussion of sustaining and funding the effort to develop a key
                         national indicator system focused on the need for forming a public/private
                         partnership. Such a partnership would need a structure that provided a
                         broad, independent, flexible, and responsive base for the effort.
                         Participants expressed the view that unless the effort is a partnership
                         between public and private entities, it will not be sustained.

                         Participants agreed that creating a national coordinating committee and a
                         variety of task forces was the best governance mechanism with which to



                         Page 39                                                          GAO-03-672SP Key National Indicators Forum
Forum




initiate a national indicators initiative. Such an initiative would include all
forum participants and representatives from any other institutions
interested in the effort would be invited to participate. As of May 7, 2003,
the organizations that have agreed to participate are: AAU, The Conference
Board, the Council for Excellence in Government, GAO, ICMA, The
National Academies, the National Association of Asian American
Professionals, OMB, and the White House Council on Environmental
Quality.

The consensus was that the national coordinating committee should be
viewed in a facilitative and catalyst role as opposed to a day-to-day
leadership role. To help promote this role of the committee, it was agreed
that an individual committee member would chair the committee for one
meeting and then the chairmanship would rotate to another member. The
first meeting of the national coordinating committee will be held in the
spring of 2003 at the offices of the Council for Excellence in Government
and be chaired by the Council’s President.

The national coordinating committee’s most important tasks will probably
involve providing effective communications, creating a strategy, and
obtaining resources. Specifically, the committee will need to focus on
articulating the national indicator initiative’s purpose, organizing process
and structure, agreeing on governing principles, identifying additional
participants, and developing short- and long-term plans. One participant
noted, and the rest agreed, that while they voted to move forward with this
effort, there were many conceptions of what “this effort” is. Participants
noted that objectives need to be clarified and agreement reached on how to
proceed. They agreed that developing a charter and set of principles for
the initiative would be addressed at the first meeting of the coordinating
committee.

Committee task forces could focus on a variety of issues such as
conducting research, creating short- and long-term action plans,
capitalizing operations, identifying possible organizational models to
sustain the effort in the long term, and investigating communications and
technology solutions. To be successful, people with different skill sets will
need to be brought into the effort. Specifically, experts in communication
and technology, not just experts in data and indicators, need to be involved.
The exact number and functions of all the task forces have yet to be
decided.




Page 40                                GAO-03-672SP Key National Indicators Forum
Forum




The funding arrangement for this effort was cited as a tremendously
important issue since it will require a substantial amount of time and
resources to start and sustain a true national effort. A brief discussion on
the topic concluded that a range of possible funding alternatives, including
private sources and federal funds, need to be studied.




Page 41                               GAO-03-672SP Key National Indicators Forum
Appendix I

Forum Participants                                                                                    AA
                                                                                                       ppp
                                                                                                         ep
                                                                                                          ned
                                                                                                            n
                                                                                                            x
                                                                                                            id
                                                                                                             e
                                                                                                             x
                                                                                                             Iis




Derek Bok               The 300th Anniversary University Professor and President Emeritus,
                        Harvard University

Donald Borut            Executive Director, National League of Cities

Charles Bowsher         Former Comptroller General of the United States, U.S. General Accounting
                        Office

Heinrich Brungger       Director, Statistics Division, The United Nations Economic Commission for
                        Europe

Philip M. Burgess       President, National Academy of Public Administration

Lala Camerer            Deputy Director, Global Access Division, Center for Public Integrity

Richard Cavanagh        President, The Conference Board

E. William Colglazier   Executive Officer, The National Academies

Rita Colwell            Director, National Science Foundation

Michael Delli Carpini   Director, Public Policy Program, The Pew Charitable Trusts

Gene Dodaro             Chief Operating Officer, U.S. General Accounting Office

Cynthia Fagnoni         Managing Director, Education, Workforce, and Income Security, U.S.
                        General Accounting Office

Scott Farrow            Chief Economist, U.S. General Accounting Office

Harvey Fineberg         President, Institute of Medicine, The National Academies

William Galston         Professor and Director, University of Maryland Institute for Philosophy and
                        Public Policy

Gaston Gianni           Vice-Chair, President’s Council on Integrity and Efficiency and Inspector
                        General, Federal Deposit Insurance Corporation




                        Page 42                              GAO-03-672SP Key National Indicators Forum
                      Appendix I
                      Forum Participants




John Graham           Administrator, Office of Information and Regulatory Affairs, Office of
                      Management and Budget

Robert Groves         Director, Survey Research Center, University of Michigan

Judith Gueron         President, Manpower Demonstration Research Corporation

Hermann Habermann     Deputy Director, U.S. Bureau of the Census

Nils Hasselmo         President, Association of American Universities

Harry Hatry           Director, Public Management Program, The Urban Institute

Theodore Heintz       White House Council on Environmental Quality

Christopher Hoenig    Director, Strategic Issues, U.S. General Accounting Office

Helen Hsing           Director-Special Projects, Office of the Comptroller General, U.S. General
                      Accounting Office

Robert Kaplan         Professor, Harvard Business School

Nancy Kingsbury       Managing Director, Applied Research and Methods, U.S. General
                      Accounting Office

Andrew Kohut          Director, Pew Research Center for the People and the Press

John Koskinen         Deputy Mayor/City Administrator, District of Columbia

Bill Kovach           Chairman, Committee of Concerned Journalists

Risa Lavizzo-Mourey   President and CEO, The Robert Wood Johnson Foundation

Rosemary Marcuss      Deputy Director, Bureau of Economic Analysis

Sylvia Mathews        Chief Operating Officer and Executive Director, The Bill & Melinda Gates
                      Foundation




                      Page 43                               GAO-03-672SP Key National Indicators Forum
                          Appendix I
                          Forum Participants




Thomas McCool             Managing Director, Financial Markets and Community Investment, U.S.
                          General Accounting Office

Patricia McGinnis         President and CEO, The Council for Excellence in Government

Sara Melendez             Professor of Nonprofit Management, The George Washington University
                          and Former President, The Independent Sector

Alex Michalos             Director, Institute for Social Research and Evaluation, University of
                          Northern British Columbia

Marc Miringoff            Founder and Director, The Fordham Institute for Innovation in Social
                          Policy

Daniel Mulhollan          Director, Congressional Research Service

Janet Norwood             Former Commissioner, Bureau of Labor Statistics

Robert O’Neill            Executive Director, International City/County Management Association

Robert Parker             Chief Statistician, U.S. General Accounting Office

Victor Rezendes           Managing Director, Strategic Issues, U.S. General Accounting Office

Martha Farnsworth Riche   Former Director, U.S. Bureau of the Census

Dorothy Ridings           President and CEO, Council on Foundations

John Rolph                Chairman, Committee on National Statistics, The National Academies

Thomas Sawyer             Former Representative, State of Ohio, 14th District

William Scanlon           Managing Director, Health Care, U.S. General Accounting Office

Max Singer                Board Member and Senior Fellow, Hudson Institute

Pete Smith                President and CEO, Private Sector Council




                          Page 44                               GAO-03-672SP Key National Indicators Forum
                              Appendix I
                              Forum Participants




Edward Sondik                 Director, National Center for Health Statistics, Department of Health and
                              Human Services

Edward Spar                   Executive Director, Council of Professional Associations on Federal
                              Statistics

Jeffrey Steinhoff             Managing Director, Financial Management and Assurance, U.S. General
                              Accounting Office

F. Michael Taylor             President, National Association of Local Government Auditors

Michael Teitelbaum            Program Director, Alfred P. Sloan Foundation

Dennis Trewin                 Chief Statistician, Australia

Kathleen Utgoff               Commissioner, Bureau of Labor Statistics

David M. Walker               Comptroller General of the United States, U.S. General Accounting Office

Katherine Wallman             Chief Statistician of the United States, U.S. Office of Management and
                              Budget

Andrew White                  Director, Committee on National Statistics, The National Academies

Jacquelyn Williams-Bridgers   Managing Director, Strategic Planning and External Liaison, U.S. General
                              Accounting Office

Vincent Yee                   National President, National Association of Asian American Professionals

Randall Yim                   Managing Director, Homeland Security and Justice, U.S. General
                              Accounting Office




                              Page 45                              GAO-03-672SP Key National Indicators Forum
Appendix II

Illustrative Indicators by Information Area for
USA Series 0.5 1                                                                                        Appendx
                                                                                                              Ii




                Community Information Area1

                •   Rate of volunteering, through nonprofit or charitable organizations
                •   Youth rates of volunteering
                •   Charitable contributions as a percentage of incomes
                •   Attendance at events and institutions that address the national heritage
                    (such as monuments, historical sites, and national parks)
                •   Attendance at performing arts, by categories
                •   Participation in organized sports
                •   Voting rates
                •   Reported hate crimes
                •   Allocation of free time
                •   Homelessness

                Crime/Public Safety Information Area

                • Crime victimization rates (by subgroups such as age, sex, and
                  race/ethnic origin)
                • National crime rate
                • Violent crime rate
                • Property crime rate
                • Incarceration (as percentage of population, by age rates and by race/
                  ethnic origin)
                • Deaths due to transportation accidents
                • Deaths due to fires
                • Proportion of jail inmates who committed offense to get money for
                  drugs
                • Percentage of working age population providing protective services
                • Percentage of population afraid to walk alone after dark

                Ecology Information Area

                •   Level of nitrogen oxide as a percentage of acceptable levels
                •   Level of sulfur oxide as a percentage of acceptable levels
                •   Level of carbon dioxide as a percentage of acceptable levels
                •   Per capita water consumption
                •   Some measure of water quality, for example, percentage of population
                    with access to safe drinking water


                1
                  The indicators and information areas for USA Series 0.5 were proposed by Dr. Martha
                Farnsworth Riche to facilitate a dialogue among the forum’s participants.




                Page 46                                      GAO-03-672SP Key National Indicators Forum
Appendix II
Illustrative Indicators by Information Area
for USA Series 0.5




•   Change in status of species at risk of loss
•   Protected areas as a proportion of vulnerable areas
•   Emissions of greenhouse gases per capita
•   Net greenhouse gas emissions per Gross Domestic Product
•   Reduction of emissions of toxic substances

Economic Information Area

•   Real Gross Domestic Product (GDP)
•   Real GDP per employed person
•   Labor force participation
•   Unemployment
•   Expenditures on Research and Development as a share of GDP
•   Real disposable income per capita
•   Median household net worth
•   Composition of wage rates (good jobs/bad jobs)
•   Poverty
•   Home ownership

Education Information Area

• Percentage of the population aged 25 and over that has completed
  postsecondary education
• National Assessment of Educational Progress (NAEP) or other measure
  of literacy equivalent to high school graduation
• Percentage of the population aged 15 to 29 that is neither enrolled in nor
  has completed high school
• Enrollment in science and engineering (National Science Foundation)
• Mathematics test scores (NAEP)
• Percentage of population with computer literacy and computer access
• Safe schools
• Gap in attainment by race and ethnic origin and other relevant factors
  (e.g., disability)
• Adult education participation/access

Governance Information Area

• Proportion of high elected offices (Congress, mayors, governors, etc.)
  held by women, minorities, etc.
• Proportion of high-appointed offices held by women, minorities, etc.
• Information about the “legal enforcement of constitutional guarantees
  of civil liberties”



Page 47                                       GAO-03-672SP Key National Indicators Forum
Appendix II
Illustrative Indicators by Information Area
for USA Series 0.5




• Civil rights: Enforcement data? Prevalence of complaints?
• Successful management of the voting franchise—for example,
  proportion of ballots that are disqualified
• Some measure of tax expenditures that reflects how effective the
  government is in taking care of the citizenry
• Some measure of how well government agencies are providing fair
  access to public services and utilities
• Some measure of how the law treats/does not treat Americans equally
• Some measure of the existence of an effective safety net
• Proportion of residents who believe that the nation is “on the right
  track”

Health Information Area

•   Overweight and obesity
•   Life expectancy—at birth, at different policy-relevant ages
•   Health/active life expectancy
•   Infant/child/youth mortality (i.e., successful survival to adulthood)
•   Disability limitations—as represented by inability to perform normal
    activities of daily living
•   Physical activity
•   Tobacco use
•   Substance abuse
•   Immunization
•   A measure of access to health care—availability, affordability, etc., for
    example, personal expenditures for health care as a percentage of per
    capita income

Social Support Information Area

•   Elderly living alone and in poverty
•   Proportion of elderly for whom Social Security is more than a “floor”
•   Older Americans who are involuntarily unemployed
•   Housing costs as a percentage of income for older Americans
•   Percentage of older Americans unable to perform certain physical
    functions
•   Proportion of children receiving child care, by source
•   Proportion of children whose diet is “poor”
•   Proportion of youth ages 16 to 19 neither enrolled in school or working
•   Adolescent birth rate
•   Family reading to young children




Page 48                                       GAO-03-672SP Key National Indicators Forum
Appendix III

Selected Bibliography on Indicator Systems                                                                 Appendx
                                                                                                                 iI




Comprehensive Indicator   Abbott, Robert M., Scott D. Johnson, and Tracy Dieckhoner. Embedding
Systems                   Sustainability in the Business of City Government: An Opportunity for
                          Seattle (Vancouver, Canada: Abbott Strategies, ND).
                          http://www.abbottstrategies.com/Papers/pdf/embedsustain.pdf.
                          (downloaded May 2003).

                              Embedding Sustainability in the Business of City Government
                              discusses ways to create a framework for integrating issues of
                              sustainability into decision making in Seattle city government.

                          Australian Bureau of Statistics. Measuring Australia’s Progress
                          2002:Indicators of Australia’s Progress (Canberra, Australia: 2003).
                          http://www.abs.gov.au. (downloaded May 2003).

                              Measuring Australia’s Progress uses a discussion of human capital,
                              social capital, natural capital, and financial capital indicators to asses
                              the extent to which Australia has progressed.

                          Bok, Derek. The State of the Nation. Cambridge, Mass.: Harvard University
                          Press, 1996.

                              The State of the Nation examines the areas of economic prosperity,
                              quality of life, equality of opportunity, personal security, and societal
                              values, and compares the progress made in these areas with progress
                              made in other countries.

                          The Boston Foundation. The Wisdom of Our Choices: Boston’s Indicators
                          of Progress, Change and Sustainability 2000. (Boston, Mass.: 2000).
                          http://www.tbf.org/boston/boston-L1.asp. (downloaded May 2003).

                              The Wisdom of Our Choices provides indicators of civic involvement,
                              the economy, education, public health, and other measures of well-
                              being.

                          Chang, Ping. State of the Region 2002: Measuring Progress in the 21st
                          Century (Los Angeles, Calif.: Southern California Association of
                          Governments, 2002). http://www.scag.ca.gov/publications/index.htm
                          (downloaded May 2003).

                              State of the Region 2002 assesses southern California’s performance
                              with respect to three overall goals: raise the standard of living, enhance
                              the quality of life, and foster equal access to resources.



                          Page 49                                 GAO-03-672SP Key National Indicators Forum
Appendix III
Selected Bibliography on Indicator Systems




Chicago Metropolis 2020. 2002 Metropolis Index (Chicago, Ill.: 2002).
http://www.chicagometropolis2020.org/10_5.htm. (downloaded May 2003).

    The 2002 Metropolis Index is intended to give residents of the region
    benchmarks to assess how the region is doing, and to help them
    consider what must be done to sustain the region’s status as a globally
    competitive region.

Committee on Geography, Committee on Identifying Data Needs for Place-
Based Decision Making. Community and Quality of Life: Data Needs for
Informed Decision Making (Washington, D.C.: National Academy Press,
2002). http://www.nap.edu/catalog/10262.html. (downloaded May 2003).

    Community and Quality of Life examines the concept of livable
    communities, the selection of livability indicators, data needs, and
    measurement and analysis issues related to the indicators.

The Conference Board of Canada. Performance and Potential 2001-2002.
Ottawa, Canada: 2002.

    Performance and Potential 2001-2002 updates the top 40 performance
    indicators and demonstrates that overall performance remains average
    among the six countries used to benchmark Canada’s performance.

Global Reporting Initiative. 2002 Sustainability Reporting Guidelines
(Boston, Mass.: 2002). http://www.globalreporting.org/guidelines/2002.asp.
(downloaded May 2003).

    The 2002 Sustainability Reporting Guidelines organizes
    “sustainability reporting” in terms of economic, environmental, and
    social performance (also know as the triple bottom line).

Maine Economic Growth Council. Measures of Growth 2002 (Augusta,
Maine: 2002). http://mdf.org/megc/growth02/. (downloaded May 2003).

    Measures of Growth 2002 provides the results of 60 indicators in the
    areas of the economy, community, and the environment.




Page 50                                      GAO-03-672SP Key National Indicators Forum
Appendix III
Selected Bibliography on Indicator Systems




Miringoff, Marc and Marque-Luisa Miringoff. The Social Health of the
Nation: How America Is Really Doing. New York, N.Y.: Oxford University
Press, 1999.

    The Social Health of the Nation presents a variety of indicators of
    social well-being over several decades.

Minnesota Planning. Minnesota Milestones 2002 (St. Paul, Minn.: 2002).
http://www.mnplan.state.mn.us/mm/index.html. (downloaded May 2003).

    Minnesota Milestones 2002 reports on 70 progress indicators to
    determine whether the state is achieving 19 publicly determined goals
    in the areas of people, community and democracy, economy, and
    environment.

The National Audit Office, United Kingdom, Good Practice in Performance
Reporting in Executive Agencies and Non-Departmental Public Bodies
(London, England: Stationery office, 2000).
http://www.nao.gov.uk/publications/nao_reports/9900272.pdf (downloaded
May 2003).

    Good Practice in Performance Reporting in Executive Agencies and
    Non-Departmental Public Bodies discusses good practices in
    government performance reporting to ensure transparent, accountable,
    and efficient government services.

New York City Department of City Planning. 2000/2001 Report on Social
Indicators. (New York, N.Y.: 2001).
http://www.nyc.gov/html/dcp/html/pub/socind00.html (downloaded May
2003).

    2000/2001 Report on Social Indicators is a compendium of data on the
    economic, social, physical, and environmental health of the city. The
    data are compiled from city, state, and federal sources and summarized
    on either a calendar or fiscal year basis.




Page 51                                      GAO-03-672SP Key National Indicators Forum
Appendix III
Selected Bibliography on Indicator Systems




The Oregon Progress Board. Achieving the Oregon Shines Vision: The
2001 Benchmark Performance Report (Salem, Oreg.: March 2001).
http://www.econ.state.or.us/opb/2001report/reporthome.htm. (downloaded
May 2003).

    Achieving the Oregon Shines Vision is a report on the comprehensive
    effort to describe progress Oregonians have made in achieving their
    year 2000 targets for 90 benchmarks.

President of the Treasury Board. Canada’s Performance 2002 (Ottawa,
Canada: 2002). http://www.tbs-sct.gc.ca/report/govrev/02/cp-rc_e.asp.
(downloaded May 2003).

    Canada’s Performance 2002 reports on the quality of life of Canadians
    in such areas as economic opportunity, health, the environment, and
    the strength and safety of communities.

Steering Committee Review of Commonwealth/State Services, Australia.
Report on Government Services 2001 (Melbourne, Australia: 2001).
http://www.pc.gov.au/gsp/2001/index.html. (downloaded May 2003).

    Report on Government Services 2001 details the performance of
    government service provision in Australia in education, health, justice,
    emergency management, community services, and housing.

United Nations General Assembly. Implementation of the United Nations
Millennium Declaration: Follow up to the Outcome of the Millennium
Summit A/57/270. New York, N.Y.: 2002.

    Implementation of the United Nations Millennium Declaration:
    Follow up to the Outcome of the Millennium Summit details the
    progress that the United Nations has made on its millennium
    development goals, which are (1) halve extreme poverty and hunger,
    (2) achieve universal primary education, (3) empower women and
    promote equality between women and men, (4) reduce under five
    mortality by two-thirds, (5) reduce maternal mortality by three-
    quarters, (6) reverse the spread of disease especially AIDS/HIV and
    malaria, (7) ensure environmental sustainability, and (8) create a global
    partnership for development with targets for aid, trade, and debt relief.




Page 52                                      GAO-03-672SP Key National Indicators Forum
                                Appendix III
                                Selected Bibliography on Indicator Systems




                                United Nations Population Fund. State of the World Population
                                2002:People, Poverty and Possibilities (New York, N.Y.: 2002).
                                http://www.unfpa.org/swp/2002/english/ch1/index.htm. (downloaded May
                                2003).

                                    State of the World Population 2002 provides a variety of demographic
                                    and economic data about people in various regions of the world as well
                                    as some data on individual nations.

                                University at Buffalo Institute for Local Governance and Regional Growth,
                                State of the Region Progress Report 2000 (Buffalo, N.Y.: 2000).
                                http://regional-institute.buffalo.edu/sotr/repo/repo00/default.html.
                                (downloaded May 2003).

                                    State of the Region Progress Report 2000 offers a first update of the
                                    1999 baseline report with two components--one focused on the data-
                                    driven performance measures, the other a second look at the
                                    opportunities and challenges that will shape Buffalo-Niagara's progress
                                    into the new century.

Specialized Indicator Systems   Annie E. Casey Foundation. Kids Count Data Book 2002 (Baltimore, Md:
                                2002). http://www.aecf.org/kidscount/kc2002/. (downloaded May 2002).

                                    Kids Count provides information about the physical health, mental
                                    health, economic well-being, and educational achievements of children
                                    in the United States. Data are available nationwide and for each state.

                                Chrvala, Carole A. and Roger J. Bulger, Eds. Leading Health Indicators for
                                Healthy People 2010: Final Report. Committee on Leading Health
                                Indicators for Healthy People 2010, Division of Health Promotion and
                                Disease Prevention, Institute of Medicine. Washington, DC: National
                                Academy Press, 1999.

                                    Leading Indicators for Healthy People 2010 describes the efforts of
                                    the Committee on Leading Health Indicators to develop leading health
                                    indicator sets that could focus on health and social issues and evoke a
                                    response and action from the general public and the traditional
                                    audiences for the Healthy People report series.




                                Page 53                                      GAO-03-672SP Key National Indicators Forum
Appendix III
Selected Bibliography on Indicator Systems




Committee to Evaluate Indicators for Monitoring Aquatic and Terrestrial
Environments, Board on Environmental Studies and Toxicology, Water
Science and Technology Board, Commission on Geosciences,
Environment, and Resources, National Research Council. Ecological
Indicators for the Nation. Washington, D.C.: National Academy Press,
2000.

    Ecological Indicators for the Nation suggests criteria for selecting
    useful ecological indicators, provides methods for integrating complex
    ecological information in indicators that are useful, proposes indicators
    that would meet these criteria, examines the state of data that would be
    used to calculate these indicators, and offers guidance on
    communicating and storing ecological indicators.

Committee on Measuring and Improving Infrastructure Performance,
Board on Infrastructure and the Constructed Environment, Commission on
Engineering and Technical Systems, National Research Council.
Measuring and Improving Infrastructure Performance (Washington, DC:
National Academy Press, 1995). http://www.nap.edu/catalog/4929.html.
(downloaded May 2003).

    Measuring and Improving Infrastructure Performance addresses the
    issue of measuring the efficiency with which our infrastructure allows
    people and goods to move, provides adequate safe drinking water,
    provides energy, removes waste, and so on, which is crucial to being
    able to manage the assets that our infrastructure represents.

Cooper, Ronald S. and Stephen A. Merrill, Eds. Industrial Research and
Innovation Indicators: Report of a Workshop. Board on Science,
Technology, and Economic Policy, National Research Council (Washington,
D.C.: National Academy Press, 1997).
http://www.nap.edu/catalog/5976.html. (downloaded May 2003).

    Industrial Research and Innovation Indicators is the report of a
    workshop held to discuss methods of improving the measurement, data
    collection and analysis of indicators on industrial research and
    innovation, as well as examining ways in which this information could
    be integrated into measures of firm and national performance.




Page 54                                      GAO-03-672SP Key National Indicators Forum
Appendix III
Selected Bibliography on Indicator Systems




Council of Economic Advisors, Executive Office of the President. The
Economic Report of the President (Washington, D.C.: Government Printing
Office, 2003).
http://w3.access.gpo.gov/usbudget/fy2004/pdf/2003_erp.pdf. (downloaded
May 2003).

    The Economic Report of the President is a discussion of selected
    economic issues and tables of economic data prepared by the Council
    of Economic Advisors.

The Conference Board. Business Cycle Indicators (New York, N.Y.: 2002).
http://www.tcb-indicators.org/Us/LatestReleases/index.cfm. (downloaded
May 2003).

    Business Cycle Indicators provides monthly economic indicators for
    the United States, such as the leading economic indicators, the
    coincident indicators, and the lagging indicators.

Federal Interagency Forum on Aging Related Statistics. Older Americans
2000: Key Indicators of Well Being (Washington, D.C.: 2000).
http://www.agingstats.gov/chartbook2000/default.htm. (downloaded May
2003).

    Older Americans: 2000 contains statistics regarding the population,
    economics, health status, health risks and behaviors, and health care of
    older United States Citizens.

Federal Interagency Forum on Child and Family Statistics. America’s
Children: Key National Indicators of Well-Being 2002 (Washington, D.C.:
Government Printing Office, 2002). http://childstats.gov. (downloaded May
2003).

    American’s Children provides 24 key indicators on the well-being of
    children in the areas of economic security, health, behavior and social
    environment, and education.




Page 55                                      GAO-03-672SP Key National Indicators Forum
Appendix III
Selected Bibliography on Indicator Systems




United States Department of Health and Human Services. Healthy People
2010: Understanding and Improving Health. (Washington, D.C.: 2000).
http://www.healthypeople.gov/. (downloaded May 2002).

    Healthy People 2010 provides a comprehensive set of disease
    prevention and health promotion objectives for the United States to
    achieve by 2010.

The H. John Heinz III Center for Science, Economics, and the Environment.
The State of the Nation’s Ecosystems: Measuring the Lands, Waters, and
Living Resources of the United States (Cambridge, U.K.: Cambridge
University Press, 2002). http://www.heinzctr.org/ecosystems/index.htm.
(downloaded May 2003).

    The State of the Nation’s Ecosystem is a blueprint for periodic
    reporting on the condition and use of ecosystems in the United States.

Mislevy, Robert J. and Kaeli T. Knowles, Eds. Performance Assessments for
Adult Education: Exploring the Measurement Issues Report of a
Workshop. Committee for the Workshop on Alternatives for Assessing
Adult Education and Literacy Programs, Board on Testing and Assessment,
Center for Education, Division of Behavioral and Social Sciences and
Education, National Research Council. (Washington, DC: National
Academy Press, 2002). http://www.nap.edu/catalog/10366.html.
(downloaded May 2003).

    Performance Assessments for Adult Education examines a variety of
    ways of measuring learning gains in adult basic education classes in
    light of the requirement that states evaluate adult students' progress as
    mandated by the Workforce Investment Act.

Nordhaus, William D. and Edward C. Kokkelenberg, Eds. Nature's
Numbers: Expanding the National Economic Accounts to Include the
Environment. Panel on Integrated Environmental and Economic
Accounting, Committee on National Statistics, Commission and Behavioral
and Social Sciences and Education, National Research Council
(Washington, DC: National Academy Press, 1999).
http://www.nap.edu/catalog/6374.html. (downloaded May 2003).

    Nature's Numbers examines the issues surrounding the question of
    broadening the U.S. National Income and Product Accounts to include




Page 56                                      GAO-03-672SP Key National Indicators Forum
Appendix III
Selected Bibliography on Indicator Systems




    activities that relate to natural resources and the environment to
    provide a more comprehensive picture of the economy.

Norwood, Janet and Jamie Casey, Eds. Key Transportation Indicators:
Summary of a Workshop. Committee on National Statistics, Division of
Behavioral and Social Sciences and Education, National Research Council.
Washington, DC: National Academy Press, 2002.

    Key Transportation Indicators discusses efforts to review current
    transportation indicators and issues associated with their uses as well
    considering what kinds of additional indicators are need.

Pellegrino, James W., Lee R. Jones, and Karen J. Mitchell, Eds. Grading the
Nation's Report Card: Evaluating NAEP and Transforming the
Assessment of Educational Progress. Committee on the Evaluation of
National and State Assessment, Commission on Behavioral and Social
Sciences and Education, National Research Council. (Washington, D.C.:
National Academy Press, 1999). http://www.nap.edu/catalog/6296.html.
(downloaded May 2003).

    Grading the Nation’s Report Card describes the National Assessment
    of Educational Progress' national assessment, the state assessment
    program, the student performance standards, and the extent to which
    the results are reasonable, valid, and informative to the public.

Schultze, Charles L. and Christopher Mackie, Eds. At What Price?
Conceptualizing and Measuring Cost-of Living and Price Indexes. Panel
on Conceptual, Measurement, and Other Statistical Issues in Developing
Cost-of-Living Indexes, Committee on National Statistics, Division of
Behavioral and Social Sciences and Education, National Research Council
(Washington, D.C.: National Academy Press, 2002).
http://www.nap.edu/catalog/10131.html. (downloaded May 2003).

    At What Price? discusses the conceptual, measurement, statistical, and
    data issues in the development of cost-of-living indexes and assesses
    the appropriate use of such indexes as for indexing federal programs
    and other purposes.

Starke, Linda, ed. State of the World 2002: Special World Summit Edition
(W.W. Norton and Co.: 2002). http://www.worldwatch.org/pubs/sow/2002/.
(downloaded May 2003).




Page 57                                      GAO-03-672SP Key National Indicators Forum
             Appendix III
             Selected Bibliography on Indicator Systems




                 State of the World 2002 provides information on a variety of issues in
                 sustainable development, such as climate change, farming, and toxic
                 chemicals.

             UNICEF. The State of the World’s Children 2003 (Oxford, N.Y.: Oxford
             University Press, 2002). http://www.unicef.org/pubsgen/sowc03/index.html.
             (downloaded May 2003).

                 The State of the World’s Children 2003 contains a comprehensive set of
                 economic and social indicators on the well-being of children
                 worldwide.

             The World Health Organization. World Health Report 2002 (Geneva: 2002).
             http://www.who.int/whr/. (downloaded May 2003).

                 World Health Report 2002 measures the amount of disease, disability,
                 and health that can be attributed to certain risks and calculates how
                 much of the burden is preventable.

Background   Berry, David, Patrice Flynn, and Theodore Heintz. “Sustainability and
             Quality of Life Indicators: Toward the Integration of Economic, Social and
             Environmental Measures,” Indicators: The Journal of Social Health vol. 1,
             no. 4 (Fall 2002).

                 “Sustainability and Quality of Life Indicators” provides discussion of
                 approaches to integrate social, economic, and environmental indicators
                 and expand the scope of our national data system.

             Caplow, Theodore, Louis Hicks, and Ben J. Wattenberg. The First
             Measured Century. Washington, D.C.: AEI Press, 2001.

                 The First Measured Century describes how using statistics to measure
                 social conditions gained importance throughout the United States from
                 1900 through 2000.

             Gross, Betram M. Social Intelligence for America's Future: Explorations
             in Societal Problems. Boston, Mass: Allyn and Bacon, Inc., 1969.

                 Social Intelligence for America’s Future is part of a “trial run” social
                 report ranging from learning and health to crime and the arts. It
                 discusses information methodology and the use of data to guide public
                 policy.



             Page 58                                      GAO-03-672SP Key National Indicators Forum
Appendix III
Selected Bibliography on Indicator Systems




Michalos, Alex C. Observations on Key National Performance Indicators.
Paper presented at the Forum on Key National Indicators, Washington,
D.C., February 2003.

    Observations on Key National Performance Indicators discusses
    several integrated performance systems on the national, international,
    and state levels.

Miringoff, Marc, Marque-Luisa Miringoff, and Sandra Opdycke. Social
Indicators: What We Need To Make Them Count. Paper presented at the
Forum on Key National Indicators, Washington, D.C., February 2003..

    Social Indicators addresses the need for social indicators to enhance
    consideration of policy issues.

Riche, Martha Farnsworth. The United States of America Developing Key
National Indicators. Paper presented at the Forum on Key National
Indicators, Washington, D.C., February 2003..

    The United States of America Developing Key National Indicators
    offers a framework to assess indicators and provides a preliminary
    draft of what an indicator set might look like for the United States.

Slater, Courtenay M. and Martin H. David, Eds. Measuring the Government
Sector of the U.S. Economic Accounts. Committee on National Statistics,
Commission on Behavioral and Social Science and Education, National
Research Council (Washington, DC: National Academy Press, 1998).
http://www.nap.edu/catalog/6208.html. (downloaded May 2003)

    Measuring the Government Sector of the U.S. Economic Accounts
    summarizes the discussion on and makes recommendations regarding
    the way the government sector is presented in U.S. economics accounts
    and how it could be brought into line with the International System of
    Accounts, which would allow for better cross-national comparisons.

U.S. Department of Health, Education, and Welfare, Toward a Social
Report. Washington, D.C.: U.S. Government Printing Office, 1969.

    Toward a Social Report discusses how social reporting can improve
    the nation’s ability to chart its social progress and to promote more
    informed policy decisions.




Page 59                                      GAO-03-672SP Key National Indicators Forum
Appendix IV

Selected Web Sites on Indicator Systems                                                        Appendx
                                                                                                     iIV




Multinational   European System of Social Indicators
                http://www.social-science-
                gesis.de/en/social_monitoring/social_indicators/EU_Reporting/eusi.htm

                Global Reporting Initiative - Sustainability Reporting Guidelines
                http://www.globalreporting.org/guidelines/2002.asp

                Measurement and Indicators for Sustainable Development - IISDnet
                http://www.iisd.org/measure/default.asp

                Organisation for Economic Co-operation and Development Worldwide
                Statistical Sources
                http://cs3-hq.oecd.org/scripts/stats/source/index.htm

                The International Society for Quality-of-Life Studies
                http://www.cob.vt.edu/market/isqols/

                United Nations High Commissioner for Refugees - The State of The World’s
                Refugees http://www.unhcr.ch/pubs/sowr2000/sowr2000toc.htm

                United Nations Human Development Report 2002
                http://www.undp.org/hdr2002/

                United Nations Millennium Development Goals
                http://www.un.org/millenniumgoals/index.html

                United Nations Population Fund - State of World Population 2002
                http://www.unfpa.org/swp/swpmain.htm

                United Nations Statistics Division - Social Indicators
                http://unstats.un.org/unsd/demographic/social/default.htm

                United Nations Statistics Division
                http://unstats.un.org/unsd/

                World Bank’s Millenium Development Goals
                http://www.developmentgoals.org/

                Worldwatch Institute State of the World 2002
                http://www.worldwatch.org/pubs/sow/2002/




                Page 60                               GAO-03-672SP Key National Indicators Forum
           Appendix IV
           Selected Web Sites on Indicator Systems




National   Australian Bureau of Statistics - Measuring Australia’s Progress
           http://www.abs.gov.au/

           Canadian Council on Social Indicators
           http://www.ccsd.ca/soc_ind.html

           Conference Board of Canada
           http://www.conferenceboard.ca/

           FedStats Home Page
           http://www.fedstats.gov/

           Heinz Center - The State of the Nation’s Ecosystems
           http://www.heinzctr.org/ecosystems/index.htm

           Interagency Working Group on Sustainable Development Indicators
           http://www.sdi.gov/

           Redefining Progress
           http://www.redefiningprogress.org/

           Treasury Board of Canada - Societal Indicators
           http://www.tbs-sct.gc.ca/rma/eppi-ibdrp/SI/si_e.htm

           STAT-USA - State of the Nation
           http://www.stat-usa.gov/econtest.nsf

           United Kingdom Government Sustainable Development Indicators
           http://www.sustainable-development.gov.uk/indicators/

           United Kingdom National Audit Office
           http://www.nao.gov.uk/

           University of Toronto Performance Indicators for Governance
           http://www.utoronto.ca/provost/perf98

           University of Washington Human Services Policy Center
           http://www.hspc.org/

           White House - Latest Federal Government Statistics
           http://www.whitehouse.gov/news/fsbr.html




           Page 61                                   GAO-03-672SP Key National Indicators Forum
                         Appendix IV
                         Selected Web Sites on Indicator Systems




Regional or Multistate   List of Performance Indicators for the Buffalo-Niagara Region
                         http://www.regional-institute.buffalo.edu/sotr/repo/indi.html

                         Northeast Midwest Institute Home Page
                         http://www.nemw.org/

                         Northwest Area Foundation Indicator Web site
                         http://www.indicators.nwaf.org/

                         Regional Research Institute, West Virginia University
                         http://www.rri.wvu.edu/

                         Worcester Regional Research Bureau
                         http://www.wmrb.org/CCPM

State                    Living with the Future in Mind - New Jersey’s 1999 Sustainable State Report
                         http://www.njfuture.org/HTMLSrc/SSR/index.html

                         Maine Marks for Children, Families and Communities
                         http://www.mainemarks.org/

                         Minnesota Planning Home Page
                         http://www.mnplan.state.mn.us/

                         Oregon Shines - Oregon Progress Board
                         http://www.econ.state.or.us/opb/index.htm

                         Public Policy Institute of California
                         http://www.ppic.org/

                         Vermont Agency of Human Services
                         http://www.ahs.state.vt.us/

Local                    Burlington Legacy Indicators Project
                         http://maps.vcgi.org/burlingtonlegacy/

                         Center for Schools and Communities - Lemoyne, Pennsylvania
                         http://www.center-school.org/

                         Chicago Metropolis 2020
                         http://www.chicagometropolis2020.org/report.htm




                         Page 62                                   GAO-03-672SP Key National Indicators Forum
                      Appendix IV
                      Selected Web Sites on Indicator Systems




                      City and Borough of Sitka, Alaska
                      http://www.cityofsitka.com/

                      Healthy Anchorage Indicators
                      http://www.indicators.ak.org/

                      New York City Department of City Planning - Social Indicators
                      http://www.nyc.gov/html/dcp/html/pub/socind00.html

                      Portland Mulnomah Progress Board
                      http://www.p-m-benchmarks.org/tblcnts.html

                      San Diego’s Regional Planning Agency
                      http://www.sandag.cog.ca.us/

                      Southern California Association of Governments
                      http://www.scag.ca.gov/

                      Sustainable Community Roundtable, Olympia, Washington
                      http://www.olywa.net/roundtable/

                      Sustainable Seattle
                      http://www.sustainableseattle.org/

                      The Baltimore Neighborhood Indicators Alliance
                      http://www.bnia.org/about_bnia_main.html

                      The Boston Foundation
                      http://www.tbf.org/

                      The Planning Council, Norfolk, Virginia
                      http://www.theplanningcouncil.org/

Specialized Efforts   Children

                      America’s Children 2002 - Key National Indicators of Well-Being 2002
                      http://www.childstats.gov/americaschildren/

                      Child Trends DataBank
                      http://www.childtrendsdatabank.org/about.htm




                      Page 63                                   GAO-03-672SP Key National Indicators Forum
Appendix IV
Selected Web Sites on Indicator Systems




Children First for Oregon
http://www.cffo.org/

KIDS COUNT - Benchmarks of Child Well-Being
http://www.aecf.org/kidscount

United Nations - The State of the World’s Children 2000
http://www.unicef.org/sowc00/

Economy

Federal Reserve Bank of Philadelphia
http://www.phil.frb.org/

Index of the Massachusetts Innovation Economy
http://www.mtpc.org/2001index/about.htm

International Association for Research in Income and Wealth
http://www.iariw.org/

Maine Economic Growth Council
http://www.mdf.org/megc

Norwegian Ministry of Finance
http://www.odin.dep.no/fin/engelsk/

West Virginia Bureau of Employment Programs - Labor Market Information
http://www.state.wv.us/bep/lmi/

Education

California’s Public Schools Accountability Act
http://www.cde.ca.gov/psaa/

National Assessment of Educational Progress -- The Nation’s Report Card
http://nces.ed.gov/nationsreportcard/about/

White House Social Statistics Briefing Room - Education
http://www.whitehouse.gov/fsbr/education.html




Page 64                                   GAO-03-672SP Key National Indicators Forum
Appendix IV
Selected Web Sites on Indicator Systems




Elderly

Federal Interagency Forum on Aging-Related Statistics
http://www.agingstats.gov/

Administration on Aging - Performance Outcomes Measures Project
http://www.gpra.net/

Environment

Environmental Protection Agency Biological Indicators of Watershed
Health http://www.epa.gov/bioindicators

Food and Agriculture Organization of the United Nations - State of the
World’s Forests
http://www.fao.org/forestry/FO/SOFO/sofo-e.stm

Northwest Environment Watch - Home Page
http://www.northwestwatch.org/pubs_index.html

Washington Governor’s Salmon Recovery Office
http://www.governor.wa.gov/gsro/default.htm

World Association of Nuclear Operators Performance Indicators
http://www.wano.org.uk/

Health

Leading Health Indicators - Healthy People 2010
http://www.health.gov/healthypeople/LHI/lhiwhat.htm

Maryland’s Drug Early Warning System
http://www.dewsonline.org/

Pan American Health Organization
http://www.paho.org/

Partnerships for Networked Consumer Health Information Conferences
http://odphp.osophs.dhhs.gov/confrnce/




Page 65                                   GAO-03-672SP Key National Indicators Forum
                          Appendix IV
                          Selected Web Sites on Indicator Systems




                          The Quality Indicator Project - Association of Maryland Hospitals and
                          Health Systems
                          http://www.qiproject.org/

                          White House Social Statistics Briefing Room - Health
                          http://www.whitehouse.gov/fsbr/health.html

Background or Reference   Flynn Research - Measuring Contributions to Society
                          http://www.flynnresearch.com/products.htm

                          From Revolution to Reconstruction - Information on U.S. Presidents
                          http://odur.let.rug.nl/~usa/P/

                          NonProfit Pathfinder - Measuring the Impact of the Independent Sector
                          http://www.independentsector.org/pathfinder/impact/indepsec_res/biblio.h
                          tml

                          The Social Indicators Survey Center - Columbia University
                          http://www.siscenter.org/




(450175)                  Page 66                                   GAO-03-672SP Key National Indicators Forum
GAO’s Mission            The General Accounting Office, the audit, evaluation and investigative arm of
                         Congress, exists to support Congress in meeting its constitutional responsibilities
                         and to help improve the performance and accountability of the federal government
                         for the American people. GAO examines the use of public funds; evaluates federal
                         programs and policies; and provides analyses, recommendations, and other
                         assistance to help Congress make informed oversight, policy, and funding
                         decisions. GAO’s commitment to good government is reflected in its core values of
                         accountability, integrity, and reliability.


Obtaining Copies of      The fastest and easiest way to obtain copies of GAO documents at no cost is
                         through the Internet. GAO’s Web site (www.gao.gov) contains abstracts and full-
GAO Reports and          text files of current reports and testimony and an expanding archive of older
                         products. The Web site features a search engine to help you locate documents
Testimony                using key words and phrases. You can print these documents in their entirety,
                         including charts and other graphics.
                         Each day, GAO issues a list of newly released reports, testimony, and
                         correspondence. GAO posts this list, known as “Today’s Reports,” on its Web site
                         daily. The list contains links to the full-text document files. To have GAO e-mail this
                         list to you every afternoon, go to www.gao.gov and select “Subscribe to GAO
                         Mailing Lists” under “Order GAO Products” heading.


Order by Mail or Phone   The first copy of each printed report is free. Additional copies are $2 each. A check
                         or money order should be made out to the Superintendent of Documents. GAO
                         also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a single
                         address are discounted 25 percent. Orders should be sent to:
                         U.S. General Accounting Office
                         441 G Street NW, Room LM
                         Washington, D.C. 20548
                         To order by Phone:     Voice: (202) 512-6000
                                                TDD: (202) 512-2537
                                                Fax: (202) 512-6061


To Report Fraud,         Contact:
                         Web site: www.gao.gov/fraudnet/fraudnet.htm
Waste, and Abuse in      E-mail: fraudnet@gao.gov
Federal Programs         Automated answering system: (800) 424-5454 or (202) 512-7470



Public Affairs           Jeff Nelligan, Managing Director, NelliganJ@gao.gov (202) 512-4800
                         U.S. General Accounting Office, 441 G Street NW, Room 7149
                         Washington, D.C. 20548