oversight

2020 Census: Initial Research Milestones Generally Met but Plans Needed to Mitigate Highest Risks

Published by the Government Accountability Office on 2012-11-07.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                United States Government Accountability Office

GAO             Report to Congressional Requesters




November 2012
                2020 CENSUS

                Initial Research
                Milestones Generally
                Met but Plans Needed
                to Mitigate Highest
                Risks




GAO-13-53
                                              November 2012

                                              2020 CENSUS
                                              Initial Research Milestones Generally Met but Plans
                                              Needed to Mitigate Highest Risks
Highlights of GAO-13-53, a report to
congressional requesters




Why GAO Did This Study                        What GAO Found
GAO’s prior work has shown that it will       According to U.S. Census Bureau (Bureau) officials, to inform timely design
be important for the Bureau to re-            decisions, research on new methods to improve the cost effectiveness of the
examine the design of the census in           2020 Census must be accomplished early enough in the decade to confirm their
order to ensure a cost effective census       likely impact on both cost and quality. Three key efforts—(1) the use of the
in 2020. As requested, this report            Internet as a response option, (2) a potential move towards targeted address
evaluates the Bureau’s efforts to             canvassing, and (3) the possible use of administrative records to replace data
improve the cost-effectiveness of the         collected during census field operations—present the Bureau with potential
enumeration, paying particular                opportunities to reduce costs while maintaining quality. The Bureau’s 2020
attention to the following three key
                                              Research and Testing Program has 14 fiscal year 2012 projects focused on
efforts: (1) leveraging the Internet to
                                              informing design decisions related to the three key efforts. Bureau officials are
increase self-response; (2) improving
how the Bureau builds its address and
                                              also aware that the changes they are testing come with many risks, and for each
mapping databases, including a                project the Bureau has identified a number of risks and prioritized them from high
possible move to targeted address             to low. However, the Bureau has not developed mitigation or contingency plans
canvassing, and use of private-sector         for these project risks. For example, there are several risks, including tight time
geographic data; (3) and using                frames and accurate cost information, without mitigation and contingency plans.
administrative records to reduce              Additionally, GAO found that the Bureau had not developed cost estimates for
nonresponse follow-up costs. This             any of its 2020 research and testing projects as required in guidance provided to
report (1) identifies what opportunities      project teams. Unreliable cost estimates was one of the reasons the 2010
and risks, if any, the Bureau might           Census was placed on GAO’s high-risk list. Without timely cost estimates, it will
need to consider for these efforts going      be difficult for the Bureau to ensure that resources are adequate to support the
forward and (2) examines to what              research and testing program.
extent these three efforts are on track
with respect to scheduling, resources,        The Bureau met its internal deadline for submitting each of its 14 research
and other performance metrics. To             project’s plans and charters. However, not all the project plans were complete.
meet these objectives, GAO reviewed           For example, some project teams did not fully document the types of skills
Bureau documents and interviewed              needed or perform a skills gap assessment to determine the resources needed to
officials.                                    carry out their respective projects as required in the Bureau’s planning template.
                                              According to Bureau officials, they are working to document these skills sets, as
What GAO Recommends                           well as any gaps in skills. Completing this analysis is important to ensuring
GAO recommends that the Acting                sufficient resources are available for conducting the research and testing
Census Director take a number of              projects. Additionally, performance metric documentation for several projects was
actions to improve the Bureau’s               incomplete. According to Bureau guidance, project teams were to provide
research and testing for the 2020             performance metrics for measuring progress and for determining the project’s
Census, such as developing risk               final outcome. However, one team did not provide either of these required
mitigation plans, contingency plans           performance metrics, while six other teams did not include performance metrics
and cost estimates for each project,          that could be used to monitor research and testing progress. Absent these
and performance metrics and skill sets        metrics, the Bureau does not have the assurance that it will be able to avoid
for those projects that do not have           potential schedule slips or the certainty as to whether project outcomes will be
them. The Department of Commerce              adequate for making decisions.
concurred with GAO’s findings and
recommendations and provided one
clarification, which was included in the
final report. Also, the Bureau in its
comments noted that it has begun to
address GAO’s recommendations.
View GAO-13-53. For more information,
contact Robert Goldenkoff at (202) 512-2757
or goldenkoffr@gao.gov.

                                                                                      United States Government Accountability Office
Contents


Letter                                                                                         1
                       Background                                                              3
                       Key Efforts Present the Bureau with Both Opportunities and Risks        8
                       Research and Testing Project Plans Were Delivered on Schedule,
                         but Not All Project-Level Resources and Performance Metrics
                         Have Been Identified                                                20
                       Conclusion                                                            21
                       Recommendations for Executive Action                                  22
                       Agency Comments and Our Evaluation                                    23

Appendix I             Objectives, Scope, and Methodology                                     26



Appendix II            Fourteen Research Projects Will Inform Design Decisions on Key
                       Efforts (Text for Interactive Figure 3)                                28



Appendix III           Census Mapping and Address Contracts                                   30



Appendix IV            Comments from the Department of Commerce                               31



Appendix V             GAO Contact and Staff Acknowledgments                                  33



Related GAO Products                                                                          34



Tables
                       Table 1: Project Teams Involved in Development of Internet
                                Response Option                                              28
                       Table 2: Project Teams Contributing to Decision to Use
                                Administrative Records                                       28
                       Table 3: Project Teams Contributing to Decision to Implement
                                Targeted Address Canvassing                                  29




                       Page i                                               GAO-13-53 2020 Census
Figures
          Figure 1: 2020 Census Design Alternatives Show Varying Degrees
                   of Change, Risk, and Potential Cost Savings                                      4
          Figure 2: The Life Cycle of the 2020 Census Has Five Phases                               5
          Figure 3: Research Projects Will Inform Design Decisions on Key
                   Efforts                                                                          7




          Abbreviations

          ACS               American Community Survey
          Bureau            U.S. Census Bureau
          GSS               Geographic Support Systems
          LUCA              Local Update of Census Addresses
          MAF               Master Address File
          NRFU              Nonresponse follow-up
          TIGER®            Topologically Integrated Geographic Encoding and
                            Referencing
          USPS              United State Postal Service




          This is a work of the U.S. government and is not subject to copyright protection in the
          United States. The published product may be reproduced and distributed in its entirety
          without further permission from GAO. However, because this work may contain
          copyrighted images or other material, permission from the copyright holder may be
          necessary if you wish to reproduce this material separately.




          Page ii                                                            GAO-13-53 2020 Census
United States Government Accountability Office
Washington, DC 20548




                                   November 7, 2012

                                   Congressional Requesters

                                   The 2010 Census, at around $13 billion, was the most expensive
                                   headcount in our nation’s history. Meanwhile, the cost of conducting the
                                   census has, on average, nearly doubled each decade since 1970 in
                                   constant 2010 dollars. If that growth rate continues, the 2020 Census
                                   could potentially cost around $25 billion. The U.S. Census Bureau
                                   (Bureau) has recognized the need for more cost-effective approaches to
                                   conducting the enumeration and is committed to identifying and
                                   implementing innovations and improvements as necessary to conduct the
                                   2020 Census at a lower cost per housing unit than the approximately
                                   $100 per housing unit cost of the 2010 Census (in constant 2010 dollars)
                                   while still maintaining high quality. Between fiscal years 2012 and 2014,
                                   the Bureau will conduct its research and testing phase for the 2020
                                   Census in order to develop an initial design by September 2014. The
                                   decisions made in the decade’s first years shape the design and set the
                                   framework for later decisions that will affect how the census will be
                                   ultimately conducted.

                                   As requested, this report evaluates the Bureau’s research efforts to
                                   improve the cost-effectiveness of the 2020 Census enumeration, paying
                                   particular attention to the following three key efforts that past Bureau
                                   research has shown could result in substantial cost savings:

                                   •   leveraging the Internet to increase self-response;

                                   •   improving how the Bureau builds its address and mapping databases,
                                       including a possible move to targeted address canvassing (as
                                       opposed to the full address canvassing used in earlier censuses,
                                       where field staff walked nearly every street in the country to verify the
                                       address list), and use of private-sector geographic data; and




                                   Page 1                                                   GAO-13-53 2020 Census
•   using administrative records, 1 such as tax data, to reduce
    nonresponse follow-up costs.

Our objectives were to: (1) identify what opportunities and risks, if any,
the Bureau might need to consider for these research efforts going
forward and (2) examine to what extent these three research efforts are
on track with respect to scheduling, resources, and other performance
metrics.

To meet these objectives, we reviewed documentary evidence, including
2020 Census research and testing plans, strategic framework documents,
and our prior reports on the subject; and conducted interviews with
cognizant Bureau officials, as well as knowledgeable stakeholders, such
as those at the National Academy of Sciences, to determine opportunities
and risks related to the Internet, administrative records and the address
and mapping databases. We also obtained and compared Bureau-
developed research planning requirements to research deliverables in
order to identify gaps in the Bureau’s research and testing efforts as it
relates to schedule, resources, and performance metrics. Additionally, we
interviewed agency officials to identify the Bureau’s 2020 research and
testing projects, timelines, and benchmarks. We obtained and reviewed
contract-related documents related to geographic data and services, as
well as interviewed Bureau officials to determine the Bureau’s plan for
using the private sector to assist with the Bureau’s address and mapping
needs. For more information on our scope and methodology see
appendix I.

We conducted this performance audit from October 2011 to September
2012 in accordance with generally accepted government auditing
standards. Those standards require that we plan and perform the audit to
obtain sufficient, appropriate evidence to provide a reasonable basis for
our findings and conclusions based on our audit objectives. We believe
that the evidence obtained provides a reasonable basis for our findings
and conclusions based on our audit objectives.




1
 Administrative records refer to data records contained in files collected and maintained
by administrative or program agencies and commercial entities. Government and
commercial entities maintain these files for the purpose of administering programs and
providing services.




Page 2                                                             GAO-13-53 2020 Census
             The Bureau’s 2020 Census early research agenda is focused on
Background   conducting the 2020 Census at a lower cost than the 2010 Census 2 while
             maintaining high quality results. In order to do this the Bureau has
             developed a range of design alternatives for the 2020 Census. The
             Bureau’s 2020 design alternatives have potential for containing costs but
             also have the potential to increase risks. The design alternatives focus on
             options to use the Internet and other social media to increase response
             rate, target address canvassing, and expand the use of administrative
             records. Figure 1 shows the current range of six 2020 Census design
             alternatives currently being considered and how the three key efforts fit
             into them. According to the Bureau, the final 2020 design is likely to
             incorporate both existing approaches as well as design alternatives that
             have never been used in the decennial census. Greater changes to the
             overall design will result in greater potential for cost savings. However,
             greater design changes also could result in greater risk, and testing will
             be needed to identify the risks, costs, and benefits of any new
             approaches. According to the Bureau, alternative one has the lowest risk
             because it most closely mirrors the 2010 Census design (mailout of
             census forms, and in-person follow-up of non respondents) and is not
             dependent on implementing innovations such as increased use of
             administrative records and targeted address canvassing.




             2
             Per housing unit, on an inflation-adjusted basis.




             Page 3                                                 GAO-13-53 2020 Census
Figure 1: 2020 Census Design Alternatives Show Varying Degrees of Change, Risk, and Potential Cost Savings




                                        The Bureau formally kicked off initial planning for the 2020 Census on
                                        October 3, 2008, with the 2020 Census Planning Summit. At that
                                        meeting, decennial-census managers and subject matter experts
                                        discussed challenges, strengths, and weaknesses that would need to be
                                        considered during the 2020 Census strategic decision-making process.
                                        From 2009 to 2011, during the options analysis phase of 2020 Census
                                        planning, Bureau officials went through a research topic winnowing
                                        process where they: identified cost and quality drivers, conducted
                                        brainstorming sessions, received expert input (e.g., National Academy of
                                        Sciences), cataloged over 600 ideas, clustered those ideas into over 100
                                        projects with research questions, and then prioritized them against the
                                        Bureau’s Guiding Principles 3 for the 2020 Census. The Bureau identified
                                        26 research projects to inform preliminary design decisions with the goal


                                        3
                                         The Bureau’s Guiding Principles for the 2020 Census are: Honor Privacy and
                                        Confidentiality; Be Objective; Commit to Quality; Contain Costs Wherever Possible;
                                        Leverage Assets, Methods, Knowledge, and Processes across the Bureau; and Be
                                        Responsive and Transparent to Partners and Stakeholders.




                                        Page 4                                                           GAO-13-53 2020 Census
of conducting the 2020 Census at a lower cost than the 2010 Census,
while maintaining high quality.

Using the 26 research projects, Bureau officials plan to validate the
feasibility of the various strategies for reducing costs between fiscal years
2012 and 2014. Bureau officials stated that they are planning to narrow
down the design alternatives by September 2014, and that research
would continue into the next phase. Bureau officials believe that
conducting research earlier in the decade will better position them to
make more effective changes to the 2020 Census design. Figure 2 shows
the five phases of the life cycle for the 2020 Census.

Figure 2: The Life Cycle of the 2020 Census Has Five Phases




The Bureau is now in the “Early Research and Testing” phase and is
currently refining and developing its 26 project plans. These plans
include, among other requirements, project goals, objectives, scope,
limitations, and skills required, and had a set of required iterative research
and testing deliverables due 30, 60, 90, and 120 days after project kickoff.
According to Bureau documentation, this process encouraged the
development and proper consideration of ideas. The Bureau selected 17
of the 26 projects to start in fiscal year 2012. The remaining projects are
scheduled to begin in fiscal years 2013 and 2014. Of the 17 research
projects the Bureau is conducting in 2012, 14 projects are to inform
design decisions related to the Internet, administrative records, and




Page 5                                                        GAO-13-53 2020 Census
targeted address canvassing. See figure 3 for a list of these 14 research
projects and their objectives. According to the Bureau’s timeline, project
teams are to complete their work on these 14 projects by September
2014. 4




4
 A Bureau official stated that they are assessing the impact of the fiscal year 2013
Continuing Resolution on their ability to complete all of their work by September 2014.




Page 6                                                             GAO-13-53 2020 Census
 Interactive graphic Figure 3: Fourteen Research Projects Will Inform Design Decisions on Key Efforts

Move mouse over each activity to see more information afterwards CLICK                 to close the open pop-up information display panel.




            Objectives in
            Objectives in Internet
                          internet                           Objectives in Administrative                        Objectives in Targeted Address
                development
                Development                                    Records Development                                 Canvassing Development



     Project teams involved in development of           Project teams contributing to decision to               Project teams contributing to decision to
     Internet response option                           use administrative records                              implement targeted address canvassing



                                                   Source: GAO analysis of U.S. Census Bureau Information   .




      Print instructions     To print text version of this graphic, go to appendix II.

                                                 Page 7                                                                               GAO-13-53 2020 Census
                            Research on new methods likely to result in a more cost-effective 2020
Key Efforts Present         Census must be accomplished early enough in the decade to confirm
the Bureau with Both        their likely impact on both cost and quality and to inform timely design
                            decisions. The use of the Internet as a response option, a potential move
Opportunities and           towards targeted address canvassing, and the possible use of
Risks                       administrative records to replace data collected during census field
                            operations present the Bureau with opportunities to reduce costs while
                            maintaining a high quality census. As part of the 2020 Census planning
                            process the Bureau has identified both program-level and project-level
                            risks associated with these options. According to the Bureau, program-
                            level risks span the entire 2020 research and testing phase and if not
                            managed properly could jeopardize the phase’s goals and objectives. In
                            contrast, project-level risks pertain to the successful completion of
                            specific projects. However, the Bureau has not developed mitigation and
                            contingency plans for the project-level risks. Also, the Bureau has not
                            developed cost estimates for any of its 2020 research and testing projects
                            as required in guidance provided to project teams. Finally, some of the
                            project plans we reviewed were incomplete. For example, not all project
                            plans identified the necessary staff resources, while other project plans
                            failed to provide performance metrics for measuring progress and for
                            determining the project’s final outcome.


All Three Key Efforts
Present the Bureau with
Cost Saving Opportunities
Internet                    During the 2000 Census, the Bureau piloted an Internet response option
                            that had a limited number of respondents. They considered building on
                            this 2000 experience for the 2010 Census, but the Bureau decided in July
                            2006 not to include the Internet option in the design for the 2010 Census
                            because test results indicated that the Internet option did not increase the
                            overall response rate, including response among hard-to-count population
                            groups, and they had underestimated the costs of the contract that
                            included developing the Internet option. 5




                            5
                             Census Bureau, Rationale for the Decision to Eliminate the Internet Option for the 2010
                            Decennial Census, 2010 Decennial Census Program Decision Memorandum Series No.
                            14 (Washington, D.C.: July 19, 2006).




                            Page 8                                                            GAO-13-53 2020 Census
However, more recent tests conducted in 2011 showed that adding an
Internet response option increased the overall response rate. The 2011
test results, coupled with the increased prevalence and accessibility of
the Internet, led Bureau officials to commit to providing an Internet
response option for the 2020 Census. With regard to the cost for the 2020
Census, if this option can help achieve an overall increase in the
response rate it can save money, since Bureau field staff would need to
visit fewer households during nonresponse follow-up (NRFU), which is
the largest and most costly census field operation. 6 Furthermore, testing
has shown that the cost of an Internet survey is low compared to a mail
survey, which incurs printing and postage costs. Moreover, web survey
responses are generally available more quickly and are of better quality
than responses from a mail survey because there is no lag time as the
responses are captured in real time and there are reminders to prompt
the respondent if a question is unanswered. Quicker and complete
responses can also help reduce the amount of time and money spent on
following up on late or incomplete census forms. 7

In April and November 2011, the Bureau tested an Internet response
option in its American Community Survey (ACS), 8 and beginning in
January 2013, the Bureau plans to offer it as a response option. Bureau
officials have stated that they intend to build on this existing information
technology infrastructure in order to reduce the cost of implementing an
Internet option for the 2020 Census.

Given the Bureau’s decision to move forward with an Internet design
option, six research and testing projects are focused on how best to
implement an Internet response option in concert with other self-response
options. The project team with primary responsibility for coordinating the
design of the Internet response option is tasked with answering the
following research question:




6
 During NRFU the Bureau sends enumerators to collect data from households that did not
mail back their census forms. NRFU procedures instruct enumerators to make up to six
attempts to contact a household. The 2010 Census NRFU operation cost $1.6 billion.
7
 We have a separate review underway examining the Bureau’s information technology
security issues.
8
 ACS is an ongoing survey that provides data every year. Information from the survey
generates data that help determine how federal and state funds are distributed each year.




Page 9                                                            GAO-13-53 2020 Census
                                •   How does the Bureau leverage technology and new response modes
                                    (including the Internet) to increase self-response, improve
                                    nonresponse follow-up data collection strategies, while maintaining
                                    overall quality?

                                One objective of this project is to determine what contact strategy should
                                be used to inform the public that census responses can be filed via the
                                Internet in order to increase self-response (i.e., Bureau use of email,
                                social networks or phone for initial contact or for reminders). Contact
                                strategy testing will examine the impact of various Bureau instructions to
                                respondents, as well as the timing of the delivery of the mailing pieces, on
                                such factors as response rates, return rates, percentage of Internet
                                returns, and speed of returns. This project will also determine what mode
                                (paper, Internet, or phone) is best for each demographic, geographic, and
                                language group, as well as determine the effect a lack of access to
                                technology has on each demographic group. Additionally this project will
                                refine and test security features. The other five projects will cover such
                                areas as types of languages that will be offered by response mode, public
                                perception and potential concerns about an Internet response, and
                                management of the workload for multiple modes of response.

Targeted Address Canvassing     In the 2010 and earlier censuses, the Bureau mounted a full address
and Use of the Private Sector   canvassing operation, where field staff verified every housing unit in the
for Geographic Data Services    nation to update the Master Address File (MAF) and the associated
                                mapping system called TIGER® (Topologically Integrated Geographic
                                Encoding and Referencing). This labor-intensive effort was one of the
                                more expensive components of the 2010 Census: the 2010 Address
                                Canvassing Operation required 140,000 temporary workers to verify 145
                                million addresses (by going door-to-door) at a cost of $444 million, or 3
                                percent of the $13 billion total cost of the 2010 Census. One of the
                                reasons the Bureau conducted a full address canvassing for the 2010
                                Census was because it wanted to capture hidden and hard-to-capture
                                addresses, such as housing units in converted garages and large storage
                                sheds. For the 2020 Census the Bureau would like to reduce the amount
                                of field work and cost of address and map updates by targeting the
                                address canvassing operation. According to Bureau planning documents,
                                through early research and testing the Bureau plans to attempt to
                                determine the quality and stability of its address file in terms of the
                                number of changes (adds, deletes, duplicates, and corrected addresses)
                                that took place to each census block in 2010. Depending on that
                                outcome, the Bureau would target for address canvassing only those
                                blocks where the quality of the addresses was determined to be
                                inadequate.


                                Page 10                                                 GAO-13-53 2020 Census
To determine whether to target address canvassing for the 2020 Census,
the Bureau has two ongoing efforts to improve the Bureau’s map and
address databases. First, the Bureau’s Geography Division is working
with the United States Postal Service (USPS) and other federal agencies
on a new program called the Geographic Support Systems (GSS)
Initiative, which allows government agencies at all levels to regularly
share and continuously update their address lists with the Bureau.
According to current plans, USPS will continue providing the Bureau with
address updates. Likewise, tribal, state, and local governments, which
maintain address lists for purposes such as emergency response and
property assessment, will also have the opportunity to share addresses
with the Bureau throughout the decade, rather than solely 2 years prior to
the decennial, as had been the case in prior decennial censuses.
Additionally, the GSS Initiative is working to develop, and in some cases
improve, methods for identifying and capturing hidden and hard-to-
capture addresses.

As part of this first effort, the Bureau is investigating the role and possible
contributions the private sector can make in improving its address and
mapping databases. For the 2010 Census the Bureau relied heavily on
the private sector to update addresses and maps. Specifically, in June
2002, the Bureau awarded an 8-year contract of about $200 million to
improve the accuracy of its mapping database and investigate options
and methods of updating its address database. The contract, among
other tasks, corrected in TIGER® the location of streets, boundaries, and
other map features so that coordinates would be aligned with their true
geographic locations. According to Bureau officials, continued reliance on
the private sector is necessary in order to maintain the major upgrades
that were made to its address and mapping databases last decennial.
The Geography Division has nine contracts in place, totaling about $90
million dollars, to help it manage its mapping and address database (see
app. III for a list of those contracts). Some of the work being done by the
private sector includes a service contract to provide system upgrades and
maintenance to the MAF/TIGER® database and a data entry services
contract to update the MAF/TIGER® database with address information
provided by state and local governments.

Moreover, to ensure the most current address and mapping data are
available for the 2020 Census, the Bureau, in April 2012, began
conducting market research with three private-sector firms to assess how
those firms might assist the Bureau with its address and mapping needs.
Possibilities to be explored include how to best maintain address and
map data, understand private-sector innovation and technology, and


Page 11                                                   GAO-13-53 2020 Census
options for address and map data-sharing methods. According to Bureau
officials, negotiations concerning the scope, tasks, and activities to be
completed with the first contractor have taken place. However,
negotiations with the remaining two firms have not begun. Use of the
private sector for mapping is an area we have previously reported needs
to be explored. Specifically, in December 2010, we listed a number of
reexamination areas which have particular implications for controlling
costs and improving accuracy and raised the question: To what extent
can private-sector and other sources of information such as maps,
address lists, and geographic databases be employed to help support the
census? 9

The second effort to improve the Bureau’s address and mapping
database is centered on the 2020 Research and Testing Program. There
are three research projects designed to inform the Bureau’s decision
about the extent to which it might be able to conduct a targeted address
canvassing operation. Two projects will use modeling to predict where
coverage errors (e.g., missed or duplicate addresses) occur in the MAF.
These models will be used to assess the quality of data sources, types of
errors, and what information, such as a missing map location for an
address, is not in the MAF. If the Bureau decides to conduct targeted
address canvassing, this information will be used to determine which
areas meet acceptable quality standards, as well as where targeted
address canvassing would be effective. According to the Bureau,
collectively the two projects are to answer the following question:

•    As the related GSS Initiative proceeds, how will the Bureau determine
     the required level of quality needed in the address frame to conduct
     an accurate census and then measure the quality of the continually
     updated MAF for that purpose?

The third research project in this effort, the Local Update of Census
Addresses (LUCA) Program Improvement project, 10 will examine how to
modify the Bureau’s existing partnership program in consultation with the


9
 GAO, 2010 Census: Data Collection Operations Were Generally Completed as Planned,
but Long-standing Challenges Suggest Need for Fundamental Reforms, GAO-11-193
(Washington, D.C.: Dec. 14, 2010).
10
  The LUCA Program was established by the Census Address List Improvement Act of
1994 to improve the accuracy of census’ address list by exchanging information with tribal,
state, and local governments.




Page 12                                                            GAO-13-53 2020 Census
                         GSS Initiative. The LUCA program is a partnership program that provides
                         an opportunity for local and tribal governments to review and update
                         individual address information or block-by-block address counts from the
                         MAF and associated geographic information in the TIGER® database just
                         prior to the decennial. The goal of LUCA is to improve the completeness
                         and accuracy of address information. This research project’s objective is
                         to determine the best way to modify and improve LUCA procedures to
                         ensure compatibility with continual updating of address information
                         throughout the decade. The project will also determine whether it is
                         necessary for local and tribal governments to actively participate in LUCA,
                         a once-a-decade operation, if they have been submitting addresses and
                         mapping data continuously throughout the decade as part of the GSS
                         Initiative.

Administrative Records   Administrative records are a growing source of information on individuals
                         and households. For purposes of the decennial census, the Bureau is
                         considering administrative records from government agencies, including
                         tax data and Medicare records, as well as commercial sources to identify
                         persons associated with a particular household address. 11 During the
                         2010 Census, the Bureau made limited use of administrative records. For
                         example, the Bureau used USPS files to update its address list, and
                         federal agency records (such as those from the Department of Defense)
                         were used to count military and federal civilian employees stationed
                         outside of the United States.

                         For 2020, in its first attempt to estimate potential costs of various design
                         alternatives for the 2020 Census, the Bureau estimated that by expanding
                         the use of administrative records it could save up to $2 billion by reducing
                         the workload for several operations, including NRFU, address building,
                         and quality assurance. 12 Depending on the results of the research and
                         testing projects, Bureau officials plan to build a composite of quality
                         administrative records from various sources (i.e., federal agencies, state
                         and local governments, and commercial sources) that it can use to
                         reduce or replace costly field work. Thus, using administrative records


                         11
                           The Bureau’s access to and use of administrative records is governed by agency-
                         specific statutes. For example, the Bureau has access to tax data under 26 U.S.C.
                         § 6103(j)(1) “for the purpose of, but only to the extent necessary in, the structuring of
                         censuses ... and conducting related statistical activities.”
                         12
                           The amount of and quality of administrative records the Bureau is able to collect will
                         impact the amount of cost savings the Bureau is able to realize.




                         Page 13                                                               GAO-13-53 2020 Census
could allow the Bureau to reduce the scale of its NRFU operation and
reduce the need for field office space and staff. The 2012 research and
testing projects are also focused on identifying the best available
administrative records to use for address frame building. Administrative
data from other sources would be combined with USPS data, thus
allowing the Bureau to continuously update its MAF throughout the
decade. Finally, administrative records could reduce the cost of quality
control, according to Bureau officials. In order to ensure the accuracy of
the field work conducted during census field operations, such as NRFU,
the Bureau sends quality control staff into the field to verify the work
completed by a random sample of enumerators. Bureau officials are
determining for 2020 whether administrative records can be used as a
quality control check, rather than sending staff into the field.

There are nine research and testing projects that will inform the Bureau’s
decision making on the extent to which the Bureau can expand the use of
administrative records in order to reduce costs. These research and
testing efforts are focused on determining whether the quality of
administrative records is sufficient to be used for this purpose. Initial
testing is to be completed and decisions on these matters are to be made
by September 2014. Of the nine, there are two primary administrative
records research projects. They are tasked with answering the following
research questions:

•   How can the Bureau best develop and maintain an independent
    collection of administrative records and assess the quality of those
    records (best sources and methods)?

•   How can the Bureau leverage administrative records (including
    commercial files) to significantly reduce decennial census cost while
    maintaining quality?

The objective of one of these projects is to acquire, process, and analyze
administrative records from federal, state, and commercial sources to
assess their utility for the 2020 Census. The other project will research
and test methods to enhance NRFU operations with administrative
records, such as replacing phone or in-person collected data with
administrative data.




Page 14                                                 GAO-13-53 2020 Census
The Bureau Has Identified   Bureau officials are aware that the changes they want to make to the
2020 Census Program-        decennial census come with many risks. The Bureau has identified and
Level and Project-Level     prioritized from high to low both program-level and project-level risks.
                            According to the Bureau, program-level risks span the entire 2020
Risks but Has Not           research and testing phase and if not managed properly could jeopardize
Developed Project-Level     the phase’s goals and objectives. In contrast, project-level risks pertain to
Mitigation and              the successful completion of projects. In accordance with best risk
Contingency Plans           management practices, 13 the Bureau has identified and drafted mitigation
                            plans for all of the 14 program-level risks it has identified. In May 2012 the
                            Bureau designated two of the 14 program-level risks as high risk:

                            •    Timely research and testing results: The 2020 research and testing
                                 strategy involves “many small field tests to research design
                                 alternatives with an accelerated, agile, and informed decision-making
                                 process for incorporating changes to the 2020 Census design.”
                                 According to the Bureau, if the research and testing results are late,
                                 then decisions will not be made on time and the program may not be
                                 ready to move out of the research and testing phase on schedule.
                                 According to the Bureau’s 2020 Census Risk Management Mitigation
                                 Plans, officials have begun implementing a mitigation strategy for this
                                 risk, including documenting clear roles and responsibilities and
                                 communicating them to staff; establishing a schedule that contains
                                 decision milestones and tracking interdependencies of research
                                 results and testing projects; and reviewing ongoing project status
                                 including updates on dependencies, risks, and metrics.

                            •    Significant budget cuts: The Bureau reported that if its funding for
                                 2020 Census planning is significantly reduced for fiscal year 2013,
                                 key projects will have to be delayed and as a result the Bureau could
                                 have major technical and operational difficulties making preliminary
                                 design decisions. According to Bureau documents, to mitigate this
                                 risk, Bureau officials have briefed staff from House of Representatives
                                 Subcommittee for Commerce, Justice, Science and Related
                                 Agencies, Committee on Appropriations, and the Office of
                                 Management and Budget on the implications of reduced funding and
                                 plan to brief oversight committee staff. In addition, Bureau officials




                            13
                              Broadly defined, risk management is a strategic process for helping policymakers make
                            decisions about assessing risk, allocating finite resources, and taking actions under
                            conditions of uncertainty.




                            Page 15                                                         GAO-13-53 2020 Census
     reported they have analyzed budgetary needs for fiscal year 2013 and
     prioritized projects to manage limited resources.

Related to these risks, a June 2012 congressional hearing held by the
Joint Economic Committee discussed the impact of eliminating or
reducing funding to the ACS. According to Bureau officials, such action
would have a significant impact on the 2020 Census research and testing
program because ACS is one of its testing platforms. This risk is not listed
as a program-level risk, or part of the current risk mitigation plans, but
Bureau managers told us they are strategizing on what they will do if ACS
is not funded and will update the risk mitigation plan accordingly.

The Bureau has identified lack of support from external stakeholders as a
medium program-level risk. Bureau officials are concerned that if external
stakeholders, such as Congress, the Office of Management and Budget,
and the National Academy of Sciences, do not support research on
alternative design options, then the research and testing agenda will have
to be redesigned. To manage this risk, Bureau officials have
communicated the alternative design options to a wide range of external
stakeholders, but are concerned that support may diminish as time
passes and stakeholders change. To mitigate this risk, the Bureau is
engaging external stakeholders in communications regarding 2020
research and testing efforts and seeking feedback on Bureau plans.
According to Bureau documents, the Bureau has met with external
stakeholders, including congressional staff on its House of
Representatives and Senate oversight committees, to discuss 2020
Census planning, and is working on a strategy for keeping stakeholders
informed as plans progress.

Unlike the program-level risks, risk mitigation and contingency plans have
not been drafted to address the project-specific risks. Specifically, each of
the Bureau’s research and testing projects has its own risk register 14 that
identifies and prioritizes the risks associated with that project, but no risk
mitigation or contingency plans have been developed for these project-
level risks. Several risks appear in multiple project team risk registers,
including tight time frames and accurate cost information. For example,
one Internet-related project listed that there is a risk that if tests are not


14
  A risk register contains the results of a projects risk analysis and includes risk
descriptions, probability of occurrence, cost impact, schedule impact, and identifies the
Bureau official with ownership of the risk.




Page 16                                                             GAO-13-53 2020 Census
timed properly, and if time frames are too tight then they may not be able
to apply what they have learned from one test to the next. Similarly, one
of the administrative records project teams was concerned that one year
to integrate data, build a model, and conduct adequate analysis of
administrative records was “not realistic.” In another example concerning
cost data, two teams stated that without accurate cost data at the project
level, research results will not be sufficient to inform the design of the
2020 Census.

Project teams also identified project-level risks directly related to the use
of administrative records in place of data collected by enumerators during
field visits. Among these risks is access to administrative records to build
a high quality compilation of administrative records for each household
across the country. To accomplish this task, the Bureau needs timely
access to data collected by the federal government and, as previously
mentioned, is currently compiling administrative records from various
federal agencies including the Departments of Housing and Urban
Development and Health and Human Services. However, not all agencies
are authorized to provide their data to the Bureau thus limiting, and in
some cases preventing, the Bureau’s use of that data. Bureau officials
provided us with two examples of agency administrative record sources
that they are not authorized to access: records maintained under the
Family Educational Rights and Privacy Act and the National Directory of
New Hires. 15 Bureau officials said they would like to have access to these
and other data sources for statistical purposes in order to further increase
the number and improve the quality of available administrative records.
According to Bureau officials, reviewing and determining whether they
have access to administrative records with every agency is a time
consuming process and the Bureau would like to have the ability to fully
access and use agency administrative data, especially since the
confidentiality of census data are protected by Title 13. 16 To move this
forward, according to a senior Bureau official, legislation might need to be
enacted that would compel federal agencies to provide the Bureau with
access to administrative data for the decennial census. However, the



15
  The Family Educational Rights and Privacy Act protects the privacy of student education
records including personally identifiable information (name, address, social security
number, etc.) from those records. The National Directory of New Hires contains wage and
employment information that is used by state agencies to enforce child support payments.
16
 13 U.S.C. § 9.




Page 17                                                          GAO-13-53 2020 Census
                          enactment of such legislation is likely to be a time consuming process
                          that raises serious policy concerns including whether the Bureau has
                          made a sufficiently strong case that it needs access to all federally
                          collected data and the resulting impact on personal privacy protections.

                          According to Bureau officials, the project teams did not submit risk
                          mitigation and contingency plans for the project-level risks as required at
                          the 90-day checkpoint because after reviewing all of the project-level risk
                          registers the project leaders first wanted to unduplicate similar risks
                          across projects in order to prevent redundancies. Bureau officials were
                          not able to provide a date for when risk mitigation and contingency plans
                          would be available. However, they stated they are committed to
                          managing risk in order to avoid the missteps encountered in 2010 when
                          the Bureau was on our high-risk list due to weaknesses in managing IT,
                          operational planning, and cost estimating. 17 In our prior work, we reported
                          that risk mitigation involves identifying, analyzing, prioritizing, and
                          documenting risks, and ideally more than one alternative should be
                          assessed. 18 The Department of Commerce’s Inspector General has
                          recommended that risk management activities begin from the outset of
                          the current decennial census life cycle, rather than just before field
                          operations. Without mitigation plans the 2020 Census planning team may
                          not be able to fully manage risks associated with these projects.


Project-Level Cost        The Bureau did not develop cost estimates for any of its 2020 research
Estimates Have Not Been   and testing projects by the 90-day checkpoint as required in guidance
Developed                 provided to project teams. The importance of reliable cost estimates is
                          underscored by the Bureau’s experience leading up to the 2010 Census,
                          where we found that the Bureau’s cost estimates lacked detailed
                          documentation on data sources and significant assumptions and was not
                          comprehensive because it did not include all costs. 19 Partly as a result,
                          some operations had substantial variances between their initial cost



                          17
                            GAO, Information Technology: Significant Problems of Critical Automation Program
                          Contribute to Risks Facing 2010 Census, GAO-08-550T (Washington, D.C.: Mar. 5, 2008).
                          18
                            GAO, 2010 Census: Cost and Design Issues Need to Be Addressed Soon, GAO-04-37
                          (Washington, D.C.: Jan. 15, 2004).
                          19
                            GAO, 2010 Census: Census Bureau Should Take Action to Improve the Credibility and
                          Accuracy of Its Cost Estimate for the Decennial Census, GAO-08-554 (Washington, D.C.:
                          Jun. 16, 2008).




                          Page 18                                                        GAO-13-53 2020 Census
estimates and their actual costs. For example, the 2010 address
canvassing operation cost $88 million more than the original estimate of
$356 million, an overrun of about 25 percent; and the Bureau’s 2010
NRFU operation cost $1.6 billion, about $660 million, or 29 percent, less
than the Bureau initially estimated.

According to 2020 Census planning documents, at the 90-day checkpoint
project teams were to: (1) prepare and submit cost estimate requests
needed for the project; (2) document estimates for completing the work
for the project; (3) identify and document funds needed for additional
resources and requirements, and (4) prepare and submit requests for
additional funds. According to a senior Bureau official, cost estimates
have not yet been completed because more work needs to be done to
determine project costs and they are in the process of formulating cost
estimates for each project.

According to our cost estimate guide, 20 a cost estimate provides an
assessment of the costs most likely to be incurred and should include the
requirements, resources, and tasks that must be accomplished. As the
Bureau moves forward with its 2020 research and testing program,
project cost estimates will allow the Bureau to plan for the resources
required to conduct necessary field testing, ensure there are sufficient
funds, and adjust testing accordingly. In addition, the Bureau could use
these cost estimates to help determine how budget cuts might hinder the
census research and testing program’s progress or effectiveness. Thus, if
the Bureau does not develop cost estimates in a timely manner it will be
difficult for the Bureau to ensure that resources are adequate to support
research and testing projects.




20
 GAO, GAO Cost Estimating and Assessment Guide: Best Practices for Developing and
Managing Capital Program Costs, GAO-09-3SP (Washington D.C.: Mar. 2, 2009).




Page 19                                                     GAO-13-53 2020 Census
                        The Bureau’s early research and testing is currently refining the project
Research and Testing    plans associated with the14 projects related to the Internet, administrative
Project Plans Were      records, and targeted address canvassing. Each of the research and
                        testing projects has a set of required deliverables due 30, 60, 90, and 120
Delivered on            days after project kickoff. We determined that each project team had
Schedule, but Not All   provided the 120-day required deliverables on schedule to Bureau
Project-Level           managers in June 2012. The research and testing documents required by
                        the 2020 Census planning team and delivered by the 14 project teams
Resources and           describe management and technical plans for each project, and include a
Performance Metrics     team charter, project plan, resources, preliminary schedule, and field
                        tests. These documents are important because they serve as the
Have Been Identified    baseline for each of the research and testing projects.

                        However, not all resource requirements were documented. For example,
                        some project teams did not fully document the types of skills needed, or
                        perform a skills gap assessment to determine the resources needed to
                        carry out their respective projects as required in the Bureau’s planning
                        template (the template lays out in detail what is to be included in the
                        deliverables). Bureau officials plan to use this skills information to allocate
                        staff to ensure teams have the skills necessary to complete their projects.
                        While 11 project teams provided all the required information, 2 project
                        teams conducting research on issues that will inform decisions on
                        targeted address canvassing and expanded use of administrative
                        records, respectively, did not provide documentation that a skills gap
                        analysis was conducted and another project researching expanded use of
                        administrative records did not provide the necessary skill sets. 21
                        According to Bureau officials, they are working to document these skills
                        sets, as well as any gaps. Completing this analysis is important to
                        ensuring sufficient resources are available for conducting the research
                        and testing projects.

                        We also found incomplete performance metric documentation for several
                        projects. According to Bureau guidance, project teams were to provide
                        performance metrics (including the methodology for monitoring and
                        evaluating project performance), as well as exit criteria for determining
                        each project’s final outcome. However, the “Enhancing Demographic


                        21
                          The LUCA Program Improvement and Supplementing and Supporting NRFU with
                        Administrative Records project teams did not provide documentation that a skills gap
                        analysis was conducted. The Enhancing Demographic Analysis project team did not
                        provide the necessary skill sets.




                        Page 20                                                           GAO-13-53 2020 Census
             Analysis” team that is researching the use of administrative records to
             assess the accuracy of census counts at the national level did not provide
             any of the required performance metrics, while six teams 22 failed to
             include performance metrics that could be used to monitor research and
             testing progress. According to Bureau officials, not all teams followed the
             project templates, and those teams have been asked to adhere to the
             requirements and provide all necessary information. Project performance
             metrics are a key element of effective project planning. As we have noted
             in prior work on project planning, measurable performance goals should
             be identified and performance data should be gathered to determine
             progress and whether desired outcomes have been achieved. 23 Absent
             such metrics, the Bureau does not have the assurance that it will be able
             to avoid potential slips in the research and testing schedule, or certainty
             as to whether project outcomes will be adequate for making decisions.


             Just over 7 years remain until Census Day 2020. While this might seem
Conclusion   like an ample amount of time to shore up the Bureau’s planning process
             and take steps to control costs, past experience has shown that the chain
             of interrelated preparations that need to occur at specific times and in the
             right sequence leave little room for delay or missteps. According to the
             Bureau, if 2010 operations are repeated in 2020, continued growth in
             population size and complexity will likely lead to an unsustainable
             increase in census cost. Further, traditional enumeration methods may no
             longer effectively produce a high quality census. To contain costs and
             maintain quality, bold innovations in both planning for and conducting the
             2020 Census will be required.

             The early research and testing phase represents a critical stage in
             preparing for the 2020 Census. At this time Bureau management is
             shaping the next decennial census as it determines what new operations
             will be a part of the 2020 Census design, which operations need to be
             revised, what risks remain, and how those risks will be mitigated. We
             have highlighted some of the cost savings that can be attained through
             three new operational changes being considered—the use of the Internet



             22
               The six project teams were: Master Address File (MAF) Error Model, Independent MAF
             Quality Assessment, LUCA Program Improvement, Workload Management, Matching
             Process Improvement, and Contact Frame.
             23
              GAO-04-37.




             Page 21                                                        GAO-13-53 2020 Census
                      as a self-response option, replacing enumerator collected data with
                      administrative records, and targeting only certain addresses for field
                      verification.

                      However, these innovations also come with risks that need to be
                      managed. Specifically, while the Bureau has identified project-level risks
                      for the 14 projects we reviewed, it has not developed risk mitigation plans
                      for them. Given the number of changes the Bureau would like to make for
                      2020 it is imperative that risks are sufficiently managed and that
                      mitigation plans be in place. For example, the Bureau needs to manage
                      the risk for accessing federally collected administrative data that it will use
                      for the 2020 Census. While it currently has access to some federally
                      collected data, the Bureau would like access to all federally collected
                      data, which according to the Bureau would require a statutory change.
                      However, such a change raises serious policy concerns including the
                      impact on personal privacy protections, and most likely would be a time
                      consuming process. Therefore, it is imperative that risk mitigation and
                      contingency plans be in place if the Bureau is unable to successfully gain
                      access to all federally collected data. Other areas that need attention
                      include ensuring that all research and testing projects have reliable cost
                      estimates, so as to avoid research and funding shortfalls, that
                      performance metrics are identified to ensure project goals are monitored
                      and met, and that skills are identified to ensure the selection and
                      assignment of appropriate staff to each project.


                      We recommend that the Acting Secretary of Commerce require the Under
Recommendations for   Secretary for Economic Affairs who oversees the Economics and
Executive Action      Statistics Administration, as well as the Acting Director of the U.S.
                      Census Bureau, to take the following three actions to improve the
                      Bureau’s Research and Testing for the 2020 Census, and thus better
                      position the Bureau to carry out a cost-effective decennial census:

                      •   Develop risk mitigation and contingency plans for all projects to
                          ensure that risks are adequately managed to minimize their effect on
                          the project.

                      •   Develop cost estimates for each project.

                      •   Ensure documentation for projects are complete, including specifying
                          the performance metrics that will be the basis for determining that
                          each of the projects has completed its work and identifying skills




                      Page 22                                                   GAO-13-53 2020 Census
                         needed to inform the selection and assignment of appropriate staff to
                         each research project.

                     The Acting Secretary of Commerce provided written comments on a draft
Agency Comments      of this report on October 22, 2012. The comments are reprinted in
and Our Evaluation   appendix IV. The Department of Commerce agreed with the assessments
                     and recommendations of the report. In addition, the Acting Secretary of
                     Commerce provided a technical comment and suggestion where
                     additional context might be needed, and we revised the report to reflect
                     this comment.

                     The Bureau in its comments stated that our report accurately represented
                     the extensive work that has been completed during the early research
                     and testing phase for the 2020 Census. To address our concerns, the
                     Bureau noted that it is now focusing on managing risks at the project level
                     and will begin obtaining more specific cost estimation details for each
                     project. The Bureau also agrees with our recommendations to ensure that
                     performance metrics and skill sets are identified for all projects and
                     teams. Finally, the Bureau stated that these efforts are either already
                     underway or are planned as a major focus during fiscal year 2013.


                     As arranged with your offices, unless you publicly announce the contents
                     of this report earlier, we plan no further distribution until 30 days after the
                     date of this report. At that time, we will send copies of this report to the
                     Acting Secretary of Commerce, the Under Secretary of Economic Affairs,
                     the Acting Director of the U.S. Census Bureau, and interested
                     congressional committees. The report also is available at no charge on
                     the GAO website at http://www.gao.gov.




                     Page 23                                                   GAO-13-53 2020 Census
If you have any questions on matters discussed in this report, please
contact me at (202) 512-2757 or by e-mail at goldenkoffr@gao.gov.
Contact points for our Offices of Congressional Relations and Public
Affairs may be found on the last page of this report. Key contributors to
this report are located in appendix V.




Robert Goldenkoff
Director
Strategic Issues




Page 24                                                 GAO-13-53 2020 Census
List of Requesters

The Honorable Thomas R. Carper
Chairman
The Honorable Scott Brown
Ranking Member
Subcommittee on Federal Financial Management, Government
  Information, Federal Services, and International Security
Committee on Homeland Security and Governmental Affairs
United States Senate

The Honorable Danny K. Davis
Ranking Member
Subcommittee on Health Care, District of Columbia, Census,
   and the National Archives
Committee on Oversight and Government Reform
House of Representatives




Page 25                                             GAO-13-53 2020 Census
Appendix I: Objectives, Scope, and
              Appendix I: Objectives, Scope, and
              Methodology



Methodology

              This report evaluates the U.S. Census Bureau’s (Bureau) efforts to
              improve the cost-effectiveness of the enumeration, paying particular
              attention to the following three key efforts that past Bureau research has
              shown could result in substantial cost savings: (1) leveraging the Internet
              to increase self-response; (2) improving how the Bureau builds its
              address and mapping databases, including a possible move to targeted
              address canvassing, and use of private-sector geographic data; and (3)
              using administrative records, such as tax data, to reduce nonresponse
              follow-up costs. To meet our objectives of identifying what opportunities
              and risks, if any, the Bureau might need to consider for these efforts
              going forward, and examining to what extent these three efforts are on
              track with respect to scheduling, resources, and other performance
              metrics, we reviewed Bureau documents pertaining to the early planning
              of the 2020 Census. These generally consisted of the Bureau’s
              operational plans, strategic framework documents, strategies and
              planning memorandums, timelines, and benchmarks. Many of these
              documents were considered draft, but Bureau officials said they were
              sufficiently developed for purposes of our review. To describe the
              Bureau’s efforts to use private-sector firms to assist with its address and
              mapping needs, we spoke to agency officials and reviewed the Bureau’s
              current mapping and address contracts and its request for information to
              solicit input from private-sector firms. We interviewed Bureau officials
              responsible for the early planning of the 2020 Census and inquired about
              their process for designing their research and testing program, as well as
              opportunities and risks presented by these three key efforts.

              We reviewed our prior reports on and other reports on leading practices
              to identify gaps in the Bureau’s research and testing efforts as it relates to
              schedule, resources, and performance metrics. We also reviewed reports
              from and spoke to knowledgeable stakeholders, such as officials at the
              National Academy of Sciences to determine opportunities and risks
              related to the Internet, administrative records, and the address and
              mapping databases. In addition, we spoke with Bureau officials and
              reviewed our prior work to identify obstacles the Bureau may need to
              address moving forward; for example, legal barriers to the Bureau’s
              access to administrative records.

              To assess the Bureau’s progress in meeting its stated goals for the
              research and testing program, we evaluated whether each of the 14
              research projects the Bureau initiated in 2012 had provided the required
              deliverables: the 120 day project plan, the project schedule, the project
              charter, and the project risk register. We also performed a content
              analysis of Bureau planning documents. For this analysis we examined


              Page 26                                                  GAO-13-53 2020 Census
Appendix I: Objectives, Scope, and
Methodology




the deliverables for each of the 14 research projects. We developed a
data collection instrument to record the presence or absence of 11 pieces
of information, or elements, in the deliverables for each project. The 11
elements were identified through a review of the Bureau’s project plan
template which outlines information that each project was expected to
report by the end of the 120-day initiation phase. A coding dictionary was
created to provide a uniform set of coding criteria for each element coded.
Each element could be coded as present, partially present, or absent.
During the coding process, two analysts independently coded documents
for each project using the coding dictionary as a guide. In the next phase,
the two coders met to reconcile any discrepancies between their coding
decisions and then made a final determination for that element. In cases
of disagreement, the engagement methodologist was consulted as a
tiebreaker. Following the coding process, the team analyzed the number
of present and absent elements to identify patterns for each element
across the 14 projects, as well as the completeness of documentation for
each project.

This engagement did not examine the Bureau’s information technology
policies, procedures, and information security in depth because our
information technology team is working on two engagements that cover
these issues.




Page 27                                                GAO-13-53 2020 Census
Appendix II: Fourteen Research Projects Will
                                          Appendix II: Fourteen Research Projects Will
                                          Inform Design Decisions on Key Efforts (Text
                                          for Interactive Figure 3)


Inform Design Decisions on Key Efforts (Text
for Interactive Figure 3)
Table 1: Project Teams Involved in Development of Internet Response Option

Project team                               Objectives in Internet development
Optimizing Self-Response                   Develop requirements for the Internet response option and coordinate the relationship
                                           between different response modes (i.e., Internet, paper, telephone)
Questionnaire Content, Design, and Mode    Determine the number of languages that will be used in the Internet response option
Study
Workload Management                        Develop the infrastructure that will support the Internet response option
Coding, Editing and Imputation Study       Help determine what prompts to include in an Internet response option to improve data
                                           quality and respondents’ ease of use
Privacy and Confidentiality Study          Identify public perception and concerns about responding to the Census via the Internet
Contact Frame                              Provide alternate contact methods, such as e-mail addresses
                                          Source: GAO analysis of U.S. Census Bureau information.




Table 2: Project Teams Contributing to Decision to Use Administrative Records

Project team                              Objectives in administrative records development
Alternative Administrative Records        Acquire, process, and analyze administrative records from federal, state, and commercial
Composite                                 sources to assess their utility for the 2020 Census
Supplementing and Supporting              Research and test methods to replace or supplement NRFU data collected in person with
Nonresponse Follow-up (NRFU) with         administrative data
Administrative Records
Local Update of Census Addresses          Explore the use of administrative records to validate new addresses
(LUCA) Program Improvement
 Coding, Editing and Imputation Study     Research the use of administrative records as a source to obtain missing household and
                                          address information
Enhancing Demographic Analysis            Research the use of administrative records to assess the accuracy of Census counts at
                                          the national level
Improving Quality Control                 Examine the use of administrative records to supplement or replace field work in quality
                                          control operations
Privacy and Confidentiality Study         Identify public perceptions related to the use of administrative records for enumeration
                                          purposes
Matching Process Improvement              Examine methods to ensure that the Census Bureau can accurately match administrative
                                          records to individuals and housing units
Contact Frame                             Acquire and process administrative records for the purpose of using them for alternate
                                          contact information
                                          Source: GAO analysis of U.S. Census Bureau information.




                                          Page 28                                                             GAO-13-53 2020 Census
                                         Appendix II: Fourteen Research Projects Will
                                         Inform Design Decisions on Key Efforts (Text
                                         for Interactive Figure 3)




Table 3: Project Teams Contributing to Decision to Implement Targeted Address Canvassing

Project team                              Objectives in targeted address canvassing development
Master Address File (MAF) Error Model     Develop a statistical model of errors (e.g., missing or duplicate addresses) in the MAF
Independent MAF Quality Assessment        Use the MAF Error Model to assess the quality of the MAF and determine whether
                                          targeted address canvassing would be effective, and, if so, where it should be performed
Local Update of Census Addresses (LUCA) Examine what modifications to the Bureau’s existing LUCA program will be required to
Program Improvement                     support targeted address canvassing
                                         Source: GAO analysis of U.S. Census Bureau information.




                                         Page 29                                                            GAO-13-53 2020 Census
Appendix III: Census Mapping and Address
                                          Appendix III: Census Mapping and Address
                                          Contracts



Contracts


Contractor                  Description of services                                                               Contract value
Acquis Inc.                 Develop, implement, and support a system which allows the Census Bureau                   $1,361,780
                            (Bureau) to update its Master Address File (MAF) with TIGER® (Topologically
                            Integrated Geographic Encoding and Referencing) data, the U.S. mapping data, as
                            the data is received.
ASRC Research and           Analyze mapping data as it is provided by state and local governments and other           $9,998,400
Technology Solutions        agencies, and upload changes into TIGER®.
Caliper Corporation         Customize and maintain software so that over 40,000 state, local, and tribal              $2,049,675
                            governments can update and verify the Bureau’s address and mapping databases.
                                                                                                                               a
ERDAS, Inc.                 Supply digital map files and digital aerial imagery to maintain and update the MAF/         $26,015
                            TIGER® database.
ESRI, Inc.                  Customize, install, and provide technical support for software to process and edit        $1,091,468
                            files to outline geographic areas, and support mapping applications, including the
                            use of handheld computers to collect and edit data.
                                                                                                                               a
Group 1 Software, Inc.      Provide software to match addresses from surveys and other agencies to the MAF/             $14,016
                            TIGER® database.
Gunnison Consulting Group, Develop and maintain software which automatically places text on Bureau maps so            $2,103,727
Inc.                       all map features are clearly identified.
Oracle America, Inc.        Update and consolidate the infrastructure, servers and database, of the MAF/              $9,973,265
                            TIGER® system.
Sabre Systems, Inc.         Provide services for the modernization of the Bureau’s MAF/TIGER® system,                $62,927,907
                            including expansion, design, programming, operation and maintenance of the
                            infrastructure supporting the MAF/ TIGER® system.
Total                                                                                                                $89,546,253
                                          Source: GAO analysis of U.S. Census Bureau information.
                                          a
                                          Cost of the most recent purchase.




                                          Page 30                                                            GAO-13-53 2020 Census
Appendix IV: Comments from the
             Appendix IV: Comments from the Department
             of Commerce



Department of Commerce




             Page 31                                     GAO-13-53 2020 Census
Appendix IV: Comments from the Department
of Commerce




Page 32                                     GAO-13-53 2020 Census
Appendix V: GAO Contact and Staff
                  Appendix V: GAO Contact and Staff
                  Acknowledgments



Acknowledgments

                  Robert Goldenkoff, (202) 512-2757 or goldenkoffr@gao.gov
GAO Contact
                  Other key contributors to this report include Lisa Pearson, Assistant
Staff             Director; David Bobruff; Robert Gebhart; Richard Hung; Kirsten B.
Acknowledgments   Lauber; Andrea Levine; and Timothy Wexler.




                  Page 33                                                GAO-13-53 2020 Census
Related GAO Products
             Related GAO Products




             2020 Census: Sustaining Current Reform Efforts Will Be Key to a More
             Cost-Effective Enumeration. GAO-12-905T. Washington, D.C.: July 18,
             2012.

             2020 Census: Additional Steps Are Needed to Build on Early Planning.
             GAO-12-626. Washington, D.C.: May 17, 2012.

             Decennial Census: Additional Actions Could Improve the Census
             Bureau’s Ability to Control Costs for the 2020 Census. GAO-12-80.
             Washington, D.C.: January 24, 2012.

             2010 Census: Preliminary Lessons Learned Highlight the Need for
             Fundamental Reforms. GAO-11-496T. Washington, D.C.: April 6, 2011.

             2010 Census: Data Collection Operations Were Generally Completed as
             Planned, but Long-standing Challenges Suggest Need for Fundamental
             Reforms. GAO-11-193. Washington, D.C.: December 14, 2010.

             GAO Cost Estimating and Assessment Guide: Best Practices for
             Developing and Managing Capital Program Costs. GAO-09-3SP.
             Washington D.C.: March 2, 2009.

             2010 Census: Census Bureau Should Take Action to Improve the
             Credibility and Accuracy of Its Cost Estimate for the Decennial Census.
             GAO-08-554. Washington, D.C.: June 16, 2008.

             Information Technology: Significant Problems of Critical Automation
             Program Contribute to Risks Facing 2010 Census. GAO-08-550T.
             Washington, D.C.: March 5, 2008.

             2010 Census: Cost and Design Issues Need to Be Addressed Soon.
             GAO-04-37. Washington, D.C.: January 15, 2004.

             High-Risk Series: Quick Reference Guide. GAO/HR-97-2. Washington,
             D.C.: February 1997.




(450946)
             Page 34                                               GAO-13-53 2020 Census
GAO’s Mission         The Government Accountability Office, the audit, evaluation, and
                      investigative arm of Congress, exists to support Congress in meeting its
                      constitutional responsibilities and to help improve the performance and
                      accountability of the federal government for the American people. GAO
                      examines the use of public funds; evaluates federal programs and
                      policies; and provides analyses, recommendations, and other assistance
                      to help Congress make informed oversight, policy, and funding decisions.
                      GAO’s commitment to good government is reflected in its core values of
                      accountability, integrity, and reliability.

                      The fastest and easiest way to obtain copies of GAO documents at no
Obtaining Copies of   cost is through GAO’s website (http://www.gao.gov). Each weekday
GAO Reports and       afternoon, GAO posts on its website newly released reports, testimony,
                      and correspondence. To have GAO e-mail you a list of newly posted
Testimony             products, go to http://www.gao.gov and select “E-mail Updates.”

Order by Phone        The price of each GAO publication reflects GAO’s actual cost of
                      production and distribution and depends on the number of pages in the
                      publication and whether the publication is printed in color or black and
                      white. Pricing and ordering information is posted on GAO’s website,
                      http://www.gao.gov/ordering.htm.
                      Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
                      TDD (202) 512-2537.
                      Orders may be paid for using American Express, Discover Card,
                      MasterCard, Visa, check, or money order. Call for additional information.
                      Connect with GAO on Facebook, Flickr, Twitter, and YouTube.
Connect with GAO      Subscribe to our RSS Feeds or E-mail Updates. Listen to our Podcasts.
                      Visit GAO on the web at www.gao.gov.
                      Contact:
To Report Fraud,
Waste, and Abuse in   Website: http://www.gao.gov/fraudnet/fraudnet.htm
                      E-mail: fraudnet@gao.gov
Federal Programs      Automated answering system: (800) 424-5454 or (202) 512-7470

                      Katherine Siggerud, Managing Director, siggerudk@gao.gov, (202) 512-
Congressional         4400, U.S. Government Accountability Office, 441 G Street NW, Room
Relations             7125, Washington, DC 20548

                      Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800
Public Affairs        U.S. Government Accountability Office, 441 G Street NW, Room 7149
                      Washington, DC 20548




                        Please Print on Recycled Paper.