United States Government Accountability Office GAO Report to Congressional Requesters November 2012 2020 CENSUS Initial Research Milestones Generally Met but Plans Needed to Mitigate Highest Risks GAO-13-53 November 2012 2020 CENSUS Initial Research Milestones Generally Met but Plans Needed to Mitigate Highest Risks Highlights of GAO-13-53, a report to congressional requesters Why GAO Did This Study What GAO Found GAO’s prior work has shown that it will According to U.S. Census Bureau (Bureau) officials, to inform timely design be important for the Bureau to re- decisions, research on new methods to improve the cost effectiveness of the examine the design of the census in 2020 Census must be accomplished early enough in the decade to confirm their order to ensure a cost effective census likely impact on both cost and quality. Three key efforts—(1) the use of the in 2020. As requested, this report Internet as a response option, (2) a potential move towards targeted address evaluates the Bureau’s efforts to canvassing, and (3) the possible use of administrative records to replace data improve the cost-effectiveness of the collected during census field operations—present the Bureau with potential enumeration, paying particular opportunities to reduce costs while maintaining quality. The Bureau’s 2020 attention to the following three key Research and Testing Program has 14 fiscal year 2012 projects focused on efforts: (1) leveraging the Internet to informing design decisions related to the three key efforts. Bureau officials are increase self-response; (2) improving how the Bureau builds its address and also aware that the changes they are testing come with many risks, and for each mapping databases, including a project the Bureau has identified a number of risks and prioritized them from high possible move to targeted address to low. However, the Bureau has not developed mitigation or contingency plans canvassing, and use of private-sector for these project risks. For example, there are several risks, including tight time geographic data; (3) and using frames and accurate cost information, without mitigation and contingency plans. administrative records to reduce Additionally, GAO found that the Bureau had not developed cost estimates for nonresponse follow-up costs. This any of its 2020 research and testing projects as required in guidance provided to report (1) identifies what opportunities project teams. Unreliable cost estimates was one of the reasons the 2010 and risks, if any, the Bureau might Census was placed on GAO’s high-risk list. Without timely cost estimates, it will need to consider for these efforts going be difficult for the Bureau to ensure that resources are adequate to support the forward and (2) examines to what research and testing program. extent these three efforts are on track with respect to scheduling, resources, The Bureau met its internal deadline for submitting each of its 14 research and other performance metrics. To project’s plans and charters. However, not all the project plans were complete. meet these objectives, GAO reviewed For example, some project teams did not fully document the types of skills Bureau documents and interviewed needed or perform a skills gap assessment to determine the resources needed to officials. carry out their respective projects as required in the Bureau’s planning template. According to Bureau officials, they are working to document these skills sets, as What GAO Recommends well as any gaps in skills. Completing this analysis is important to ensuring GAO recommends that the Acting sufficient resources are available for conducting the research and testing Census Director take a number of projects. Additionally, performance metric documentation for several projects was actions to improve the Bureau’s incomplete. According to Bureau guidance, project teams were to provide research and testing for the 2020 performance metrics for measuring progress and for determining the project’s Census, such as developing risk final outcome. However, one team did not provide either of these required mitigation plans, contingency plans performance metrics, while six other teams did not include performance metrics and cost estimates for each project, that could be used to monitor research and testing progress. Absent these and performance metrics and skill sets metrics, the Bureau does not have the assurance that it will be able to avoid for those projects that do not have potential schedule slips or the certainty as to whether project outcomes will be them. The Department of Commerce adequate for making decisions. concurred with GAO’s findings and recommendations and provided one clarification, which was included in the final report. Also, the Bureau in its comments noted that it has begun to address GAO’s recommendations. View GAO-13-53. For more information, contact Robert Goldenkoff at (202) 512-2757 or email@example.com. United States Government Accountability Office Contents Letter 1 Background 3 Key Efforts Present the Bureau with Both Opportunities and Risks 8 Research and Testing Project Plans Were Delivered on Schedule, but Not All Project-Level Resources and Performance Metrics Have Been Identified 20 Conclusion 21 Recommendations for Executive Action 22 Agency Comments and Our Evaluation 23 Appendix I Objectives, Scope, and Methodology 26 Appendix II Fourteen Research Projects Will Inform Design Decisions on Key Efforts (Text for Interactive Figure 3) 28 Appendix III Census Mapping and Address Contracts 30 Appendix IV Comments from the Department of Commerce 31 Appendix V GAO Contact and Staff Acknowledgments 33 Related GAO Products 34 Tables Table 1: Project Teams Involved in Development of Internet Response Option 28 Table 2: Project Teams Contributing to Decision to Use Administrative Records 28 Table 3: Project Teams Contributing to Decision to Implement Targeted Address Canvassing 29 Page i GAO-13-53 2020 Census Figures Figure 1: 2020 Census Design Alternatives Show Varying Degrees of Change, Risk, and Potential Cost Savings 4 Figure 2: The Life Cycle of the 2020 Census Has Five Phases 5 Figure 3: Research Projects Will Inform Design Decisions on Key Efforts 7 Abbreviations ACS American Community Survey Bureau U.S. Census Bureau GSS Geographic Support Systems LUCA Local Update of Census Addresses MAF Master Address File NRFU Nonresponse follow-up TIGER® Topologically Integrated Geographic Encoding and Referencing USPS United State Postal Service This is a work of the U.S. government and is not subject to copyright protection in the United States. The published product may be reproduced and distributed in its entirety without further permission from GAO. However, because this work may contain copyrighted images or other material, permission from the copyright holder may be necessary if you wish to reproduce this material separately. Page ii GAO-13-53 2020 Census United States Government Accountability Office Washington, DC 20548 November 7, 2012 Congressional Requesters The 2010 Census, at around $13 billion, was the most expensive headcount in our nation’s history. Meanwhile, the cost of conducting the census has, on average, nearly doubled each decade since 1970 in constant 2010 dollars. If that growth rate continues, the 2020 Census could potentially cost around $25 billion. The U.S. Census Bureau (Bureau) has recognized the need for more cost-effective approaches to conducting the enumeration and is committed to identifying and implementing innovations and improvements as necessary to conduct the 2020 Census at a lower cost per housing unit than the approximately $100 per housing unit cost of the 2010 Census (in constant 2010 dollars) while still maintaining high quality. Between fiscal years 2012 and 2014, the Bureau will conduct its research and testing phase for the 2020 Census in order to develop an initial design by September 2014. The decisions made in the decade’s first years shape the design and set the framework for later decisions that will affect how the census will be ultimately conducted. As requested, this report evaluates the Bureau’s research efforts to improve the cost-effectiveness of the 2020 Census enumeration, paying particular attention to the following three key efforts that past Bureau research has shown could result in substantial cost savings: • leveraging the Internet to increase self-response; • improving how the Bureau builds its address and mapping databases, including a possible move to targeted address canvassing (as opposed to the full address canvassing used in earlier censuses, where field staff walked nearly every street in the country to verify the address list), and use of private-sector geographic data; and Page 1 GAO-13-53 2020 Census • using administrative records, 1 such as tax data, to reduce nonresponse follow-up costs. Our objectives were to: (1) identify what opportunities and risks, if any, the Bureau might need to consider for these research efforts going forward and (2) examine to what extent these three research efforts are on track with respect to scheduling, resources, and other performance metrics. To meet these objectives, we reviewed documentary evidence, including 2020 Census research and testing plans, strategic framework documents, and our prior reports on the subject; and conducted interviews with cognizant Bureau officials, as well as knowledgeable stakeholders, such as those at the National Academy of Sciences, to determine opportunities and risks related to the Internet, administrative records and the address and mapping databases. We also obtained and compared Bureau- developed research planning requirements to research deliverables in order to identify gaps in the Bureau’s research and testing efforts as it relates to schedule, resources, and performance metrics. Additionally, we interviewed agency officials to identify the Bureau’s 2020 research and testing projects, timelines, and benchmarks. We obtained and reviewed contract-related documents related to geographic data and services, as well as interviewed Bureau officials to determine the Bureau’s plan for using the private sector to assist with the Bureau’s address and mapping needs. For more information on our scope and methodology see appendix I. We conducted this performance audit from October 2011 to September 2012 in accordance with generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based on our audit objectives. We believe that the evidence obtained provides a reasonable basis for our findings and conclusions based on our audit objectives. 1 Administrative records refer to data records contained in files collected and maintained by administrative or program agencies and commercial entities. Government and commercial entities maintain these files for the purpose of administering programs and providing services. Page 2 GAO-13-53 2020 Census The Bureau’s 2020 Census early research agenda is focused on Background conducting the 2020 Census at a lower cost than the 2010 Census 2 while maintaining high quality results. In order to do this the Bureau has developed a range of design alternatives for the 2020 Census. The Bureau’s 2020 design alternatives have potential for containing costs but also have the potential to increase risks. The design alternatives focus on options to use the Internet and other social media to increase response rate, target address canvassing, and expand the use of administrative records. Figure 1 shows the current range of six 2020 Census design alternatives currently being considered and how the three key efforts fit into them. According to the Bureau, the final 2020 design is likely to incorporate both existing approaches as well as design alternatives that have never been used in the decennial census. Greater changes to the overall design will result in greater potential for cost savings. However, greater design changes also could result in greater risk, and testing will be needed to identify the risks, costs, and benefits of any new approaches. According to the Bureau, alternative one has the lowest risk because it most closely mirrors the 2010 Census design (mailout of census forms, and in-person follow-up of non respondents) and is not dependent on implementing innovations such as increased use of administrative records and targeted address canvassing. 2 Per housing unit, on an inflation-adjusted basis. Page 3 GAO-13-53 2020 Census Figure 1: 2020 Census Design Alternatives Show Varying Degrees of Change, Risk, and Potential Cost Savings The Bureau formally kicked off initial planning for the 2020 Census on October 3, 2008, with the 2020 Census Planning Summit. At that meeting, decennial-census managers and subject matter experts discussed challenges, strengths, and weaknesses that would need to be considered during the 2020 Census strategic decision-making process. From 2009 to 2011, during the options analysis phase of 2020 Census planning, Bureau officials went through a research topic winnowing process where they: identified cost and quality drivers, conducted brainstorming sessions, received expert input (e.g., National Academy of Sciences), cataloged over 600 ideas, clustered those ideas into over 100 projects with research questions, and then prioritized them against the Bureau’s Guiding Principles 3 for the 2020 Census. The Bureau identified 26 research projects to inform preliminary design decisions with the goal 3 The Bureau’s Guiding Principles for the 2020 Census are: Honor Privacy and Confidentiality; Be Objective; Commit to Quality; Contain Costs Wherever Possible; Leverage Assets, Methods, Knowledge, and Processes across the Bureau; and Be Responsive and Transparent to Partners and Stakeholders. Page 4 GAO-13-53 2020 Census of conducting the 2020 Census at a lower cost than the 2010 Census, while maintaining high quality. Using the 26 research projects, Bureau officials plan to validate the feasibility of the various strategies for reducing costs between fiscal years 2012 and 2014. Bureau officials stated that they are planning to narrow down the design alternatives by September 2014, and that research would continue into the next phase. Bureau officials believe that conducting research earlier in the decade will better position them to make more effective changes to the 2020 Census design. Figure 2 shows the five phases of the life cycle for the 2020 Census. Figure 2: The Life Cycle of the 2020 Census Has Five Phases The Bureau is now in the “Early Research and Testing” phase and is currently refining and developing its 26 project plans. These plans include, among other requirements, project goals, objectives, scope, limitations, and skills required, and had a set of required iterative research and testing deliverables due 30, 60, 90, and 120 days after project kickoff. According to Bureau documentation, this process encouraged the development and proper consideration of ideas. The Bureau selected 17 of the 26 projects to start in fiscal year 2012. The remaining projects are scheduled to begin in fiscal years 2013 and 2014. Of the 17 research projects the Bureau is conducting in 2012, 14 projects are to inform design decisions related to the Internet, administrative records, and Page 5 GAO-13-53 2020 Census targeted address canvassing. See figure 3 for a list of these 14 research projects and their objectives. According to the Bureau’s timeline, project teams are to complete their work on these 14 projects by September 2014. 4 4 A Bureau official stated that they are assessing the impact of the fiscal year 2013 Continuing Resolution on their ability to complete all of their work by September 2014. Page 6 GAO-13-53 2020 Census Interactive graphic Figure 3: Fourteen Research Projects Will Inform Design Decisions on Key Efforts Move mouse over each activity to see more information afterwards CLICK to close the open pop-up information display panel. Objectives in Objectives in Internet internet Objectives in Administrative Objectives in Targeted Address development Development Records Development Canvassing Development Project teams involved in development of Project teams contributing to decision to Project teams contributing to decision to Internet response option use administrative records implement targeted address canvassing Source: GAO analysis of U.S. Census Bureau Information . Print instructions To print text version of this graphic, go to appendix II. Page 7 GAO-13-53 2020 Census Research on new methods likely to result in a more cost-effective 2020 Key Efforts Present Census must be accomplished early enough in the decade to confirm the Bureau with Both their likely impact on both cost and quality and to inform timely design decisions. The use of the Internet as a response option, a potential move Opportunities and towards targeted address canvassing, and the possible use of Risks administrative records to replace data collected during census field operations present the Bureau with opportunities to reduce costs while maintaining a high quality census. As part of the 2020 Census planning process the Bureau has identified both program-level and project-level risks associated with these options. According to the Bureau, program- level risks span the entire 2020 research and testing phase and if not managed properly could jeopardize the phase’s goals and objectives. In contrast, project-level risks pertain to the successful completion of specific projects. However, the Bureau has not developed mitigation and contingency plans for the project-level risks. Also, the Bureau has not developed cost estimates for any of its 2020 research and testing projects as required in guidance provided to project teams. Finally, some of the project plans we reviewed were incomplete. For example, not all project plans identified the necessary staff resources, while other project plans failed to provide performance metrics for measuring progress and for determining the project’s final outcome. All Three Key Efforts Present the Bureau with Cost Saving Opportunities Internet During the 2000 Census, the Bureau piloted an Internet response option that had a limited number of respondents. They considered building on this 2000 experience for the 2010 Census, but the Bureau decided in July 2006 not to include the Internet option in the design for the 2010 Census because test results indicated that the Internet option did not increase the overall response rate, including response among hard-to-count population groups, and they had underestimated the costs of the contract that included developing the Internet option. 5 5 Census Bureau, Rationale for the Decision to Eliminate the Internet Option for the 2010 Decennial Census, 2010 Decennial Census Program Decision Memorandum Series No. 14 (Washington, D.C.: July 19, 2006). Page 8 GAO-13-53 2020 Census However, more recent tests conducted in 2011 showed that adding an Internet response option increased the overall response rate. The 2011 test results, coupled with the increased prevalence and accessibility of the Internet, led Bureau officials to commit to providing an Internet response option for the 2020 Census. With regard to the cost for the 2020 Census, if this option can help achieve an overall increase in the response rate it can save money, since Bureau field staff would need to visit fewer households during nonresponse follow-up (NRFU), which is the largest and most costly census field operation. 6 Furthermore, testing has shown that the cost of an Internet survey is low compared to a mail survey, which incurs printing and postage costs. Moreover, web survey responses are generally available more quickly and are of better quality than responses from a mail survey because there is no lag time as the responses are captured in real time and there are reminders to prompt the respondent if a question is unanswered. Quicker and complete responses can also help reduce the amount of time and money spent on following up on late or incomplete census forms. 7 In April and November 2011, the Bureau tested an Internet response option in its American Community Survey (ACS), 8 and beginning in January 2013, the Bureau plans to offer it as a response option. Bureau officials have stated that they intend to build on this existing information technology infrastructure in order to reduce the cost of implementing an Internet option for the 2020 Census. Given the Bureau’s decision to move forward with an Internet design option, six research and testing projects are focused on how best to implement an Internet response option in concert with other self-response options. The project team with primary responsibility for coordinating the design of the Internet response option is tasked with answering the following research question: 6 During NRFU the Bureau sends enumerators to collect data from households that did not mail back their census forms. NRFU procedures instruct enumerators to make up to six attempts to contact a household. The 2010 Census NRFU operation cost $1.6 billion. 7 We have a separate review underway examining the Bureau’s information technology security issues. 8 ACS is an ongoing survey that provides data every year. Information from the survey generates data that help determine how federal and state funds are distributed each year. Page 9 GAO-13-53 2020 Census • How does the Bureau leverage technology and new response modes (including the Internet) to increase self-response, improve nonresponse follow-up data collection strategies, while maintaining overall quality? One objective of this project is to determine what contact strategy should be used to inform the public that census responses can be filed via the Internet in order to increase self-response (i.e., Bureau use of email, social networks or phone for initial contact or for reminders). Contact strategy testing will examine the impact of various Bureau instructions to respondents, as well as the timing of the delivery of the mailing pieces, on such factors as response rates, return rates, percentage of Internet returns, and speed of returns. This project will also determine what mode (paper, Internet, or phone) is best for each demographic, geographic, and language group, as well as determine the effect a lack of access to technology has on each demographic group. Additionally this project will refine and test security features. The other five projects will cover such areas as types of languages that will be offered by response mode, public perception and potential concerns about an Internet response, and management of the workload for multiple modes of response. Targeted Address Canvassing In the 2010 and earlier censuses, the Bureau mounted a full address and Use of the Private Sector canvassing operation, where field staff verified every housing unit in the for Geographic Data Services nation to update the Master Address File (MAF) and the associated mapping system called TIGER® (Topologically Integrated Geographic Encoding and Referencing). This labor-intensive effort was one of the more expensive components of the 2010 Census: the 2010 Address Canvassing Operation required 140,000 temporary workers to verify 145 million addresses (by going door-to-door) at a cost of $444 million, or 3 percent of the $13 billion total cost of the 2010 Census. One of the reasons the Bureau conducted a full address canvassing for the 2010 Census was because it wanted to capture hidden and hard-to-capture addresses, such as housing units in converted garages and large storage sheds. For the 2020 Census the Bureau would like to reduce the amount of field work and cost of address and map updates by targeting the address canvassing operation. According to Bureau planning documents, through early research and testing the Bureau plans to attempt to determine the quality and stability of its address file in terms of the number of changes (adds, deletes, duplicates, and corrected addresses) that took place to each census block in 2010. Depending on that outcome, the Bureau would target for address canvassing only those blocks where the quality of the addresses was determined to be inadequate. Page 10 GAO-13-53 2020 Census To determine whether to target address canvassing for the 2020 Census, the Bureau has two ongoing efforts to improve the Bureau’s map and address databases. First, the Bureau’s Geography Division is working with the United States Postal Service (USPS) and other federal agencies on a new program called the Geographic Support Systems (GSS) Initiative, which allows government agencies at all levels to regularly share and continuously update their address lists with the Bureau. According to current plans, USPS will continue providing the Bureau with address updates. Likewise, tribal, state, and local governments, which maintain address lists for purposes such as emergency response and property assessment, will also have the opportunity to share addresses with the Bureau throughout the decade, rather than solely 2 years prior to the decennial, as had been the case in prior decennial censuses. Additionally, the GSS Initiative is working to develop, and in some cases improve, methods for identifying and capturing hidden and hard-to- capture addresses. As part of this first effort, the Bureau is investigating the role and possible contributions the private sector can make in improving its address and mapping databases. For the 2010 Census the Bureau relied heavily on the private sector to update addresses and maps. Specifically, in June 2002, the Bureau awarded an 8-year contract of about $200 million to improve the accuracy of its mapping database and investigate options and methods of updating its address database. The contract, among other tasks, corrected in TIGER® the location of streets, boundaries, and other map features so that coordinates would be aligned with their true geographic locations. According to Bureau officials, continued reliance on the private sector is necessary in order to maintain the major upgrades that were made to its address and mapping databases last decennial. The Geography Division has nine contracts in place, totaling about $90 million dollars, to help it manage its mapping and address database (see app. III for a list of those contracts). Some of the work being done by the private sector includes a service contract to provide system upgrades and maintenance to the MAF/TIGER® database and a data entry services contract to update the MAF/TIGER® database with address information provided by state and local governments. Moreover, to ensure the most current address and mapping data are available for the 2020 Census, the Bureau, in April 2012, began conducting market research with three private-sector firms to assess how those firms might assist the Bureau with its address and mapping needs. Possibilities to be explored include how to best maintain address and map data, understand private-sector innovation and technology, and Page 11 GAO-13-53 2020 Census options for address and map data-sharing methods. According to Bureau officials, negotiations concerning the scope, tasks, and activities to be completed with the first contractor have taken place. However, negotiations with the remaining two firms have not begun. Use of the private sector for mapping is an area we have previously reported needs to be explored. Specifically, in December 2010, we listed a number of reexamination areas which have particular implications for controlling costs and improving accuracy and raised the question: To what extent can private-sector and other sources of information such as maps, address lists, and geographic databases be employed to help support the census? 9 The second effort to improve the Bureau’s address and mapping database is centered on the 2020 Research and Testing Program. There are three research projects designed to inform the Bureau’s decision about the extent to which it might be able to conduct a targeted address canvassing operation. Two projects will use modeling to predict where coverage errors (e.g., missed or duplicate addresses) occur in the MAF. These models will be used to assess the quality of data sources, types of errors, and what information, such as a missing map location for an address, is not in the MAF. If the Bureau decides to conduct targeted address canvassing, this information will be used to determine which areas meet acceptable quality standards, as well as where targeted address canvassing would be effective. According to the Bureau, collectively the two projects are to answer the following question: • As the related GSS Initiative proceeds, how will the Bureau determine the required level of quality needed in the address frame to conduct an accurate census and then measure the quality of the continually updated MAF for that purpose? The third research project in this effort, the Local Update of Census Addresses (LUCA) Program Improvement project, 10 will examine how to modify the Bureau’s existing partnership program in consultation with the 9 GAO, 2010 Census: Data Collection Operations Were Generally Completed as Planned, but Long-standing Challenges Suggest Need for Fundamental Reforms, GAO-11-193 (Washington, D.C.: Dec. 14, 2010). 10 The LUCA Program was established by the Census Address List Improvement Act of 1994 to improve the accuracy of census’ address list by exchanging information with tribal, state, and local governments. Page 12 GAO-13-53 2020 Census GSS Initiative. The LUCA program is a partnership program that provides an opportunity for local and tribal governments to review and update individual address information or block-by-block address counts from the MAF and associated geographic information in the TIGER® database just prior to the decennial. The goal of LUCA is to improve the completeness and accuracy of address information. This research project’s objective is to determine the best way to modify and improve LUCA procedures to ensure compatibility with continual updating of address information throughout the decade. The project will also determine whether it is necessary for local and tribal governments to actively participate in LUCA, a once-a-decade operation, if they have been submitting addresses and mapping data continuously throughout the decade as part of the GSS Initiative. Administrative Records Administrative records are a growing source of information on individuals and households. For purposes of the decennial census, the Bureau is considering administrative records from government agencies, including tax data and Medicare records, as well as commercial sources to identify persons associated with a particular household address. 11 During the 2010 Census, the Bureau made limited use of administrative records. For example, the Bureau used USPS files to update its address list, and federal agency records (such as those from the Department of Defense) were used to count military and federal civilian employees stationed outside of the United States. For 2020, in its first attempt to estimate potential costs of various design alternatives for the 2020 Census, the Bureau estimated that by expanding the use of administrative records it could save up to $2 billion by reducing the workload for several operations, including NRFU, address building, and quality assurance. 12 Depending on the results of the research and testing projects, Bureau officials plan to build a composite of quality administrative records from various sources (i.e., federal agencies, state and local governments, and commercial sources) that it can use to reduce or replace costly field work. Thus, using administrative records 11 The Bureau’s access to and use of administrative records is governed by agency- specific statutes. For example, the Bureau has access to tax data under 26 U.S.C. § 6103(j)(1) “for the purpose of, but only to the extent necessary in, the structuring of censuses ... and conducting related statistical activities.” 12 The amount of and quality of administrative records the Bureau is able to collect will impact the amount of cost savings the Bureau is able to realize. Page 13 GAO-13-53 2020 Census could allow the Bureau to reduce the scale of its NRFU operation and reduce the need for field office space and staff. The 2012 research and testing projects are also focused on identifying the best available administrative records to use for address frame building. Administrative data from other sources would be combined with USPS data, thus allowing the Bureau to continuously update its MAF throughout the decade. Finally, administrative records could reduce the cost of quality control, according to Bureau officials. In order to ensure the accuracy of the field work conducted during census field operations, such as NRFU, the Bureau sends quality control staff into the field to verify the work completed by a random sample of enumerators. Bureau officials are determining for 2020 whether administrative records can be used as a quality control check, rather than sending staff into the field. There are nine research and testing projects that will inform the Bureau’s decision making on the extent to which the Bureau can expand the use of administrative records in order to reduce costs. These research and testing efforts are focused on determining whether the quality of administrative records is sufficient to be used for this purpose. Initial testing is to be completed and decisions on these matters are to be made by September 2014. Of the nine, there are two primary administrative records research projects. They are tasked with answering the following research questions: • How can the Bureau best develop and maintain an independent collection of administrative records and assess the quality of those records (best sources and methods)? • How can the Bureau leverage administrative records (including commercial files) to significantly reduce decennial census cost while maintaining quality? The objective of one of these projects is to acquire, process, and analyze administrative records from federal, state, and commercial sources to assess their utility for the 2020 Census. The other project will research and test methods to enhance NRFU operations with administrative records, such as replacing phone or in-person collected data with administrative data. Page 14 GAO-13-53 2020 Census The Bureau Has Identified Bureau officials are aware that the changes they want to make to the 2020 Census Program- decennial census come with many risks. The Bureau has identified and Level and Project-Level prioritized from high to low both program-level and project-level risks. According to the Bureau, program-level risks span the entire 2020 Risks but Has Not research and testing phase and if not managed properly could jeopardize Developed Project-Level the phase’s goals and objectives. In contrast, project-level risks pertain to Mitigation and the successful completion of projects. In accordance with best risk Contingency Plans management practices, 13 the Bureau has identified and drafted mitigation plans for all of the 14 program-level risks it has identified. In May 2012 the Bureau designated two of the 14 program-level risks as high risk: • Timely research and testing results: The 2020 research and testing strategy involves “many small field tests to research design alternatives with an accelerated, agile, and informed decision-making process for incorporating changes to the 2020 Census design.” According to the Bureau, if the research and testing results are late, then decisions will not be made on time and the program may not be ready to move out of the research and testing phase on schedule. According to the Bureau’s 2020 Census Risk Management Mitigation Plans, officials have begun implementing a mitigation strategy for this risk, including documenting clear roles and responsibilities and communicating them to staff; establishing a schedule that contains decision milestones and tracking interdependencies of research results and testing projects; and reviewing ongoing project status including updates on dependencies, risks, and metrics. • Significant budget cuts: The Bureau reported that if its funding for 2020 Census planning is significantly reduced for fiscal year 2013, key projects will have to be delayed and as a result the Bureau could have major technical and operational difficulties making preliminary design decisions. According to Bureau documents, to mitigate this risk, Bureau officials have briefed staff from House of Representatives Subcommittee for Commerce, Justice, Science and Related Agencies, Committee on Appropriations, and the Office of Management and Budget on the implications of reduced funding and plan to brief oversight committee staff. In addition, Bureau officials 13 Broadly defined, risk management is a strategic process for helping policymakers make decisions about assessing risk, allocating finite resources, and taking actions under conditions of uncertainty. Page 15 GAO-13-53 2020 Census reported they have analyzed budgetary needs for fiscal year 2013 and prioritized projects to manage limited resources. Related to these risks, a June 2012 congressional hearing held by the Joint Economic Committee discussed the impact of eliminating or reducing funding to the ACS. According to Bureau officials, such action would have a significant impact on the 2020 Census research and testing program because ACS is one of its testing platforms. This risk is not listed as a program-level risk, or part of the current risk mitigation plans, but Bureau managers told us they are strategizing on what they will do if ACS is not funded and will update the risk mitigation plan accordingly. The Bureau has identified lack of support from external stakeholders as a medium program-level risk. Bureau officials are concerned that if external stakeholders, such as Congress, the Office of Management and Budget, and the National Academy of Sciences, do not support research on alternative design options, then the research and testing agenda will have to be redesigned. To manage this risk, Bureau officials have communicated the alternative design options to a wide range of external stakeholders, but are concerned that support may diminish as time passes and stakeholders change. To mitigate this risk, the Bureau is engaging external stakeholders in communications regarding 2020 research and testing efforts and seeking feedback on Bureau plans. According to Bureau documents, the Bureau has met with external stakeholders, including congressional staff on its House of Representatives and Senate oversight committees, to discuss 2020 Census planning, and is working on a strategy for keeping stakeholders informed as plans progress. Unlike the program-level risks, risk mitigation and contingency plans have not been drafted to address the project-specific risks. Specifically, each of the Bureau’s research and testing projects has its own risk register 14 that identifies and prioritizes the risks associated with that project, but no risk mitigation or contingency plans have been developed for these project- level risks. Several risks appear in multiple project team risk registers, including tight time frames and accurate cost information. For example, one Internet-related project listed that there is a risk that if tests are not 14 A risk register contains the results of a projects risk analysis and includes risk descriptions, probability of occurrence, cost impact, schedule impact, and identifies the Bureau official with ownership of the risk. Page 16 GAO-13-53 2020 Census timed properly, and if time frames are too tight then they may not be able to apply what they have learned from one test to the next. Similarly, one of the administrative records project teams was concerned that one year to integrate data, build a model, and conduct adequate analysis of administrative records was “not realistic.” In another example concerning cost data, two teams stated that without accurate cost data at the project level, research results will not be sufficient to inform the design of the 2020 Census. Project teams also identified project-level risks directly related to the use of administrative records in place of data collected by enumerators during field visits. Among these risks is access to administrative records to build a high quality compilation of administrative records for each household across the country. To accomplish this task, the Bureau needs timely access to data collected by the federal government and, as previously mentioned, is currently compiling administrative records from various federal agencies including the Departments of Housing and Urban Development and Health and Human Services. However, not all agencies are authorized to provide their data to the Bureau thus limiting, and in some cases preventing, the Bureau’s use of that data. Bureau officials provided us with two examples of agency administrative record sources that they are not authorized to access: records maintained under the Family Educational Rights and Privacy Act and the National Directory of New Hires. 15 Bureau officials said they would like to have access to these and other data sources for statistical purposes in order to further increase the number and improve the quality of available administrative records. According to Bureau officials, reviewing and determining whether they have access to administrative records with every agency is a time consuming process and the Bureau would like to have the ability to fully access and use agency administrative data, especially since the confidentiality of census data are protected by Title 13. 16 To move this forward, according to a senior Bureau official, legislation might need to be enacted that would compel federal agencies to provide the Bureau with access to administrative data for the decennial census. However, the 15 The Family Educational Rights and Privacy Act protects the privacy of student education records including personally identifiable information (name, address, social security number, etc.) from those records. The National Directory of New Hires contains wage and employment information that is used by state agencies to enforce child support payments. 16 13 U.S.C. § 9. Page 17 GAO-13-53 2020 Census enactment of such legislation is likely to be a time consuming process that raises serious policy concerns including whether the Bureau has made a sufficiently strong case that it needs access to all federally collected data and the resulting impact on personal privacy protections. According to Bureau officials, the project teams did not submit risk mitigation and contingency plans for the project-level risks as required at the 90-day checkpoint because after reviewing all of the project-level risk registers the project leaders first wanted to unduplicate similar risks across projects in order to prevent redundancies. Bureau officials were not able to provide a date for when risk mitigation and contingency plans would be available. However, they stated they are committed to managing risk in order to avoid the missteps encountered in 2010 when the Bureau was on our high-risk list due to weaknesses in managing IT, operational planning, and cost estimating. 17 In our prior work, we reported that risk mitigation involves identifying, analyzing, prioritizing, and documenting risks, and ideally more than one alternative should be assessed. 18 The Department of Commerce’s Inspector General has recommended that risk management activities begin from the outset of the current decennial census life cycle, rather than just before field operations. Without mitigation plans the 2020 Census planning team may not be able to fully manage risks associated with these projects. Project-Level Cost The Bureau did not develop cost estimates for any of its 2020 research Estimates Have Not Been and testing projects by the 90-day checkpoint as required in guidance Developed provided to project teams. The importance of reliable cost estimates is underscored by the Bureau’s experience leading up to the 2010 Census, where we found that the Bureau’s cost estimates lacked detailed documentation on data sources and significant assumptions and was not comprehensive because it did not include all costs. 19 Partly as a result, some operations had substantial variances between their initial cost 17 GAO, Information Technology: Significant Problems of Critical Automation Program Contribute to Risks Facing 2010 Census, GAO-08-550T (Washington, D.C.: Mar. 5, 2008). 18 GAO, 2010 Census: Cost and Design Issues Need to Be Addressed Soon, GAO-04-37 (Washington, D.C.: Jan. 15, 2004). 19 GAO, 2010 Census: Census Bureau Should Take Action to Improve the Credibility and Accuracy of Its Cost Estimate for the Decennial Census, GAO-08-554 (Washington, D.C.: Jun. 16, 2008). Page 18 GAO-13-53 2020 Census estimates and their actual costs. For example, the 2010 address canvassing operation cost $88 million more than the original estimate of $356 million, an overrun of about 25 percent; and the Bureau’s 2010 NRFU operation cost $1.6 billion, about $660 million, or 29 percent, less than the Bureau initially estimated. According to 2020 Census planning documents, at the 90-day checkpoint project teams were to: (1) prepare and submit cost estimate requests needed for the project; (2) document estimates for completing the work for the project; (3) identify and document funds needed for additional resources and requirements, and (4) prepare and submit requests for additional funds. According to a senior Bureau official, cost estimates have not yet been completed because more work needs to be done to determine project costs and they are in the process of formulating cost estimates for each project. According to our cost estimate guide, 20 a cost estimate provides an assessment of the costs most likely to be incurred and should include the requirements, resources, and tasks that must be accomplished. As the Bureau moves forward with its 2020 research and testing program, project cost estimates will allow the Bureau to plan for the resources required to conduct necessary field testing, ensure there are sufficient funds, and adjust testing accordingly. In addition, the Bureau could use these cost estimates to help determine how budget cuts might hinder the census research and testing program’s progress or effectiveness. Thus, if the Bureau does not develop cost estimates in a timely manner it will be difficult for the Bureau to ensure that resources are adequate to support research and testing projects. 20 GAO, GAO Cost Estimating and Assessment Guide: Best Practices for Developing and Managing Capital Program Costs, GAO-09-3SP (Washington D.C.: Mar. 2, 2009). Page 19 GAO-13-53 2020 Census The Bureau’s early research and testing is currently refining the project Research and Testing plans associated with the14 projects related to the Internet, administrative Project Plans Were records, and targeted address canvassing. Each of the research and testing projects has a set of required deliverables due 30, 60, 90, and 120 Delivered on days after project kickoff. We determined that each project team had Schedule, but Not All provided the 120-day required deliverables on schedule to Bureau Project-Level managers in June 2012. The research and testing documents required by the 2020 Census planning team and delivered by the 14 project teams Resources and describe management and technical plans for each project, and include a Performance Metrics team charter, project plan, resources, preliminary schedule, and field tests. These documents are important because they serve as the Have Been Identified baseline for each of the research and testing projects. However, not all resource requirements were documented. For example, some project teams did not fully document the types of skills needed, or perform a skills gap assessment to determine the resources needed to carry out their respective projects as required in the Bureau’s planning template (the template lays out in detail what is to be included in the deliverables). Bureau officials plan to use this skills information to allocate staff to ensure teams have the skills necessary to complete their projects. While 11 project teams provided all the required information, 2 project teams conducting research on issues that will inform decisions on targeted address canvassing and expanded use of administrative records, respectively, did not provide documentation that a skills gap analysis was conducted and another project researching expanded use of administrative records did not provide the necessary skill sets. 21 According to Bureau officials, they are working to document these skills sets, as well as any gaps. Completing this analysis is important to ensuring sufficient resources are available for conducting the research and testing projects. We also found incomplete performance metric documentation for several projects. According to Bureau guidance, project teams were to provide performance metrics (including the methodology for monitoring and evaluating project performance), as well as exit criteria for determining each project’s final outcome. However, the “Enhancing Demographic 21 The LUCA Program Improvement and Supplementing and Supporting NRFU with Administrative Records project teams did not provide documentation that a skills gap analysis was conducted. The Enhancing Demographic Analysis project team did not provide the necessary skill sets. Page 20 GAO-13-53 2020 Census Analysis” team that is researching the use of administrative records to assess the accuracy of census counts at the national level did not provide any of the required performance metrics, while six teams 22 failed to include performance metrics that could be used to monitor research and testing progress. According to Bureau officials, not all teams followed the project templates, and those teams have been asked to adhere to the requirements and provide all necessary information. Project performance metrics are a key element of effective project planning. As we have noted in prior work on project planning, measurable performance goals should be identified and performance data should be gathered to determine progress and whether desired outcomes have been achieved. 23 Absent such metrics, the Bureau does not have the assurance that it will be able to avoid potential slips in the research and testing schedule, or certainty as to whether project outcomes will be adequate for making decisions. Just over 7 years remain until Census Day 2020. While this might seem Conclusion like an ample amount of time to shore up the Bureau’s planning process and take steps to control costs, past experience has shown that the chain of interrelated preparations that need to occur at specific times and in the right sequence leave little room for delay or missteps. According to the Bureau, if 2010 operations are repeated in 2020, continued growth in population size and complexity will likely lead to an unsustainable increase in census cost. Further, traditional enumeration methods may no longer effectively produce a high quality census. To contain costs and maintain quality, bold innovations in both planning for and conducting the 2020 Census will be required. The early research and testing phase represents a critical stage in preparing for the 2020 Census. At this time Bureau management is shaping the next decennial census as it determines what new operations will be a part of the 2020 Census design, which operations need to be revised, what risks remain, and how those risks will be mitigated. We have highlighted some of the cost savings that can be attained through three new operational changes being considered—the use of the Internet 22 The six project teams were: Master Address File (MAF) Error Model, Independent MAF Quality Assessment, LUCA Program Improvement, Workload Management, Matching Process Improvement, and Contact Frame. 23 GAO-04-37. Page 21 GAO-13-53 2020 Census as a self-response option, replacing enumerator collected data with administrative records, and targeting only certain addresses for field verification. However, these innovations also come with risks that need to be managed. Specifically, while the Bureau has identified project-level risks for the 14 projects we reviewed, it has not developed risk mitigation plans for them. Given the number of changes the Bureau would like to make for 2020 it is imperative that risks are sufficiently managed and that mitigation plans be in place. For example, the Bureau needs to manage the risk for accessing federally collected administrative data that it will use for the 2020 Census. While it currently has access to some federally collected data, the Bureau would like access to all federally collected data, which according to the Bureau would require a statutory change. However, such a change raises serious policy concerns including the impact on personal privacy protections, and most likely would be a time consuming process. Therefore, it is imperative that risk mitigation and contingency plans be in place if the Bureau is unable to successfully gain access to all federally collected data. Other areas that need attention include ensuring that all research and testing projects have reliable cost estimates, so as to avoid research and funding shortfalls, that performance metrics are identified to ensure project goals are monitored and met, and that skills are identified to ensure the selection and assignment of appropriate staff to each project. We recommend that the Acting Secretary of Commerce require the Under Recommendations for Secretary for Economic Affairs who oversees the Economics and Executive Action Statistics Administration, as well as the Acting Director of the U.S. Census Bureau, to take the following three actions to improve the Bureau’s Research and Testing for the 2020 Census, and thus better position the Bureau to carry out a cost-effective decennial census: • Develop risk mitigation and contingency plans for all projects to ensure that risks are adequately managed to minimize their effect on the project. • Develop cost estimates for each project. • Ensure documentation for projects are complete, including specifying the performance metrics that will be the basis for determining that each of the projects has completed its work and identifying skills Page 22 GAO-13-53 2020 Census needed to inform the selection and assignment of appropriate staff to each research project. The Acting Secretary of Commerce provided written comments on a draft Agency Comments of this report on October 22, 2012. The comments are reprinted in and Our Evaluation appendix IV. The Department of Commerce agreed with the assessments and recommendations of the report. In addition, the Acting Secretary of Commerce provided a technical comment and suggestion where additional context might be needed, and we revised the report to reflect this comment. The Bureau in its comments stated that our report accurately represented the extensive work that has been completed during the early research and testing phase for the 2020 Census. To address our concerns, the Bureau noted that it is now focusing on managing risks at the project level and will begin obtaining more specific cost estimation details for each project. The Bureau also agrees with our recommendations to ensure that performance metrics and skill sets are identified for all projects and teams. Finally, the Bureau stated that these efforts are either already underway or are planned as a major focus during fiscal year 2013. As arranged with your offices, unless you publicly announce the contents of this report earlier, we plan no further distribution until 30 days after the date of this report. At that time, we will send copies of this report to the Acting Secretary of Commerce, the Under Secretary of Economic Affairs, the Acting Director of the U.S. Census Bureau, and interested congressional committees. The report also is available at no charge on the GAO website at http://www.gao.gov. Page 23 GAO-13-53 2020 Census If you have any questions on matters discussed in this report, please contact me at (202) 512-2757 or by e-mail at firstname.lastname@example.org. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. Key contributors to this report are located in appendix V. Robert Goldenkoff Director Strategic Issues Page 24 GAO-13-53 2020 Census List of Requesters The Honorable Thomas R. Carper Chairman The Honorable Scott Brown Ranking Member Subcommittee on Federal Financial Management, Government Information, Federal Services, and International Security Committee on Homeland Security and Governmental Affairs United States Senate The Honorable Danny K. Davis Ranking Member Subcommittee on Health Care, District of Columbia, Census, and the National Archives Committee on Oversight and Government Reform House of Representatives Page 25 GAO-13-53 2020 Census Appendix I: Objectives, Scope, and Appendix I: Objectives, Scope, and Methodology Methodology This report evaluates the U.S. Census Bureau’s (Bureau) efforts to improve the cost-effectiveness of the enumeration, paying particular attention to the following three key efforts that past Bureau research has shown could result in substantial cost savings: (1) leveraging the Internet to increase self-response; (2) improving how the Bureau builds its address and mapping databases, including a possible move to targeted address canvassing, and use of private-sector geographic data; and (3) using administrative records, such as tax data, to reduce nonresponse follow-up costs. To meet our objectives of identifying what opportunities and risks, if any, the Bureau might need to consider for these efforts going forward, and examining to what extent these three efforts are on track with respect to scheduling, resources, and other performance metrics, we reviewed Bureau documents pertaining to the early planning of the 2020 Census. These generally consisted of the Bureau’s operational plans, strategic framework documents, strategies and planning memorandums, timelines, and benchmarks. Many of these documents were considered draft, but Bureau officials said they were sufficiently developed for purposes of our review. To describe the Bureau’s efforts to use private-sector firms to assist with its address and mapping needs, we spoke to agency officials and reviewed the Bureau’s current mapping and address contracts and its request for information to solicit input from private-sector firms. We interviewed Bureau officials responsible for the early planning of the 2020 Census and inquired about their process for designing their research and testing program, as well as opportunities and risks presented by these three key efforts. We reviewed our prior reports on and other reports on leading practices to identify gaps in the Bureau’s research and testing efforts as it relates to schedule, resources, and performance metrics. We also reviewed reports from and spoke to knowledgeable stakeholders, such as officials at the National Academy of Sciences to determine opportunities and risks related to the Internet, administrative records, and the address and mapping databases. In addition, we spoke with Bureau officials and reviewed our prior work to identify obstacles the Bureau may need to address moving forward; for example, legal barriers to the Bureau’s access to administrative records. To assess the Bureau’s progress in meeting its stated goals for the research and testing program, we evaluated whether each of the 14 research projects the Bureau initiated in 2012 had provided the required deliverables: the 120 day project plan, the project schedule, the project charter, and the project risk register. We also performed a content analysis of Bureau planning documents. For this analysis we examined Page 26 GAO-13-53 2020 Census Appendix I: Objectives, Scope, and Methodology the deliverables for each of the 14 research projects. We developed a data collection instrument to record the presence or absence of 11 pieces of information, or elements, in the deliverables for each project. The 11 elements were identified through a review of the Bureau’s project plan template which outlines information that each project was expected to report by the end of the 120-day initiation phase. A coding dictionary was created to provide a uniform set of coding criteria for each element coded. Each element could be coded as present, partially present, or absent. During the coding process, two analysts independently coded documents for each project using the coding dictionary as a guide. In the next phase, the two coders met to reconcile any discrepancies between their coding decisions and then made a final determination for that element. In cases of disagreement, the engagement methodologist was consulted as a tiebreaker. Following the coding process, the team analyzed the number of present and absent elements to identify patterns for each element across the 14 projects, as well as the completeness of documentation for each project. This engagement did not examine the Bureau’s information technology policies, procedures, and information security in depth because our information technology team is working on two engagements that cover these issues. Page 27 GAO-13-53 2020 Census Appendix II: Fourteen Research Projects Will Appendix II: Fourteen Research Projects Will Inform Design Decisions on Key Efforts (Text for Interactive Figure 3) Inform Design Decisions on Key Efforts (Text for Interactive Figure 3) Table 1: Project Teams Involved in Development of Internet Response Option Project team Objectives in Internet development Optimizing Self-Response Develop requirements for the Internet response option and coordinate the relationship between different response modes (i.e., Internet, paper, telephone) Questionnaire Content, Design, and Mode Determine the number of languages that will be used in the Internet response option Study Workload Management Develop the infrastructure that will support the Internet response option Coding, Editing and Imputation Study Help determine what prompts to include in an Internet response option to improve data quality and respondents’ ease of use Privacy and Confidentiality Study Identify public perception and concerns about responding to the Census via the Internet Contact Frame Provide alternate contact methods, such as e-mail addresses Source: GAO analysis of U.S. Census Bureau information. Table 2: Project Teams Contributing to Decision to Use Administrative Records Project team Objectives in administrative records development Alternative Administrative Records Acquire, process, and analyze administrative records from federal, state, and commercial Composite sources to assess their utility for the 2020 Census Supplementing and Supporting Research and test methods to replace or supplement NRFU data collected in person with Nonresponse Follow-up (NRFU) with administrative data Administrative Records Local Update of Census Addresses Explore the use of administrative records to validate new addresses (LUCA) Program Improvement Coding, Editing and Imputation Study Research the use of administrative records as a source to obtain missing household and address information Enhancing Demographic Analysis Research the use of administrative records to assess the accuracy of Census counts at the national level Improving Quality Control Examine the use of administrative records to supplement or replace field work in quality control operations Privacy and Confidentiality Study Identify public perceptions related to the use of administrative records for enumeration purposes Matching Process Improvement Examine methods to ensure that the Census Bureau can accurately match administrative records to individuals and housing units Contact Frame Acquire and process administrative records for the purpose of using them for alternate contact information Source: GAO analysis of U.S. Census Bureau information. Page 28 GAO-13-53 2020 Census Appendix II: Fourteen Research Projects Will Inform Design Decisions on Key Efforts (Text for Interactive Figure 3) Table 3: Project Teams Contributing to Decision to Implement Targeted Address Canvassing Project team Objectives in targeted address canvassing development Master Address File (MAF) Error Model Develop a statistical model of errors (e.g., missing or duplicate addresses) in the MAF Independent MAF Quality Assessment Use the MAF Error Model to assess the quality of the MAF and determine whether targeted address canvassing would be effective, and, if so, where it should be performed Local Update of Census Addresses (LUCA) Examine what modifications to the Bureau’s existing LUCA program will be required to Program Improvement support targeted address canvassing Source: GAO analysis of U.S. Census Bureau information. Page 29 GAO-13-53 2020 Census Appendix III: Census Mapping and Address Appendix III: Census Mapping and Address Contracts Contracts Contractor Description of services Contract value Acquis Inc. Develop, implement, and support a system which allows the Census Bureau $1,361,780 (Bureau) to update its Master Address File (MAF) with TIGER® (Topologically Integrated Geographic Encoding and Referencing) data, the U.S. mapping data, as the data is received. ASRC Research and Analyze mapping data as it is provided by state and local governments and other $9,998,400 Technology Solutions agencies, and upload changes into TIGER®. Caliper Corporation Customize and maintain software so that over 40,000 state, local, and tribal $2,049,675 governments can update and verify the Bureau’s address and mapping databases. a ERDAS, Inc. Supply digital map files and digital aerial imagery to maintain and update the MAF/ $26,015 TIGER® database. ESRI, Inc. Customize, install, and provide technical support for software to process and edit $1,091,468 files to outline geographic areas, and support mapping applications, including the use of handheld computers to collect and edit data. a Group 1 Software, Inc. Provide software to match addresses from surveys and other agencies to the MAF/ $14,016 TIGER® database. Gunnison Consulting Group, Develop and maintain software which automatically places text on Bureau maps so $2,103,727 Inc. all map features are clearly identified. Oracle America, Inc. Update and consolidate the infrastructure, servers and database, of the MAF/ $9,973,265 TIGER® system. Sabre Systems, Inc. Provide services for the modernization of the Bureau’s MAF/TIGER® system, $62,927,907 including expansion, design, programming, operation and maintenance of the infrastructure supporting the MAF/ TIGER® system. Total $89,546,253 Source: GAO analysis of U.S. Census Bureau information. a Cost of the most recent purchase. Page 30 GAO-13-53 2020 Census Appendix IV: Comments from the Appendix IV: Comments from the Department of Commerce Department of Commerce Page 31 GAO-13-53 2020 Census Appendix IV: Comments from the Department of Commerce Page 32 GAO-13-53 2020 Census Appendix V: GAO Contact and Staff Appendix V: GAO Contact and Staff Acknowledgments Acknowledgments Robert Goldenkoff, (202) 512-2757 or email@example.com GAO Contact Other key contributors to this report include Lisa Pearson, Assistant Staff Director; David Bobruff; Robert Gebhart; Richard Hung; Kirsten B. Acknowledgments Lauber; Andrea Levine; and Timothy Wexler. Page 33 GAO-13-53 2020 Census Related GAO Products Related GAO Products 2020 Census: Sustaining Current Reform Efforts Will Be Key to a More Cost-Effective Enumeration. GAO-12-905T. Washington, D.C.: July 18, 2012. 2020 Census: Additional Steps Are Needed to Build on Early Planning. GAO-12-626. Washington, D.C.: May 17, 2012. Decennial Census: Additional Actions Could Improve the Census Bureau’s Ability to Control Costs for the 2020 Census. GAO-12-80. Washington, D.C.: January 24, 2012. 2010 Census: Preliminary Lessons Learned Highlight the Need for Fundamental Reforms. GAO-11-496T. Washington, D.C.: April 6, 2011. 2010 Census: Data Collection Operations Were Generally Completed as Planned, but Long-standing Challenges Suggest Need for Fundamental Reforms. GAO-11-193. Washington, D.C.: December 14, 2010. GAO Cost Estimating and Assessment Guide: Best Practices for Developing and Managing Capital Program Costs. GAO-09-3SP. Washington D.C.: March 2, 2009. 2010 Census: Census Bureau Should Take Action to Improve the Credibility and Accuracy of Its Cost Estimate for the Decennial Census. GAO-08-554. Washington, D.C.: June 16, 2008. Information Technology: Significant Problems of Critical Automation Program Contribute to Risks Facing 2010 Census. GAO-08-550T. Washington, D.C.: March 5, 2008. 2010 Census: Cost and Design Issues Need to Be Addressed Soon. GAO-04-37. Washington, D.C.: January 15, 2004. High-Risk Series: Quick Reference Guide. GAO/HR-97-2. Washington, D.C.: February 1997. (450946) Page 34 GAO-13-53 2020 Census GAO’s Mission The Government Accountability Office, the audit, evaluation, and investigative arm of Congress, exists to support Congress in meeting its constitutional responsibilities and to help improve the performance and accountability of the federal government for the American people. GAO examines the use of public funds; evaluates federal programs and policies; and provides analyses, recommendations, and other assistance to help Congress make informed oversight, policy, and funding decisions. GAO’s commitment to good government is reflected in its core values of accountability, integrity, and reliability. The fastest and easiest way to obtain copies of GAO documents at no Obtaining Copies of cost is through GAO’s website (http://www.gao.gov). Each weekday GAO Reports and afternoon, GAO posts on its website newly released reports, testimony, and correspondence. To have GAO e-mail you a list of newly posted Testimony products, go to http://www.gao.gov and select “E-mail Updates.” Order by Phone The price of each GAO publication reflects GAO’s actual cost of production and distribution and depends on the number of pages in the publication and whether the publication is printed in color or black and white. Pricing and ordering information is posted on GAO’s website, http://www.gao.gov/ordering.htm. Place orders by calling (202) 512-6000, toll free (866) 801-7077, or TDD (202) 512-2537. Orders may be paid for using American Express, Discover Card, MasterCard, Visa, check, or money order. Call for additional information. Connect with GAO on Facebook, Flickr, Twitter, and YouTube. Connect with GAO Subscribe to our RSS Feeds or E-mail Updates. Listen to our Podcasts. Visit GAO on the web at www.gao.gov. Contact: To Report Fraud, Waste, and Abuse in Website: http://www.gao.gov/fraudnet/fraudnet.htm E-mail: firstname.lastname@example.org Federal Programs Automated answering system: (800) 424-5454 or (202) 512-7470 Katherine Siggerud, Managing Director, email@example.com, (202) 512- Congressional 4400, U.S. Government Accountability Office, 441 G Street NW, Room Relations 7125, Washington, DC 20548 Chuck Young, Managing Director, firstname.lastname@example.org, (202) 512-4800 Public Affairs U.S. Government Accountability Office, 441 G Street NW, Room 7149 Washington, DC 20548 Please Print on Recycled Paper.
2020 Census: Initial Research Milestones Generally Met but Plans Needed to Mitigate Highest Risks
Published by the Government Accountability Office on 2012-11-07.
Below is a raw (and likely hideous) rendition of the original report. (PDF)