oversight

Information Technology: Census Bureau Needs to Implement Key Management Practices

Published by the Government Accountability Office on 2012-09-18.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                 United States Government Accountability Office

GAO              Report to Congressional Requesters




September 2012
                 INFORMATION
                 TECHNOLOGY
                 Census Bureau Needs
                 to Implement Key
                 Management Practices




GAO-12-915
                                               September 2012

                                               INFORMATION TECHNOLOGY
                                               Census Bureau Needs to Implement Key
                                               Management Practices
Highlights of GAO-12-915, a report to
congressional requesters




Why GAO Did This Study                         What GAO Found
The 2010 Decennial Census, at a cost           The U.S. Census Bureau (Census Bureau) has drafted a new investment
of approximately $13 billion, was the          management plan, system development methodology, and requirements
most expensive headcount in our                development and management processes to improve its ability to manage
nation’s history. Prior to the 2010            information technology (IT) investments and system development, but additional
Decennial Census, the Census Bureau            work is needed to ensure these processes are effective and successfully
experienced significant challenges in          implemented across the bureau. GAO and others have identified the importance
managing its information systems               of implementing critical processes within an agency to allow it to select, control,
leading to cost overruns and                   and evaluate its IT investments and effectively manage system development.
performance shortfalls which increased         The bureau has developed a new draft investment management plan which
the cost of the 2010 census by almost          contains policies and guidance for managing IT projects; however, the plan does
$3 billion. Given the bureau’s extensive       not explain when investments with cost or schedule variances should be
use of IT in collecting, analyzing, and        escalated to higher-level boards for review, or when managers should provide
distributing information, GAO was              updated investment information to a planned bureau-wide tracking tool. The
asked to determine to what extent the
                                               bureau has also developed a new system development methodology guide, but
bureau has developed (1) effective
                                               the guide has critical gaps. For example, although there are five development
policies, procedures, and processes for
managing IT investments and system
                                               process models allowed, including the traditional sequential approach and newer
development; and (2) effective                 more iterative approaches, the guide does not explain how to adapt processes
practices for acquiring and maintaining        and related work products for newer iterative approaches. Furthermore, while the
IT human capital skills. To address            bureau has developed new draft requirements development and management
these objectives, GAO identified               processes for system development within individual bureau directorates, it has
leading practices in these areas,              not established a consistent process bureau-wide as GAO recommended in
reviewed bureau policies and                   2005. Lack of a consistent bureau-wide process contributed to significant cost
procedures to determine whether they           and performance issues in the 2010 Decennial Census. Although the bureau
followed these practices, and                  plans to begin operational development for the 2020 Decennial Census in fiscal
interviewed bureau officials.                  year 2015, it has not finalized plans for implementing its new investment
                                               management and system development processes across the bureau. Until the
What GAO Recommends                            bureau takes additional action to finalize and implement consistent, bureau-wide
To strengthen and improve the Census
                                               processes, it faces the risk that IT governance issues that adversely affected the
Bureau’s management of IT, GAO                 2010 Decennial Census will also impact the 2020 Decennial Census.
recommends that the Acting Secretary           The bureau has begun to take steps to improve its IT workforce planning;
of Commerce take eight actions,                however, many key practices consistent with principles for effective workforce
including improvements to guidance for         planning remain to be put in place. In particular, there is no bureau-wide
its planned IT investment process, a           coordination of these workforce planning efforts. Each directorate is responsible
consistent requirements development            for its own IT workforce planning and the bureau has not established any efforts
and management process, an
                                               to coordinate activities among directorates. While the bureau identified mission
implementation plan and time frames
                                               critical IT occupations and began an assessment of select mission critical
for its investment management
process and system development                 competencies in June 2011, it does not plan to perform a bureau-wide IT
methodology, and coordination of IT            competency assessment until the fall of 2012. Until bureau-wide IT workforce
workforce planning efforts. In written         planning processes are established and the bureau develops specific plans to
comments, the Acting Secretary                 conduct an IT skills inventory and gap analysis, the bureau faces the risk that the
concurred with our recommendations             appropriate IT workforce will not be in place to effectively develop and manage
and described steps the bureau was             multimillion dollar investments in information systems and technology that will be
taking to implement them.                      needed for the 2020 Decennial Census.

View GAO-12-915. For more information,
contact David A. Powner at (202) 512-9286 or
pownerd@gao.gov.

                                                                                       United States Government Accountability Office
Contents


Letter                                                                                   1
               Background                                                                2
               Census Bureau Has Begun to Improve Its Investment Management
                 and System Development Processes, but Has Not Finalized
                 Plans for Implementation                                              17
               Census Bureau Does Not Have Bureau-wide Workforce Planning
                 Practices for IT Staff                                                29
               Conclusions                                                             31
               Recommendations for Executive Action                                    32
               Agency Comments                                                         33

Appendix I     Objectives, Scope, and Methodology                                      35



Appendix II    Comments from the Department of Commerce                                38



Appendix III   GAO Contact and Staff Acknowledgments                                   43



Tables
               Table 1: Assessment of Census Bureau’s Enterprise Investment
                        Management Plan against the ITIM Framework                     19
               Table 2: Census Bureau’s Implementation of the Investment
                        Management Plan and System Development Methodology             26
               Table 3: Census Bureau’s IT Workforce Planning Practices                29


Figures
               Figure 1: Organizational Chart of the Census Bureau                       3
               Figure 2: Fundamental Phases of the IT Investment Approach                6
               Figure 3: The Five Information Technology Investment
                        Management Stages of Maturity with Critical Processes            7
               Figure 4: Agile Development Compared with Traditional Waterfall
                        Development                                                    10




               Page i                                GAO-12-915 Census Management Practices
Abbreviations

Census Bureau              U.S. Census Bureau
CMMI                       Capability Maturity Model Integration
IT                         information technology
ITIM                       information technology investment management
SEI                        Software Engineering Institute




This is a work of the U.S. government and is not subject to copyright protection in the
United States. The published product may be reproduced and distributed in its entirety
without further permission from GAO. However, because this work may contain
copyrighted images or other material, permission from the copyright holder may be
necessary if you wish to reproduce this material separately.




Page ii                                         GAO-12-915 Census Management Practices
United States Government Accountability Office
Washington, DC 20548




                                   September 18, 2012

                                   The Honorable Thomas R. Carper
                                   Chairman
                                   The Honorable Scott P. Brown
                                   Ranking Member
                                   Subcommittee on Federal Financial Management,
                                      Government Information, Federal Services,
                                      and International Security
                                   Committee on Homeland Security
                                      and Governmental Affairs
                                   United States Senate

                                   The Honorable Danny K. Davis
                                   Ranking Member
                                   Subcommittee on Health Care, District of Columbia,
                                     Census, and the National Archives
                                   Committee on Oversight and Government Reform
                                   House of Representatives

                                   Providing current and relevant data about the economy and people of the
                                   United States is the mission of the U.S. Census Bureau (Census Bureau)
                                   of the Department of Commerce. The data collected are vital for
                                   determining reapportionment and redistricting of the congressional
                                   districts for the U.S. House of Representatives; realigning the boundaries
                                   of the legislative districts of each state; allocating money for federal
                                   financial assistance; and providing a social, demographic, and economic
                                   profile of the nation’s people to guide policy decisions at each level of
                                   government. To improve the coverage, accuracy, and efficiency of
                                   gathering data from the public, the Census Bureau relies on automation
                                   and information technology (IT).

                                   Given the Census Bureau’s extensive use of IT in collecting, analyzing,
                                   and distributing information, you asked us to determine to what extent the
                                   Census Bureau has developed (1) effective policies, procedures, and
                                   processes for managing IT investments and system development; and (2)
                                   effective practices for acquiring and maintaining IT human capital skills.

                                   To meet these objectives, we reviewed the bureau’s policies and
                                   procedures related to IT investment management, system development,
                                   and human capital management. We also interviewed bureau officials
                                   responsible for providing oversight in these areas to learn how the bureau


                                   Page 1                                 GAO-12-915 Census Management Practices
             is implementing changes to those management processes. For more
             information on our scope and methodology, see appendix I.

             We conducted this performance audit from February 2012 through
             September 2012 in accordance with generally accepted government
             auditing standards. Those standards require that we plan and perform the
             audit to obtain sufficient, appropriate evidence to provide a reasonable
             basis for our findings and conclusions based on our audit objectives. We
             believe that the evidence obtained provides a reasonable basis for our
             findings and conclusions based on our audit objectives.


             The Census Bureau’s mission is to collect and provide comprehensive
Background   data about the nation’s people and economy. Core activities include
             conducting decennial, economic, and government censuses; conducting
             demographic and economic surveys; managing international
             demographic and socioeconomic databases; providing technical advisory
             services to foreign governments; and performing other activities such as
             producing official population estimates and projections.

             The Census Bureau is part of the Department of Commerce and is in the
             department’s Economics and Statistics Administration, led by the Under
             Secretary for Economic Affairs. The Census Bureau is headed by a
             Director and is organized into directorates corresponding to key
             programmatic and administrative functions as depicted in figure 1.




             Page 2                                GAO-12-915 Census Management Practices
Figure 1: Organizational Chart of the Census Bureau




Role of IT at the Census                According to the bureau, while planning, taking, processing, and
Bureau                                  publishing the results of censuses and surveys still requires the work of
                                        thousands of people, advances over the years have been made in the
                                        speed of collection, analysis, and publication of data through the
                                        development of mechanical and electronic tools. For nearly 100 years,
                                        census data were tabulated by clerks who made tally marks or added
                                        columns of figures with a pen or a pencil. As the nation grew and there
                                        were more people, items, and characteristics to count, speedier tabulation
                                        methods had to be invented or the results of one census would not be
                                        processed before it was time for the next one. In 1880, the bureau used a
                                        “tabulating machine”—a wooden box in which a roll of paper was
                                        threaded past an opening where a clerk marked the tallies in various
                                        columns and then added up the marks when the roll was full—that made
                                        tabulating at least twice as fast as the previous manual process.




                                        Page 3                                 GAO-12-915 Census Management Practices
By 1950, mechanical tabulating improved; its speed had increased to
2,000 items per minute. In 1951, the first large-scale electronic computer,
UNIVAC I, was designed and built specifically for the Census Bureau.
This machine was able to tabulate 4,000 items per minute. From 1970 on,
the bureau took advantage of new high-speed composers that converted
the data on computer tape directly to words and numbers on off-set
negative film used in publishing. Beginning in the mid-1980s, some
statistics were made available on diskettes for use in microcomputers and
users began to obtain statistics online. In the later 1980s, the bureau
began testing CD-ROM (compact disk/read-only memory) laser disks as a
medium for releasing data.

The 2000 Census demonstrated probably the biggest leap forward in the
use of technology for collecting and disseminating data. According to the
bureau, its previous response scanning system (which dated to the
1950s) was replaced with optical character recognition technology,
allowing the bureau to design a respondent-friendly (instead of machine-
friendly) questionnaire in which write-in responses could also be captured
electronically. In addition, the bureau’s previous online data system from
the 1990s evolved into online data available through the Census Bureau’s
website. 1 Further technological advances were made in the 2010 census
through the use of handheld computers for certain parts of Census
operations and integration of Global Positioning System information into
Census Bureau maps. Although specific technical decisions for the 2020
Census remain to be made, both ongoing Census operations and the
next decennial census will be highly reliant on the effective use of
information technology.

The 2010 Decennial Census cost $13 billion and was the costliest U.S.
census in history. One reason for the high cost was the increased use of
paper-based processing over what was originally intended due to
performance issues with key IT systems, which increased the cost of the
census by up to $3 billion. The total cost of the census was 56 percent
more than the $8.1 billion 2000 Decennial Census (in constant 2010
dollars). Based on past trends, if the growth rate continues unchecked,
the census could cost approximately $25 billion in 2020. 2 For 2020, the


1
See http://www.census.gov.
2
 GAO, 2010 Census: Preliminary Lessons Learned Highlight the Need for Fundamental
Reforms, GAO-11-496T (Washington, D.C.: Apr. 6, 2011).




Page 4                                      GAO-12-915 Census Management Practices
                          bureau intends to focus on several measures to reduce costs, including
                          better use of IT, but still is planning to spend roughly $12 to $18 billion to
                          conduct the census.

                          To support its IT operations for all of its activities, including those related
                          to decennial censuses, the Census Bureau reported that it plans to spend
                          $384 million on major IT investments in fiscal year 2012. Of this, $130
                          million is to be spent on systems managed by the IT Directorate and $254
                          million is to be spent on systems managed in other directorates. To
                          support these efforts, a bureau official from the Human Resource Division
                          reported that as of July 2012, the bureau employed 1,148 IT staff among
                          its approximately 14,000 employees. 3 IT staff are spread throughout the
                          bureau: the IT Directorate has 256 staff, Economic Programs has 262
                          staff, the Decennial Census has 156 staff, Field Operations has 185 staff,
                          and Demographic Programs has 123 staff.


Overview of Investment    GAO’s Information Technology Investment Management (ITIM)
Management and GAO’s IT   framework can be used by agencies to improve their organizational
Investment Management     processes and measure progress in attaining them. 4 A central tenet of
                          this framework is the select/control/evaluate model. Figure 2 illustrates
Maturity Framework
                          the central components of this model.




                          3
                           The bureau defines IT staff as those staff with the Office of Personnel Management
                          0391, 1550, or 2210 job series.
                          4
                           GAO, Information Technology Investment Management: A Framework for Assessing and
                          Improving Process Maturity (Version 1.1), GAO-04-394G (Washington, D.C.: March
                          2004).




                          Page 5                                         GAO-12-915 Census Management Practices
Figure 2: Fundamental Phases of the IT Investment Approach




During the select phase the organization (1) identifies and analyzes each
project’s risks and returns before committing significant funds to any
project and (2) selects those IT projects that will best support its mission
needs. This process should be repeated each time funds are allocated to
projects, reselecting even ongoing investments as described below.

During the control phase the organization ensures that, as projects
develop and investment expenditures continue, the project continues to
meet mission needs at the expected levels of cost and risk. If the project
is not meeting expectations or if problems have arisen, steps are quickly
taken to address the deficiencies. If mission needs have changed, the
organization is able to adjust its objectives for the project and
appropriately modify expected project outcomes.

During the evaluate phase, actual versus expected results are compared
after a project has been fully implemented. This is done to (1) assess the
project’s impact on mission performance, (2) identify any changes or
modifications to the project that may be needed, and (3) revise the
investment management process based on lessons learned.

The ITIM framework consists of five progressive stages of maturity that
an agency can achieve in its investment management capabilities. The
maturity stages are cumulative; that is, in order to attain a higher stage,
an agency must institutionalize all of the critical processes at the lower
stages, in addition to the higher stage critical processes.



Page 6                                    GAO-12-915 Census Management Practices
The framework’s five maturity stages (see fig. 3) represent steps toward
achieving stable and mature processes for managing IT investments. The
successful attainment of each stage leads to improvement in the
organization’s ability to manage its investments. With the exception of the
first stage, each maturity stage is composed of critical processes that
must be implemented and institutionalized. These critical processes are
further broken down into key practices that describe the types of activities
that an organization should be performing to successfully implement each
critical process. It is not unusual for an organization to be performing key
practices from more than one maturity stage at the same time. However,
our research shows that agency efforts to improve investment
management capabilities should focus on implementing all the lower-
stage practices before addressing the higher-stage practices.

Figure 3: The Five Information Technology Investment Management Stages of
Maturity with Critical Processes




Stage 2 critical processes lay the foundation by establishing successful,
predictable, and repeatable investment control processes at the project
level. Stage 3 is where the agency moves from project-centric processes
to portfolio-based processes and evaluates potential investments
according to how well they support the agency’s missions, strategies, and
goals. Organizations implementing these Stage 2 and 3 practices have in
place selection, control, and evaluation processes that are consistent with




Page 7                                    GAO-12-915 Census Management Practices
the Clinger-Cohen Act of 1996. 5 Stages 4 and 5 require the use of
evaluation techniques to continuously improve both investment processes
and portfolios in order to achieve strategic outcomes.

The ITIM framework can be used to assess the maturity of an agency’s
investment management processes and as a tool for organizational
improvement. The overriding purpose of the framework is to encourage
investment processes that promote business value and mission
performance, reduce risk, and increase accountability and transparency
in the decision-making process. We have used the framework in several
of our evaluations and a number of agencies have adopted it. 6 These
agencies have used ITIM for purposes ranging from self-assessment to
redesign of their IT investment management processes.

Effective management of federal IT investments remains an ongoing
challenge. In December 2010, the White House released a plan to reform
federal IT management that includes greater attention to several of the
management processes described in ITIM. The plan includes efforts to
increase accountability for IT investments, strengthen IT program
management, increase the authority of agency chief information officers,
and strengthen the ability of agency investment review boards to oversee
agency IT investments. 7




5
As relevant here, 40 U.S.C. § 11312.
6
 GAO, Information Technology: DHS Needs to Further Define and Implement Its New
Governance Process, GAO-12-818 (Washington, D.C.: July 25, 2012); Information
Technology: IRS Has a Strong Oversight Process but Needs to Improve How It Continues
Funding Ongoing Investments, GAO-11-587 (Washington, D.C.: July 20, 2011);
Information Technology: HUD Needs to Better Define Commitments and Disclose Risks
for Modernization Projects in Future Expenditure Plans, GAO-11-72 (Washington, D.C.:
Nov. 23, 2010); Information Technology: HUD Needs to Strengthen Its Capacity to
Manage and Modernize Its Environment, GAO-09-675 (Washington, D.C.: July 31, 2009);
Information Technology: FDA Needs to Establish Key Plans and Processes for Guiding
Systems Modernization Efforts, GAO-09-523 (Washington D.C.: June 2, 2009); and
Information Technology: SSA Has Taken Key Steps for Managing Its Investments, but
Needs to Strengthen Oversight and Fully Define Policies and Procedures, GAO-08-1020
(Washington, D.C.: Sept. 12, 2008).
7
 Office of Management and Budget, 25 Point Implementation Plan to Reform Federal
Information Technology Management (Washington, D.C.: Dec. 9, 2010).




Page 8                                       GAO-12-915 Census Management Practices
System Development   The decision to invest in IT often leads to the acquisition or development
Methodologies        of IT systems. To manage that development, organizations often employ
                     a system development methodology. There are several different
                     methodologies that can be used to develop IT systems, which range from
                     the traditional waterfall model to the spiral model and to iterative models
                     such as the Agile model.

                     •   The waterfall model begins with requirements development and
                         continues sequentially through other phases—design, build, and
                         test—using the output of one phase as the input to the next to develop
                         a finished product at the end. This model allows the status of a
                         development project to be easily identified and tracked based on the
                         current phase of the project.

                     •   The spiral model uses a risk-based approach to incrementally build a
                         system by cycling through the four development phases. Using this
                         model, each spiral, or incremental cycle, typically starts by
                         determining the development objectives and scope for the increment.
                         Next, alternative solutions are evaluated and risk management
                         techniques are employed to identify and reduce risks. Then, a product
                         for the increment (such as a prototype) is developed. Finally, the
                         product is evaluated to determine whether the increment’s initial
                         objectives have been met.

                     •   The Agile model focuses on short-duration, small-scope
                         development phases that produce segments of a functional product.
                         This model operates with similar phases to the traditional waterfall
                         model—requirements, design, build, and test—but uses a shorter
                         development cycle to achieve multiple iterations in similar time
                         frames. Recently, several agencies have tried Agile, as it calls for
                         producing software in small, short increments. Shorter, more
                         incremental approaches to IT development have been identified as
                         having the potential to improve the way in which the federal
                         government develops and implements IT. In a recent report, we
                         identified 32 practices and approaches as effective for applying Agile
                         software development methods to IT projects. 8 Officials who have
                         used Agile methods on federal projects generally agreed that these
                         practices are effective. In addition, the Office of Management and



                     8
                      GAO, Software Development: Effective Practices and Federal Challenges in Applying
                     Agile Methods, GAO-12-681 (Washington, D.C.: July 27, 2012).




                     Page 9                                       GAO-12-915 Census Management Practices
                                           Budget recently issued guidance that advocates the use of shorter
                                           delivery time frames, 9 an approach consistent with Agile. See figure 4
                                           for a comparison of the Agile and waterfall development methods.

Figure 4: Agile Development Compared with Traditional Waterfall Development




                                       According to the Software Engineering Institute (SEI) Capability Maturity
                                       Model Integration (CMMI) for Development, a noted reference for best
                                       practices in system life-cycle development processes, when establishing
                                       an organizational process, the organization should establish and maintain
                                       criteria and guidelines that can be tailored for a particular project based




                                       9
                                        Office of Management and Budget, 25 Point Implementation Plan to Reform Federal
                                       Information Technology Management (Washington, D.C.: Dec. 9, 2010) and Immediate
                                       Review of Financial Systems IT Projects, M-10-26 (Washington, D.C.: June 28, 2010).




                                       Page 10                                       GAO-12-915 Census Management Practices
                         on the development model chosen, and other issues, such as customer
                         needs, cost, schedule, and technical difficulty. 10

                         Also according to SEI, within a given system development model, a
                         number of specific development activities should be addressed. These
                         include requirements development and requirements management.
                         Requirements development includes activities such as identifying
                         desirable functionality and quality attributes through an analysis of
                         scenarios with relevant stakeholders; analyzing and qualifying
                         functionality required by end users; and partitioning requirements into
                         groups based on established criteria such as similar functionality to
                         facilitate and focus the requirements analysis. Requirements
                         management provides management of the business and system
                         requirements, and identification of inconsistencies among requirements
                         and the project’s plans and work products.


GAO and Office of        A strategic approach to human capital management includes viewing
Personnel Management     personnel as assets whose value can be enhanced by investing in them.
Guidance Help Federal    Such an approach enables an organization to use their people effectively
                         and to determine how well they integrate human capital considerations
Agencies Strategically   into daily decision making and planning for mission results. It also helps
Manage Human Capital     organizations to remain aware of and be prepared for current and future
                         needs as an organization, and ensure that personnel have the
                         knowledge, skills, and abilities needed to pursue the mission of the
                         organization.

                         In 2003, we identified a set of key practices for effective strategic human
                         capital management, including workforce planning. 11 These practices are
                         based on our reports and testimonies, reviews of studies by leading
                         workforce planning organizations, and interviews with officials from the
                         Office of Personnel Management and other federal agencies. Strategic
                         workforce planning addresses two critical needs: (1) aligning an
                         organization’s human capital program with its current and emerging
                         mission and programmatic goals and (2) developing long-term strategies



                         10
                           Software Engineering Institute, CMMI for Development, Version 1.3, CMU/SEI-2010-TR-
                         033 (Pittsburgh, Pa: November 2010).
                         11
                          GAO, Human Capital: Key Principles for Effective Strategic Workforce Planning,
                         GAO-04-39 (Washington, D.C.: Dec. 11, 2003).




                         Page 11                                       GAO-12-915 Census Management Practices
for acquiring, developing, and retaining staff to achieve programmatic
goals. While agency approaches to workforce planning will vary, we and
the Office of Personnel Management have identified key practices in
effective strategic workforce planning, 12 six of which are:

•    Align workforce planning with strategic planning and budget
     formulation.

•    Involve top management, employees, and other stakeholders in
     developing, communicating, and implementing the strategic workforce
     plan.

•    Identify the critical skills 13 and competencies 14 that will be needed to
     achieve current and future programmatic results.

•    Develop strategies that are tailored to address gaps in number,
     deployment, and alignment of human capital approaches for enabling
     and sustaining the contributions of all critical skills and competencies.

•    Build the capability needed to address administrative, educational,
     and other requirements important to support workforce planning
     strategies.

•    Monitor and evaluate the agency’s progress toward its human
     capital goals and the contribution that human capital results have
     made toward achieving programmatic results.




12
  GAO-04-39; GAO, Workforce Planning: Interior, EPA, and the Forest Service Should
Strengthen Linkages to Their Strategic Plans and Improve Evaluation, GAO-10-413
(Washington, D.C.: Mar. 31, 2010); and Office of Personnel Management, Human Capital
Assessment and Accountability Framework— Systems, Standards, and Metrics
(http://www.opm.gov/hcaaf_resource_center/).
13
  The Chief Information Officers Council defines a skill as a granular or discrete ability
related to a specific product or technology.
14
  The Office of Personnel Management defines a competency as a measurable pattern of
knowledge, skills, abilities, behaviors, and other characteristics that an individual needs to
successfully perform a work role or occupational function.




Page 12                                           GAO-12-915 Census Management Practices
Activities for              We have previously reported on the challenges associated with
Implementing New            implementing a new IT governance framework, such as the Census
Organizational Governance   Bureau is trying to do. 15 Implementing a new governance framework and
                            system development methodology are challenging tasks that can be
Processes
                            aided by having robust implementation plans. Such a plan is instrumental
                            in helping agencies coordinate and guide improvement efforts. As we
                            have previously reported, several steps are important for successfully
                            implementing new organizational governance processes related to
                            investment management and system development. 16 For example,
                            organizations should:

                            •    Have a commitment from agency leadership to putting the process in
                                 place. Buy-in of key stakeholders should be obtained to ensure that
                                 their perspectives are considered and to facilitate adoption. This
                                 includes obtaining top management support and creating forums for
                                 involving business representatives.

                            •    Select an implementation team and develop a detailed
                                 implementation plan that lays out a roadmap for implementing the
                                 new process. An effective implementation team should include key
                                 stakeholders from both business and IT components. An
                                 implementation plan should build on existing strengths and
                                 weaknesses; specify measurable goals, objectives, and milestones;
                                 specify needed resources; assign responsibility and accountability for
                                 accomplishing tasks; and be approved by senior-level management.
                                 On the other end, measures to assess progress in meeting the
                                 objectives of the implementation efforts should be developed and
                                 should include lessons learned.

                            •    Perform pilot testing of the new process to evaluate the process and
                                 identify potential problems. Pilot testing is an effective—and usually
                                 necessary—tool for moving the agency successfully to full
                                 implementation. Pilot testing allows the agency to (1) evaluate the
                                 soundness of the proposed process in actual practice, (2) identify and



                            15
                             GAO, Information Technology: Treasury Needs to Strengthen Its Investment Board
                            Operations and Oversight, GAO-07-865 (Washington, D.C.: July 23, 2007).
                            16
                              GAO, Information Technology: DHS Needs to Further Define and Implement Its New
                            Governance Processes, GAO-12-818 (Washington, D.C.: July 25, 2012); and Business
                            Process Reengineering Assessment Guide, Version 3, GAO/AIMD-10.1.15 (Washington,
                            D.C.: May 1997).




                            Page 13                                     GAO-12-915 Census Management Practices
                                 correct problems with the new design, and (3) refine performance
                                 measures. Also, successful pilot testing will help strengthen support
                                 for full-scale implementation from employees, outside stakeholders,
                                 Congress, and the public, and help secure the funding needed for a
                                 smooth rollout.

                             •   Develop a formal evaluation process to determine the effectiveness of
                                 the new process in meeting the agency’s goals. The team should
                                 develop a formal evaluation process to determine the efficiency and
                                 effectiveness of the new process, both during pilot tests and full
                                 implementation, in meeting the agency’s performance goals. The
                                 process should also allow the agency to pinpoint trouble spots, so that
                                 corrective actions can be developed quickly.


Prior GAO and Inspector      Our prior work has identified the importance of having sound
General Reports Identified   management processes in place to help the bureau as it manages
Management Challenges        multimillion dollar investments needed for its decennial census. For the
                             last decennial, we issued multiple reports and testimonies from 2005
                             through 2010 on weaknesses in the Census Bureau’s management and
                             testing of key 2010 Decennial Census IT systems. For example, in June
                             2005, we found that while the Census Bureau had initiated key practices
                             in areas such as providing investment oversight, project planning,
                             requirements management, and risk management, they were not fully and
                             consistently performed across the bureau. 17 Accordingly, we made
                             recommendations to the Census Bureau to develop procedures to ensure
                             consistent investment management and decision-making practices and to
                             institutionalize a process improvement initiative, such as the CMMI
                             framework, to strengthen bureau-wide system development and
                             management processes. We noted that unless these recommendations
                             were implemented, the bureau would face increased risk that cost
                             overruns, schedule slippages, and performance shortfalls would occur
                             and it would not be able to effectively manage its multimillion dollar
                             investments in IT.

                             As development of the IT systems progressed, these problems were
                             realized. In 2007, we reviewed the status of four key IT acquisitions


                             17
                               GAO, Information Technology Management: Census Bureau Has Implemented Many
                             Key Practices, but Additional Actions Are Needed, GAO-05-661 (Washington, D.C: June
                             16, 2005).




                             Page 14                                      GAO-12-915 Census Management Practices
needed for the 2010 Decennial Census. 18 The bureau was still in the
process of addressing our 2005 recommendations and our review found
that there were increases in cost estimates and projected cost overruns of
at least $51 million for the Field Data Collection Automation program 19
due to changes in requirements. There were also schedule slippages with
two other projects. Furthermore, these four projects were not consistently
implementing key risk management practices. We concluded that unless
the bureau addressed our recommendations to strengthen system testing
and risk management activities, there would be an increased probability
that decennial systems would not be delivered on schedule and within
budget or perform as expected.

Subsequently, in March 2008, we added the 2010 Decennial Census to
our list of high-risk programs in part because of long-standing
weaknesses in the Census Bureau’s IT acquisition and contract
management function, difficulties in developing reliable life-cycle cost
estimates, and key operations that were not tested under operational
conditions. 20 We also testified on significant risks facing the 2010 census.
In particular, we testified in March 2008 that the Field Data Collection
Automation program was experiencing significant problems, including
schedule delays and cost increases from changes in requirements, which
required additional work and staffing. 21 In April 2008, the Census Bureau
dropped the use of handheld devices developed as part of this program
for nonresponse follow-up and reverted to a paper-based operation,
requiring the development of a Paper-Based Operations Control System
to manage the operation. Dropping the use of handhelds for nonresponse
follow-up and replacing it with the paper-based system increased the cost
of the 2010 Decennial Census by up to $3 billion.



18
  GAO, Information Technology: Census Bureau Needs to Improve Its Risk Management
of Decennial Systems, GAO-08-79 (Washington, D.C.: Oct. 5, 2007).
19
  The Field Data Collection Automation program, originally estimated to cost $596 million,
was intended to use handheld mobile devices to support field data collection for address
canvassing to verify addresses and for nonresponse follow-up, or following up in person
with respondents who failed to return the mail questionnaire. In April 2008, the Census
Bureau decided not to use the handheld devices for nonresponse follow-up, but did
continue to use the devices for other decennial census operations.
20
 GAO, High-Risk Series, An Update, GAO-09-271 (Washington, D.C.: January 2009).
21
 GAO, Information Technology: Significant Problem of Critical Automation Program
Contribute to Risks Facing 2010 Census, GAO-08-550T (Washington, D.C.: Mar. 5, 2008).




Page 15                                         GAO-12-915 Census Management Practices
In March 2009, we reported that the bureau continued to face a number
of problems related to testing of key IT systems, such as the Paper-
Based Operations Control Systems, that included weaknesses in test
plans and schedules, and a lack of executive-level oversight and
guidance. 22 We recommended that the bureau complete key system-
testing activities and improve testing oversight and guidance or the
Census Bureau would face the risk that systems were not thoroughly
tested and or would perform as planned. Later that year, we reported in
November 2009 that the bureau had not finalized detailed requirements
for releases of the Paper-Based Operations Control System, which put
the system at risk for cost increases, schedule delays, or performance
shortfalls. 23 Although the bureau worked aggressively to improve the
Paper-Based Operations Control System, we reported in December 2010
that the system had experienced significant issues when it was put in
operation. 24 The bureau attributed these issues, in part, due to a
compressed development and testing schedule, as well as inadequate
performance and interface testing.

At a cost of about $13 billion, 2010 was the costliest decennial census in
history. While the 2010 census was removed from GAO’s high-risk list in
February 2011, we reported in April 2011 that the bureau needed to
continue to improve key practices for managing IT and strengthen its
ability to develop reliable life-cycle cost estimates. 25

More recently, in May 2012, we reported on the Census Bureau’s early
planning efforts for the 2020 census. 26 We noted that the bureau’s early
planning and preparation efforts were consistent with most leading
practices in each of the three management areas we reviewed:


22
  GAO, Information Technology: Census Bureau Testing of 2010 Decennial Systems Can
Be Strengthened, GAO-09-262 (Washington, D.C.: Mar. 5, 2009).
23
  GAO, 2010 Census: Census Has Made Progress on Schedule and Operational Control
Tools, but Needs to Prioritize Remaining System Requirements, GAO-10-59 (Washington,
D.C.: Nov. 13, 2009).
24
  GAO, 2010 Census: Data Collection Operations Were Generally Completed As
Planned, but Long-Standing Challenges Suggest Need for Fundamental Reform,
GAO-11-193 (Washington, D.C.: Dec. 14, 2010).
25
 GAO-11-496T.
26
 GAO, 2020 Census: Additional Steps Are Needed to Build on Early Planning,
GAO-12-626 (Washington, D.C.: May 17, 2012).




Page 16                                      GAO-12-915 Census Management Practices
                       organizational transformation, long-term project planning, and strategic
                       workforce planning, but we did identify opportunities for improvement. 27

                       In addition, the Department of Commerce Office of the Inspector General
                       recently identified several management challenges the Census Bureau
                       faces as it prepares for the 2020 Decennial Census. 28 In June 2011, the
                       Office of the Inspector General noted that the bureau needed to
                       implement improved project planning and management techniques early
                       in the decade to address the weaknesses in project management, cost
                       estimation, and risk management. Officials from the Census Bureau
                       stated that those recommendations were consistent with their current
                       plans. 29


                       The Census Bureau has drafted a new investment management plan,
Census Bureau Has      system development methodology, and requirements development and
Begun to Improve Its   management processes to improve its ability to manage IT investments
                       and system development and to address our prior recommendations.
Investment             While the bureau’s investment plan adapts key practices outlined in the
Management and         ITIM framework, additional work is needed to develop guidelines such as
System Development     when investment review boards should escalate investments with cost or
                       schedule variances and when managers should provide updated
Processes, but Has     investment information to the enterprise portfolio management tool. In
Not Finalized Plans    addition, while the system development methodology lays out a
                       foundation for development activities at the Census Bureau, it lacks
for Implementation     guidance on tailoring the methodology to development models other than
                       the traditional waterfall model. Furthermore, while the bureau has
                       developed new draft requirements development and management
                       processes for system development, it has not established a consistent
                       process bureau-wide as we recommended in 2005. Finally, the bureau
                       has not finalized plans for implementing these processes across the
                       bureau or for ensuring they are in place for managing investments and



                       27
                          This report did not include the bureau’s IT investment management practices as part of
                       its scope.
                       28
                         U.S. Department of Commerce Office of Inspector General, Top Management
                       Challenges Face the Department of Commerce, OIG-11-015 (Washington, D.C.: Dec. 20,
                       2010).
                       29
                         U.S. Department of Commerce, Office of Inspector General, Census Bureau - Census
                       2010: Final Report to Congress, OIG-11-030-I (Washington, D.C.: June 27, 2011).




                       Page 17                                         GAO-12-915 Census Management Practices
                           developing systems necessary for the 2020 Decennial Census to help
                           avoid a repeat of the cost and performance issues that occurred during
                           the 2010 Decennial Census.


Census Bureau Has Not      The bureau has developed a new investment management plan, called
Finalized Its Investment   the Enterprise Investment Management Plan, which is to apply to all
Management Plan            investments, including IT investments. The draft plan outlines a portfolio
                           management process that is to operate in two interdependent cycles, one
                           to align current and planned investments to ensure the right investments
                           are selected to support the bureau’s mission, and one to monitor the
                           development, deployment, and operation of approved investments in
                           order to ensure new projects are developed as planned and ongoing
                           systems are regularly evaluated for their impact on and relevancy to the
                           bureau’s mission.

                           The plan outlines key investment management roles and responsibilities
                           for various groups within the bureau. The groups include the

                           •   Operating Committee, which is comprised of the bureau’s senior
                               executive team, including the Deputy Director, who chairs the
                               committee, and associate directors for all nine of the bureau’s
                               directorates, one of whom is also the Chief Information Officer of the
                               bureau. The Operating Committee has ultimate responsibility for
                               directing the bureau’s resource allocations and overseeing program
                               performance. It also has overall responsibility for all IT investments
                               costing more than $10 million that are high priority and medium or
                               high priority investments that cost more than $50 million.

                           •   Office of Risk Management and Program Evaluation, which is to
                               manage the bureau’s enterprise investment portfolio, review business
                               cases for major investments and all other projects and portfolio
                               investments within the bureau, regardless of cost or priority, and track
                               project, program, and portfolio performance information. The office
                               began operating in January 2011.

                           •   Directorate, division, and program-level investment review boards,
                               which are to assess, review, and prioritize all existing and proposed
                               investments at the appropriate directorate, division, or program level,
                               and escalate investment issues to a higher-level board when required.
                               Directorate-level review boards have overall responsibility for
                               investments that are high priority and cost less than $10 million,
                               medium-priority investments between $10 million and $50 million, and



                           Page 18                                 GAO-12-915 Census Management Practices
                                                 low-priority investments costing more than $50 million. Division and
                                                 program-level boards have responsibility for medium-priority
                                                 investments of less than $10 million and low-priority investments that
                                                 cost less than $50 million.

                                             As shown in the following table, the bureau’s new investment
                                             management plan is consistent with key practices outlined in the ITIM
                                             framework for Stage 2, including having documented policies and
                                             procedures in place for identifying IT projects that support business needs
                                             and selecting investments for funding. However, other policies within the
                                             plan only partially address the ITIM framework. In particular, the plan
                                             does not include guidelines for the membership of investment review
                                             boards, the frequency of board meetings, and the thresholds for
                                             escalating issues to higher-level boards, as these decisions are left up to
                                             individual directorates to determine. Table 1 summarizes our assessment
                                             of the policies contained in the bureau’s draft plan against relevant
                                             practices in Stage 2 of the ITIM framework.

Table 1: Assessment of Census Bureau’s Enterprise Investment Management Plan against the ITIM Framework

Practice                                               Included in
category       Specific practice                       policy?       Summary of assessment
Instituting    A process for creating and defining     Partially     An investment board, the Operating Committee, has been
investment     the membership, guiding policies,                     established and responsibilities and authorities have been
board          operations, roles, responsibilities,                  outlined in the draft plan. In addition, the plan calls for the
operations     and authorities of one or more                        creation of investment review boards at the directorate level as
               investment review boards is                           well as at the division and program levels. However, the
               established.                                          membership of these boards and the frequency of meetings are
                                                                     not specified in the plan. According to the Chief of the Office of
                                                                     Risk Management and Program Evaluation, these decisions will
                                                                     be left up to the individual directorates to decide. However, by
                                                                     doing so, the bureau may not be able to ensure there is
                                                                     consistency in individual review board operations when
                                                                     decisions are made regarding investments.
Meeting        Documented policies and                Yes            The bureau’s draft plan outlines the process for identifying IT
business       procedures for identifying IT projects                projects that support business needs and mission. The bureau’s
needs          or systems that support the                           business mission with stated goals and objectives is
               organization’s ongoing and future                     documented in the U.S. Census Bureau Strategic Plan FY2007-
               business needs are established. A                     2012.
               business mission with stated goals
               and objectives is also documented.
Selecting an   Policies and procedures for            Yes            The draft plan describes processes for the concurrent review of
investment     selecting new IT investments as well                  proposals by executives, the use of predefined criteria to
               as reselecting ongoing investments                    analyze these proposals, and the process by which executives
               have been developed, which also                       choose to fund some proposals and not others.
               include integrating funding as part of
               the process.




                                             Page 19                                        GAO-12-915 Census Management Practices
Practice                                                Included in
category      Specific practice                         policy?                 Summary of assessment
Providing     Policies and procedures are               Partially               The draft plan describes the Operating Committee, which
investment    documented for management                                         serves as the manager of the bureau’s enterprise portfolio and
oversight     oversight of IT projects and                                      has ultimate responsibility for directing the Census Bureau’s
              systems.                                                          resource allocation and overseeing program performance. The
                                                                                Operating Committee currently reviews investments on a yearly
                                                                                basis. However, the management of most program components
                                                                                in the portfolio is the responsibility of the individual program
                                                                                owners. In addition, while the plan indicates that issues with
                                                                                investments will be escalated from the directorate’s investment
                                                                                review board to the Operating Committee when there are cost,
                                                                                risk, schedule, or impact issues, no consistent criteria or
                                                                                thresholds have been established for escalation. The Chief of
                                                                                the Office of Risk Management and Program Evaluation stated
                                                                                that individual boards will be responsible for determining these
                                                                                thresholds. However, by doing so, the bureau may not be able
                                                                                to ensure that issues with specific investments are consistently
                                                                                reported to higher-level boards.
Capturing     Policies and procedures for            Partially                  The plan includes information on the use of an enterprise
investment    identifying and collecting information                            portfolio management tool, which will consolidate investment
information   about IT projects to support the                                  information from all directorates into one centralized database.
              investment management process                                     According to the Chief of the Office of Risk Management and
              are documented.                                                   Program Evaluation, individual directorates will be able to use
                                                                                other applications to manage their projects, but they will be
                                                                                required to extract information from their applications and
                                                                                provide it to the enterprise portfolio management tool central
                                                                                database. Census Bureau officials said that financial
                                                                                information in the enterprise portfolio management tool will be
                                                                                updated on a monthly basis, but the bureau has not set a time
                                                                                frame for when project managers should provide periodic
                                                                                updates to the tool, or provided dates for when this would be
                                                                                done.
                                            Source: GAO analysis of Census Bureau documentation and interviews with bureau officials.


                                            Note: Full implementation of a practice in the ITIM framework requires that a practice be stated in a
                                            policy and also be in use. This table only assesses whether the practice is stated in the bureau’s
                                            policy.

                                            In addition to lacking key guidelines for investment review board
                                            operations and the enterprise portfolio management tool, the plan is still a
                                            draft. The Chief of the Office of Risk Management and Program
                                            Evaluation stated that the plan would be finalized in late September 2012.
                                            According to the official, it has taken the bureau time to finalize the plan
                                            due to its review process with stakeholders, which included holding
                                            desktop exercises with key staff in various directorates to walk through
                                            the new governance processes and obtain feedback, and the naming of a
                                            new Deputy Director for the bureau.

                                            While development of a draft investment management plan is a useful
                                            first step toward more rigorous investment management, until the Office


                                            Page 20                                                              GAO-12-915 Census Management Practices
                            of Risk Management and Program Evaluation establishes guidelines for
                            the frequency and membership of the investment review boards,
                            thresholds for escalating cost or schedule variance issues, and time
                            frames for project managers to make periodic updates of investment
                            information in its enterprise portfolio management tool, the bureau is likely
                            to face inconsistent application of its investment management plan.


Census Bureau Has           In February 2012, the bureau’s IT and 2020 Census Directorates, with the
Created a New System        assistance of a contractor, developed a new system development life-
Development                 cycle methodology to improve the bureau’s ability to develop IT systems
                            and to address our prior 2005 recommendation to strengthen bureau-
Methodology, but It Does    wide system development and management processes. 30 The main
Not Address Modifications   elements of the bureau’s new methodology include:
Needed for Newer Models
of Software Development     •     Ten defined life-cycle phases, including initiation, concept
                                  development, planning, requirements analysis, design, development,
                                  integration and test, deployment, operations and maintenance, and
                                  disposition, along with corresponding activities and work products that
                                  must be completed.

                            •     Five development process models, including waterfall, prototyping,
                                  incremental, iterative, 31 and spiral, that project managers can use
                                  when developing new systems or modifying or adding functionality to
                                  existing systems.

                            •     A tool that helps project managers choose the appropriate
                                  development process model.

                            •     An appendix that helps project managers identify the work products to
                                  complete for each phase of the life-cycle based on the project’s life-
                                  cycle cost and priority.

                            Although the methodology lays out a foundation for system development
                            activities at the bureau, it has critical gaps. In particular, SEI’s CMMI for
                            Development, a noted reference for best practices in system life-cycle
                            development processes, recommends that an organization establish and



                            30
                                GAO-05-661.
                            31
                                One example of an iterative development process model is Agile.




                            Page 21                                         GAO-12-915 Census Management Practices
maintain criteria and guidelines that can be tailored to suit changing
situations. 32 However, the bureau’s methodology is based on a waterfall
development model and work products identified in the guide are only tied
to categories of projects assigned according to each project’s life-cycle
cost and priority. The guide does not have guidance on how to adapt the
process and related work products to alternate development models.
Specifically, since other life-cycle models do not follow the waterfall model
in terms of phases, types of activities, and work products for each phase,
the methodology should explain how it can be adapted to alternate
software development models, including identifying mandatory work
products for other non-waterfall development models and phases. For
example, one project manager for a pilot project of the new guide that is
using an iterative process development model confirmed that he had
difficulty in using the guide and had to deviate from it as he was
developing work products.

The current Chief Information Officer stated that the bureau’s process
does not currently include alternate development models because the
methodology is in its first iteration and he wanted to evaluate the new
process with pilots before making changes or additions. Census Bureau
officials also acknowledged that the system development life-cycle
methodology will not be as useful with other life-cycle models, such as an
iterative model, and that they intend to develop additional guidance for
non-waterfall models. However, they could not provide dates for when
development of this guidance would be started or finalized.

Both private and public-sector organizations are making increasing use of
non-waterfall development models as a means of reducing development
times and reducing risk for software projects. 33 Until the bureau specifies
how to adapt the new system development life-cycle development
methodology to non-waterfall models, the methodology will be of limited
use for managing system development and acquisition efforts.




32
  Software Engineering Institute, CMMI for Development, Version 1.3, CMU/SEI-2010-TR-
033 (Pittsburgh, Pa: November 2010).
33
  Alternative models allow for developing software in increments or stages, adding
additional capabilities until a full system is developed, which can provide a better return on
investment. In addition, alternative models have the potential to address changes in
requirements more readily than a waterfall model. See GAO-12-681.




Page 22                                           GAO-12-915 Census Management Practices
Both IT and 2020 Census    SEI states that a disciplined process for developing and managing
Directorates Have          requirements can help reduce the risks of developing or acquiring a
Independently Drafted      system. The practices underlying requirements development and
                           management include eliciting, documenting, verifying and validating, and
New Requirements           managing requirements through a system’s life cycle. This set of activities
Development and            translates customer needs from statements of high-level business
Management Processes,      requirements into validated, testable system requirements. A well-defined
but Bureau-wide Approach   and managed requirements baseline can, in addition, improve
Is Needed                  understanding among stakeholders and increase stakeholder buy-in and
                           acceptance of the resulting system.

                           Both the IT Directorate and the 2020 Census Directorate have drafted
                           new requirements development and management processes, though only
                           the IT Directorate’s guidance has been finalized. In particular, the IT
                           Directorate has developed a new process, the Application Services
                           Division’s Requirements Elicitation, Analysis, and Documentation
                           Process, for working with customers to develop requirements for projects
                           during the requirements phase. This includes activities for creating a
                           requirements work plan, identifying high-level and detailed project
                           requirements, as well as assessing the feasibility and managing the risks
                           associated with these requirements. The 2020 Census Directorate has
                           developed a draft Requirements Engineering Management Plan that
                           establishes processes for developing both enterprise-level mission
                           requirements for the 2020 Decennial Census and specific project-level
                           business, capability, and solution requirements. The process includes
                           four stages for developing and managing these sets of requirements
                           including discovery, analysis, agreement, and solution acceptance, and
                           outlines roles and responsibilities for stakeholders.

                           While both the IT and 2020 Census Directorates have established new
                           requirements development and management processes, which are to be
                           used in the requirements analysis phase of the bureau’s new system
                           development methodology, the bureau has not established a consistent
                           process bureau-wide as we have previously recommended. In 2005 we
                           found that individual project teams within the bureau had not consistently
                           implemented key practices for requirements management and we
                           recommended that a consistent approach be established bureau-wide.
                           These weaknesses in the bureau’s processes for requirements
                           management were not sufficiently addressed and we reported in 2007,
                           2008, and 2009 that these issues contributed to increases in life-cycle




                           Page 23                                 GAO-12-915 Census Management Practices
cost estimates and cost overruns of hundreds of millions of dollars for key
investments necessary for the 2010 Decennial Census. 34 Furthermore,
the bureau had developed a handheld device for the 2010 census that did
not operate as intended because of the lack of a robust requirements
process. Instead, the bureau had to rely on a paper-based system to
replace the handheld devices for key aspects of the census, which
increased the cost of the 2010 census by up to $3 billion.

Nevertheless, both the Chief Information Officer, who heads the IT
Directorate and the Associate Director of the 2020 Census Directorate, as
well as the Chief of the Office of Risk Management and Program
Evaluation, stated that there are no plans to standardize a requirements
development and management process across the bureau. Currently, for
future investments, bureau officials will decide which directorate
requirements process to use, depending on which directorate has
responsibility for developing the system. For example, the 2020 Census
and IT Directorates are currently working together to develop a new
electronic document management system 35 intended for the 2020
Decennial Census. For this project, only the IT Directorate’s requirements
process is being used because the IT Directorate has overall
responsibility for developing the system.

While the 2020 Census and IT Directorate’s processes both contain
useful elements of a requirements management process, neither process
is sufficient by itself. Specifically, the IT Directorate has guidance for
eliciting, developing, and documenting requirements for projects, but
although the 2020 Directorate’s process mentions project requirements,
the guidance does not provide clear steps for developing these
requirements. In contrast, the 2020 Census Directorate’s guidance
provides clear steps for developing strategic mission and business
requirements, but the IT Directorate’s guidance does not. Moreover, each
of these guidance documents lacks information on how these
requirements processes will be integrated into the bureau’s new system
development life-cycle methodology. Bureau officials stated that they
hope to integrate processes from the two directorates sometime in the



34
 GAO-08-79, GAO-08-550T, and GAO-10-59.
35
  The Electronic Document Management System will be used to streamline the American
Community Service Office and 2020 Census program processes for developing,
reviewing, and approving documents at the program and project levels.




Page 24                                     GAO-12-915 Census Management Practices
                          future in preparation for the 2020 Decennial Census, but did not specify
                          when this would occur.

                          By utilizing overlapping requirements development and management
                          processes without a specified time frame for when these processes will
                          be integrated, the bureau is increasing the risk that IT investments,
                          particularly those intended for the 2020 Decennial Census, will face cost
                          overruns, schedule slippages, and performance shortfalls. Until the
                          bureau establishes and implements a consistent requirements
                          development and management process across the bureau that has clear
                          guidance for developing requirements at the strategic mission, business,
                          and project levels and is integrated with its new system development
                          methodology, it will not have assurance that requirements for IT systems
                          intended for the 2020 Decennial Census will be effectively developed or
                          managed.


Census Bureau Has Not     While the bureau has drafted a new investment management plan, and
Finalized Plans for       system development methodology, including requirements development
Implementing Investment   and management processes, key activities for effectively implementing
                          these processes across the bureau remain to be undertaken. In
Management and System     particular, while the bureau’s leadership has made a commitment to
Development Processes     putting the new investment plan in place across the bureau, no specific
across the Bureau         plans have been made to implement the system methodology bureau-
                          wide. In addition, detailed implementation plans for putting these
                          processes in place, including having sufficient pilot testing and formal
                          evaluations to determine the effectiveness of the new processes remain
                          to be developed. Table 2 shows our assessment of the bureau’s
                          implementation efforts based on our prior work on implementing new
                          processes within an organization. 36




                          36
                           GAO-12-818 and GAO/AIMD-10.1.15.




                          Page 25                                GAO-12-915 Census Management Practices
Table 2: Census Bureau’s Implementation of the Investment Management Plan and System Development Methodology

Key activity                           Assessment    Summary of assessment
Agency leadership makes                Partially     Agency leadership has played a key role in establishing the new investment
commitment to putting the process in   implemented   management structure and the Office of Risk Management and Program
place                                                Evaluation. The Operating Committee, composed of senior executive
                                                     leadership in the bureau, has been established and is chaired by the Deputy
                                                     Director of the bureau. It has begun meeting on a yearly basis to review IT
                                                     investments. However, no time frames for implementing the investment
                                                     management plan across the bureau have been established.
                                                     The Chief Information Officer, who heads the IT Directorate, and the
                                                     Associate Director of the 2020 Census Directorate have spearheaded the
                                                     initiative to create a new system development methodology and have
                                                     directed their staffs to work to implement the methodology within their
                                                     directorates. However, the bureau has not made specific plans to implement
                                                     the methodology across the bureau. The Chief Information Officer stated
                                                     that because of the culture change associated with establishing such a
                                                     standardized process at the bureau, it was important to take a gradual
                                                     approach to implementation. Regarding requirements management, both
                                                     directorates have developed new processes and have indicated that these
                                                     processes will be integrated sometime in the future; however they did not
                                                     specify when this would occur.
A detailed implementation plan is      Not           The Chief of the Office of Risk Management and Program Evaluation
developed that lays out a road map     implemented   originally stated that the bureau intended to finalize its plan in August 2012.
for implementing the new process                     However, the office did not provide a draft plan or milestones for us to
                                                     review. As of August 2012, the chief stated that the implementation plan
                                                     would not be finalized and approved until September 2012. In addition, the
                                                     office has not finalized plans for implementing the enterprise portfolio
                                                     management tool across the bureau.
                                                     The Chief Information Officer delegated ownership of the system
                                                     methodology, including the requirements development and management
                                                     processes, to the Application Services Division within the IT Directorate and
                                                     established a project team to coordinate implementation. The Chief of the
                                                     Application Services Division stated that his division was beginning to
                                                     develop a formal implementation plan for the system development
                                                     methodology, but did not provide any more specific information.




                                           Page 26                                       GAO-12-915 Census Management Practices
Key activity                          Assessment            Summary of assessment
Pilot testing of the new process is   Partially             The Office of Risk Management and Program Evaluation has not finalized
performed to evaluate the process     implemented           plans for pilot testing the new investment management process or the
and identify potential problems                             enterprise investment management tool, though two directorates have been
                                                            tentatively identified to help pilot these efforts.
                                                            The bureau is currently piloting the new system development methodology
                                                            and using the IT Directorate’s requirements process on two projects
                                                            managed by the IT Directorate: the Electronic Document Management
                                                            System and an effort to upgrade and consolidate Linux servers within the
                                                            bureau. However, one of the two pilots—the Linux project—is of limited
                                                            usefulness in testing the system development life cycle because it is not a
                                                            system development effort. In addition, the second pilot—the Electronic
                                                            Document Management System—is a limited system development effort
                                                            that is intended to take 3 months. Although both the Chief Information
                                                            Officer, who heads the IT Directorate, and the Associate Director of the
                                                            2020 Census Directorate stated that they intend to use the system
                                                            development life-cycle methodology in the future, there are no specific plans
                                                            for additional pilots.
A formal evaluation process is        Not                   The Office of Risk Management and Program Evaluation has not identified a
developed to determine the            implemented           formal evaluation process for evaluating the pilots or the implementation of
effectiveness of the new process in                         the new investment management governance structure.
meeting the agency’s goals                                  The project manager for the system development life-cycle methodology has
                                                            initiated weekly meetings to obtain feedback on the use of the methodology
                                                            for the pilots. However, an evaluation process has not yet been established.
                                          Source: GAO analysis of Census Bureau documentation and interviews with bureau officials.


                                          With respect to the investment management plan, the Chief of the Office
                                          of Risk Management and Program Evaluation said that the bureau had
                                          not finalized a plan for implementing the new investment management
                                          structure because the office was still in the process of incorporating
                                          feedback on the plan. For the system development methodology, the
                                          Chief Information Officer stated that an implementation plan had not been
                                          finalized because more work was needed to refine the methodology. The
                                          methodology would be implemented across the bureau once it had been
                                          refined and all issues were resolved.

                                          As we noted in 2005, the bureau’s lack of a consistent bureau-wide
                                          approach for IT investment management contributed to the bureau not
                                          effectively and efficiently managing multimillion dollar investments,
                                          including taking consistent and appropriate action when cost, schedule, or
                                          performance expectations were not being met. In addition, a lack of a
                                          consistent approach for system development and management, including
                                          requirements development and management, led to project teams
                                          managing systems in an ad hoc manner and increased the risk that cost




                                          Page 27                                                              GAO-12-915 Census Management Practices
overruns, schedule slippages, and performance shortfalls would, and did,
occur, as we reported in 2007 and 2009. 37

According to the bureau’s timeline for 2020 Decennial Census planning, it
will begin operational development and system testing starting in fiscal
year 2015. While the exact design for 2020 information systems is not yet
defined, it is likely to be complex, involve multiple directorates, and use
both contractors and bureau staff based on the prior decennial census
and the bureau’s initial plans for 2020.

Unless the investment management plan and system development life-
cycle methodology, including a requirements development and
management process, are fully implemented by this time, the Census
Bureau will face increased risk that similar challenges that occurred in the
2010 Decennial Census will occur for the 2020 Decennial Census’s
multimillion dollar investments. In addition, although only full
implementation can identify all the potential problems with the new
investment management plan and system methodology, until the bureau
conducts additional pilot testing, including various software development
models and projects of the scope and complexity of those needed for the
2020 Decennial Census, there is increased risk of not identifying potential
problems with the investment management plan and system development
life-cycle methodology.

Moreover, while feedback sessions on issues with implementing
processes can be useful, until the Office of Risk Management and
Program Evaluation and the Application Services Division establish a
documented evaluation process to assess the effectiveness of the new
processes, the bureau will lack assurance as to whether these processes
are effective in meeting the bureau’s goals. Lastly, while any significant
change cannot be accomplished overnight, clear leadership and
deadlines are essential to implement changes. Failure to address these
issues in a timely manner puts the bureau at risk of the same cost
overrun, schedule slippage, and performance shortfall issues that
affected the previous census.




37
 GAO-08-79 and GAO-10-59.




Page 28                                 GAO-12-915 Census Management Practices
                                              Over the past year, the Census Bureau has taken limited steps to develop
Census Bureau Does                            IT human capital practices, including identifying critical IT occupations
Not Have Bureau-                              and select competencies and conducting an inventory of these
                                              competencies among its IT staff in June 2011. However, many key
wide Workforce                                practices remain to be implemented. In particular, the Census Bureau has
Planning Practices for                        not developed a bureau-wide IT workforce plan, identified gaps in
IT Staff                                      mission-critical IT occupations, skills, and competencies, or developed
                                              strategies to address gaps. Table 3 summarizes our assessment of the
                                              bureau’s efforts against key principles for effective workforce planning. 38

Table 3: Census Bureau’s IT Workforce Planning Practices

Principle                                Assessment     Summary of assessment
Align workforce planning with            Not            According to bureau officials, the bureau has not developed a bureau-wide IT
                                                                                              a
strategic planning and budget            implemented    workforce plan since at least 2009 and only one directorate, the IT
formulation                                             Directorate, has created an IT workforce plan. According to bureau officials,
                                                        each directorate is responsible for establishing its own workforce planning,
                                                        including IT planning for its staff, as IT staff are distributed across the bureau.
                                                        According to the Chief of the Human Resources Division, the bureau has
                                                        provided no guidance to the directorates on preparing workforce plans or
                                                        established any efforts to coordinate workforce plans between them.
Involve top management,                  Not            The bureau has not involved top management or key stakeholders in strategic
employees and other stakeholders         implemented    IT workforce planning because each directorate is responsible for developing
in developing, communicating, and                       their own plans. Although the IT Directorate has established a draft IT
implementing the strategic                              workforce plan, bureau officials from the IT Directorate stated there was limited
workforce plan                                          involvement from management and employees in developing this draft plan
                                                        due to budget constraints and lack of sufficient staff resources.
Identify critical occupations, skills,   Partially      In June 2011, the bureau conducted a pilot assessment of the competencies
and competencies and analyze             implemented    of its IT workforce across the bureau, with the intent of conducting a bureau-
workforce gaps                                          wide competency inventory assessment of all employees in fall 2012. As part
                                                        of this pilot effort, the Human Resources Division identified the critical IT
                                                        occupations needed, but did not identify critical IT skills and only partially
                                                        identified critical IT competencies. In addition, the bureau has not yet used this
                                                        information to conduct an IT skills gap analysis.
Develop strategies tailored to        Not               Work has not been started in this area for the bureau’s IT workforce because a
address workforce gaps and human implemented            gap analysis for critical occupations, skills, and competencies has not been
capital conditions in critical skills                   performed. The bureau does have existing strategies for other staff that could
and competencies that need                              be used to address gaps for IT staff once the gaps are identified.
attention
Build capacity to address workforce      Not            Work has not been started in this area for the bureau’s IT workforce because
strategies                               implemented    the bureau has not completed a gap analysis of critical IT occupations, skills,
                                                        and competencies.




                                              38
                                                We reported on the bureau’s early management of strategic workforce planning for the
                                              2020 Decennial Census for areas other than IT in May 2012. See GAO-12-626.




                                              Page 29                                           GAO-12-915 Census Management Practices
Principle                       Assessment          Summary of assessment
Monitor and evaluate progress   Not                 The bureau has not yet developed an IT workforce plan, and so is not able to
toward achieving workforce      implemented         monitor and evaluate progress in addressing its workforce planning and
planning and strategic goals                        strategic goals.
                                     Source: GAO analysis of Census Bureau documentation and interviews with bureau officials.

                                     a
                                      The bureau has developed a bureau-wide human capital management plan but it does not
                                     specifically address IT.

                                     According to officials in the Human Resources Division, the bureau has
                                     not yet begun key workforce planning activities because it first needs to
                                     complete an initial bureau-wide competency assessment. In planning for
                                     this assessment, the bureau conducted a pilot assessment of IT
                                     competencies in June 2011. Officials stated that the pilot assessment
                                     provided several lessons that they intend to use for the bureau-wide
                                     competency assessment. For instance, while managers in the
                                     directorates with IT staff found that the information from the pilot
                                     assessment was useful, those managers were more interested in
                                     identifying the current skills of their IT staff than the competency
                                     information that was gathered. The bureau’s Human Resources Division
                                     therefore collected additional information for managers in May 2012. The
                                     bureau is to conduct the bureau-wide competency assessment, including
                                     a reassessment of IT competencies, in late 2012.

                                     Once this assessment is completed, the bureau’s directorates are to
                                     conduct additional workforce planning activities. For example, bureau
                                     officials stated that individual directorates plan to conduct gap analyses
                                     for mission-critical skills and competencies. According to the bureau’s
                                     Human Resources Division, once the gap analysis is completed, the
                                     bureau is to begin refining existing strategies and its capacity to address
                                     workforce gaps. The bureau provided a document with high-level goals to
                                     incorporate results of its competency assessment into individual
                                     directorate workforce plans by July 2013. However, it did not provide time
                                     frames for when specific activities would be completed for its IT
                                     workforce, such as a gap analysis for occupations, skills, or competencies
                                     of IT staff, nor did it provide plans to integrate these activities bureau-
                                     wide.

                                     The Chief of the Human Resources Division stated that the Census
                                     Bureau has traditionally been decentralized in its IT workforce planning
                                     efforts because its IT staff is located in several directorates. The bureau
                                     has also faced budget constraints in conducting workforce planning
                                     specifically for its IT staff but had recently begun to undertake efforts in
                                     this area within the IT Directorate after a new division chief was hired.



                                     Page 30                                                              GAO-12-915 Census Management Practices
              Effective IT workforce planning efforts are critical to ensuring the bureau
              has the appropriate workforce in place to achieve its mission and
              strategic goals, particularly in regards to the 2020 Decennial Census.
              While the bureau’s Human Resources Division plans to undertake a
              thorough assessment of the competencies of its workforce this year,
              unless the division establishes a repeatable process for performing skills
              assessments and gap analysis that can be implemented in a timely
              manner, managers may not be able to make decisions to address any
              skills gaps in preparation for the 2020 Decennial Census. In addition, until
              the Human Resources Division establishes a process for directorates to
              coordinate on IT workforce planning in line with key principles for effective
              workforce planning, the bureau may not have sufficient assurance that it
              has the appropriate IT workforce needed for 2020 Decennial Census
              activities.


              While the Census Bureau has begun to make improvements to
Conclusions   investment management and system development processes, including
              requirements development and management, more work remains to be
              done to refine these processes and implement them across the bureau.
              As we have previously reported, a lack of robust processes in these areas
              contributed to the cost overruns, schedule slippages, and performance
              shortfalls in key IT investments that were needed for the 2010 Decennial
              Census, which increased its cost by up to $3 billion. However, the bureau
              has not established key guidelines and thresholds within its investment
              management plan, nor has it developed guidance to tailor the bureau’s
              new system development methodology to alternate development models,
              or established plans to implement these new processes across the
              bureau. Until the bureau takes action in these key areas, there is the risk
              that similar issues will arise for the 2020 Decennial Census.

              Furthermore, having an IT workforce that has the appropriate mission-
              critical skills and competencies will be necessary to help the bureau
              effectively develop and manage its multimillion dollar investments in
              information systems and technology. While the bureau has begun to
              improve its IT human capital practices including conducting an inventory
              of select IT competencies, many key workforce planning practices remain
              to be put in place, including conducting gap analyses and integrating IT
              workforce planning bureau-wide. Having effective IT workforce planning
              practices will help ensure the bureau can achieve its mission and
              strategic goals for the 2020 Decennial Census.




              Page 31                                  GAO-12-915 Census Management Practices
                      To strengthen and improve the bureau’s new investment management,
Recommendations for   system development, and IT workforce management processes, we
Executive Action      recommend that the Acting Secretary of Commerce direct the Under
                      Secretary for Economic Affairs who oversees the Economics and
                      Statistics Administration, as well as the Acting Director of the U.S.
                      Census Bureau, take eight actions to address weaknesses in the
                      following IT management areas:

                      •   Establish guidelines for the frequency and membership of bureau
                          investment review boards and thresholds for these boards to escalate
                          cost or schedule variance issues to higher-level boards.

                      •   Establish time frames for project managers to provide periodic
                          updates of investment information in the enterprise investment
                          management tool.

                      •   Adapt the bureau’s new system development life-cycle methodology,
                          including the mandatory work products, activities, and phases of the
                          project, to the additional software development models beyond the
                          waterfall model that are specified in the methodology.

                      •   Establish and implement a consistent requirements development and
                          management process across the bureau that is integrated with its new
                          system development life-cycle methodology and includes guidance for
                          developing requirements at the strategic mission, business, and
                          project levels.

                      •   Finalize a plan for implementing the Enterprise Investment
                          Management Plan, including time frames for implementation by fiscal
                          year 2015, pilot testing of the new process, and a documented
                          evaluation process.

                      •   Establish a plan for implementing the new system development life-
                          cycle methodology, including requirements development and
                          management processes, across the bureau, to include time frames for
                          implementation by fiscal year 2015, additional pilots of the
                          methodology prior to full implementation, and a documented
                          evaluation process.

                      •   Establish a repeatable process for performing IT skills assessments
                          and gap analysis that can be implemented in a timely manner.

                      •   Establish a process for directorates to coordinate on IT workforce
                          planning, including: (1) aligning IT workforce planning with strategic


                      Page 32                                  GAO-12-915 Census Management Practices
                      planning and budget formulation; (2) involving appropriate
                      stakeholders and staff from each directorate; (3) identifying critical
                      occupations, skills, and competencies, and analyzing workforce gaps;
                      (4) developing strategies to address IT workforce gaps; (5) building
                      capacity to address workforce gaps; and (6) monitoring and
                      evaluating IT workforce planning efforts across the bureau, and
                      ensure this process is implemented across the bureau.



                  We received comments from the Acting Secretary of Commerce on a
Agency Comments   draft of this report. The comments are included in appendix II. In its
                  comments, the department stated that the Census Bureau concurred with
                  our eight recommendations and outlined steps it was taking to implement
                  the recommendations. The bureau acknowledged that the Enterprise
                  Investment Management Plan was in draft, but indicated it had deployed
                  components of the plan, such as initiating governing boards in three
                  directorates. The bureau also noted that its enterprise portfolio
                  management tool had been placed into production in July and that the
                  bureau was working to migrate to the new tool.

                  Regarding the system development life-cycle methodology, the bureau
                  stated that the methodology was one of several initiatives to improve the
                  bureau’s delivery of IT services. The bureau said that later iterations of
                  the methodology would include more iterative models such as Agile that
                  were applicable to its business needs. The bureau also indicated it was
                  planning to include feedback from its pilot projects in the revised
                  methodology and to provide additional guidance on required
                  documentation.

                  For its requirements management processes, the bureau stated it was
                  planning to integrate the bureau’s different requirements management
                  processes and emphasized that joint commitment of the 2020 Decennial
                  Census and IT Directorates was critical to the success of a requirements
                  management process for the 2020 Decennial Census.

                  The bureau also indicated that it has provided extensive training and
                  education for its IT workforce even though it lacked an integrated plan
                  based on best practices. The bureau plans to develop a workforce plan to
                  incorporate strategies for hiring, developing, and contracting to meet its
                  identified requirements. It noted, however, that this could be affected by
                  potential budget cuts. The bureau stated its bureau-wide assessment and
                  competency gap analysis would be completed by fall 2012.



                  Page 33                                 GAO-12-915 Census Management Practices
We are sending copies of this report to the Acting Secretary of
Commerce, the Senior Advisor to the Acting Director and the Deputy
Director of the U.S. Census Bureau, and interested congressional
committees. The report also is available at no charge on GAO’s website
at http://www.gao.gov.

If you or your staffs have any questions on the matters discussed in this
report, please contact me at (202) 512-9286 or pownerd@gao.gov.
Contact points for our Offices of Congressional Relations and Public
Affairs may be found on the last page of this report. GAO staff who made
major contributions to this report are listed in appendix III.




David A. Powner
Director, Information Technology
  Management Issues




Page 34                                GAO-12-915 Census Management Practices
Appendix I: Objectives, Scope, and
              Appendix I: Objectives, Scope, and
              Methodology



Methodology

              Our objectives were to evaluate the (1) effectiveness of the Census
              Bureau’s policies, procedures, and processes for managing information
              technology (IT) investments and system development and (2) the Census
              Bureau’s development of effective practices for acquiring and maintaining
              IT human capital skills.

              To address our first objective, we reviewed the effectiveness of policies
              and procedures in two areas—investment management and system
              development and management. We compared the bureau’s new IT
              investment management policies and procedures in its draft Enterprise
              Investment Management Plan to the criteria for policies and procedures
              associated with maturity stage 2 of the Information Technology
              Investment Management (ITIM) framework. 1 The ITIM framework consists
              of five progressive stages of maturity that an agency can achieve in its
              investment management capabilities. The maturity stages are cumulative;
              that is, in order to attain a higher stage, an agency must first
              institutionalize all of the critical processes at the lower stages. To
              determine whether the bureau satisfied the criteria for maturity stage 2 we
              compared the bureau’s policies and procedures to the critical processes
              outlined in stage 2. We did not evaluate the bureau at maturity stage 1
              because our prior review in 2005 determined that it had passed that
              stage. 2 We did not evaluate the Census Bureau at maturity stages 3, 4, or
              5 because our prior work has shown that an agency should focus on
              implementing all practices associated with a lower phase before
              addressing the higher-stage practices. We also interviewed officials from
              the Office of Risk Management and Program Evaluation regarding the
              development and implementation of the plan across the bureau.

              To assess the effectiveness of the bureau’s processes for managing
              system development, including requirements development and
              management, we reviewed the bureau’s System Development Life Cycle
              Users’ Guide and project templates, Requirements Engineering
              Management Plan, and the Application Services Division’s Requirements
              Elicitation, Analysis, and Documentation Process, and compared these
              documents with Software Engineering Institute’s (SEI) Capability Maturity
              Model Integration for Development criteria in two areas: establishing
              organizational processes, and requirements management and


              1
                  GAO-04-394G.
              2
               GAO-05-661.




              Page 35                                 GAO-12-915 Census Management Practices
Appendix I: Objectives, Scope, and
Methodology




development. Although SEI specifies criteria in numerous areas, we
focused on establishing organizational processes because the bureau’s
system development methodology is newly developed. We focused on
requirements development and management because of the bureau’s
challenges in managing requirements for the 2010 Decennial Census. For
each of the two areas, we analyzed bureau plans and procedures to
determine if the practices described were consistent with those in the SEI
criteria. In addition, we interviewed officials from the IT Directorate and
2020 Census Directorate regarding development and implementation of
the guide and the requirements management and development
processes.

To assess the Census Bureau’s efforts to effectively implement these
new investment management and system development processes across
the bureau, we interviewed officials from the Office of Risk Management
and Program Evaluation, IT Directorate, and 2020 Census Directorate
regarding implementation efforts and reviewed related documentation,
which we compared to our reported best practices for implementing new
organizational governance processes. 3 We evaluated whether these
efforts satisfied each of the components of each key activity from the best
practices and assigned ratings of “implemented”, “partially implemented”,
or “not implemented” based on that assessment. A rating of “partially
implemented” was given if the bureau’s activities satisfied at least one
component of the key activity.

To address our second objective, we reviewed bureau workforce planning
documents, including the Human Capital Management Plan (FY 2011-
2016 ), IT Directorate’s 2011-2016 Strategic Information Technology
Plan, and the IT Directorate’s workforce plan, and other bureau
documentation related to the bureau’s pilot workforce competency
assessment and compared these to the six leading principles for
workforce planning that we and the Office of Personnel Management
have identified to determine whether the bureau’s practices were
consistent with these principles. We evaluated whether the bureau’s
activities satisfied each of the detailed practices in a principle and
assigned ratings of “implemented”, “partially implemented”, or “not
implemented” based on that assessment. A rating of “partially
implemented” was assessed if the bureau’s activities satisfied at least one



3
GAO-12-818 and GAO/AIMD-10.1.15.




Page 36                                 GAO-12-915 Census Management Practices
Appendix I: Objectives, Scope, and
Methodology




of the detailed practices in the principle. We also interviewed officials from
the Human Resources Division and IT Directorate to obtain information
about the pilot assessment and the bureau’s workforce planning efforts.

We performed our work from March 2012 through September 2012 in
accordance with generally accepted government auditing standards.
Those standards require that we plan and perform the audit to obtain
sufficient, appropriate evidence to provide a reasonable basis for our
findings and conclusions based on our audit objectives. We believe that
the evidence obtained provides a reasonable basis for our findings and
conclusions based on our audit objectives.




Page 37                                  GAO-12-915 Census Management Practices
Appendix II: Comments from the Department
             Appendix II: Comments from the Department
             of Commerce



of Commerce




             Page 38                                     GAO-12-915 Census Management Practices
Appendix II: Comments from the Department
of Commerce




Page 39                                     GAO-12-915 Census Management Practices
Appendix II: Comments from the Department
of Commerce




Page 40                                     GAO-12-915 Census Management Practices
Appendix II: Comments from the Department
of Commerce




Page 41                                     GAO-12-915 Census Management Practices
Appendix II: Comments from the Department
of Commerce




Page 42                                     GAO-12-915 Census Management Practices
Appendix III: GAO Contact and Staff
                  Appendix III: GAO Contact and Staff
                  Acknowledgments



Acknowledgments

                  David A. Powner at (202) 512-9286 or pownerd@gao.gov
GAO Contact
                  In addition to the contact listed above, the following staff made key
Staff             contributions to this report: Vijay D’Souza (Assistant Director); Justin
Acknowledgments   Booth; Nancy Glover; Valerie Hopkins; Paul Middleton; Tarunkant
                  Mithani; and Karl Seifert.




(311265)
                  Page 43                                  GAO-12-915 Census Management Practices
GAO’s Mission         The Government Accountability Office, the audit, evaluation, and
                      investigative arm of Congress, exists to support Congress in meeting its
                      constitutional responsibilities and to help improve the performance and
                      accountability of the federal government for the American people. GAO
                      examines the use of public funds; evaluates federal programs and
                      policies; and provides analyses, recommendations, and other assistance
                      to help Congress make informed oversight, policy, and funding decisions.
                      GAO’s commitment to good government is reflected in its core values of
                      accountability, integrity, and reliability.

                      The fastest and easiest way to obtain copies of GAO documents at no
Obtaining Copies of   cost is through GAO’s website (http://www.gao.gov). Each weekday
GAO Reports and       afternoon, GAO posts on its website newly released reports, testimony,
                      and correspondence. To have GAO e-mail you a list of newly posted
Testimony             products, go to http://www.gao.gov and select “E-mail Updates.”

Order by Phone        The price of each GAO publication reflects GAO’s actual cost of
                      production and distribution and depends on the number of pages in the
                      publication and whether the publication is printed in color or black and
                      white. Pricing and ordering information is posted on GAO’s website,
                      http://www.gao.gov/ordering.htm.
                      Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
                      TDD (202) 512-2537.
                      Orders may be paid for using American Express, Discover Card,
                      MasterCard, Visa, check, or money order. Call for additional information.
                      Connect with GAO on Facebook, Flickr, Twitter, and YouTube.
Connect with GAO      Subscribe to our RSS Feeds or E-mail Updates. Listen to our Podcasts.
                      Visit GAO on the web at www.gao.gov.
                      Contact:
To Report Fraud,
Waste, and Abuse in   Website: http://www.gao.gov/fraudnet/fraudnet.htm
                      E-mail: fraudnet@gao.gov
Federal Programs      Automated answering system: (800) 424-5454 or (202) 512-7470

                      Katherine Siggerud, Managing Director, siggerudk@gao.gov, (202) 512-
Congressional         4400, U.S. Government Accountability Office, 441 G Street NW, Room
Relations             7125, Washington, DC 20548

                      Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800
Public Affairs        U.S. Government Accountability Office, 441 G Street NW, Room 7149
                      Washington, DC 20548




                        Please Print on Recycled Paper.