oversight

Bureau of Land Management: Plan Needed to Sustain Progress in Establishing IT Investment Management Capabilities

Published by the Government Accountability Office on 2003-09-12.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                 United States General Accounting Office

GAO              Report to the Subcommittee on Interior
                 and Related Agencies, Committee on
                 Appropriations, House of
                 Representatives

September 2003
                 BUREAU OF LAND
                 MANAGEMENT
                 Plan Needed to
                 Sustain Progress in
                 Establishing IT
                 Investment
                 Management
                 Capabilities




GAO-03-1025
                 a
                                                September 2003


                                                BUREAU OF LAND MANAGEMENT

                                                Plan Needed to Sustain Progress in
Highlights of GAO-03-1025, a report to the      Establishing IT Investment Management
Subcommittee on Interior and Related
Agencies, Committee on Appropriations,          Capabilities
House of Representatives




The mission of the Department of                BLM has made progress in establishing its ITIM capabilities. Specifically,
the Interior’s Bureau of Land
Management (BLM) is to maintain                 • BLM has established most of the key practices associated with building an
the health, diversity, and                        investment foundation (see table). For example, the bureau has
productivity of the public lands for              established a board for managing IT investments, implemented processes
the use and enjoyment of present                  to ensure that IT projects support business needs and meet users’
and future generations. BLM                       requirements, and established a process for selecting IT proposals. In
employs about 11,000 people, with                 addition, the bureau has efforts under way to address the key practices it
information technology (IT)                       has not yet established.
playing a critical role in helping
BLM perform its responsibilities.               • BLM has also initiated efforts to manage its investments as a portfolio. For
The bureau estimates that it will                 example, it has established a council to support portfolio management
spend about $146 million on IT                    activities and begun defining portfolio selection criteria. BLM has also
initiatives in fiscal year 2003.                  begun performing postimplementation reviews to learn lessons that will
                                                  help define and implement an IT investment evaluation process. However
GAO was asked to evaluate BLM’s                   BLM’s progress to date in defining practices for managing its investments
IT investment management (ITIM)                   as a portfolio has been limited because, according to its officials, its
capabilities and determine the                    investment board first focused its resources on establishing the processes
bureau’s plans for improving these                associated with building the IT investment management foundation.
capabilities. GAO’s evaluation was
based on applying its ITIM maturity
framework, which identifies critical            Although BLM has made progress in developing its IT investment process, it
processes for successful IT                     has not yet developed a plan to guide its efforts in this area and, as a result,
investment management.                          may not be able to successfully establish more mature ITIM processes.
                                                According to the chief information officer, this is because BLM wanted to
                                                develop an ITIM plan that is integrated with improvement plans for other IT
                                                management areas, and the results of the comprehensive assessment that
GAO recommends that the
Secretary of the Interior direct                were to be used as the basis for such a plan were obtained only in June 2003.
BLM’s Director to develop and                   BLM officials agree that this plan is necessary for guiding improvement
implement a plan for making ITIM                efforts and stated their intention to develop one. Developing such a plan will
improvements that is based on                   help BLM sustain progress made to date.
GAO’s ITIM stage 2 and 3 critical
processes; specifies measurable                 Summary of Results for Investment Foundation Critical Processes and Key Practices
goals, outcomes, and needed
                                                                                                                                          Percentage of
resources; and assigns clear                                                                                                              key practices
responsibility for tasks. Progress                Critical process                          Purpose                                       executed
should be measured and reported                   IT investment board     To define and establish the governing board(s) responsible          100%
periodically to Interior.                         operation               for selecting, controlling, and evaluating IT investments.

BLM agreed with our findings and                  IT project and          To create and maintain an IT project inventory to assist in          42%
                                                  system identification   managerial decision making.
recommendations and noted that it
had begun developing a plan for                   IT project oversight    To regularly determine each IT project's progress toward cost        91%
                                                                          and schedule milestones using established criteria and take
making ITIM improvements in                                               corrective actions when milestones are not achieved.
accordance with our
                                                  Business needs          To ensure that each IT project supports the organization's          100%
recommendations.                                  identification          business needs and meets users' needs.
www.gao.gov/cgi-bin/getrpt?GAO-03-1025.           Proposal selection      To ensure that an established, structured process is used to         83%
                                                                          select new IT proposals.
To view the full product, including the scope     Total                                                                                        84%
and methodology, click on the link above.
For more information, contact Linda D.
Koontz at (202) 512-6240 or                     Source: GAO.
koontzl@gao.gov.
Contents



Letter                                                                                                   1
                            Results in Brief                                                             2
                            Background                                                                   3
                            BLM Has Established Many of the Key Practices for Effective
                              Investment Management                                                     12
                            BLM Does Not Have a Plan to Guide IT Investment Management
                              Improvement Efforts                                                       27
                            Conclusions                                                                 28
                            Recommendations for Executive Action                                        28
                            Agency Comments and Our Evaluation                                          29


Appendixes
             Appendix I:    Objectives, Scope, and Methodology                                          31
             Appendix II:   Comments from the Bureau of Land Management                                 34


Tables                      Table 1: Stage 2 Critical Processes—Building the Investment
                                     Foundation                                                         13
                            Table 2: Summary of Results for Stage 2 Critical Processes and Key
                                     Practices                                                          14
                            Table 3: IT Investment Board Operation                                      16
                            Table 4: IT Project and System Identification                               19
                            Table 5: IT Project Oversight                                               21
                            Table 6: Business Needs Identification                                      24
                            Table 7: Proposal Selection                                                 26


Figures                     Figure 1: BLM’s IT Investment Management Process                             9
                            Figure 2: The Five Stages of Maturity with Critical Processes               11




                            Page i                       GAO-03-1025 BLM’s IT Investment Management Process
Contents




Abbreviations

BLM          Bureau of Land Management
CIO          chief information officer
IT           information technology
ITIB         Information Technology Investment Board
ITIM         information technology investment management

 This is a work of the U.S. government and is not subject to copyright protection in the
 United States. It may be reproduced and distributed in its entirety without further
 permission from GAO. However, because this work may contain copyrighted images or
 other material, permission from the copyright holder may be necessary if you wish to
 reproduce this material separately.




Page ii                            GAO-03-1025 BLM’s IT Investment Management Process
A
United States General Accounting Office
Washington, D.C. 20548



                                    September 12, 2003                                                                               Leter




                                    The Honorable Charles H. Taylor
                                    Chairman
                                    The Honorable Norman D. Dicks
                                    Ranking Minority Member
                                    Subcommittee on Interior
                                      and Related Agencies
                                    Committee on Appropriations
                                    House of Representatives

                                    The mission of the Department of the Interior’s Bureau of Land
                                    Management (BLM) is to maintain the health, diversity, and productivity of
                                    public lands for the use and enjoyment of present and future generations.
                                    To carry out the functions involved in managing the public lands, BLM
                                    employs a workforce of about 11,000 employees located in headquarters in
                                    Washington, D.C., state offices, field offices, and national centers
                                    specializing in training, fire management support, science and technology,
                                    human resources management, information resources management, and
                                    business services. Information technology (IT) plays a critical role in
                                    helping BLM carry out its responsibilities. The bureau estimates that it will
                                    spend about $146 million on IT initiatives in fiscal year 2003.

                                    This report is one of two in response to your request to evaluate the
                                    Department of the Interior’s IT investment management process.1 As
                                    agreed, the objectives of our review were to (1) evaluate BLM’s IT
                                    investment management capabilities against the key practices defined in
                                    our IT investment management assessment framework2 and (2) determine
                                    the agency’s plans for improving these capabilities. We performed our work
                                    in accordance with generally accepted government accounting standards.
                                    Details on our objectives, scope, and methodology are contained in
                                    appendix I.


                                    1
                                     The second report, U.S. General Accounting Office, Information Technology Departmental
                                    Leadership Crucial to Success of Investment Reforms at Interior, GAO-03-1028
                                    (Washington, D.C.: Sept. 12, 2003) addresses (1) the Department of the Interior’s capabilities
                                    for managing the agency’s IT investments, including its ability to effectively overseee bureau
                                    processes, and (2) the department’s actions and plans to improve these capabilities.
                                    2
                                     U.S. General Accounting Office, Information Technology Investment Management: A
                                    Framework for Assessing and Improving Process Maturity (Exposure Draft), GAO/AIMD-
                                    10.1.23 (Washington, D.C.: May 2000).




                                    Page 1                               GAO-03-1025 BLM’s IT Investment Management Process
Results in Brief   BLM has made progress in establishing key practices for effectively
                   managing its IT investments, including the foundational practices for
                   selecting and controlling IT investments, which provide assurance that
                   projects selected meet organizational needs and will be completed on time
                   and within budget, and the practices for managing investments as a
                   portfolio. Specifically,

                   • BLM has established most of the key practices (about 85 percent)
                     associated with building the investment foundation. For example, BLM
                     has defined and established an investment board for managing IT
                     investments, implemented processes to ensure that projects support
                     business needs and meet users’ requirements, and established a process
                     for selecting proposals. Although the agency has not yet, for example,
                     defined policies and procedures for collecting information into its
                     project and system inventory to make informed investment
                     management decisions or fully defined criteria for analyzing and
                     prioritizing new proposals, it recognizes the importance of resolving
                     these issues and has efforts under way to address them.

                   • BLM has also initiated efforts to manage its investments as a portfolio—
                     that is, as a set of competing IT investments—and has begun to address
                     the evaluation of projects and systems after development using
                     postimplementation reviews,3 a critical process associated with the
                     most capable organizations. For example, BLM has established a
                     council to support portfolio management activities and begun defining
                     portfolio selection criteria. BLM has also performed
                     postimplementation reviews on a limited basis to learn lessons that will
                     help define and implement a robust IT investment evaluation process.

                   Although BLM is clearly committed to further developing its IT investment
                   process, it has not developed a plan to guide its efforts to do so, and, as a
                   result, may not be able to successfully establish mature IT investment
                   management processes. According to the chief information officer (CIO),
                   this is because BLM wanted to develop a plan that is integrated with
                   improvement plans for other IT management areas, and the results of the
                   comprehensive assessment that were to be used as the basis for such a plan
                   were obtained only in June 2003. BLM officials stated their intention to



                   3
                   BLM refers to postimplementation reviews as postdeployment reviews.




                   Page 2                           GAO-03-1025 BLM’s IT Investment Management Process
                  develop a plan to guide improvement efforts. By developing such a plan,
                  BLM would be able to sustain progress made to date.

                  To strengthen BLM’s investment management capability, we are
                  recommending that BLM develop and implement a plan aimed at
                  addressing the weaknesses identified in this report. In written comments
                  on a draft of this report (reprinted in app. II), BLM’s Director agreed with
                  our findings and recommendations and stated that they represented a fair
                  and accurate evaluation of the bureau’s status and progress towards IT
                  investment management maturity. BLM also provided additional technical
                  comments, which we have incorporated into the report as appropriate.



Background        BLM, an agency of the U.S. Department of the Interior, manages 261 million
                  surface acres and an additional 700 million acres of subsurface mineral
                  estate throughout the nation. The agency’s mission is to maintain the
                  health, diversity, and productivity of public lands for the use and enjoyment
                  of present and future generations. To carry out this mission, BLM employs
                  a workforce of about 11,000 employees located in headquarters in
                  Washington, D.C., 12 state offices, 130 field offices, and national centers
                  specializing in training, fire management support, science and technology,
                  human resources management, information resources management, and
                  business services.



BLM’s Use of IT   BLM collects, analyzes, and records a tremendous amount of business
                  information about the public lands and resources, ranging from land title to
                  recreational usage to wildlife habitat. These data are mainly geographic in
                  character and are best understood when displayed and analyzed in spatial
                  form using automated geographic information systems. Numerous
                  parties—including public land users; educational institutions; public
                  interest groups; other federal, state, tribal, and local agencies; and the
                  scientific community—use these data or information to make thousands of
                  business decisions each year.

                  The central focus of BLM’s IT strategy is to develop integrated systems that
                  help BLM meet national and local needs in managing the lands and natural
                  resources, while supporting the mission and goals outlined in BLM’s
                  Strategic Plan. For example, BLM uses its Automated Fluid Minerals
                  Support System to support its Oil and Gas Program. The system helps
                  match mineral estate information to information on existing wells,



                  Page 3                        GAO-03-1025 BLM’s IT Investment Management Process
                            facilities, permits, and inspections and provides an automated system for
                            granting permits to BLM customers and creating reports. Currently, BLM
                            issues approximately 41,000 permits and reports and conducts about 19,000
                            inspections per year. BLM also uses its Management Information System,
                            which provides a Web-enabled, business information, budgetary, financial,
                            and program performance system so that simple data analysis can be
                            performed benefiting the entire bureau. This system provides managers
                            with up-to-date business data to make cost effective decisions concerning
                            the management of BLM’s people, resources, and natural resources for
                            several of Interior’s mission goals. BLM’s estimated IT expenditures are
                            $146.45 million for fiscal year 2003.



Prior Weaknesses in BLM’s   Between 1995 and 2001, we conducted reviews and issued several reports
IT Investment Management    on problems and risks that threatened the successful development and
                            deployment of BLM’s modernization of its Automated Land and Mineral
Process
                            Record System. In a recent report addressing this issue, we noted, among
                            other things, that after 15 years and about $411 million obligated, the
                            project was terminated because the Initial Operating Capability module—a
                            major component of the system—did not meet BLM’s business needs and
                            therefore could not be deployed.4 We also reported that the absence of
                            adequate investment management processes and practices at BLM was a
                            significant factor contributing to the failure of the system. Accordingly, we
                            recommended that the Secretary of the Interior direct BLM to take certain
                            actions to help it strengthen its investment management process. BLM has
                            been working on improving its process since that time.



BLM’s Approach to           BLM has assigned several individuals and groups with responsibilities for
Investment Management       managing national IT investments, that is investments which, among other
                            things, are considered major applications or general support systems; have
                            a life-cycle value of greater than $500,000; or will affect multiple states,
                            centers, or business areas.5 These individuals and groups and their roles
                            are described below.

                            4
                             U.S. General Accounting Office, Land Management Systems: Status of BLM’s Actions to
                            Improve Information Technology Management, GAO/AIMD-00-67 (Washington, D.C.: Feb.
                            27, 2000).
                            5
                             Other criteria for national investments include investments between $50,000 and $500,000
                            for which a state or center does not have its own ITIB, and operations and maintenance
                            investments that are not included in program base funding.




                            Page 4                             GAO-03-1025 BLM’s IT Investment Management Process
• National Information Technology Investment Board (ITIB)—Chaired
  by BLM’s Deputy Director of Operations, this board is responsible for
  selecting, controlling, and evaluating all national IT investments.
  Members include the CIO, Chief Financial Officer, Assistant Directors
  from the business units, two State Directors, an Associate State
  Director, the CIO Council Chair, the Bureau Architecture Chair, a Fire
  and Aviation Portfolio Representative, and several ex officio members
  including Interior’s CIO, the bureau Architect, and managers from the
  System Coordination Office and the Investment Management Group.

• System Coordination Office—Created in June 2000 to support a
  number of IT management functions, the System Coordination Office,
  among other things, is responsible for coordinating the screening of all
  IT investments and projects to ensure that they are in line with the
  bureau’s selection, control, and evaluation criteria, and monitors project
  performance (scope, schedule, and budget). The office is also
  responsible for coordinating the development of a project management
  curriculum and mentoring and developing a cadre of trained and
  experienced project managers.

• Investment Management Group—Responsibilities of this group include
  coordinating the development and maintenance of the bureau’s IT
  investment portfolio, ensuring that all investments fit within budget
  constraints, and providing investment updates and forecasting as
  needed.

• Information Technology Portfolio Management Council—Chartered in
  June 2003, but established about a year ago, this council serves as an
  advisory council to the ITIB and is responsible, among other things, for
  applying business-related rating and ranking criteria to BLM’s portfolio,
  performing trade-off analyses, and working with the Investment
  Management Group to develop funding strategies. The council is also
  responsible for ensuring that investments are clearly tied to the mission
  and strategic plans (both business and information resources
  management) of the bureau and selected by a consistent, repeatable,




Page 5                        GAO-03-1025 BLM’s IT Investment Management Process
   objective process. Members include national IT portfolio managers6 for
   each of the directorates, representatives from the state portfolios and
   the Bureau Enterprise Architecture Team, and members from the
   System Coordination Office and the Investment Management Group.

• Bureau Enterprise Architecture Team—Responsible for ensuring that
  investment proposals and business cases are aligned with the bureau
  enterprise architecture’s business processes, data, applications, and
  technology components.7

• Project proponent—Responsible, among other things, for leading the
  development of the investment proposal, coordinating and championing
  the development of the business case, and working with the project
  manager throughout the life cycle of the project.

• Project sponsor—A field, center, or Washington office manager who
  authorizes the development of a business case. The project sponsor
  shifts roles to become the system owner when the project moves into
  operations and maintenance. The project sponsor is responsible for
  selecting a project manager, approving all project documentation, and
  participating in a management oversight role throughout the planning,
  design, development, testing, acceptance, and deployment of the
  project.

• Project manager—Responsible for developing the project plan and
  leading and managing the project. The project manager reports directly
  to the project sponsor. Ultimately, it is the project managers who



6
 Portfolio managers are responsible for ensuring that all IT investment proposals, national
systems, and externally directed systems are planned, developed, maintained, and
distributed according to the bureau’s ITIM process and configuration management, and are
in conformance with established bureau hardware and software standards, as well as the
bureau’s architecture and technical reference model; summarizing all operations and
maintenance costs associated with existing and proposed systems; ensuring that all new
project proposals address the discontinuance of existing systems as a cost saving measure;
and working directly with each state or center that proposes moving a state, center, or local
application to a national system.
7
 An enterprise architecture is an investment blueprint that defines, both in logical terms
(including business functions and applications) and in technical terms (including hardware,
software, data communications, and security), how an organization operates today (current
architecture), how it intends to operate tomorrow (target architecture), and a road map for
making the transition from today to tomorrow.




Page 6                              GAO-03-1025 BLM’s IT Investment Management Process
                   are responsible for successfully managing and completing one or more
                   projects approved by the ITIB.

                The bureau has also defined a three-phase IT investment management
                process which involves selecting proposed IT projects (select phase),
                controlling ongoing projects through development (control phase), and
                evaluating projects that have been deployed (evaluate phase). Each phase
                comprises multiple stages that have entrance and exit criteria defined in
                the IT Investment Management Process guide that must be satisfied before
                a project can move from one stage to the next stage or phase in the
                process. The System Coordination Office tracks projects’ progress through
                the various stages, ensuring that they comply with the processes defined in
                the guide. The national ITIB stays abreast of projects’ performance through
                quarterly report reviews. The board is also directly involved in key
                milestone (i.e., stage) reviews.

Select Phase    The purpose of the phase is to ensure that BLM chooses the IT projects that
                best support its mission and align with the bureau’s architecture. During
                this phase, the project proponent and portfolio manager are expected to
                collaborate to develop an investment proposal.

                The System Coordination Office is responsible for reviewing the proposal
                and ensuring that issues are identified and resolved. Finally, the ITIB is to
                review the proposal and either approve it, approve it with stipulations,
                return it for further analysis, or reject it. If the ITIB approves the proposal,
                the project manager and project proponent are to work to develop a more
                elaborate business case. The System Coordination Office reviews the
                business case and coordinates the reviews performed by other groups (e.g.,
                the Bureau Enterprise Architecture Team). The Office then makes
                recommendations for approval to the ITIB on the basis of these reviews.

                At the end of the select phase, a project plan is to be developed that defines
                the strategies for managing the project. According to BLM officials, to date
                the ITIB has placed more emphasis on this phase than on the other two.

Control Phase   Once selected for inclusion in the bureau’s IT portfolio, each project is to
                be managed by a trained or experienced IT project manager and monitored
                by the System Coordination Office and ITIB on a quarterly basis throughout
                its life cycle.

                Included within the project’s plan, which is developed at the end of the
                select phase, are milestones for architecture, technical, and project



                Page 7                         GAO-03-1025 BLM’s IT Investment Management Process
                 management reviews. Factors such as project risk, complexity, and cost
                 determine the scope and frequency of each of these milestone reviews.
                 Projects that fall short of meeting their predetermined budget, schedule, or
                 scope requirements are to be reviewed by the ITIB, who works with the
                 project managers to develop an appropriate course of action. If this issue
                 arises, the ITIB must decide whether to continue the project; rebaseline the
                 scope, schedule, or budget; or to terminate the project. Ultimately, all
                 decisions that are carried out are a result of the ITIB voting process.

Evaluate Phase   Once a project has been fully implemented and accepted by the users and
                 system owner, the System Coordination Office and ITIB are responsible for
                 monitoring its schedule and budget quarterly. BLM has also, on a limited
                 basis, begun performing postimplementation reviews—BLM refers to these
                 as postdeployment reviews— in which a project’s actual results are to be
                 evaluated against expected results to compare realized to estimated
                 benefits and assess the project’s impact on mission performance.
                 Necessary changes or modifications to the project are to be identified, and
                 technical compliance with the bureau enterprise architecture is also to be
                 assessed. The main objective of the postimplementation review is to derive
                 lessons learned, which may lead to investment management process
                 improvements and opportunities for improving business processes (which
                 in turn provide input into the select phase). To date, BLM has performed
                 postimplementation reviews for two systems.

                 Figure 1 illustrates BLM’s IT investment management process phases and
                 stages. The highlighted stages represent those for which the ITIB must
                 make an approval decision before a project can move forward.




                 Page 8                        GAO-03-1025 BLM’s IT Investment Management Process
Figure 1: BLM’s IT Investment Management Process




                                       Page 9      GAO-03-1025 BLM’s IT Investment Management Process
ITIM Maturity Framework   On the basis of research into the IT investment management practices of
                          leading private- and public-sector organizations, we have developed an
                          information technology investment management (ITIM) maturity
                          framework.8 This framework identifies critical processes for successful IT
                          investments organized into a framework of five increasingly mature stages.
                          The ITIM is intended to be used as both a management tool for
                          implementing these processes incrementally and an evaluation tool for
                          determining an organization’s current level of maturity. The overriding
                          purpose of the framework is to encourage investment processes that
                          increase business value and mission performance, reduce risk, and
                          increase accountability and transparency in the decision process. This
                          framework has been used in several GAO evaluations9 and adopted by a
                          number of agencies. These agencies have used ITIM for purposes ranging
                          from self-assessment to redesign of their IT investment management
                          processes.

                          ITIM is a hierarchical model comprising five “maturity stages.” These
                          maturity stages represent steps toward achieving stable and mature
                          processes for managing IT investments. Each stage builds upon the lower
                          stages; the successful achievement of each stage leads to improvement in
                          the organization’s ability to manage its investments. With the exception of
                          the first stage, each maturity stage is composed of “critical processes” that
                          must be implemented and institutionalized for the organization to achieve
                          that stage. These critical processes are further broken down into key
                          practices that describe the types of activities that an organization should be
                          performing to successfully implement each critical process. An
                          organization may be performing key practices from more than one maturity
                          stage at one time. This is not unusual, but efforts to improve investment
                          management capabilities should focus on becoming compliant with lower
                          stage practices before addressing higher stage practices.


                          8
                          GAO/AIMD-10.1.23.
                          9
                           U.S. General Accounting Office, Information Technology: INS Needs to Strengthen Its
                          Investment Management Capability, GAO-01-146 (Washington, D.C.: Dec. 29, 2000);
                          Information Technology Management: Social Security Administration Practices Can Be
                          Improved, GAO-01-961 (Washington, D.C.: Aug. 21, 2001); Information Technology: DLA
                          Needs to Strengthen Its Investment Management Capability, GAO-02-314 (Washington,
                          D.C.: Mar. 15, 2002); United States Postal Service: Opportunities to Strengthen IT
                          Investment Management Capabilities, GAO-03-3 (Washington D.C.: Oct. 15, 2002);
                          Information Technology: Executive Office for U.S. Attorneys Needs to Institutionalize Key
                          IT Management Disciplines, GAO-03-751 (Washington, D.C.: July 25, 2003).




                          Page 10                            GAO-03-1025 BLM’s IT Investment Management Process
Stage 2 in the ITIM framework encompasses building a sound investment
management process—by developing the capability to control projects so
that they finish predictably within established cost and schedule
expectations—and establishing basic capabilities for selecting new IT
projects. Stage 3 requires that an organization continually assess proposed
and ongoing projects as parts of a complete investment portfolio: an
integrated and competing set of investment options. This approach enables
the organization to consider the relative costs, benefits, and risks of newly
proposed investments along with those previously funded and to identify
the optimal mix of IT investments to meet its mission, strategies, and goals.
Stages 4 and 5 require the use of evaluation techniques to continuously
improve both the investment portfolio and investment processes to better
achieve strategic outcomes. Figure 2 shows the five maturity stages and the
associated critical processes.



Figure 2: The Five Stages of Maturity with Critical Processes




As defined by the model, each critical process consists of “key practices”
that must be executed to implement the critical process.




Page 11                          GAO-03-1025 BLM’s IT Investment Management Process
BLM Has Established        In order to have the capabilities to effectively manage IT investments, an
                           agency should (1) have basic, project-level control and selection practices
Many of the Key            in place (stage 2 capabilities) and (2) manage its projects as a portfolio of
Practices for Effective    investments, treating them as an integrated package of competing
                           investment options and pursuing those that best meet the strategic goals,
Investment                 objectives, and mission of the agency (stage 3 capabilities). In addition, an
Management                 agency would be well served by implementing capabilities for improving its
                           investment management process (stage 4 capabilities).

                           BLM has executed the majority of the project-level control and selection
                           practices. The bureau has also initiated efforts to manage its projects as a
                           portfolio and performed two postimplementation reviews to learn lessons
                           to improve its investment management process. When BLM implements all
                           critical processes associated with building an investment foundation and
                           managing its projects as a portfolio, the bureau will have greater
                           confidence that it has selected the mix of projects that best supports its
                           strategic goals and that the projects will be managed to successful
                           completion.



BLM Has Implemented Most   At ITIM stage 2 maturity, an organization has attained repeatable,
of the Key Practices       successful IT project-level investment control processes and basic
                           selection processes. Through these processes, the organization can identify
Required to Establish      expectation gaps early and take appropriate steps to address them.
an IT Investment           According to ITIM, critical processes at stage 2 include (1) defining
Foundation                 investment review board10 operations, (2) collecting information about
                           existing investments, (3) developing project-level investment control
                           processes, (4) identifying the business needs for each IT project, and
                           (5) developing a basic process for selecting new IT proposals. Table 1
                           discusses the purpose for each of the stage 2 critical processes.




                           10
                             An investment review board is a decision-making body made up of senior program,
                           financial, and information managers that is responsible for making decisions about
                           investments or projects.




                           Page 12                            GAO-03-1025 BLM’s IT Investment Management Process
Table 1: Stage 2 Critical Processes—Building the Investment Foundation

Critical process                 Purpose
IT investment board operation    To define and establish the governing board(s)
                                 responsible for selecting, controlling, and evaluating IT
                                 investments.
IT project and system            To create and maintain an IT project inventory to assist in
identification                   managerial decision making.
IT project oversight             To regularly determine each IT project's progress toward
                                 cost and schedule milestones using established criteria
                                 and take corrective actions when milestones are not
                                 achieved.
Business needs identification    To ensure that each IT project supports the
for IT projects                  organization's business needs and meets users' needs.
Proposal selection               To ensure that an established, structured process is
                                 used to select new IT proposals.
Source: GAO.


To its credit, BLM has put in place about 85 percent of the key practices
associated with stage 2 critical processes.11 The bureau has satisfied all the
key practices associated with establishing the governing boards
responsible for managing IT investments and ensuring that IT projects
support organizational needs and meet users’ needs. It has satisfied a
majority of the key practices associated with proposal selection and IT
project oversight and is working on incorporating the use of an IT project
and system inventory into its IT investment management process. Table 2
summarizes the status of BLM’s critical processes for stage 2, showing how
many associated key practices it has executed.




11
 BLM hired a contractor to perform an independent assessment of its ITIM stage 2
capabilities, which was completed in January 2003. The assessment, which indicated that
BLM had established the majority of key practices for four out of five critical processes,
provided an important source of data for our evaluation.




Page 13                             GAO-03-1025 BLM’s IT Investment Management Process
                                Table 2: Summary of Results for Stage 2 Critical Processes and Key Practices

                                                                                Total required
                                                                Key practices       by critical    Percentage of key
                                Critical process                    executed          process     practices executed
                                IT investment board operation              6                 6                 100%
                                IT project and system                      3                 7                 42%
                                identification
                                IT project oversight                      10                11                 91%
                                Business needs identification              8                 8                 100%
                                Proposal selection                         5                 6                 83%
                                Total                                     32                38                 84%
                                Source: GAO.



BLM Has Established an          The creation of decision-making bodies or boards is central to the IT
Investment Board for Managing   investment management process. At the stage 2 level of maturity,
IT Investments                  organizations define one or more boards, provide resources to support
                                their operations, and appoint members who have expertise in both
                                operational and technical aspects of proposed investments. Resources
                                provided to support the operations of IT investment boards typically
                                include top management’s participation in creating the board(s) and
                                defining their scope and formal evidence acknowledging management’s
                                support for board decisions. The boards operate according to a written IT
                                investment process guide tailored to the organization’s unique
                                characteristics, thus ensuring that consistent and effective management
                                practices are implemented across the organization. Once board members
                                are selected, the organization ensures that they are knowledgeable about
                                policies and procedures for managing investments. Organizations at the
                                stage 2 level of maturity also take steps to ensure that executives and line
                                managers support and carry out the decisions of the IT investment board.
                                According to ITIM, an IT investment management process guide should be
                                a key authoritative document that the organization uses to initiate and
                                manage IT investment processes and should provide a comprehensive
                                foundation for policies and procedures developed for all other related
                                processes. (The complete list of key practices is provided in table 3.)

                                BLM has executed all the key practices for this critical process. For
                                example, in 1998, the bureau established an IT Investment Board (the ITIB)
                                to manage national investments. With the development of the IT
                                Investment Management Process guide in 2001, BLM provided the board
                                and all involved parties (i.e., project managers and sponsors, portfolio



                                Page 14                          GAO-03-1025 BLM’s IT Investment Management Process
managers, investment management group) with specifics concerning
responsibilities and procedures.12 This guide is centered on a project’s life
cycle and requisite decision points (phases and stages) in the investment
process from the submission of a proposal to a postimplementation review.

The board is also adequately resourced, with the main support being
provided by the System Coordination Office, whose responsibilities include
developing and modifying the bureau’s criteria for selecting, controlling,
and evaluating potential and existing IT investments and documenting,
recording, and transmitting decisions made by the board. Experienced
senior-level officials from both business and IT areas are members of the
board and exhibit the core competencies required by the investment
management process. Finally, all actions by the board are well documented
using meeting minutes and records of decision. In June 2003, an action-item
tracking matrix was introduced. This matrix is used to identify and track
ITIB-approved decisions and assigned responsibilities to ensure that the
board’s decisions are carried out. By executing all key practices associated
with creating and defining investment board operations, BLM has greater
assurance that the ITIB will effectively carry out its responsibilities.

Table 3 shows the rating for each key practice required to implement the
critical process for establishing IT investment board operation at the stage
2 level of maturity. Each of the “Executed” ratings shown below represents
instances where, based on the evidence provided by BLM officials, we
concluded that the specific key practices were executed by the
organization.




12
 According to BLM “Instruction Memorandum 2001-222,” this policy expired on September
30, 2002. BLM still continues to use this version to guide IT investment management efforts.
A newer version is currently in draft, but no expected release date was identified.




Page 15                             GAO-03-1025 BLM’s IT Investment Management Process
Table 3: IT Investment Board Operation

Type of practice   Key practice                  Rating     Summary of evidence
Organizational     1. An organization-specific Executed     BLM’s IT Investment Management Process guide and associated memo
commitments        IT investment process guide              issued in September of 2001 direct the national ITIB’s operations. The IT
                   is created to direct each                investment board’s charter updated in April 2003 also defines operating
                   board's operations.                      procedures for the board.



                   2. Organization executives    Executed   BLM established the System Coordination Office to, among other things,
                   and line managers support                support the investment management process. The office’s
                   and carry out IT investment              responsibilities include ensuring that the ITIB’s decisions are carried out
                   board decisions.                         by documenting and communicating these decisions and tracking
                                                            actions to support them. The ITIB recently adopted an action-item
                                                            tracking matrix to better track decisions made, assigned responsibilities,
                                                            and established time frames related to board decisions.
Prerequisites      1. Adequate resources are     Executed   A number of resources support the ITIB’s operations. They include the
                   provided for operating each              System Coordination Office which, among other things, screens IT
                   IT investment board.                     investment proposals and monitors project performance; the Investment
                                                            Management Group, which helps ensure that IT investments fit within
                                                            BLM’s budget plan; and an IT Portfolio Management Council that is
                                                            responsible for providing the board with an enterprise view of the
                                                            bureau’s various portfolios. The BLM intranet also features an “ITIB
                                                            Membership Guide Book” site that contains links to information on the
                                                            ITIM process and related resources.
                   2. Board members            Executed     Board members understand the investment board’s policies and
                   understand the investment                procedures and have experience in making investment management
                   board's policies and                     decisions. High-level training has been provided to members during past
                   procedures and exhibit core              board meetings on an informal basis. In addition, BLM plans to provide
                   competencies in using the                more formal training that addresses the core competencies that
                   IT investment approach via               members should have in order to more effectively contribute to the ITIM
                   training, education, or                  process. This first formal training session is expected to be conducted in
                   experience.                              November 2003.

Activities         1. Each IT investment board Executed     The ITIB membership includes both IT and business knowledge. It
                   is created and defined with              includes the CIO (also the Vice Chair); and representatives from the
                   board membership                         System Coordination Office; state, center, and field offices; Bureau
                   integrating both IT and                  Enterprise Architecture Team; and budget areas.
                   business knowledge.
                   2. Each IT investment board Executed     ITIB operations are guided by BLM’s IT Investment Management
                   operates according to                    Process guide, its associated instruction memo, and the ITIB charter.
                   written policies and                     The guide specifies procedures for selecting, controlling, and evaluating
                   procedures in the                        investments, while the ITIB charter contains specific operating
                   organization-specific IT                 procedures.
                   investment process guide.
Source: GAO.




                                             Page 16                           GAO-03-1025 BLM’s IT Investment Management Process
BLM Has Not Defined Policies    An IT project and system inventory provides information to investment
and Procedures for Collecting   decision makers to help evaluate the impacts and opportunities created by
Information into the Budget     proposed or continuing investments. This inventory (which can take many
Planning System to Make         forms) should, at a minimum, identify the organization’s IT projects
Informed Investment             (including new and existing systems) and a defined set of relevant
Management Decisions            investment management information about them (e.g., purpose, owner, life-
                                cycle stage, budget cost, physical location, and interfaces with other
                                systems). Information from the IT project and system inventory can, for
                                example, help identify systems across the organization that provide similar
                                functions and help avoid the commitment of additional funds for redundant
                                systems and processes. It can also help determine more precise
                                development and enhancement costs by informing decision makers and
                                other managers of interdependencies among systems and how potential
                                changes in one system can affect the performance of other systems.
                                According to ITIM, effectively managing an IT project and system inventory
                                requires, among other things, (1) identifying projects and systems,
                                collecting relevant information about them, and capturing this information
                                in a repository; (2) assigning responsibility for managing the inventory
                                process and ensuring that the inventory meets the needs of the investment
                                management process; (3) developing written policies and procedures for
                                maintaining the project and system inventory; (4) making information from
                                the inventory available to staff and managers throughout the organization
                                so they can use it, for example, to build business cases and support
                                activities to select and control projects; and (5) maintaining the inventory
                                and its information records to contribute to future investment selections
                                and assessments. (The full list of key practices is provided in table 4.)

                                BLM has executed three out of seven of the key practices for IT project and
                                system identification. For example, the bureau is using its target
                                application architecture and Budget Planning System13 to collect
                                information on its IT projects and systems to make informed IT investment
                                management decisions; according to CIO officials, the architecture is used
                                for the information it contains on BLM’s business processes and supporting
                                data, applications, and technology, while the Budget Planning System is
                                used for the financial information on the investments. Resources have been
                                assigned to support activities related to identifying IT projects and systems,
                                including the Bureau Enterprise Architecture Team and the Budget


                                13
                                 The Budget Planning System is part of BLM's larger Financial Management System. It
                                contains financial data on individual investments. These data can be directly tied to annual
                                budget submissions.




                                Page 17                             GAO-03-1025 BLM’s IT Investment Management Process
Planning System system owners. According to BLM, all national projects
and systems are in both the target application architecture and Budget
Planning System (although BLM officials told us that they have planned a
meeting to determine whether additional requirements are needed for the
Budget Planning System to effectively serve as an inventory for investment
management purposes).

Despite these strengths, policies and procedures for collecting project and
system information in the Budget Planning System for investment
management purposes have not yet been defined. However, the CIO has
directed teams composed of the System Coordination Office, portfolio
managers, the Investment Management Group, and system owners of the
Budget Planning System to “identify the ownership of each process
associated with the IT project and system inventory.” This step would form
the basis for policies and procedures relating to the collection (and use) of
information in the inventory. Until BLM defines these policies and
procedures, it cannot adequately ensure that its inventory can be relied
upon as an effective tool to assist in investment decision making. Table 4
shows the rating for each key practice required to implement the critical
process for IT project and system identification at the stage 2 level of
maturity and summarizes the evidence that supports these ratings.




Page 18                       GAO-03-1025 BLM’s IT Investment Management Process
Table 4: IT Project and System Identification

Type of practice Key practice                             Rating         Summary of evidence
Organizational   1. The organization has written policies Not executed   BLM is using its enterprise architecture and Budget
commitments      and procedures for identifying its IT                   Planning System to collect information about its IT
                 projects and systems and collecting, in                 projects and systems. However, the bureau has not yet
                 an inventory, information about the IT                  defined the policies and procedures associated with
                 projects and systems that is relevant to                collecting information in the Budget Planning System for
                 the investment management process.                      investment management purposes.

                 2. An official is assigned responsibility Executed      According to the ITIB charter, it is the ITIB’s responsibility
                 for managing the IT project and system                  to manage the IT project and system identification process
                 identification process and ensuring that                and ensure that the inventory meets the needs of the
                 the inventory meets the needs of the                    investment management process. According to BLM’s
                 investment management process.                          self-assessment, in practice, this responsibility has been
                                                                         delegated to the Bureau Enterprise Architecture Team for
                                                                         the architecture and to the System Coordination Office for
                                                                         the Budget Planning System.

Prerequisite     1. Adequate resources are provided for Executed         Resources for IT project and system identification
                 identifying IT projects and systems and                 activities include contractor support for the enterprise
                 collecting relevant information into an                 architecture and the Budget Planning System system
                 inventory.                                              owners.
Activities       1. The organization's IT projects and    Executed       According to BLM officials, all national projects and
                 systems are identified, and specific                    systems are in the enterprise architecture’s target
                 information about them is collected in                  application component and in the Budget Planning
                 an inventory.                                           System.
                 2. Changes to IT projects and systems Not executed      According to BLM, changes to projects and systems are
                 are identified, and change information                  not reliably being maintained in the inventory.
                 is maintained in the inventory.
                 3. Information from the inventory is     Not executed   Information from the enterprise architecture and the
                 available on demand to decision                         Budget Planning System is available to decision makers
                 makers and other affected parties.                      and other affected parties through BLM’s intranet site.
                                                                         However, this information is not reliable because,
                                                                         according to BLM, it is not being maintained in a
                                                                         consistent manner (see activity 2).
                 4. The IT project and system inventory Not executed     According to BLM, changes to projects and systems are
                 and its information records are                         not reliably being maintained in the inventory.
                 maintained to contribute to future
                 investment selections and
                 assessments.

Source: GAO.



Projects’ Progress Toward Cost               Investment boards should effectively oversee IT projects throughout all
and Schedule Milestones Is                   life-cycle phases (concept, design, testing, implementation, and
Regularly Determined                         operations/maintenance). At the stage 2 level of maturity, investment
                                             boards should review each project’s progress toward predefined cost and



                                             Page 19                          GAO-03-1025 BLM’s IT Investment Management Process
schedule expectations, using established criteria and performance
measures, and take corrective actions to address cost and milestone
variances. According to ITIM, effective project oversight requires, among
other things, (1) having written policies and procedures for project
management; (2) developing and maintaining an approved management
plan for each IT project; (3) making up-to-date cost and schedule data for
each project available to the oversight boards; (4) reviewing each project’s
performance by regularly comparing actual cost and schedule data with
expectations; (5) ensuring that corrective actions for each under-
performing project are documented, agreed to, implemented, and tracked
until the desired outcome is achieved; and (6) having written policies and
procedures for oversight of IT projects. (The complete list of key practices
is provided in table 5.)

BLM has in place all but one of the key practices associated with effective
project oversight. Project management policies and high-level procedures
are defined in the IT Investment Management Process guide, associated
memoranda, and in best practices guidance. In addition, project oversight
policies and procedures are defined in the guide and associated
memoranda, which, among other things, require the involvement and
approval of the board at key stages in a project’s life cycle. For example,
according to the guide, the ITIB must review and approve investment
proposals before they can be developed into business cases. Further, once
a project has been approved, the ITIB reviews up-to-date cost, schedule,
and scope information quarterly and analyzes this information against
predetermined performance expectations. The board also determines
corrective actions for projects that have not met performance
expectations.

We verified that cost, schedule, and scope information was submitted to
the ITIB quarterly and analyzed against expectations and, when significant
variances occurred, corrective actions were determined for the three
projects we reviewed.14 This involved rebaselining the project plan based
on schedule slippages or increased costs. In all cases, the ITIB was
responsible for this decision.

Notwithstanding these strengths, as discussed in the previous section,
BLM’s IT project and systems inventory has not yet been developed to the


14
 The three projects we reviewed—PayCheck, National Integrated Land System, and
Antivirus—are described in appendix I.




Page 20                          GAO-03-1025 BLM’s IT Investment Management Process
                                                point where information is consistently collected and maintained to make
                                                informed investment management decisions. This increases the risk that
                                                the ITIB will not have at its disposal reliable information for supporting
                                                project and portfolio investment decisions and oversight.

                                                Table 5 shows the rating for each key practice required to implement the
                                                critical process for project oversight at the stage 2 level of maturity and
                                                summarizes the evidence that supports these ratings.



Table 5: IT Project Oversight

Type of
practice         Key practice                        Rating     Summary of evidence
Organizational   1. The organization has written     Executed   BLM’s policy and procedures for managing IT investments are defined
commitments      policies and procedures for                    in the IT Investment Management Process guide and its associated
                 project management.                            instruction memo.
                 2. The organization has written     Executed   The ITIB charter and IT Investment Management Process guide
                 policies and procedures for                    specify that the board shall perform IT project oversight by regularly
                 management oversight of IT                     analyzing each IT investment’s planned progress toward cost and
                 projects.                                      schedule milestones against actual progress/performance. Instruction
                                                                memos define procedures for submitting up-to-date project
                                                                information to the ITIB quarterly. In addition, specific procedures for
                                                                determining corrective actions for projects that have not met
                                                                predetermined performance standards were approved and adopted
                                                                by the ITIB in June 2003.
Prerequisites    1. Adequate resources are          Executed    Adequate resources are provided to assist the board in overseeing IT
                 provided to assist the board(s) in             projects. Specifically, the System Coordination Office has the primary
                 overseeing IT projects.                        responsibility of maintaining an integrated project schedule and
                                                                conducting IT project oversight on all investments as they move
                                                                through the ITIB decision process.
                 2. Each IT project has and          Executed   The IT Investment Management Process guide requires that the
                 maintains an approved project                  project management plan be developed so that investments can be
                 management plan that includes                  managed to achieve technical cost, schedule, performance, and risk
                 cost and schedule controls.                    management objectives. The three projects we reviewed have project
                                                                management plans. In addition, we verified that the ITIB and System
                                                                Coordination Office applied cost and schedule controls in their
                                                                quarterly reviews of these projects.
                 3. An IT investment board is        Executed   BLM’s board has been in operation since 1998. It is responsible for
                 operating.                                     selecting, controlling, and evaluating all national IT investments.
                 4. Information from the IT project Not         Information from the IT project and system inventory is available to
                 and system inventory is used by executed       the ITIB. However, according to BLM, this information is not being
                 the IT investment board as                     maintained reliably.
                 applicable.




                                                Page 21                         GAO-03-1025 BLM’s IT Investment Management Process
(Continued From Previous Page)
Type of
practice        Key practice                       Rating     Summary of evidence
Activities      1. Each project’s up-to-date cost Executed    BLM has a policy memo requiring project managers to submit up-to-
                and schedule data are provided                date cost, schedule, and scope information to the System
                to the appropriate IT investment              Coordination Office quarterly, and this information is provided to the
                board.                                        ITIB. We verified that this was the case for the three projects that we
                                                              reviewed.
                2. Using established criteria, the Executed   The ITIB oversees each IT project’s performance by comparing actual
                IT investment board regularly                 cost, schedule, and scope data to expectations quarterly. Quarterly
                oversees each IT project’s                    project status reports indicate whether reported data fall within an
                performance by comparing                      acceptable range when compared to expected cost and schedule
                actual cost and schedule data to              (red/yellow/green). We verified that this was the case for the three
                expectations.                                 projects that we reviewed.
                3. The IT investment board         Executed   Special reviews were conducted for the three projects we reviewed
                performs special reviews of                   when they did not meet predetermined performance standards.
                projects that have not met
                predetermined performance
                standards.
                4. Appropriate corrective actions Executed    For each of the projects we reviewed, there are ITIB decision papers
                for each under-performing                     documenting the corrective actions that have been agreed upon by
                project are defined, documented,              the ITIB and project manager. These actions include rebaselining the
                and agreed to by the IT                       project plan to account for schedule slippage or increased costs.
                investment board and the project
                manager.
                5. Corrective actions are        Executed     The board recently approved an action-item tracking matrix that tracks
                implemented and tracked until                 board decisions, parties responsible, and associated time frames.
                the desired outcome is achieved.              According to project documentation, corrective actions were tracked
                                                              until implemented for the three projects we reviewed.
Source: GAO.



Processes Ensure That IT                      Defining business needs for each IT project helps ensure that projects
Projects Support Business Needs               support the organization’s mission goals and meet users’ needs. This
and Meet Users’ Needs                         critical process creates the link between the organization’s business
                                              objectives and its IT management strategy. According to ITIM, effectively
                                              identifying business needs requires, among other things, (1) defining the
                                              organization’s business needs or stated mission goals, (2) identifying users
                                              for each project who will participate in the project’s development and
                                              implementation, (3) training IT staff adequately in identifying business
                                              needs, and (4) defining business needs for each project. (The complete list
                                              of key practices is provided in table 6.)

                                              BLM has executed all the key practices for this critical process. The
                                              bureau’s IT Investment Management Process guide requires that business
                                              needs and associated users of each IT project be identified in the
                                              investment proposal and business case stages of the select phase. BLM also
                                              has detailed procedures for developing these two documents that call for



                                              Page 22                         GAO-03-1025 BLM’s IT Investment Management Process
identifying business needs and associated users. Resources for identifying
business needs and associated users include the project sponsor, project
proponent, and the System Coordination Office and detailed procedures
and associated templates for developing investment proposals and
business cases. Bureau specific business needs are defined in the Bureau
of Land Management’s Strategic Plan for fiscal years 2000–2005, and
projects are also often linked to the Department of the Interior’s strategic
goals and goals of the President’s Management Agenda.15 In addition,
individuals responsible for managing projects at BLM can adequately
identify business needs; according to BLM, the bureau’s practice is to select
staff from business units as project managers (instead of staff from the IT
unit). Finally, according to BLM officials, all national applications are in the
Budget Planning System and target enterprise architecture, which are
repositories of investment information that BLM is planning on using to
make informed investment management decisions. For the three projects
we reviewed, business needs and associated users were identified in
business case or project planning documents, and users were involved in
project management throughout the life cycle of the project through, for
example, chartered user-group meetings, structured walk-throughs,
prerelease workshops, and conference calls. Because it is executing all key
practices associated with business needs identification, BLM can have
greater assurance that its projects will support business needs and meet
users’ needs.

Table 6 shows the rating for each key practice required to implement the
critical process for business needs identification at the stage 2 level of
maturity and summarizes the evidence that supports these ratings.




15
 The President's Management Agenda is a strategy for improving the management and
performance of the federal government. The Agenda contains five governmentwide and nine
agency-specific goals to improve federal management and deliver results that matter to the
American people.




Page 23                            GAO-03-1025 BLM’s IT Investment Management Process
Table 6: Business Needs Identification

Type of practice Key practice                            Rating     Summary of evidence
Organizational   1. The organization has written     Executed       BLM has guidance for developing investment proposals and
commitment       policies and procedures for                        business cases that calls for identifying the business needs (and
                 identifying the business needs (and                the associated users) of each IT project.
                 the associated users) of each IT
                 project.
Prerequisites    1. Adequate resources are provided Executed        BLM has adequate resources for identifying business needs
                 for identifying business needs and                 and associated users. They include the project managers,
                 associated users.                                  portfolio managers, and the System Coordination Office.
                 2. The organization has defined         Executed   The Bureau of Land Management’s Strategic Plan for fiscal
                 business needs or stated mission                   years 2000–2005 defines the agency’s mission goals.
                 goals.
                 3. IT staff are trained in business     Executed   According to BLM officials, individuals responsible for managing
                 needs identification.                              projects at BLM generally come from the business/program
                                                                    areas. Therefore, they can adequately identify business needs.
                 4. IT projects and systems are          Executed   According to BLM officials, all national IT investments and most
                 identified in the IT project and                   state and center IT investments are in the enterprise
                 system inventory.                                  architecture and Budget Planning System, which together
                                                                    represent the IT project and system inventory.
Activities       1. The business needs for each IT       Executed   BLM requires that business needs for each IT project be
                 project are clearly identified and                 identified in investment proposals and business cases. We
                 defined.                                           verified that the business needs for the three projects we
                                                                    reviewed were clearly identified and defined.
                 2. Specific users are identified for    Executed   BLM requires that specific users be identified for each IT project
                 each IT project.                                   in investment proposals and business cases. We verified that
                                                                    specific users were identified for the three projects we reviewed.
                 3. Identified users participate in      Executed   According to BLM officials, users are involved in project
                 project management throughout a                    management throughout a project’s life cycle. For the three
                 project’s life cycle.                              projects were reviewed, users were involved in project
                                                                    management through various types of meetings.
Source: GAO.



Proposal Selection Process Is                  Selecting new IT proposals requires an established and structured process
Established, but Criteria for                  to ensure informed decision making and infuse management
Analyzing and Prioritizing                     accountability. According to ITIM, this critical process requires, among
Investments Have Not Been                      other things, (1) making funding decisions for new IT proposals according
Fully Defined                                  to an established process; (2) providing adequate resources to proposal
                                               selection activities; (3) using an established proposal selection process;
                                               (4) analyzing and ranking new IT proposals according to established
                                               selection criteria, including cost and schedule criteria; and (5) designating
                                               an official to manage the proposal selection process. (The complete list of
                                               key practices is provided in table 7.)




                                               Page 24                        GAO-03-1025 BLM’s IT Investment Management Process
BLM has executed five of the six key practices associated with proposal
selection. For example, the IT Investment Management Process guide
defines a multistage selection process (the select phase), including
developing a business case and project plan that executives and project
proponents follow. Resources for proposal selection activities include
project proponents and the System Coordination Office. As previously
noted, detailed procedures and a template have been defined for
developing new IT proposals. In addition, according to BLM officials,
funding decisions are made through the budget process, which BLM is
working on to better integrate with the investment management process.

Despite these strengths, the key practice associated with analyzing and
prioritizing new proposals according to established criteria has not yet
been executed because the various criteria for doing so have not yet been
fully defined. The Bureau Enterprise Architecture Team has defined criteria
it uses to determine projects’ compliance with the bureau enterprise
architecture’s business processes, data, applications, and technology
components. Another set of criteria is being developed to assess proposals
for their business value. These criteria are intended to be used by the IT
Portfolio Management Council to screen proposals before they are
reviewed by the ITIB. The ITIB members believe that these criteria are key
to analyzing new proposals and have charged the IT Portfolio Management
Council with developing draft criteria. The council intends to finalize the
criteria in time for their use in the spring of 2004 to evaluate proposals for
the fiscal year 2006 budget. Until BLM finalizes its proposal selection
criteria and uses them to analyze and prioritize proposals, the bureau will
not be adequately assured that it is consistently and objectively selecting
proposals that best meet the needs and priorities of the agency.

Table 7 shows the rating for each key practice required to implement the
critical process for proposal selection at the stage 2 level of maturity and
summarizes the evidence that supports these ratings.




Page 25                       GAO-03-1025 BLM’s IT Investment Management Process
Table 7: Proposal Selection

Type of practice   Key practice                                 Rating         Summary of evidence
Organizational     1. Executives and managers follow an         Executed       BLM’s IT Investment Management Process guide
commitments        established selection process.                              defines a selection process that executives and
                                                                               project proponents follow.
                   2. An official is designated to manage the   Executed       The IT Investment Management Process guide and
                   proposal selection process.                                 the ITIB charter assign this responsibility to the
                                                                               ITIB.
Prerequisite       1. Adequate resources are provided for       Executed       Adequate resources are provided for proposal
                   proposal selection activities.                              selection activities. They include the project
                                                                               proponents and portfolio managers who develop
                                                                               investment proposals and business cases and the
                                                                               System Coordination Office who supports them in
                                                                               these activities.
Activities         1. The organization uses a structured        Executed       BLM has defined procedures for developing new IT
                   process to develop new IT proposals.                        proposals.
                   2. Executives analyze and prioritize new IT Not executed    Executives analyze and prioritize new IT proposals;
                   proposals according to established selection                however, according to BLM officials, prioritization is
                   criteria.                                                   not based on established criteria.
                   3. Executives make funding decisions for     Executed       According to BLM officials, executives make
                   new IT proposals according to an                            funding decisions for new IT proposals through
                   established process.                                        BLM’s budget process.

Source: GAO.




BLM Has Initiated Efforts to                 Once an agency has attained stage 2 maturity, it needs to establish
Manage Its Investments as a                  capabilities for managing its investments as a portfolio (stage 3). Such
                                             capabilities enable the agency to consider its investments comprehensively
Portfolio and Improve Its
                                             so that the collective investments optimally address its mission, strategic
Investment Management                        goals, and objectives. Stage 3 capabilities include (1) defining portfolio
Process                                      selection criteria, (2) engaging in project-level investment analysis, and (3)
                                             aligning the authority of IT investment boards. In addition, establishing
                                             higher level stage capabilities—for example performing
                                             postimplementation reviews—can help an agency improve its investment
                                             management process.

                                             BLM has initiated efforts to manage its investments as a portfolio. BLM has
                                             defined procedures for aligning the national ITIB with subordinate
                                             boards.16 Portfolio categories have been defined that correspond to BLM’s
                                             organizational units (e.g., Minerals, Realty, and Resource Protection;
                                             Information Resource Management). Moreover, the board revised its
                                             charter in April 2003 to include portfolio management responsibilities (i.e.,



                                             Page 26                          GAO-03-1025 BLM’s IT Investment Management Process
                      stage 3) and established an IT Portfolio Management Council to support it
                      in carrying out these responsibilities.17 At a higher level stage, BLM has
                      begun to address postimplementation reviews, a critical process associated
                      with the most capable organizations. BLM has begun performing
                      postimplementation reviews on a limited basis to learn lessons for
                      improving both project management and investment management
                      processes. To date, the bureau has conducted two such reviews.

                      Compared with the progress at stage 2, BLM’s progress to date in defining
                      practices for higher level maturity stages has been limited because,
                      according to its officials, the ITIB first focused its resources on establishing
                      the processes associated with building the IT investment management
                      foundation. Full implementation of the critical processes associated with
                      portfolio management will provide BLM with the capability to determine
                      whether it is selecting the mix of products that best meet the bureau’s
                      mission needs. Implementing critical processes at higher level stages will
                      equip BLM with the capabilities it needs to improve its investment
                      management processes.



BLM Does Not Have a   We have previously reported that to effectively implement ITIM processes,
                      agencies need to be guided by a plan that (1) is based on an assessment of
Plan to Guide IT      strengths and weaknesses; (2) specifies measurable goals, objectives, and
Investment            milestones; (3) specifies needed resources; and (4) assigns clear
                      responsibility and accountability for accomplishing well-defined tasks. In
Management            addition, these plans should be approved by senior management.18
Improvement Efforts
                      Although a plan was developed a few years ago to establish the practices
                      currently in place, BLM does not have a plan to guide further improvement
                      of its investment management process. An independent assessment of
                      BLM's ITIM process relative to stage 2 of our IT investment management
                      framework was completed in January 2003, but BLM has not yet used the
                      results of this assessment to develop an improvement plan. According to
                      the CIO, this is because BLM intends to develop a plan integrating

                      16
                       Most state and center offices have their own boards for managing IT investments that fall
                      below the thresholds of the national IT investments’ criteria.
                      17
                        The Portfolio Management Council was formed in July 2002 and has met several times
                      since. The council’s charter was approved in June 2003.
                      18
                           GAO-02-314, GAO-03-3, GAO-03-751.




                      Page 27                             GAO-03-1025 BLM’s IT Investment Management Process
                      improvements for IT investment management and other IT management
                      areas, and the results of the comprehensive assessment to be used as a
                      basis for this integrated plan were not received until June 2003. BLM
                      officials recognize the importance of having a plan to guide their
                      improvement efforts, however, and stated their commitment to developing
                      one, although they do not have a specific time frame for doing so. Until
                      BLM develops this plan, the bureau risks losing the momentum it has
                      gained in implementing its ITIM process.



Conclusions           BLM has made good progress in defining and establishing its investment
                      management process in the 2 years since we reported that the lack of such
                      a process had largely contributed to the failure of a key program. By
                      establishing most of the key practices associated with building the
                      investment foundation, the bureau has strengthened its basic capabilities
                      for selecting and controlling projects and positioned itself to develop the
                      processes for managing its investments as a portfolio. Critical to BLM’s
                      success going forward will be the development of an implementation
                      plan—preferably integrated with implementation plans for improving other
                      IT management areas—to (1) guide and establish accountability for
                      executing the stage 2 key practices that we noted needed to be addressed
                      and (2) proceeding with efforts to define and implement stage 3 key
                      practices. Without this plan, BLM risks not being able to sustain the
                      progress made to date in establishing its investment management process.



Recommendations for   To strengthen BLM’s IT investment management capability and address the
                      weaknesses discussed in this report, we recommend that the Secretary of
Executive Action      the Department of the Interior direct the BLM Director to develop and
                      implement a plan for improving its IT investment management process that
                      is based on GAO’s ITIM stage 2 and 3 critical processes. The plan should, at
                      a minimum, provide for accomplishing the following:

                      • implementing the recently approved procedures for determining
                        corrective actions for projects that have not met performance
                        expectations;

                      • defining and implementing policies and procedures for collecting
                        project and system information in the Budget Planning System for
                        investment management purposes;




                      Page 28                       GAO-03-1025 BLM’s IT Investment Management Process
                      • fully defining criteria for analyzing and prioritizing new IT proposals;
                        and

                      • proceeding with plans to define and implement all stage 3 critical
                        processes, which are necessary for portfolio management.

                      In developing the plan, the BLM Director should ensure that it (1) is based
                      on the results of the bureau’s recent assessment of ITIM stage 2
                      capabilities; (2) specifies measurable goals, objectives, milestones, and
                      outcomes; (3) specifies needed resources; and (4) assigns clear
                      responsibility and accountability for accomplishing well-defined tasks. In
                      implementing the plan, the Director should ensure that the needed
                      resources are provided and that progress is measured and reported
                      periodically to the Secretary of the Interior.



Agency Comments and   In written comments on a draft of this report (reprinted in app. II), BLM’s
                      Director agreed with our findings and recommendations and stated that
Our Evaluation        they represented a fair and accurate evaluation of the bureau’s status and
                      progress towards IT investment management maturity. The Director also
                      noted that BLM has begun developing a plan, in accordance with our
                      recommendations, to (1) complete the key practices for reaching GAO’s
                      ITIM stage 2 maturity and (2) identify the goals, time frames, outcomes,
                      and resources needed to reach stage 3 maturity. BLM provided additional
                      technical comments, which we have incorporated into the report as
                      appropriate.


                      We are sending copies of this report to interested congressional
                      committees. We are also sending copies to the Director of the Office of
                      Management and Budget, the Secretary of the Interior, and BLM’s Director
                      and CIO. We also will make copies available to others upon request. In
                      addition, the report will be available at no charge on the GAO Web site at
                      www.gao.gov.

                      Should you or your offices have questions on matters discussed in this
                      report, please contact me at (202) 512-6240, or Lester P. Diamond, Assistant




                      Page 29                       GAO-03-1025 BLM’s IT Investment Management Process
Director, at (202) 512-7957. We can also be reached by E-mail at at
koontzl@gao.gov, or diamondl@gao.gov, respectively. Key contributors to
this assignment were Jamey A. Collins, Sabine R. Paul, and Sophia
Harrison.




Linda D. Koontz
Director, Information Management Issues




Page 30                     GAO-03-1025 BLM’s IT Investment Management Process
Appendix I

Objectives, Scope, and Methodology                                                           AA
                                                                                              ppp
                                                                                                ep
                                                                                                 ned
                                                                                                   n
                                                                                                   x
                                                                                                   id
                                                                                                    e
                                                                                                    x
                                                                                                    Iis




              The objectives of our review were to (1) evaluate the Bureau of Land
              Management’s (BLM) IT investment management capabilities against the
              key practices defined in GAO’s IT investment management assessment
              framework and (2) determine the agency’s plans for improving these
              capabilities.

              To address our first objective, we assessed the extent to which BLM
              satisfied the five critical processes identified in stage 2 of GAO’s
              Information Technology Investment Management (ITIM) framework.1 We
              applied the framework as it is described in the exposure draft, except that
              we used a revised version of the IT Asset Inventory critical process, called
              IT Project and System Identification, after discussions with departmental
              officials at the beginning of this engagement. This revised critical process
              has been used in our evaluations since June 2001. We did not formally
              assess BLM’s progress in establishing capabilities found in stages 3, 4, and 5
              because BLM acknowledged that it had so far primarily focused on stage 2
              and had not executed many key practices in higher maturity stages. In
              addition, we limited our review to BLM’s management of its national
              investments because they represent the investments of greater cost and
              impact to the organization.

              To determine whether BLM had implemented the critical processes
              associated with stage 2, we reviewed the results of a self-assessment of
              stage 2 practices using GAO’s ITIM framework and validated and updated
              the results of the self-assessment through document reviews and
              interviews with officials. We reviewed written policies, procedures, and
              guidance and other documentation providing evidence of executed
              practices, including BLM’s IT Investment Management Process guide,
              various board/council charters, and instruction memorandums. We also
              reviewed national Information Technology Investment Board (ITIB)
              meeting materials, including quarterly status reports, meeting minutes,
              records of decision, and matrices tracking action items through
              completion. We interviewed several BLM officials, including system
              coordination office officials, portfolio managers, and ITIB members. We
              also attended a 2-day national ITIB meeting in March 2003.

              As part of our analysis, we selected three IT projects as case studies to
              verify application of the critical processes and practices. We selected
              projects that (1) supported different BLM functional areas (directorates),


              1
              GAO/AIMD-10.1.23.




              Page 31                       GAO-03-1025 BLM’s IT Investment Management Process
Appendix I
Objectives, Scope, and Methodology




(2) were in different life-cycle phases, and (3) required various levels of
funding. The three projects are the following:

• PayCheck—The objective of PayCheck, currently in the evaluate phase,
  is to allow employees to input their time and attendance data using an
  automated system. Previously, the employee developed a hard-copy
  time sheet, and then a timekeeper keyed the same information into the
  payroll system. By allowing the employees to enter their own data,
  PayCheck changed the business process and eliminated the duplication
  of manual effort. This project, with an estimated life-cycle cost of
  $1,681,000, is the responsibility of the National Human Resources
  Management Center, which supports BLM’s human resources function.

• National Integrated Land System—The objective of this system, which is
  currently in a mixed life cycle,2 is to provide a process to collect,
  maintain, and store survey- and parcel-based land information that
  meets the common, shared business needs of land title and land
  resource management. This system will provide agencies, BLM’s
  partners, and the public with business solutions for the management of
  cadastral records and land parcel information in a Geographic
  Information System environment, accessible via the Internet. This
  project, which has an estimated life-cycle cost of $31.3 million, is under
  the Minerals, Realty, and Resource Protection directorate. This
  directorate is responsible for managing commercial energy and mineral
  production from the public lands.

• Antivirus—The objective of Antivirus, currently in the control phase, is
  to renew or replace BLM’s existing antivirus contract that is expiring to
  provide antivirus coverage to all simple mail transfer protocol gateways,
  mail servers, other servers, desktops, and laptops. In addition, BLM is
  seeking to provide improved enterprise management and reporting
  capabilities, as well as a more automated methodology for deploying
  virus update files across the bureau. This project is being carried out by
  the Information Resources Management directorate, which provides IT
  services to BLM states, centers, and partners in support of the bureau’s
  mission. The estimated life cycle cost for Antivirus is $800,600.




2
 The National Integrated Land System has four different modules. Three of these modules
are in the “control” phase and the fourth is in the “evaluate” phase.




Page 32                              GAO-03-1025 BLM’s IT Investment Management Process
Appendix I
Objectives, Scope, and Methodology




For these projects, we reviewed project management documentation, such
as business cases, project plans, and quarterly reports. We also reviewed
user-group meeting minutes and analyzed national ITIB decision
documents related to each of the projects. We also interviewed the project
managers for these projects.

We compared the evidence collected from our document reviews and
interviews to the key practices in ITIM. We rated the key practices as
“executed” on the basis of whether the agency demonstrated (by providing
evidence of performance) that it had met the criteria of the key practice. A
key practice was rated as ”not executed” when we found insufficient
evidence of a practice during the review, or when we determined that there
were significant weaknesses in BLM’s execution of the key practice.

To address our second objective, we interviewed officials from the System
Coordination Office, whose main responsibility it is to oversee and ensure
that BLM’s IT investment management process is implemented and
followed; the chief information officer; and other national ITIB members to
determine efforts undertaken to improve IT investment management
processes. We also reviewed an improvement plan developed about 3 years
ago based on strengths and weaknesses of BLM’s IT investment
management process at that time, and the results of the comprehensive IT
management assessment BLM officials stated they plan to use as a basis for
an integrated plan for improving IT investment management and other IT
management areas.

We conducted our work at BLM Headquarters in Washington, D.C., from
March through July 2003, in accordance with generally accepted
government auditing standards.




Page 33                              GAO-03-1025 BLM’s IT Investment Management Process
Appendix II

Comments from the Bureau of Land
Management                                                               Appendx
                                                                               Ii




(310363)      Page 34   GAO-03-1025 BLM’s IT Investment Management Process
GAO’s Mission            The General Accounting Office, the audit, evaluation and investigative arm of
                         Congress, exists to support Congress in meeting its constitutional responsibilities
                         and to help improve the performance and accountability of the federal government
                         for the American people. GAO examines the use of public funds; evaluates federal
                         programs and policies; and provides analyses, recommendations, and other
                         assistance to help Congress make informed oversight, policy, and funding
                         decisions. GAO’s commitment to good government is reflected in its core values of
                         accountability, integrity, and reliability.


Obtaining Copies of      The fastest and easiest way to obtain copies of GAO documents at no cost is
                         through the Internet. GAO’s Web site (www.gao.gov) contains abstracts and full-
GAO Reports and          text files of current reports and testimony and an expanding archive of older
                         products. The Web site features a search engine to help you locate documents
Testimony                using key words and phrases. You can print these documents in their entirety,
                         including charts and other graphics.
                         Each day, GAO issues a list of newly released reports, testimony, and
                         correspondence. GAO posts this list, known as “Today’s Reports,” on its Web site
                         daily. The list contains links to the full-text document files. To have GAO e-mail this
                         list to you every afternoon, go to www.gao.gov and select “Subscribe to
                         e-mail alerts” under the “Order GAO Products” heading.


Order by Mail or Phone   The first copy of each printed report is free. Additional copies are $2 each. A check
                         or money order should be made out to the Superintendent of Documents. GAO
                         also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a single
                         address are discounted 25 percent. Orders should be sent to:
                         U.S. General Accounting Office
                         441 G Street NW, Room LM
                         Washington, D.C. 20548
                         To order by Phone:     Voice: (202) 512-6000
                                                TDD: (202) 512-2537
                                                Fax: (202) 512-6061


To Report Fraud,         Contact:
                         Web site: www.gao.gov/fraudnet/fraudnet.htm
Waste, and Abuse in      E-mail: fraudnet@gao.gov
Federal Programs         Automated answering system: (800) 424-5454 or (202) 512-7470



Public Affairs           Jeff Nelligan, Managing Director, NelliganJ@gao.gov (202) 512-4800
                         U.S. General Accounting Office, 441 G Street NW, Room 7149
                         Washington, D.C. 20548
United States                  Presorted Standard
General Accounting Office      Postage & Fees Paid
Washington, D.C. 20548-0001           GAO
                                 Permit No. GI00
Official Business
Penalty for Private Use $300
Address Service Requested