oversight

Land Management Systems: Major Software Development Does Not Meet BLM's Business Needs

Published by the Government Accountability Office on 1999-03-04.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                          United States General Accounting Office

GAO                       Testimony
                          Before the Subcommittee on Interior and Related
                          Agencies, Committee on Appropriations, House of
                          Representatives


For Release on Delivery
Expected at
9 a.m.
                          LAND MANAGEMENT
Thursday,
March 4, 1999             SYSTEMS

                          Major Software
                          Development Does Not
                          Meet BLM’s Business
                          Needs
                          Statement of Joel C. Willemssen
                          Director, Civil Agencies Information Systems
                          Accounting and Information Management Division




GAO/T-AIMD-99-102
                         Mr. Chairman and Members of the Subcommittee:

                         Thank you for inviting us to participate in today’s hearing on the Bureau of
                         Land Management’s (BLM) Automated Land and Mineral Record System
                         project, also known as the ALMRS/Modernization. The ALMRS/
                         Modernization was intended to provide modern computer and
                         telecommunications equipment and office automation for over 200 offices
                         nationwide as well as software to more efficiently record, maintain, and
                         retrieve land description, ownership, and use information to support BLM,
                         other federal programs, and interested parties. As you requested, I will
                         discuss (1) the history of the project, (2) the results of our reviews,
                         including the key reasons for problems, and (3) where we believe BLM
                         should go from here.

                         Mr. Chairman, BLM spent over 15 years and estimates that it invested about
                         $411 million planning and developing the ALMRS/Modernization, only to
                         have the major software component—known as the ALMRS Initial
                         Operating Capability (IOC)—fail. As a result of that failure, the bureau
                         decided not to deploy ALMRS IOC at this time.

                         We have previously reported on the significant problems and risks that
                         BLM has encountered. We have made many recommendations to reduce
                         those risks; however, BLM has been slow to implement some
                         recommendations and has not yet fully implemented others. The bureau
                         now needs to determine whether it can salvage any of the more than
                         $67-million reported investment in ALMRS IOC software, by analyzing the
                         software to determine if it can be cost-beneficially modified to meet BLM’s
                         needs. In addition, to reduce the risk that future efforts will result in
                         similar failures, BLM should assess its information technology investment
                         practices and systems acquisition capabilities.

                         We performed our work from July 1998 through February 1999, in
                         accordance with generally accepted government auditing standards.



The ALMRS/               During the energy boom of the early 1980s, BLM found that it could not
                         handle the case processing workload associated with a growing number of
Modernization: A Brief   applications for oil and gas leases. The bureau recognized that to keep up
History                  with increased demand, it needed to automate its manual records and case
                         processing activities. Therefore, in the mid-1980s, it began planning to
                         acquire an automated land and mineral case processing system. At that



                         Page 1                                                     GAO/T-AIMD-99-102
time, BLM estimated that the life-cycle cost of such a system would be
about $240 million.

In 1988 BLM expanded the scope of the system to include a land
information system (LIS). The expanded system was to provide automated
information systems and geographic information systems technology 1
capabilities to support other land management functions, such as land use
and resource planning. BLM combined the LIS with a project to modernize
the bureau's computer and telecommunications equipment, and estimated
the total life-cycle cost of this combined project to be $880 million.

The project was reduced in scope in 1989 to respond to concern about the
high cost and named the ALMRS/Modernization. The project consisted of
three major components—the ALMRS IOC, a geographic coordinate
database,2 and the modernization of BLM's computer and
telecommunications infrastructure and rehost of selected management and
administrative systems. Estimated life-cycle costs were $575 million (later
reduced to $403 million), and BLM planned to complete the entire project
by the end of fiscal year 1996.

The ALMRS IOC was to be the flagship of the ALMRS/Modernization, and
was to replace various manual and ad hoc automated systems. The bureau
designated the ALMRS IOC a critical system for (1) automating land and
mineral records, (2) supporting case processing activities, including leasing
oil and gas reserves and recording valid mining claims, and (3) providing
information for land and resource management activities, including timber
sales and grazing leases. The system was expected to more efficiently
record, maintain, and retrieve land description, ownership, and use
information to support BLM, other federal programs, and interested
parties. It was to do this by using the new computer and
telecommunications equipment that was deployed throughout the bureau,
integrating multiple databases into a single geographically referenced
database, shortening the time to complete case processing activities, and
automating costly manual records.




1 A geographic information system is computer technology designed to assemble, store, manipulate, and
display geographically referenced data, such as the location of a lake or oil well.

2 Wepreviously reported significant cost overruns and milestone slippages on an earlier project to
develop the database. See Land Management Systems: Extensive Cost Increases and Delays in BLM’s
Major Data Base Project (GAO/IMTEC-91-55, August 5, 1991).




Page 2                                                                                GAO/T-AIMD-99-102
Despite the promise of ALMRS IOC to significantly improve business
operations, repeated problems with its development have prevented
deployment. For example, during a user evaluation test in May 1996,
problems were reported involving unacceptably slow system performance.
Subsequent testing in 1996 uncovered 204 high-priority software problems,
which delayed project completion by about a year. In testing conducted in
November 1997, BLM encountered workstation failures and slowdowns
caused by insufficient workstation memory and by problems discovered in
two BLM-developed software applications. Some of these problems had
been identified in earlier tests but had not been corrected. Additional
testing uncovered software errors that resulted in missing, incorrect, and
incomplete data, and error files that contained accurate data. As a result of
these problems, BLM postponed the Operational Assessment Test and
Evaluation (OAT&E) that had been scheduled for December 1997. The
OAT&E was to determine whether ALMRS IOC was ready to be deployed to
the first state office.

In October 1998, the OAT&E was conducted and showed that ALMRS IOC
was not ready to be deployed because it did not meet requirements. During
the test, users reported several problems, including that ALMRS IOC (1) did
not support BLM’s business activities, (2) was too complex, and
(3) significantly impeded worker productivity. For example, one tester
reported that entering data for a $10 sale of a commodity, such as gravel,
required an hour of data entry using ALMRS IOC, whereas with the existing
system, the same transaction would have taken about 10 minutes. Users
also reported that system response time problems were severe or
catastrophic at all test sites. One user said “It is ridiculous to spend 2 or 3
hours to enter information in this system, when it takes 30 minutes to an
hour to process the information into the legacy system.” Finally, users
reported data converted from legacy databases were not accurate, and that
validation of the converted legacy data required inordinate effort and time.

Because these problems are significant, senior BLM officials have decided
that ALMRS IOC is not currently deployable. According to BLM, it
obligated about $411 million on the ALMRS/Modernization project between
fiscal years 1983 and 1998, of which more than $67 million was spent to
develop ALMRS IOC software. The $67 million does not include ALMRS
IOC costs that are part of other cost categories, such as costs for work
performed from fiscal years 1983 through 1988, project management,
computer and telecommunications hardware and software, data
management, and systems operation and maintenance. The reported




Page 3                                                       GAO/T-AIMD-99-102
                                             obligations associated with the major cost categories of the ALMRS/
                                             Modernization are summarized in table 1.



Table 1: Reported ALMRS/Modernization Obligations, Fiscal Years 1983 Through 1998

Cost category                 Explanation                                                                                         Obligations
FY 1983-1988 obligations      •   Data collection
                              •   Concept development
                              •   Requirements definition
                              •   Contract preparation                                                                           $32,925,000
Project management            • BLM and contractors’ costs for project management, oversight, and administration
                              • BLM salaries, benefits, and travel                                                                 74,690,940
ALMRS IOC software            • ALMRS Release 1 and 2
development                                                                                                                        67,547,220
Administrative systems        • Rehost of selected management and administrative systems from outdated
modernization                   mainframe computers to BLM’s modern, networked environment                                          8,198,466
Computer and                  • Acquiring and installing new hardware and software to support office automation
telecommunications hardware     and administrative functions
and software                                                                                                                     121,348,325
Data management               • BLM and contractor costs to collect, edit, and convert BLM land and mineral
                                program related data
                              • Ongoing work to establish the geographic coordinate database
                              • Conversion of BLM data from legacy systems into the ALMRS database                                 85,476,518
Operations and maintenance    • Configure, operate, and maintain administrative, database, and e-mail software,
                                telecommunications and computer hardware, and operating systems software
                              • Contract maintenance fees
                              • BLM and contractor labor costs                                                                     20,957,296
Total ALMRS modernization                                                                                                       $411,143,765
                                             Source: BLM; amounts include funding from all sources. We did not independently verify these data.


                                             Senior BLM officials told us that although ALMRS IOC is not currently
                                             deployable, BLM has benefited from the ALMRS/Modernization work. BLM
                                             has deployed about 6,000 workstations throughout the bureau, provided
                                             office automation capabilities, and implemented a national
                                             telecommunications network with electronic mail and internet access,
                                             which has enhanced communications and enabled BLM to communicate
                                             with other federal agencies. BLM’s view of the benefits received, however,
                                             does not reflect the fact that it has not realized the significant business-
                                             related benefits and improvements ALMRS IOC was to provide.




                                             Page 4                                                                      GAO/T-AIMD-99-102
Our Reviews Have      Mr. Chairman, since May 1995 we have reported many problems and risks
                      that threatened the successful development and deployment of the ALMRS/
Shown Long-Standing   Modernization. Our reports have discussed these issues, their causes, and
Project Weaknesses    our recommended corrective actions.3 BLM has been slow to implement
                      some of our recommendations and has not yet fully implemented others.
                      Following is a summary of the problems, causes, and associated
                      recommendations we have reported.

                      • BLM did not develop a system architecture or formulate a concept of
                        operations before designing and developing the ALMRS/Modernization.
                        A system architecture describes the components of a system, their
                        interrelationships, and principles and guidelines governing their design
                        and evolution. A concept of operations describes how an organization
                        would use planned information technology to perform its business
                        operations and accomplish its missions. Designing and developing the
                        project without a system architecture and concept of operations
                        unnecessarily increased the risk that the ALMRS/Modernization would
                        not meet the business and information needs of the bureau.
                      • BLM has never had a credible project schedule, reliable milestones, or a
                        critical path to manage the development and deployment of the ALMRS/
                        Modernization. As a result, BLM has not known with any certainty how
                        long it would take and, therefore, how much it would cost to complete
                        the ALMRS/Modernization. Because BLM has not implemented our
                        recommendation to establish a credible project schedule, the ALMRS/
                        Modernization has been driven by self-imposed deadlines. In trying to
                        meet those deadlines, BLM has deferred some tasks until after
                        completion of the project, and has not corrected all problems when it
                        found them because doing so would cause it to miss the self-imposed
                        project deadlines.
                      • BLM faced serious risks because it had not established a robust
                        configuration management program for the ALMRS/Modernization.
                        Configuration management is essential to controlling the composition of
                        and changes to computer and network systems components and
                        documentation. The lack of configuration management increased the
                        risks that system modifications could lead to undesirable consequences,


                      3 Land Management Systems:   Progress and Risks in Developing BLM’s Land and Mineral Record System
                      (GAO/AIMD-95-180, August 31, 1995), Land Management Systems: BLM Faces Risks in Completing the
                      Automated Land and Mineral Record System (GAO/AIMD-97-42, March 19, 1997), Land Management
                      Systems: Information on BLM’s Automated Land and Mineral Record System Release 2 Project (GAO/
                      AIMD-97-109R, June 6, 1997), and Land Management Systems: Actions Needed in Completing the
                      Automated Land and Mineral Record System Development (GAO/AIMD-98-107, May 15, 1998).




                      Page 5                                                                      GAO/T-AIMD-99-102
    such as causing system failures, endangering system integrity,
    increasing security risks, and degrading system performance. In
    response to our recommendation, BLM later developed a configuration
    management plan and related policies and procedures for the ALMRS/
    Modernization. We planned to review field office implementation of the
    configuration management program after completion of the ALMRS
    IOC; however, we have not done so because the system was not
    deployed.
•   BLM incurred serious risks because it had not established a security
    plan or security architecture for the ALMRS/Modernization. The lack of
    such security controls increased risks to the confidentiality, integrity,
    and availability of stored and processed data. BLM recently completed
    work in response to our recommendation. It performed a risk analysis,
    developed a system security plan and architecture, identified
    management and operational controls, and developed disaster and
    recovery plan procedures. As with configuration management, we
    planned to review field office implementation of the security program
    after completion of the ALMRS IOC, but have not done so because the
    system was not deployed.
•   BLM invited serious risks because it had not established transition plans
    to guide the incorporation of ALMRS IOC into its daily operations.
    Deploying a major information system that people will use to do their
    jobs requires careful planning to avoid business and operational
    problems. Without transition plans, BLM increased the risk that using
    ALMRS IOC would disrupt, rather than facilitate, its work processes and
    ability to conduct land and mineral management business. In response
    to our recommendation, BLM developed transition plans; however, the
    plans were not adequate. They did not outline needed changes in
    organizational roles, responsibilities, and interrelationships, or address
    issues such as how state and subordinate offices would deal with oil and
    gas, mining, and solid mineral business process changes that would
    result from implementing ALMRS IOC.
•   BLM faced serious risks because it had not established operations and
    maintenance plans. The lack of plans increased the risk that the bureau
    would not meet its automation objectives or the daily needs of its
    offices. BLM developed operations and maintenance plans in response
    to our recommendation. We expected to review field office
    implementation of the operations and maintenance plans after
    completion of the ALMRS IOC; however, we have not done so because
    the system was not deployed.
•   BLM invited serious risks because it planned to stress test only the
    ALMRS IOC component—state and district offices, ALMRS IOC servers,



Page 6                                                       GAO/T-AIMD-99-102
                          terminals, and workstations. This increased the risk that BLM would
                          deploy the ALMRS IOC nationwide without knowing whether the
                          ALMRS/Modernization—ALMRS IOC, office automation, e-mail,
                          administrative systems, and various departmental, state, and district
                          software applications in a networked environment—would perform as
                          intended during peak workloads. BLM agreed to fully stress test the
                          entire ALMRS/Modernization before deploying the ALMRS IOC
                          component throughout the bureau.
                        • BLM did not develop a Year 2000 contingency plan to ensure that critical
                          legacy systems could operate after January 1, 2000, if the ALMRS IOC
                          could not be delivered in 1999. We recommended that BLM develop a
                          Year 2000 contingency plan to ensure continued use of those critical
                          legacy systems ALMRS IOC was to replace. BLM implemented this
                          recommendation and began executing the plan in 1998, when it became
                          clear that ALMRS IOC would not be fully implemented by the end of
                          1999.



Where BLM Should Go     At this point, BLM has made an enormous investment in software that does
                        not meet its business needs. At the same time, it has not adopted
From Here               information technology management practices required by recent
                        legislation or suggested by industry best practices. Because of its large
                        investment, BLM should analyze ALMRS IOC to determine whether the
                        software can be cost-beneficially modified to meet the bureau’s needs. In
                        addition, to reduce the risk that future information technology efforts will
                        result in a similar outcome, BLM should assess its investment management
                        practices and its systems acquisition capabilities. Until these assessments
                        and subsequent improvement actions are taken, BLM will not be
                        adequately prepared to undertake any sizable system acquisition.


Analysis of ALMRS IOC   We believe that since BLM has invested over $67 million to develop the
Software Is Needed      ALMRS IOC software, the bureau should thoroughly analyze the software
                        to determine whether it can be modified to meet users’ needs and at what
                        cost. This analysis should be part of an overall effort to identify and assess
                        all viable alternatives, including (1) using or modifying ALMRS IOC
                        software, (2) modifying or evolving existing land and recordation systems,
                        (3) acquiring commercial, off-the-shelf software, or (4) developing new
                        systems. The alternative analysis should clearly identify the risks, costs,
                        and benefits of each alternative, and should be performed only after BLM is
                        assured that it has fully verified its current business requirements. In this
                        regard, senior BLM officials said they are performing an analysis to



                        Page 7                                                       GAO/T-AIMD-99-102
                         determine where ALMRS IOC failed to meet users’ expectations and critical
                         business requirements.


Assessment of BLM’s      According to the acting land and resources information systems program
Information Technology   manager, BLM is beginning to develop plans for future information
                         technology modernization. These plans are to identify alternatives to
Investment Management    deploying ALMRS IOC, and evaluate those alternatives based on cost,
Practices Is Needed      functionality, and return on investment. BLM also plans to document its
                         current and planned business processes and systems architectures as part
                         of this effort.

                         While such planning is necessary, BLM also needs to assess its investment
                         management practices to help avoid future problems. The Clinger-Cohen
                         Act of 1996 seeks to maximize the return on investments in information
                         systems by requiring agencies to institute sound capital investment
                         decision-making. Under the act, agencies must design and implement a
                         process for maximizing the value and assessing and managing the risks of
                         information technology acquisitions.

                         An information technology investment process is an integrated approach
                         that provides for data-driven selection, control, and evaluation of
                         information technology investments.4 The investment process is
                         comprised of three phases. The first phase involves selecting investments
                         using quantitative and qualitative criteria for comparing and setting
                         priorities for information technology projects. The second phase includes
                         monitoring and controlling selected projects through progress reviews at
                         key milestones to compare the expected costs, risks encountered, and
                         performance benefits realized to date. These progress reviews are
                         essential for senior managers to decide whether to continue, accelerate,
                         modify, or terminate a selected project. The third phase involves a
                         postimplementation review or evaluation of fully implemented projects to
                         compare actuals against estimates, assess performance, and identify areas
                         where future decision-making can be improved.

                         According to senior BLM officials, the bureau has established an
                         Information Technology Investment Board to provide support for its capital


                         4 Thisprocess is documented in our Assessing Risks and Returns: A Guide for Evaluating Federal
                         Agencies’ IT Investment Decision-Making, Version 1 (GAO/AIMD-10.1.13, February 1997) and OMB’s
                         Evaluating Information Technology Investments: A Practical Guide, Office of Management and Budget,
                         Version 1.0, November 1995.




                         Page 8                                                                      GAO/T-AIMD-99-102
planning processes. It intends to apply more rigorous, structured
processes to analyze its information technology investments and select,
control, and evaluate information technology investment alternatives.
Until such processes are fully in place, the bureau cannot be assured that
future investments will be properly selected, managed, and evaluated using
sound investment criteria to provide effective support for the bureau’s
mission and goals.

Further, to ensure that information technology investment processes are
carried out adequately, the Clinger-Cohen Act also requires agencies to
assess the knowledge and skills of its executive and management staff to
meet agencies’ information resources management requirements, and to
take steps to rectify any deficiencies. The Software Engineering Institute5
(SEI) has identified the need for organizations to focus on information
resources management capabilities.6 Organizations should improve their
capabilities using a process to characterize the maturity of their workforce
practices, guide a program of workforce development, set priorities for
immediate actions, and establish a culture of software engineering
excellence.

According to senior BLM officials, the bureau examined the kind of skills
that its field office computer specialists had, and identified the skills they
would need. However, the officials recognize that this was not the same as
the more comprehensive assessment suggested by SEI. Such assessments
are needed to better identify and manage information technology
investments. Consequently, the bureau should evaluate and, where needed,
enhance the knowledge and skills of its staff to help ensure that the
investment management processes it puts in place can be effectively
carried out by its information resources management organization.




5
 SEI, located at Pittsburgh’s Carnegie Mellon University, is a nationally recognized, federally funded
research and development center established to address software development issues.

6 Software   Engineering Institute, People Capability Maturity Model, CMU/SEI-95-MM-02, September
1995.




Page 9                                                                            GAO/T-AIMD-99-102
                          Finally, the Clinger-Cohen Act requires agencies to develop, maintain, and
                          facilitate the implementation of a sound and integrated information
                          technology architecture.7 An information technology architecture provides
                          a comprehensive blueprint that systematically details the breadth and
                          depth of an organization’s mission-based mode of operation. An
                          architecture provides details first in logical terms, such as defining
                          business functions, providing high-level descriptions of information
                          systems and their interrelationships, and specifying information flows; and
                          second in technical terms, such as specifying hardware, software, data,
                          communications, security, and performance characteristics. By enforcing
                          an information technology architecture to guide and constrain a
                          modernization program, an agency can preclude inconsistent systems
                          design and development decisions, and the resulting suboptimal
                          performance and excess cost.

                          As I discussed earlier, BLM did not develop a system architecture before
                          designing and developing the ALMRS/Modernization. This is a key reason
                          why ALMRS IOC did not meet the bureau’s business needs. BLM still has
                          not developed an architecture that documents its business processes and
                          the technology and systems that support them. BLM needs to develop an
                          information technology architecture to guide its future investment plans.


BLM Needs to Assess Its   Research by SEI has shown that defined and repeatable processes for
Systems Acquisition       managing software acquisition are critical to an organization’s ability to
                          consistently deliver high-quality information systems on time and within
Capabilities              budget. These critical management processes include project planning,
                          requirements management, software project tracking and oversight,
                          software quality assurance, software configuration management, and
                          change control management.8 To assist organizations in evaluating and
                          enhancing systems acquisition capabilities and processes, SEI has
                          developed models for conducting software process assessments and




                          7
                           The Clinger-Cohen Act of 1996 gives the Chief Information Officer of an executive agency the
                          responsibility for developing, maintaining, and facilitating the implementation of a sound and
                          integrated information technology architecture. An information technology architecture is sometimes
                          referred to as a system architecture.

                          8 Definitions
                                     of these processes were obtained from the Software Engineering Institute’s Capability
                          Maturity Model for Software, Version 1.1,  1993 Carnegie Mellon University.




                          Page 10                                                                       GAO/T-AIMD-99-102
                    software capability evaluations to determine the state of their capabilities
                   and identify areas requiring improvement.9

                   BLM also needs an independent assessment of its systems acquisition
                   capabilities, and must ensure that it uses sound systems acquisition
                   processes. As I discussed earlier, BLM did not develop several key
                   management controls for the ALMRS/Modernization. BLM did not develop
                   a credible project schedule or develop adequate transition plans. In
                   addition, the lack of a configuration management program, security plan
                   and architecture, and operations and maintenance plans further increased
                   BLM’s risks. These problems indicate the need for BLM to ensure that the
                   deficiencies in its systems acquisition capabilities and processes are
                   acknowledged and corrected. Until such assessments are completed and
                   corrective action taken, BLM should not undertake any sizable systems
                   acquisition or development efforts.

                   Mr. Chairman, that concludes my statement. I would be happy to respond
                   to any questions that you or other members of the Subcommittee may have
                   at this time.




                   9 SEIhas developed two models to assist organizations in assessing the maturity of their software
                   development processes. These models are the Capability Maturity Model for Software and the Software
                   Acquisition Capability Maturity Model. Capability Maturity Model SM is the service trademark of
                   Carnegie Mellon University, and CMM is registered in the U.S. Patent and Trademark Office.




(511470)   Leter   Page 11                                                                      GAO/T-AIMD-99-102
Page 12   GAO/T-AIMD-99-102
Ordering Information

The first copy of each GAO report and testimony is free.
Additional copies are $2 each. Orders should be sent to the
following address, accompanied by a check or money order made
out to the Superintendent of Documents, when necessary, VISA and
MasterCard credit cards are accepted, also.

Orders for 100 or more copies to be mailed to a single address are
discounted 25 percent.

Orders by mail:

U.S. General Accounting Office
P.O. Box 37050
Washington, DC 20013

or visit:

Room 1100
700 4th St. NW (corner of 4th and G Sts. NW)
U.S. General Accounting Office
Washington, DC

Orders may also be placed by calling (202) 512-6000
or by using fax number (202) 512-6061, or TDD (202) 512-2537.

Each day, GAO issues a list of newly available reports and
testimony. To receive facsimile copies of the daily list or any list
from the past 30 days, please call (202) 512-6000 using a touchtone
phone. A recorded menu will provide information on how to obtain
these lists.

For information on how to access GAO reports on the INTERNET,
send an e-mail message with “info” in the body to:

info@www.gao.gov

or visit GAO’s World Wide Web Home Page at:

http://www.gao.gov
United States                       Bulk Rate
General Accounting Office      Postage & Fees Paid
Washington, D.C. 20548-0001           GAO
                                 Permit No. GI00
Official Business
Penalty for Private Use $300

Address Correction Requested