United States General Accounting Office GAO Report to the Chairman, Committee on Indian Affairs, U.S. Senate April 1999 INDIAN TRUST FUNDS Interior Lacks Assurance That Trust Improvement Plan Will Be Effective GAO/AIMD-99-53 United States GAO General Accounting Office Washington, D.C. 20548 Leter Accounting and Information Management Division Leter April 28, 1999 B-280590 Letter The Honorable Ben Nighthorse Campbell Chairman, Committee on Indian Affairs United States Senate Dear Mr. Chairman: This report responds to your request that we evaluate the Department of Interior’s High-Level Implementation Plan (High Level Plan) for improving its management of the Indian trust funds and resources under its control. This plan focuses on correcting many of the long-standing problems with Indian trust operations, which include inadequate accounting and information management systems; backlogs in asset appraisals, ownership determination, and record keeping; and poor internal controls. As discussed with your office, we agreed to assess whether Interior has reasonable assurance that (1) the High Level Plan provides an effective solution for addressing these long-standing problems, and (2) its acquisition of a new asset and land records management service will cost effectively satisfy trust management needs. Results in Brief Interior does not have reasonable assurance that its High Level Plan for improving Indian trust operations provides an effective solution for addressing long-standing management weaknesses. The plan (1) recognizes the severity of long-standing weaknesses in managing trust fund assets, (2) identifies 13 projects intended to improve information systems, enhance the accuracy and completeness of its data regarding the ownership and lease of Indian lands, and address deficiencies with respect to records management, training, policy and procedures, and internal controls, and (3) assigns responsibility for oversight and management of the 13 projects. However, Interior has not properly analyzed its information technology needs which are essential to the overall success of the plan. Until Interior develops an information systems architecture addressing all of its trust management functions, it cannot ensure that its information systems will not be duplicative or incompatible or will optimally support its needs across all business areas. Interior also does not know whether its acquisition of a new service for managing Indian assets and land records will cost effectively meet trust management needs. Before deciding to contract with a service vendor, Interior did not adequately define important service requirements or Page 1 GAO/AIMD-99-53 Indian Trust Funds B-280590 sufficiently analyze technical alternatives. Nor did Interior take the steps needed to minimize acquisition risks. In particular, it did not develop a risk management plan, ensure that the vendor’s system could work with Interior’s data and systems, or establish realistic project time frames. Thus, Interior faces an unnecessarily high risk that the service will not meet its general business and specific performance needs, and it lacks the means for dealing with this risk. Background The Secretary of the Interior is responsible for administering the government's trust responsibilities to tribes and Indians, including managing about $3 billion in Indian trust funds and administering about 54 million acres of Indian land. Management of the Indian trust funds and assets has long been characterized by inadequate accounting and information systems; untrained and inexperienced staff; backlogs in appraisals, ownership determinations, and recordkeeping; the lack of a master lease file and an accounts receivable system; inadequate written policies and procedures; and poor internal controls. To address these long-standing problems, the Congress created the Office of the Special Trustee for American Indians (OST) and required the Special Trustee to develop a comprehensive strategic plan for trust fund management. In April 1997, the Special Trustee submitted a strategic plan to the Congress, but Interior did not fully support the plan. At this Committee’s July 1997 hearing on the Special Trustee’s strategic plan, we testified on the results of our analysis of the strategic plan and provided our assessment of needed actions related to implementation issues that we had identified during that analysis.1 On August 22, 1997, the Secretary of the Interior indicated that he and the Special Trustee for American Indians had agreed on the problems that needed to be solved immediately and called for the development of a high level implementation plan within 60 days. The High Level Plan was issued about 11 months later on July 31, 1998. In developing the High Level Plan, Interior did not prepare a documented analysis of its mission-related and administrative processes. Rather, it took the problems identified in the Secretary’s memorandum one by one and proposed separate projects to 1 Financial Management: Indian Trust Fund Strategic Plan (GAO/T-AIMD-97-138, July 30, 1997). Page 2 GAO/AIMD-99-53 Indian Trust Funds B-280590 address each. Later, at the Secretary’s direction, an additional project was added. The 13 separate projects are shown in table 1. The projects are directed at improving systems, enhancing the accuracy and completeness of Interior’s data regarding the ownership and lease of Indian lands, and correcting deficiencies with respect to records management, training, policy and procedures, and internal controls within 3 years. For each project, the plan assigns management responsibility and identifies some supporting tasks, critical milestones, and resource estimates. Some of the projects are already being implemented. For example, a new Trust Funds Accounting System has already been deployed at several Interior sites. We did not assess the status or effectiveness of this project or other individual projects. Instead, we focused on whether Interior has assurance that the information technology aspects of the plan, which are essential to the success of the majority of the projects and therefore the overall plan, were properly planned and executed. Table 1: Thirteen Projects for Improving Indian Trust Management Project Description 1 OST Trust Financial Records OST will standardize and verify Individual Indian Monies (IIM) system data for trust Clean Up resource records, and correct and establish an inventory of hard copy records for each trust fund account. 2 Bureau of Indian Affairs (BIA) Trust BIA trust resource records will be cleaned up to ensure timely ownership and land Resource Records Cleanup status data. Processing backlogs will be worked off to update existing and future trust resource management systems data essential to ensure that income distribution and resource management functions can operate from timely data. 3 BIA Probate Backlog BIA will inventory, identify, and develop action plans and procedures to eliminate probate backlog. 4 Office of Hearings and Appeals OHA will inventory, identify, and develop action plans and procedures to eliminate OHA (OHA) Probate Backlog probate backlog. 5 BIA Appraisal Program This project includes an assessment of the present BIA appraisal program, policies and procedures; reviews of staff qualifications; determination of the adherence to uniform Standards of Professional Appraisal Practices; and development of corrective action plans, as appropriate. 6 Trust Funds Accounting System A proven commercial off-the-shelf (COTS) trust accounting system will be acquired, using a service bureau approach, to replace the present BIA IIM accounting module. Page 3 GAO/AIMD-99-53 Indian Trust Funds B-280590 Project Description 7 Trust Asset and Accounting The Department will evaluate, acquire, and pilot, standardized, proven COTS general Management System (TAAMS) trust management system technology (Master Lease, Billings and Accounts Receivable, and Collection subsystems) to the extent practicable. Following successful testing and piloting, the TAAMS system will proceed to full implementation across BIA, replacing the present BIA Integrated Records Management System. 8 BIA Land Records Information This project contemplates the modernization of BIA’s official title system to provide on- System (LRIS) Enhancements line and up-to-date legal and beneficial title ownership and encumbrance for all Indian lands and resources, including automated calculation of data storage of fractional interests and automated chain-of-title processes and information. 9 Minerals Management Service MMS will design, develop, and implement new core business processes for MMS’ (MMS) System Reengineering royalty management functions, with supporting systems. 10 Records Management A joint records management solution for Interior trust records will be developed and implemented, involving OST, BIA, MMS, Bureau of Land Management (BLM), OHA and other relevant Interior offices. The project scope includes Indian trust records management, storage, access, control and disposition and contemplates electronic recordkeeping, including imaging technology. 11 Policy and Procedures Interior trust policies and procedures will be inventoried, reviewed, and, where appropriate, revised or established. This project specifically involves and includes representatives of OST, BIA, MMS, BLM, OHA, and other departmental offices involved in Indian trust management. 12 Training This project will plan and deliver both trust management and employee skills training relevant to delivery of Interior’s trust fiduciary responsibilities to American Indians. Training will be provided across the Interior trust workforce and include tribes and participating contractors. 13 Internal Controls This project will systematically address documented internal control deficiencies in Indian trust management, item by item, that have been identified through internal and external audit, congressional oversight and outside reviews. Corrective actions will be validated and/or designed to assure resolution of all internal control weaknesses. Source: Department of the Interior July 1998 High Level Implementation Plan. Interior estimates that it will spend $147.4 million from fiscal years 1997 through 2000 on this effort. About $60 million of this amount is to be spent on developing and improving information systems, $54 million on data cleanup, $17 million on records management, $8 million on training, and $8 million on all other activities. Scope and The objectives of our review were to assess whether Interior has reasonable assurance that (1) the High Level Plan provides an effective Methodology solution for addressing long-standing problems with Interior’s Indian trust Page 4 GAO/AIMD-99-53 Indian Trust Funds B-280590 responsibilities and (2) its acquisition of a new asset and land records management service will cost effectively satisfy trust management needs. To determine whether Interior has reasonable assurance that the High Level Plan provides an effective solution for addressing Interior’s long- standing problems with its Indian trust responsibilities, we • reviewed the Clinger-Cohen Act of 19962 and current technical literature3 as a basis for assessing the information technology aspects of the High-Level Plan; • reviewed the process that was used to develop the plan; • reviewed the Strategic Plan that was produced by Interior’s Special Trustee for American Indians; • met with senior Interior officials responsible for developing the plan, including Interior’s Chief Information Officer, Chief Financial Officer, Deputy Special Trustee, and the Interior contractor who assisted in the development of the plan; and • analyzed the High Level Plan for internal consistency and compliance with generally accepted best practices. We focused on the information technology aspects of the plan because they are essential to its success. To determine whether Interior has reasonable assurance that its acquisition of a new asset and land records management service will cost effectively satisfy trust management needs, we 2Public Law 104-106. The Clinger-Cohen Act requires agencies to analyze their missions and, based on the analysis, revise mission-related and administrative processes, as appropriate, before making significant investments in information technology used to support those missions. 3 For example, we reviewed GAO’s framework for designing and developing system architectures; the Project Management Institute’s Guide to the Project Management Body of Knowledge; and the Software Engineering Institute’s guidance on software development and software acquisitions. Page 5 GAO/AIMD-99-53 Indian Trust Funds B-280590 • reviewed the Clinger-Cohen Act of 1996; federal policy governing acquisition efforts including Office of Management and Budget guidance and Federal Information Processing Standards; and other current literature to determine the statutory and administrative requirements and best practices that should be used in acquiring software-intensive services such as the asset and land records service; 4 • reviewed Interior documents relating to this acquisition, including the Request for Information, vendor responses, and the Request for Proposals. We did not review the selection process or documents produced as part of this process subsequent to the issuance of the Request for Proposals; and • met with senior Interior officials responsible for acquiring the service, including Interior’s Chief Information Officer, Chief Financial Officer, Special Trustee, and the Interior contractor who assisted in the acquisition of the new service. We performed our work at the Department of the Interior, Office of the Special Trustee, and Bureau of Indian Affairs in Washington, D.C., from July 1998 through November 1998 in accordance with generally accepted government auditing standards. We requested comments on a draft of this report from the Secretary of the Interior. On March 19, 1999, the Assistant Secretary for Policy, Management and Budget provided us with written comments, which are discussed in the “Agency Comments and Our Evaluation” section of this report and reprinted in appendix I. 4 For example, we reviewed the Software Engineering Institute’s Software Acquisition Capability Maturity Model SM (Capability Maturity ModelSM is a service mark of Carnegie Mellon University, and CMM is a registered trademark) which provides a logical and widely accepted framework for baselining an organization’s current process capabilities (i.e., strengths and weaknesses) and assessing whether an organization has the necessary process discipline in place to repeat earlier successes on similar projects. Page 6 GAO/AIMD-99-53 Indian Trust Funds B-280590 Without Systems Despite the fact that Interior plans for its components to independently improve information systems or acquire information management services, Architecture, Interior at a cost of about $60 million, it has not yet defined an integrated Lacks Assurance That architecture for Indian trust operations. The Clinger-Cohen Act requires the Chief Information Officer to develop and maintain an information the Plan Provides an systems architecture. Without a target architecture, agencies are at risk of Effective Solution to building and buying systems that are duplicative, incompatible, and Long-Standing unnecessarily costly to maintain and interface. Problems In 1992, we issued a report5 defining a comprehensive framework for designing and developing system architectures. This framework specifies (1) the logical or business component of an architecture which serves as the basis for (2) the technical or systems component. The logical component ensures that the systems meet the business needs of the organization. It provides a high-level description of the organization’s mission and target concept of operations; the business functions being performed and the relationships among functions; the information needed to perform the functions; the users and locations of the functions and information; and the information systems needed to support the agency’s business needs. The technical component ensures that the systems are interoperable, function together efficiently and are cost-effective over their life cycles. The technical component details specific standards and approaches that will be used to build systems, including hardware, software, communications, data management, security, and performance characteristics. Experience shows that without a target architecture, agencies risk building and buying systems that are duplicative, incompatible, and unnecessarily costly to maintain and interface. For example: • In February 1997, we reported6 that the Federal Aviation Administration’s (FAA) lack of a complete architecture resulted in 5Strategic Information Planning: Framework for Designing and Developing System Architectures (GAO/IMTEC-92-51, June 1992). 6 Air Traffic Control: Complete and Enforced Architecture Needed for FAA Systems Modernization (GAO/AIMD-97-30, February 3, 1997). Page 7 GAO/AIMD-99-53 Indian Trust Funds B-280590 incompatibilities among its air traffic control systems that (1) required higher-than-need-be system development, integration, and maintenance costs and (2) reduced overall system performance. Without having architecturally defined requirements and standards governing information and data structures and communications, FAA was forced to spend an additional $38 million to acquire a system dedicated to overcoming incompatibilities between systems. • In May 1998, we reported7 that the Customs Service’s architecture was incomplete and ineffectively enforced, and that, according to a contractor, Customs components had developed and implemented incompatible systems, which increased modernization risks and implementation costs. • In July 1997, we reported8 that because it lacked a system architecture, the Department of Education had made limited progress in integrating its National Student Loan Data System with other student financial aid databases. Moreover, without an architecture, the department could not correct long-standing problems resulting from a lack of integration across its student financial aid systems. • In July 1995, we reported9 that because its architecture was incomplete and did not define the interfaces and standards needed to ensure the successful integration of its Tax System Modernization projects, IRS was at increased risk of developing unreliable systems that would not work together effectively and would require costly redesign. Without an architecture for Indian trust operations, Interior has no assurance that the 13 projects delineated in the High Level Plan and the systems supporting them are cost-effective and are not duplicative, inconsistent, and incompatible. In fact, in reviewing the High Level Plan, we found indications that Interior was already encountering these problems. For example: • Three weeks after the plan was issued, Interior recognized that TAAMS and LRIS were so closely related that they should be merged into a single project. The BIA Probate Backlog project and the OHA Probate 7 Customs Service Modernization: Architecture Must Be Complete and Enforced to Effectively Build and Maintain Systems (GAO/AIMD-98-70, May 5, 1998). 8Student Financial Aid Information: Systems Architecture Needed to Improve Program’s Efficiency (GAO/AIMD-97-122, July 29, 1997). 9 Tax Systems Modernization: Management and Technical Weaknesses Must Be Corrected If Modernization Is To Succeed (GAO/AIMD-95-156, July 26, 1995). Page 8 GAO/AIMD-99-53 Indian Trust Funds B-280590 Backlog project also appear to be closely related; however, Interior did not thoroughly analyze the relationship between these two efforts in formulating the High Level Plan and did not determine whether, like TAAMS and LRIS, they should be combined. • The High Level Plan shows that the BIA Probate Backlog and the OHA projects depend on the TAAMS project to provide them with a case tracking system by the end of 1998. This system is to manage the flow of probate cases through BIA and OHA and enable management to identify resources needed to eliminate the backlog. However, in describing TAAMS, the High Level Plan does not mention the case management system. Further, according to Interior officials, development of the case tracking system under TAAMS is not scheduled to be funded until fiscal year 2000, and delivery is not planned before September 2000. • Although Interior has already initiated several projects to “clean” data that will be used by TAAMS, it has not yet defined the data elements that this project needs. Until Interior defines the logical characteristics of its business environment and uses them to establish technical standards and approaches, it will remain at risk of investing in projects that are redundant and incompatible, and do not satisfy Indian trust management requirements cost effectively. Interior Does Not In undertaking its effort to acquire a new asset and land record management service, Interior failed to follow a sound process for ensuring Know if New Asset and that the most cost-effective technical alternative was selected and reducing Record Management acquisition risks. Specifically, Interior did not adequately define important service requirements or sufficiently analyze technical alternatives. Further, Service Will Cost Interior did not develop an overall risk management plan, require the Effectively Satisfy contractor to demonstrate its system could work with Interior’s data and Trust Management systems, or establish realistic project time frames. Needs Interior’s Decision to Interior intended to acquire TAAMS as a commercial-off-the-shelf (COTS) Acquire a Service for system. With this goal in mind, in May 1998, Interior issued a Request for Information. The responses from vendors were evaluated using a 15- Managing Assets and Land category form. After this survey was completed, Interior decided to Records combine the TAAMS project with the LRIS project and to obtain the needed functionality of these combined projects by acquiring a trust asset information management service using a COTS system. Under this Page 9 GAO/AIMD-99-53 Indian Trust Funds B-280590 approach, a contractor would manage Interior-provided land and trust account data in a contractor-owned and maintained data center while Interior would perform its trust management functions by remotely accessing contractor-provided applications that run in the data center. Service Requirements Were To help ensure successful acquisition of a software-intensive service, Not Adequately Defined information technology experts recommend that organizations establish and maintain a common and unambiguous definition of requirements (e.g., function, performance, help desk operations, data characteristics, security, etc.) among the acquisition team, the service users, and the contractor.10 The requirements must be consistent with one another, verifiable, and traceable to higher level business or functional requirements. Poorly defined, vague or conflicting requirements can result in a service which does not meet business needs or which cannot be delivered on schedule and within budget. Interior did not follow a sound process for defining requirements. First, Interior did not define high-level functional requirements for projects contained in the High Level Plan to help guide the requirements development process for each of the individual projects. For this effort, such high-level functional requirements might have included the following. • The contractor’s system will contain the necessary data to support the financial information needs of the probate function. • Records management policies and procedures will be consistent with departmental guidelines. • Sensitive but unclassified data, such as data covered by the Privacy Act, will be encrypted in accordance with Federal Information Processing Standards whenever they are transmitted outside of the facility that generated the data. • Data elements must conform to applicable departmental naming conventions and formats specified in the data dictionary. • Automated records must be maintained in a form that ensures land ownership records can be traced back to the original source of the ownership. 10 For example, the Software Engineering Institute’s Software Acquisition Capability Maturity Model includes requirements development as a key practice. Page 10 GAO/AIMD-99-53 Indian Trust Funds B-280590 By not defining high-level functional requirements, Interior lacks assurance that the projects it develops and acquires will meet its business needs. Second, while Interior specified general service requirements in its request for proposal such as the need for the contractor to (1) administer all databases, (2) perform maintenance operations outside BIA’s normal working hours, (3) provide configuration management of data center hardware and software, and (4) perform daily, weekly, and monthly backup of operational data and archiving, it did not clearly specify all of BIA’s requirements, including its functional, security, and data management requirements. For example: • While Interior stated that the system “shall include safeguards against conflicts of interest, abuse, or self-dealing,” it did not define these terms. A definition of these terms in the context of Indian trust operations is necessary to design and determine the adequacy of proposed system safeguards and approaches. • In discussing system security, Interior (1) specified an inappropriate technology for encrypting data,11 (2) did not specify how long system passwords should be, and (3) did not require password verification features.12 • Interior did not define key data management requirements, including what data elements were needed to meet Interior’s information requirements and whether existing systems contained the necessary data elements. Technical Alternatives Were The Clinger-Cohen Act requires agencies to establish a process to assess Not Sufficiently Analyzed the value and risks of information technology investments, including consideration of quantitatively expressed projected net, risk-adjusted 11Encryption involves the transformation of original text (known as plaintext or cleartext) into unintelligible text (also known as ciphertext). The requirement in the Request for Proposal stated that “[t]he Contractor shall provide a method of connectivity that allows secure transmission of data utilizing 64-bit Public Key Encryption.” Public key encryption systems (also called asymmetric cryptography) are designed so that the key used for encryption is different from the key used for decryption. Public key systems are used to encrypt the keys that are used by systems using symmetric key cryptography to encrypt data. (This is commonly referred to as key management.) They are not used for encrypting large amounts of data such as that in TAAMS because public key algorithms are (1) slow (symmetric key systems are generally at least 1,000 times faster) and (2) vulnerable to certain types of computer attacks which depend upon analyzing extensive amounts of encrypted data. 12 When a user enters the desired password, a password checking program will compare the password to a wordlist and a series of rules to ensure that it cannot be easily guessed. Page 11 GAO/AIMD-99-53 Indian Trust Funds B-280590 return-on-investment, and specific quantitative and qualitative criteria for comparing and prioritizing alternative information technology projects. Only by comparing the costs, benefits, and risks of a full range of technical options can agencies ensure that the best approaches are selected. Interior did not thoroughly analyze technical alternatives before choosing a vendor to provide the asset and land records management service. First, Interior did not assess the desirability of satisfying its requirements by (1) modifying existing legacy systems, (2) acquiring a COTS product and using existing Interior infrastructure resources, (3) building a system that would provide the necessary capability, or (4) acquiring a service. Second, in surveying the availability of COTS products, Interior did not perform a gap analysis which would systematically and quantitatively compare and contrast these products against Interior’s requirements based on functional, technical, and cost differences. Specifically, although Interior concluded based on the results of its Request for Information that none of the COTS products available from responding vendors would meet all its requirements, Interior did not determine, for each COTS product, which requirements could not be satisfied and how difficult and expensive it would be to make the needed modifications. For example, Interior did not determine whether all needed data elements could be represented conveniently and manipulated effectively by each COTS product. Third, in acquiring a service, Interior did not consider how its information, once it had been loaded into a contractor’s system, would be retrieved by Interior for subsequent use when the contract was terminated. Because Interior did not compare the costs and benefits of a full range of technical options, it has no assurance that it selected the most cost-effective alternative. Acquisition Risk Was Not According to information technology experts, a key practice associated Minimized with successful information technology service acquisitions is to formally identify risks as early as possible and adjust the acquisition to mitigate those risks.13 An effective risk management process, among other things, includes (1) developing an acquisition risk management plan to document 13 For example, the Software Engineering Institute includes acquisition risk management as a key practice in its Software Acquisition Capability Maturity Model because it is considered by software experts to be an integral part of the solicitation, project performance management, and contract performance management processes. Page 12 GAO/AIMD-99-53 Indian Trust Funds B-280590 the procedures that will be used to manage risk throughout the project, (2) conducting risk management activities in accordance with the plan (e.g., identifying risks, taking mitigation actions, and tracking actions to completion), and (3) preparing realistic cost and schedule estimates for the services being acquired. In acquiring its new TAAMS service, Interior did not carry out critical risk management steps. First, Interior did not develop a risk management plan. Without this plan, Interior has no disciplined means to predict and mitigate risks, such as the risk that the service will not (1) meet performance and business requirements, (2) work with Interior’s systems, and/or (3) be delivered on schedule and within budget. Second, in structuring a capabilities demonstration for the contractor’s system, Interior did not require the contractor to use Interior-provided data. Ensuring that the contractor’s system can work with data unique to Interior is important since some data elements, such as fractionated ownership interests, are not commonly used in the private sector. Third, in structuring the capabilities demonstration, Interior did not require the contractor to demonstrate that its system could interface with Interior’s Trust Fund Accounting System and a Mineral Management Service system. As a result, Interior will not know whether the contractor’s system can interoperate with its legacy systems. Fourth, Interior did not prepare a realistic project management schedule. Organizations following sound software acquisition practices would typically (1) identify the specific activities that must be performed to produce the various project deliverables, (2) identify and document dependencies, (3) estimate the amount of time needed to complete the activities, and (4) analyze the activity sequences, durations, and resource requirements. By contrast, Interior used the Secretary’s stated expectation that all Indian trust fund-related improvements should occur within a 3- year period beginning in 1998 as a starting point for developing the TAAMS project schedule. 14 14 Memorandum from the Secretary to the Special Trustee for American Indians, Assistant Secretary for Indian Affairs, Deputy Commissioner for Indian Affairs, Director of the Minerals Management Service, and Director of the Bureau of Land Management, dated August 22, 1997. This memorandum stated the Secretary’s overall expectations for Interior’s improvement effort. Page 13 GAO/AIMD-99-53 Indian Trust Funds B-280590 Because it did not establish clear requirements and did not take critical steps to manage risk effectively, Interior has no assurance that the new asset and land records management service will meet its specific performance, security, and data management needs or that the service can be delivered on schedule and within budget. Conclusions Interior cannot realistically expect to develop compatible and optimal information systems without first developing an information systems architecture for Indian trust operations. If it proceeds to implement the projects outlined in the High Level Plan without taking these steps, individual improvement efforts such as the initiative to acquire a service for managing assets and land records may well incur cost and schedule overruns and fail to satisfy Interior’s trust management needs. Recommendations To ensure that Interior’s information systems are compatible and effectively satisfy Interior’s business needs, we recommend that, before making major investments in information technology systems to support trust operations, the Secretary direct the Chief Information Officer to develop an information systems architecture for Indian trust operations that (1) provides a high-level description of Interior’s mission and target concept of operations, (2) defines the business functions to be performed and the relationships among functions; the information needed to perform the functions; the users and locations of the functions and information; and the information systems needed to support the department’s business needs, (3) identifies the improvement projects to be undertaken, specifying what they will do, how they are interrelated, what data they will exchange, and what their relative priorities are, and (4) details specific standards and approaches that will be used to build or acquire systems, including hardware, software, communications, data management, security, and performance characteristics. To reduce the risks we identified with the effort to acquire a service for managing assets and land records, we recommend that the Secretary of the Interior direct the Chief Information Officer to (1) clearly define and validate functional requirements, security requirements, and data management requirements, (2) develop and implement an effective risk management plan, and (3) ensure that all project decisions are based on objective data and demonstrated project accomplishments, and are not schedule driven. Page 14 GAO/AIMD-99-53 Indian Trust Funds B-280590 Agency Comments and In its written comments on a draft of this report, Interior states that our oversight provides a valuable perspective and allows Interior to benefit Our Evaluation from our experience in dealing with similar issues at other agencies. However, Interior disagrees with the report’s conclusions and does not indicate whether it will implement the recommendations. In disagreeing with the report’s first conclusion (that Interior does not have reasonable assurance that its High Level Plan for improving Indian trust operations provides an effective solution for addressing long-standing management weaknesses), Interior states that although it recognizes the importance of a formal architecture and does not yet have one, the “lack of a formal architecture is not a significant impediment to success in this case, given the use of proven COTS products.” Interior also expresses confidence because this effort is smaller than the modernization efforts that have failed at other agencies like FAA. This position is not valid. The decision to use COTS products does not compensate for the lack of an integrated information system architecture for Indian trust operations. Such an architecture would have identified and preferably reengineered the business functions of trust operations, and then mapped these into information systems to support the business functions. Just choosing COTS products from the marketplace does not accomplish the same purpose. In fact, the close relationship between business functions and IT is the reason we focus on all 13 projects in the High Level Plan as a whole, even though, as Interior points out in its comments, only 4 of the projects are information technology systems projects. Further, small efforts, like IRS’ $17 million Cyberfile project, 15 as well as large ones, like FAA’s modernization, have failed due to poor program management, including lack of an architecture. With an estimated cost of $60 million for IT systems and an additional $54 million for data cleanup, the information systems supporting the 13 projects will have to be effectively managed if they are to succeed. Interior bases its decision to proceed with its IT acquisitions without a formal architecture (and without an estimated date for completing one) on the “pressing need for more responsive Indian trust systems.” However, moving to implement complex systems before developing an architecture 15 Tax Systems Modernization: Management and Technical Weaknesses Must Be Overcome To Achieve Success (GAO/T-AIMD-96-75, March 26, 1996) and Tax Systems Modernization: Cyberfile Project Was Poorly Planned and Managed (GAO/AIMD-96-140, August 26, 1996). Page 15 GAO/AIMD-99-53 Indian Trust Funds B-280590 does not expedite solutions. Instead, it greatly increases the chance of building duplicative systems, introducing potential integration problems, and perpetuating inefficient and overlapping business processes that currently exist in Indian trust operations. This is especially true in the case of TAAMS as Interior does not yet know whether the COTS product can effectively work with other Interior systems or with Interior-provided data. Also, as Interior notes in its comments, it consolidated TAAMS and LRIS from two separate projects into one because the “consolidation eliminated duplication within each system (80% of the data is shared), made better use of limited resources, and eliminated potential integration issues.” Similarly, Interior states that it is now considering streamlining the probate process and consolidating the BIA and OHA probate projects. Had Interior developed a sound architecture, it would have systematically identified the shared data and overlapping business processes before proposing either TAAMS and LRIS or BIA probate and OHA probate as separate projects in the High Level Plan. Moreover, it would have done the analysis needed to know whether other duplications and/or inconsistencies exist among its projects. Interior also disagrees with the report’s second conclusion that Interior does not have reasonable assurance that its acquisition of the new asset and land records management (TAAMS/LRIS) service will cost effectively satisfy trust management needs. Our report bases this conclusion on findings that Interior did not follow sound processes for defining TAAMS/ LRIS requirements, thoroughly analyzing technical alternatives before selecting an approach, or managing technical risk. Interior states that its requirements were adequately defined and that its requirements definition process consisted of conducting several requirements reviews with the end-user community and deciding “early on to adopt the business processes afforded through implementation of the COTS product.” Just as deciding to use COTS products does not compensate for the lack of an integrated system architecture for Indian trust operations, selecting a COTS product before thoroughly analyzing requirements does not constitute an effective requirements definition process. Further, while Interior says that it will adopt the business processes afforded through implementation of the COTS product, it has at the same time recognized that the COTS product does not meet all of its requirements and will have to be modified. For example, Interior must modify the COTS product to handle fractionated interests and title requirements that are unique to Indian ownership. Page 16 GAO/AIMD-99-53 Indian Trust Funds B-280590 Interior does not directly address the finding that it did not thoroughly analyze technical alternatives before choosing a vendor and a COTS product to provide asset and land records management services. As discussed in the report, these technical alternatives include (1) modifying existing legacy systems, (2) acquiring a COTS product and using existing Interior infrastructure resources, (3) building a system to provide the necessary capability, or (4) acquiring a service. Instead, Interior dismisses any use of the legacy systems, stating that the systems “. . . employ both outdated software products and processing techniques . . . ,” and “. . . would require a virtual rewrite;” does not address the second and third alternative at all; and states once again, without having performed a gap analysis, that “the use of COTS product, combined with a service bureau approach, does provide the Department an economical and timely solution.” Because it has not thoroughly analyzed all technical alternatives and does not have convincing, objective evidence to support its decision, there is no assurance that Interior has selected the most cost-effective alternative. Interior then describes several actions which it feels minimizes acquisition risk. Specifically, it “. . . established a risk management plan shortly after awarding the TAAMS contract”; will have other contractors review the work of the TAAMS contractor; and will evaluate the results of pilot testing. Because all of these actions occur after the vendor was selected and the contract awarded, they are not relevant to our finding that Interior did not follow a sound process for selecting an approach and, therefore, does not have reasonable assurance that its trust management needs will be met cost effectively. In its comments, Interior says “. . . a rigorous, standard approach was not used in identifying the requirements for TAAMS . . .”; and “. . . we would have preferred to use actual BIA data [in Operational Capabilities Demonstrations], but given the time constraints, we decided to use scripts . . .”. Further, Interior recognizes that it had to correct resulting errors identified in our report. Specifically, the Request for Proposal and/or the contract for TAAMS had to be changed to clarify terms such as “conflicts of interest, abuse, and self-dealing”; to correct the mistaken reference to Public Key encryption; and to require monthly delivery to the government of all data to facilitate import into other applications. However, because Interior does not explicitly recognize the flaws in its processes and does not acknowledge the relationship between these weaknesses and the errors that have already occurred, it has not committed to correcting these weaknesses and it is likely to repeat similar errors in the future. Page 17 GAO/AIMD-99-53 Indian Trust Funds B-280590 Interior also raises several subsidiary issues. It asserts that our review was incomplete because we did not assess the TAAMS vendor selection process, which, in Interior’s opinion, was necessary to determine if the TAAMS acquisition was cost effective. The objective of our audit was not to determine how Interior selected its vendor; it was to determine whether Interior had done the analysis needed to determine what was required and to select an approach to the project that would be cost-beneficial. How Interior selected its vendor is not relevant to that objective and was therefore not within the scope of our audit. Interior claims that we stopped the audit work “prematurely.” However, Interior does not cite any significant events that occurred or critical corrections made since the audit ended that would alter our conclusions. In fact, during the review, we evaluated every document provided by Interior. Moreover, this review was initiated, performed, and concluded after its objectives were completed according to its established schedule. The only deviation from schedule was made to accommodate Interior’s request for an additional 6 business days to comment on this report. Interior is concerned that we focused only on the TAAMS/LRIS project and therefore, were not in a position to make broad statements about the High Level Plan. In focusing on all IT aspects of the plan, we assessed the interrelationships of the individual 13 projects as well as the overall process for developing the plan. This enabled us to determine that Interior did not have reasonable assurance that the High Level Plan provides an effective solution for addressing its long-standing management weaknesses. We assessed the TAAMS/LRIS project because it was ongoing during our review, is one of the major IT projects in the High Level Plan, and illustrates fundamental problems with Interior’s approach. Finally, Interior states that once it deploys TAAMS, it will have the means to reengineer its business processes to the “industry standard.” This runs counter to the basic tenets of reengineering, that is, organizations should first reengineer business processes and then assess and acquire or build systems necessary to support those processes. This enables organizations to ensure that they implement optimal technical solutions and that they do not limit their business process alternatives or entrench themselves in ineffective ways of doing business. Interior needs to implement our recommendations to substantially reduce the risk to key IT systems in trust management operations. Interior’s Page 18 GAO/AIMD-99-53 Indian Trust Funds B-280590 comments are provided in their entirety in appendix I along with our detailed evaluation of them. We are sending copies of this report to Senator Daniel K. Inouye, Vice Chairman, Senate Committee on Indian Affairs and to Senator Robert C. Byrd, Senator Joseph I. Lieberman, Senator Ted Stevens, and Senator Fred Thompson, and to Representative Dan Burton, Representative George Miller, Representative David Obey, Representative Henry A. Waxman, Representative C.W. Bill Young, and Representative Don Young, in their capacities as Chairmen and Ranking Minority Members of the Senate Committee on Appropriations, Senate Committee on Governmental Affairs, House Committee on Appropriations, House Committee on Resources, and House Committee on Government Reform. We are also sending copies of this report to the Honorable Jacob J. Lew, Director, Office of Management and Budget, and to other interested congressional committees and Members of Congress. Copies will also be made available to others upon request. If you have any questions about this report, please call me at (202) 512- 6415. Other major contributors to this report are listed in appendix II. Sincerely yours, Dr. Rona B. Stillman Chief Scientist for Computers and Telecommunications Page 19 GAO/AIMD-99-53 Indian Trust Funds Appendix I Comments From the Department of the Interior AppIexndi Note: GAO comments supplementing those in the report text appear at the end of this appendix. Page 20 GAO/AIMD-99-53 Indian Trust Funds Appendix I Comments From the Department of the Interior Page 21 GAO/AIMD-99-53 Indian Trust Funds Appendix I Comments From the Department of the Interior Page 22 GAO/AIMD-99-53 Indian Trust Funds Appendix I Comments From the Department of the Interior See comment 1. See comment 2. See comment 3. Page 23 GAO/AIMD-99-53 Indian Trust Funds Appendix I Comments From the Department of the Interior See comment 4. See comment 5. See comment 6. See comment 7. Page 24 GAO/AIMD-99-53 Indian Trust Funds Appendix I Comments From the Department of the Interior See comment 8. Page 25 GAO/AIMD-99-53 Indian Trust Funds Appendix I Comments From the Department of the Interior See comment 9. Page 26 GAO/AIMD-99-53 Indian Trust Funds Appendix I Comments From the Department of the Interior The following are GAO’s comments on Interior’s March 19, 1999, letter responding to a draft of this report. GAO Comments 1. According to Interior’s High Level Plan (page 70), five projects are classified as data cleanup projects: OST data cleanup, BIA data cleanup, BIA probate backlog, OHA probate backlog, and BIA appraisal program. According to the schedules provided in the High Level Plan (pages 64 through 67) OST data cleanup was initiated in January 1998 and BIA data cleanup project began in August 1998. 2. Our intent was to present the sequence of events chronologically, not to imply that there was a change in direction in the middle of the TAAMS acquisition. We clarified the language in the report to reflect this more precisely. 3. The report does not state that the High Level Plan should include all high-level requirements. Our report makes the point that the high-level requirements for all 13 projects were not defined anywhere. 4. Although Interior’s letter indicates otherwise, neither the RFP nor the amendment included any definitions for the terms “conflicts of interest, abuse and self-dealing.” In subsequent correspondence to us, Interior officials told us that they believe these terms are commonly used and do not require additional definition. However, Interior requires that TAAMS implement safeguards to identify incidents of conflicts of interest, abuse, and self-dealing. Precise definition of requirements, not assumptions about “common usage” for terms that by their nature are subject to broad interpretation, is needed to implement systems features effectively. 5. The TAAMS RFP states this requirement as follows: “Access to the system shall at a minimum require unique user IDs with passwords. The system shall record unsuccessful attempts . . .” The parenthetical phrase discussing password length does not appear. After receiving a draft of this report, Interior issued an amendment to the contract containing the phrase. This is another example of inadequate requirement definition that Interior is addressing piecemeal and ad hoc, without correcting the fundamental process weaknesses that caused the problem. 6. Section J of the RFP contains a collection of data elements from different legacy systems, but it is not a data dictionary for TAAMS. Because the data elements required by TAAMS were not defined prior to asking Page 27 GAO/AIMD-99-53 Indian Trust Funds Appendix I Comments From the Department of the Interior vendors to respond to the TAAMS RFP, Interior has no assurance that the vendor’s product can handle all data elements crucial to Indian trust operations. 7. We are not suggesting a priori that the legacy system is a viable solution. Neither we nor Interior can make informed decisions without analyzing relevant data. We are pointing out that, consistent with the Clinger-Cohen Act and good IT investment practices, Interior should have evaluated all technical alternatives before selecting one. 8. Interior has quoted this statement out of context. The full sentence from our draft report states: “Specifically, although Interior concluded based on the results of its Request for Information that none of the COTS products available from responding vendors would meet all its requirements, Interior did not determine, for each COTS product, which requirements could not be satisfied and how difficult and expensive it would be to make the needed modifications." Our point is that Interior did not perform a gap analysis on products available in the marketplace to determine whether the COTS approach was optimum. According to a Mitretek official, the Mitretek study was completed after the Request for Proposal was issued and was intended to serve as the government’s independent cost estimate for use in source selection. 9. Interior is in error. While all projects do, indeed, contain some elements of risk, our point was that Interior was incurring and not mitigating unnecessarily high levels of risk because it does not have an integrated architecture for Indian trust operations and has not corrected fundamental weaknesses in its IT management processes. Page 28 GAO/AIMD-99-53 Indian Trust Funds Appendix I Major Contributors to This Report AppIexndi Accounting and Mike Koury, Assistant Director Naba Barkakati, Technical Assistant Director Information Chris Martin, Technical Assistant Director Management Division, Lou Schuster, Senior Auditor Cristina Chaplain, Communications Analyst Washington, D.C. Office of General Tom Armstrong, Assistant General Counsel Franklin Jackson, Senior Attorney Counsel (913827) eL rtet Page 29 GAO/AIMD-99-53 Indian Trust Funds Appendix I Page 30 GAO/AIMD-99-53 Indian Trust Funds Ordering Information The first copy of each GAO report and testimony is free. Additional copies are $2 each. Orders should be sent to the following address, accompanied by a check or money order made out to the Superintendent of Documents, when necessary, VISA and MasterCard credit cards are accepted, also. Orders for 100 or more copies to be mailed to a single address are discounted 25 percent. Orders by mail: U.S. General Accounting Office P.O. Box 37050 Washington, DC 20013 or visit: Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders may also be placed by calling (202) 512-6000 or by using fax number (202) 512-6061, or TDD (202) 512-2537. Each day, GAO issues a list of newly available reports and testimony. To receive facsimile copies of the daily list or any list from the past 30 days, please call (202) 512-6000 using a touchtone phone. A recorded menu will provide information on how to obtain these lists. For information on how to access GAO reports on the INTERNET, send an e-mail message with “info” in the body to: email@example.com or visit GAO’s World Wide Web Home Page at: http://www.gao.gov United States Bulk Rate General Accounting Office Postage & Fees Paid Washington, D.C. 20548-0001 GAO Permit No. GI00 Official Business Penalty for Private Use $300 Address Correction Requested
Indian Trust Funds: Interior Lacks Assurance That Trust Improvement Plan Will Be Effective
Published by the Government Accountability Office on 1999-04-28.
Below is a raw (and likely hideous) rendition of the original report. (PDF)