oversight

Bureau of Indian Affairs Schools: New Facilities Management Information System Promising, but Improved Data Accuracy Needed

Published by the Government Accountability Office on 2003-07-28.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

             United States General Accounting Office

GAO          Report to Congressional Committees




July 2003
             BUREAU OF INDIAN
             AFFAIRS SCHOOLS

             New Facilities
             Management
             Information System
             Promising, but
             Improved Data
             Accuracy Needed




GAO-03-692
                                               July 2003


                                               BUREAU OF INDIAN AFFAIRS SCHOOLS

                                               New Facilities Management Information
Highlights of GAO-03-692, a report to          System Promising, but Improved Data
Congressional Committees
                                               Accuracy Needed



The Bureau of Indian Affairs (BIA)             FMIS is designed to address the previous data system’s shortcomings and
is responsible for providing over              appears to have the capability to meet BIA’s management needs if the data
48,000 children with a safe place to           that are entered into FMIS are correct and timely. The old system was hard
learn. In response to concerns that            to use and did not readily provide data for maintenance and repair efforts.
data in its old information system             FMIS’s design appears to overcome these weaknesses. For example, FMIS
did not accurately reflect the
condition of facilities, BIA acquired
                                               has features that help facility managers make data more consistent, as well
a new system, called the Facilities            as tools for helping managers develop cost estimates for maintenance and
Management Information System                  construction projects.
(FMIS). GAO was asked to
determine whether FMIS addresses               BIA’s contractor has been correcting the data that were transferred to FMIS
the old system’s weaknesses and                from the previous system, but issues such as software compatibility
meets BIA’s management needs,                  problems between the contractor’s system and FMIS have delayed entry of
whether BIA has finished validating            some of the data for more than 1 year. BIA officials say that these problems
the accuracy of data entered into              are being addressed. They said the delay has not affected their ability to
FMIS from the old system, and how              prioritize or fund repair and construction projects, and our review of the
well the quality control measures              data indicated that most newly identified deficiencies will not need to be
are working for ensuring the
accuracy of new data being entered
                                               addressed for 2 to 5 years. Our review of data from 14 BIA schools and
into the system from individual                observations during site visits disclosed no instances in which these data
schools.                                       problems resulted in an unsafe learning environment for children.

                                               Most measures for controlling the quality of new data BIA employees are
                                               entering into the system for individual schools are not working well. BIA
GAO recommends that BIA                        has established a multilevel review process and training programs to help
establish better guidance and                  ensure that such data entries are complete and accurate, but BIA’s
performance expectations for
                                               contractor, in reviewing data at the end of this process, continues to find
employees who are responsible for
entering and reviewing the                     that nearly half of the proposed data entries coming through the system are
accuracy and completeness of                   inaccurate and incomplete. Data entries from one-third of 102 schools that
FMIS data. GAO also recommends                 entered data show a 100-percent error rate. As a result, BIA officials
that BIA officials periodically                continue to rely on their contractor to ensure that FMIS reflects accurate
analyze the extent and type of data            and complete data on the condition of BIA’s facilities.
errors being found during review in
order to identify training needs and           BIA School in South Dakota
other strategies for addressing any
continuing problems.




www.gao.gov/cgi-bin/getrpt?GAO-03-692.

To view the full report, including the scope
and methodology, click on the link above.
For more information, contact Marnie S.
Shaul (202) 512-6778.
Contents


Letter                                                                                     1
               Results in Brief                                                            2
               Background                                                                  5
               FMIS’s Design Addresses Previous Information System’s
                 Shortcomings and Appears to Have the Capability to Meet BIA’s
                 Needs                                                                     8
               Inventory and Backlog Data Have Been Validated, but Not
                 Transferred to FMIS                                                     13
               Control Measures in Place for Ensuring Data Quality Are Largely
                 Ineffective                                                             22
               Conclusion                                                                26
               Recommendation for Executive Action                                       27
               Agency Comments on Our Evaluation                                         27

Appendix I     Objectives, Scope, and Methodology                                        30



Appendix II    Comments from the Department of the Interior                              32



Appendix III   GAO Contacts and Staff Acknowledgments                                    34
               GAO Contacts                                                              34
               Staff Acknowledgments                                                     34


Tables
               Table 1. Description of Backlog Categories and Ranks                      10
               Table 2: School Facilities Listed in the FMIS Inventory Directly
                        Used by Children                                                 14
               Table 3: Categories of Unfunded FMIS Backlog Entries as of
                        October 2002                                                     17
               Table 4: Modifications to the Backlog Data from AME’s 100 Percent
                        Validation in 1999                                               18
               Table 5: Estimated Total Cost Changes to the FMIS Backlog for 14
                        Schools                                                          20
               Table 6: Extent of Disapproved Entries in FMIS among 102 Schools
                        Submitting Data, August 2001 to December 2002.                   23




               Page i                            GAO-03-692 BIA School Facilities Management
Figure
         Figure 1: FMIS Design and Implementation Timeline                                         7




         Abbreviations

         AME               Applied Management Engineering, Inc.
         BIA               Bureau of Indian Affairs
         FACCOM            Facility Construction Operations and Maintenance
         FMIS              Facilities Management Information System
         O&M               operations and maintenance
         OFMC              Office of Facilities Management and Constructions




         This is a work of the U.S. Government and is not subject to copyright protection in the
         United States. It may be reproduced and distributed in its entirety without further
         permission from GAO. It may contain copyrighted graphics, images or other materials.
         Permission from the copyright holder may be necessary should you wish to reproduce
         copyrighted materials separately from GAO’s product.




         Page ii                                    GAO-03-692 BIA School Facilities Management
United States General Accounting Office
Washington, DC 20548




                                   July 31, 2003

                                   Congressional Committees:

                                   The Bureau of Indian Affairs (BIA), which is responsible for ensuring that
                                   over 48,000 Indian students have school facilities that provide them with a
                                   safe and quality place to learn, works with Indian tribes to operate and
                                   maintain nearly 2,200 buildings at 171 elementary and secondary schools.1
                                   During fiscal year 2002, BIA’s budget for operating, maintaining,
                                   constructing, and repairing its school facilities was nearly $348 million.
                                   For many years, BIA and tribal employees have been tracking the
                                   condition of their school facilities using an automated information system.
                                   Tribal Indian education organizations2 and BIA have had longstanding
                                   concerns that the information system was inadequate for managing school
                                   facilities and did not accurately reflect how many buildings are in good
                                   condition or disrepair. As a result, BIA did not have accurate and complete
                                   information to determine how much annual funding is needed at each
                                   school site for heating, lighting, and other operating expenses, as well as
                                   how much current and future funding is needed at each site for making
                                   repairs and capital improvements, such as patching a roof or upgrading a
                                   fire alarm system.

                                   Recognizing the need to address these concerns, BIA is now in its third
                                   year of implementing a new Facilities Management Information System
                                   (FMIS) to replace its old information system.3 Prior to implementing FMIS,
                                   however, and to ensure that this new system met BIA’s management
                                   needs, BIA hired a consulting firm to identify weaknesses with the old
                                   system and recommend improvements for designing a new one. Once the


                                   1
                                    During school year 2001-2002, in addition to the 171 elementary and secondary schools,
                                   BIA funded 14 peripheral dormitories. These dormitories are established on reservations
                                   for Indian students who attend nearby public schools.
                                   2
                                    We discussed the issues concerning the school facilities with the Association of
                                   Community Tribal Schools, the National Indian School Board Association, the Native
                                   American Grant Schools Association, and the Association of Navajo Community Controlled
                                   School Boards.
                                   3
                                    BIA uses the FMIS to manage its entire Facilities Management Program. FMIS processes
                                   information about all BIA-owned or BIA-funded facilities. However, facilities other than
                                   educational facilities were considered to be out of our scope and were not included in our
                                   analysis.



                                   Page 1                                     GAO-03-692 BIA School Facilities Management
                   new system was designed and BIA was prepared to implement it, BIA
                   hired an engineering firm to validate all of the data to be transferred from
                   the old system into FMIS to ensure that the data were accurate and
                   complete. To maintain the integrity of these data over time, BIA also
                   introduced various quality control measures for helping ensure that facility
                   management staff would accurately enter new data into FMIS. Because of
                   concerns about the quality of the data in the former system, the Congress
                   directed us to conduct a review in this area.4 After discussions with
                   cognizant congressional committees and several Indian education
                   organizations about the information that would be most helpful, we
                   focused our efforts on (1) whether BIA’s new facilities management
                   information system addresses the former system’s shortcomings and
                   meets BIA’s needs for managing school facilities; (2) the status of BIA’s
                   effort to validate the accuracy and completeness of the data being
                   transferred from the old system into FMIS; and (3) how well BIA’s quality
                   control measures are working to ensure that new data entered into FMIS
                   are accurate and complete.

                   To conduct our review of BIA’s new information system, we reviewed the
                   studies that evaluated BIA’s old information system and determined the
                   extent that BIA’s new system implements recommendations. We evaluated
                   data currently in the FMIS database and accompanied BIA’s contractor on
                   selected site visits designed to validate and update information in the
                   database. We also interviewed school officials at 8 BIA-funded schools in
                   Arizona, New Mexico, and South Dakota. We selected these schools to
                   obtain a mix based on differences in their size, geographic location, and
                   type of school, such as day school or boarding school. We also interviewed
                   officials from BIA’s Office of Facilities Management and Construction,
                   Division of Safety and Risk Management, and Office of Indian Education
                   Programs, as well as officials from various Indian education organizations.
                   See appendix I for a more complete description of our scope and
                   methodology.

                   We conducted our work between June 2002 and July 2003 in accordance
                   with generally accepted government auditing standards.


                   FMIS is designed to address the previous information system’s
Results in Brief   shortcomings and appears to have the capability to meet BIA’s needs for


                   4
                       The mandate for this review is from Public Law 107-110.




                   Page 2                                        GAO-03-692 BIA School Facilities Management
managing its school facilities if the data that are entered are correct and
timely. Problems with the old system included difficulty of use and
inaccurate data. FMIS is designed as an easy-to-use system that allows
facility managers at the schools to enter data without having to remember
lengthy codes or compute calculations by hand. Many facility managers at
the schools we visited said FMIS is easier to use and contains features that
address weaknesses they had identified with the previous system. FMIS
also has features that can help BIA employees make accurate and
consistent data entries, including more specific categories and priority
rankings for repairs and improvements, and a software tool that helps
facility managers develop cost estimates for repair, replacement, or
construction projects. FMIS incorporates several modules to provide BIA
managers with enough information to help them make consistent
decisions for disbursing funds across the schools and for accurately and
consistently prioritizing repair projects and capital improvements.

BIA’s engineering contractor has identified needed corrections to the
inventory and condition data that were transferred to FMIS from the old
information system, but BIA has not made all of the data corrections in
FMIS. As of October 2002, FMIS contained information on almost 2,200
buildings at 171 elementary and secondary schools and showed an
estimated cost of over $642 million for repairs, capital improvements, and
construction to improve the facilities’ condition and meet changing
program needs. Regarding the inventory data used to provide schools with
annual funds for operations and maintenance (O&M), BIA’s contractor
completed its data validation effort in February 2003, and BIA expects that
the FMIS data will be updated by August 2003. BIA officials expect that
this corrected inventory data will enhance their ability to provide schools
with the appropriate amount of funding beginning in fiscal year
2004. Regarding the data used to identify and prioritize repair projects and
capital improvements, BIA’s contractor has been completing its cyclical
review of the condition of school facilities on time. The contractor
completed its first review of all school sites in 1999, identifying almost
75,000 adjustments that increased the list of repairs and improvements by
more than $265 million. In fiscal year 2002, the contractor visited
33 schools and identified corrections to add, delete, or adjust the FMIS
data. However, because of issues such as software compatibility problems
between the contractor’s system and FMIS, BIA had not entered the
results of these condition assessments into FMIS for over 1 year. BIA and
contractor officials said that these compatibility problems are being
addressed and that the delay did not significantly affect BIA’s ability to
appropriately prioritize or fund repair and construction projects or
negatively affect the children’s learning environment. Our review of the


Page 3                              GAO-03-692 BIA School Facilities Management
data for 14 of the schools showed that none of the unentered data were for
urgent or safety deficiencies that needed immediate attention; most were
maintenance deficiencies that would need attention over the next 2 to
5 years. Further, in our visits to 8 BIA schools, we did not observe that the
children were in facilities that did not provide them with a safe place to
learn.

The ability of FMIS to provide accurate and complete data depends on BIA
employees entering correct and timely information, but control measures
BIA has established for ensuring data quality have been largely ineffective.
BIA established a process whereby each data entry proposed by school
facility managers is reviewed for accuracy by BIA engineering personnel at
both an agency and a regional office, with final review and approval by
BIA’s engineering contractor and the central office before being accepted
into FMIS. Despite this process, however, BIA’s contractor continues to
find that nearly half of data entries proposed by facility managers and
approved by agency and regional office personnel are inaccurate and
incomplete, with a third of 102 schools that entered data experiencing a
100 percent error rate. Despite these ongoing problems, BIA officials have
not established criteria and performance goals that define their
expectations for the accuracy and completeness of data entered into
FMIS. In addition, BIA officials do not analyze the number and content of
data errors that would allow them to identify the type of additional
guidance or training needed, or target locations that could benefit from
more comprehensive technical assistance. BIA officials responsible for
FMIS said that one reason they have not taken such action is because they
had no line authority over the BIA employees that enter and review the
FMIS data, and such action would have had to occur at a higher level
within BIA. BIA officials said that an ongoing reorganization of the agency
and collaboration between BIA offices offers an opportunity to address
performance issues of BIA staff. Until BIA employee performance
improves, BIA officials responsible for FMIS continue to rely on their
contractor to ensure that FMIS reflects accurate and complete data about
the condition of BIA’s facilities.

We are making recommendations to the Secretary of the Department of
the Interior to direct the Assistant Secretary for Indian Affairs and the
Office of Indian Education Programs to take steps to improve the ability of
BIA employees to maintain the integrity of FMIS data. These
recommendations include establishing criteria, guidance, and performance
measures for maintaining accurate, complete, and timely FMIS data. These
recommendations also include a requirement to periodically analyze the



Page 4                              GAO-03-692 BIA School Facilities Management
             number and content of data errors in order to identify strategies for
             correcting and preventing them.

             BIA administers funding for the operation, maintenance, construction, and
Background   repair of school facilities at 171 elementary and secondary schools in
             23 states. These schools are located primarily in rural areas and small
             towns and serve Indian students living on or near reservations. Many of
             these schools include not only educational buildings, but also dormitories
             and supporting infrastructure such as water and sewer systems. BIA
             operates 64 of the schools directly while the others are operated by tribes
             through separate grant or contract agreements. We previously reported on
             issues related to the condition of BIA school facilities in 1997 and 2001.5

             BIA’s Office of Facilities Management and Construction (OFMC) is
             responsible for overseeing FMIS. At both BIA-operated and tribal-operated
             schools, it is the responsibility of the facility managers to enter data about
             the inventory and condition of their schools into the system. Prior to
             acceptance into FMIS, these draft data entries are reviewed and approved
             by facility managers at BIA agency and regional offices respectively,
             before final review and approval by a BIA contractor and BIA’s central
             office.

             For 22 years BIA relied on its Facility Construction Operations and
             Maintenance (FACCOM) system to maintain inventory data for its annual
             O&M program, as well as “backlog” data that reflect repairs and
             improvements needed outside of the annual maintenance program to
             improve the facilities’ condition now and in the future. These data assist
             BIA in monitoring the status of facilities repair and new construction
             projects and identifying funding needs for O&M and renovation. However,
             as BIA’s needs began to change, BIA managers realized that FACCOM had
             limitations and acknowledged that there were serious concerns with the
             accuracy and completeness of these data.

             As shown in figure 1, BIA’s efforts to replace FACCOM began in 1995 when
             one of its contractors issued a report about the FACCOM system’s
             shortcomings and recommended actions for improvement. In 1995, BIA
             entered into a contract with Anteon Corporation, a system developer, to


             5
             See U.S. General Accounting Office, School Facilities: Reported Condition and Costs to
             Repair Schools Funded by Bureau of Indian Affairs, GAO/HEHS-98-47 (Washington D.C.:
             Dec. 31, 1997) and BIA and DOD Schools: Student Achievement and Other Characteristics
             Often Differ from Public Schools’, GAO-01-934 (Washington D.C.: Sept. 28, 2001).




             Page 5                                  GAO-03-692 BIA School Facilities Management
design the new management information system. Relying on government
standards,6 BIA worked with Anteon Corporation to design the new
system and address FACCOM’s shortcomings. In 1999, BIA contracted
with an engineering firm, Applied Management Engineering, Inc. (AME), to
conduct a survey at all school sites in order to validate the schools’
condition data and to verify the presence of buildings and their use.
According to BIA officials, after AME validated each school’s inventory
and condition data, and BIA approved it, the data were accepted into
FMIS.




6
 The standards are the Federal Information Processing Standards from the National
Institute of Standards and Technology.




Page 6                                    GAO-03-692 BIA School Facilities Management
Figure 1: FMIS Design and Implementation Timeline



    1995                1996             1997            1998                1999                2000                2001                 2002              2003
    March               BIA began        Anteon              Anteon          January             March               June                 January           February
    The contracting     reviewing and    Corporation         Corporation     BIA awarded         AME completed       FMIS was             AME began its     AME completed
    firm's report is    evaluating its   began               continued       AME                 a 100 percent       available            second cycle      its validation and
    issued that         management       designing and       designing and   a contract to       validation of the   at all BIA offices   to reassess and   verificaton of
    evaluated           needs to         developing          developing      verify and          schools' backlog    and schools          validate the      the schools'
    FACCOM's            determine an     FMIS.               FMIS.           validate the        data and            nationwide.          backlog data      inventory data.
    weaknesses and      appropriate                                          schools'            of the inventory                         for 20 percent
    recommended         design for the                                       backlog data        associated with     November             of the schools    April
    actions to verify   information                                          and to validate     the backlog.        BIA awarded          each year.        BIA awarded AME
    existing data and   system.                                              the inventory                           AME a contract                         a contract to
    ensure integrity                                                         associated          June                to verify and                          continue its work
    of future data.                                                          with the backlog.   BIA transferred     validate each                          of verifying and
                                                                                                 the AME-            of the school's                        validating the
    April                                                                                        validated           inventory data.                        schools' inventory
    BIA awarded                                                                                  backlog data                                               and backlog. AME
    Anteon                                                                                       into FMIS.                                                 began its third cycle
    Corporation                                                                                                                                             to reassess and
    a contract to                                                                                August                                                     validate the backlog
    design a new                                                                                 AME began                                                  data for 33 percent
    facilities                                                                                   its first cycle                                            of the schools
    management                                                                                   to reassess                                                each year.a
    information                                                                                  and validate
    system.                                                                                      the backlog                                                August
                                                                                                 data for 20                                                BIA expects to
                                                                                                 percent of the                                             transfer the AME-
                                                                                                 schools each                                               validated inventory
                                                                                                 year.                                                      data into FMIS.

                                                                                                 November
                                                                                                 BIA began
                                                                                                 implementing
                                                                                                 FMIS at BIA
                                                                                                 offices and
                                                                                                 schools.


Source: BIA data.
                                                         a
                                                         PL. 107-110 required BIA to update the data in FMIS every 3 years.


                                                         From fiscal years 1995 through 2002, BIA spent nearly $12.5 million to
                                                         develop and begin implementing FMIS. These costs include about
                                                         $8 million for contractor expenses and over $2.6 million for BIA in-house
                                                         expenses, which covered the design of FMIS and ongoing technical
                                                         support. During fiscal years 1999- 2003, BIA spent about $13 million for the
                                                         AME contract covering the validation of inventory and condition data and
                                                         other engineering support activities. To operate FMIS, BIA expects to
                                                         spend about $1.7 million annually through fiscal year 2006 for contractor
                                                         expenses and about $250,000 for in-house expenses. In fiscal year
                                                         2007, BIA hopes to move to an annual steady rate of about $750,000 for
                                                         contractor costs and about $250,000 for in-house costs. To continue having
                                                         AME reassess and validate the schools’ inventory and condition data, BIA


                                                         Page 7                                                      GAO-03-692 BIA School Facilities Management
                       projects to spend over $8.3 million from fiscal years 2004 through 2006 on
                       contract expenses.


                       Recognizing the FACCOM system’s shortcomings, BIA worked with its
FMIS’ Design           system developer to design a new management information system that
Addresses Previous     would assist in resolving many of the weaknesses identified with the old
                       system, including those related to difficulty of use and accuracy of data.
Information System’s   FMIS is more user friendly and it is designed to meet facility managers’
Shortcomings and       needs at all levels within BIA by serving as both an information
                       management system and as a project management tool. FMIS incorporates
Appears to Have the    modules, including the inventory and backlog modules, which help facility
Capability to Meet     managers make decisions regarding the condition of the school facilities
BIA’s Needs            to provide a safe environment for their students. The inventory module
                       contains information such as the physical characteristics and use of
                       buildings and is used to make funding decisions for annual operating
                       expenses and routine maintenance. The backlog module contains data that
                       tracks detailed information about the physical condition of a school’s
                       facility and is used to prioritize and fund repair projects, capital
                       improvements, and construction.


FMIS Is Designed to    FMIS is designed to better support the day-to-day activities of the facility
Resolve Shortcomings   management staff by being more user friendly. FMIS is a Windows-based
with the Previous      system that provides a point-and-click feature, which makes it easy to
                       navigate the system without having to remember codes. This is important
Information System     for FMIS users, because some facility managers have little prior exposure
                       to computers. Facility managers we interviewed at the school sites and
                       agency and regional offices said that compared to FACCOM, FMIS is
                       better and easier to use. One facility manager in Arizona said FMIS is
                       easier to use because the system automatically sends messages to him
                       when changes are made to the data, allowing him to instantly see when
                       updates have been made to his school’s data. Another facility manager in
                       Arizona said FMIS’s automated functions, such as drop-down menus,
                       make it user friendly. Finally, according to BIA officials, FACCOM was
                       only accessible by about 3-4 percent of facility managers, which did not
                       include facility managers located at the schools. FMIS is designed to be
                       accessible by all of BIA’s facility managers, including those at the school
                       sites, via an Internet connection. Although most of the facility managers
                       we visited at the schools said FMIS was better compared to FACCOM,




                       Page 8                              GAO-03-692 BIA School Facilities Management
many had been unable to access FMIS at school sites since December of
2001 due to a court order that shut down access to BIA’s Internet site.7
These facility managers had to travel to agency or regional offices to enter
data or had to forward data to these offices for data entry.

FMIS is also designed to help BIA employees improve the accuracy of the
data, in part through automated mechanisms that help facility managers
consistently describe the category and rank of backlog entries and the
funding needed to address them.8 One difficulty under FACCOM was that
entries listed in the backlog were often categorized incorrectly with
inflated priorities, making it difficult to determine which projects needed
immediate attention. To address this problem, FMIS is designed to restrict
who can enter safety deficiencies, which are given first priority for
funding. In addition, FMIS is designed to only accept entries that have
been reviewed and approved by three different levels of BIA management
and the contractor. Further, BIA refined the definitions of how backlog
items should be categorized and ranked to help facility managers use the
definitions consistently. The definitions now include nine categories and a
ranking system for determining the priority of the items entered into the
backlog (see table 1). These nine categories describe whether the
deficiency at the school affects safety and endangers students’ lives;
violates an environmental, disability, or energy standard; is a maintenance
or capital improvement item; or requires an emergency repair. In addition,
the system ranks items using a scale from one through three—with one
describing the most severe deficiency. The backlog entries ranked as a
“1” will most likely be funded first because they are the highest priority.
FMIS requires facility managers to enter an associated category and rank
for all items entered into the backlog. Such a process, with its greater



7
 In the Cobell v. Norton litigation concerning the government’s management of Native
American trust funds, a U.S. District Court judge, on December 5, 2001, ordered the
Department of the Interior to disconnect from the Internet all information technology
systems that house or provide access to individual Indian trust data and all computers
within the custody and control of Interior, its employees, and contractors that have access
to individual trust data. In an order dated December 17, 2001, the judge held that Interior
could reconnect systems to the Internet with the approval of a court-appointed Special
Master. As of June 16, 2003, the BIA facilities management information system had not
been reconnected to the Internet.
8
 Entries to FMIS are identified through physical inspections and entered by the BIA
contractor, the Division of Safety and Risk Management, facilities management field staff,
or the Division of Environmental Services. Cost estimates are developed from these
inspections and are routed through the contractor from either the facilities management
field staff or the Division of Environmental Services, according to BIA officials.




Page 9                                     GAO-03-692 BIA School Facilities Management
specificity in how to categorize and rank deficiencies, can help facility
managers improve consistency in the data entered for all 171 schools.

Table 1. Description of Backlog Categories and Ranks

 Category of violation   Code and rank
 Emergency               U-1—an unforeseen event in which danger exists that could
                         reasonably be expected to cause death, physical harm,
                         or property damage.
 Safety                  S-1—serious deficiency that poses a threat to safety and
                         health, such as fire safety violations.
                         S-2—moderate deficiency such as poor lighting or trip or fall
                         hazards.
 Physical plant          M-1—deficiency related to the structural, mechanical, or
 (maintenance)           electrical systems that render it inoperable, such as the
                         deterioration of a roof that causes interior building
                         damage.
                         M-2—deficiency related to the facility, systems, or grounds,
                         such as replacing worn door locks that are inoperable
                         M-3—functional facility equipment exceeds its normal life
                         expectancy.
 Handicapped             H-1—serious code deficiency, such as the lack of accessible
                         door hardware.
                         H-2—violation of codes and standards, such as the lack of
                         code compliant accessible handrails.
 Environmental           X-1—serious code deficiency that poses a threat to life or
                         property, such as removing friable asbestos in occupied
                         areas.
                         X-2—code deficiency, such as removing asbestos floor tiles
                         from a building.
 Predictive renewals     R-3—backlogs identified for future planning to determine the
                         life cycle needs beyond the 5-year plan.
 New construction        C-1—to replace buildings with serious code/safety deficiencies
                         or to abate numerous high cost code violations that
                         meet or exceed the replacement cost rule.
                         C-2—to accommodate functional or programmatic needs, such
                         as replacing an undersized dining room to
                         accommodate the student population.
 Programmatic capital    P-2—to change the functional space or to accommodate
 improvements            programmatic space needs, such as retrofitting an
                         existing classroom into a computer laboratory.
 Energy                  E-2—violation of energy codes and standards, such as
                         upgrading or replacing inefficient heating systems.
                         E-3—deficiencies, which when corrected will reduce energy
                         use, such as replacing weather seals on exterior doors.
Source: BIA data.




Page 10                                  GAO-03-692 BIA School Facilities Management
                             To help facility managers develop accurate and consistent cost estimates
                             to address backlog items, FMIS is designed to operate with a software
                             program that helps facility managers accomplish industry standard cost
                             estimates for replacement, renovation, or construction projects over
                             $5,000. A facility manager at a school site we visited said that this software
                             tool eliminates the need to make calculations by hand, and thus greatly
                             assists him in estimating accurate costs for school projects.

                             Another accuracy-related area that plagued FACCOM was that projects
                             would continue on the active backlog list even after completion. The FMIS
                             backlog module is designed with a “backlog completion screen” that
                             stores completion dates, costs, and narrative comments. This function
                             helps facility managers monitor the length of time that funded backlog
                             items remained on the backlog without being completed. Unlike FACCOM,
                             FMIS is designed with a tracking function that identifies the name of the
                             person who entered or updated a particular backlog item. Managers can
                             use this function to seek clarification or justification for items that have
                             not been completed within a reasonable timeframe.


FMIS Is Designed to Assist   The FMIS inventory module, one of six modules in the system,9 is designed
BIA in Making Funding        to assist BIA in determining operations and maintenance funding for its
Decisions for Annual         school facilities. Specifically, the inventory module is designed to manage
                             information about all of BIA’s school buildings, rooms, towers (such as a
Operating Expenses and       water tower), and grounds, along with their associated inventory items,
Routine Maintenance          such as stairs, sidewalks, or playgrounds. The inventory module also
                             details if the property is owned, operated, or maintained by BIA directly or
                             under contract or grant. The inventory module is designed with the
                             capacity to integrate with other FMIS modules in order to generate reports
                             and provide detailed documentation for federal funding purposes.

                             At the schools we visited, several facility managers said the inventory
                             function in FMIS helps them to better manage their school facilities. For
                             example, one school facility manager in Arizona said FMIS helps him more
                             accurately keep track of his school’s inventory and allowed him to enter
                             the information that is necessary for BIA to make good funding decisions.
                             Another facility manager in South Dakota said that because only one staff



                             9
                               The six modules that make up FMIS are inventory, budget, project management, backlog,
                             reports, and work tickets. BIA is in the process of implementing a seventh module, the
                             work plan module.




                             Page 11                                   GAO-03-692 BIA School Facilities Management
                             person can enter information into the inventory module, his school is able
                             to maintain consistency in inventory changes and additions.


FMIS Is Designed to Help     The FMIS backlog module is designed to help BIA officials prioritize and
BIA Prioritize and Justify   make funding decisions for needed repair projects and capital
Funding for Repair           improvements. The backlog collects and tracks condition data related to
                             deficiencies, capital improvements, or construction for specific inventory
Projects and Capital         items, such as classrooms, sidewalks, or utility systems. These data are
Improvements                 entered into FMIS by a facility manager, by safety officers as a part of a
                             safety inspection, or by BIA’s contractor. Although the backlog module
                             can store information about any identified deficiency, the only items that
                             are reported as part of BIA’s backlog are those with an estimated cost of
                             more than $1,000 to fix. These deficiencies, which may be grouped
                             together to form repair, replacement, or construction projects are
                             maintained in the backlog until funded and complete. During our site
                             visits, many of the facility managers said the backlog module helped them
                             to better manage their facilities. For example, one school facility manager
                             in Arizona said FMIS’s ability to store digital pictures was helpful because
                             a picture of a deficiency could be sent to the regional facility manager and
                             reviewed without the facility manager traveling nearly 284 miles to the
                             school.

                             Facility managers at schools use the information in the backlog module to
                             justify funding needs for repair projects and capital improvements at their
                             schools. BIA management officials allocating funding among the schools
                             said the data in the backlog module allow them to determine which
                             deficiencies are related to student safety and need to be addressed
                             immediately, and which are related to capital improvements, such as roof
                             replacement, that are planned for the future. BIA officials said they use the
                             backlog data to help improve the physical condition of their schools in
                             order to provide a safe and healthy learning environment for the students.




                             Page 12                             GAO-03-692 BIA School Facilities Management
                             BIA’s engineering contractor has corrected the inventory and backlog data
Inventory and                that existed in the old data system, but BIA has not transferred all
Backlog Data Have            corrected information to the new FMIS. The contractor completed its
                             validation of the inventory data in February 2003, and BIA plans to transfer
Been Validated, but          the corrected data into the FMIS by August 2003. BIA officials expect that
Not Transferred to           the corrected inventory data, in conjunction with improvement to the
                             existing funding formula,10 will enhance their ability to better match
FMIS                         funding with annual expenses for utilities and routine maintenance at each
                             school site. For the backlog data, BIA’s contractor is in the third year of
                             the second cycle review of the condition of BIA-funded schools as
                             planned. In fiscal year 2002, the contractor visited 33 schools and
                             identified corrections to add, delete, or adjust the FMIS data. BIA,
                             however, had not entered the results of these condition assessments into
                             FMIS for over 1 year. BIA officials attribute the long delays in correcting
                             the FMIS condition data to a revised process for verifying contractor data
                             and to software compatibility problems that they say are being addressed.


Contractor Has Finished      The FMIS inventory module contains data on BIA facilities, including
Correcting Inventory Data;   almost 2,200 separate buildings that are occupied by or used for BIA-
BIA Plans to Transfer the    funded schools. More than 50 percent (1,146) of the buildings are used
                             directly by children, as shown in table 2.
Data into FMIS Later in
2003




                             10
                               BIA’s contractor is developing a new formula for allocating operations and maintenance
                             funding to the schools. BIA expects to implement this formula in fiscal year 2005, after
                             consulting with the tribes about the change, according to BIA officials.




                             Page 13                                   GAO-03-692 BIA School Facilities Management
Table 2: School Facilities Listed in the FMIS Inventory Directly Used by Children

    Buildings in the inventory used by children                            Number of buildings
    Schools                                                                                    796
    Dormitories                                                                                153
    Gymnasium/sports facilities                                                                  49
    Multipurpose/auditorium                                                                      64
    Library                                                                                      21
    Kitchen/cafeteria                                                                            63
    Total                                                                                    1,146
Source: BIA data.

Note: Most of the remaining buildings included: fire stations, shops, garages, pump houses, heating
and utility plants, sewage and water treatment plants, warehouses, storage facilities, communications
facilities and equipment, animal shelters, and greenhouses.


Accurate and up-to-date inventory data are crucial to the operation of the
entire FMIS because other modules rely on inventory data for planning
and prioritizing the work and for identifying and prioritizing funding
needs. For example:

•     School facility managers access inventory data when planning and
      scheduling routine maintenance of their facilities, grounds, and
      equipment. For example, one FMIS module acts as a scheduling tool to
      inform facilities managers about work, such as preventive
      maintenance, that needs to be done to buildings, equipment, and other
      physical assets listed in the inventory.

•     BIA’s Office of Facilities Management and Construction uses inventory
      data in the formula that determines the amount of operations and
      maintenance funding allocated to each school location. Distribution of
      this funding is calculated using a formula that includes such inventory
      data as the square footage of rooms in each building and systems that
      support the facility such as heating and cooling systems. Funding
      distributions have been a particular source of contention for Office of
      Indian Education officials, who told us that inaccuracies in the
      inventory data have led to inequities in how the money is apportioned.
      Accurate data that are collected using a methodology that is consistent
      from site to site is a necessary component for demonstrating the
      fairness of the process.

BIA’s engineering contractor, AME, has remained on schedule in its effort
to improve the accuracy of the inventory information currently in FMIS. In
conducting this effort, AME visited each school and collected inventory



Page 14                                         GAO-03-692 BIA School Facilities Management
data using a standardized, industry-based approach to help ensure that
information on all facilities is uniformly collected and recorded. AME
completed the first phase of this effort in 2000, when it visited each school
to verify and update the inventory data listed in the FACCOM before its
transfer to FMIS in that year, according to a BIA official. This phase, which
focused on the more general aspects of the inventory, was aimed at such
matters as identifying which buildings were still in use, the use of the
facility, and who owns it. AME completed the second phase of the
improvement effort in February 2003. This second phase, which took
longer than the first phase, involved a more extensive measurement of the
buildings and updated the drawings of floor space, grounds, infrastructure,
and utility lines. BIA’s current plans call for replacing the existing data in
FMIS with this updated and corrected data by August 2003.

Our preliminary review of the new data generated during the second phase
of AME’s work indicates that the inventory figures may change
considerably for some schools. At our request, AME officials provided
revised square footage data for more than 90 buildings (such as classroom
buildings, dormitories, multipurpose buildings, and offices) at 13 different
schools. Overall, the revised measurements for these buildings decreased
the total square footage by about 3 percent, but the range in increases and
decreases at each school varied significantly. For 9 of the schools the
decrease in square footage ranged from less than 1 percent to more than
13 percent; increases for the remaining 4 schools, ranged from less than
1 percent to almost 18 percent. We do not know if these results will be
typical for all schools.11 One BIA official indicated, however, that some
schools were likely to experience greater changes than others in the
square footage that would qualify for O&M funding.

BIA officials said that the improved data, along with improvements to the
funding formula, will help ensure the various schools that their share of
O&M funding was objectively and accurately determined. However,
whether this corrected inventory data will be transferred to FMIS in time
for making fiscal year 2004 funding decisions is uncertain. BIA may yet
face some implementation problems as it moves into the final months of
putting this information in place. For example, during 2002, BIA attempted
to run the O&M funding formula using the existing FMIS inventory data for


11
 BIA officials cited three primary reasons for square footage variance (1) new buildings or
additions to existing buildings that have not been added to the inventory, (2) unoccupied
buildings that should be categorized as “non-maintained” and (3) those buildings where
ownership and occupancy are unclear.




Page 15                                    GAO-03-692 BIA School Facilities Management
                              the first time and experienced problems with the data and software. While
                              the agency has had a year to work out these problems, introduction of the
                              updated inventory data may hold its own unforeseen problems. If such
                              problems are encountered and remain unresolved, a BIA official told us
                              that the agency would continue to use the data currently in the system to
                              allocate the O&M funding for fiscal year 2004.


Contractor Is on Schedule     The backlog data in FMIS reflects actions and funding needed to improve
in Validating Backlog Data,   the condition of facilities and infrastructure at the various schools now
but BIA Is Behind             and in the future. Most of the items listed in the backlog provide detail for
                              repairs needed over the next 5 years to correct what is wrong with a
Schedule in Incorporating     facility such as a leaky roof, the presence of asbestos, or a violation of
the Updated Information       handicapped codes and standards. However, FMIS also includes entries
                              for capital improvements that will need to be addressed beyond 5 years to
                              upgrade specific building components such as replacing lighting and
                              power systems, siding, and carpeting as well as future construction to
                              replace, renovate, and add buildings to accommodate program needs.12
                              Accurate and up-to-date backlog data are important because FMIS
                              contains formulas that use these data to allow BIA to make informed
                              decisions not only about which projects are in greatest need of attention,
                              but also how much money is needed to fund them each year. As of
                              October 2002, for example, BIA schools had a backlog of unfunded repairs
                              and improvements with an estimated cost over $640 million (see table 3);
                              FMIS shows that almost two-thirds of this amount may be needed within
                              the next 5 years.




                              12
                                   In the FMIS, these entries are categorized as “predictive renewals.”




                              Page 16                                        GAO-03-692 BIA School Facilities Management
Table 3: Categories of Unfunded FMIS Backlog Entries as of October 2002

 Dollars in millions
                                   Deficiencies      Deficiencies   Deficiencies                             New
                                      should be        should be      should be    Planned work      construction
                                     addressed     addressed in 1 addressed in 3   scheduled for and other capital
 Category                          within 1 year       to 2 years     to 5 years    5 to 10 years  improvements          Total
 Emergency—a danger
 exists that could cause
 physical harm or property
 damage, such as severe
 damage to a roof from a
                                               a
 storm.                                 $1,939                                                                         $1,939a
 Safety—deficiency that
 affects safety and health,
 such as fire safety.                    65,304          $20,700            $62                                        86,066
 Physical plant
 (maintenance)—structural,
 mechanical, or electrical
 deficiency, such as aging
 roof or defective boiler.               32,960           53,499         154,115                                      240,573
 Handicapped—violates
 code and standards, such
 as lack of accessible door
 hardware.                                1,926           14,989             63                                        16,978
 Environmental—deficiency
 such as the presence of
 friable asbestos in occupied
 areas or leaking storage
 tanks.                                   1,441           13,766                                                       15,207
 Predictive renewals—to
 anticipate replacement of
 building components, such
 as carpeting or roofing
                                                                                                b
 systems.                                                                               $73,895                        73,895b
 New construction—to
 replace entire school or
 buildings.                                                                                               $112,468    112,468
 Programmatic capital
 improvements—to renovate
 and add to buildings to
 meet program needs.                                                                                         42,301    42,301
 Energy—violates energy
 standards, such as
 inefficient heating or lighting
 systems.                                                                 53,186                                       53,186
 Total                                 $103,570         $102,954       $207,426          $73,895          $154,769 $642,613
                                          (16%)            (16%)          (32%)            (12%)             (24%)     (100%)
Source: BIA data.




                                             Page 17                               GAO-03-692 BIA School Facilities Management
a
  A BIA official said that the amounts requested in the emergency category were for reimbursement of
funds already spent to correct any outstanding deficiencies.
b
 The estimated amount for predictive renewals is not reported to the Congress as part of the backlog
total because these entries are not yet deficiencies, but BIA is anticipating the time period in which
they will become deficiencies and need to be addressed, according to a BIA official.
AME currently validates the backlog for each school in a 3-year cycle.
During its first review, in 1999, AME conducted a 100-percent validation of
the backlog data prior to transferring that data from the old FACCOM
system to the new FMIS backlog module. In that review, AME updated the
backlog data by confirming entries already in the system, updating the
costs estimated to conduct the work, deleting entries for duplicate items
or completed work, and identifying new entries. Results of the validation
effort, as shown in table 4, increased the backlog by more than
$265 million; almost 28 percent of the total backlog of $960 million that
existed in 1999.

Table 4: Modifications to the Backlog Data from AME’s 100-Percent Validation in
1999

    Dollars in millions
    Change to backlog data                     Number of changes                      Cost change
    New deficiencies                                           39,143                      $865,850
                                                                                                     a
    Modifications                                              14,579
    Completed work                                              6,964                      (169,928)
    Deleted (duplicate entries)                                14,186                      (430,348)
    Total changes                                              74,872                      $265,574
Source: BIA data.
a
Data were not available.


AME has since started the second review of each school, which consists of
updating this information for a certain percentage of the facilities each
year. AME updated 20 percent of the facilities in each year 2001 and
2002. For 2003, AME is on track to increase the percentage of facilities
reviewed each year to 34 percent to comply with recent changes in the
law.13 These updates involve visually inspecting the architectural,
structural, mechanical, and electrical components of each facility to
determine if action is still needed and to update the estimated costs. In



13
 Under Public Law 107-110 (2002), section 1125(b)(6), BIA is required to periodically
update the FMIS; in fulfilling this requirement, BIA’s contractor is updating the inventory
and assessing the condition of about one-third of the school facilities each year.




Page 18                                          GAO-03-692 BIA School Facilities Management
addition, AME identifies new deficiencies, including the extent to which
handicapped accessibility requirements are met, and verifying estimated
costs.

Our review of some updates conducted during fiscal year 2002 indicates
that AME’s reviews will continue to result in substantial changes in
backlog data. We obtained data for 14 of the schools reviewed between
February and April 2002.14 For the 14 schools, the AME update resulted in
a net increase of almost $11 million (see table 5) in the unfunded FMIS
backlog of more than $39 million, an increase of about 28 percent. Part of
the change involved modifications to deficiencies already in the backlog
inventory, such as revising cost estimates and deleting projects that had
been completed or no longer needed but were still listed in the backlog as
ongoing. However, a large part of the change involved adding new
deficiencies to the backlog. In all, there were almost 650 new backlog
entries, and more than 75 percent of these entries were for deficiencies
identified at school facilities and dormitories, where children are the
primary occupants.15 For example, there were more than 200 entries for
dormitories with an estimated cost of almost $4 million. None of these
entries were for urgent or safety deficiencies that needed immediate
attention; most were maintenance deficiencies that will need attention
over the next 2 to 5 years, such as repairing lighting and plumbing systems,
carpeting, and ceilings.




14
     This was slightly more than 40 percent of the 33 schools AME reviewed during 2002.
15
     Includes dormitories, gymnasiums, libraries, and classroom facilities.




Page 19                                        GAO-03-692 BIA School Facilities Management
Table 5: Estimated Total Cost Changes to the FMIS Backlog for 14 Schools

 Dollars in millions
 Category                                                     Net cost change to backlog
 Physical plant (maintenance)                                                        $5,536
 Safety                                                                                (329)
 Environmental                                                                          (76)
 Handicapped                                                                          1,120
 New construction—school replacement                                                  1,903
 Programmatic capital improvements—renovations to                                     1,140
 meet program needs
 Energy                                                                                 228
 Predictive renewals                                                                  1,397
 Total change                                                                       $10,919
Source: BIA data.


While the contractor’s field assessments are proceeding on schedule, there
have been significant delays in incorporating the 2002 updated backlog
data into FMIS. In fiscal year 2001, the first year of the updates, the
contractor assessed the condition of 39 schools and transferred the data
from its information system to the FMIS without problems, according to
BIA and contractor officials. However, officials said that in fiscal year
2002, the untimely transfer of data from the field assessments to the
contractor’s information system16 and software compatibility problems
between the contractor’s information system and FMIS delayed the update
of backlog data for 33 schools for over a year. These implementation
problems occurred for such reasons as the following:

•    BIA added a new function to the FMIS, which took 6 months to
     implement, rather than the 2 months that they had planned, according
     to a BIA official. This new function involved using the FMIS, for the
     first time, to generate the O&M funding amounts to be distributed to
     the various schools.

•    A change to the backlog category and ranking system in FMIS created
     duplicate entries that took time to resolve. In the 2002 assessments,
     some deficiencies that were already in the system were recategorized,
     and when the update was transferred to FMIS a duplicate entry was


16
  Typically, the contractor takes about 9 weeks to collect, analyze, and transfer data from
the contractor’s information system to FMIS, according to BIA and AME officials.




Page 20                                     GAO-03-692 BIA School Facilities Management
   created instead of overwriting the old entry. According to BIA and
   contractor officials, this software compatibility problem has largely
   been addressed. As of April 2003, data for 27 of the 33 schools had been
   entered into FMIS.

A BIA official told us that delays in introducing these updates into the
FMIS backlog could have some impact on their ability to prioritize or fund
repair and capital improvement projects, but not a significant impact for
two reasons. First, the deficiencies that receive the highest points in the
project ranking system are safety deficiencies, which are identified and
updated in an annual safety inspection by BIA’s safety officers and not
AME. Second, the most critical deficiencies at the schools were identified
in the first assessment in 1999, and during this second assessment, AME is
finding very few deficiencies that would be funded within 1 to 2 years that
were not already in the system. However, one official acknowledged that
since the FMIS ranks the schools for major repair and capital
improvement projects based on the points applied to each deficiency, if
AME’s assessment indicated an increase in the severity of a deficiency and
that change was not reflected in the system, that school could be ranked
lower than if the data were up to date.

Our site visits to 8 BIA-funded schools did not disclose any instances
where serious problems were not being addressed. While facilities
management staff and school principals pointed out problems with their
facilities, we did not observe that the children were in an unsafe learning
environment with obvious safety or repair issues. The problems we
observed were either of a less serious nature, or if serious, were being
addressed in some form. For example, at a boarding school in Arizona the
principal said that the fire alarm system for the school building and
dormitory had been improperly installed and had to be replaced. While
waiting for the funding for a new system, which had been approved, they
had to use funds from the school budget to hire extra people to stand fire
watch 24 hours a day.




Page 21                             GAO-03-692 BIA School Facilities Management
                           The ability of FMIS to provide accurate and complete data depends on BIA
Control Measures in        employees entering correct and timely information, but review processes
Place for Ensuring         and training programs BIA has established for ensuring data quality have
                           been largely ineffective. Although adherence to federal control standards17
Data Quality Are           are a major part of providing reasonable assurance that the objectives of
Largely Ineffective        the agency are being achieved, half of the entries proposed by BIA
                           employees are incorrect or incomplete and are flagged by BIA’s
                           contractor. Discussions with BIA employees indicate that some employees
                           are unclear about their responsibilities in maintaining the accuracy of
                           FMIS data, and the high error rates in data entry indicate that additional
                           training is needed in some locations to improve performance. BIA has not
                           analyzed the information it has available about the content and origination
                           of the data errors to determine the type of additional training that might be
                           needed or to target locations with the highest error rates for technical
                           assistance. Further, BIA has not established criteria or performance goals
                           that define its expectations for the accuracy and completeness of the data
                           for employees that enter and review this information. BIA’s Office of
                           Facilities Management and Construction, which manages FMIS, did not
                           until recently have authority to establish criteria or performance goals for
                           agency and regional office personnel that are responsible for reviewing
                           and approving data entries by school facility managers. BIA’s OFMC still
                           does not have similar authority over school facility managers who
                           originate most data entries. Under BIA’s current organization, such action
                           would have to be taken by the Office of Indian Education Programs.

Despite BIA Review         BIA established a multilevel review process as a control to prevent
Process, Its Contractor    problems related to inaccurate and incomplete data entries to BIA’s
Rejects Half of All FMIS   information system. In this process, each entry that school facility
                           managers propose for the backlog is first reviewed and approved by BIA
Data Entries               facility management personnel at an agency and a regional office before it
                           is sent to BIA’s contractor, AME, for review and approval,18 with final
                           approval by BIA’s central office. After approval has been obtained from
                           each level, the status of the entry is changed from “draft” to “accepted” in
                           FMIS, according to BIA.



                           17
                            See U.S. General Accounting Office, Standards for Internal Control in the Federal
                           Government, GAO/AIMD-00-21.3.1, (Washington D.C., Nov. 1999).
                           18
                            Because changes to the square footage in the inventory affects operations and
                           maintenance funding, additions or subtractions to the inventory are also reviewed at the
                           agency and regional offices before it goes to the central office for final approval; the
                           contractor, however, does not review these entries.




                           Page 22                                    GAO-03-692 BIA School Facilities Management
                             Although BIA established this multilevel review process to improve the
                             quality of the data entered into FMIS, AME continues to reject half of the
                             proposed entries because they are inaccurate and incomplete. For
                             example, between August 2001 and December 2002, out of more than
                             650 entries to the backlog made by facilities management staff from
                             102 schools, more than 300, or almost 50 percent, were rejected by the
                             contractor. BIA documents show that the incidence of the errors among
                             the 102 schools was widespread. In all, 73 of the 102 schools entering data
                             had one or more entries disapproved, and 33 of them had all entries
                             disapproved (see table 6).

                             Table 6: Extent of Disapproved Entries in FMIS among 102 Schools Submitting
                             Data, August 2001 to December 2002.

                              Percentage of entries disapproved           Number of schools in each category
                              0                                                                            29
                              1-25                                                                         12
                              26-50                                                                        12
                              51-74                                                                        12
                              75-99                                                                          4
                              100                                                                          33
                             Source: BIA data.




Roles and Responsibilities   Facility management staff at the regional and agency offices appear to
May Be Unclear to            have the necessary background to fulfill their responsibilities for screening
Employees Reviewing          and correcting inaccurate and incomplete data entries; however, it may be
                             unclear to some reviewers what their role is in this process. Federal
Data Entries                 control standards require that employees have the requisite knowledge,
                             skills, and abilities to perform their job appropriately and have clear roles
                             and responsibilities outlined in their job descriptions. Among the facilities
                             management staffs in the agency and regional offices that we interviewed,
                             most had an engineering background or had significant experience in
                             facilities management and they were aware that maintaining accurate and
                             complete data in the FMIS was important. However, facilities managers we
                             interviewed at two regional offices each had a different view about their
                             role in the review process. One said that it was important that the backlog
                             entries are consistent and that two staff had been designated to review the
                             FMIS entries from the schools and agencies in the region before being sent
                             on to the central office for review by the contractor. The second manager
                             said he reviewed the entries so that he knew what new additions were




                             Page 23                               GAO-03-692 BIA School Facilities Management
                          being proposed for the backlog; he did not consider it his role to critique
                          the entries for accuracy and completeness.


Training Program May be   Although most BIA locations have staffs that have received training to use
Insufficient to Prepare   FMIS, the extent of the errors indicates that employees may not be
Users                     receiving the kind of training needed to create accurate and complete data
                          entries. Standards for internal control in the federal government include a
                          commitment to competence, which includes the provision that employees
                          receive the appropriate training necessary to improve their performance.
                          BIA has developed a training program that is intended to provide FMIS
                          users with sufficient information to operate the system. All FMIS users
                          receive 40 hours of training before they are given a password that allows
                          them to access the system, according to BIA; a review of BIA’s training log
                          indicates that about 70 percent of the schools have at least one staff
                          member onsite that has received this training.19

                          While training appears to be adequate in terms of providing staff with the
                          basic skills needed to use the computer-based applications, what appears
                          to be lacking in the training program is more specific instruction and
                          guidance on the kinds of information that are needed to enter an accurate
                          and complete deficiency to the backlog. BIA officials acknowledged that a
                          user manual to provide this type of guidance was lacking and should be
                          developed. BIA officials said developing the FMIS training has been a
                          challenge because the facilities management staffs have different levels of
                          knowledge about computers—for some facility managers, FMIS training
                          was their first exposure to using a computer. The training programs were
                          developed to meet the needs of this diverse group of users, according to
                          the contractor that developed and provides the training. In our site visits
                          we asked users about the training that they had received for using FMIS;
                          almost all of the staff that we interviewed said that they were pleased with
                          the training they received and believed that it had prepared them for using
                          FMIS. In addition, they said that when they did have problems they
                          contacted central office with questions and/or problems with using the
                          system, the response was prompt and very helpful.

                          However, the engineering contractor indicated that the varying levels of
                          experience and expertise is a difficulty affecting staff’s ability to input data



                          19
                            For those schools that do not have trained staff, the facility manager at the agency office
                          is responsible for updating the data, according to a BIA official.




                          Page 24                                      GAO-03-692 BIA School Facilities Management
                             successfully. He said that there are more than 180 sites with education
                             buildings for which data must be entered and the level of knowledge and
                             expertise about facilities and the kinds of information needed for an FMIS
                             entry varies widely—particularly in those sites where the turnover rate of
                             facilities management staff is high. We were unable to obtain
                             comprehensive data on the turnover rate for the facility management staff,
                             but among the schools that we visited, the facilities managers’ length of
                             employment at their current location ranged from 2 years to 17 years.


BIA Does Not Use             BIA officials have information on the number and reasons that data entries
Information on Error Rates   are rejected at each location, but said that they had not used this
to Improve Employee          information to provide performance counseling to employees or modify
                             the training and guidance in this regard. Standards for internal control in
Performance                  the federal government include the provision that employees receive the
                             feedback necessary to improve their performance. Analyzing the extent
                             and content of data errors would be helpful to determine the type and
                             amount of additional training and guidance needed to improve employee
                             performance at schools, agencies, and regional offices and target them
                             appropriately for technical assistance. BIA data we reviewed indicated
                             that the reasons for disapproval generally fell into one of four groups and
                             were consistent with historical data entry problems experienced under the
                             old FACCOM system. For example,

                             •   more detailed description was needed for the deficiency; a roof repair,
                                 for example, required specific information about the kind of roof and
                                 its size;

                             •   questionable cost estimates involving labor rates or material costs;

                             •   duplicate entry for a deficiency already in the backlog; and

                             •   wrong category and/or rank for the entry such as categorizing the
                                 replacement of asbestos floor tiles as maintenance rather than
                                 environmental, which are funded from different sources.

                             Of these problems, most of the rejected backlog entries generally related
                             to insufficient detail to accurately estimate the cost to address the
                             deficiency, according to an AME official who reviewed and rejected many
                             of these entries. The kind of detail that is needed to successfully enter a
                             deficiency can be seen in an example involving the repair of a leaking roof.
                             To adequately estimate the cost for this repair, information is needed
                             about the size of the area that needs repair or replacement, the



                             Page 25                              GAO-03-692 BIA School Facilities Management
                       composition of the roof (such as asphalt shingles or tile), and other
                       associated components (such as whether skylights or gutters are present
                       and whether they also need to be replaced). We reviewed one entry that
                       had been rejected by the contractor because it came through the review
                       process without information about the roof’s size or its composition.


Employees Not Held     The ineffectiveness of BIA’s guidance, training, and review processes to
Accountable for FMIS   minimize inaccurate or incomplete data entries by its employees suggests
Data Integrity         that accountability is another issue that deserves attention. Federal
                       standards require that agencies clearly establish authority and
                       responsibility for achieving agency goals and hold their employees
                       accountable for performing their assigned responsibilities in a competent
                       manner. During our review, the organization of BIA was such that the
                       office that manages FMIS did not have line authority to establish
                       performance criteria and standards for the BIA employees that entered
                       and reviewed the FMIS data.

                       In April 2003, BIA announced a new organization plan. This plan may offer
                       OFMC greater opportunity to establish performance criteria and standards
                       for facility managers at agency and regional offices. BIA officials told us
                       the reorganization would not provide OFMC with line authority for facility
                       managers at schools. However, the Director of the Office of Indian
                       Education programs said that his office would work with OFMC to
                       establish comparable performance criteria and standards for school
                       facility managers.


                       FMIS is designed to assist BIA employees improve the quality of
Conclusion             information used to manage school facilities, but the quality of the
                       decisions that BIA makes for managing the operations and maintenance,
                       repair, and construction of its facilities is directly dependent upon BIA
                       employees entering correct and timely information. Currently, the FMIS
                       data that BIA uses for making its decisions are improving as the data are
                       updated by its contractor and entered into FMIS—to date the inventory
                       data have been updated and the contractor is in the third year of assessing
                       the condition of the schools and updating the backlog for the second time.
                       However, challenges remain in BIA’s efforts to improve the quality of data
                       entered by its employees. Although BIA has implemented controls for
                       ensuring the accuracy and completeness of the FMIS data entered and
                       reviewed by BIA employees, they do not work effectively. Without the role
                       of the contractor as a reviewer of new entries by field staff and in
                       conducting site visits to verify and update the data, the quality of the FMIS


                       Page 26                             GAO-03-692 BIA School Facilities Management
                     data could quickly become inaccurate and out of date. BIA has not taken
                     the necessary steps to hold its staff accountable for data accuracy or to
                     use the available information on why the problems exist to develop
                     training programs and target technical assistance where it is needed. Such
                     actions are needed if BIA is to rely on its employees, rather than the
                     contractor, to ensure that it provides a safe and quality learning
                     environment for Indian children.

                     To better enable BIA to rely on its employees for maintaining accurate and
Recommendation for   complete information in the FMIS, we recommend that the Office of the
Executive Action     Assistant Secretary of Indian Affairs

                     •   establish data standards for accuracy and completeness of FMIS data
                         and related performance criteria for BIA employees who are
                         responsible for entering and reviewing the data and

                     •   analyze available error data and use this information to provide its
                         employees with the necessary training, guidance, and technical
                         assistance to improve performance.


                     We provided a draft of this report to the Department of the Interior for its
Agency Comments on   review and comment. Interior’s comments are provided in appendix II. In
Our Evaluation       its written comments, Interior agreed with our findings and
                     recommendations and said that BIA is establishing a special working
                     group to develop a plan to address our recommendations. In addition, BIA
                     will consider our comments and observations as it continues to develop
                     and implement the FMIS.


                     We will send copies of this report to the Secretary of the Department of
                     Interior, relevant congressional committees, Indian education
                     organizations, and other interested parties, and will make copies available
                     to others upon request. In addition, the report will be available at no
                     charge on GAO’s Web site at http://www.gao.gov. Please contact me at




                     Page 27                              GAO-03-692 BIA School Facilities Management
(202) 512-6778 if you or your staff have any questions about this report.
Other major contributors to this report are listed in appendix III.




Marnie S. Shaul
Director, Education, Workforce,
 and Income Security Issues




Page 28                             GAO-03-692 BIA School Facilities Management
List of Congressional Committees

The Honorable Ben Nighthorse Campbell
Chairman
The Honorable Daniel K. Inouye
Vice Chairman, Committee on Indian Affairs
United States Senate

The Honorable Judd Gregg
Chairman
The Honorable Edward Kennedy
Ranking Minority Member
Committee on Health, Education, Labor, and Pensions
United States Senate

The Honorable Ted Stevens
Chairman
The Honorable Robert C. Byrd
Ranking Minority Member
Committee on Appropriations
United States Senate

The Honorable Richard Pombo
Chairman
The Honorable Nick J. Rahall, II
Ranking Minority Member
Committee on Resources
House of Representatives

The Honorable John A. Boehner
Chairman
The Honorable George Miller
Ranking Minority Member
Committee on Education and the Workforce
House of Representatives

The Honorable C.W. Bill Young
Chairman
The Honorable David R. Obey
Ranking Minority Member
Committee on Appropriations
House of Representatives



Page 29                            GAO-03-692 BIA School Facilities Management
              Appendix I: Objectives, Scope, and
Appendix I: Objectives, Scope, and
              Methodology



Methodology

              The objectives of this study were to determine (1) whether the Bureau of
              Indian Affairs (BIA) new facilities management information system (FMIS)
              addresses the former system’s shortcomings and meets BIA’s needs for
              managing school facilities; (2) the status of BIA’s effort to validate the
              accuracy and completeness of the data being transferred from the old
              system into FMIS; and (3) how well BIA’s quality control measures are
              working to ensure that new data entered into FMIS are accurate and
              complete.

              To determine the extent to which FMIS was designed to address
              weaknesses of the previous data processing system and how the new
              system meets BIA’s facility management needs we reviewed contractor
              reports, BIA documentation on the FMIS, and interviewed contractors and
              BIA headquarters, regional, agency, and school facility management staff.
              First, we reviewed the needs assessment studies conducted by
              independent contractors to identify old system weaknesses and the
              recommendations made for addressing the system problems. We then
              reviewed the FMIS documentation to determine whether the system
              addressed weaknesses identified in the needs assessment. In addition, we
              conducted interviews with some of the contractors hired by BIA to build
              and implement the system. We also interviewed BIA officials at the Office
              of Facilities Management and Construction, the Division of Safety and
              Risk Management, and the Office of Indian Education Programs about
              improvements in the new system and how it meets their management
              needs. Finally, to understand how well school facility management staff
              received FMIS, we conducted site visits to 8 schools in Arizona, New
              Mexico, and South Dakota. We selected these schools to obtain a mix
              based on their differences in size, geographic location, type of school
              (i.e., grade level, day school, or on-reservation boarding school), and
              whether it was BIA-operated or tribal-operated. At these schools, we
              interviewed facility managers, education line officers, principals, and tribal
              officials. We also interviewed facility management staff at two regional
              and five agency offices that provide facility management services to the
              schools.

              To determine the extent to which the FMIS inventory and backlog data are
              accurate and complete we used three methods. First, we obtained data
              from two sources (1) a copy of the FMIS database and (2) a copy of the
              contractor file of backlog data from the fiscal year 2002 condition
              assessments of 14 schools. The data in these files were assessed for
              reliability, which included looking for missing data, the relationship of one
              data element to another, values beyond a given range, and dates outside of
              valid time frames. We determined that the data were sufficiently reliable


              Page 30                              GAO-03-692 BIA School Facilities Management
Appendix I: Objectives, Scope, and
Methodology




for the purposes of this report. We also calculated summary statistics of
the data from these files. Second, we evaluated BIA’s multilevel review
process. For this analysis, we obtained data on the number of backlog
entries made by facility managers and the number that had been accepted
and rejected by the contractor. In addition, we reviewed the log of entries
that had been rejected by the contractor to understand the reasons for
these rejections. We also interviewed the contractor and BIA central office
and regional and agency facility management staff about this review
process. Finally, we accompanied BIA’s engineering contractor on site
visits to 2 schools in Arizona where we evaluated their methodologies for
data collection and validation of the inventory and backlog data.

To determine how well BIA’s internal control measures are working for its
FMIS we first reviewed our standards on internal controls to identify
controls that apply to an organization’s management of an information
system. We then compared the BIA controls with those identified in our
reports to evaluate the effectiveness of the BIA controls. Additionally, we
interviewed staff at the BIA central office about the internal controls they
had in place for the FMIS. We also interviewed staff at the regional and
agency offices and the schools about one of the controls—the
effectiveness of the training they received.




Page 31                              GAO-03-692 BIA School Facilities Management
             Appendix II: Comments from the Department of the Interior
Appendix II: Comments from the Department
of the Interior




             Page 32                                  GAO-03-692 BIA School Facilities Management
Appendix II: Comments from the Department of the Interior




Page 33                                  GAO-03-692 BIA School Facilities Management
                  Appendix III: GAO Contacts and Staff
Appendix III: GAO Contacts and Staff
                  Acknowledgments



Acknowledgments

                  Cindy Ayers, Assistant Director (206) 654-5591
GAO Contacts      Nancy Purvine, Analyst-in-Charge (206) 287-4878


                  In addition to the individuals mentioned above, Jessica Botsford, Maya
Staff             Chakko, Terry Dorn, David Gill, Barbara Johnson, Nathan Morris, Stan
Acknowledgments   Stenerson, and Michelle Zapata made significant contributions to this
                  report.




(130156)
                  Page 34                                GAO-03-692 BIA School Facilities Management
                         The General Accounting Office, the audit, evaluation and investigative arm of
GAO’s Mission            Congress, exists to support Congress in meeting its constitutional responsibilities
                         and to help improve the performance and accountability of the federal
                         government for the American people. GAO examines the use of public funds;
                         evaluates federal programs and policies; and provides analyses,
                         recommendations, and other assistance to help Congress make informed
                         oversight, policy, and funding decisions. GAO’s commitment to good government
                         is reflected in its core values of accountability, integrity, and reliability.


                         The fastest and easiest way to obtain copies of GAO documents at no cost is
Obtaining Copies of      through the Internet. GAO’s Web site (www.gao.gov) contains abstracts and full-
GAO Reports and          text files of current reports and testimony and an expanding archive of older
                         products. The Web site features a search engine to help you locate documents
Testimony                using key words and phrases. You can print these documents in their entirety,
                         including charts and other graphics.
                         Each day, GAO issues a list of newly released reports, testimony, and
                         correspondence. GAO posts this list, known as “Today’s Reports,” on its Web site
                         daily. The list contains links to the full-text document files. To have GAO e-mail
                         this list to you every afternoon, go to www.gao.gov and select “Subscribe to daily
                         E-mail alert for newly released products” under the GAO Reports heading.


Order by Mail or Phone   The first copy of each printed report is free. Additional copies are $2 each. A
                         check or money order should be made out to the Superintendent of Documents.
                         GAO also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a
                         single address are discounted 25 percent. Orders should be sent to:
                         U.S. General Accounting Office
                         441 G Street NW, Room LM
                         Washington, D.C. 20548
                         To order by Phone:     Voice:    (202) 512-6000
                                                TDD:      (202) 512-2537
                                                Fax:      (202) 512-6061


                         Contact:
To Report Fraud,
                         Web site: www.gao.gov/fraudnet/fraudnet.htm
Waste, and Abuse in      E-mail: fraudnet@gao.gov
Federal Programs         Automated answering system: (800) 424-5454 or (202) 512-7470


                         Jeff Nelligan, Managing Director, NelliganJ@gao.gov (202) 512-4800
Public Affairs           U.S. General Accounting Office, 441 G Street NW, Room 7149
                         Washington, D.C. 20548