oversight

General Services Administration: STAR-PBS' New Program for Tracking and Managing Real Property

Published by the Government Accountability Office on 1999-10-12.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                United States General Accounting Office

GAO             Report to Congressional Requesters




October 1999

                GENERAL SERVICES
                ADMINISTRATION
                STAR—PBS’ New
                Program for Tracking
                and Managing Real
                Property




GAO/GGD-00-12
United States General Accounting Office                                              General Government Division
Washington, D.C. 20548




                                     B-281769
                                     October 12, 1999

                                     The Honorable Bob Franks
                                     Chairman
                                     The Honorable Robert Wise, Jr.
                                     Ranking Democratic Member
                                     Subcommittee on Economic Development,
                                       Public Buildings, Hazardous Materials,
                                       and Pipeline Transportation
                                     Committee on Transportation and Infrastructure
                                     House of Representatives

                                     This report responds to your request for a review of the General Services
                                     Administration’s (GSA) Public Buildings Service’s (PBS) System for
                                     Tracking and Administering Real Property (STAR). Your Subcommittee
                                     was concerned about problems that were being reported to it concerning
                                     STAR. As agreed with the Subcommittee, our objectives were to determine

                                 •   the functions STAR performs;
                                 •   whether users were having problems using STAR;
                                 •   the steps PBS had taken or is taking to address any user problems; and
                                 •   the actions PBS had taken or plans to take to ensure that STAR data are
                                     accurate, reliable, and consistent.

                                     As agreed with your offices, we did not review the technical aspects of
                                     STAR because GSA’s Office of Inspector General (IG) is reviewing the
                                     technical and systemic issues relating to STAR. The IG expects to issue its
                                     report by the end of 1999.

                                     STAR is a real estate inventory management software application that
Results in Brief                     maintains data on projects, leases, buildings, and space assignmentsthat
                                     is, who is in what space—in an integrated database environment. The data
                                     in STAR are used to generate bills to clients for their assigned space,
                                     develop budget plans, track and manage leases, and evaluate performance.
                                     STAR’s input/output functions include

                                 •   direct on-screen access to the data,
                                 •   the ability to update the data,
                                 •   access to standard reports, and
                                 •   the ability to sort data and create ad hoc reports.




                                     Page 1         GAO/GGD-00-12 STAR—Program for Tracking and Managing Real Property
  B-281769




  Users identified problems with STAR as well as benefits derived from
  using the system. The 128 users we interviewed identified 18 types of
  problems that they had experienced using STAR. These problems included
  complaints such as taking too long to enter data and having too many steps
  or screens. According to the STAR project manager, all of the problems
  that were reported to us by users had been reported to the STAR project
  team through the contractor-operated STAR Help Desk. Also, STAR users
  had submitted nearly 8,000 requests, which were recorded on service
  tickets, to the Help Desk between October 1997 and August 1999. These
  tickets, many of them concerning the same issue, identified problems that
  users reported having when working with STAR as well as STAR
  enhancement suggestions and other issues related to STAR’s operations.
  Further, 116 of the 128 users we interviewed saw at least 1 benefit from
  using STAR, such as data being current and readily available.

  The STAR Help Desk and a PBS Change Review Team (CRT) were set up
  to solve problems identified by STAR users and to resolve other issues
  with STAR. By the end of November 1999, changes and enhancements will
  have resulted in eight versions of STAR. Further, additional contract help
  had been provided to assist the regions with STAR problems and training.

  PBS has taken various actions to try to ensure that STAR has accurate,
  reliable, and consistent data.

• STAR has been designed with built-in features to help ensure the reliability
  of some data elements. For example, work queues were to automatically
  generate requests for such things as approvals of leases, corrections, or
  notifications of missing information; work queues could also be used
  manually to send information.
• PBS ran edits against data before they were transferred to STAR and ran
  additional edits after data were transferred.
• PBS initiated an effort to validate all space assignments.
• PBS conducted a special study and produced a report that identified data
                                                                   1
  problems and made recommendations for solving the problems.
• A team was set up to determine how to implement the report’s
  recommendations. PBS has started implementing some of these
  recommendations.
• PBS is developing a quality control system.




  1
      Data Issues in PBS, PBS, April 1999.




  Page 2                GAO/GGD-00-12 STAR—Program for Tracking and Managing Real Property
             B-281769




             With the introduction of STAR and other changes, a major cultural change
Background   has occurred in GSA’s facilities management. According to the PBS report,

             “The simultaneous launches of New Pricing, STAR, Occupancy Agreements, and a host of
             other re-engineered business line processes and software constitutes the most significant
             change to the way PBS does its business since the creation of the Federal Buildings Fund.”

             The mission of PBS is to effectively manage over 8,000 owned and leased
             federal real properties in the United States. PBS had previously used a
             computer program called PLUS+ to track and manage its real properties.
             In 1994, PBS’ Central Office decided that the systems it used to support its
             mission were antiquated and needed to be replaced. New software was
             needed to support real property inventory, leasing, space assignments, and
             billing-related activities. PBS did market research analyses from March
             1994 through October 1995, evaluating various real estate software
             packages.

             In March 1994, PBS began its initial efforts to obtain new software. It
             published a notice in the Commerce Business Daily seeking sources to
             supply software products to support management operations for leased
             and owned public buildings. Two responses to this notice were received.
             However, PBS determined that neither of these sources met its functional
             needs.

             PBS also hired a contractor to identify any existing property management
             software that might meet its needs. The contractor developed a list of 400
             software products that it said spanned all facets of PBS’ business
             functions. Using PBS’ needs as criteria, this list was reduced to 36
             products, then to 6 products for further study. However, PBS concluded
             that none of these products was capable of satisfying all of its minimum
             requirements, such as the ability and capacity to concurrently manage over
             8,000 owned and leased properties.

             During discussions with real estate companies and Fortune 500 companies
             about real estate practices and procedures, PBS discovered a proprietary
             system called Permanent Record of Managed Property Transactions
             (PROMPT). This system had been implemented by AT&T during the
             summer of 1994. According to PBS officials, PBS ran a 4-month pilot test in
             1995 on PROMPT. During this test, 30 percent of PBS’ data were
             diagrammed and selected data elements on leases and space assignments,
             as well as other areas, were entered into the system. The test was
             completed in October 1995. PBS concluded that although PROMPT could
             meet its needs, the data were not in the format that it wanted.



             Page 3           GAO/GGD-00-12 STAR—Program for Tracking and Managing Real Property
              B-281769




              In July 1996, an acquisition plan for STAR was approved by GSA’s
              Associate Administrator for Acquisition Policy. This plan called for a sole
              source contract with AT&T for the use of PROMPT, which PBS renamed
              STAR. On January 17, 1997, the contract, which included a license
              agreement granting use of PROMPT in perpetuity unless canceled by the
              licensee, was signed with AT&T. The contract was for labor hours on a
              best-effort performance basis to convert PROMPT version 2.7 into STAR,
              which met PBS’ requirements. The base contract was for $8.3 million. Of
              this amount, $4.3 million was for the perpetual right to use the national
              license; the remaining $4.0 million was for software maintenance, a STAR
              reference guide, training courses and manuals, technical support services,
              hotline support, and travel. The contract also included several options. If
              all of these options were exercised, the total cost of the contract would
              have been $17.0 million. On October 19, 1998, the quantities in one of the
              options—option four—were increased adding 1,000 users, 310 training
              classes, 2,900 sets of training manuals, and 23,200 staff hours of technical
              support. This raised the value of the contract to $21.8 million, which was
              an increase of $4.8 million.

              PBS began rolling out STAR in October 1997, and it was available in all
              regions by January 1998. The first rent bills generated by STAR were sent
              to agencies in July 1998.

              PBS officials said that the introduction of STAR, among other things, has
              led to a cultural change that has had many effects in PBS. The officials
              noted that three changes particularly, affected the realty specialists who
              maintain STAR. First, realty specialists are now required to enter the data
              directly into the STAR system; second, they are to be held directly
              responsible for data accuracy; and third, they are responsible for billing
              the client agencies. In the past, the realty specialists filled out data entry
              sheets and data entry clerks entered the data. It was not clear at that time
              who was responsible for the data accuracy. Further, in the past, realty
              specialists were not involved in the actual billing of the clients.

              To determine what functions STAR performs, we observed how STAR
Scope and     operates; interviewed officials responsible for the development of STAR;
Methodology   and reviewed contract documents, training manuals, and the draft desk
              reference manual for STAR.

              To identify problems that users were experiencing using STAR, we
              judgmentally selected four regions, two of which were identified by PBS
              officials as having problems implementing STAR, one that was the lead
              region for STAR, and one from which congressional staff had received



              Page 4         GAO/GGD-00-12 STAR—Program for Tracking and Managing Real Property
                     B-281769




                     complaints. We also included the headquarters unit to get its perspective
                     on STAR. PBS officials identified the users of STAR at these locations, of
                     which realty specialists and realty assistants were identified as the major
                     users. We then interviewed 128 users, 81 of which were realty specialists
                     or realty assistants, in the 5 locations and asked them about the problems
                     they have had using STAR. Our results cannot be projected to the total
                     universe of STAR users.

                     Further, we reviewed Help Desk reports that organized user complaints
                     about STAR into subject categories. Because of the way that the issues
                     were categorized in the report and the actual number of items reported to
                     the Help Desk—almost 8,000—we were not able to directly compare our
                     interview-identified problems with those reported to the Help Desk.
                     However, we discussed the problems that users had identified in our
                     interviews, and problems that had been reported to the Help Desk, with
                     the STAR project manager. Further, we reviewed some Help Desk reports
                     that showed specific problems identified.

                     To identify the steps that PBS has taken or is taking to deal with user
                     problems, we interviewed officials who were responsible for addressing
                     STAR problems and reviewed documents on changes being made to
                     correct problems and improve STAR.

                     Finally, to determine what PBS has done or plans to do to ensure STAR
                     data integrity, we interviewed officials and reviewed plans addressing PBS’
                     efforts to ensure that STAR data are accurate, reliable, and consistent. We
                     did not independently assess the accuracy, reliability, and consistency of
                     STAR data.

                     We did our work at GSA/PBS Headquarters and the National Capital
                     Region in Washington, D.C.; the Southeast Sunbelt Region in Atlanta, GA;
                     the New England Region in Boston, MA; and the Mid-Atlantic Region in
                     Philadelphia, PA. Our work was done in accordance with generally
                     accepted government auditing standards. We requested comments on a
                     draft of this report from the Administrator of GSA. GSA’s written
                     comments are discussed at the end of this letter.


STAR—A System for    STAR is a real estate inventory management software application that
                     maintains data on projects, leases, buildings, and space assignmentsthat
Tracking and         is, who is in what space—in an integrated database environment. The data
Administering Real   in STAR have been used to generate bills to clients for their assigned
Property


                     Page 5        GAO/GGD-00-12 STAR—Program for Tracking and Managing Real Property
                                              B-281769




                                              space, develop budget plans, track and manage leases, and evaluate
                                              performance. STAR’s input/output functions include

                                          •   direct on-screen access to the data,
                                          •   the ability to update the data,
                                          •   access to standard reports, and
                                          •   the ability to sort data and create ad hoc reports.

                                              To develop the fiscal year 2001 budget, STAR data are being used to
                                              support the development of the over $5.5 billion rental revenue estimate
                                              and as the basis for the Galaxy system’s roughly $3.1 billion estimate of
                                                                       2
                                              PBS lease expenditures.

                                              STAR has seven major modules: (1) Space Management, (2) Client Billing,
                                              (3) Project Administration, (4) Billing, (5) Work Queue, (6) Administration,
                                              and (7) Reports. Table 1 describes each of these modules.


Table 1: Description of STAR’s Seven Major Modules
Module                                    Description
Space Management                          Supports the recording and processing of data related to buildings, space, occupants, and
                                          occupancy. Data stored include building location, square footage, lease terms, types of
                                          space, occupant, and payment schedules.
Client Billing                            Supports the development, documentation, and implementation of specific occupancy
                                          agreements with GSA clients. The data stored include client terms, space identification,
                                          and client-specific rates.
Project Administration                    Supports the tracking of actions taken to satisfy a federal agency’s space request. These
                                          actions include new lease acquisitions, consolidations, and constructing a new room or
                                          expanding space.
Billing                                   Generates monthly bills to client agencies on the basis of their space assignment;
                                          supports miscellaneous adjustments, such as debits or credits, to bills that have already
                                          been sent to the client. The data stored include rent to be billed and miscellaneous debits
                                          and credits.
Work Queue                                Supports an on-line to do list for realty services users and for interoffice communications.
                                          It has five types of work queues—approval, copy, corrections, action required, and
                                          notification.
Administration                            Manages administrative tasks related to STAR. Accessible only by contractor.
Reports                                   Supports standardized reports on space management, project administration, and billing.
                                          Also provides report writer for writing ad hoc reports.
                                              Source: GAO analysis of PBS documents.




                                              2
                                               Galaxy is a program that is the primary tool used to manage the rental-of-space account. Galaxy will
                                              receive source data from STAR. A proposal to enhance Galaxy is currently under review. A statement
                                              of work to accomplish this objective is currently being developed.




                                              Page 6             GAO/GGD-00-12 STAR—Program for Tracking and Managing Real Property
                          B-281769




                          Each of these modules has data entry screens. For example, the Space
                          Management module has a Building Details screen for entering specific
                          building information and a Lease Details screen for entering specific lease
                          information. STAR is supported by a contractor-operated Help Desk to aid
                          users if they have problems with the STAR modules and screens.

                          PBS identified the following benefits of STAR.

                        • Movement of data and information systems closer to the end user.
                        • Direct access to project, lease, building, and space assignment data.
                        • Electronic notification of expiring leases, unassigned space, and other
                          information.
                        • Integration of information technology into business practices.
                        • Direct input eliminates the need for data transcribers and precludes
                          duplicative keypunching.
                        • Easy report generation and exporting of data to another program.
                        • No need to memorize data element numbers, it uses English.
                        • Tutorials on policies and procedures can be placed into help screens of the
                          system.
                        • Will allow for direct update of agency information.
                        • Improves corporate competitiveness.

                          In the future, PBS hopes to expand STAR’s capabilities by adding more
                          features, such as direct access by an agency to view the information that
                          STAR maintains concerning the agency’s assigned space.

                          The 128 users we interviewed identified problems they had experienced
Users Identified          using STAR as well as several benefits they saw being derived from the use
Problems and Benefits               3
                          of STAR. Of the problems identified by users, 17 were mentioned by at
for STAR                  least 10 of the users. One problem was mentioned by only one user but
                          was included in this report because it involves accounting for rent
                                      4
                          revenues. According to the STAR project manager, each type of problem
                          that was reported to us had been reported to the STAR project team
                          through the STAR Help Desk and logged into 16 categories. According to
                          the project manager, the PBS project team has taken actions to (1) resolve
                          the reported problems that the team believed were legitimate and (2) add
                          the requested enhancements that the team believed were appropriate.

                          3
                           Of the approximately 46 problems identified by users, we combined 9 problems into 4 generic
                          problems because of their similarity. We then reported only those problems identified by 10 or more
                          users, except in 1 case. All four of the generic problems were included in our report.
                          4
                           One person identified the problem that STAR could not record the rent revenues from the National
                          Antenna Program.




                          Page 7             GAO/GGD-00-12 STAR—Program for Tracking and Managing Real Property
                                    B-281769




                                    Further, 116 of the users we interviewed saw at least 1 benefit from using
                                    STAR.

                                    Table 2 gives a breakdown of the problems that individuals we interviewed
                                    identified.

Table 2: Identified User Problems
                                                                                                                        Number of
                                                                                                                  users identifying
                                    Type of problem                                                                       problem
                                    Takes long time to enter data, too many steps/screens.a                                      99
                                    System slow at times, you get system errors.                                                 92
                                    Have to enter the same information repeatedly in different                                   87
                                    modules because modules are not linked.
                                    Printing requires multiple steps.                                                           73
                                    Not user friendly.                                                                          71
                                    Realty specialist can go into any lease in region, data                                     55
                                    security issues.
                                    STAR will not let you save work until certain information                                   55
                                    has been filled in.
                                    No history for R620 reports, authorizes payment by                                          39
                                    finance, only a digest. Realty specialists believed a history
                                    is needed in STAR.
                                    Cannot enter same date twice in the payment schedule                                        29
                                    screen. This results in the dummying of dates on the
                                    Payment Schedule screen.
                                    Hard-to-use report writer for ad hoc reports because you                                    27
                                    have to download data to another program module by
                                    module. Cannot access all fields in STAR.a
                                    STAR prompts are not clear.                                                                 27
                                    Work queues—You are getting too many that you do not                                        24
                                    need, getting work queues on items that are not the realty
                                    specialist’s responsibility. These waste the user’s time. a
                                    Bills—need to explain the manual adjustments on the bill.                                   22
                                    Need to calculate joint use space to the same number of
                                    decimal places for the bill as STAR does internally. Billing
                                    adjustments cumbersome to make.a
                                    Once a change has been approved, you cannot correct                                         21
                                    an error without initiating a database change through the
                                    STAR Help Desk.
                                    STAR lacks a Desk Reference Manual.                                                         20
                                    Realty specialist does not use STAR often enough to                                         19
                                    become proficient.
                                    STAR lacks a standard report called a combined space                                        13
                                    report.
                                    Cannot enter rent for National Antenna Program.b                                             1
                                    a
                                    We have combined nine identified problems into four generic problems.
                                    b
                                    This problem was included because it involves accounting for rent revenues.
                                    Source: Compiled by GAO from interviews with STAR users.




                                    Page 8            GAO/GGD-00-12 STAR—Program for Tracking and Managing Real Property
                                               B-281769




                                               The project manager said that the STAR project team had taken action or
                                               action was being taken on some of the identified problems. Other
                                               problems relate to inherent parts of STAR that cannot be easily fixed or
                                               occurred because of decisions made during the STAR developmental
                                               process. Further, according to another PBS official, user problems may
                                               also be a reaction to learning new business processes, such as New
                                               Pricing, and other changes.

                                               Table 3 contains PBS’ responses to examples of the problems we identified
                                               through interviews.


Table 3: PBS’ Responses to Examples of User Problems
Problem                                  PBS response to the problem
Printing requires multiple steps.        STAR has excellent data manipulation capabilities but the users are not used to
                                         downloading in order to print. Also, it would be very expensive to make STAR print directly
                                         from the screen.
STAR will not let you save work until    To ensure that information was entered in STAR, it was designed not to let you save work
certain information has been filled in.  until certain data entry blanks have been filled in.
Realty specialist can go into any lease  All realty specialists in a region can go into all leases, if they have been granted access to
in region, data security issues.         leases. It will be up to each region to decide if it wants to limit access to a greater extent. It
                                         can be done.
No history for R620 reports, authorizes  It was decided during development not to have a history of the R620 reports in STAR.
payment by finance, only a digest.       There is a digest of the reports on STAR and a hard copy of the individual R620s can be
Realty specialists believed a history    located in the lease file.
is needed in STAR.
Work queues—You are getting too          Work queues serve an important role in STAR by keeping users informed of problems and
many that you do not need, getting       other issues. The work queue process is being streamlined in a release scheduled for
work queues on items that are not        November 1999.
the realty specialist’s responsibility.
These waste the user’s time.
Once a change has been approved,         Because of the approval process in STAR for making changes, you have to go to the Help
you cannot correct an error without      Desk for a database change to correct an error once a change has been approved.
initiating a database change through
the STAR Help Desk.
                                               Source: Table 2 of this report and interview with the STAR project manager.


                                               The STAR developmental process also affected STAR, particularly in
                                               relation to the user friendliness problem. According to the project
                                               manager, the time frame for developing STAR covered from January 1997
                                               until October 1997. A different team developed each module in STAR, and
                                               communications among the teams may not have been effective because of
                                               the time constraints. The project manager thought that the lack of
                                               communications may have resulted in inconsistencies and duplication
                                               among modules. Further, she said that STAR’s development was hindered
                                               because during the development phase it was learned that STAR would
                                               have to handle both the old pricing policy for client space and the new



                                               Page 9             GAO/GGD-00-12 STAR—Program for Tracking and Managing Real Property
B-281769




pricing policy for client space, not just the new pricing policy as originally
planned. This resulted in having to change STAR’s design. Also, she
suggested that PBS may have released STAR to the regions before it was
ready, resulting in some of the problems that have been identified by users.

The project manager said that a cultural change occurred for realty
specialists that probably affected their view of STAR. Before STAR and
other business changes, realty specialists were not responsible for entering
and maintaining data using a computer or for billing client agencies for
their assigned space. With the introduction of STAR, New Pricing, New
Billing, and the Occupancy Agreement Tool, additional workloads were
                                5
placed on the realty specialist. Also, each initiative had its own
implementation problems that the realty specialists had to deal with in
doing their jobs.

The nearly 8,000 service tickets handled by the Help Desk reflect PBS-wide
problems and other issues reported by users. PBS had classified these
tickets into 16 different categories. According to PBS officials, the same
problem may have been reported on many different service tickets. Table 4
shows the Help Desk’s breakdown of these service tickets received and the
status of the tickets by the 16 categories, as of August 12, 1999. Definitions
of the categories can be found in appendix I.




5
 Through New Pricing, PBS has introduced a new way of calculating the rent that client agencies must
pay. Through New Billing, PBS has changed its billing process, such as billing on a monthly basis
instead of quarterly and making the reality specialists responsible for billing. The Occupancy
Agreement Tool is a new software package that is to be used to document agreements on such things
as square footage and cost between PBS and its client agencies.




Page 10            GAO/GGD-00-12 STAR—Program for Tracking and Managing Real Property
                                     B-281769




Table 4: Help Desk-Identified User
Problems, Ranked by the Number of                                                          Number of
Service Tickets Received for the                                                       service tickets
Problem                              Category of problem                                     received              Closed    Open
                                     Troublea                                                    3,540              3,496      44
                                     Database change                                             2,080              2,033      47
                                     Inquiry                                                       659                652       7
                                     Enhancement—application                                       590                429     161
                                     Rent bill                                                     364                359       5
                                     Trouble reports                                               189                178      11
                                     Enhancement—ad hoc reports                                     94                 50      44
                                     Inquiry reports                                                90                 89       1
                                     Enhancement—report standard                                    88                 61      27
                                     Security request                                               78                 78       0
                                     Table update                                                   59                 55       4
                                     Policy and procedures                                          40                 33       7
                                     Training                                                       28                 27       1
                                     Maintenance—nonscheduled                                       14                 14       0
                                     Conversion issues                                              11                 11       0
                                     Maintenance—scheduled                                           4                  3       1
                                     Total                                                       7,928              7,568     360
                                     a
                                     Originally, this category was used as a catchall until other categories were defined.
                                     Source: Help Desk contractor document.

                                     Trouble tickets refer to any part of the system that the user may be having
                                     problems with, except for the report function, including essential
                                     nonfunctioning software. The contractor is responsible for fixing the
                                     problems associated with all tickets categorized as trouble at no cost to
                                     PBS. The three enhancement categories contain requests to make changes
                                     to STAR. Any changes to STAR resulting from these tickets categorized as
                                     enhancements have to be negotiated between PBS and the contractor, with
                                     PBS’ paying for the change. The CRT leader said the team had processed
                                     511 enhancement tickets, of which about 44 were still open. The official
                                     said that this tally and the contractor’s tally of tickets categorized as
                                     enhancements, as shown in table 4, do not match because the two had not
                                     been recently reconciled. She said the contractor’s system probably had
                                     some items in the enhancement categories that were not enhancements
                                     and CRT had probably closed some other tickets that were not yet
                                     recorded in the contractor’s computer database as closed.

                                     While the 128 users we interviewed reported problems with STAR, 116 of
                                     them saw some positive benefit from STAR. Users identified the following
                                     as the top five benefits.




                                     Page 11            GAO/GGD-00-12 STAR—Program for Tracking and Managing Real Property
                      B-281769




                  •   Information is current and readily accessible.
                  •   Can see that changes to data have been accepted by the system.
                  •   STAR makes it easy to make minor changes.
                  •   STAR directly controls billing and lease payments.
                  •   It is good to have control over the data for which you are responsible.

                      As previously mentioned, as part of the STAR project, a contractor-
PBS’ Efforts to       operated Help Desk and PBS’ CRT were set up to solve STAR problems.
Improve STAR          The Help Desk and CRT have handled about 8,000 service tickets, of which
                      95 percent have been closed. Further, other actions, such as additional
                      contract help, are now available to the regions to help with STAR problems
                      and training.

                      The Help Desk is to record user complaints, enhancement suggestions, or
                      other issues concerning the use of STAR on service tickets. CRT reviews
                      and determines how to handle enhancement tickets and trouble tickets
                      that require a fix through a new release of STAR. Guidance on the review
                      process states that no action on a service ticket usually means that a ticket
                      resulted from a user’s momentary frustration with the system, a passing
                      thought of a user, or that other users had already identified the problem.

                      If the enhancement or trouble service ticket is accepted, CRT is to edit and
                      define the ticket to clearly identify the issue. CRT is also to assign each of
                      these tickets a priority—high, normal, or low. Working with the contractor,
                      CRT then decides how to implement any change considered needed. By
                      the end of November 1999, eight versions of STAR will have been issued.
                      According to PBS reports, STAR releases 3.0, and 4.0, combined, have
                      about 40 enhancements. Version 4.0, which is to be released by the end of
                      November 1999, for example, is to address the problems of (1) having to
                      enter the same information repeatedly in different modules because
                      modules are not linked and (2) not having the ability to enter the same
                      date twice in the Payment Schedule screen. A category named “zero square
                      foot leases” is also to be added to STAR to cover the national antenna
                                                                               6
                      program in a release scheduled for the spring of 2000. As of August 12,
                      1999, additional enhancement tickets are still open, although the exact
                      number was unclear because PBS had not reconciled the data between the
                      CRT and the computer-based Help Desk report. Other efforts to improve
                      STAR include the following:



                      6
                      Zero square foot leases will also cover other non-space revenue producing items, such as docks and
                      wareyards.




                      Page 12            GAO/GGD-00-12 STAR—Program for Tracking and Managing Real Property
                            B-281769




                        • A desk reference manual for STAR is to be issued in November 1999. This
                          manual should help with the problem of the need to have better prompts
                          that make where to go clearer. The users should be able to look up what to
                          do.
                        • A report is under development that would be similar to the combined
                          space report, which provided such information as building address, square
                          footage, and lease-specific information.
                        • Renewed emphasis has been placed on examining a preview of the bill in
                          STAR before actual bills are sent.
                        • An ad hoc exception report on bills is being sent to the regions each month
                          to catch errors more quickly.
                        • PBS has set a specific accuracy goal that targets a specific data problem
                          for improvement and rewards regions for meeting the improvement goal.
                          We were told that this data accuracy goal would be changed on a regular
                          basis. It currently focuses on the issue of chargebacks and missing rates.

                            PBS knows that it has data problems, and these problems have been well
PBS’ Efforts to Ensure      documented. Examples of the problems follow:
Accurate, Reliable, and
Consistent Data in      • The GSA IG concluded that over half of the space assignment drawings are
                          off by more than 5 percent.
STAR                    • Price Waterhouse Coopers, in its audit of GSA’s fiscal year 1998 financial
                          statements, found a reportable condition that improvements are needed in
                          the controls over the integrity of rent and leasing data used to manage the
                          Federal Buildings Fund.
                        • Exception reports and queries indicate that there is significant regional
                          variation in how well data in STAR are maintained, how well PBS produces
                          occupancy agreements, and the accuracy of rent billings. PBS had to do
                          extensive reconciliation of all congressional rent bills, the Federal Bureau
                          of Investigation had to tell PBS that it was not being billed for $3.0 million
                          worth of assignments, and 40 percent of the first wave of Census Bureau
                          offices that PBS had delivered had not been billed as of January 1999.

                            PBS said it has taken various actions to ensure that STAR has accurate,
                            reliable, and consistent data.

                        • STAR was designed with built-in features to help ensure the reliability of
                          some data elements.
                        • PBS ran edits against the data to be transferred to STAR and ran additional
                          edits after data were transferred.
                        • PBS has an initiative to validate all space assignments.




                            Page 13       GAO/GGD-00-12 STAR—Program for Tracking and Managing Real Property
  B-281769




• A special study was done that produced a report identifying data problems
                                                       7
  and made recommendations for solving the problems.
• A team was set up to determine how to implement the report’s
  recommendations.
• PBS is developing a quality control system.

  Controls have been built into STAR to make the program provide more
  accurate, reliable, and consistent data. For example, STAR has fields that
  have to be filled in before you can go to the next step of your task. This is
  to ensure that all of the required data for a task are entered. Another
  example is work queues, which automatically generate requests for such
  things as approvals of leases, corrections, or notifications of missing
  information and may also be used manually to send information. This
  feature is also designed to automatically notify the budget analyst
  responsible for rent expenditures of changes so that the analyst may
  anticipate the effect of the change on expenditures. STAR makes certain
  calculations using data entered in the database to reduce the chance of
  human error.

  During the transition to STAR, a PBS team was formed to work with each
  of the teams that was developing STAR modules to assist in data transfer.
  According to the team leader, this team developed diagnostic edits for
  each module that were run on a region-by-region basis to determine
  missing or incomplete data before the transfer of data to STAR. Further,
  AT&T wrote a conversion program that included a list of edits to run
  against the data once they were transferred to STAR to identify missing or
  incomplete data. Finally, the post-conversion STAR team was tasked with
  regularly identifying queries, exception reports, and flags that are to be
  added to STAR to help identify pockets of bad data.

  To further ensure accurate, reliable, and consistent data, PBS established a
  Go Team in January 1999 to identify and quantify the real and perceived
  problems with inventory data—space assignment drawings, lease files,
  assignment files, rent rates, bills, and information in STAR and Occupancy
  Agreements. For example, as previously mentioned, the GSA IG found half
  of the space assignment drawings were off by more than 5 percent. The Go
  Team found considerable problems with how well PBS was creating final
  rent rates—the proper calculation and application of fees and other
  charges. In a spot check of 20 Client Billing Records, the team found minor
  errors in 100 percent of the cases. Also, the team found errors in parking
  information. In one region, a discrepancy between total parking and
  7
      Data Issues in PBS.




  Page 14               GAO/GGD-00-12 STAR—Program for Tracking and Managing Real Property
    B-281769




    assigned parking, which should match in STAR, was found in about 5
    percent of the buildings. This problem could affect revenues. Further, the
    team found errors in design gross square feet and gross square feet, which
    should match in STAR. In three regions in particular, substantial variances
    between the design gross square feet and the gross square feet were found.
    Again, this could affect revenues.

    In the team’s April 1999 report, it identified the causes of known data
    problems and identified actions that needed to be taken to fix data
    problems. The Go Team report identified the following as the root causes
    of data problems:

• the lack of accountability and ownership of data;
• the lack of clear, consistent, complete policies associated with PBS’ new
  business practices;
• incomplete training;
• bad data to begin with;
• PBS’ reorganization that moved or de-emphasized assignment functions;
• migration from the old computer program to STAR;
• personnel turnover, downsizing, and workload; and
• changing standards, such as moving from occupiable square feet to usable
  square feet to rentable square feet.

    In addition to those actions already under way that were discussed under
    the previous section of this report, the Go Team identified issues that it
    said must be addressed to improve data quality. These issues were

•   management responsibilities,
•   accountability and ownership,
•   training and tools, and
•   reports and measures.

    Table 5 lists examples of the suggested changes made by the team.
    Appendix II lists all of the suggested changes.




    Page 15       GAO/GGD-00-12 STAR—Program for Tracking and Managing Real Property
                                               B-281769




Table 5: Examples of Go Team-Identified Issues and Recommendations
Type of issue                             Recommendations
Management’s Responsibilities             •Establish national policies, answer questions, and resolve disputes in a timely manner.
                                          •Change the Central Office culture of “Information is power so I won’t share it.”
                                          •Ensure that changed business practices and new tools are adopted at the working level.
Accountability and Ownership              •Job descriptions should include data integrity.
                                          •Competency testing and certification should be established for employees who need to
                                          understand New Pricing, Occupancy Agreements, and STAR.
                                          •Performance ratings should address the issue of employee proficiency in the use of tools
                                          and databases required in their specific job functions. Employees should be evaluated on
                                          the integrity of their data and the promptness and accuracy of their input.
Training and Tools                        •Need to make sure that everyone who uses STAR gets the right kind of training.
                                          •Cross training opportunities between business lines should be encouraged.
                                          •More changed business practices training should be scheduled.
Reports and Measures                      •Develop meaningful performance measures for data integrity. Any measure that is
                                          adopted should identify the magnitude of the data problems and encourage additional
                                          accountability and ownership.
                                          •Possible performance measures for data include the number of exceptions/flags
                                          generated for each region in STAR and the number and dollar value of chargebacks.
                                               Source: Data Issues in PBS, PBS, April 1999.

                                               In May 1999, another team was established to determine how to implement
                                               the first team’s recommendations. The second team developed a set of six
                                               prioritized action plans that include both temporary stop-gap fixes and
                                               permanent solutions. Table 6 lists the prioritized action plans, in priority
                                               order.


Table 6: PBS’ Prioritized Action Plans by Priority
Action plan                                 Purpose
1. Education and Involvement                Initially educate internally and externally for implementation of new billing in October 1999.
                                            Address long-term educational and training needs associated with data accuracy.
2. Making Decision Process More             Address the mechanics of decisionmaking, suggest some standards that should exist for
   Effective                                seeking decisions, and propose some solutions for publicizing decisions once made.
3. Ensuring Data Accuracy and               Establish standards for accuracy, and provide tools for people to use in dealing with PBS
   Appropriate Use                          data.
4. Business Process Integration and         Address approaches ranging from process improvementse.g., billing processto
   Simplification                           systems improvementse.g., software standardizationto correction of current data and
                                            prevention of future data corruption.
5. Motivational Issues With Data and        Suggest various rewards/consequences and approaches to motivate users and customers
   Systems                                  to clean up and maintain data in all PBS systems.
6. Overcoming and Capitalizing Upon         Identify regional variances, and decide which variances are problems and which are not.
   Regional Variances
                                               Source: PBS Go Team implementation paper.




                                               Page 16           GAO/GGD-00-12 STAR—Program for Tracking and Managing Real Property
                      B-281769




                      The team developed the education and involvement action plan to a higher
                      degree of detail than the other five plans due to the extreme time
                      constraints for delivering results on these issues. The PBS Commissioner
                      advised us that under this action plan, as of September 23, 1999, an intense
                      training effort, called Educate 2000, was completed in each region and the
                      central office on the effects of full implementation of New Pricing in
                      October 1999. Also, according to the Commissioner, regions began holding
                      customer workshops in late September to inform them of the changes in
                      their STAR bills and provide updates on Occupancy Agreements. Further,
                      he said that a Customer Advisory Board has been established, and its first
                      meeting is scheduled for October 29, 1999.

                      For action plans two, five, and six, team leaders have been appointed.
                      Action plans three and four are being run by the Office of the Chief
                      Information Officer and were delayed while a new Chief Information
                      Officer was appointed. A new Chief Information Officer was appointed in
                      July 1999.

                      PBS is also in the final stages of hiring a contractor to develop and
                      document a quality control system that validates real estate and billing
                      data residing in STAR. The contractor will provide on-site support to
                      regions in such areas as developing diagnostic reports. PBS expects to
                      award the contract in early fiscal year 2000.

                      On September 23, 1999, the PBS Commissioner provided written
Agency Comments and   comments on a draft of this report. He stated that the report is largely
Our Evaluation        factual and balanced and does a good job of capturing the magnitude of
                      the effort undertaken to move GSA and PBS from 1970s batch process
                      computing to a modern relational database platform. In the comments, the
                      Commissioner also briefly summarized (1) PBS’ transition from its 1970s
                      system to STAR; (2) the cultural changes that drove the need to provide its
                      customers with more reliable information and to measure its own
                      performance in hard, quantifiable terms as envisioned by the Government
                      Performance and Results Act; and (3) actions being taken to improve the
                      data in STAR and elsewhere, including a spatial validation effort using
                      computer-assisted drawings and the development of a quality control
                      system to validate real estate and billing data residing in STAR. He also
                      provided technical clarifications that have been included in this report
                      where appropriate. The Commissioner’s letter is reproduced in appendix
                      III.




                      Page 17       GAO/GGD-00-12 STAR—Program for Tracking and Managing Real Property
B-281769




We are sending copies of this report to Senator Ben Nighthorse Campbell,
Chairman, and Senator Byron L. Dorgan, Ranking Minority Member,
Subcommittee on Treasury and General Government, Senate Committee
on Appropriations; Senator George V. Voinovich, Chairman, and Senator
Max S. Baucus, Ranking Minority Member, Subcommittee on
Transportation and Infrastructure, Senate Committee on Environment and
Public Works; Representative Jim Kolbe, Chairman, and Representative
Steny H. Hoyer, Ranking Minority Member, Subcommittee on Treasury,
Postal Service, and General Government, House Committee on
Appropriations; the Honorable David J. Barram, Administrator, GSA; and
to others upon request.

If you have any questions regarding this report, please call me or Ron King
at (202) 512-8387. Key contributors to this assignment were Tom Keightley
and Mike Yacura.




Bernard L. Ungar
Director, Government Business
  Operations Issues




Page 18       GAO/GGD-00-12 STAR—Program for Tracking and Managing Real Property
Page 19   GAO/GGD-00-12 STAR—Program for Tracking and Managing Real Property
Contents



Letter                                                                                                1


Appendix I                                                                                           22
                         Conversion Issues                                                           22
Definitions of STAR      Database Change                                                             22
Help Desk Ticket         Enhancement—Application                                                     22
                         Enhancement— Ad Hoc Report                                                  22
Types                    Enhancement—Report Standard                                                 22
                         Inquiry                                                                     22
                         Inquiry Reports                                                             22
                         Maintenance—Nonscheduled                                                    22
                         Maintenance—Scheduled                                                       22
                         Policy and Procedures                                                       23
                         Rent Bill                                                                   23
                         Security Request                                                            23
                         Table Update                                                                23
                         Training                                                                    23
                         Trouble                                                                     23
                         Trouble Reports                                                             23


Appendix II                                                                                          24

Data Issues Identified
by PBS’ Go Team and
Its Recommendations
Appendix III                                                                                         26

Comments From the
General Services
Administration
Tables                   Table 1: Description of STAR’s Seven Major Modules                           6
                         Table 2: Identified User Problems                                            8
                         Table 3: PBS’ Responses to Examples of User Problems                         9
                         Table 4: Help Desk-Identified User Problems, Ranked by                      11
                           the Number of Service Tickets Received for the
                           Problem
                         Table 5: Examples of Go Team-Identified Issues and                          16
                           Recommendations



                         Page 20      GAO/GGD-00-12 STAR—Program for Tracking and Managing Real Property
Contents




Table 6: PBS’ Prioritized Action Plans by Priority                           16




Abbreviations

CRT            Change Review Team
GSA            General Services Administration
IG             Office of Inspector General
PBS            Public Buildings Service
PROMPT         Permanent Record of Managed Property Transactions
STAR           System for Tracking and Administering Real Property




Page 21       GAO/GGD-00-12 STAR—Program for Tracking and Managing Real Property
Appendix I

Definitions of STAR Help Desk Ticket Types


                     Problems or situations pertaining to data that have been converted from
Conversion Issues    the old system.



                     Requests to have data changed within the database, requires written
Database Change      approval.



                     Requests to change the software.
Enhancement—
Application


                     Requests to change an existing ad hoc report or add a new ad hoc report.
Enhancement—
Ad Hoc Report


                     Requests to change an existing standard report or add a new standard
Enhancement—Report   report.
Standard

                     Any question that is being asked to gather information.
Inquiry

                     Calls specifically related to questions about reports.
Inquiry Reports

                     Requests for maintenance for unexpected problems.
Maintenance—
Nonscheduled
                     Issues regarding the monthly maintenance and operation schedule.
Maintenance—
Scheduled


                     Page 22        GAO/GGD-00-12 STAR—Program for Tracking and Managing Real Property
                        Appendix I
                        Definitions of STAR Help Desk Ticket Types




                        Questions concerning policies and procedures.
Policy and Procedures

                        Requests regarding a rent bill or the rent billing process.
Rent Bill

                        Requests on any security issue coming through the Help Desk.
Security Request

                        Request to change information in a table that supports the System for
Table Update            Tracking and Administering Real Property (STAR).



                        Requests that can be solved or partially solved by additional training.
Training

                        Calls about any part of the system the user may be having problems with,
Trouble                 except reports, including bugs and essential nonfunctioning software.



                        Calls about any problems the user may be having regarding reports.
Trouble Reports




                        Page 23         GAO/GGD-00-12 STAR—Program for Tracking and Managing Real Property
Appendix II

Data Issues Identified by PBS’ Go Team and
Its Recommendations

Type of issue                   Recommendations
Management’s Responsibilities   •Establish national policies, answer questions, and resolve disputes in a timely manner.
                                •Change the Central Office culture of “Information is power so I won’t share it.”
                                •Ensure that changed business practices and new tools are adopted at the working level.
                                •Prepare employees for the business practices and system changes and emphasize the
                                importance of needed training.
                                •Take a proactive position regarding data integrity and commit the resources needed to
                                get the job done.
                                •Resolve issues quickly, so employees do not disengage completely from the issue,
                                making it even worse.
                                •Instill in all employees a big picture perspective to avoid the “That’s not my job attitude.”
                                •Become familiar with the systems the employees must use on a daily basis in order to
                                appreciate the effort needed to learn the system.
                                •Address, buy-into, and communicate a policy on data input by contractors.
                                •Close the gaps that exist between the organizational vision of upper management and
                                the changes that need to take place in business processes and roles.
                                •Decide and publish Central Office roles and organizational structure on who ultimately
                                decides what. Establish accountability for business decisions.
                                •Managers must see the advantages of acting proactively—this must become a core
                                value in PBS.
                                •Establish a management information steering committee that will establish policy, fiscal
                                controls and resolve conflicts.
                                •Start having regular meetings to go over project data.
Accountability and Ownership    •Job descriptions should include data integrity.
                                •Competency testing and certification should be established for employees who need to
                                understand New Pricing, Occupancy Agreements, and STAR.
                                •Performance ratings should address the issue of employee proficiency in the use of tools
                                and databases required in their specific job functions. Employees should be evaluated on
                                the integrity of their data and the promptness and accuracy of their input.
                                •Deal with the perception that STAR and Computer-aided Design are wonderful systems
                                that absolve us of our individual responsibility to review, track, and know our data.
                                •If individual specialists are to be responsible for their data, then any data manipulation
                                that takes place system-wide must be coordinated and communicated.
                                •Stop continually shifting employees from one area to another.
                                •Decide how much access to propriety data our contractors can/should have.
Training and Tools              •Need to make sure that everyone who uses STAR gets the right kind of training.
                                •Cross training opportunities between business lines should be encouraged.
                                •More changed business practices training should be scheduled.
                                •Develop systems that can get information to the people who need it.
                                •Set up recurring training/certification programs.
                                •Expand STAR training beyond realty specialists.
                                •Conduct more billing training, emphasizing how and when to do billing adjustments.
                                •Plan just-in-time training and train internally before taking it to the customers.
                                •Regional business lines need to identify their areas for immediate training.
                                •Training for affected personnel must be mandatory and enforced by directors,
                                supervisors, and team leaders.
                                •Look at integrated training of multiple tools—teach STAR, Occupancy Agreements, and
                                the client billing policy as an integrated training session.
                                •Regions need to take advantage of the available training resources recently procured by
                                the STAR national training team.




                                Page 24           GAO/GGD-00-12 STAR—Program for Tracking and Managing Real Property
                                   Appendix II
                                   Data Issues Identified by PBS’ Go Team and Its Recommendations




Type of issue                      Recommendations
Training and Tools (Cont.)         •Regional “Power Users” should be utilized in training and instructing other regional users
                                   in some type of regular scheduled sessions to keep users up to date on changes and
                                   enhancements to the databases and internal processes.
                                   •Identify any strong regional training programs as best practices, and have the rest of
                                   PBS learn from them.

 Medium/long term actions            •Some type of evaluation or testing of users needs to be developed to ensure that they
                                   understand the database(s) and process required for them.
                                     •A training infrastructure should be developed in each region that provides for regularly
                                   scheduled sessions on all of the various databases utilized by GSA. Training sessions
                                   should be interactive.
 Customer training and marketing
                                     •Develop national, regional, and local dialogs/workshops with our customers so they
                                   fully understand new pricing, billing, and the changes coming to existing assignments
                                   before they arrive October 1, 1999.
Reports and Measures               •Develop meaningful performance measures for data integrity. Any measure that is
                                   adopted should identify the magnitude of the data problems and encourage additional
                                   accountability and ownership. Possible performance measures for data include the
                                   following:
                                     •Number of exceptions/flags generated for each region in STAR.
                                     •Number and dollar value of chargebacks.
                                     •Number of unopened work queues over some number of days old in STAR.
                                     •Develop a workload measure that would help management determine when to deploy
                                   additional resources, if possible.
                                     •Number of billing adjustments.
                                     •Number of customer complaints with response times for resolution.




                                   Page 25          GAO/GGD-00-12 STAR—Program for Tracking and Managing Real Property
Appendix III

Comments From the General Services
Administration




               Page 26   GAO/GGD-00-12 STAR—Program for Tracking and Managing Real Property
Appendix III
Comments From the General Services Administration




Page 27         GAO/GGD-00-12 STAR—Program for Tracking and Managing Real Property
Ordering Information

The first copy of each GAO report and testimony is free.
Additional copies are $2 each. Orders should be sent to the
following address, accompanied by a check or money order made
out to the Superintendent of Documents, when necessary. VISA
and MasterCard credit cards are accepted, also. Orders for 100 or
more copies to be mailed to a single address are discounted 25
percent.

Order by mail:

U.S. General Accounting Office
P.O. Box 37050
Washington, DC 20013

or visit:

Room 1100
700 4th St. NW (corner of 4th and G Sts. NW)
U.S. General Accounting Office
Washington, DC

Orders may also be placed by calling (202) 512-6000 or by using
fax number (202) 512-6061, or TDD (202) 512-2537.

Each day, GAO issues a list of newly available reports and
testimony. To receive facsimile copies of the daily list or any list
from the past 30 days, please call (202) 512-6000 using a touch-
tone phone. A recorded menu will provide information on how to
obtain these lists.

For information on how to access GAO reports on the INTERNET,
send e-mail message with “info” in the body to:

info@www.gao.gov

or visit GAO’s World Wide Web Home Page at:

http://www.gao.gov
United States                       Bulk Rate
General Accounting Office      Postage & Fees Paid
Washington, D.C. 20548-0001           GAO
                                Permit No. G100
Official Business
Penalty for Private Use $300

Address Correction Requested




(240329)