United States General Accounting Office GAO Report to Congressional Requesters October 1999 GENERAL SERVICES ADMINISTRATION STAR—PBS’ New Program for Tracking and Managing Real Property GAO/GGD-00-12 United States General Accounting Office General Government Division Washington, D.C. 20548 B-281769 October 12, 1999 The Honorable Bob Franks Chairman The Honorable Robert Wise, Jr. Ranking Democratic Member Subcommittee on Economic Development, Public Buildings, Hazardous Materials, and Pipeline Transportation Committee on Transportation and Infrastructure House of Representatives This report responds to your request for a review of the General Services Administration’s (GSA) Public Buildings Service’s (PBS) System for Tracking and Administering Real Property (STAR). Your Subcommittee was concerned about problems that were being reported to it concerning STAR. As agreed with the Subcommittee, our objectives were to determine • the functions STAR performs; • whether users were having problems using STAR; • the steps PBS had taken or is taking to address any user problems; and • the actions PBS had taken or plans to take to ensure that STAR data are accurate, reliable, and consistent. As agreed with your offices, we did not review the technical aspects of STAR because GSA’s Office of Inspector General (IG) is reviewing the technical and systemic issues relating to STAR. The IG expects to issue its report by the end of 1999. STAR is a real estate inventory management software application that Results in Brief maintains data on projects, leases, buildings, and space assignmentsthat is, who is in what space—in an integrated database environment. The data in STAR are used to generate bills to clients for their assigned space, develop budget plans, track and manage leases, and evaluate performance. STAR’s input/output functions include • direct on-screen access to the data, • the ability to update the data, • access to standard reports, and • the ability to sort data and create ad hoc reports. Page 1 GAO/GGD-00-12 STAR—Program for Tracking and Managing Real Property B-281769 Users identified problems with STAR as well as benefits derived from using the system. The 128 users we interviewed identified 18 types of problems that they had experienced using STAR. These problems included complaints such as taking too long to enter data and having too many steps or screens. According to the STAR project manager, all of the problems that were reported to us by users had been reported to the STAR project team through the contractor-operated STAR Help Desk. Also, STAR users had submitted nearly 8,000 requests, which were recorded on service tickets, to the Help Desk between October 1997 and August 1999. These tickets, many of them concerning the same issue, identified problems that users reported having when working with STAR as well as STAR enhancement suggestions and other issues related to STAR’s operations. Further, 116 of the 128 users we interviewed saw at least 1 benefit from using STAR, such as data being current and readily available. The STAR Help Desk and a PBS Change Review Team (CRT) were set up to solve problems identified by STAR users and to resolve other issues with STAR. By the end of November 1999, changes and enhancements will have resulted in eight versions of STAR. Further, additional contract help had been provided to assist the regions with STAR problems and training. PBS has taken various actions to try to ensure that STAR has accurate, reliable, and consistent data. • STAR has been designed with built-in features to help ensure the reliability of some data elements. For example, work queues were to automatically generate requests for such things as approvals of leases, corrections, or notifications of missing information; work queues could also be used manually to send information. • PBS ran edits against data before they were transferred to STAR and ran additional edits after data were transferred. • PBS initiated an effort to validate all space assignments. • PBS conducted a special study and produced a report that identified data 1 problems and made recommendations for solving the problems. • A team was set up to determine how to implement the report’s recommendations. PBS has started implementing some of these recommendations. • PBS is developing a quality control system. 1 Data Issues in PBS, PBS, April 1999. Page 2 GAO/GGD-00-12 STAR—Program for Tracking and Managing Real Property B-281769 With the introduction of STAR and other changes, a major cultural change Background has occurred in GSA’s facilities management. According to the PBS report, “The simultaneous launches of New Pricing, STAR, Occupancy Agreements, and a host of other re-engineered business line processes and software constitutes the most significant change to the way PBS does its business since the creation of the Federal Buildings Fund.” The mission of PBS is to effectively manage over 8,000 owned and leased federal real properties in the United States. PBS had previously used a computer program called PLUS+ to track and manage its real properties. In 1994, PBS’ Central Office decided that the systems it used to support its mission were antiquated and needed to be replaced. New software was needed to support real property inventory, leasing, space assignments, and billing-related activities. PBS did market research analyses from March 1994 through October 1995, evaluating various real estate software packages. In March 1994, PBS began its initial efforts to obtain new software. It published a notice in the Commerce Business Daily seeking sources to supply software products to support management operations for leased and owned public buildings. Two responses to this notice were received. However, PBS determined that neither of these sources met its functional needs. PBS also hired a contractor to identify any existing property management software that might meet its needs. The contractor developed a list of 400 software products that it said spanned all facets of PBS’ business functions. Using PBS’ needs as criteria, this list was reduced to 36 products, then to 6 products for further study. However, PBS concluded that none of these products was capable of satisfying all of its minimum requirements, such as the ability and capacity to concurrently manage over 8,000 owned and leased properties. During discussions with real estate companies and Fortune 500 companies about real estate practices and procedures, PBS discovered a proprietary system called Permanent Record of Managed Property Transactions (PROMPT). This system had been implemented by AT&T during the summer of 1994. According to PBS officials, PBS ran a 4-month pilot test in 1995 on PROMPT. During this test, 30 percent of PBS’ data were diagrammed and selected data elements on leases and space assignments, as well as other areas, were entered into the system. The test was completed in October 1995. PBS concluded that although PROMPT could meet its needs, the data were not in the format that it wanted. Page 3 GAO/GGD-00-12 STAR—Program for Tracking and Managing Real Property B-281769 In July 1996, an acquisition plan for STAR was approved by GSA’s Associate Administrator for Acquisition Policy. This plan called for a sole source contract with AT&T for the use of PROMPT, which PBS renamed STAR. On January 17, 1997, the contract, which included a license agreement granting use of PROMPT in perpetuity unless canceled by the licensee, was signed with AT&T. The contract was for labor hours on a best-effort performance basis to convert PROMPT version 2.7 into STAR, which met PBS’ requirements. The base contract was for $8.3 million. Of this amount, $4.3 million was for the perpetual right to use the national license; the remaining $4.0 million was for software maintenance, a STAR reference guide, training courses and manuals, technical support services, hotline support, and travel. The contract also included several options. If all of these options were exercised, the total cost of the contract would have been $17.0 million. On October 19, 1998, the quantities in one of the options—option four—were increased adding 1,000 users, 310 training classes, 2,900 sets of training manuals, and 23,200 staff hours of technical support. This raised the value of the contract to $21.8 million, which was an increase of $4.8 million. PBS began rolling out STAR in October 1997, and it was available in all regions by January 1998. The first rent bills generated by STAR were sent to agencies in July 1998. PBS officials said that the introduction of STAR, among other things, has led to a cultural change that has had many effects in PBS. The officials noted that three changes particularly, affected the realty specialists who maintain STAR. First, realty specialists are now required to enter the data directly into the STAR system; second, they are to be held directly responsible for data accuracy; and third, they are responsible for billing the client agencies. In the past, the realty specialists filled out data entry sheets and data entry clerks entered the data. It was not clear at that time who was responsible for the data accuracy. Further, in the past, realty specialists were not involved in the actual billing of the clients. To determine what functions STAR performs, we observed how STAR Scope and operates; interviewed officials responsible for the development of STAR; Methodology and reviewed contract documents, training manuals, and the draft desk reference manual for STAR. To identify problems that users were experiencing using STAR, we judgmentally selected four regions, two of which were identified by PBS officials as having problems implementing STAR, one that was the lead region for STAR, and one from which congressional staff had received Page 4 GAO/GGD-00-12 STAR—Program for Tracking and Managing Real Property B-281769 complaints. We also included the headquarters unit to get its perspective on STAR. PBS officials identified the users of STAR at these locations, of which realty specialists and realty assistants were identified as the major users. We then interviewed 128 users, 81 of which were realty specialists or realty assistants, in the 5 locations and asked them about the problems they have had using STAR. Our results cannot be projected to the total universe of STAR users. Further, we reviewed Help Desk reports that organized user complaints about STAR into subject categories. Because of the way that the issues were categorized in the report and the actual number of items reported to the Help Desk—almost 8,000—we were not able to directly compare our interview-identified problems with those reported to the Help Desk. However, we discussed the problems that users had identified in our interviews, and problems that had been reported to the Help Desk, with the STAR project manager. Further, we reviewed some Help Desk reports that showed specific problems identified. To identify the steps that PBS has taken or is taking to deal with user problems, we interviewed officials who were responsible for addressing STAR problems and reviewed documents on changes being made to correct problems and improve STAR. Finally, to determine what PBS has done or plans to do to ensure STAR data integrity, we interviewed officials and reviewed plans addressing PBS’ efforts to ensure that STAR data are accurate, reliable, and consistent. We did not independently assess the accuracy, reliability, and consistency of STAR data. We did our work at GSA/PBS Headquarters and the National Capital Region in Washington, D.C.; the Southeast Sunbelt Region in Atlanta, GA; the New England Region in Boston, MA; and the Mid-Atlantic Region in Philadelphia, PA. Our work was done in accordance with generally accepted government auditing standards. We requested comments on a draft of this report from the Administrator of GSA. GSA’s written comments are discussed at the end of this letter. STAR—A System for STAR is a real estate inventory management software application that maintains data on projects, leases, buildings, and space assignmentsthat Tracking and is, who is in what space—in an integrated database environment. The data Administering Real in STAR have been used to generate bills to clients for their assigned Property Page 5 GAO/GGD-00-12 STAR—Program for Tracking and Managing Real Property B-281769 space, develop budget plans, track and manage leases, and evaluate performance. STAR’s input/output functions include • direct on-screen access to the data, • the ability to update the data, • access to standard reports, and • the ability to sort data and create ad hoc reports. To develop the fiscal year 2001 budget, STAR data are being used to support the development of the over $5.5 billion rental revenue estimate and as the basis for the Galaxy system’s roughly $3.1 billion estimate of 2 PBS lease expenditures. STAR has seven major modules: (1) Space Management, (2) Client Billing, (3) Project Administration, (4) Billing, (5) Work Queue, (6) Administration, and (7) Reports. Table 1 describes each of these modules. Table 1: Description of STAR’s Seven Major Modules Module Description Space Management Supports the recording and processing of data related to buildings, space, occupants, and occupancy. Data stored include building location, square footage, lease terms, types of space, occupant, and payment schedules. Client Billing Supports the development, documentation, and implementation of specific occupancy agreements with GSA clients. The data stored include client terms, space identification, and client-specific rates. Project Administration Supports the tracking of actions taken to satisfy a federal agency’s space request. These actions include new lease acquisitions, consolidations, and constructing a new room or expanding space. Billing Generates monthly bills to client agencies on the basis of their space assignment; supports miscellaneous adjustments, such as debits or credits, to bills that have already been sent to the client. The data stored include rent to be billed and miscellaneous debits and credits. Work Queue Supports an on-line to do list for realty services users and for interoffice communications. It has five types of work queues—approval, copy, corrections, action required, and notification. Administration Manages administrative tasks related to STAR. Accessible only by contractor. Reports Supports standardized reports on space management, project administration, and billing. Also provides report writer for writing ad hoc reports. Source: GAO analysis of PBS documents. 2 Galaxy is a program that is the primary tool used to manage the rental-of-space account. Galaxy will receive source data from STAR. A proposal to enhance Galaxy is currently under review. A statement of work to accomplish this objective is currently being developed. Page 6 GAO/GGD-00-12 STAR—Program for Tracking and Managing Real Property B-281769 Each of these modules has data entry screens. For example, the Space Management module has a Building Details screen for entering specific building information and a Lease Details screen for entering specific lease information. STAR is supported by a contractor-operated Help Desk to aid users if they have problems with the STAR modules and screens. PBS identified the following benefits of STAR. • Movement of data and information systems closer to the end user. • Direct access to project, lease, building, and space assignment data. • Electronic notification of expiring leases, unassigned space, and other information. • Integration of information technology into business practices. • Direct input eliminates the need for data transcribers and precludes duplicative keypunching. • Easy report generation and exporting of data to another program. • No need to memorize data element numbers, it uses English. • Tutorials on policies and procedures can be placed into help screens of the system. • Will allow for direct update of agency information. • Improves corporate competitiveness. In the future, PBS hopes to expand STAR’s capabilities by adding more features, such as direct access by an agency to view the information that STAR maintains concerning the agency’s assigned space. The 128 users we interviewed identified problems they had experienced Users Identified using STAR as well as several benefits they saw being derived from the use Problems and Benefits 3 of STAR. Of the problems identified by users, 17 were mentioned by at for STAR least 10 of the users. One problem was mentioned by only one user but was included in this report because it involves accounting for rent 4 revenues. According to the STAR project manager, each type of problem that was reported to us had been reported to the STAR project team through the STAR Help Desk and logged into 16 categories. According to the project manager, the PBS project team has taken actions to (1) resolve the reported problems that the team believed were legitimate and (2) add the requested enhancements that the team believed were appropriate. 3 Of the approximately 46 problems identified by users, we combined 9 problems into 4 generic problems because of their similarity. We then reported only those problems identified by 10 or more users, except in 1 case. All four of the generic problems were included in our report. 4 One person identified the problem that STAR could not record the rent revenues from the National Antenna Program. Page 7 GAO/GGD-00-12 STAR—Program for Tracking and Managing Real Property B-281769 Further, 116 of the users we interviewed saw at least 1 benefit from using STAR. Table 2 gives a breakdown of the problems that individuals we interviewed identified. Table 2: Identified User Problems Number of users identifying Type of problem problem Takes long time to enter data, too many steps/screens.a 99 System slow at times, you get system errors. 92 Have to enter the same information repeatedly in different 87 modules because modules are not linked. Printing requires multiple steps. 73 Not user friendly. 71 Realty specialist can go into any lease in region, data 55 security issues. STAR will not let you save work until certain information 55 has been filled in. No history for R620 reports, authorizes payment by 39 finance, only a digest. Realty specialists believed a history is needed in STAR. Cannot enter same date twice in the payment schedule 29 screen. This results in the dummying of dates on the Payment Schedule screen. Hard-to-use report writer for ad hoc reports because you 27 have to download data to another program module by module. Cannot access all fields in STAR.a STAR prompts are not clear. 27 Work queues—You are getting too many that you do not 24 need, getting work queues on items that are not the realty specialist’s responsibility. These waste the user’s time. a Bills—need to explain the manual adjustments on the bill. 22 Need to calculate joint use space to the same number of decimal places for the bill as STAR does internally. Billing adjustments cumbersome to make.a Once a change has been approved, you cannot correct 21 an error without initiating a database change through the STAR Help Desk. STAR lacks a Desk Reference Manual. 20 Realty specialist does not use STAR often enough to 19 become proficient. STAR lacks a standard report called a combined space 13 report. Cannot enter rent for National Antenna Program.b 1 a We have combined nine identified problems into four generic problems. b This problem was included because it involves accounting for rent revenues. Source: Compiled by GAO from interviews with STAR users. Page 8 GAO/GGD-00-12 STAR—Program for Tracking and Managing Real Property B-281769 The project manager said that the STAR project team had taken action or action was being taken on some of the identified problems. Other problems relate to inherent parts of STAR that cannot be easily fixed or occurred because of decisions made during the STAR developmental process. Further, according to another PBS official, user problems may also be a reaction to learning new business processes, such as New Pricing, and other changes. Table 3 contains PBS’ responses to examples of the problems we identified through interviews. Table 3: PBS’ Responses to Examples of User Problems Problem PBS response to the problem Printing requires multiple steps. STAR has excellent data manipulation capabilities but the users are not used to downloading in order to print. Also, it would be very expensive to make STAR print directly from the screen. STAR will not let you save work until To ensure that information was entered in STAR, it was designed not to let you save work certain information has been filled in. until certain data entry blanks have been filled in. Realty specialist can go into any lease All realty specialists in a region can go into all leases, if they have been granted access to in region, data security issues. leases. It will be up to each region to decide if it wants to limit access to a greater extent. It can be done. No history for R620 reports, authorizes It was decided during development not to have a history of the R620 reports in STAR. payment by finance, only a digest. There is a digest of the reports on STAR and a hard copy of the individual R620s can be Realty specialists believed a history located in the lease file. is needed in STAR. Work queues—You are getting too Work queues serve an important role in STAR by keeping users informed of problems and many that you do not need, getting other issues. The work queue process is being streamlined in a release scheduled for work queues on items that are not November 1999. the realty specialist’s responsibility. These waste the user’s time. Once a change has been approved, Because of the approval process in STAR for making changes, you have to go to the Help you cannot correct an error without Desk for a database change to correct an error once a change has been approved. initiating a database change through the STAR Help Desk. Source: Table 2 of this report and interview with the STAR project manager. The STAR developmental process also affected STAR, particularly in relation to the user friendliness problem. According to the project manager, the time frame for developing STAR covered from January 1997 until October 1997. A different team developed each module in STAR, and communications among the teams may not have been effective because of the time constraints. The project manager thought that the lack of communications may have resulted in inconsistencies and duplication among modules. Further, she said that STAR’s development was hindered because during the development phase it was learned that STAR would have to handle both the old pricing policy for client space and the new Page 9 GAO/GGD-00-12 STAR—Program for Tracking and Managing Real Property B-281769 pricing policy for client space, not just the new pricing policy as originally planned. This resulted in having to change STAR’s design. Also, she suggested that PBS may have released STAR to the regions before it was ready, resulting in some of the problems that have been identified by users. The project manager said that a cultural change occurred for realty specialists that probably affected their view of STAR. Before STAR and other business changes, realty specialists were not responsible for entering and maintaining data using a computer or for billing client agencies for their assigned space. With the introduction of STAR, New Pricing, New Billing, and the Occupancy Agreement Tool, additional workloads were 5 placed on the realty specialist. Also, each initiative had its own implementation problems that the realty specialists had to deal with in doing their jobs. The nearly 8,000 service tickets handled by the Help Desk reflect PBS-wide problems and other issues reported by users. PBS had classified these tickets into 16 different categories. According to PBS officials, the same problem may have been reported on many different service tickets. Table 4 shows the Help Desk’s breakdown of these service tickets received and the status of the tickets by the 16 categories, as of August 12, 1999. Definitions of the categories can be found in appendix I. 5 Through New Pricing, PBS has introduced a new way of calculating the rent that client agencies must pay. Through New Billing, PBS has changed its billing process, such as billing on a monthly basis instead of quarterly and making the reality specialists responsible for billing. The Occupancy Agreement Tool is a new software package that is to be used to document agreements on such things as square footage and cost between PBS and its client agencies. Page 10 GAO/GGD-00-12 STAR—Program for Tracking and Managing Real Property B-281769 Table 4: Help Desk-Identified User Problems, Ranked by the Number of Number of Service Tickets Received for the service tickets Problem Category of problem received Closed Open Troublea 3,540 3,496 44 Database change 2,080 2,033 47 Inquiry 659 652 7 Enhancement—application 590 429 161 Rent bill 364 359 5 Trouble reports 189 178 11 Enhancement—ad hoc reports 94 50 44 Inquiry reports 90 89 1 Enhancement—report standard 88 61 27 Security request 78 78 0 Table update 59 55 4 Policy and procedures 40 33 7 Training 28 27 1 Maintenance—nonscheduled 14 14 0 Conversion issues 11 11 0 Maintenance—scheduled 4 3 1 Total 7,928 7,568 360 a Originally, this category was used as a catchall until other categories were defined. Source: Help Desk contractor document. Trouble tickets refer to any part of the system that the user may be having problems with, except for the report function, including essential nonfunctioning software. The contractor is responsible for fixing the problems associated with all tickets categorized as trouble at no cost to PBS. The three enhancement categories contain requests to make changes to STAR. Any changes to STAR resulting from these tickets categorized as enhancements have to be negotiated between PBS and the contractor, with PBS’ paying for the change. The CRT leader said the team had processed 511 enhancement tickets, of which about 44 were still open. The official said that this tally and the contractor’s tally of tickets categorized as enhancements, as shown in table 4, do not match because the two had not been recently reconciled. She said the contractor’s system probably had some items in the enhancement categories that were not enhancements and CRT had probably closed some other tickets that were not yet recorded in the contractor’s computer database as closed. While the 128 users we interviewed reported problems with STAR, 116 of them saw some positive benefit from STAR. Users identified the following as the top five benefits. Page 11 GAO/GGD-00-12 STAR—Program for Tracking and Managing Real Property B-281769 • Information is current and readily accessible. • Can see that changes to data have been accepted by the system. • STAR makes it easy to make minor changes. • STAR directly controls billing and lease payments. • It is good to have control over the data for which you are responsible. As previously mentioned, as part of the STAR project, a contractor- PBS’ Efforts to operated Help Desk and PBS’ CRT were set up to solve STAR problems. Improve STAR The Help Desk and CRT have handled about 8,000 service tickets, of which 95 percent have been closed. Further, other actions, such as additional contract help, are now available to the regions to help with STAR problems and training. The Help Desk is to record user complaints, enhancement suggestions, or other issues concerning the use of STAR on service tickets. CRT reviews and determines how to handle enhancement tickets and trouble tickets that require a fix through a new release of STAR. Guidance on the review process states that no action on a service ticket usually means that a ticket resulted from a user’s momentary frustration with the system, a passing thought of a user, or that other users had already identified the problem. If the enhancement or trouble service ticket is accepted, CRT is to edit and define the ticket to clearly identify the issue. CRT is also to assign each of these tickets a priority—high, normal, or low. Working with the contractor, CRT then decides how to implement any change considered needed. By the end of November 1999, eight versions of STAR will have been issued. According to PBS reports, STAR releases 3.0, and 4.0, combined, have about 40 enhancements. Version 4.0, which is to be released by the end of November 1999, for example, is to address the problems of (1) having to enter the same information repeatedly in different modules because modules are not linked and (2) not having the ability to enter the same date twice in the Payment Schedule screen. A category named “zero square foot leases” is also to be added to STAR to cover the national antenna 6 program in a release scheduled for the spring of 2000. As of August 12, 1999, additional enhancement tickets are still open, although the exact number was unclear because PBS had not reconciled the data between the CRT and the computer-based Help Desk report. Other efforts to improve STAR include the following: 6 Zero square foot leases will also cover other non-space revenue producing items, such as docks and wareyards. Page 12 GAO/GGD-00-12 STAR—Program for Tracking and Managing Real Property B-281769 • A desk reference manual for STAR is to be issued in November 1999. This manual should help with the problem of the need to have better prompts that make where to go clearer. The users should be able to look up what to do. • A report is under development that would be similar to the combined space report, which provided such information as building address, square footage, and lease-specific information. • Renewed emphasis has been placed on examining a preview of the bill in STAR before actual bills are sent. • An ad hoc exception report on bills is being sent to the regions each month to catch errors more quickly. • PBS has set a specific accuracy goal that targets a specific data problem for improvement and rewards regions for meeting the improvement goal. We were told that this data accuracy goal would be changed on a regular basis. It currently focuses on the issue of chargebacks and missing rates. PBS knows that it has data problems, and these problems have been well PBS’ Efforts to Ensure documented. Examples of the problems follow: Accurate, Reliable, and Consistent Data in • The GSA IG concluded that over half of the space assignment drawings are off by more than 5 percent. STAR • Price Waterhouse Coopers, in its audit of GSA’s fiscal year 1998 financial statements, found a reportable condition that improvements are needed in the controls over the integrity of rent and leasing data used to manage the Federal Buildings Fund. • Exception reports and queries indicate that there is significant regional variation in how well data in STAR are maintained, how well PBS produces occupancy agreements, and the accuracy of rent billings. PBS had to do extensive reconciliation of all congressional rent bills, the Federal Bureau of Investigation had to tell PBS that it was not being billed for $3.0 million worth of assignments, and 40 percent of the first wave of Census Bureau offices that PBS had delivered had not been billed as of January 1999. PBS said it has taken various actions to ensure that STAR has accurate, reliable, and consistent data. • STAR was designed with built-in features to help ensure the reliability of some data elements. • PBS ran edits against the data to be transferred to STAR and ran additional edits after data were transferred. • PBS has an initiative to validate all space assignments. Page 13 GAO/GGD-00-12 STAR—Program for Tracking and Managing Real Property B-281769 • A special study was done that produced a report identifying data problems 7 and made recommendations for solving the problems. • A team was set up to determine how to implement the report’s recommendations. • PBS is developing a quality control system. Controls have been built into STAR to make the program provide more accurate, reliable, and consistent data. For example, STAR has fields that have to be filled in before you can go to the next step of your task. This is to ensure that all of the required data for a task are entered. Another example is work queues, which automatically generate requests for such things as approvals of leases, corrections, or notifications of missing information and may also be used manually to send information. This feature is also designed to automatically notify the budget analyst responsible for rent expenditures of changes so that the analyst may anticipate the effect of the change on expenditures. STAR makes certain calculations using data entered in the database to reduce the chance of human error. During the transition to STAR, a PBS team was formed to work with each of the teams that was developing STAR modules to assist in data transfer. According to the team leader, this team developed diagnostic edits for each module that were run on a region-by-region basis to determine missing or incomplete data before the transfer of data to STAR. Further, AT&T wrote a conversion program that included a list of edits to run against the data once they were transferred to STAR to identify missing or incomplete data. Finally, the post-conversion STAR team was tasked with regularly identifying queries, exception reports, and flags that are to be added to STAR to help identify pockets of bad data. To further ensure accurate, reliable, and consistent data, PBS established a Go Team in January 1999 to identify and quantify the real and perceived problems with inventory data—space assignment drawings, lease files, assignment files, rent rates, bills, and information in STAR and Occupancy Agreements. For example, as previously mentioned, the GSA IG found half of the space assignment drawings were off by more than 5 percent. The Go Team found considerable problems with how well PBS was creating final rent rates—the proper calculation and application of fees and other charges. In a spot check of 20 Client Billing Records, the team found minor errors in 100 percent of the cases. Also, the team found errors in parking information. In one region, a discrepancy between total parking and 7 Data Issues in PBS. Page 14 GAO/GGD-00-12 STAR—Program for Tracking and Managing Real Property B-281769 assigned parking, which should match in STAR, was found in about 5 percent of the buildings. This problem could affect revenues. Further, the team found errors in design gross square feet and gross square feet, which should match in STAR. In three regions in particular, substantial variances between the design gross square feet and the gross square feet were found. Again, this could affect revenues. In the team’s April 1999 report, it identified the causes of known data problems and identified actions that needed to be taken to fix data problems. The Go Team report identified the following as the root causes of data problems: • the lack of accountability and ownership of data; • the lack of clear, consistent, complete policies associated with PBS’ new business practices; • incomplete training; • bad data to begin with; • PBS’ reorganization that moved or de-emphasized assignment functions; • migration from the old computer program to STAR; • personnel turnover, downsizing, and workload; and • changing standards, such as moving from occupiable square feet to usable square feet to rentable square feet. In addition to those actions already under way that were discussed under the previous section of this report, the Go Team identified issues that it said must be addressed to improve data quality. These issues were • management responsibilities, • accountability and ownership, • training and tools, and • reports and measures. Table 5 lists examples of the suggested changes made by the team. Appendix II lists all of the suggested changes. Page 15 GAO/GGD-00-12 STAR—Program for Tracking and Managing Real Property B-281769 Table 5: Examples of Go Team-Identified Issues and Recommendations Type of issue Recommendations Management’s Responsibilities •Establish national policies, answer questions, and resolve disputes in a timely manner. •Change the Central Office culture of “Information is power so I won’t share it.” •Ensure that changed business practices and new tools are adopted at the working level. Accountability and Ownership •Job descriptions should include data integrity. •Competency testing and certification should be established for employees who need to understand New Pricing, Occupancy Agreements, and STAR. •Performance ratings should address the issue of employee proficiency in the use of tools and databases required in their specific job functions. Employees should be evaluated on the integrity of their data and the promptness and accuracy of their input. Training and Tools •Need to make sure that everyone who uses STAR gets the right kind of training. •Cross training opportunities between business lines should be encouraged. •More changed business practices training should be scheduled. Reports and Measures •Develop meaningful performance measures for data integrity. Any measure that is adopted should identify the magnitude of the data problems and encourage additional accountability and ownership. •Possible performance measures for data include the number of exceptions/flags generated for each region in STAR and the number and dollar value of chargebacks. Source: Data Issues in PBS, PBS, April 1999. In May 1999, another team was established to determine how to implement the first team’s recommendations. The second team developed a set of six prioritized action plans that include both temporary stop-gap fixes and permanent solutions. Table 6 lists the prioritized action plans, in priority order. Table 6: PBS’ Prioritized Action Plans by Priority Action plan Purpose 1. Education and Involvement Initially educate internally and externally for implementation of new billing in October 1999. Address long-term educational and training needs associated with data accuracy. 2. Making Decision Process More Address the mechanics of decisionmaking, suggest some standards that should exist for Effective seeking decisions, and propose some solutions for publicizing decisions once made. 3. Ensuring Data Accuracy and Establish standards for accuracy, and provide tools for people to use in dealing with PBS Appropriate Use data. 4. Business Process Integration and Address approaches ranging from process improvementse.g., billing processto Simplification systems improvementse.g., software standardizationto correction of current data and prevention of future data corruption. 5. Motivational Issues With Data and Suggest various rewards/consequences and approaches to motivate users and customers Systems to clean up and maintain data in all PBS systems. 6. Overcoming and Capitalizing Upon Identify regional variances, and decide which variances are problems and which are not. Regional Variances Source: PBS Go Team implementation paper. Page 16 GAO/GGD-00-12 STAR—Program for Tracking and Managing Real Property B-281769 The team developed the education and involvement action plan to a higher degree of detail than the other five plans due to the extreme time constraints for delivering results on these issues. The PBS Commissioner advised us that under this action plan, as of September 23, 1999, an intense training effort, called Educate 2000, was completed in each region and the central office on the effects of full implementation of New Pricing in October 1999. Also, according to the Commissioner, regions began holding customer workshops in late September to inform them of the changes in their STAR bills and provide updates on Occupancy Agreements. Further, he said that a Customer Advisory Board has been established, and its first meeting is scheduled for October 29, 1999. For action plans two, five, and six, team leaders have been appointed. Action plans three and four are being run by the Office of the Chief Information Officer and were delayed while a new Chief Information Officer was appointed. A new Chief Information Officer was appointed in July 1999. PBS is also in the final stages of hiring a contractor to develop and document a quality control system that validates real estate and billing data residing in STAR. The contractor will provide on-site support to regions in such areas as developing diagnostic reports. PBS expects to award the contract in early fiscal year 2000. On September 23, 1999, the PBS Commissioner provided written Agency Comments and comments on a draft of this report. He stated that the report is largely Our Evaluation factual and balanced and does a good job of capturing the magnitude of the effort undertaken to move GSA and PBS from 1970s batch process computing to a modern relational database platform. In the comments, the Commissioner also briefly summarized (1) PBS’ transition from its 1970s system to STAR; (2) the cultural changes that drove the need to provide its customers with more reliable information and to measure its own performance in hard, quantifiable terms as envisioned by the Government Performance and Results Act; and (3) actions being taken to improve the data in STAR and elsewhere, including a spatial validation effort using computer-assisted drawings and the development of a quality control system to validate real estate and billing data residing in STAR. He also provided technical clarifications that have been included in this report where appropriate. The Commissioner’s letter is reproduced in appendix III. Page 17 GAO/GGD-00-12 STAR—Program for Tracking and Managing Real Property B-281769 We are sending copies of this report to Senator Ben Nighthorse Campbell, Chairman, and Senator Byron L. Dorgan, Ranking Minority Member, Subcommittee on Treasury and General Government, Senate Committee on Appropriations; Senator George V. Voinovich, Chairman, and Senator Max S. Baucus, Ranking Minority Member, Subcommittee on Transportation and Infrastructure, Senate Committee on Environment and Public Works; Representative Jim Kolbe, Chairman, and Representative Steny H. Hoyer, Ranking Minority Member, Subcommittee on Treasury, Postal Service, and General Government, House Committee on Appropriations; the Honorable David J. Barram, Administrator, GSA; and to others upon request. If you have any questions regarding this report, please call me or Ron King at (202) 512-8387. Key contributors to this assignment were Tom Keightley and Mike Yacura. Bernard L. Ungar Director, Government Business Operations Issues Page 18 GAO/GGD-00-12 STAR—Program for Tracking and Managing Real Property Page 19 GAO/GGD-00-12 STAR—Program for Tracking and Managing Real Property Contents Letter 1 Appendix I 22 Conversion Issues 22 Definitions of STAR Database Change 22 Help Desk Ticket Enhancement—Application 22 Enhancement— Ad Hoc Report 22 Types Enhancement—Report Standard 22 Inquiry 22 Inquiry Reports 22 Maintenance—Nonscheduled 22 Maintenance—Scheduled 22 Policy and Procedures 23 Rent Bill 23 Security Request 23 Table Update 23 Training 23 Trouble 23 Trouble Reports 23 Appendix II 24 Data Issues Identified by PBS’ Go Team and Its Recommendations Appendix III 26 Comments From the General Services Administration Tables Table 1: Description of STAR’s Seven Major Modules 6 Table 2: Identified User Problems 8 Table 3: PBS’ Responses to Examples of User Problems 9 Table 4: Help Desk-Identified User Problems, Ranked by 11 the Number of Service Tickets Received for the Problem Table 5: Examples of Go Team-Identified Issues and 16 Recommendations Page 20 GAO/GGD-00-12 STAR—Program for Tracking and Managing Real Property Contents Table 6: PBS’ Prioritized Action Plans by Priority 16 Abbreviations CRT Change Review Team GSA General Services Administration IG Office of Inspector General PBS Public Buildings Service PROMPT Permanent Record of Managed Property Transactions STAR System for Tracking and Administering Real Property Page 21 GAO/GGD-00-12 STAR—Program for Tracking and Managing Real Property Appendix I Definitions of STAR Help Desk Ticket Types Problems or situations pertaining to data that have been converted from Conversion Issues the old system. Requests to have data changed within the database, requires written Database Change approval. Requests to change the software. Enhancement— Application Requests to change an existing ad hoc report or add a new ad hoc report. Enhancement— Ad Hoc Report Requests to change an existing standard report or add a new standard Enhancement—Report report. Standard Any question that is being asked to gather information. Inquiry Calls specifically related to questions about reports. Inquiry Reports Requests for maintenance for unexpected problems. Maintenance— Nonscheduled Issues regarding the monthly maintenance and operation schedule. Maintenance— Scheduled Page 22 GAO/GGD-00-12 STAR—Program for Tracking and Managing Real Property Appendix I Definitions of STAR Help Desk Ticket Types Questions concerning policies and procedures. Policy and Procedures Requests regarding a rent bill or the rent billing process. Rent Bill Requests on any security issue coming through the Help Desk. Security Request Request to change information in a table that supports the System for Table Update Tracking and Administering Real Property (STAR). Requests that can be solved or partially solved by additional training. Training Calls about any part of the system the user may be having problems with, Trouble except reports, including bugs and essential nonfunctioning software. Calls about any problems the user may be having regarding reports. Trouble Reports Page 23 GAO/GGD-00-12 STAR—Program for Tracking and Managing Real Property Appendix II Data Issues Identified by PBS’ Go Team and Its Recommendations Type of issue Recommendations Management’s Responsibilities •Establish national policies, answer questions, and resolve disputes in a timely manner. •Change the Central Office culture of “Information is power so I won’t share it.” •Ensure that changed business practices and new tools are adopted at the working level. •Prepare employees for the business practices and system changes and emphasize the importance of needed training. •Take a proactive position regarding data integrity and commit the resources needed to get the job done. •Resolve issues quickly, so employees do not disengage completely from the issue, making it even worse. •Instill in all employees a big picture perspective to avoid the “That’s not my job attitude.” •Become familiar with the systems the employees must use on a daily basis in order to appreciate the effort needed to learn the system. •Address, buy-into, and communicate a policy on data input by contractors. •Close the gaps that exist between the organizational vision of upper management and the changes that need to take place in business processes and roles. •Decide and publish Central Office roles and organizational structure on who ultimately decides what. Establish accountability for business decisions. •Managers must see the advantages of acting proactively—this must become a core value in PBS. •Establish a management information steering committee that will establish policy, fiscal controls and resolve conflicts. •Start having regular meetings to go over project data. Accountability and Ownership •Job descriptions should include data integrity. •Competency testing and certification should be established for employees who need to understand New Pricing, Occupancy Agreements, and STAR. •Performance ratings should address the issue of employee proficiency in the use of tools and databases required in their specific job functions. Employees should be evaluated on the integrity of their data and the promptness and accuracy of their input. •Deal with the perception that STAR and Computer-aided Design are wonderful systems that absolve us of our individual responsibility to review, track, and know our data. •If individual specialists are to be responsible for their data, then any data manipulation that takes place system-wide must be coordinated and communicated. •Stop continually shifting employees from one area to another. •Decide how much access to propriety data our contractors can/should have. Training and Tools •Need to make sure that everyone who uses STAR gets the right kind of training. •Cross training opportunities between business lines should be encouraged. •More changed business practices training should be scheduled. •Develop systems that can get information to the people who need it. •Set up recurring training/certification programs. •Expand STAR training beyond realty specialists. •Conduct more billing training, emphasizing how and when to do billing adjustments. •Plan just-in-time training and train internally before taking it to the customers. •Regional business lines need to identify their areas for immediate training. •Training for affected personnel must be mandatory and enforced by directors, supervisors, and team leaders. •Look at integrated training of multiple tools—teach STAR, Occupancy Agreements, and the client billing policy as an integrated training session. •Regions need to take advantage of the available training resources recently procured by the STAR national training team. Page 24 GAO/GGD-00-12 STAR—Program for Tracking and Managing Real Property Appendix II Data Issues Identified by PBS’ Go Team and Its Recommendations Type of issue Recommendations Training and Tools (Cont.) •Regional “Power Users” should be utilized in training and instructing other regional users in some type of regular scheduled sessions to keep users up to date on changes and enhancements to the databases and internal processes. •Identify any strong regional training programs as best practices, and have the rest of PBS learn from them. Medium/long term actions •Some type of evaluation or testing of users needs to be developed to ensure that they understand the database(s) and process required for them. •A training infrastructure should be developed in each region that provides for regularly scheduled sessions on all of the various databases utilized by GSA. Training sessions should be interactive. Customer training and marketing •Develop national, regional, and local dialogs/workshops with our customers so they fully understand new pricing, billing, and the changes coming to existing assignments before they arrive October 1, 1999. Reports and Measures •Develop meaningful performance measures for data integrity. Any measure that is adopted should identify the magnitude of the data problems and encourage additional accountability and ownership. Possible performance measures for data include the following: •Number of exceptions/flags generated for each region in STAR. •Number and dollar value of chargebacks. •Number of unopened work queues over some number of days old in STAR. •Develop a workload measure that would help management determine when to deploy additional resources, if possible. •Number of billing adjustments. •Number of customer complaints with response times for resolution. Page 25 GAO/GGD-00-12 STAR—Program for Tracking and Managing Real Property Appendix III Comments From the General Services Administration Page 26 GAO/GGD-00-12 STAR—Program for Tracking and Managing Real Property Appendix III Comments From the General Services Administration Page 27 GAO/GGD-00-12 STAR—Program for Tracking and Managing Real Property Ordering Information The first copy of each GAO report and testimony is free. Additional copies are $2 each. Orders should be sent to the following address, accompanied by a check or money order made out to the Superintendent of Documents, when necessary. VISA and MasterCard credit cards are accepted, also. Orders for 100 or more copies to be mailed to a single address are discounted 25 percent. Order by mail: U.S. General Accounting Office P.O. Box 37050 Washington, DC 20013 or visit: Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders may also be placed by calling (202) 512-6000 or by using fax number (202) 512-6061, or TDD (202) 512-2537. Each day, GAO issues a list of newly available reports and testimony. To receive facsimile copies of the daily list or any list from the past 30 days, please call (202) 512-6000 using a touch- tone phone. A recorded menu will provide information on how to obtain these lists. For information on how to access GAO reports on the INTERNET, send e-mail message with “info” in the body to: firstname.lastname@example.org or visit GAO’s World Wide Web Home Page at: http://www.gao.gov United States Bulk Rate General Accounting Office Postage & Fees Paid Washington, D.C. 20548-0001 GAO Permit No. G100 Official Business Penalty for Private Use $300 Address Correction Requested (240329)
General Services Administration: STAR-PBS' New Program for Tracking and Managing Real Property
Published by the Government Accountability Office on 1999-10-12.
Below is a raw (and likely hideous) rendition of the original report. (PDF)