oversight

Public Housing: HUD Should Improve the Usefulness and Accuracy of Its Management Assessment Program

Published by the Government Accountability Office on 1997-01-29.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                 United States General Accounting Office

GAO              Report to the Chairman, Subcommittee
                 on Housing and Community Opportunity,
                 Committee on Banking and Financial
                 Services, House of Representatives

January 1997
                 PUBLIC HOUSING
                 HUD Should Improve
                 the Usefulness and
                 Accuracy of Its
                 Management
                 Assessment Program




GAO/RCED-97-27
      United States
GAO   General Accounting Office
      Washington, D.C. 20548

      Resources, Community, and
      Economic Development Division

      B-275648

      January 29, 1997

      The Honorable Rick Lazio
      Chairman, Subcommittee on Housing
        and Community Opportunity
      Committee on Banking and
        Financial Services
      House of Representatives

      Dear Mr. Chairman:

      This report responds to your request that we review the Department of Housing and Urban
      Development’s (HUD) use of its Public Housing Management Assessment Program (PHMAP).
      Specifically, the report discusses (1) HUD’s use and implementation of the program at its field
      offices, (2) public housing authorities’ (PHA) PHMAP scores over the first 4 years of the program,
      and (3) limits on any additional uses for the program.

      We are sending copies of this report to the Secretary of Housing and Urban Development and
      will make copies available to others upon request. Major contributors to this report are listed in
      appendix VI.

      Please call me at (202) 512-7631 if you or your staff have any questions.

      Sincerely yours,




      Judy A. England-Joseph
      Director, Housing and Community
        Development Issues
Executive Summary


                 At a current annual cost of $5.4 billion, the Department of Housing and
Purpose          Urban Development (HUD) subsidizes the operation, maintenance, and
                 modernization of the nation’s public housing, a $90 billion investment that
                 provides homes to 3 million people. Because HUD provides this subsidy to
                 more than 3,000 independent, state-chartered public housing authorities,
                 the Congress holds HUD responsible for ensuring that these authorities
                 efficiently provide safe and decent housing and protect the federal
                 investment in their properties. However, the Public Housing Management
                 Assessment Program (PHMAP)—HUD’s primary tool for measuring housing
                 authorities’ performance—has been criticized as unreliable, inaccurate,
                 and at times conflicting with good property management practices.
                 Nevertheless, because no other measurement tool exists, the Congress and
                 HUD have proposed at different times using the program as a basis for
                 deregulating or rewarding high-scoring housing authorities.

                 Stressing the need for HUD to hold housing authorities accountable while
                 making better use of the data PHMAP produces, the Chairman of the
                 Subcommittee on Housing and Community Opportunity, House
                 Committee on Banking and Financial Services, asked GAO to review HUD’s
                 implementation and use of PHMAP. Specifically, the Chairman asked GAO to
                 determine

             •   whether HUD’s field offices are using PHMAP and complying with the
                 program’s statutory and regulatory requirements to monitor and provide
                 technical assistance to housing authorities;
             •   whether PHMAP scores have increased and how HUD uses the program to
                 inform HUD’s Secretary and the Congress about the performance of
                 housing authorities; and
             •   whether PHMAP scores are consistently accurate and can be considered a
                 generally accepted measure of good property management.


                 The National Affordable Housing Act of 1990 directed HUD to use certain
Background       indicators, including vacancy rates and the percentage of rents
                 uncollected, to assess the management performance of local housing
                 authorities. The act also directed HUD to determine the cause of an
                 authority’s management problems, commit both HUD and the housing
                 authority to a specific course of corrective action, and document agreed
                 upon corrective actions in memorandums of agreement. To meet the act’s
                 requirements, HUD developed PHMAP to annually obtain data from each
                 housing authority on 12 basic indicators of management performance,
                 such as vacancy rates and operating expenses. On the basis of aggregate



                 Page 2                                          GAO/RCED-97-27 Public Housing
Executive Summary




performance against these indicators, HUD calculates a score from 0 to 100
for each authority and assigns one of the following three designations:
“troubled performer” for a score less than 60, “standard performer” for a
score between 60 and less than 90, and “high performer” for a score 90 or
above.

HUD’s field offices have nearly 800 staff devoted to oversight of housing
authorities and implementation of the full range of the Department’s
public housing programs, including PHMAP. Because HUD’s field offices are
responsible for implementing PHMAP, HUD expects them to ensure housing
authorities meet PHMAP’s requirements and provide technical assistance
when an authority has problems doing so. Technical assistance can cover
a wide variety of activities, such as a focused review of an authority’s
compliance with HUD’s rules and regulations, discussions over the
telephone, on-site reviews of HUD-funded modernization work, or
suggestions for improving an authority’s occupancy rate and rent
collection procedures.

Field offices depend on each authority to submit and certify to the
accuracy of about half the data that lead to the overall PHMAP score; the
balance of the information HUD uses comes from its existing information
system for tracking expenditures from major grants. With each troubled
authority, the act requires HUD to perform an independent management
assessment to determine the causes of an authority’s problems and then
enter into a binding memorandum of agreement stipulating the problems
that authority needs to address and an approach and a timetable to resolve
them. Also, when a troubled authority’s new PHMAP score would cause HUD
to remove its troubled designation, HUD’s policy has been to require its
field offices to verify the accuracy and completeness of the data the
housing authority submitted. Standard- and high-performing authorities
that fail any of the 12 indicators must submit a plan for improving their
performance in that indicator.

While HUD’s primary use of PHMAP has been to identify troubled housing
authorities and target technical assistance to them, the Congress and HUD
have proposed to use PHMAP for other purposes. In 1994, the Senate
Committee on Banking, Housing, and Urban Affairs proposed additional
flexibility for housing authorities that had achieved PHMAP scores over 90.
In its fiscal year 1997 budget request, HUD proposed to give
high-performing housing authorities bonuses based in part on their PHMAP
scores.




Page 3                                           GAO/RCED-97-27 Public Housing
                           Executive Summary




                           Most of HUD’s field offices are using PHMAP to identify troubled housing
Results in Brief           authorities and target HUD’s limited technical assistance resources.
                           However, the field offices have not been systematically using the
                           assessment program, as required by statutes and regulations, to monitor
                           housing authorities’ progress in improving their performance and target
                           technical assistance to them. For example, the field offices have generally
                           not been meeting the act’s requirement to enter into memorandums of
                           agreement with troubled authorities, nor have the field offices consistently
                           met HUD’s requirement that housing authorities document plans to correct
                           low scores in individual performance indicators. Also, the impact of a 1995
                           reorganization of the field offices’ functions and current departmental
                           downsizing continue to influence some offices’ ability to provide technical
                           assistance.

                           Performance scores generally have increased during the first 4 full years of
                           the program. With average scores increasing, the total number of troubled
                           housing authorities has decreased, and the greatest proportion of those
                           that are troubled are the smallest authorities—those managing fewer than
                           100 units. The proportion of high-performing authorities has increased
                           steadily from about 33 percent in 1992 to over 50 percent in 1995.
                           High-performing authorities manage nearly 50 percent of all public
                           housing units. Periodically, HUD officials provide the Secretary and the
                           Congress information on the performance of all housing authorities as
                           well as the number of troubled authorities.

                           HUD’s confirmatory reviews of the information underlying assessment
                           scores have shown the scores to be inaccurate in half the instances when
                           such reviews were performed. Regardless of the scores’ accuracy, HUD and
                           public housing industry officials do not believe that the management
                           assessment program comprehensively assesses how well local housing
                           authorities manage their properties. This is because the assessment
                           program does not include indicators to specifically measure overall
                           housing quality or the quality of maintenance.



Principal Findings

Field Offices’ Use of      GAO visited 5 of HUD’s 49 field offices and sent a survey to all 49 offices
PHMAP and Related          asking about their use of PHMAP. Officials in these offices generally found
                           PHMAP useful to identify troubled housing authorities and target limited
Oversight Tools Has Been
                           technical assistance and oversight resources. However, field offices
Limited

                           Page 4                                            GAO/RCED-97-27 Public Housing
    Executive Summary




    reported that they have not been systematically complying with PHMAP’s
    statutory and regulatory follow-up requirements for all housing
    authorities.

    The extent to which HUD’s field offices used PHMAP to provide technical
    assistance to housing authorities varies widely. In part, this difference
    stems from how different field offices have interpreted their role in
    helping authorities improve performance on PHMAP indicators as well as
    their overall operations. Some field offices told GAO that they interpret
    their role in providing oversight and technical assistance narrowly,
    generally limiting their assistance to advice, information on complying
    with federal rules and regulations, and suggestions for solving
    management problems. Other offices were more willing to get involved in
    a housing authority’s operations. For example, staff from one field office
    spent several days at a troubled authority to help it set up proper tenant
    rent records and waiting lists.

    Regarding noncompliance with follow-up requirements, GAO found the
    following:

•   In 1992, HUD’s field offices entered into 29 percent of the required
    memorandums of agreement; by 1995, just 18 percent of the housing
    authorities that should have been operating under a memorandum of
    agreement actually were. Primarily, field offices said the reason they did
    not enter into these required agreements with troubled housing authorities
    is that the housing authorities had already corrected or were in the
    process of correcting their management deficiencies.
•   Even though HUD requires an improvement plan to address each
    performance indicator an authority fails unless that authority can correct
    the deficiency within 90 days, 31 percent of HUD’s field offices had not
    ensured that housing authorities had developed such plans.
•   Field offices generally did not meet HUD’s requirement to confirm the
    accuracy of all PHMAP scores that were high enough to remove the troubled
    designation from a housing authority. In fiscal year 1995, HUD’s field offices
    confirmed fewer than 30 percent of the scores that should have been
    confirmed. While 13 offices performed none of the required confirmatory
    reviews, some of these same offices performed confirmatory reviews of
    other standard- or high-performing authorities. Although no minimum
    level of activity is required, in 1995 HUD confirmed just over 6 percent of all
    scores, with some field offices performing no confirmatory reviews and
    others performing 10 or more.




    Page 5                                             GAO/RCED-97-27 Public Housing
                                 Executive Summary




                             •   At the five field offices GAO visited, limited use was made of the
                                 independent financial and compliance audits conducted annually at each
                                 authority. Over a year ago, HUD began to require that these audits certify
                                 that the PHMAP data from each authority were accurate and complete; HUD
                                 added this requirement because it does not have the resources to confirm
                                 each score every year. Nonetheless, few HUD staff in the field offices GAO
                                 visited were aware of this requirement or used the audits to better focus
                                 their oversight and technical assistance.

                                 The technical assistance HUD staff find most effective at improving the
                                 performance of housing authorities is often the type of help least
                                 frequently provided. HUD’s field offices told GAO that technical assistance
                                 and oversight are most effective when they provide it on-site at the local
                                 authority. For example, while most staff in field offices said confirmatory
                                 reviews, which must take place on-site, were one of the most effective
                                 ways to provide technical assistance, discussions over the telephone were
                                 the most common form of technical assistance. Some field offices cited
                                 resource constraints—a lack of staff, travel funds, or expertise—as the
                                 main reason for not meeting follow-up requirements or visiting housing
                                 authorities more often; others opted not to enforce requirements when
                                 they believed the authorities were already addressing their problems.


Scores Are Increasing, but       Average PHMAP scores have increased over the life of the program, rising
HUD Recognizes Database          from an average of 83 in 1992 to 86 in 1995, the last year of complete data.
Flaws Limit Program’s Use        In addition, HUD’s database of PHMAP scores indicates the following two
                                 trends:

                             •   The number of high-performing authorities grew each year, from almost a
                                 third in 1992 (33 percent) to over half in 1995 (57 percent).
                             •   The number of troubled housing authorities has declined from 130 in 1992
                                 to 83 in 1995. However, by 1995, the smallest housing authorities—those
                                 managing fewer than 100 units—accounted for a greater share of those
                                 designated as troubled than when the program began. In 1995, half of all
                                 housing authorities HUD designated as troubled were small.

                                 GAO found missing, inaccurate, and inconsistent data in HUD’s primary
                                 database for storing PHMAP scores. Nevertheless, HUD makes regular,
                                 periodic use of the database to provide information to the Secretary and
                                 the Congress on all housing authorities’ scores and the number of troubled
                                 authorities at any given time. However, before providing this information
                                 to others, HUD first manually verifies much of the data it draws from this



                                 Page 6                                            GAO/RCED-97-27 Public Housing
                               Executive Summary




                               system. Senior HUD officials acknowledged these problems with the
                               database and added that they are currently working to address data
                               accuracy and reliability problems as well as improve their ability to
                               correct errors sooner.


The Questionable               PHMAP  scores are often changed after HUD confirms the data used to
Accuracy and Validity of       support the scores. In commenting on this report, HUD said that most
PHMAP Scores Limit the         confirmatory reviews are conducted of high-risk housing authorities
                               whose data are most susceptible to being found inaccurate. Over
Program’s Usefulness           half—58 percent—of the changes HUD made to PHMAP scores resulted in
                               HUD’s lowering the score by an average of 14 points; 42 percent of the
                               changes resulted in HUD’s raising the score by an average of 8 points.
                               Typically, HUD changes a PHMAP score after such a review for several
                               reasons, including the housing authority’s failure to report correctly the
                               required data or its failure to maintain documentation to support its data.

                               PHMAP scores are not a generally accepted measure of good property
                               management. HUD officials, as well as representatives of public housing
                               industry associations and professional property management consultants,
                               told GAO that the PHMAP indicators do not assess all major aspects of a
                               housing authority’s performance. For example, PHMAP does not include an
                               independent on-site inspection of the condition of an authority’s housing,
                               so it does not adequately assess the quality of modernization work or
                               routine maintenance. These same HUD officials and industry
                               representatives also told GAO that PHMAP does not always allow for
                               extenuating circumstances that can lead to decisions inconsistent with
                               good property management. For example, a housing authority can improve
                               its PHMAP score on the tenants accounts receivable indicator by writing off
                               as uncollectible past due rents from vacated tenants, but PHMAP would not
                               measure how diligent an effort the authority had undertaken to collect the
                               rent.


                               GAO   recommends that the Secretary of HUD
Recommendations
                           •   provide guidance to its field offices that clearly (1) articulates their
                               minimally acceptable roles regarding oversight and assistance to housing
                               authorities and (2) emphasizes the importance of using the results of the
                               independent audits to better target the Department’s limited technical
                               assistance resources.




                               Page 7                                           GAO/RCED-97-27 Public Housing
                      Executive Summary




                      Furthermore, because scores are not consistently accurate and PHMAP does
                      not measure all aspects of property management, GAO recommends that
                      HUD


                  •   not consider additional uses for PHMAP, including using scores as criteria
                      for funding bonuses, until it determines that PHMAP meets an acceptable
                      level of accuracy and more comprehensively measures property
                      management performance and
                  •   require its field offices to confirm the PHMAP scores of housing authorities
                      with scores low enough that the authorities are at risk of being designated
                      troubled.



                      GAO  provided a draft of this report to HUD for its review and comment. HUD
Agency Comments       agreed with GAO’s recommendations and described the steps that the
                      Department has begun taking to implement them. However, HUD expressed
                      concern that the draft report (1) used potentially inaccurate data from
                      HUD’s PHMAP database, (2) incorrectly assumed that PHMAP was intended to
                      be an all-encompassing system that measures both management
                      performance and physical housing conditions, (3) neglected to place PHMAP
                      in a historical perspective by discussing HUD’s previous systems for
                      assessing and identifying troubled housing authorities, and (4) reached
                      incorrect conclusions regarding the reliability of all PHMAP scores on the
                      basis of the results of confirmatory reviews of the high-risk authorities
                      most susceptible to discrepancies in their PHMAP data.

                      GAO used the best data available for this review. GAO recognizes and
                      discusses in the report the inaccuracies in the PHMAP data. Where HUD was
                      able to provide more accurate data than its PHMAP database reports, GAO
                      used that data in this report. GAO did not assume that PHMAP should be a
                      complete measure of both performance and physical conditions. This
                      report describes the current uses of PHMAP data and addresses how the
                      program’s limitations affect its suitability for additional purposes. GAO did
                      not provide a historical perspective on the program because discussing
                      HUD’s previous systems for assessing and identifying troubled housing
                      authorities did not contribute to the review’s objectives of evaluating HUD’s
                      use of PHMAP, trends in scores, or limits on additional uses for the program.
                      Finally, GAO did not reach a conclusion about the reliability of all housing
                      authorities’ PHMAP scores. The report discusses only the reliability of PHMAP
                      scores for those housing authorities whose scores are so low that they




                      Page 8                                            GAO/RCED-97-27 Public Housing
Executive Summary




may be at risk of being designated troubled. Statements have been added
to the report to clarify this point.

HUD’s written comments are presented in appendix V and GAO’s responses
are discussed at the end of each chapter as appropriate.




Page 9                                         GAO/RCED-97-27 Public Housing
Contents



Executive Summary                                                                                2


Chapter 1                                                                                       12
                         Indicator Grades Determine the PHMAP Score, Performance                14
Introduction               Designation, Required Follow-Up, and Incentives
                         HUD’s Field Offices Implement PHMAP                                    16
                         Objectives, Scope, and Methodology                                     18

Chapter 2                                                                                       21
                         HUD Uses PHMAP to Identify Troubled Housing Authorities, but           21
Although Field Offices     Technical Assistance Varies
Use PHMAP to             HUD’s Infrequent Use of Some Oversight Tools May Not                   24
                           Adequately Improve the Performance of PHAs or Target
Identify Troubled          Technical Assistance
PHAs, Compliance         Conclusions                                                            31
With Statutory and       Recommendation                                                         31
                         Agency Comments                                                        31
Agency Follow-Up
Requirements Has
Been Limited
Chapter 3                                                                                       33
                         Most PHMAP Scores Are Increasing and Fewer Housing                     33
Although PHMAP             Authorities Are Troubled
Scores Have Risen,       HUD Recognizes Database Flaws and Plans Corrections                    37
                         Agency Comments                                                        39
HUD Recognizes That
Flaws in the
Program’s Database
Limit Its Use
Chapter 4                                                                                       40
                         Accuracy of Scores and Validity as a Management Assessment             40
The Questionable           Tool Limit Uses for PHMAP
Accuracy of PHMAP’s      The Congress and HUD Have Proposed to Use PHMAP as a Basis             43
                           for Deregulation and Funding Bonuses
Scores and the           Conclusions                                                            45
Program’s Validity       Recommendations                                                        45
Limit Its Usefulness     Agency Comments                                                        46




                         Page 10                                      GAO/RCED-97-27 Public Housing
             Contents




Appendixes   Appendix I: Average PHMAP Score by Geographic Region, Fiscal              48
               Years 1992-95
             Appendix II: Number of Troubled PHAs by Geographic Region,                50
               Fiscal Years 1992-95
             Appendix III: Number of Standard-Performing PHAs by                       52
               Geographic Region, Fiscal Years 1992-95
             Appendix IV: Number of High-Performing PHAs by Geographic                 54
               Region, Fiscal Years 1992-95
             Appendix V: Comments From the Department of Housing and                   56
               Urban Development
             Appendix VI: Major Contributors to This Report                            61

Tables       Table 1.1: Twelve PHMAP Indicators                                        13
             Table 2.1: PHAs Receiving a Confirmatory Review From a HUD                28
               Field Office, Fiscal Years 1992-95
             Table 3.1: Average PHMAP Score by PHA Size Category, Fiscal               34
               Years 1992-95
             Table 3.2: Number of PHAs by PHMAP Performance Category,                  35
               Fiscal Years 1992-95
             Table 3.3: Failed Indicators by Percentage of PHAs, Fiscal Years          36
               1992-95
             Table 4.1: Changes in PHMAP Scores After HUD’s Field Offices              41
               Performed Confirmatory Reviews in Fiscal Year 1995

Figures      Figure 2.1: Percentage of Troubled PHAs Operating Under a                 26
               MOA, Fiscal Years 1992-95
             Figure 2.2: Number of Confirmatory Reviews Performed by HUD               29
               Field Offices, Fiscal Year 1995
             Figure 3.1: Number of Troubled PHAs by Size, Fiscal Years                 37
               1992-95



             Abbreviations

             GAO        General Accounting Office
             HUD        Department of Housing and Urban Development
             IG         Office of Inspector General
             MOA        memorandum of agreement
             PHA        public housing authority
             PHMAP      Public Housing Management Assessment Program
             SMIRPH     System for Management Information Retrieval-Public
                             Housing


             Page 11                                         GAO/RCED-97-27 Public Housing
Chapter 1

Introduction


               Because public housing represents a $90 billion investment on the part of
               the federal government since the program’s inception in 1937 and because
               the Department of Housing and Urban Development (HUD) currently
               spends $5.4 billion a year on operating subsidies and modernization grants
               for this housing, interest remains keen in knowing how well local public
               housing authorities (PHA) are managing their properties. The PHAs, through
               which HUD provides these subsidies and grants, house 3 million
               low-income people, many of whom are elderly or disabled. The Congress
               holds HUD responsible for ensuring that the authorities provide safe and
               decent housing, operate their developments efficiently, and protect the
               federal investment in their properties.

               The National Affordable Housing Act of 1990 required HUD to develop
               indicators to assess the management performance of PHAs.1 This law
               became the framework through which HUD developed one of its primary
               oversight tools for housing authorities, the Public Housing Management
               Assessment Program (PHMAP). Primarily, PHMAP establishes objective
               standards for HUD to evaluate and monitor the management operations of
               all PHAs to identify those that are troubled. According to HUD, PHMAP also
               allows the Department to identify ways to reward high-performing PHAs as
               well as improve the management practices of troubled PHAs. The program
               also allows PHAs’ governing bodies, management officials, residents, and
               the local community to better understand and identify specific program
               areas needing improvement.

               To help improve public housing management, the National Affordable
               Housing Act of 1990, as amended (the act), required HUD to develop
               indicators to assess the performance of PHAs in all the major aspects of
               their management operations. The act required HUD to use certain
               indicators as well as provided discretion for the Secretary of HUD to
               develop up to five additional indicators that the Department deemed
               appropriate. HUD implemented PHMAP by using the 12 indicators listed in
               table 1.1, the first seven of which are those required by statute.




               1
               P.L. 101-625, Section 502(a), as amended by the departments of Veterans Affairs and Housing and
               Urban Development, and Independent Agencies Appropriations Act of 1992.



               Page 12                                                         GAO/RCED-97-27 Public Housing
                                     Chapter 1
                                     Introduction




Table 1.1: Twelve PHMAP Indicators
                                     PHMAP indicator                                     Measurement
                                     1.Vacancy number and percentage                     Number and percentage of vacancies,
                                                                                         including progress made within the
                                                                                         previous 3 years to reduce vacancies
                                     2.Modernization                                     Amount and percentage of funds
                                                                                         unexpended after 3 years
                                     3.Rents uncollected                                 Balance uncollected as a percentage of
                                                                                         total rents to be collected
                                     4.Energy consumption                                Increase in annual consumption
                                     5.Unit turnaround                                   Average time required to repair and
                                                                                         reoccupy vacant units
                                     6.Outstanding workorders                            Proportion of maintenance workorders
                                                                                         outstanding
                                     7.Annual inspection and condition of units          Percentage of units and systems not
                                     and systems                                         inspected to determine preventive
                                                                                         maintenance or modernization needs
                                     8.Tenants accounts receivable                       Percentage of monies owed to the
                                                                                         authority by current residents
                                     9.Operating reserves                                Percentage of operating reserves
                                                                                         maintained by the authority
                                     10.Routine operating expenses                       Level of operating expenses compared to
                                                                                         operating income and federal subsidy
                                     11.Resident initiatives                             Existence of a partnership between
                                                                                         residents and the authority to promote
                                                                                         opportunities for self-sufficiency and other
                                                                                         programs
                                     12.Development                                      Ability to develop additional public housing
                                                                                         units

                                     Because some indicators are more important than others in measuring
                                     management performance, HUD assigns them added weight in determining
                                     the overall score.2 HUD considers the indicators for vacancies, rents
                                     uncollected, annual inspection and condition of units and systems, and
                                     resident initiatives most indicative of good property management and
                                     delivery of services to residents, so each one has a greater weight than
                                     other indicators.

                                     After reviewing existing procedures and extensively consulting with a
                                     group of PHAs, public housing industry groups, private management firms,
                                     resident groups, and HUD staff in field offices, HUD has significantly revised
                                     the PHMAP indicators. HUD’s revisions to PHMAP, published December 30,
                                     1996, eliminated three indicators; consolidated four other indicators into

                                     2
                                     According to a HUD official, these weights reflect the Department’s determination that they are the
                                     most important indicators of good property management.



                                     Page 13                                                           GAO/RCED-97-27 Public Housing
                            Chapter 1
                            Introduction




                            two; and added one new indicator, security.3 These revisions primarily
                            address the performance indicators on which housing authorities report
                            data, not HUD’s use of PHMAP data.


                            Annually, PHAs receive a grade of “A” through “F” for each of the twelve
Indicator Grades            indicators that apply to their operations. HUD uses a formula that reflects
Determine the               the weights assigned to each indicator, converts indicator grades into
PHMAP Score,                points, totals each PHA’s points, and divides that total by the maximum
                            total the PHA could have achieved to arrive at a percentage. That
Performance                 percentage, a number between 0 and 100, is the PHMAP score.
Designation, Required
                                 draws data on the performance of a housing authority from two
Follow-Up, and              HUD
                            sources to determine the authority’s PHMAP score. First, the housing
Incentives                  authority submits data to HUD for about half of the PHMAP indicators and
                            certifies that this information is accurate and complete.4 HUD assigns
                            grades to each of these indicators according to a comparison of the
                            authority’s data and HUD’s criteria for grades “A” through “F.” The balance
                            of the information HUD uses comes from its own information system for
                            tracking expenditures from major grants. This system contains the
                            financial and other types of data the field offices need to grade the
                            remaining indicators for which the PHAs do not provide data. The field
                            offices use this data and the PHA-certified data to determine indicator
                            scores, the PHMAP score, and the PHA’s performance designation.

                            The PHMAP score is HUD’s starting point for both the performance
                            designation it assigns to a PHA and, depending on that designation, the
                            extent of follow-up required of the PHA to correct deficiencies identified
                            during the PHMAP assessment. Generally, HUD uses three designations to
                            describe the performance of PHAs:

                        •   troubled performers are those scoring less than 60 percent;5
                        •   standard performers are those scoring between 60 and less than
                            90 percent; and
                        •   high performers are those scoring 90 percent or more.

                            3
                             HUD eliminated the tenants accounts receivable, routine operating expenses, and development
                            indicators. HUD also consolidated unit turnaround into the vacancy indicator and energy consumption
                            into the financial management (formerly operating reserves) indicator.
                            4
                             PHAs certify the data for the following indicators: vacancy number and percentage, rents uncollected,
                            unit turnaround, outstanding work orders, annual inspection and condition of units and systems, and
                            resident initiatives.
                            5
                             PHMAP includes an additional designation, modernization troubled (mod-troubled), which can apply
                            to any PHA that scores less than 60 percent on the modernization indicator.



                            Page 14                                                           GAO/RCED-97-27 Public Housing
Chapter 1
Introduction




HUD has the discretion to withhold the troubled designation or award the
high performer designation if a PHA’s score is within 10 points of the
threshold for either designation and HUD determines that its score results
from the physical condition and/or neighborhood environment of that
authority’s units rather than from the PHA’s poor management practices.

If a housing authority is designated as troubled, it faces several mandatory
follow-up activities and/or corrective actions to improve performance and
remove the troubled designation. Specifically, the act requires HUD to
perform an independent management assessment of the troubled PHA’s
overall operations to identify the causes of the deficiencies that led to its
poor PHMAP score. HUD uses private contractors to perform these
independent assessments.

HUD  expects the independent assessments to form the basis for the second
requirement for troubled PHAs—the memorandum of agreement (MOA). A
memorandum of agreement is a binding contract between HUD and a
troubled PHA to identify solutions to its management problems and pursue
those solutions in a way that is significant, expeditious, and lasting.
Among other things, HUD requires that the MOA address the specific
responsibilities of HUD and the PHA, the resources each will commit to
resolving the authority’s problems, the annual and quarterly performance
targets for improving its performance on PHMAP indicators, and the
incentives for it to meet its performance targets as well as sanctions for
failing to do so. A PHA’s initial MOA generally lasts 18 months so that it can
complete a second-year agreement with HUD, if necessary, before the first
expires.6

HUD’s regulations for implementing PHMAP require standard- and
high-performing PHAs to develop improvement plans for every PHMAP
indicator on which the PHA received an “F,” unless the PHA can correct the
deficiency within 90 days;7 HUD may also choose to require these plans for
indicators receiving scores of “D” or “E” when failure to raise the grade
might pose significant added risk. An improvement plan documents how
and when the PHA plans to correct deficiencies. Although similar in content
and scope to a memorandum of agreement, improvement plans differ in
that (1) PHAs develop and submit them to HUD for approval rather than



6
 Second and subsequent year agreements are necessary only as long as the PHA remains troubled.
7
 Mandatory improvement plans would not apply to the modernization indicator because any PHA
receiving an F on this would be designated mod-troubled.



Page 15                                                         GAO/RCED-97-27 Public Housing
                      Chapter 1
                      Introduction




                      negotiate them with HUD officials and (2) they are not a binding
                      contractual commitment between the PHA and HUD.

                      When HUD first implemented PHMAP, it offered high-performers a variety of
                      incentives, primarily regulatory relief from various reporting requirements.
                      These incentives included less frequent reviews of changes to a PHA’s
                      operating budget and, for those performing well on the modernization
                      indicator, no prior HUD review for architects’ or engineers’ contracts. In
                      addition to regulatory relief, high-performing PHAs receive a HUD certificate
                      of commendation and public recognition for their performance.

                      In its fiscal year 1997 budget request, HUD proposed an additional
                      PHMAP-based    incentive for high-performing PHAs when it sought to create a
                      $500 million capital bonus fund (as part of the $3.2 billion it sought for its
                      public housing capital fund).8 To be eligible for a bonus, a PHA would have
                      to be a PHMAP high performer and have undertaken substantive efforts to
                      obtain education and job training for its residents. However, the Congress
                      chose not to fund the bonus proposal for public housing or any of HUD’s
                      other major programs, in part because of concerns about HUD’s ability to
                      accurately and reliably track the performance of bonus recipients.


                      With nearly 800 staff devoted to oversight of housing authorities and
HUD’s Field Offices   implementation of the full range of HUD’s public housing programs, its field
Implement PHMAP       offices have the bulk of the Department’s responsibility for the day-to-day
                      implementation of PHMAP. Field offices’ PHMAP responsibilities include
                      determining the indicator grades and PHMAP scores, negotiating
                      memorandums of agreement, approving PHAs’ improvement plans, and
                      monitoring their progress in meeting the goals the MOA or improvement
                      plan set forth. To determine a housing authority’s PHMAP score, a field
                      office relies on that PHA to provide about half the data that leads to the
                      overall PHMAP score and certify the data’s accuracy. As a result, the overall
                      PHMAP score and everything it influences—from incentives for high
                      performers to sanctions for troubled PHAs—are very much a joint effort
                      and a shared responsibility.

                      A PHA may also request to exclude or modify the data HUD should consider
                      in computing its PHMAP score. An exclusion means that the indicator (or
                      one or more of its components) is entirely excluded from calculations to
                      determine the PHMAP score. For example, PHAs with no ongoing

                      8
                       The public housing capital fund would have consolidated public housing modernization, development,
                      and several other capital repair and replacement programs.



                      Page 16                                                         GAO/RCED-97-27 Public Housing
Chapter 1
Introduction




modernization or development programs are automatically excluded from
being assessed on those indicators. Modifying the data for an indicator
allows HUD to consider unique or unusual circumstances by exempting
some of the data HUD usually requires the PHA to consider. The PHA still
receives a score for the indicator, but the score would not reflect the data
associated with the PHA’s unique or unusual circumstances. For example, a
PHA operating under a court order not to collect tenants’ rent at specific
developments until it corrects deficiencies the court had identified can
seek to exempt those units in its developments from being considered in
its indicator score for rents uncollected. A PHA always has the right to
appeal a field office’s decision about modifications, exclusions, indicator
scores, or the performance designation. However, after those appeals have
been exhausted, the field office certifies the PHA’s PHMAP score, assigns a
final performance designation, and proceeds with any required
improvement plans, MOAs, or other necessary follow-up.

When a troubled authority’s new PHMAP score is high enough to cause HUD
to remove its troubled designation, HUD’s policy is to require the field
office to verify the accuracy and completeness of the new data submitted
by the housing authority. HUD also requires the field office to conduct a
confirmatory review to verify the data the PHA had certified as well as the
accuracy of the data HUD had obtained from its own information system.
HUD’s guidance for implementing PHMAP stipulates that a confirmatory
review must take place on-site at the PHA and cannot be accomplished
through remote monitoring.

HUD’s  field offices may choose to conduct some confirmatory reviews of
standard- and high-performing PHAs’ PHMAP certifications. HUD expects its
field offices to choose these PHAs according to the risk they pose and focus
on those with the highest potential for fraud, waste, mismanagement, or
poor performance. Some of the factors HUD field offices may consider in
analyzing the risk associated with a PHA’s PHMAP certification include size
(number of units), borderline troubled designation (5 percent above or
below the percentage for the designation), and negative trends in overall
or individual indicator scores over several years.

In May 1995, HUD expanded the scope of the annual independent audit
each PHA receives in order to improve the Department’s ability to
determine whether PHA-certified data are accurate. The annual audit,
conducted pursuant to the requirements of the Single Audit Act, examines
the housing authority’s financial statements, internal controls, and
compliance with HUD’s rules and regulations. Housing authorities are



Page 17                                          GAO/RCED-97-27 Public Housing
                         Chapter 1
                         Introduction




                         responsible for selecting their own auditors and submitting the results of
                         the audits to their HUD field office. Field offices are responsible for
                         reviewing the audits to ensure they meet all of HUD’s requirements and,
                         when they have approved the audit, reimbursing housing authorities for
                         them. In fiscal year 1995, these independent audits cost HUD about
                         $8 million for all housing authorities.

                         HUD now requires the independent auditors to determine whether a
                         housing authority has adequate documentation for the data it submits to
                         HUD for its PHMAP certification. According to HUD officials, because the
                         Department’s resources are too limited to conduct annual confirmatory
                         reviews of most housing authorities, they expected to use the results of
                         these audits to better focus HUD’s attention, oversight, and technical
                         assistance. In addition to paying for the audits, HUD expects its field offices
                         to use the results as part of a risk assessment to determine which housing
                         authorities should get the most sustained attention and technical
                         assistance.


                         Stressing the need for HUD to hold housing authorities accountable while
Objectives, Scope,       making better use of the data that PHMAP produces, the Chairman of the
and Methodology          Subcommittee on Housing and Community Opportunity, House
                         Committee on Banking and Financial Services, asked GAO to review HUD’s
                         use and implementation of PHMAP. As agreed with the Chairman’s office,
                         we reviewed

                     •   whether HUD’s field offices are using PHMAP and complying with the
                         program’s statutory and regulatory requirements to monitor and provide
                         technical assistance to housing authorities,
                     •   whether PHMAP scores have increased and how HUD uses the program to
                         inform HUD’s Secretary and the Congress about the performance of
                         housing authorities, and
                     •   whether PHMAP scores are consistently accurate and can be considered a
                         generally accepted measure of good property management.

                         We developed information from several different sources to address
                         questions concerning the usefulness of PHMAP to HUD and other uses for
                         which PHMAP may not be appropriate. To determine PHMAP’s usefulness to
                         HUD, we interviewed officials and collected information on technical
                         assistance activities at both the Department’s headquarters and field
                         offices. At HUD’s headquarters, we analyzed a variety of documents
                         pertaining to PHMAP and discussed the program’s use as a basis for



                         Page 18                                            GAO/RCED-97-27 Public Housing
Chapter 1
Introduction




technical assistance with the Offices of the Deputy Assistant Secretaries
under HUD’s Assistant Secretary for Public and Indian Housing.

At HUD’s field offices, our approach was twofold. First, we surveyed them
via fax questionnaire to obtain data on the use of PHMAP, such as the
number of confirmatory reviews each field office performs and how useful
such program tools as improvement plans have been.9 This data reflect
responses from all of HUD’s public housing field offices.10 Second, we
visited five HUD field offices to review their use of PHMAP in more depth and
to supplement the information we had gathered in our survey.11 We
judgmentally selected the five field offices because of their geographic
distribution, variations in the number of HUD staff in each office as well as
the number of PHAs each oversees, and variations in average PHMAP scores
for the PHAs reporting to those offices.

To provide information on PHAs’ PHMAP scores, we relied on existing data
from HUD sources, including HUD’s primary public housing database, the
System for Management Information Retrieval-Public Housing (SMIRPH).
From this database, we extracted the module containing housing
authorities’ PHMAP data, including the PHMAP scores and individual indicator
grades. Our analysis covers federal fiscal years 1992 through 1995 because
the first fiscal year in which the rules governing PHMAP took effect was
1992 and the most recent year for which all PHMAP scores were complete at
the time of our review was 1995.

We did not systematically verify the accuracy of HUD’s data or conduct a
reliability assessment of HUD’s database. In performing our analysis we
found erroneous and incomplete information for a few PHAs, ranging from
1 to 3 percent of the total. We confirmed this with HUD officials, who
attributed the errors to mistakes in data input or the field office’s having
entered incomplete scores. However, because we used these data in
context with additional evidence we obtained directly from HUD’s field
offices and we did not focus on the scores of specific PHAs or small groups


9
 HUD headquarters does not collect or centrally maintain the information for which we surveyed the
field offices.
10
  Although 51 of HUD’s 79 field offices have public housing oversight responsibilities, 2 of those 51
have assumed the workload of another field office due to attrition or temporary vacancies in the public
housing division. In these cases, the field office handling the workload provided a combined response
reflecting data for both office’s jurisdictions. As a result, our data is drawn from 49 responses, but it
does reflect the PHMAP-related activities of all of HUD’s public housing field offices.
11
 We visited HUD’s field offices in Birmingham, Alabama; Kansas City, Missouri; Minneapolis,
Minnesota; San Antonio, Texas; and San Francisco, California.



Page 19                                                             GAO/RCED-97-27 Public Housing
Chapter 1
Introduction




of PHAs, we believe our conclusions about overall trends in scores are
valid.

Throughout the course of our work, because the number of PHAs reporting
PHMAP scores is too great for us to visit a representative sample, we
consulted with several prominent groups representing the public housing
industry to discuss HUD’s uses for PHMAP as well as their perspectives on
the program’s ability to measure the performance of public housing
authorities. These groups include the Council of Large Public Housing
Authorities, the National Association of Housing and Redevelopment
Officials, and the Public Housing Authorities Directors Association.

We provided a draft of this report to HUD for review and comment. HUD’s
comments appear in appendix V and are addressed at the end of each
applicable chapter.

We performed our work from January through December 1996 in
accordance with generally accepted government auditing standards.




Page 20                                         GAO/RCED-97-27 Public Housing
Chapter 2

Although Field Offices Use PHMAP to
Identify Troubled PHAs, Compliance With
Statutory and Agency Follow-Up
Requirements Has Been Limited
                            HUD’s field offices use PHMAP scores for their primary intended purposes: as
                            a standard, objective means to identify troubled housing authorities; to
                            compare performance among PHAs; and to identify when, where, and how
                            to target HUD’s limited resources for technical assistance. However,
                            beyond identifying troubled authorities and what they need, the amounts
                            and kinds of technical assistance HUD provides varies because its field
                            offices interpret their responsibilities differently—some choose to be
                            actively involved while others adopt a hands-off approach. Furthermore,
                            HUD’s 1995 reorganization of its field offices adversely affected some
                            offices’ ability to provide technical assistance while others adapted to
                            changed expectations and resumed providing as much assistance as they
                            did before the reorganization.


                            As part of HUD’s oversight of public housing, the PHMAP score is an
HUD Uses PHMAP to           important tool for identifying troubled authorities so HUD can focus
Identify Troubled           technical assistance and monitoring on them. The most common types of
Housing Authorities,        technical assistance that HUD’s 49 public housing field offices provided all
                            PHAs were telephone consultations, training, and participation in
but Technical               conferences. However, we found differences in how field offices defined
Assistance Varies           their roles in providing PHAs technical assistance as well as some
                            innovations in how others provided that assistance. For example, some
                            field offices have encouraged high-performing PHAs to provide “peer
                            assistance” to lower performers. Many of the differences in assistance
                            were due to variations in field offices’ interpretations of their roles and the
                            impact of HUD’s 1995 reorganization of its field offices. HUD headquarters
                            officials believe that more training for all field staff and leadership from
                            field office managers would help achieve more quality and consistency
                            among field offices in providing technical assistance.


HUD Uses PHMAP to           Officials in 40 of HUD’s 49 field offices rated PHMAP as being of “utmost” or
Target PHAs for Technical   “major” importance in identifying which housing authorities need the most
Assistance                  technical assistance. According to field office staff, PHMAP provides
                            standard indicators to objectively measure an authority’s performance. In
                            addition, some staff said that because PHAs have a strong aversion to
                            failing performance scores and try to avoid failure, they are confident that
                            when PHAs report information that results in low scores or failing grades,
                            the data and the resulting scores are accurate. Because an accumulation of
                            low or failing scores results in a PHA’s being designated troubled, HUD staff
                            are confident that those PHAs PHMAP identifies as the worst-performing
                            housing authorities are accurately designated as troubled performers.



                            Page 21                                            GAO/RCED-97-27 Public Housing
                           Chapter 2
                           Although Field Offices Use PHMAP to
                           Identify Troubled PHAs, Compliance With
                           Statutory and Agency Follow-Up
                           Requirements Has Been Limited




                           Some field office staff also use declining PHMAP scores to provide an early
                           warning of management problems and to identify which PHAs could need
                           additional technical assistance. In addition, the staff use PHMAP’s 12
                           individual indicator grades to better focus their limited technical
                           assistance resources and thereby maximize the benefits PHAs receive from
                           HUD’s assistance. For example, one field office developed a package of
                           technical assistance for the “resident initiatives” indicator because many
                           PHAs failed this indicator. The package of assistance included sample
                           policies and procedures for operating resident programs. Another field
                           office developed assistance specifically for small housing authorities
                           because many of them were having trouble renting their units when they
                           became vacant (thus failing PHMAP’s unit turnaround indicator). Among
                           other things, that field office provided its small PHAs an extensive list of
                           suggestions on how and where to better market their units.

                           Most technical assistance from HUD’s field offices consisted of telephone
                           consultations, training sessions, and industry conferences. HUD also
                           provided assistance—although limited because of time constraints—at the
                           time of a PHMAP confirmatory review. During telephone consultations,
                           several offices we visited answered questions from housing authority staff
                           and helped the executive directors of new housing authorities better
                           understand public housing regulations and operations. Training sessions
                           covered these and other topics and provided more details than telephone
                           discussions. In addition, to increase the amount of personal contact they
                           have with housing authority staff and to provide technical assistance, field
                           office staff said they regularly participate in conferences hosted by public
                           housing industry associations.


Field Offices’             Field offices’ interpretations of their obligation to improve the
Interpretations of Their   performance of housing authorities influences the type of technical
Role and Their Recent      assistance they provide. For example, officials in one field office did not
                           believe that it was HUD’s role to manage PHAs’ operations. Instead, they
Reorganization Influence   believed that the role of their field office should be limited to providing
the Level and Types of     information on compliance with federal rules and regulations and to
Technical Assistance       suggesting solutions to management problems. This field office avoids
                           showing PHAs how to manage their developments because the staff believe
                           that they do not have sufficient expertise and that the housing authorities
                           would view this advice as intrusive.

                           In contrast, staff at other field offices that we visited believed they are
                           obligated to tell PHAs what must be done to correct management



                           Page 22                                            GAO/RCED-97-27 Public Housing
Chapter 2
Although Field Offices Use PHMAP to
Identify Troubled PHAs, Compliance With
Statutory and Agency Follow-Up
Requirements Has Been Limited




deficiencies because HUD is responsible for ensuring that PHAs use federal
funds efficiently and effectively to provide safe, decent housing. For
example, staff from one field office spent several days at a troubled
authority to help it set up proper tenant rent records and waiting lists.

In addition to differences in how they view their role to directly assist
PHAs, we found differences in the extent to which field offices use outside
resources to help their housing authorities. Some field offices told us that
to compensate for a shortage of resources from HUD, they help PHAs in
their jurisdiction by encouraging technical assistance from other PHAs
rather than providing it themselves. For example, some of the field offices
arranged for high-performing PHAs to provide peer assistance to authorities
with management problems. One field office persuaded staff from a
high-performing PHA to temporarily manage a small authority that
unexpectedly lost its executive director. Another field office recruited a
high-performing PHA to help another one develop a system for inspecting
its housing units.

In 1995, HUD reorganized the field offices and changed the responsibilities
of the staff who oversee and assist PHAs. Before the reorganization, most
field office staff were generalists and broadly understood federal housing
regulations and PHA operations. After the reorganization, however, the
responsibilities of individual field office staff became more specialized to
focus on the rules and regulations of specific public housing operations.1
This specialization confused some staff in field offices and housing
authorities as well as impaired the ability of some field offices to provide
technical assistance. For example, field office staff we visited said that
some specialists do not have the skills needed to do their jobs because
many of them did not have the work experience or requisite training for
the specialists’ positions; the staff also noted that HUD had not provided
sufficient training for the staff to understand the reorganization and their
new responsibilities. The staff also said that the reorganization was a
source of confusion for PHAs. Before the reorganization, a housing
authority could call one employee at HUD’s field office to answer all its
questions; afterward, a housing authority generally needed to call several
different staff at HUD’s field office to answer questions.

Adjusting to the reorganization differed across field offices. At one field
office, staff resisted the reorganization because they did not want to
become specialists and they recognized that technical assistance to the

1
 The five areas in which field office staff now specialize are finance and budget; marketing, leasing,
and management; facilities management; community relations and involvement; and organization,
management, and personnel.



Page 23                                                             GAO/RCED-97-27 Public Housing
                       Chapter 2
                       Although Field Offices Use PHMAP to
                       Identify Troubled PHAs, Compliance With
                       Statutory and Agency Follow-Up
                       Requirements Has Been Limited




                       PHAs suffered as a result. For example, the staff now disagree over who is
                       responsible for overseeing certain PHA operations. They also have resisted
                       working together to provide technical assistance and have not been
                       sharing PHMAP information to develop the best plan for correcting
                       management deficiencies. Other field offices we visited adapted to the
                       reorganization. Staff in these field offices worked cooperatively to build on
                       the skills of the experienced staff. For example, one field office continues
                       to assign each housing authority to only one staff member who provides or
                       coordinates all technical assistance to that authority. The responsible staff
                       member, however, belongs to a team of staff from all operational areas
                       who work together to solve each PHA’s problems.

                       Officials at HUD headquarters, including the Deputy Assistant Secretary for
                       Public and Assisted Housing Operations, acknowledged that some field
                       offices had difficulty adjusting to the reorganization. They stated that
                       although adequate training was crucial to the reorganization’s success,
                       some field offices either did not seek it or did not take the need for it
                       seriously, despite the availability of training funds for field staff. HUD
                       officials continue to emphasize the importance and availability of training
                       and expect field office management to assess the staff’s skills and
                       expertise and request the appropriate training. These officials believe that
                       because of limited staff resources, now and in the future, the
                       reorganization is the best way for field offices to provide effective
                       oversight and technical assistance to PHAs. Furthermore, they believe that
                       managers of the field offices must take a more active leadership role in
                       directing their staff to work together.


                       The act and HUD’s requirements for how field offices use PHMAP provide for
HUD’s Infrequent Use   several tools to guide improvements in a housing authority’s performance
of Some Oversight      and thereby raise its indicator grades and PHMAP score. These tools include
Tools May Not          the memorandums of agreement (MOA), improvement plans, confirmatory
                       reviews, and the annual independent audits. While such tools as MOAs and
Adequately Improve     improvement plans generally apply to PHAs designated as troubled or
the Performance of     failing specific indicators, a confirmatory review is mandatory for any PHA
                       coming off HUD’s troubled list and an independent audit is mandatory for
PHAs or Target         all PHAs. Nonetheless, we found that the compliance of field offices with
Technical Assistance   statutory requirements and HUD’s guidance for using these tools has been
                       inadequate and infrequent.

                       Furthermore, HUD has not determined whether these statutory or agency
                       requirements are effective, adequately improve housing authority



                       Page 24                                          GAO/RCED-97-27 Public Housing
                             Chapter 2
                             Although Field Offices Use PHMAP to
                             Identify Troubled PHAs, Compliance With
                             Statutory and Agency Follow-Up
                             Requirements Has Been Limited




                             performance, or help the field offices better target limited technical
                             assistance resources. As a result, HUD has little information to determine
                             which of these tools best improve a PHA’s performance and which tools its
                             field offices can use most effectively to offset their declining resources.


Field Offices Make Limited   Over 90 percent of the field offices we surveyed reported that on-site visits
Use of Oversight Tools       to the housing authorities were one of the most effective means to ensure
                             compliance with PHMAP requirements and provide technical assistance.
                             Officials at one field office responded that PHAs under its jurisdiction
                             believed that on-site visits from HUD staff to provide technical assistance
                             were essential to maintaining effective operations. Yet, most field office
                             staff we visited made fewer personal visits to housing authorities than they
                             felt were necessary because of limited staff resources and travel funds.
                             Field office staff told us, for example, that their workload has increased
                             because their offices have been unable to replace staff who have left the
                             agency. With less time available for on-site visits, direct monitoring of the
                             PHAs’ performance has occurred less frequently. In addition, some field
                             office staff said that they could rarely justify to their management using
                             limited staff and travel resources to visit a PHA that is more than a 1-day
                             trip from the office unless that authority’s PHMAP score was below 60.


Memorandums of               Although HUD is required by law to enter into MOAs with troubled housing
Agreement                    authorities to improve management performance, few field offices have
                             done so. Figure 2.1 shows that the percentage of troubled PHAs operating
                             under an MOA has been decreasing since 1992. Furthermore, in fiscal year
                             1995, only 3 of HUD’s 32 field offices that had troubled PHAs were fully in
                             compliance with the requirement to enter into an MOA with each troubled
                             authority.2




                             2
                              For fiscal year 1995, the 49 field offices responded that a total of 71 housing authorities were troubled,
                             but only 13 had MOAs. However, HUD’s PHMAP database reports 150 housing authorities as troubled
                             in fiscal year 1995. HUD headquarters officials told us that our survey information was probably more
                             accurate than its database.



                             Page 25                                                              GAO/RCED-97-27 Public Housing
                                     Chapter 2
                                     Although Field Offices Use PHMAP to
                                     Identify Troubled PHAs, Compliance With
                                     Statutory and Agency Follow-Up
                                     Requirements Has Been Limited




Figure 2.1: Percentage of Troubled
PHAs Operating Under a MOA, Fiscal   100   Percentage of troubled housing authorities
Years 1992-95
                                      90

                                      80

                                      70

                                      60

                                      50

                                      40

                                           29
                                      30
                                                     23
                                                               20
                                      20                                 18


                                      10

                                       0

                                             1992      1993      1994     1995
                                             Fiscal year



                                     Source: HUD field offices’ responses to GAO’s questionnaire.




                                     The primary reason HUD’s field offices told us that they did not enter into
                                     these required agreements with troubled housing authorities is that the
                                     PHAs had already corrected or were in the process of correcting their
                                     management deficiencies. However, HUD headquarters officials told us
                                     they did not accept this as a valid reason for not meeting the requirement
                                     and questioned how the field offices could be sure the housing authorities
                                     were no longer troubled.


Improvement Plans                    When a PHA fails any of PHMAP’s 12 performance indicators, HUD requires
                                     the responsible HUD field office to obtain a plan from that PHA for
                                     improving its performance and to track its progress against the plan.
                                     However, we found that nearly a third—31 percent—of HUD’s field offices
                                     had not ensured that local housing authorities had developed these plans.
                                     We also found examples of PHAs’ plans lacking specific strategies and time
                                     frames for correcting management deficiencies. For example, one PHA’s
                                     plan for a failing “rents uncollected” indicator simply stated that the
                                     housing authority would start collecting rent. Although field office staff
                                     acknowledged that the PHA also needed to update its standard tenant lease



                                     Page 26                                                        GAO/RCED-97-27 Public Housing
                            Chapter 2
                            Although Field Offices Use PHMAP to
                            Identify Troubled PHAs, Compliance With
                            Statutory and Agency Follow-Up
                            Requirements Has Been Limited




                            and develop a rent collection policy to improve this indicator grade, they
                            said that they had not yet had the time to contact the PHA to revise its plan.

                            HUD  requires its field offices to monitor the progress of housing authorities
                            in implementing improvement plans to ensure PHAs meet the quarterly and
                            annual performance targets in their plans. However, four of the five field
                            offices we visited told us they do not follow up with the PHAs to determine
                            the status of improvement plans or whether the plans had corrected the
                            management deficiencies. Field office staff said that they did not have time
                            to track the effectiveness of the plans because their workloads have been
                            increasing due to decreasing numbers of staff.

                            HUD  headquarters officials confirmed that systematic tracking of the field
                            offices’ success in obtaining improvement plans or executing MOAs has not
                            been done. They emphasized that responsibility for implementing PHMAP
                            rests with the field offices and said that limited efforts were underway to
                            ensure field offices do more to use these tools and measure their
                            effectiveness. However, they could not tell us whether troubled PHAs
                            without MOAs had improved their scores and left the troubled list without
                            such oversight, nor could they tell us whether improvement plans are
                            instrumental in improving indicator scores.

Field Offices Confirm Few   When a troubled housing authority receives a new PHMAP score that is high
PHMAP Scores                enough to remove that designation, HUD requires that the field office
                            confirm the score’s accuracy by verifying that the PHA’s improvements
                            have been effective before removing the troubled designation. However,
                            we found most field offices are not meeting this requirement. In 1995, for
                            example, HUD’s field offices confirmed less than 30 percent of the scores
                            that should have been confirmed. HUD officials acknowledged that the
                            infrequency of confirmatory reviews by its field offices hampers the
                            program’s credibility and integrity. Because it has done so few
                            confirmatory reviews, HUD cannot say that most scores are accurate, nor
                            can it say that most troubled PHAs that raised their scores above 60 really
                            are no longer troubled. The HUD Inspector General (IG) recently noted that
                            without more confirmatory reviews, the self-reporting nature of PHMAP
                            creates a temptation for PHAs to manipulate data to raise their scores.3 In
                            fiscal year 1995, 24 of the 49 field offices had housing authorities with
                            PHMAP scores high enough to remove them from HUD’s troubled list, but
                            only 11 of the 24 field offices performed all or some of the required



                            3
                            Limited Review of the Public Housing Management Assessment Program (Audit Related
                            Memorandum No. 96-PH-101-0801), July 1996, HUD Office of Inspector General.



                            Page 27                                                     GAO/RCED-97-27 Public Housing
                                     Chapter 2
                                     Although Field Offices Use PHMAP to
                                     Identify Troubled PHAs, Compliance With
                                     Statutory and Agency Follow-Up
                                     Requirements Has Been Limited




                                     confirmatory reviews.4 The remaining 13 offices performed none of the
                                     required confirmatory reviews. Nonetheless, some of these same 13 field
                                     offices performed discretionary confirmatory reviews of other housing
                                     authorities that had not been classified as troubled. In one case, a field
                                     office had just one housing authority whose new PHMAP score was high
                                     enough to remove its troubled designation. Although the field office did
                                     not perform a confirmatory review for that authority until the next fiscal
                                     year, it did complete nine confirmatory reviews of standard- or
                                     high-performing housing authorities. HUD headquarters officials told us
                                     that although they encourage the field offices to do as many additional,
                                     discretionary confirmatory reviews as possible, they expect field offices to
                                     complete the mandatory reviews first. They also told us that limited
                                     resources kept them from monitoring the performance of field offices on
                                     these reviews.

                                     In addition to the field offices’ lack of compliance with HUD’s requirement
                                     for performing confirmatory reviews, few offices are performing
                                     discretionary confirmatory reviews. Over the life of the program, HUD has
                                     confirmed 6.7 percent of all PHMAP scores. Table 2.1 shows that since the
                                     program began in 1992, HUD has confirmed no more than 8 percent of all
                                     PHMAP scores in any given year (see table 2.1).


Table 2.1: PHAs Receiving a
Confirmatory Review From a HUD                            1992                1993                1994                1995
Field Office, Fiscal Years 1992-95   Number of PHAs       2,372               3,071               3,071               3,077
                                     submitting
                                     PHMAP
                                     certifications
                                     Number of            137                 225                 241                 200
                                     confirmatory
                                     reviews
                                     completed
                                     Percentage of    5%                      7%                  8%                  6%
                                     PHAs receiving a
                                     confirmatory
                                     review




                                     4
                                     In fiscal year 1995, a total of 58 PHAs reported new PHMAP scores that would have taken them off
                                     HUD’s troubled list. HUD’s field offices confirmed 15 (26 percent) of these 58 PHMAP scores.



                                     Page 28                                                         GAO/RCED-97-27 Public Housing
                                     Chapter 2
                                     Although Field Offices Use PHMAP to
                                     Identify Troubled PHAs, Compliance With
                                     Statutory and Agency Follow-Up
                                     Requirements Has Been Limited




                                     To expand on fiscal year 1995 data, nine field offices performed no
                                     confirmatory reviews, over two thirds performed five or fewer, and 4
                                     offices performed 10 or more confirmatory reviews (see fig. 2.2).5


Figure 2.2: Number of Confirmatory
Reviews Performed by HUD Field
Offices, Fiscal Year 1995                                                                    8%
                                                                                             >10 Reviews (4 Field offices)

                                                                                             0 Reviews (9 Field offices)




                                                   • 18%


                                               •
                                                                       53% •                 1-5 Reviews (26 Field offices)

                                               20%
                                                 •




                                                                                             6-10 Reviews (10 Field offices)



                                     Source: HUD field offices’ responses to GAO’s questionnaire.




                                     Recognizing that PHMAP scores may not be as accurate as they could or
                                     should be to give the program integrity and credibility, HUD has added new
                                     requirements and begun initiatives to improve the accuracy of the scores
                                     and strengthen the program. HUD currently requires its field offices to
                                     confirm the PHMAP scores of housing authorities whose scores have risen
                                     to 60 or above, thereby removing them from the troubled list. Recently,
                                     HUD formed a team of “expert” field office staff to develop review
                                     guidelines and to perform confirmatory reviews at selected housing

                                     5
                                      HUD’s Oklahoma City Field Office reported having one staff member and performing no confirmatory
                                     reviews in fiscal year 1995. We did not include that office’s response in our analysis, however, because
                                     its responsibilities for overseeing public housing had been transferred to HUD’s Ft. Worth office after
                                     the bombing of the Oklahoma City Federal Building.



                                     Page 29                                                            GAO/RCED-97-27 Public Housing
                               Chapter 2
                               Although Field Offices Use PHMAP to
                               Identify Troubled PHAs, Compliance With
                               Statutory and Agency Follow-Up
                               Requirements Has Been Limited




                               authorities whose new PHMAP scores meet HUD’s criteria for a mandatory
                               confirmatory review. HUD officials expect this team to perform as many as
                               12 confirmatory reviews in 1 year, during which they will focus primarily
                               on large, high-risk housing authorities.

Field Offices Are Not Using    In May 1995, HUD expanded the scope of the mandatory annual financial
Independent Audits to Verify   audits of PHAs to require that auditors review the records underlying a
Data Provided by PHAs          PHA’s self-reported PHMAP data.6 HUD expects the financial audits to verify
                               that the PHAs’ data are accurate and complete and that the PHAs have
                               adequate documentation to support their submissions. HUD adopted this
                               requirement because the field offices do not have sufficient resources to
                               confirm each PHA’s score every year. Moreover, HUD officials told us that
                               further departmental downsizing will limit its field offices’ ability to
                               provide meaningful technical assistance, including confirmatory reviews.
                               As a result, HUD expects that the PHMAP review in the annual audit can help
                               ensure the integrity of housing authorities’ PHMAP data and should be a
                               valuable tool for aiding the field offices to identify those housing
                               authorities most needing technical assistance.

                               HUD  does not consider the auditors’ analysis to be a confirmatory review
                               because the auditors do not verify the information HUD maintains in its
                               information system. Furthermore, even though the auditors certify that a
                               housing authority has documentation to support the data it submitted to
                               HUD, they do not verify that some of the activities reflected in that data
                               were actually performed by that authority. For example, while the auditors
                               verify that a PHA has data indicating it has met the requirements for the
                               indicator on conducting annual inspections of all of its housing units and
                               major systems (e.g., heating, plumbing, and electrical), the auditors do not
                               verify that those inspections actually took place.

                               Although the independent audit requirement has been in place since
                               May 1995, few of the staff in the five field offices we visited were aware of
                               it. Before field offices authorize payment for an annual audit, HUD
                               headquarters officials said that they expect field offices to review the
                               audits for quality and completeness and verify that the audits addressed all
                               appropriate areas of the PHAs’ operations, including the PHMAP. However,
                               field office staff said that they had not seen an audit of a housing authority




                               6
                                Annually, HUD requires and pays for a single financial and compliance audit of every public housing
                               authority.



                               Page 30                                                           GAO/RCED-97-27 Public Housing
                  Chapter 2
                  Although Field Offices Use PHMAP to
                  Identify Troubled PHAs, Compliance With
                  Statutory and Agency Follow-Up
                  Requirements Has Been Limited




                  that tested the reliability of its PHMAP submission.7 HUD also expects the
                  field offices to consider significant audit findings in deciding which PHAs
                  need additional oversight or assistance. HUD officials acknowledged,
                  however, that the independent auditors may need training to better
                  understand HUD’s expectations of them, regulations, and PHMAP system as
                  well as the operations of PHAs. Similarly, these officials noted that staff in
                  HUD’s field offices need training and guidance in how to better use the
                  annual independent audit.


                  One of the key challenges HUD faces in the coming years is effectively
Conclusions       downsizing the Department while maintaining the needed level of
                  oversight at public housing authorities. However, HUD is currently not
                  maintaining a consistent, minimally acceptable level of oversight at all
                  housing authorities because of the variance in how field offices interpret
                  their roles to provide that oversight as well as their lack of systematic
                  compliance with follow-up requirements. Furthermore, because field
                  offices are not making enough use of the independent audits’ verification
                  of PHMAP data to target their technical assistance, HUD is not using the
                  resources it has to effectively determine which housing authorities’ scores
                  are most likely to be inaccurate. As a result, HUD is not ensuring that the
                  housing authorities most in need of oversight and assistance are receiving
                  it and thereby improving their performance. Continued departmental
                  downsizing likely will cause HUD to leverage its existing resources to
                  achieve a minimally acceptable level of oversight. This oversight is needed
                  for HUD to be reasonably confident that all housing authorities are using
                  federal funds appropriately, managing and maintaining their developments
                  properly, and reporting accurately their performance information.


                  To make better use of the limited resources it has to devote to the
Recommendation    oversight of public housing, we recommend that HUD provide guidance to
                  its field offices that clearly (1) articulates their minimally acceptable roles
                  regarding oversight and assistance to housing authorities and
                  (2) emphasizes the importance of using the results of the independent
                  audits to better target HUD’s limited technical assistance resources.


                  HUD agreed with our findings regarding oversight of public housing
Agency Comments   authorities and stated that it has begun taking steps to address this

                  7
                   HUD’s IG evaluated some housing authorities’ annual independent audits that included a PHMAP
                  review. The IG found that the auditors had problems confirming the reliability of the PHMAP
                  submissions because the PHAs lacked documentation to support the self-reported indicator grades.



                  Page 31                                                         GAO/RCED-97-27 Public Housing
Chapter 2
Although Field Offices Use PHMAP to
Identify Troubled PHAs, Compliance With
Statutory and Agency Follow-Up
Requirements Has Been Limited




recommendation. These steps include a wide variety of training and other
activities to (1) explain the revisions HUD is making to PHMAP;
(2) reemphasize the need for and importance of statutory and agency
follow-up requirements, such as memorandums of agreement,
improvement plans, and confirmatory reviews; and (3) update HUD’s
guidance to its field offices regarding their PHMAP and other oversight
responsibilities.




Page 32                                        GAO/RCED-97-27 Public Housing
Chapter 3

Although PHMAP Scores Have Risen, HUD
Recognizes That Flaws in the Program’s
Database Limit Its Use
                       According to a HUD database of PHMAP scores, average PHMAP scores have
                       increased over the life of the program from an average of 83 in 1992 to 86
                       in 1995 (the last year of complete data). The number of high-performing
                       housing authorities increased, with more than half of all authorities
                       designated high performers in 1995, and the number of troubled
                       authorities decreased. However, the smallest housing authorities—those
                       with fewer than 100 units—now make up a greater proportion of those
                       designated troubled than when the program began. During our analysis of
                       this database, we found omissions of key data, such as the number of units
                       under a PHA’s management and its performance designation. We also found
                       inconsistencies between PHMAP scores and the assigned performance
                       designations. Notwithstanding these weaknesses, the database represents
                       the most complete data available on PHA performance over time.


                       Nationwide, average PHMAP scores generally increased over the 4 years of
Most PHMAP Scores      the program for which we analyzed data. By 1995, over half of all public
Are Increasing and     housing authorities were high performers. Subsequent analysis showed
Fewer Housing          little regional variation in how well they scored on PHMAP. While the overall
                       increases in PHMAP scores held true for all sizes of PHAs, the largest ones
Authorities Are        had scores consistently lower than the national average. With average
Troubled               scores increasing, the number of PHAs with scores low enough for HUD to
                       designate them as troubled also decreased. The number of troubled
                       authorities reached 83 in 1995, with half of that total consisting of the
                       smallest housing authorities (those managing fewer than 100 units).


Average PHMAP Scores   The average PHMAP score for all housing authorities rose from about 83 in
Increased              1992 to 86 in 1995. This increase held true for PHAs of all sizes, although
                       large PHAs—those with more than 1,250 units—consistently scored lower
                       than the national average (see table 3.1). In fiscal year 1995, 151 large PHAs
                       accounted for approximately 5 percent of all PHAs reporting PHMAP scores,
                       but they operated nearly 60 percent of all public housing units.
                       Consequently, while more PHAs had higher scores, more units were under
                       the control of PHAs with somewhat lower scores.




                       Page 33                                            GAO/RCED-97-27 Public Housing
                                        Chapter 3
                                        Although PHMAP Scores Have Risen, HUD
                                        Recognizes That Flaws in the Program’s
                                        Database Limit Its Use




Table 3.1: Average PHMAP Score by PHA Size Category, Fiscal Years 1992-95
PHA size
category                  1992                     1993                       1994                                        1995
(number of         Number of     PHMAP     Number of        PHMAP      Number of                    PHMAP        Number of          PHMAP
units)a               PHAs         score        PHAs          score       PHAs                       score          PHAs             score
No size datab             4           54                  4              43                3                41             7               31
1-99                  1,453           83             1,471               84           1,481                 87        1,488                87
100-499               1,241           83             1,262               85           1,266                 88        1,269                87
500-1,249               243           83               242               84             243                 86          242                86
1,250 or more           149           78               151               80             151                 81          151                83
All sizes             3,090           83             3,130               84           3,144                 87        3,157                86
                                        a
                                         All size categories were calculated on the number of managed units for fiscal year 1995. HUD
                                        does not maintain information on the number of units managed in previous years. To the extent
                                        that specific PHAs reduced or increased the number of units under their jurisdiction in previous
                                        years, the current size categories may not represent previous years.
                                        b
                                            The database did not contain size information for these PHAs.

                                        Source: GAO’s analysis of data from HUD’s System for Management Information Retrieval-Public
                                        Housing (SMIRPH) database.



                                        Appendix I provides average PHMAP scores for PHAs for all of HUD’s field
                                        offices for fiscal years 1992 through 1995.


The Majority of PHAs Were               By fiscal year 1995, more than half—about 57 percent—of all public
High Performers                         housing authorities were designated as high performers. As shown in table
                                        3.2, the number of high performing authorities grew each year, rising from
                                        1,033 (33 percent) in 1992 to 1,791 (57 percent) in 1995. Also, by 1995,
                                        nearly 50 percent of all public housing units were under the management
                                        of high-performing authorities.




                                        Page 34                                                             GAO/RCED-97-27 Public Housing
                                     Chapter 3
                                     Although PHMAP Scores Have Risen, HUD
                                     Recognizes That Flaws in the Program’s
                                     Database Limit Its Use




Table 3.2: Number of PHAs by PHMAP
Performance Category, Fiscal Years   Performance
                                     category                Units (in                          Number of PHAs
1992-95
                                     (score)              thousands)a              1992            1993             1994             1995
                                     Troubled (<60)                 214             130              118             101                83
                                     Standard                       443           1,927            1,719           1,358            1,216
                                     (60-<90)
                                     High (90-100)                  656           1,033            1,293           1,685            1,791
                                     All                          1,313           3,090            3,130           3,144            3,090
                                     performance
                                     categories
                                     a
                                         HUD’s SMIRPH database contains the number of units for only fiscal year 1995.

                                     Source: GAO’s analysis of data from HUD’s SMIRPH database.




Little Variation Among               Our analysis showed little regional variation in PHMAP scores. The regional
Regions                              differences we found were slightly greater than those associated with the
                                     size of housing authorities, but no region was significantly below the
                                     national average. Likewise, there was little variation among the regions in
                                     the percentage of troubled PHAs under their jurisdiction. For example, in
                                     fiscal year 1995, 5 percent of all PHAs nationwide were troubled, but within
                                     the 10 regions we analyzed, the percentage of troubled housing authorities
                                     ranged from 2 to 9 percent.

                                     Appendixes I-IV provide detailed information on average PHMAP scores as
                                     well as the number of troubled, standard- and high-performing PHAs,
                                     respectively, for each HUD field office.


PHAs Consistently Failed             Despite some improvement in overall scores, some indicators were more
Some Indicators                      problematic for PHAs than others. As shown in table 3.3, with the exception
                                     of 1 year, PHAs consistently had the most difficulty with the energy
                                     consumption indicator—which had the highest failure rate for 1992, 1994,
                                     and 1995.1 Similarly, the indicators for unit turnaround, tenants accounts
                                     receivable, and operating expenses proved troublesome, with 10 percent
                                     or more of all PHAs failing them in 1995.



                                     1
                                      This indicator measures the annual increase in the housing authority’s energy consumption. Housing
                                     authorities with no increase from year to year receive an A for the indicator; those whose consumption
                                     increases receive lower grades. In commenting on this report, HUD officials noted that in some cases
                                     failing this indicator was related less to PHA performance than to such conditions as regional weather
                                     variations or an inappropriate baseline to measure this indicator.



                                     Page 35                                                           GAO/RCED-97-27 Public Housing
                                   Chapter 3
                                   Although PHMAP Scores Have Risen, HUD
                                   Recognizes That Flaws in the Program’s
                                   Database Limit Its Use




Table 3.3: Failed Indicators by
Percentage of PHAs, Fiscal Years                                          Percentage of PHAs that failed
1992-95                            PHMAP indicator                1992               1993              1994             1995
                                   1. Vacancy                       5.5                4.2               3.6              2.9
                                   number and
                                   percentage
                                   2. Modernization                 1.9                1.4               1.4              1.5
                                   3. Rents                         4.1                3.1               3.0              2.2
                                   uncollected
                                   4. Energy                       18.0               12.4              14.3            15.3
                                   consumption
                                   5. Unit turnaround              16.4               13.6              12.9            11.2
                                   6. Outstanding                   3.6                2.1               2.4              1.7
                                   workorders
                                   7. Annual                        2.2                2.3               2.1              1.9
                                   inspection and
                                   condition of units
                                   and systems
                                   8. Tenants                      16.9               15.2              14.2            12.8
                                   accounts
                                   receivable
                                   9. Operating                     6.3                5.8               4.8              7.4
                                   reserves
                                   10. Routine                     11.2               10.8              10.1            10.0
                                   operating
                                   expenses
                                   11. Resident                    13.4               26.7              11.0              5.9
                                   initiatives
                                   12. Development                  8.5                6.2               6.2              3.0

                                   Source: GAO’s analysis of data from HUD’s SMIRPH database.


                                   A HUD official explained that the high failure rate in 1993 for the indicator
                                   measuring resident initiatives occurred because the PHAs were not paying
                                   attention to this indicator. In 1992, all PHAs received an automatic “C” for
                                   this indicator because HUD had not provided enough information on the
                                   requirements for grades “A” through “F” until after the assessment period
                                   started. This official said that many PHAs assumed they would receive an
                                   automatic “C” the next year as well, even though HUD had stated in 1992
                                   that the automatic grade was a one-time occurrence. This official added
                                   that most field offices followed up by providing technical assistance to the
                                   PHAs with failing grades and were able to resolve the problems in the
                                   following year. This appears to be supported by the decline of the failure
                                   rate over the following 2 years to less than 6 percent in 1995.



                                   Page 36                                                      GAO/RCED-97-27 Public Housing
                                      Chapter 3
                                      Although PHMAP Scores Have Risen, HUD
                                      Recognizes That Flaws in the Program’s
                                      Database Limit Its Use




Smaller PHAs Were More                While the total number of troubled housing authorities declined—130 were
Likely to Be Troubled                 troubled in 1992 compared to 83 in 1995—more of those PHAs were
                                      concentrated among the smallest housing authorities than when the
                                      program began. The percentage of troubled PHAs that were
                                      small—managing fewer than 100 units each—grew from 32 percent of all
                                      troubled authorities in 1992 to 49 percent in 1995 (see fig. 3.1).


Figure 3.1: Number of Troubled PHAs
by Size, Fiscal Years 1992-95         70   Number of troubled PHAs
                                      65
                                      60
                                      55
                                      50
                                      45
                                      40
                                      35
                                      30
                                      25
                                      20
                                      15
                                      10
                                       5
                                       0

                                             1992                 1993               1994                1995



                                                     1-99 units

                                                     100-499 units

                                                     500-1,249 units

                                                     1,250 or more units



                                      Note: For each fiscal year, the figures exclude six or fewer PHAs for which there was no
                                      information on size in HUD’s database.


                                      Source: HUD’s SMIRPH database.




                                      We found missing, inaccurate, and inconsistent data in HUD’s SMIRPH
HUD Recognizes                        database, the primary database for storing PHMAP scores. A HUD official
Database Flaws and                    attributed these problems to data input problems at the field offices.
Plans Corrections                     Although HUD headquarters makes regular, periodic use of this database, it



                                      Page 37                                                          GAO/RCED-97-27 Public Housing
Chapter 3
Although PHMAP Scores Have Risen, HUD
Recognizes That Flaws in the Program’s
Database Limit Its Use




must also manually verify much of the information before providing it to
HUD’s Secretary, Members of Congress, and others. HUD’s General Deputy
Assistant Secretary for Public and Indian Housing acknowledged that the
SMIRPH database, as currently implemented, does not produce a complete,
accurate list of troubled PHAs and that HUD is in the process of making it
more reliable and useful.

We found that the number of troubled authorities (150) for fiscal year 1995
that we derived from the database was inaccurate when we compared it to
the number reported (83) as of December 20, 1995, by HUD’s Management
Assessment Coordinator. We also found performance designations that
were inconsistent with PHMAP scores. In 1995, for the 150 PHAs we found to
be troubled, HUD had designated 42 as high performers, 7 as standard, and
51 had no designation. Among high-performing PHAs in 1995, of the 1,791
PHAs that we found that had PHMAP scores of 90 or higher, HUD had
designated one as troubled, 43 as standard, and 325 had no performance
designation. We also found some omissions in the database. Data, such as
the number of units and performance designations, had not been entered
for all PHAs. For example, we found that the database did not have size
information on 18 PHAs from fiscal years 1992 through 1995. We also found
that no designations had been entered for 132 PHAs with scores less than
60 and 1,037 PHAs with scores 90 or higher.

HUD’s Management Assessment Coordinator stated that these problems
with missing, inaccurate, and inconsistent data occurred because field
offices either (1) did not enter the information at all or (2) entered it
incorrectly. These instances of inconsistent or missing data suggest that
basic system safeguards do not exist to prevent field offices from making
these data entry errors or omitting essential PHMAP data.

While HUD officials who oversee PHMAP and the Department’s field offices
acknowledged problems with the database, they added that the program’s
redesign includes changes that will address the problems with data
accuracy and reliability. HUD officials told us they plan to change
procedures for entering information on PHAs into the database to allow
field offices to update PHA data on a real-time basis and to make immediate
corrections when they find errors or omissions. These procedural changes
will also enable HUD headquarters staff to access field office data directly
and allow ongoing reviews of the information for accuracy and
completeness. HUD officials also believe that the changes will increase
control over the information from the field offices and help ensure that the
information in the SMIRPH database is accurate.



Page 38                                          GAO/RCED-97-27 Public Housing
                  Chapter 3
                  Although PHMAP Scores Have Risen, HUD
                  Recognizes That Flaws in the Program’s
                  Database Limit Its Use




                  HUD  expressed concern that our draft report used data from the SMIRPH
Agency Comments   database that HUD had not verified for accuracy. HUD noted that it is
                  making changes to the database that will improve headquarters’ ability to
                  find and correct data errors that have been entered by staff at its field
                  offices.

                  To address HUD’s concern that we used inaccurate, unverified data from its
                  database to analyze PHMAP data on housing authorities’ scores by size and
                  region, we recalculated the number of troubled housing authorities by size
                  category for 1995 using data HUD verified with its field offices; we also
                  modified this report to reflect a more accurate and lower number of
                  troubled housing authorities in 1995. Recalculating the number of troubled
                  authorities by size did not change our conclusion that a greater proportion
                  of the authorities that HUD verified as being troubled are those with fewer
                  than 100 units. In fact, while HUD’s database indicates that 44 percent of
                  troubled authorities in 1995 were small, HUD’s verified list of troubled
                  authorities indicates 49 percent were small. Furthermore, although HUD
                  officials told us that a manually-verified list of troubled authorities for
                  1992 was not available, they agreed with our conclusion that the smallest
                  housing authorities make up a greater proportion of troubled housing
                  authorities in 1995 than in 1992.

                  Because our draft report presented no analysis of data on a regional basis
                  (only data as drawn from HUD’s database) and because we draw no
                  conclusions in that regard in this report, we have retained appendixes I-IV,
                  which show average PHMAP scores and the number of troubled, standard,
                  and high-performers in HUD’s regions. Where HUD provided us with
                  manually verified data—particularly in appendix II showing troubled
                  authorities—we have modified the appendixes to reflect the more
                  accurate data.




                  Page 39                                          GAO/RCED-97-27 Public Housing
Chapter 4

The Questionable Accuracy of PHMAP’s
Scores and the Program’s Validity Limit Its
Usefulness
                        Our review and those of others indicate that PHMAP scores are often
                        inaccurate, imprecise, and must be changed when HUD verifies the data
                        that public housing authorities have submitted to support their scores.
                        Furthermore, professional property managers and others in the public
                        housing industry question whether PHMAP can capture all aspects of
                        management operations. Although HUD has taken some steps to help
                        ensure that future scores are more accurate than they have been over the
                        program’s first 4 years, these steps will be resource-intensive and do not
                        address all of the program’s limitations. In the past, both HUD and the
                        Congress have proposed additional uses for PHMAP, such as deregulating
                        and awarding bonuses to PHAs with high PHMAP scores. However, until
                        greater confidence exists that individual scores are accurate and HUD
                        brings greater validity to PHMAP as a comprehensive measure of
                        management operations, such additional uses for the program may not be
                        appropriate.


                        After performing on-site reviews of selected PHAs to confirm the accuracy
Accuracy of Scores      of their PHMAP scores, HUD’s field offices changed half of the scores. In
and Validity as a       commenting on this report, HUD indicated that most confirmatory reviews
Management              involved high-risk PHAs, whose PHMAP data have been most susceptible to
                        being found inaccurate. In similar reviews, HUD’s independent assessment
Assessment Tool         contractors as well as HUD’s IG found that many scores or grades for
Limit Uses for PHMAP    specific indicators were inaccurate. To better identify PHAs that need
                        oversight and technical assistance, HUD staff often supplement their
                        decision-making with other measures of management problems to get a
                        more complete picture of an authority’s performance. Professional
                        property managers and industry representatives agreed that more
                        information is needed than PHMAP provides to give a complete picture of
                        how well a PHA’s management is performing.


After Confirmatory      After performing confirmatory reviews of 200 PHAs in fiscal year 1995,
Reviews, PHMAP Scores   HUD’s 49 field offices changed 98 PHMAP scores (see table 4.1).

Change Significantly




                        Page 40                                         GAO/RCED-97-27 Public Housing
                                          Chapter 4
                                          The Questionable Accuracy of PHMAP’s
                                          Scores and the Program’s Validity Limit Its
                                          Usefulness




Table 4.1: Changes in PHMAP Scores
After HUD’s Field Offices Performed                                                   Number of housing
Confirmatory Reviews in Fiscal Year       Change in                                           authorities                  Average change
1995                                      PHMAP score                                (percentage of total)                       in points
                                          No change                                                   96 (49)                                   0
                                          Scores lowered                                              57 (29)                              –14
                                          Scores raised                                               41 (21)                                  +8
                                          Note: The field offices had not reported the final PHMAP scores for 6 out of the 200 confirmatory
                                          reviews because the reviews’ results were being finalized at the time the offices responded to our
                                          questionnaire. As a result, this table reflects results from 194 of the 200 confirmatory reviews
                                          performed in fiscal year 1995.



                                          In several cases, the changes HUD made to PHMAP scores also meant HUD
                                          would have to change the performance designation of those PHAs. For
                                          example, HUD

                                      •   lowered the scores of 14 PHAs enough to designate them as troubled,
                                      •   raised the scores of 4 troubled PHAs to 60 points or higher, and
                                      •   raised the scores of 10 standard-performing PHAs to 90 or higher.

                                          Both of HUD’s independent assessment contractors as well as HUD’s IG have
                                          reviewed PHMAP data to confirm the accuracy of PHAs’ scores. For example,
                                          in 1993, the IG confirmed the scores of 12 housing authorities. As a result
                                          of this review, the IG concluded that the PHMAP scores for 9 of the 12 PHAs
                                          should be lowered because 3 of them fell below 60, a score which should
                                          have warranted the troubled designation. In a second report on PHMAP, the
                                          IG reported that six of HUD’s field offices reduced over half of the scores
                                          they reviewed. Similarly, one of HUD’s independent assessment contractors
                                          reported that for the 30 assessments it has performed at troubled housing
                                          authorities, it found 21 indicator grades and/or PHMAP scores that were
                                          inaccurate. Over 50 percent of the contractor’s assessments resulted in
                                          lowering the indicator grades to an “F.” The contractor most often lowered
                                          the indicators used to measure outstanding workorders and annual
                                          inspections of housing conditions and systems.

                                          Several reasons explain why HUD and others changed so many PHMAP
                                          scores after performing a confirmatory review. Some field office staff said
                                          these scores changed because the PHAs did not understand all the
                                          requirements of PHMAP and therefore misreported their data. They also told
                                          us that PHMAP is particularly difficult for smaller housing authorities whose
                                          limited staff can find HUD’s paperwork requirements overwhelming.




                                          Page 41                                                          GAO/RCED-97-27 Public Housing
                               Chapter 4
                               The Questionable Accuracy of PHMAP’s
                               Scores and the Program’s Validity Limit Its
                               Usefulness




                               HUD  staff do not believe many PHAs intentionally try to deceive the
                               Department by reporting false PHMAP information. Instead, they, as well as
                               the contractor staff, said that the PHAs often have insufficient
                               documentation to support the data they must submit to the field offices or
                               do not understand how HUD wants them to report the information. For
                               example, while a PHA may report the average number of days their housing
                               units have been vacant, the PHA may not have the tenant files to document
                               when the previous tenants moved out and when the new tenants’ leases
                               took effect. Without supporting documentation or evidence of a system to
                               track unit turnaround, HUD assigns an “F” to this indicator. Similarly, a PHA
                               may be providing support programs for its residents, but fail to understand
                               that its board of commissioners must approve those programs to receive a
                               passing grade on PHMAP’s indicator for resident initiatives. Typically, when
                               HUD’s field office staff find examples, such as these, during a confirmatory
                               review, they use the correct data to recalculate the housing authority’s
                               grade for each of the affected indicators.


HUD and Industry               HUD’s field office staff did not use PHMAP alone to assess the management
Professionals Supplement       performance of its public housing authorities. Although they agreed that
                               PHMAP accurately identifies troubled authorities, several staff said that they
PHMAP With Additional
                               consider other factors besides PHMAP indicators to supplement their
Factors to Evaluate            decision-making for the other authorities they oversee. They said that
Management Performance         some PHAs with scores over 90 have management problems that the
                               program’s indicators do not measure. Other factors used by some HUD staff
                               to identify the potential for management problems at standard- and
                               high-performing authorities include

                           •   the failure of a PHA to implement consistent and effective operating
                               policies and procedures,
                           •   the frequency of changes in the executive leadership and the continued
                               interference into a PHA’s daily operations by its board of commissioners,
                           •   the number and the type of telephone calls received from a PHA’s residents
                               and staff, and
                           •   any adverse news stories about a PHA.

                               Staff at the five field offices we visited said that they believed some
                               housing authorities with high PHMAP scores were not operating their
                               housing programs efficiently or effectively. These field offices differed,
                               however, in how they treated those PHAs. Staff at two field offices told us
                               that although they use the scores to determine which PHAs need on-site




                               Page 42                                           GAO/RCED-97-27 Public Housing
                         Chapter 4
                         The Questionable Accuracy of PHMAP’s
                         Scores and the Program’s Validity Limit Its
                         Usefulness




                         reviews, they would not let a high score prevent them from visiting an
                         authority they believed had serious management problems.

                         The HUD IG also questioned whether or not PHMAP scores accurately
                         measure the management performance of public housing authorities. The
                         IG’s reviews of high- and standard-performing PHAs found instances of
                         fraud and program abuse. For example, the IG reported that the executive
                         director of a high-performing PHA had charged over $62,000 in ineligible
                         expenses, including excessive compensatory time, unsupported travel
                         costs, and health and insurance benefits for his divorced spouse. Another
                         PHA executive director falsified PHMAP data to obtain a high-performing
                         designation. After reviewing the operations of a standard-performing PHA,
                         the IG also cited numerous program abuses and mismanagement. The IG
                         concluded that although PHMAP could be a useful tool to assess PHAs, the
                         program was too unreliable for HUD to make oversight decisions.

                         Other public housing professionals—property managers and those
                         representing industry associations—agreed that more information is
                         needed than PHMAP provides to give a complete picture of how well a PHA is
                         managed. For example, they noted that PHMAP does not automatically
                         include an on-site observation and inspection of a PHA’s housing
                         developments. One association noted that while a PHA could improve its
                         PHMAP score by simply writing off more past due rents from former tenants
                         as uncollectible to improve its grade on the indicator for rents uncollected,
                         its PHMAP score would not measure how diligent an effort it had
                         undertaken to collect the rent. Another industry association official knew
                         of several examples of PHAs that were making good property management
                         decisions, such as choosing to perform deferred maintenance when a unit
                         became vacant rather than rent it immediately, that ironically led to lower
                         PHMAP scores. Citing a similar situation, HUD has agreed that occasionally
                         the best decision for a PHA is to take an action that yields a lower PHMAP
                         score, and that the score should not be the sole driving force influencing a
                         PHA’s decisions.



                         While HUD’s primary use of PHMAP has been to identify troubled housing
The Congress and         authorities and target technical assistance to them, the Congress and HUD
HUD Have Proposed        have proposed to use this program for other purposes. In 1994, the Senate
to Use PHMAP as a        Committee on Banking, Housing, and Urban Affairs proposed some
                         deregulation and additional flexibility for those authorities that had
Basis for Deregulation   achieved PHMAP scores of 90 or above. In addition, in its fiscal year 1997
and Funding Bonuses      budget request, HUD proposed to give high-performing PHAs bonuses based



                         Page 43                                          GAO/RCED-97-27 Public Housing
Chapter 4
The Questionable Accuracy of PHMAP’s
Scores and the Program’s Validity Limit Its
Usefulness




in part on their PHMAP scores. Because PHMAP scores do not always
measure the true management performance of the PHAs, the benefits of
these proposals need to be weighed against the possibility of granting
undeserved flexibility and awards.

To encourage individual PHAs to be more innovative, the Banking
Committee proposed limited deregulation and additional flexibility for
high-performing PHAs in two ways. First, it proposed permitting a PHA that
generates income over a certain level to exclude that income from
calculations of its need for a subsidy from HUD to operate and manage its
properties.1 At that time, each dollar of extra income that a PHA generated
reduced its subsidy by a dollar, thereby creating a disincentive to generate
additional income from sources other than rent. Second, the Committee
proposed to waive all but a few key regulations—such as
nondiscrimination, equal opportunity, and tenant income eligibility—so
high-performing PHAs could have more flexibility to bring innovative
solutions to local problems and achieve more efficient operations.

In its fiscal year 1997 budget request, HUD proposed to award $500 million
to high-performing PHAs as bonuses based, in part, on their PHMAP scores.
As we reported in our testimony in June 1996 and as we found in the
course of our work on this report, HUD does not confirm the scores of high
performers and generally accepts them.2 In our June 1996 testimony, we
recommended that the Congress consider not appropriating the bonus
funding until HUD develops adequate performance measures and
supporting information systems. The HUD appropriations bill which the
Congress approved and the President signed did not contain funding for
performance bonuses.

The three associations representing the public housing industry and the
professional property managers that we interviewed all opposed or had
strong reservations about using PHMAP scores for purposes other than
identifying troubled housing authorities and targeting technical assistance
to them. They also believed that other uses would be inappropriate
because of the limited number of confirmatory reviews the field offices
perform and the proportion of PHMAP scores that have been changed after a
review. Two of the associations did not believe that PHMAP scores


1
 Housing authorities receive operating subsidies from HUD each year to make up the difference
between the rent they are allowed to charge their tenants and the expected costs of operating their
developments.
2
 Housing and Urban Development: Comments on HUD’s FY 1997 Budget Request
(GAO/T-RCED-96-205, June 17, 1996).



Page 44                                                           GAO/RCED-97-27 Public Housing
                      Chapter 4
                      The Questionable Accuracy of PHMAP’s
                      Scores and the Program’s Validity Limit Its
                      Usefulness




                      adequately measured the management performance of housing authorities
                      because they thought some PHAs that received high scores did not provide
                      their residents with decent, safe housing. The professional property
                      management firm that independently verified some scores also agreed that
                      the usefulness of these scores is limited. Because this firm has
                      recommended lowering many scores after an independent assessment, the
                      firm lacks confidence in the scores’ accuracy and does not believe that the
                      program provides enough information about the management
                      performance of PHAs for HUD to make effective funding decisions.


                      In recent years, both the Congress and HUD have proposed additional uses
Conclusions           for PHMAP, such as bonuses to reward those housing authorities with the
                      highest scores. While PHMAP has provided a quantifiable means to assess
                      the management performance of housing authorities, the scores are not
                      sufficiently accurate for detailed comparisons of performance. Although
                      HUD is currently working to enhance the accuracy of these scores, they do
                      not yet provide a comprehensive, generally accepted way to assess the
                      performance of PHAs. To be useful for other purposes, not only would
                      these scores have to be more accurate, but the program would have to be
                      expanded to provide a more comprehensive measure of public housing
                      authorities’ management operations.

                      Because HUD does not frequently confirm most scores—confirmatory
                      reviews have focused on troubled PHAs—HUD does not know how many
                      authorities are not receiving the proper designation. When HUD does
                      confirm scores, it changes half of them—and more than half of these
                      changes result in HUD’s lowering the score. We found that when HUD lowers
                      a PHMAP score, it does so by an average of 14 points. If this average change
                      held true for housing authorities in general, then HUD may not be properly
                      designating as troubled those authorities currently scoring between 60 and
                      the low 70s whose scores should be lower. As a result, those authorities
                      are not receiving the oversight and technical assistance HUD should be
                      providing to improve their performance.


                      We recommend that until it establishes a cost-effective means to ensure
Recommendations       consistently accurate scores, HUD should

                  •   not consider additional uses for PHMAP, including using its scores as
                      criteria for funding bonuses, until it determines that PHMAP meets an




                      Page 45                                          GAO/RCED-97-27 Public Housing
                      Chapter 4
                      The Questionable Accuracy of PHMAP’s
                      Scores and the Program’s Validity Limit Its
                      Usefulness




                      acceptable level of accuracy and more comprehensively measures
                      property management performance and
                  •   require its field offices to confirm the PHMAP scores of housing authorities
                      with scores low enough that they are at risk of being designated troubled.


                      HUD  agreed with our findings and recommendations. When we met with
Agency Comments       HUD officials, including the General Deputy Assistant Secretary for Public
                      and Indian Housing, to discuss a draft of this report, they told us that the
                      Department is no longer considering additional uses for PHMAP, such as
                      using scores as criteria for funding bonuses. Even in the absence of using
                      PHMAP for such purposes, we believe that it is important that HUD works to
                      ensure scores are more consistently accurate and have, therefore, retained
                      this recommendation. HUD has begun taking steps to address our
                      recommendation that it confirm PHMAP scores of those housing authorities
                      that are at risk of being designated troubled but expressed concern that it
                      may not have sufficient resources to fully implement this
                      recommendation.

                      HUD  expressed three concerns relating to the information and conclusions
                      presented in this chapter of our report. HUD believed that this chapter
                      (1) assumes that PHMAP was intended to be an all-inclusive assessment
                      system for property management, (2) does not place PHMAP in a historical
                      perspective, and (3) reaches incorrect conclusions regarding the overall
                      reliability of PHMAP scores.

                      We do not believe that we characterize PHMAP’s purpose as being an
                      all-inclusive measure of property management. Our discussion of the
                      program does not state that this is the purpose of PHMAP. Rather, the report
                      discusses how the program’s limitations—including its intentional design
                      not to be a complete performance measure—affect its suitability for
                      additional purposes, such as those proposed in recent years by HUD and
                      the Congress. HUD agreed that there is a perception that PHMAP is an
                      all-encompassing system to assess the performance of PHAs and stated it is
                      taking steps to address this misperception. Seeking to clarify the
                      program’s purpose, HUD added language to its recently revised interim
                      PHMAP rule (published in the December 30, 1996, Federal Register), that the
                      program’s indicators reflect performance in only specific areas.

                      HUD correctly states that this report does not provide a historical
                      perspective of PHMAP by discussing previous HUD systems for assessing and
                      identifying troubled housing authorities. We believe that such information



                      Page 46                                           GAO/RCED-97-27 Public Housing
Chapter 4
The Questionable Accuracy of PHMAP’s
Scores and the Program’s Validity Limit Its
Usefulness




would not contribute substantially to our report’s three objectives to
evaluate HUD’s use of the current program, provide trends in PHMAP scores
from fiscal years 1992 through 1995, and discuss limitations in the
program’s design and implementation that affect its usefulness for
purposes other than identifying troubled housing authorities and targeting
assistance to them. Therefore, we have not added the historical
information HUD suggested to the report.

Finally, HUD is concerned that we have incorrectly reached conclusions
about the reliability of all PHMAP scores based on the results of
confirmatory reviews of high-risk authorities. HUD noted that the accuracy
of the scores of these PHAs does not necessarily represent the accuracy of
all PHMAP scores because the data provided by these PHAs are most
susceptible to being inaccurate. Our report did not reach a conclusion
about the reliability of all housing authorities’ scores because of the
changes that resulted from confirmatory reviews. This report discusses the
reliability of PHMAP scores for housing authorities whose scores are low
enough that they may be at risk of being designated troubled. We have
added language to the report to clarify this point.




Page 47                                         GAO/RCED-97-27 Public Housing
Appendix I

Average PHMAP Score by Geographic
Region, Fiscal Years 1992-95


                                  Average PHMAP score
Region                    1992         1993             1994                    1995
Great Plains               82.2        83.1             86.9                    82.9
Des Moines                 84.4        82.3             88.9                    92.1
Kansas City                81.7        81.5             86.9                    88.3
Omaha                      84.0        88.4             88.3                    90.4
St. Louis                  78.9        78.7             83.6                    53.1
Mid-Atlantic               80.1        81.3             83.1                    85.7
Baltimore                  81.8        84.3             82.8                    84.8
Charleston                 76.0        79.7             82.2                    86.1
Philadelphia               80.9        81.4             85.9                    87.4
Pittsburgh                 80.1        80.5             81.3                    83.2
Richmond                   84.5        84.7             84.2                    88.7
District of Columbia       72.5        71.7             73.9                    76.3
Midwest                    83.6        86.1             88.8                    88.9
Chicago                    75.4        77.2             82.9                    83.9
Cincinnati                 77.6        78.8             84.8                    87.1
Cleveland                  77.9        80.8             83.9                    85.5
Columbus                   80.7        85.1             86.3                    90.7
Detroit                    82.4        84.0             87.6                    87.9
Grand Rapids               85.5        86.3             91.2                    90.2
Indianapolis               82.6        87.3             89.6                    89.5
Milwaukee                  90.2        92.5             93.1                    91.5
Minneapolis                86.5        90.1             90.6                    90.7
Northwest/Alaska           89.9        92.9             92.9                    92.6
Anchorage                  77.8        86.1             94.9                    99.2
Portland                   90.6        93.9             94.1                    95.4
Seattle                    89.6        92.0             91.4                    89.4
New York/New Jersey        81.8        84.9             88.2                    89.7
Buffalo                    84.6        86.9             90.0                    91.2
New York                   79.6        85.0             83.4                    90.5
Newark                     80.9        83.7             88.9                    88.4
New England                83.4        85.3             89.3                    89.2
Boston                     83.6        82.9             88.0                    90.4
Hartford                   74.4        76.9             83.7                    78.3
Manchester                 91.9        94.1             95.1                    94.0
Providence                 79.9        87.3             89.9                    92.4
                                                                          (continued)




                       Page 48                          GAO/RCED-97-27 Public Housing
                      Appendix I
                      Average PHMAP Score by Geographic
                      Region, Fiscal Years 1992-95




                                               Average PHMAP score
Region                    1992                        1993                        1994                        1995
Pacific/Hawaii            85.4                        85.4                         86.9                        87.2
Honolulu                  81.5                        68.9                         73.8                        70.6
Los Angeles               89.1                        90.0                         90.4                        91.0
Phoenix                   85.4                        83.3                         88.2                        83.8
Sacramento                86.1                        81.9                         75.5                        85.5
San Francisco             82.9                        84.9                         87.4                        87.7
Rocky Mountains           87.8                        87.9                         86.8                        91.5
Denver                    87.8                        87.9                         86.8                        91.5
Southeast/Caribbean       82.9                        85.4                         87.9                        84.6
Atlanta                   81.7                        83.7                         85.5                        87.9
Birmingham                85.8                        87.3                         89.7                        90.8
Columbia                  84.2                        86.5                         91.7                        94.0
Greensboro                82.5                        85.6                         88.8                        89.9
Jackson                   82.6                        86.6                         86.4                        87.4
Jacksonville              79.6                        83.3                         86.0                        83.3
Louisville                86.9                        88.4                         91.0                        60.6
Knoxville                 85.4                        87.7                         91.9                        90.1
Nashville                 77.1                        82.4                         85.4                        85.3
Caribbean                 44.5                        36.7                         48.2                        32.0
Southwest                 80.3                        80.8                         84.0                        85.7
Albuquerque               74.8                        70.4                         72.7                        83.7
Beaumont                  79.1                        78.4                         83.7                        84.1
Ft. Worth                 80.5                        79.5                         82.9                        85.9
Houston                   80.1                        78.3                         80.8                        82.7
Little Rock               86.3                        90.5                         91.6                        91.5
New Orleans               76.5                        78.6                         81.3                        79.5
Oklahoma City             80.0                        81.1                         84.0                        86.1
San Antonio               79.8                        79.9                         85.5                        87.0

                      Source: GAO’s analysis of data from the Department of Housing and Urban Development’s (HUD)
                      System for Management Information Retrieval-Public Housing (SMIRPH) database.




                      Page 49                                                     GAO/RCED-97-27 Public Housing
Appendix II

Number of Troubled PHAs by Geographic
Region, Fiscal Years 1992-95


                                     Number of troubled PHAs
Region                    1992             1993                1994                    1995
Great Plains                 10              13                  12                       7
Des Moines                       1             3                  1                       1
Kansas City                      1             3                  5                       2
Omaha                            0             1                  1                       1
St. Louis                        8             6                  5                       3
Mid-Atlantic                     7             7                 11                       6
Baltimore                        0             0                  1                       0
Charleston                       1             0                  0                       0
Philadelphia                     2             2                  4                       3
Pittsburgh                       2             3                  4                       2
Richmond                         1             1                  1                       0
District of Columbia             1             1                  1                       1
Midwest                      24              16                  12                      12
Chicago                      12              10                   7                       5
Cincinnati                       1             0                  0                       0
Cleveland                        2             0                  1                       1
Columbus                         0             0                  0                       0
Detroit                          4             5                  3                       2
Grand Rapids                     1             0                  0                       0
Indianapolis                     2             0                  0                       2
Milwaukee                        0             0                  0                       1
Minneapolis                      2             1                  1                       1
Northwest/Alaska                 0             0                  0                       1
Anchorage                        0             0                  0                       0
Portland                         0             0                  0                       0
Seattle                          0             0                  0                       1
New York/New Jersey          10                5                  6                       3
Buffalo                          0             1                  1                       0
New York                         4             2                  3                       1
Newark                           6             2                  2                       2
New England                  12                5                  2                       5
Boston                           2             0                  0                       0
Hartford                         7             5                  2                       4
Manchester                       0             0                  0                       1
Providence                       3             0                  0                       0
                                                                                 (continued)




                       Page 50                                 GAO/RCED-97-27 Public Housing
                      Appendix II
                      Number of Troubled PHAs by Geographic
                      Region, Fiscal Years 1992-95




                                             Number of troubled PHAs
Region                    1992                       1993                          1994                    1995
Pacific/Hawaii                  2                        2                            2                       4
Honolulu                        0                        0                            0                       0
Los Angeles                     0                        0                            0                       0
Phoenix                         1                        1                            1                       2
Sacramento                      0                        0                            0                       0
San Francisco                   1                        1                            1                       2
Rocky Mountains                 2                        4                            8                       0
Denver                          2                        4                            8                       0
Southeast/Caribbean         30                          30                           19                      58
Atlanta                         9                       12                            7                       5
Birmingham                      1                        1                            0                       0
Columbia                        2                        2                            0                       0
Greensboro                      3                        3                            0                       1
Jackson                         2                        0                            1                       1
Jacksonville                    2                        4                            3                       3
Louisville                      2                        1                            2                       1
Knoxville                       0                        0                            0                       1
Nashville                       7                        4                            4                       2
Caribbean                       2                        1                            1                       1
Southwest                   33                          36                           29                      28
Albuquerque                     4                        9                            8                       3
Beaumont                        3                        4                            0                       1
Ft. Worth                       7                        7                            8                       6
Houston                         1                        1                            0                       1
Little Rock                     3                        1                            0                       1
New Orleans                 12                           7                            9                      12
Oklahoma City                   1                        1                            0                       0
San Antonio                     2                        6                            4                       4
All regions                130                         118                         101                      150

                      Source: GAO’s analysis of data from HUD’s SMIRPH database.




                      Page 51                                                      GAO/RCED-97-27 Public Housing
Appendix III

Number of Standard-Performing PHAs by
Geographic Region, Fiscal Years 1992-95


                                     Number of standard-performing PHAs
Region                    1992                   1993                     1994                    1995
Great Plains               235                    213                     157                      141
Des Moines                   30                    31                       18                      12
Kansas City                  99                    95                       62                      60
Omaha                        67                    47                       40                      34
St. Louis                    39                    40                       37                      35
Mid-Atlantic               121                    115                       89                      78
Baltimore                    15                    13                       12                      13
Charleston                   25                    25                       22                      20
Philadelphia                 36                    33                       17                      11
Pittsburgh                   26                    27                       20                      18
Richmond                     13                    12                       13                      11
District of Columbia             6                  5                        5                       5
Midwest                    304                    246                     215                      205
Chicago                      73                    67                       61                      61
Cincinnati                       6                  6                        6                       4
Cleveland                    12                    13                        9                       9
Columbus                     22                    12                       13                       9
Detroit                      24                    19                       19                      17
Grand Rapids                 44                    40                       23                      28
Indianapolis                 27                    20                       17                      13
Milwaukee                    36                    30                       24                      24
Minneapolis                  60                    39                       43                      40
Northwest/Alaska             24                    13                       10                      10
Anchorage                        1                  1                        0                       0
Portland                     11                     5                        4                       3
Seattle                      12                     7                        6                       7
New York/New Jersey        107                     93                       67                      54
Buffalo                      31                    28                       19                      15
New York                     18                    15                       14                       6
Newark                       58                    50                       34                      33
New England                  88                    84                       61                      51
Boston                       40                    49                       31                      22
Hartford                     21                    17                       15                      18
Manchester                   10                     5                        4                       4
Providence                   17                    13                       11                       7
                                                                                            (continued)




                       Page 52                                            GAO/RCED-97-27 Public Housing
                      Appendix III
                      Number of Standard-Performing PHAs by
                      Geographic Region, Fiscal Years 1992-95




                                      Number of standard-performing PHAs
Region                    1992                       1993                          1994                    1995
Pacific/Hawaii              42                          46                           31                      27
Honolulu                        2                        2                            2                       2
Los Angeles                     8                        9                            9                       8
Phoenix                         8                        8                            4                       3
Sacramento                      4                        7                            6                       3
San Francisco               20                          20                           10                      11
Rocky Mountains             49                          44                           31                      35
Denver                      49                          44                           31                      35
Southeast/Caribbean        459                         421                          334                     310
Atlanta                    110                         110                          103                      94
Birmingham                  80                          73                           60                      49
Columbia                    19                          19                            9                       7
Greensboro                  63                          58                           43                      31
Jackson                     30                          27                           25                      25
Jacksonville                49                          37                           36                      47
Louisville                  52                          46                           23                      20
Knoxville                   22                          18                            9                       9
Nashville                   34                          32                           25                      27
Caribbean                       0                        1                            1                       1
Southwest                  497                         445                          363                     331
Albuquerque                 26                          18                           17                      18
Beaumont                    56                          55                           48                      42
Ft. Worth                  130                         128                           97                      86
Houston                     16                          16                           16                      11
Little Rock                 56                          39                           28                      31
New Orleans                 69                          62                           54                      53
Oklahoma City               78                          74                           65                      51
San Antonio                 66                          53                           38                      39
All regions              1,927                       1,719                         1,358                  1,242

                      Source: GAO’s analysis of data from HUD’s SMIRPH database.




                      Page 53                                                      GAO/RCED-97-27 Public Housing
Appendix IV

Number of High-Performing PHAs by
Geographic Region, Fiscal Years 1992-95


                                     Number of high-performing PHAs
Region                    1992                 1993                   1994                    1995
Great Plains               102                  122                   179                      200
Des Moines                   18                  15                     30                      36
Kansas City                  36                  39                     70                      75
Omaha                        31                  50                     57                      63
St. Louis                    17                  18                     22                      26
Mid-Atlantic                 42                  48                     74                      90
Baltimore                        3                5                      6                       6
Charleston                       6                7                     10                      12
Philadelphia                 12                  15                     30                      37
Pittsburgh                       8                6                     13                      17
Richmond                     12                  14                     13                      16
District of Columbia             1                1                      2                       2
Midwest                    200                  266                   303                      316
Chicago                      13                  21                     30                      34
Cincinnati                       2                3                      3                       5
Cleveland                        5                6                      9                       9
Columbus                         2               12                     11                      15‘
Detroit                      18                  23                     25                      28
Grand Rapids                31‘                  36                     53                      48
Indianapolis                 12                  21                     24                      26
Milwaukee                    59                  65                     71                      71
Minneapolis                  58                  79                     77                      80
Northwest/Alaska             34                  45                     48                      47
Anchorage                        0                0                      1                       1
Portland                     18                  24                     25                      26
Seattle                      16                  21                     22                      20
New York/New Jersey          46                  65                     90                     106
Buffalo                      20                  22                     31                      36
New York                     10                  15                     15                      25
Newark                       16                  28                     44                      45
New England                  67                  78                   102                      112
Boston                       24                  17                     34                      44
Hartford                         5               11                     15                      12
Manchester                   33                  38                     39                      38
Providence                       5               12                     14                      18
                                                                                        (continued)




                       Page 54                                        GAO/RCED-97-27 Public Housing
                      Appendix IV
                      Number of High-Performing PHAs by
                      Geographic Region, Fiscal Years 1992-95




                                         Number of high-performing PHAs
Region                    1992                       1993                          1994                    1995
Pacific/Hawaii              35                          31                           46                      49
Honolulu                        0                        0                            0                       0
Los Angeles                 14                          13                           13                      14
Phoenix                         6                        6                           10                      11
Sacramento                      3                        0                            1                       4
San Francisco               12                          12                           22                      20
Rocky Mountains             64                          67                           77                      84
Denver                      64                          67                           77                      84
Southeast/Caribbean        281                         355                          459                     487
Atlanta                     51                          79                           91                     101
Birmingham                  63                          70                           85                      96
Columbia                    19                          19                           31                      34
Greensboro                  31                          36                           54                      65
Jackson                     19                          24                           27                      27
Jacksonville                22                          35                           41                      30
Louisville                  52                          59                           81                      85
Knoxville                   10                          14                           23                      23
Nashville                   14                          19                           26                      26
Caribbean                       0                        0                            0                       0
Southwest                  162                         216                          307                     342
Albuquerque                     6                        9                           13                      16
Beaumont                    11                          11                           22                      27
Ft. Worth                   36                          38                           68                      83
Houston                         3                        3                            4                       8
Little Rock                 51                          70                           82                      78
New Orleans                 16                          29                           34                      34
Oklahoma City               23                          27                           37                      51
San Antonio                 16                          29                           47                      45
All regions              1,033                       1,293                         1,685                  1,833

                      Source: GAO’s analysis of data from HUD’s SMIRPH database.




                      Page 55                                                      GAO/RCED-97-27 Public Housing
Appendix V

Comments From the Department of Housing
and Urban Development




             Page 56           GAO/RCED-97-27 Public Housing
Appendix V
Comments From the Department of Housing
and Urban Development




Page 57                                   GAO/RCED-97-27 Public Housing
Appendix V
Comments From the Department of Housing
and Urban Development




Page 58                                   GAO/RCED-97-27 Public Housing
Appendix V
Comments From the Department of Housing
and Urban Development




Page 59                                   GAO/RCED-97-27 Public Housing
Appendix V
Comments From the Department of Housing
and Urban Development




Page 60                                   GAO/RCED-97-27 Public Housing
Appendix VI

Major Contributors to This Report


                       Lawrence J. Dyckman, Associate Director
Resources,             Eric Marts, Assistant Director
Community, and         Carol Anderson-Guthrie
Economic               Curtis Groves
                       Bill MacBlane
Development Division   Luann Moy
                       Terri Russell




(385602)               Page 61                                   GAO/RCED-97-27 Public Housing
Ordering Information

The first copy of each GAO report and testimony is free.
Additional copies are $2 each. Orders should be sent to the
following address, accompanied by a check or money order
made out to the Superintendent of Documents, when
necessary. VISA and MasterCard credit cards are accepted, also.
Orders for 100 or more copies to be mailed to a single address
are discounted 25 percent.

Orders by mail:

U.S. General Accounting Office
P.O. Box 6015
Gaithersburg, MD 20884-6015

or visit:

Room 1100
700 4th St. NW (corner of 4th and G Sts. NW)
U.S. General Accounting Office
Washington, DC

Orders may also be placed by calling (202) 512-6000
or by using fax number (301) 258-4066, or TDD (301) 413-0006.

Each day, GAO issues a list of newly available reports and
testimony. To receive facsimile copies of the daily list or any
list from the past 30 days, please call (202) 512-6000 using a
touchtone phone. A recorded menu will provide information on
how to obtain these lists.

For information on how to access GAO reports on the INTERNET,
send an e-mail message with "info" in the body to:

info@www.gao.gov

or visit GAO’s World Wide Web Home Page at:

http://www.gao.gov




PRINTED ON    RECYCLED PAPER
United States                       Bulk Rate
General Accounting Office      Postage & Fees Paid
Washington, D.C. 20548-0001           GAO
                                 Permit No. G100
Official Business
Penalty for Private Use $300

Address Correction Requested