oversight

Assessing Performance for Programs to Preserve and Revitalize HUD-Assisted Affordable Housing

Published by the Department of Housing and Urban Development, Office of Inspector General on 2016-09-14.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

      Assessing Performance for Programs
        To Preserve and Revitalize HUD-
          Assisted Affordable Housing




                 Program Evaluations Division


Washington DC   Report Number: 2016-OE-0003     September 14, 2016
                                       MEMORANDUM
                                       September 14, 2016


TO:          Priya Jayachandran
             Deputy Assistant Secretary, Office of Multifamily Housing, HT

             Thomas Davis
             Director, Office of Recapitalization, HTR

FROM:        Kathryn Saylor
             Assistant Inspector General for Evaluation, GAH

SUBJECT: Assessing Performance for Programs To Preserve and Revitalize HUD-Assisted
         Affordable Housing

Attached is the report on our performance assessment of programs to preserve and revitalize
affordable housing within the U.S. Department of Housing and Urban Development (HUD),
Office of Multifamily Housing Programs (Multifamily). We performed this project at the
request of Multifamily officials to assist their efforts to monitor and evaluate program operations.
This review was conducted by Zelos, LLC, for HUD’s Office of Inspector General.

Zelos observed five areas in which Multifamily could improve program performance assessment
and made six recommendations. HUD concurred with the recommendations and provided
information on improvements in process and actions it planned to initiate, along with
implementation dates. Those improvements satisfied the intent of our recommendations. HUD’s
complete response is provided in appendix D.

Please furnish us copies of any correspondence or directives issued as a result of the evaluation.

If you have any questions, please contact me at 202-809-3093 or Nikki Tinsley at 443-822-8285.

Attachment




                                          Office of Inspector General
                                               Office of Evaluation
                                    451 7th Street SW, Washington DC 20024
                                   Phone (202) 708-0430, Fax (202) 401-2505
                                                 www.hudoig.gov
                                                                   Executive Summary
                                       Assessing Performance for Programs To
                       Preserve and Revitalize HUD-Assisted Affordable Housing
Report Number: 2016-OE-0003                                                       September 14, 2016



Purpose                       Observations

Improve U.S. Department of    •   We could find no documentation showing that Multifamily did a systematic
Housing and Urban                 program assessment for TRN or Phase I. Therefore, it did not have lessons
Development (HUD), Office         learned from these efforts to improve current programs.
of Multifamily Housing        •   Existing performance measures and reporting formats did not provide useful
Programs’ (Multifamily)           program evaluation information and did not meet Multifamily’s needs.
ability to monitor and        •   There were opportunities to improve communication and collaboration
evaluate its programs to          among HUD headquarters, regional, and local offices; provide feedback to
preserve affordable housing       grantees and host sites; and leverage other organizations’ leading practices.
and revitalize communities.   •   The Phase II program has new objectives -- to revitalize properties and
                                  integrate community resources to meet specific tenant needs. More
Background                        coordination is needed among Multifamily, CNCS, and EJW to specify
                                  activities and program operations needed to achieve Phase II objectives.
In 2012, Multifamily
                              •   Phase II goals, activities, and measures to revitalize properties and
dedicated funds to the
                                  integrate tenants with community resources should be site specific. Thus,
Tenant Resource Network
                                  each site should follow a site-specific model to achieve specific goals.
(TRN) program, a program
designed to preserve
affordable housing.
                              Recommendations
Multifamily then signed two
                              Multifamily headquarters officials should
interagency agreements
                              • Build systematic program assessment into each program. This requirement
with the Corporation for
                                  should include ongoing evaluation throughout the program to enable
National and Community
                                  effective program management and formal assessments at regular intervals
Service (CNCS) for it to
                                  to monitor and report progress against goals and objectives.
engage AmeriCorps
                              • Specify quarterly reporting requirements to provide meaningful information
Volunteers in Service to
                                  to enable officials to manage, evaluate, adjust, and improve Phase I and II
America (VISTA) members
                                  program operations.
and Equal Justice Works
                              • Obtain input from HUD field offices, provide feedback on Phase I and Phase
(EJW) to carry out Phase I
                                  II programs, and identify leading practices to share with other sites and
and Phase II programs for
                                  potentially other parts of HUD.
affordable housing
preservation and community    • Collaborate with CNCS and EJW to specify and align plans, activities, and
revitalization activities.        reporting requirements to achieve Phase II objectives, including modifying
                                  the current CNCS interagency agreement as needed.
                              • Implement a site-specific model for Phase II that identifies measurement
                                  criteria and reporting requirements and enables officials to monitor and
                                  evaluate progress.
List of Acronyms
  ACRONYM                                      DEFINITION
   CNCS     Corporation for National and Community Service
    EJW     Equal Justice Works
    HUD     U.S. Department of Housing and Urban Development
  OAMPO     Office of Asset Management and Portfolio Oversight
    OIG     Office of Inspector General
   Recap    Office of Recapitalization
    TRN     Tenant Resource Network
   VISTA    Volunteers in Service to America
Table of Contents
Background and Objectives ............................................................................................ 4
  Program History ........................................................................................................... 4
  Program Evaluation and Performance Measurement .................................................. 5
  Evaluation Objectives................................................................................................... 5
Evaluation Results........................................................................................................... 6
  Systematic Program Assessment Needed To Improve Program Operations............... 6
  Recommendation ......................................................................................................... 6
  Management Response ............................................................................................... 6
  Improvements Needed for Program Performance Reporting ....................................... 7
  Recommendations ....................................................................................................... 7
  Management Response ............................................................................................... 8
  Opportunities To Improve Program Operations ........................................................... 8
  Recommendation ......................................................................................................... 9
  Management Response ............................................................................................... 9
  More Coordination Needed To Achieve Phase II Objectives ....................................... 9
  Recommendation ....................................................................................................... 10
  Management Response ............................................................................................. 10
  Site-Specific Model Required for Phase II Objectives ................................................ 10
  Recommendation ....................................................................................................... 12
  Management Response ............................................................................................. 12
Appendixes ................................................................................................................... 13
  Appendix A – Site-Specific Model .............................................................................. 13
  Appendix B – Scope and Methodology ...................................................................... 28
  Appendix C – Summary of Recommendations .......................................................... 30
  Appendix D – Agency Comments .............................................................................. 32
                                                                                 Report number: 2016-OE-0003




Background and Objectives
The Office of Evaluation performed this project at the request of U.S. Department of Housing
and Urban Development (HUD) officials to assist the HUD Office of Multifamily Housing
Programs’ (Multifamily) efforts to monitor and evaluate program operations. Multifamily
officials wanted to know the impact programs had on preserving 1 affordable housing via tenant
outreach and advocacy activities and revitalizing 2 communities through activities that integrate
community services and resources to meet tenant needs. Multifamily specifically requested
assistance in developing measures for community revitalization efforts.

Program History
HUD made available $10 million under Section 514 of the Multifamily Assisted Housing
Reform and Affordability Act of 1997 to nonprofit organizations (grantees) through an October
2011 Notice of Funding Availability for the Tenant Resource Network (TRN) program. The
purpose of this program was to preserve affordable housing and empower tenants. 3 In June
2012, Multifamily awarded a total of $4.9 million to 15 grantees. The TRN program ended in
the summer of 2015.

In August 2014, Multifamily signed an interagency agreement with the Corporation for National
and Community Service (CNCS) and provided $2 million for CNCS to work with AmeriCorps’
Volunteers in Service to America (VISTA) members, Equal Justice Works (EJW), and host sites
(subgrantees) for Phase I, a program also designed to preserve affordable housing. Phase I began
in August 2014 and is ongoing.

In August 2015, Multifamily signed a second interagency agreement with CNCS and provided
an additional $2 million for CNCS to continue Phase I and also establish Phase II, a program
designed to revitalize affordable housing and neighboring communities.

Multifamily’s Office of Asset Management and Portfolio Oversight (OAMPO) managed the
TRN program for its duration, as well as the Phase I program operations until mid-December
2015. At that time, responsibility for Phase I and II was transferred to Multifamily’s Office of
Recapitalization (Recap). Recap is developing plans and selecting properties for Phase II, and
activities are expected to begin in the fall of 2016.

1
  Preserving and preservation refers to Multifamily programs and activities designed to keep project-based Section 8
housing assistance payments contract-assisted properties in the Multifamily housing portfolio. This process includes
renewing such contracts and keeping the properties in good physical condition.
2
  Revitalizing and revitalization refer to Multifamily programs and activities designed to address tenants of
properties going through Recap’s Rental Assistance Demonstration Program to better use neighboring community
resources to address tenants’ needs.
3
  Empower tenants refers to assisting, informing, educating and engaging tenants of Section 8 properties regarding
their rights, responsibilities, and options.

                                                         4
                                                                             Report number: 2016-OE-0003


Program Evaluation and Performance Measurement
There are two common types of systematic program assessment: program evaluation and
performance measurement: 4

      •   Program evaluations are individual systematic studies conducted periodically or on an ad
          hoc basis to assess how well a program is working. Evaluations typically assess
          achievement of program objectives and may examine aspects of program operations to
          help explain the linkages among program inputs, activities, outputs, and outcomes.
      •   Performance measurement is the ongoing monitoring and reporting of program
          accomplishments, particularly progress toward preestablished goals. Performance
          measurement focuses on whether a program has achieved its objectives, expressed in
          measurable performance standards.

Both forms of assessment – program evaluation and performance measurement − aim to support
resource allocation and other policy decisions and to improve service delivery and program
effectiveness. However, performance measurement, because of its ongoing nature, can serve as
an early warning system to management and as a vehicle for improving accountability to the
public.

Evaluation Objectives
Our evaluation focused on recent Multifamily programs designed to preserve affordable housing
and revitalize affordable housing and neighboring communities. Specifically, our objectives
were to

      •   Identify lessons learned about performance measurement related to preserving
          affordable housing activities from the TRN program and Phase I.
      •   Develop an approach to enable Recap to identify and report Phase II performance metrics
          and evaluate program operations.




4   Performance Measurement and Evaluation: Definitions and Relationships, GAO-11-646SP, May 2011

                                                      5
                                                                                Report number: 2016-OE-0003




Evaluation Results
Systematic Program Assessment Needed To Improve Program
Operations
We could find no documentation showing that OAMPO officials had assessed the effectiveness
or results of either the TRN or Phase I activities and program operations. Therefore, Recap
officials did not have lessons learned from the TRN program or Phase I to build on for Phase II
or to improve Phase I going forward.

Program evaluations are typically examinations of program performance in a context that allows
for an overall assessment of how the program is working and identifies adjustments that may
improve its results. Depending on their focus, evaluations may examine aspects of program
operations (such as in a process evaluation) or factors in the program environment that may
impede or contribute to its success, to help explain the linkages among program inputs, activities,
outputs, and outcomes. There are four types of program evaluations: outcome, process, impact,
and cost-benefit or cost-effectiveness analysis. The most relevant types for assessing the Phase I
and II programs appear to be outcome and process evaluations. 5

Recommendation
The Deputy Assistant Secretary, Office of Multifamily Housing, and the Director, Office of
Recapitalization, should

    1. Build systematic program assessment into each program. This requirement should
       include both (1) ongoing process evaluation throughout the program to enable effective
       program management and (2) formal assessments at regular intervals and at the
       program’s end to report on progress toward meeting preestablished goals and achieving
       expected outcomes.

Management Response
HUD agreed with this recommendation and planned to implement program assessment for both
Phase I and Phase II. This assessment would include both (1) ongoing, informal evaluation
based on routine reporting and program management and (2) a more formal assessment of Phase
I in the spring of 2017 and of Phase II in approximately the fall of 2017.

Management’s complete response is provided in appendix D.


5
 Outcome evaluation assesses the extent to which a program achieves its outcome-oriented objectives and focuses
on outputs and outcomes to judge program effectiveness. Process evaluation assesses the extent to which a program
operates as intended.

                                                       6
                                                                      Report number: 2016-OE-0003


Improvements Needed for Program Performance Reporting
The previous and existing performance reporting formats and requirements were not designed to
meet Multifamily’s needs, nor did they provide useful information for evaluating the TRN
program or Phase I.

The TRN program specified that grantees use the e-Logic Model to report performance on goals
and objectives, activities, outcomes, and measures. Several grantees and Multifamily OAMPO
officials said that the e-Logic Model had reporting limitations, and it was replaced with a
different reporting format; thus, the merits of the Model are not known. However, the revised
TRN program reporting format did not follow a consistent structure, and the performance
measures did not show the impact of the activities on goals and objectives to empower tenants
and preserve affordable housing. Therefore, the reports did not provide OAMPO officials with
information on the impact of the activities at individual sites or an overall assessment of program
operations and effectiveness.

CNCS and EJW primarily collected information in Phase I to monitor VISTA member
developmental and capacity-building activities. In general, the data collected and reported did
not show whether tenant outreach activities contributed to preserving affordable housing.

Both TRN and Phase I site reports included extensive narrative information on various topics not
related to program performance. This information provided occasional anecdotes, but the
narrative did not provide or support an overall assessment of the program and was not useful for
identifying leading practices to share or common issues to address.

Recap officials were developing reporting requirements for Phase II. Given Recap’s resource
constraints for program management, as well as OMB’s and Congress’ interest in knowing
program results, it is important that reports readily provide meaningful information and support
both program management and reporting to interested entities.

Recommendations
The Director, Office of Recapitalization, should

   2. Specify Phase II quarterly reporting requirements that provide meaningful information to
      enable them to manage, evaluate, adjust, and improve program operations. Recap
      officials should identify the information they need to perform their monitoring and
      reporting responsibilities. Specifically, the reports should
          • Provide the information Recap needs, versus that required by CNCS or EJW.
          • Be modified to present the information in a manner that is accessible,
              understandable, and useful.
          • Reflect progress over time.



                                                   7
                                                                                Report number: 2016-OE-0003


    3. Apply relevant aspects of the new reporting process for Phase II to Phase I reporting to
       enable Recap officials to manage and report progress for both programs.

Management Response
HUD agreed with the recommendations and intended to introduce new reporting requirements
for Phase II to CNCS and EJW based on the suggested reporting model contained within the
evaluation report (appendix A). HUD also agreed that there were relevant aspects of the
suggested reporting model that could be adapted to capture useful information on Phase I
program activities and outcomes. HUD noted and we agreed, however, that reporting would be
different between the two programs to reflect the different objectives of each program. HUD
said it would introduce a new reporting framework for negotiation with CNCS and EJW within
60 days.

Management’s complete response is provided in appendix D.

Opportunities To Improve Program Operations
Opportunities exist to improve communication and collaboration among HUD headquarters,
regional, and local offices; provide feedback to grantees and host sites; and leverage other
organizations’ leading practices.

For the TRN and Phase I programs, HUD’s local and regional officials told us that OAMPO
headquarters officials did not communicate or coordinate with them to leverage local and
regional knowledge regarding properties they monitor. In addition, several HUD local and
regional offices told us they were not aware that there was a Phase II program.

Our analysis showed that OAMPO headquarters officials did not always provide feedback to
grantees or host sites on the quarterly reports they submitted and in some cases, did not respond
to specific issues and grantee questions directed to them that affected program operations.
Therefore, some participants and sites operated without official answers to program and policy
questions and did not learn about leading practices that might improve program operations. For
example, OAMPO did not apply, on a programwide basis, letters used at a regional and local
level that reinforced regulations for property access by grantees and VISTA members.

We identified that some private organizations, The Community Builders 6 and NeighborWorks, 7
use reporting systems and tools that provide useful and meaningful information for initiatives

6
 The Community Builders’ mission is to build and sustain strong communities where people of all incomes can
achieve their full potential. The majority of its work is building and rebuilding multifamily housing.
Source: http://www.tcbinc.org/values.html
7
  NeighborWorks supports a network of more than 240 nonprofit organizations with technical assistance, grants, and
training for more than 12,000 professionals in the affordable housing and community development field every year.
Source: http://neighborworks.org/About-Us/What-We-Do


                                                        8
                                                                    Report number: 2016-OE-0003


similar to the TRN, Phase I, and Phase II programs. Recap officials could leverage these leading
practices to improve program operations and better meet program objectives.

Recommendation
The Director, Office of Recapitalization, should

   4. Obtain input from regional and local HUD offices regarding site-specific issues and
      property characteristics, provide feedback on and use the quarterly reports to identify and
      address problematic programwide operations as well as site-specific issues, and seek to
      identify leading practices or examples that could be shared with other sites and
      potentially other parts of HUD (including regional or local offices) to improve program
      effectiveness.

Management Response
HUD agreed with this recommendation and had initiatives underway and planned to improve
communication with regional and local HUD offices to aid Recap in identifying site-specific
issues and improving program effectiveness.

Management’s complete response is provided in appendix D.

More Coordination Needed To Achieve Phase II Objectives
The Phase II program has new objectives: to revitalize properties and integrate community
resources to meet specific tenant needs. This program focuses on increasing access and
resources for tenants of sites under Recap’s Rental Assistance Demonstration Program by
integrating the resources of the neighboring community. VISTA members, under the supervision
of EJW and CNCS officials, are tasked with identifying tenant needs and community resources
to address those needs.

Although Recap, CNCS, and EJW had coordinated to some extent regarding Phase II objectives
and operations, the interagency agreement had not been modified to address these objectives.
More coordination is needed to align the objectives and operations among these key
organizations. Without such communication and collaboration, Phase II objectives may not be
achieved.




                                                   9
                                                                      Report number: 2016-OE-0003


Recommendation
The Director, Office of Recapitalization, should

   5. Specify requirements to achieve Phase II objectives, including modifying the interagency
      agreement as needed, and collaborate with CNCS and EJW to specify and align plans,
      activities, and reporting requirements to achieve Phase II objectives.

Management Response
HUD agreed with this recommendation and had coordinated with CNCS and EJW to create an
agreed-upon and written Phase II project plan that would identify expected activities. HUD said
that it was communicating regularly with partners to monitor the progress of the Phase II
program implementation. HUD said it would consider additional revisions to the interagency
agreement to formally document the project plan and related activities and reporting
requirements.

Management’s complete response is provided in appendix D.

Site-Specific Model Required for Phase II Objectives
Phase II goals are to revitalize properties and integrate community resources to meet tenant
needs. Because each site will have different demographics, tenant needs, and community
resources, goals and measures will likely be site specific. Therefore, VISTA members will
perform site-specific activities. However, each site should follow a standard approach to design
and implement its program and achieve a variety of goals and enable Recap to manage, monitor,
and evaluate program operations.

Our analysis shows that a site-specific model could provide information to tailor goals, activities,
metrics, and baseline information for assessment. Such a model could include steps to identify
quarterly reporting requirements and develop a combined quarterly report to meet Recap’s needs.
Accordingly, we developed a model that Recap can use to address the items above.

The following exhibit outlines the model. Additional information is provided in appendix A.




                                                10
                                                                          Report number: 2016-OE-0003



                              Exhibit: Five-step site-specific model


            Assess site




                          Analyze data
                          and set goals



                                          Develop site
                                          success plan



                                                         Implement site
                                                          success plan


                                                                            Evaluate and
                                                                          adjust to improve
                                                                              program
                                                                             operations




The following table illustrates an example of using the model and steps to determine and
accomplish a site-specific goal: improving transportation services.

         Steps                                             Description
                           Site assessment resulted in identifying tenant issues:
                               • Tenants could not get to work or grocery stores because the
      Assess site
                                   community did not have options such as rental bikes or bus,
                                   subway, or taxi-ride share services.
     Analyze data          Data analysis resulted in setting a transportation-related goal of
     and set goals              • Establishing bus service.
                           Developed a detailed (multiyear, if appropriate) approach outlining
      Develop site
                                • Activities to be performed and
      success plan
                                • Measures that will track and monitor progress.
                           Tasks in the site success plan included
                               • Meet with city transportation officials to identify resources and
     Implement site                specific activities and actions needed to establish bus service.
      success plan             • Complete specific activities and actions.
                               • Collect performance measurement data.
                               • Develop and submit quarterly reports.
  Evaluate and adjust      Program evaluation at regular intervals included
  to improve program            • Identify progress and any challenges.
       operations               • Provide information to adjust and improve program operations.

Reporting progress against this model will provide useful, comparative information across sites.
Each site will set goals and objectives based on tenant needs and then identify how to measure

                                                  11
                                                                      Report number: 2016-OE-0003


success. The site-specific information can be combined into a programwide quarterly summary
report that meets Recap’s needs. In addition, it will provide information to assist Recap officials
in communicating with and addressing the needs of individual sites as well as overall program
operations.

Recommendation
The Director, Office of Recapitalization, should

   6. Implement a site-specific model to manage and evaluate Phase II site operations and use
      as a basis for quarterly reporting. The proposed site-specific model provides a systematic
      process to develop goals, activities, measures, and reports and monitor performance for
      Phase II. (The model is provided in appendix A.)

Management Response
HUD agreed that a site-specific model was needed to allow each site to report activities,
challenges, and accomplishments to achieve meaningful outcomes under Phase II and had begun
to discuss this approach with CNCS and EJW as a means to develop quarterly data reports.

Management’s complete response is provided in appendix D.




                                                12
                                        Report number: 2016-OE-0003




Appendixes
Appendix A – Site-Specific Model




                                   13
     Report number: 2016-OE-0003




14
     Report number: 2016-OE-0003




15
     Report number: 2016-OE-0003




16
     Report number: 2016-OE-0003




17
     Report number: 2016-OE-0003




18
     Report number: 2016-OE-0003




19
     Report number: 2016-OE-0003




20
     Report number: 2016-OE-0003




21
     Report number: 2016-OE-0003




22
     Report number: 2016-OE-0003




23
     Report number: 2016-OE-0003




24
     Report number: 2016-OE-0003




25
     Report number: 2016-OE-0003




26
     Report number: 2016-OE-0003




27
                                                                     Report number: 2016-OE-0003


Appendix B – Scope and Methodology
Our evaluation focused on recent Multifamily programs designed to preserve project-based
housing assistance payments, contract-assisted properties, and revitalize properties going
through the Rental Assistance Demonstration Program. We used U.S. Government
Accountability Office guidelines to identify gaps in the operation, management, and
performance reporting for the TRN and Phase I programs.

We identified and assessed leading practices and relevant information to develop a site-specific
model to enable Recap to identify and report performance metrics to evaluate Phase II program
operations. The site-specific model should be used to develop goals, activities, measures, and
reports for Phase II and also to monitor performance. The model incorporates information from
the TRN program and Phase I, approaches used for revitalizing communities, U.S. Government
Accountability Office guidelines for performance measures, and other documented ways of
measuring and reporting performance.

Our scope was limited to operation, management, and reporting of performance of the
programs and, thus, did not include an evaluation of whether the activities carried out were
the correct ones for preservation and revitalization.

To accomplish our objectives, we performed the following activities:

   •   Identified and reviewed past studies and relevant Office of Inspector General (OIG), U.S.
       Government Accountability Office, and other reports to gain an understanding and
       historical perspective of preserving affordable housing, revitalizing communities,
       program evaluation, and performance measures.
   •   Reviewed and analyzed
           o TRN Notice of Funding Availability and CNCS interagency agreements for Phase
               I and II.
           o Metrics reported for the TRN program and CNCS Phase I.
           o Whether the metrics used were feasible and effective for assessing the impact of
               tenant outreach activities.
           o Leading practices of private organizations carrying out programs similar to the
               TRN program and Phase I and II initiatives.
           o Other HUD place-based community development initiatives; Choice
               Neighborhoods and Promise Zone; and relevant leading practices, policies, and
               procedures.
           o Program assessment guidance for program evaluation and performance measures.
   •    Interviewed individuals from
           o HUD Multifamily OAMPO.
           o HUD Multifamily Recap.
           o HUD regional and local offices.

                                               28
                                                                  Report number: 2016-OE-0003


           o Choice Neighborhood program in HUD’s Office of Public and Indian Housing.
           o Promise Zone program in HUD’s Office of Field Policy and Management.
           o CNCS.
           o EJW.
           o TRN grantees.
           o Phase I host and sponsor site participants.
           o AmeriCorps VISTA members.
   •   Interviewed individuals from private organizations, The Community Builders and
       NeighborWorks, to learn about leading practices for place-based initiatives, data
       collection, and measuring impact.
   •   Conducted an analysis to identify gaps in program design, management, data collection,
       and reporting performance for TRN and Phase I.

We performed the evaluation from December 2015 through July 2016 at HUD headquarters in
Washington, DC. We performed work in accordance with the Council of the Inspectors General
on Integrity and Efficiency’s Quality Standards for Inspection and Evaluation, January 2012.




                                             29
                                                                 Report number: 2016-OE-0003


Appendix C – Summary of Recommendations
  OIG
          No.                        Recommendation                              Status
 report
                The Deputy Assistant Secretary, Office of Multifamily
                Housing, and the Director, Office of Recapitalization,
                should
                   1. Build systematic program assessment into each
                       program. This requirement should include both (1)
 2016-
           1
                       ongoing process evaluation throughout the program
OE-0003                to enable effective program management and (2)
                       formal assessments at regular intervals and at the
                       program’s end to report on progress toward meeting
                       preestablished goals and achieving expected
                       outcomes.

                The Director, Office of Recapitalization, should
                   2. Specify Phase II quarterly reporting requirements
                       that provide meaningful information to enable Recap
                       to manage, evaluate, adjust, and improve program
                       operations. Recap officials should identify the
                       information they need to perform their monitoring
                       and reporting responsibilities. Specifically, the
 2016-                 reports should
           2
OE-0003
                           • Provide the information Recap needs, versus
                               that required by CNCS or EJW.
                           • Be modified to present the information in a
                               manner that is accessible, understandable,
                               and useful.
                           • Reflect progress over time.

                The Director, Office of Recapitalization, should
                   3. Apply relevant aspects of the new reporting process
 2016-
           3
                       for Phase II to Phase I reporting to enable Recap
OE-0003                officials to manage and report progress for both
                       programs.

                The Director, Office of Recapitalization, should
                   4. Obtain input from regional and local HUD offices
                       regarding site-specific issues and property
                       characteristics, provide feedback on and use the
 2016-                 quarterly reports to identify and address problematic
           4
OE-0003                programwide operations as well as site-specific
                       issues, and seek to identify leading practices or
                       examples that could be shared with other sites and
                       potentially other parts of HUD (including regional or
                       local offices) to improve program effectiveness.

                                            30
                                                                 Report number: 2016-OE-0003


 OIG
          No.                        Recommendation                              Status
report


                The Director, Office of Recapitalization, should
                   5. Specify requirements to achieve Phase II objectives,
                       including modifying the interagency agreement as
 2016-
           5           needed, and collaborate with CNCS and EJW to
OE-0003
                       specify and align plans, activities, and reporting
                       requirements to achieve Phase II objectives.

                The Director, Office of Recapitalization, should
                   6. Implement a site-specific model to manage and
                       evaluate Phase II site operations and use as a basis
 2016-                 for quarterly reporting. The proposed site-specific
           6
OE-0003                model provides a systematic process to develop
                       goals, activities, measures, and reports and monitor
                       performance for Phase II.




                                            31
                                    Report number: 2016-OE-0003


Appendix D – Agency Comments




                               32
     Report number: 2016-OE-0003




33
     Report number: 2016-OE-0003




34
                                                         Report number: 2016-OE-0003




 The Office of Inspector General is an independent and objective oversight
  agency within the U.S. Department of Housing and Urban Development.
 We conduct and supervise audits, evaluations, and investigations relating
 to the Department’s programs and operations. Our mission is to promote
 economy, efficiency, and effectiveness in these programs while preventing
             and detecting fraud, abuse, and mismanagement.


    Report fraud, waste, and mismanagement in HUD programs and operations by

                       Faxing the OIG hotline: 202-708-4829
                    Emailing the OIG hotline: hotline@hudoig.gov

                          Sending written information to
                 Department of Housing and Urban Development
                         Inspector General Hotline (GFI)
                          451 7th Street, SW Room 8254
                             Washington, DC 20410

                                            Internet

                     http://www.hudoig.gov/hotline/index.php




             Program Evaluations Division

Report number: 2016-OE-0003

                                             35