OFFICE OF AUDIT
REGION 3
PHILADELPHIA, PA
U.S. Department of Housing and Urban
Development
Washington, DC
Moving to Work Demonstration Program
2013-PH-0004 SEPTEMBER 27, 2013
Issue Date: September 27, 2013
Audit Report Number: 2013-PH-0004
TO: Dominique G. Blom, Deputy Assistant Secretary for Public Housing
Investments, PI
//signed//
FROM: John P. Buck, Regional Inspector General for Audit, Philadelphia Region, 3AGA
SUBJECT: HUD’s Oversight of Its Moving to Work Demonstration Program Needs
Improvement
Attached is the U.S. Department of Housing and Urban Development (HUD), Office of
Inspector General’s (OIG) final results of our review of HUD’s monitoring and oversight of its
Moving to Work Demonstration program.
HUD Handbook 2000.06, REV-4, sets specific timeframes for management decisions on
recommended corrective actions. For each recommendation without a management decision,
please respond and provide status reports in accordance with the HUD Handbook. Please furnish
us copies of any correspondence or directives issued because of the audit.
The Inspector General Act, Title 5 United States Code, section 8L, requires that OIG post its
publicly available reports on the OIG Web site. Accordingly, this report will be posted at
http://www.hudoig.gov.
If you have any questions or comments about this report, please do not hesitate to call me at
(215) 430-6729.
September 27, 2013
HUD’s Oversight of Its Moving to Work Demonstration
Program Needs Improvement
Highlights
Audit Report 2013-PH-0004
What We Audited and Why What We Found
We audited the U.S. Department of HUD had implemented program monitoring
Housing and Urban Development’s procedures; however, its program oversight was
(HUD) oversight of the Moving to inadequate because it had not (1) implemented
Work Demonstration program based on programwide performance indicators, (2) evaluated
the HUD Office of Inspector General’s agencies’ programs according to its policy, (3)
(OIG) initiative to focus HUD evaluated agencies’ compliance with key statutory
management’s attention on problem requirements, (4) verified agencies’ self-reported
areas on which we and others have performance data, and (5) performed required annual
reported over the years. Our objective program risk assessments. As a result, HUD was
was to determine whether HUD had unable to demonstrate program results. Also, HUD
implemented adequate program controls had limited ability to assess agencies’ compliance with
to effectively monitor participant statutory program goals and lacked assurance that
agencies’ performance and ensure that agencies met key statutory requirements.
they met statutory program goals and
requirements.
What We Recommend
We recommend that the Deputy
Assistant Secretary for Public Housing
Investments establish and implement
policies to improve HUD’s
administration of the program and
ensure that agencies meet statutory
goals and requirements before
expanding the program.
TABLE OF CONTENTS
Background and Objective 3
Results of Audit
Finding: HUD Did Not Adequately Oversee Its Moving to Work
Demonstration Program 4
Scope and Methodology 12
Internal Controls 13
Follow-up on Prior Audits 14
Appendixes
A. Auditee Comments and OIG’s Evaluation 16
2
BACKGROUND AND OBJECTIVE
The Omnibus Consolidated Rescissions and Appropriations Act of 1996, dated April 26, 1996,
authorized the Moving to Work Demonstration program. The program was established to give
public housing agencies and the U.S. Department of Housing and Urban Development (HUD)
flexibility to design and test innovative, locally designed strategies in pursuit of the national goal of
delivering rental assistance more efficiently. The program’s intent is to pursue three statutory
objectives: (1) reduce cost and achieve greater cost effectiveness in Federal expenditures; (2) give
incentives to families with children in which the head of household is working; seeking work; or
preparing by participating in job training, educational programs, or programs that assist people to
obtain employment and become economically self-sufficient; and (3) increase housing choices for
low-income families.
In pursuit of these objectives, Moving to Work housing agencies are granted exceptions to many
portions of the U.S. Housing Act of 1937 and combine funds received for the Housing Choice
Voucher program and public housing capital and operating funds to use interchangeably. Agencies
are required to
• Ensure that at least 75 percent of families admitted are very low income.
• Establish a reasonable rent policy designed to encourage self-sufficiency and employment.
• Continue to assist substantially the same number of eligible low-income families that they
would have served had funding amounts not been combined.
• Maintain a mix of families comparable to those they would have served without the
demonstration.
• Ensure that the housing provided meets HUD’s housing quality standards.
HUD’s Office of Public Housing Investments, within the Office of Public and Indian Housing at
HUD headquarters, is responsible for program oversight. As of July 2013, there were 35 agencies
participating in the program.
Our objective was to determine whether HUD had implemented adequate program controls to
effectively monitor participant agencies’ performance and ensure that they met statutory program
goals and requirements.
3
RESULTS OF AUDIT
Finding: HUD Did Not Adequately Oversee Its Moving to Work
Demonstration Program
HUD generally conducted periodic visits to program participants according to its policy and
maintained documentation to support its most recent admissions into the program. However, its
program oversight was inadequate because it had not (1) implemented programwide performance
indicators, (2) evaluated agencies’ programs according to its policy, (3) evaluated agencies’
compliance with key statutory program requirements, (4) verified agencies’ self-reported
performance data, and (5) performed required annual program risk assessments. These
conditions occurred because HUD believed that implementing performance indicators would be
difficult and contrary to the program’s purpose. Also, HUD cited a lack of funding needed to
perform critical evaluations and did not have adequate procedures to verify agencies’ compliance
with key statutory requirements. Additionally, HUD lacked procedures to verify agencies’ self-
reported performance data and was not aware of the requirement to perform annual program risk
assessments. As a result, it was unable to demonstrate program results. Also, HUD had limited
ability to assess agencies’ compliance with statutory program goals and lacked assurance that
agencies met key statutory requirements. Further, HUD lacked assurance that it had properly
identified and addressed risks that might prevent participating agencies from meeting program
goals and statutory requirements.
HUD Conducted Periodic Site
Visits According to Its Policy
HUD’s Moving to Work desk guide states that it will conduct annual site visits to
participant agencies to discuss their programs and provide technical assistance.
HUD generally conducted annual site visits to agencies in accordance with its
procedures. It prepared reports to document its annual site visits. We reviewed
21 site visit reports for 4 agencies in Region 3, including the Philadelphia
Housing Authority, Housing Authority of Baltimore City, District of Columbia
Housing Authority, and Housing Authority of the City of Pittsburgh, for the
period January 2006 through October 2012. During the site visits, HUD met with
agency officials to discuss program updates, agencies’ short- and long-term
program goals, current activities, and local issues. HUD provided technical
assistance by providing verbal guidance on issues agencies encountered, such as
entering data into HUD’s Public and Indian Housing Information Center database.
HUD also toured housing developments to verify agencies’ progress.
4
Although HUD conducted annual site visits in accordance with its procedures, it
did not validate the accuracy of agencies’ self-reported performance data during
these visits. According to HUD officials, HUD was working on developing
procedures to verify the accuracy of self-reported data, either through audits or
during annual site visits.
HUD Maintained Documents
Supporting Its Most Recent
Admissions Into the Program
Section 232 of the Consolidated Appropriations Act of 2010 allowed HUD to
select four new participant agencies, which were designated as “high performing”
under the Public Housing Assessment System and administered no more than
5,000 aggregate housing vouchers and public housing units. Public and Indian
Housing Notice 2012-16 required eligible agencies to submit an application
containing an eligibility certification, evidence of capability, a Moving to Work
plan, evidence of community support and involvement, and additional
information regarding implementation of the Moving to Work plan. The notice
also described HUD’s evaluation criteria for selecting the eligible agencies. The
evaluation criteria required HUD to evaluate applications based on the following
four rating factors: (1) demonstrated capability to effectively plan, implement,
and administer the program; (2) feasibility of the Moving to Work plan; (3)
degree of resident and community support and involvement; and (4) potential
local and national impact of the Moving to Work plan. Each rating factor
included a number of points attributed to the factor, with the possible number of
points for all four factors totaling 100 points.
HUD received applications from 12 agencies seeking to participate in the program
and selected the 4 with the highest scores. We reviewed the applications and
related evaluations for the four agencies and found that HUD maintained adequate
documentation to show that it made the selections according to its policy.
HUD Lacked Programwide
Performance Indicators
HUD had not established programwide performance indicators that would allow it
to assess program results. The program was established to (1) reduce cost and
achieve greater cost effectiveness in Federal expenditures, (2) incentivize families
to become economically self-sufficient, and (3) increase housing choices for low-
income families. The Government Performance and Results Modernization Act
of 2010 requires Federal agencies to develop a performance plan for each
program activity. The plan should establish a balanced set of performance
indicators to be used in measuring or assessing progress toward each performance
5
goal, including efficiency, output, and outcome indicators. HUD’s annual
performance plans for fiscal years 2011 and 2012 established agencywide
performance indicators; however, none of the indicators was specifically
connected to the program.
HUD officials stated that programwide performance indicators were not
implemented because the program lacked standard metrics related to program
performance. Based on discussions with HUD officials and HUD’s statements
during an April 2012 U.S. Government Accountability Office (GAO) audit of the
program, it appeared that HUD did not implement standard performance metrics
because it believed doing so could be difficult and might be contrary to the nature
of the program. Due to the lack of performance indicators, HUD was unable to
demonstrate program results or show that the three main program objectives were
met. We believe that HUD could establish policies to use the results from
agencies’ self-reported performance data to develop programwide performance
indicators for assessing program results.
The U.S. Office of Management and Budget (OMB) recently issued a
memorandum, 1 which noted the need to continually improve Federal program
performance by applying existing evidence on successes, generating new
knowledge, and using experimentation and innovation to test new approaches to
program delivery. According to the memorandum, evidence would include
evaluation results, performance measures, and other relevant data analytics and
research studies, with a preference for high-quality experimental and quasi-
experimental studies. For 2015 budget proposals, OMB urged agencies to (1) use
credible evidence in formulating 2015 budget proposals and performance plans and
(2) propose new ways to develop evidence to address policy challenges. OMB
stated that agencies’ budget requests would be more likely to be fully funded if they
showed a widespread commitment to evidence and innovation. Therefore, it is vital
that HUD develop programwide performance indicators to ensure that its future
budget requests are fully funded.
HUD Failed To Evaluate
Agencies’ Programs According
to Its Policy
HUD did not evaluate agencies’ programs according to provisions in its standard
program agreement and as incorporated into its program monitoring policies.
According to the agreement, HUD would evaluate agencies’ programs during
fiscal years 2011 and 2014 to determine whether the agencies should continue in
the program. For the evaluations, HUD planned to use the following criteria: (1)
the agency complies with the standard program agreement; (2) the annual
program plans and reports have been satisfactorily completed and submitted in a
1
OMB Memorandum 13-17, Next Steps in the Evidence and Innovation Agenda, dated July 26, 2013
6
timely manner, consistent with the standard program agreement; and (3) the
agency has demonstrated, through the annual program plan and report, that it used
its program designation in accordance with Section 204 of the 1996
Appropriations Act.
HUD planned to hire an outside research consultant to perform the evaluations
and use the results to determine whether the agencies would be allowed to
continue to participate in the program or have their participation terminated.
HUD preferred to hire an outside research consultant instead of performing the
evaluation in-house because it wanted a third-party opinion regarding the
program. However, HUD did not perform the 2011 evaluation and had not
performed one since. HUD officials informed us that they did not perform the
evaluation due to a lack of funding and the fiscal year 2014 evaluation might also
be jeopardized for the same reason. As a result, HUD lacked assurance that
agencies complied with the program agreement and whether the agencies should
continue to participate in the program. Based on the lack of funding for these
evaluations, HUD should not admit additional agencies into the program until the
evaluations are performed.
HUD officials responsible for the program informed us that they had been
requesting funds to perform the program evaluation through HUD’s
Transformation Initiative 2 since January 2010. HUD estimated a cost of
approximately $3 million to perform the evaluation. In February 2013, program
officials met with HUD’s Office of Policy Development and Research officials to
further discuss the need for the evaluation. The Office of Policy Development
and Research stated that it would investigate whether Transformation Initiative
funds could be allocated to perform the evaluation in fiscal year 2013 or 2014. In
July 2013, the Office of Policy Development and Research stated that it planned
to award an evaluation contract in fiscal year 2014. It also stated that some funds
had been reserved for the evaluation and that it planned to reserve additional
funding in fiscal year 2014. Since there are indications that Congress is
considering expanding the program to include more agencies, we believe HUD
needs to evaluate the program participants to determine whether they comply with
the program agreement before expanding the program.
HUD Failed To Verify
Agencies’ Compliance With
Statutory Requirements
HUD had not implemented controls to verify whether agencies met statutory
requirements related to performance that were incorporated into the standard
2
In 2010, Congress enacted the Transformation Initiative, which makes up to 1 percent of program funds available
for the following: (1) research, evaluation, and program metrics; (2) program demonstration; (3) technical
assistance; and (4) information technology.
7
program agreement. The agreement required that agencies (1) ensure that at least
75 percent of assisted families were very low-income families as defined in the
Housing Act of 1937, (2) serve substantially the same number of families that
would have been served if they had not participated in the program, and (3)
maintain a mix of families comparable to those that would have been served
without the program. As part of agencies’ annual report, the agencies would self-
certify that they complied with the statutory requirements. Although the self-
certification process served as a control measure to determine or ensure agencies’
compliance with statutory requirements, the effectiveness of the control was
limited because HUD had no policies or procedures to periodically verify the
agencies’ self-certifications.
We determined that HUD had taken steps to verify agencies’ compliance with the
statutory requirements. For the second statutory requirement, HUD issued
guidance in January 2013, which described how it would determine agencies’
compliance with the requirement annually. 3 For the first and third statutory
requirements, HUD had implemented reporting requirements instructing agencies
to self-report data to support compliance. HUD officials informed us that they
would verify the accuracy of the self-reported data through an audit or during
annual site visits. However, HUD had not finalized policies or procedures to
verify the self-reported information and thereby fully verify agencies’ compliance
with the first and third statutory requirements.
HUD Did Not Verify Agencies’
Self-Reported Performance
Data
HUD lacked policies and procedures to verify the accuracy of key information
that agencies self-report. Attachment B of each standard program agreement
required agencies to report in their annual reports on benchmarks, metrics, and
performance information, such as reducing the number of homeless persons
without housing, reducing the processing time during the annual recertification
process, and increasing the number of housing choices for low-income
households. The information reported by agencies related to the outcomes of
ongoing activities included the number of homeless persons provided with
housing, the number of program staff hours saved annually through the rent
simplification process, and the number of units under contract to low-income
households.
We reviewed 17 annual reports from the Moving to Work agencies located within
Region 3’s jurisdiction for the period 2007 through 2012. The agencies reported
benchmarks, metrics, and performance information. However, HUD did not
3
Public and Indian Housing Notice 2013-02, Baseline Methodology for Moving to Work Public Housing Agencies,
dated January 10, 2013
8
verify the accuracy of the self-reported information. As stated above, HUD
officials stated that HUD planned to establish a means of verifying the accuracy
of self-reported information, either through audits or during annual site visits.
However, it had not finalized such procedures.
HUD Failed To Perform
Critical Annual Program Risk
Assessments
HUD did not perform annual program risk assessments in accordance with its
own policy. HUD Handbook 1840.1 required program offices to perform an
annual risk assessment of their programs or administrative functions using a
standard risk-assessment worksheet. HUD officials informed us that they did not
perform annual program risk assessments because they were not aware of the
requirement. Because HUD did not perform annual risk assessments, it lacked
assurance that it properly identified and addressed risks that might prevent
participating agencies from meeting program goals and statutory requirements.
HUD had taken steps to perform a program risk assessment. It recently
performed an internal review of potential program risk areas. It planned to
develop a basic risk assessment based on the internal review. The basic risk
assessment would eventually feed into the Office of Public and Indian Housing’s
overall risk assessment. However, the basic risk assessment had not been
finalized.
Prior GAO and OIG Reports
Disclosed Similar Issues
Prior HUD Office of Inspector General (OIG) and GAO audit reports on the
program disclosed that HUD lacked (1) programwide performance measures to
determine whether overall program objectives were met and (2) policies and
procedures to verify the reliability of agencies’ self-reported data. GAO also had
previously reported that HUD failed to perform required program risk
assessments. 4 We determined that HUD was working on resolving audit
recommendations related to these findings. In conjunction with its audit
resolution efforts, HUD had implemented revised reporting requirements for
agencies as discussed in the paragraph below. Since some of our audit findings
were consistent with findings from prior reports, HUD should continue its efforts
to ensure that it has fully addressed all audit recommendations related to the
program.
4
See the Follow-up on Prior Audits section of this report for details on prior HUD OIG and GAO audits.
9
HUD Revised Its Standard
Reporting Requirements
HUD Form 50900 specifies the elements that each agency must provide in its
annual plans and reports. 5 It requires information such as an overview of the
program goals and objectives for the year; planned capital expenditures by
development; proposed activities requesting HUD approval with baselines,
benchmarks, and metrics to assess outcomes; planned sources and uses of funds;
and a description of planned or ongoing agency-directed evaluations of the
demonstration. HUD uses the information to make determinations about the
program and respond to congressional and other inquiries regarding program
outcomes and positive practices learned throughout the program demonstration.
HUD revised Form 50900 in response to the prior GAO and HUD OIG reports.
In May 2013, OMB approved the revised form. The form includes standard
metrics for agencies to report on the progress of proposed and approved activities.
The standard metrics are organized to assist HUD in determining agencies’
compliance with the three statutory program objectives.
HUD’s updated reporting requirements will improve its program oversight.
However, it needs to implement a formal policy to develop programwide
performance indicators using results from the data it collects from agencies. It
also needs to complete its efforts to establish and implement policies to verify
agencies’ self-reported information to ensure the accuracy of data on which it
relies for various purposes.
Conclusion
HUD needs to improve its program oversight to ensure that participant agencies
meet statutory program goals and requirements. Also, since there are indications
that Congress is considering expanding the program, HUD needs to ensure that it
has implemented effective evaluation processes before program expansion. We
included four of GAO’s recommendations in our audit because they are
significant to the success of this program. In about 14 years, HUD has not shown
that this demonstration program has met its objectives of achieving cost
effectiveness, incentivizing families to become self-sufficient, and increasing
housing choices for low-income families. GAO is reviewing HUD’s revised
reporting requirements in response to its prior report. HUD can provide us a copy
of its planned actions communicated to GAO rather than developing separate
management decisions for these four recommendations.
5
Attachment B to the amended and restated Moving to Work agreement between HUD and Moving to Work
agencies
10
Recommendations
We recommend that the Deputy Assistant Secretary for Public Housing
Investments
1A. Develop programwide performance indicators based on the results from
agencies’ self-reported data.
1B. Ensure that the program is evaluated in accordance with the standard
program agreement and program monitoring policies.
1C. If Congress expands the program, delay adding participants to the program
until programwide performance indicators are in place and program
evaluation results warrant program expansion.
1D. Continue to develop and implement procedures to verify agencies’
compliance with key statutory requirements.
1E. Continue to develop and implement procedures to verify the accuracy of
agencies’ self-reported performance data.
1F. Ensure that annual risk assessments are completed as required.
11
SCOPE AND METHODOLOGY
We conducted the audit from November 2012 through June 2013 primarily at our office located
in Philadelphia, PA. The audit covered the period October 2007 through September 2012 but
was expanded as necessary to accomplish our objective.
To accomplish our objective, we
• Reviewed relevant background information, including prior HUD OIG audit reports and
GAO reports.
• Interviewed headquarters staff of HUD’s Office of Public Housing Investments and GAO
personnel.
• Reviewed applicable HUD rules, handbooks, notices, and guidance.
• Reviewed HUD’s organizational chart related to the program.
• Reviewed program agreements, annual plans, and reports for four agencies in Region 3.
• Reviewed site visit reports for four agencies in Region 3.
• Reviewed HUD’s annual performance plans for fiscal years 2011 and 2012.
• Reviewed applications and related evaluations for the four agencies HUD recently
selected to participate in the program.
• Reviewed proposed legislation to expand the program.
We nonstatistically selected as our sample four housing agencies within Region 3: the Housing
Authority of Baltimore City, the District of Columbia Housing Authority, the Philadelphia
Housing Authority, and the Housing Authority of the City of Pittsburgh.
We considered data in HUD’s Public and Indian Housing Information Center database. We used
the data obtained by HUD to determine whether agencies substantially served the same number
of families as if they had not participated in the program. The data did not materially affect our
results; thus, we considered the data adequate for our purposes.
We conducted the audit in accordance with generally accepted government auditing standards.
Those standards require that we plan and perform the audit to obtain sufficient, appropriate
evidence to provide a reasonable basis for our findings and conclusions based on our audit
objective. We believe that the evidence obtained provides a reasonable basis for our findings
and conclusions based on our audit objective.
12
INTERNAL CONTROLS
Internal control is a process adopted by those charged with governance and management,
designed to provide reasonable assurance about the achievement of the organization’s mission,
goals, and objectives with regard to
• Effectiveness and efficiency of operations,
• Reliability of financial reporting, and
• Compliance with applicable laws and regulations.
Internal controls comprise the plans, policies, methods, and procedures used to meet the
organization’s mission, goals, and objectives. Internal controls include the processes and
procedures for planning, organizing, directing, and controlling program operations as well as the
systems for measuring, reporting, and monitoring program performance.
Relevant Internal Controls
We determined that the following internal controls were relevant to our audit
objective:
• Policies and procedures – Controls that HUD has implemented to ensure that
housing agencies meet statutory goals and requirements.
We assessed the relevant controls identified above.
A deficiency in internal control exists when the design or operation of a control does
not allow management or employees, in the normal course of performing their
assigned functions, the reasonable opportunity to prevent, detect, or correct (1)
impairments to effectiveness or efficiency of operations, (2) misstatements in
financial or performance information, or (3) violations of laws and regulations on a
timely basis.
Significant Deficiency
Based on our review, we believe that the following item is a significant deficiency:
• HUD lacked controls to allow it to determine whether housing agencies met
statutory goals and requirements.
13
FOLLOW-UP ON PRIOR AUDITS
Moving to Work
Demonstration: Opportunities
Exist to Improve Information
and Monitoring - Audit Report
GAO-12-490, Dated April 19,
2012
GAO audited the program because Congress was considering expanding the
program and asked GAO to examine the program’s success in addressing the
three primary program objectives, HUD’s monitoring efforts, and the potential
benefits of and concerns about expansion. GAO found that opportunities existed
to improve HUD’s methods for evaluating the program. GAO recommended that
HUD (1) improve its guidance on reporting performance information, (2) develop
a plan for identifying and analyzing standard performance data, (3) establish
programwide performance indicators, (4) systematically identify lessons learned,
(5) clarify key terms, (6) implement a process for assessing compliance with
statutory requirements, (7) perform annual assessments of program risks, and (8)
verify the accuracy of self-reported data.
HUD generally or partially agreed with seven of the recommendations. HUD
disagreed with the recommendation that it establish programwide performance
indicators. However, GAO believed performance indicators were critical to
demonstrating program results and, thus, maintained its recommendation. In June
2013, GAO testified to Congress that HUD had revised agency reporting
requirements since it issued its report. GAO is reviewing the new requirements.
HUD expects that its revised reporting requirements will address
recommendations 1, 2, 5, 6, and 8. Although it initially disagreed with the
recommendation to implement programwide performance indicators, HUD stated
that its goal was to implement the recommendation by the end of fiscal year 2014.
Also, as stated in the report, HUD had initiated steps to perform a program risk
assessment and was working on determining an appropriate process to verify
agencies’ self-reported performance data. However, as of the date of this report,
all of the recommendations remained open.
14
Additional Details To
Supplement Our Report on
HUD’s Fiscal Years 2012 and
2011 Financial Statements -
Audit Report 2013-FO-0003,
Dated November 15, 2012
HUD OIG audited HUD’s consolidated financial statements in accordance with
the Chief Financial Officers Act of 1990. Among the deficiencies found, HUD
did not establish monitoring tools necessary to ensure that the Moving to Work
statutory goals were met, finalize or implement performance measures and the
methodology needed to evaluate the performance of agencies in achieving the
program objectives, and establish adequate tools to verify the reliability of
reported program data.
HUD OIG made the following related recommendations: (1) develop, implement,
and document methodologies to calculate and track performance measures to
enable comparability of data among agencies and ensure the reliability of reported
data; (2) develop, implement, and document standardized reporting requirements
for the program data and results for all participant agencies; (3) update the
program plan and report review procedures to include steps to verify the
reliability of presented data against HUD systems and retain all supporting
documentation as evidence of controls performed; and (4) ensure that the staffing
and funding levels for the program office are adequate to provide proper oversight
of the program.
To resolve the audit recommendations, HUD proposed actions with target dates
ranging from December 2013 to December 2014. HUD OIG agreed to HUD’s
proposed actions and target dates. All of the recommendations were open as of
the date of this report.
15
Appendix A
AUDITEE COMMENTS AND OIG’S EVALUATION
Ref to OIG Evaluation Auditee Comments
Comment 1
Comment 2
Comment 3
Comment 4
Comment 1
16
Comment 5
Comment 6
Comment 7
Comment 8
17
Comment 9
18
Comment 10
Comments 1,
2, 3, & 4
19
OIG Evaluation of Auditee Comments
Comment 1 We conducted the audit in accordance with generally accepted government
auditing standards. Those standards require that we communicate the results of
the audit. Accordingly, the report discusses the audit findings and related
recommendations.
Comment 2 As stated in the section on prior audit reports, GAO made eight recommendations
related to the program. Four of the recommendations are included in this report.
These four recommendations include the recommendation related to the need for
programwide performance indicators, on which HUD initially disagreed with
GAO. As acknowledged in the report, HUD is working on resolving most of
these recommendations. However, we believe it is important to make these same
recommendations because they have not been completely resolved, and about 14
years after the demonstration program was implemented to achieve cost
effectiveness, incentivize families to become self-sufficient, and increase housing
choices for low-income families, HUD has yet to demonstrate that the program
has met the purposes for which it was established. To reduce the reporting
burden, HUD can provide us a copy of its planned actions in place with GAO and
we will track the resolution of these recommendations through GAO’s report. We
have revised the conclusion in the finding to clarify this in the report.
Comment 3 We are encouraged that HUD agreed with the recommendations in OIG Audit
Report 2013-FO-0003 and is working on resolving those recommendations. We
are also pleased that HUD, for the most part, agrees with our audit
recommendations and is only concerned that they appear duplicative. However,
while the audit recommendations in this report appear to mirror recommendations
from the prior audit referenced, the recommendations in this report are valid as
explained in comments 5 and 9 below.
Comment 4 We have considered management’s comments and determined that all of the audit
recommendations are appropriately directed to the Office of Public and Indian
Housing. However, we have updated recommendation 1C, as explained in
comment 7, to reflect reasonable action that HUD can take to address the related
concern discussed in the audit finding.
Comment 5 The referenced recommendation from the prior OIG audit is not the same as the
recommendation in this report or the recommendation GAO issued.
Implementing performance measures to enable comparability of data among
agencies and ensuring the reliability of reported data is not the same as
implementing programwide performance indicators to determine whether the
program as a whole is meeting the purposes for which was established. HUD’s
action plan for implementing the recommendation from the prior OIG audit did
not address the need for developing programwide performance indicators. HUD
needs to establish programwide performance indicators that will allow it to
determine whether the program has produced the intended results, particularly as
20
Congress considers whether to expand the program. Therefore, the
recommendation in the prior OIG audit report should have a different
management decision and needs to be addressed separately from GAO’s
recommendation.
Comment 6 We acknowledged the Office of Public and Indian Housing’s efforts to obtain
funding for program evaluation. The Office of Public and Indian Housing is
responsible for administering the program. Also, it was the Office of Public and
Indian Housing that established, as a control, a plan to evaluate the program in
2011 and 2014 to determine whether agencies should continue in the program.
We believe this is a critical control that the Office of Pubic and Indian Housing
should implement. As stated in the report, the Office of Policy Development and
Research told OIG that it had reserved funds for a program evaluation in 2014 and
that it planned to reserve additional related funding in fiscal year 2014. Since it is
responsible for program administration, the Office of Public and Indian Housing
should coordinate with the Office of Policy Development and Research to ensure
that program evaluation occurs in accordance with its program monitoring
policies or determine another way to evaluate the program in fiscal year 2014.
We updated the recommendation to reflect that the Office of Public and Indian
Housing is responsible for ensuring that the program is evaluated.
Comment 7 We recognize that HUD does not have the authority to expand the program.
However, in a program report to Congress in August 2010, HUD recommended
that the program be expanded by up to twice its size, or from 35 to 60
participating agencies. 6 Also, in its fiscal year 2014 budget request presented to
Congress in April 2013, HUD proposed a substantial expansion of the program to
high-capacity agencies. We have updated the recommendation to emphasize that
the program must be evaluated before expansion and to delay adding program
participants until it has implemented programwide performance indicators and
determined that program evaluation results warrant program expansion.
Comment 8 We evaluated HUD’s revised Form 50900 during the audit. Although the form
includes standard metrics that were organized to assist HUD in determining
agencies’ compliance with the statutory requirements, agencies will continue to
self-report data to support compliance. However, HUD has not determined how it
will verify the data for accuracy as required by recommendation 1E. During the
audit, HUD officials stated that HUD planned to establish a means of verifying
the accuracy of self-reported data, either through audits or during annual site
visits. However, until HUD finalizes such procedures, the verification to
determine compliance will not be reliable. Therefore, the audit recommendation
will need to remain open until recommendation 1E is completed.
Comment 9 As stated above, we evaluated HUD’s revised Form 50900 during the audit and
determined that HUD needs to implement additional verification procedures.
6
Moving to Work: Interim Policy Applications and the Future of the Demonstration, August 2010
21
The referenced recommendation from the prior OIG audit asked HUD to
implement steps to verify the reliability of presented data against its systems. Our
recommendation and GAO’s recommendation did not ask for HUD to only
implement a verification process for data it has in its systems. HUD still needs to
address how it will verify self-reported information that it does not have in its
systems. Therefore, the recommendation in the prior OIG audit report needs to be
addressed separately from GAO’s recommendation. Although HUD officials
stated that HUD planned to verify agencies’ self-reported data through audits or
during annual site visits, HUD had not implemented or finalized such procedures.
Comment 10 In the GAO report, HUD agreed that it needed to conduct an annual program risk
assessment and described its plans to develop a formal risk-based strategy for
monitoring. During our audit, HUD officials stated that HUD had not performed
annual program risk assessments. According to HUD officials, the Office of
Public and Indian Housing recently performed an internal review of potential
program risk areas and planned to develop a basic risk assessment based on the
internal review.
During the exit conference, HUD officials stated that the Office of Public and
Indian Housing performed overall risk assessments that would cover the Moving
to Work program. After we received HUD’s response to the draft report, we
asked it to provide documentation on the joint assessments it stated it had
performed. HUD provided a writeup on issues related to five participant agencies
it had classified as “at risk” as of October 2012. This writeup did not address the
requirement for an annual program risk assessment as outlined in HUD Handbook
1840.1 and described in the report.
During the audit, the HUD Office of the Chief Financial Officer confirmed that
each program office is required to perform an annual risk assessment of its
programs and maintain related supporting documentation. Given the specific
statutory goals and requirements related to the program and the flexibility
awarded to participant agencies, we agree with GAO’s recommendation and
HUD’s initial response, which recognized that annual program risk assessments
were necessary. Accordingly, HUD needs to complete the program risk
assessment it said it had started and ensure that it completes annual program risk
assessments.
22
HUD's Oversight of Its Moving to Work Demonstration Program Needs Improvement
Published by the Department of Housing and Urban Development, Office of Inspector General on 2013-09-27.
Below is a raw (and likely hideous) rendition of the original report. (PDF)