oversight

FHFA's Adoption of Clear Guidance on the Review of the Enterprises' Internal Audit Work When Assessing the Sufficiency of Remediation of Serious Deficiencies Would Assist FHFA Examiners

Published by the Federal Housing Finance Agency, Office of Inspector General on 2018-03-28.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

            Federal Housing Finance Agency
                Office of Inspector General




FHFA’s Adoption of Clear Guidance
  on the Review of the Enterprises’
Internal Audit Work When Assessing
  the Sufficiency of Remediation of
  Serious Deficiencies Would Assist
          FHFA Examiners




 Evaluation Report • EVL-2018-003 • March 28, 2018
                 Executive Summary
                 The Federal Housing Finance Agency (FHFA or Agency), through its
                 Division of Enterprise Regulation (DER), is responsible for the supervision
                 of Fannie Mae and Freddie Mac (collectively, the Enterprises). When
                 conducting its supervisory responsibilities, DER may identify significant
                 deficiencies regarding risk management, risk exposure, or violations of laws,
EVL-2018-003     regulations, or orders affecting the performance or condition of a regulated
                 entity. Among these “adverse examination findings” are matters requiring
March 28, 2018   attention (MRAs), which consist of either “critical supervisory matters (the
                 highest priority) which pose substantial risk to the safety and soundness of the
                 regulated entity” or “deficiencies,” which if not corrected, could “escalate and
                 potentially negatively affect” the regulated entity.

                 FHFA expects the Enterprises to take corrective action to remediate MRAs,
                 and DER is responsible for monitoring the remediation process. When
                 Enterprise management determines that it has completed remediation of an
                 MRA, FHFA requires the Enterprise’s internal audit (IA) functions to review
                 the corrective action and “validate” that remediation has been fully
                 implemented as intended. The Enterprise then submits a closure package to
                 DER that contains documentation of IA’s validation work. Based on a review
                 of the closure package, and any other follow-up examination work that DER
                 may conduct, DER determines whether the MRA has been satisfactorily
                 addressed and notifies the Enterprise of its determination.

                 In a companion evaluation issued today, we reviewed DER’s guidance and
                 standards for reliance on the Enterprises’ IA functions when examiners assess
                 the remediation of MRAs. See FHFA Requires the Enterprises’ Internal
                 Audit Functions to Validate Remediation of Serious Deficiencies but Provides
                 No Guidance and Imposes No Preconditions on Examiners’ Use of that
                 Validation Work, EVL-2018-002, available online at
                 www.fhfaoig.gov/reports/auditsandevaluations. In this evaluation, we
                 reviewed DER’s practices for closing MRAs in order to understand (1) the
                 extent to which examiners accepted, relied on, or otherwise used IA’s
                 validation work in their assessment of the adequacy of MRA remediation and
                 (2) whether they conducted independent assessments of the adequacy of the
                 remediation. We reviewed key documentation for a sample of 22 out of 78
                 MRAs issued to Fannie Mae and Freddie Mac and closed by DER between
                 January 2015 and October 2017. We also interviewed the examination
                 managers and examiners who were responsible for closing these MRAs.

                 The examination managers and examiners we interviewed offered varying
                 explanations of the difference between relying on and leveraging IA’s
                 validation work, but provided no clear distinction between the two. They also
                 expressed no uniform view on whether they were expected to conduct any
                 testing as part of their assessment of MRA remediation. When DER
                 examiners specifically reported that testing of the sufficiency of MRA
                 remediation was conducted for the 22 MRAs in our sample, we found that
                 the examiners generally relied on the validation testing conducted by the
                 Enterprise’s IA function. We determined, from our review of examiners’ key
                 workpapers for the 22 MRAs in our sample, that almost half (9 of 22) cited
EVL-2018-003     testing completed by the IA function and did not reflect that the examiners
                 performed their own testing. For 11 of the 22 sampled MRAs, we found that
March 28, 2018
                 the examiners’ key workpapers reflected review of IA validation work, but did
                 not specifically identify testing performed either by IA or by the examiners.
                 For 2 of the 22 sampled MRAs, we found that the examiners performed their
                 own testing; in one of these instances it appears the examiner conducted
                 testing for a component of MRA remediation and relied on IA’s testing for
                 another component.

                 In the companion evaluation issuing today, we found that FHFA, unlike the
                 Federal Reserve, does not require its examiners to conclude on the overall
                 effectiveness of the Enterprises’ IA functions as a predicate to use of IA work.
                 As a result, its examiners may lack a sufficient basis to determine whether,
                 or to what extent, to use IA’s validation work, and FHFA has not issued
                 guidance on the level of assurance of the competency, objectivity, reliability,
                 and quality of IA’s validation work needed before the IA work can be used.

                 Because FHFA only issues MRAs for the most significant deficiencies,
                 determinations to close MRAs should be based on the examiners’ independent
                 assessments of the Enterprises’ remedial actions. Current FHFA guidance
                 directs examiners to independently review and assess the documents in the
                 Enterprise’s closure package, including some independent review or
                 assessment of documentation provided by the Enterprise’s business unit
                 and/or IA. As FHFA does not identify the steps that examiners should
                 undertake to assess the sufficiency of MRA remediation, we found that
                 examination managers and examiners have broad discretion in determining
                 the scope of their independent assessment of the adequacy of the remedial
                 actions. We determined, from our review of key examiner workpapers for the
                 22 MRAs in our sample, that the workpapers reflected some independent
                 assessment of the sufficiency of management’s remediation activities and/or
                 IA’s validation work for nearly all of the 22 MRAs in our sample, although
                 the scope of that assessment varied among examiners.

                 To promote consistency among examiners in determinations on MRA
                 closures, we recommended, in our companion evaluation, that FHFA provide
                 clear direction to examiners on whether, or the circumstances under which,
                 they may rely on information, analyses, or conclusions from IA when
                 assessing the sufficiency of MRA remediation. We recommend, based on our
                 findings from this evaluation, that FHFA adopt guidance that identifies the
                 work steps that should be included in examiners’ independent assessments of
                 IA’s work when assessing the sufficiency of MRA remediation and specifies
                 the conditions under which examiner testing is expected. FHFA agreed with
                 our recommendation.
EVL-2018-003
                 This report was prepared by Jacob Kennedy, Senior Investigative Evaluator,
March 28, 2018   and Philip Noyovitz, Senior Auditor. We appreciate the cooperation of FHFA
                 staff, as well as the assistance of all those who contributed to the preparation
                 of this report.

                 This report has been distributed to Congress, the Office of Management and
                 Budget, and others and will be posted on our website, www.fhfaoig.gov.




                 Angela Choy
                 Assistant Inspector General for Evaluations
TABLE OF CONTENTS ................................................................
EXECUTIVE SUMMARY .............................................................................................................2

ABBREVIATIONS .........................................................................................................................6

BACKGROUND .............................................................................................................................7
      MRA Issuance and Follow-up Process .....................................................................................7

FACTS AND ANALYSIS.............................................................................................................10
      DER Expects its Examiners to Independently Analyze MRA Closure Packages but
      Provides No Guidance on the Elements of this Independent Analysis ..................................10
      Unlike Other Federal Financial Regulators, FHFA Has Not Concluded on the
      Overall Effectiveness or Strength of the IA Functions ...........................................................11
      DER Expects its Examiners to Review the Validation Work Performed by an
      Enterprise’s IA but Provides No Guidance on the Use of the Validation Work by its
      Examiners ...............................................................................................................................12
      Where DER Examiners Reported in Their Workpapers that Testing Was Conducted
      for MRAs In Our Sample, Our Review Found that These Examiners Generally
      Relied on the Validation Testing Conducted by IA................................................................13
      Review of Key Workpapers for the 22 MRAs in the Sample Found Evidence of
      Independent Assessments by DER Examiners of the Enterprise’s Closure Packages
      for Nearly All of the Sampled MRAs .....................................................................................14

FINDINGS .....................................................................................................................................16

CONCLUSIONS............................................................................................................................16

RECOMMENDATION .................................................................................................................17

FHFA COMMENTS AND OIG RESPONSE ...............................................................................17

OBJECTIVE, SCOPE, AND METHODOLOGY .........................................................................18

APPENDIX: FHFA MANAGEMENT RESPONSE ....................................................................19

ADDITIONAL INFORMATION AND COPIES .........................................................................20




                                            OIG • EVL-2018-003 • March 28, 2018                                                               5
ABBREVIATIONS .......................................................................

AB                    Advisory Bulletin

DER                   Division of Enterprise Regulation

EIC                   Examiner-in-charge

Enterprises           Fannie Mae and Freddie Mac, collectively

Fannie Mae            Federal National Mortgage Association

Federal Reserve       Board of Governors of the Federal Reserve System and Reserve Banks

FHFA or Agency        Federal Housing Finance Agency

Freddie Mac           Federal Home Loan Mortgage Corporation

IA                    Internal Audit

MRA                   Matter Requiring Attention

OCC                   Office of the Comptroller of the Currency

OIG                   Federal Housing Finance Agency Office of Inspector General

OPB                   Operating Procedures Bulletin




                          OIG • EVL-2018-003 • March 28, 2018                          6
BACKGROUND ..........................................................................

Since 2008, FHFA has operated as both regulator and conservator of Fannie Mae and Freddie
Mac and regulator of the Federal Home Loan Banks. DER is responsible for supervising the
Enterprises and does so through targeted examinations and ongoing monitoring activities.
According to FHFA, targeted examinations enable examiners to conduct a deep or
comprehensive assessment of selected areas of high importance or risk, while the purpose of
ongoing monitoring is to analyze real-time information and to use those analyses to identify
Enterprise practices and changes in an Enterprise’s risk profile that may warrant supervisory
attention. DER also conducts ongoing monitoring or targeted examinations to assess the
Enterprises’ remediation of serious deficiencies, such as MRAs, and the Enterprises’
adherence to supervisory guidance and conservatorship directives.

MRA Issuance and Follow-up Process
During an ongoing monitoring activity or a targeted examination, DER may identify
significant deficiencies related to risk management, risk exposure, or violations of laws,
regulations, or orders affecting the performance or condition of a regulated entity. These
identified deficiencies are known as “adverse examination findings.” FHFA classifies
such examination findings into one of three categories: (1) MRAs, (2) Violations, or
(3) Recommendations. FHFA has two categories of MRAs: (a) “critical supervisory matters
(the highest priority) which pose substantial risk to the safety and soundness of the regulated
entity” and (b) “deficiencies,” which if not corrected, could “escalate and potentially
negatively affect” the regulated entity. 1 After DER issues an MRA to an Enterprise, it
requires the Enterprise to prepare and submit a written remediation plan to FHFA. DER
reviews the remediation plan and determines whether the proposed corrective actions are
sufficient to address the MRA. If DER considers the plan acceptable, the examiner-in-charge
(EIC) notifies the Enterprise in writing of DER’s non-objection to the plan. 2 During the
course of remediation, DER examiners are expected to monitor the Enterprise’s progress
through ongoing monitoring or targeted examinations.


1
 FHFA, Advisory Bulletin (AB) 2017-01, Classifications of Adverse Examination Findings, at 1-2 (Mar. 13,
2017) (online at www.fhfa.gov/SupervisionRegulation/AdvisoryBulletins/Pages/Classifications-of-Adverse-
Examination-Findings.aspx). The MRAs reviewed for this evaluation were issued prior to the release of AB
2017-01. During the review period for this evaluation, AB 2012-01 was in force. Although there were
changes to the definition of “MRA,” they are not material for purposes of this report.
2
  If the plan is not acceptable, the EIC issues an objection letter that identifies the concerns and requests a
revised remediation plan. The EIC may also issue a non-objection with conditions letter that states that if
certain conditions are met, the corrective actions proposed in the remediation plan, if implemented, would
address the concerns that led to the finding.




                                     OIG • EVL-2018-003 • March 28, 2018                                          7
When the Enterprise’s management determines that MRA remediation has been completed,
the Enterprise’s IA function reviews and “validates” that the remediation plan was
implemented as intended and that remediation is complete. 3 The Enterprise is expected to
prepare and submit a closure package to DER that includes the validation work performed by
the IA function.

DER’s examiners are expected to review and assess the documents in the closure package
along with any other follow-up work performed, and summarize and document their review
and conclusions in the form of an analysis or summary memorandum. DER, in its response
to a recommendation in OIG’s 2016 evaluation report, explained to us that it expects its
examiners to independently analyze MRA closure packages, and referred to its Examination
Manual, which states that “[e]xaminers support their conclusions through testing and
independent analysis of Enterprise information, data, documents, and other materials obtained
from management or other sources.” 4 According to DER, this independent analysis would
include some independent review or assessment of documentation provided by the
Enterprise’s business unit and/or IA to support an examiner’s recommendation to close an
MRA.

The EIC determines, based on the examiners’ work, whether the MRA has been
“satisfactorily addressed.” Finally, the results of DER’s assessment are communicated in
writing to the Enterprise.

In a companion evaluation issued today, we reviewed FHFA’s existing guidance governing
the respective roles of the Enterprise’s IA 5 and FHFA examination staff in assessing whether
MRAs have been satisfactorily remediated. 6 In that report, we observed that FHFA’s
guidance sets forth its expectations that IA assess and validate management’s remediation
of deficiencies, but does not address whether, or the circumstances under which, FHFA

3
  DER adopted this process in 2013 when it issued an operating procedures bulletin in April 2013 that called
for an Enterprise’s IA function, or other independent third party, to validate that management’s MRA
remediation was complete and consistent with the remediation plan. See DER, Operating Procedures Bulletin,
Matters Requiring Attention (MRA) Process (Apr. 23, 2013) (2013-DER-OPB-01). The 2013 operating
procedures bulletin was superseded in 2017, but the guidance governing the review and closure process did not
change materially.
4
 FHFA, Examination Manual, at 22 (Dec. 2013) (online at
www.fhfa.gov/SupervisionRegulation/Documents/ExaminationProgramOverview.pdf).
5
 FHFA, Advisory Bulletin 2016-05, Internal Audit Governance and Function, at 15 (Oct. 7, 2016) (online at
www.fhfa.gov/SupervisionRegulation/AdvisoryBulletins/Pages/Internal-Audit-Governance-and-
Function.aspx).
6
 See OIG, FHFA Requires the Enterprises’ Internal Audit Functions to Validate Remediation of Serious
Deficiencies but Provides No Guidance and Imposes No Preconditions on Examiners’ Use of that Validation
Work (Mar. 28, 2018) (EVL-2018-002) [hereinafter EVL-2018-002].




                                   OIG • EVL-2018-003 • March 28, 2018                                          8
examiners may rely on, accept, or otherwise use information, analyses, or conclusions
provided by an Enterprise’s IA function to determine whether an Enterprise has satisfactorily
addressed an MRA. 7

In this evaluation, we reviewed documentation for a sample of 22 out of 78 MRAs closed by
DER between January 2015 and October 2017 that were issued to Fannie Mae and Freddie
Mac, 8 and interviewed the examination managers and examiners who were responsible for
these MRAs. 9 Our objective was to understand the extent to which (1) examiners accepted,
relied on, or otherwise used IA’s validation work in their assessment of the adequacy of MRA
remediation and (2) whether they conducted independent assessments of the adequacy of the
remediation. 10




7
 In its management response to our companion report, FHFA agreed with our recommendation to revise its
guidance to provide clear direction to examiners on whether, or the circumstances under which, its examiners
may rely on information, analyses, or conclusions provided by an Enterprise’s IA function when assessing the
adequacy of MRA remediation and plans to issue the revised guidance by October 31, 2018.
8
  The sample of 22 MRAs represents 28% of all MRAs closed between January 2015 and October 2017, our
review period.
9
  In a previous OIG evaluation, we reviewed DER’s workpapers supporting closure of eight MRAs and
identified three instances where DER examiners appeared to have accepted an Enterprise’s IA work to
close MRAs without documented evidence of independent review of the adequacy of remediation. We
recommended that DER examiners conduct and document an independent analysis of the adequacy and
sustainability of the Enterprise’s remediation activity, or where appropriate, independently analyze the
adequacy of the Enterprise’s IA validation work. DER agreed with our recommendation and committed to
“amend its internal guidance to provide that examiners should assess any . . . closure package, or internal
audit validation of remediation activity and should include in the summary memorandum the results of that
assessment.” See OIG, FHFA’s Inconsistent Practices in Assessing Enterprise Remediation of Serious
Deficiencies and Weaknesses in its Tracking Systems Limit the Effectiveness of FHFA’s Supervision of the
Enterprises, at 17-21 (July 14, 2016) (EVL-2016-007) (online at www.fhfaoig.gov/Content/Files/EVL-2016-
007.pdf).
10
   In two separate audit reports issued today, OIG reviewed FHFA’s closure of cybersecurity MRAs that were
issued to the Enterprises. See OIG, As Allowed by its Standard, FHFA Closed Three Fannie Mae
Cybersecurity MRAs after Independently Determining the Enterprise Completed its Planned Remedial Actions
(Mar. 28, 2018) (AUD-2018-007) and OIG, FHFA Failed to Ensure Freddie Mac’s Remedial Plans for a
Cybersecurity MRA Addressed All Deficiencies; as Allowed by its Standard, FHFA Closed the MRA after
Independently Determining the Enterprise Completed its Planned Remedial Actions (Mar. 28, 2018) (AUD-
2018-008) (online at https://www.fhfaoig.gov/reports/auditsandevaluations).



                                   OIG • EVL-2018-003 • March 28, 2018                                         9
FACTS AND ANALYSIS ...............................................................

DER Expects its Examiners to Independently Analyze MRA Closure Packages but
Provides No Guidance on the Elements of this Independent Analysis

Under FHFA’s Advisory Bulletin 2012-01, which was in effect when DER closed most of the
MRAs in our sample, 11 examiners’ remediation follow-up “should include an assessment of
materials provided by the [Enterprises], discussions with the responsible parties at the
[Enterprises], and testing, if appropriate, to determine progress against a remediation plan.” 12
Also in effect at the time was guidance issued by DER, Operating Procedures Bulletin (OPB)
2013-DER-OPB-01, Matters Requiring Attention (MRA) Process, which directed:

        [u]pon completion of the [remediation] plan and management’s determination that
        the respective Enterprise has remediated the MRAs, internal audit . . . will review
        and “validate” that the [remediation] plan was implemented as intended and that
        the remediation is complete.

That OPB further instructed: “FHFA will assess the remediation of the MRA through on-
going monitoring or related targeted examination work. If additional reviews are needed,
examiners will conduct the necessary reviews to validate the remediation.” 13 Beyond these
two guidance documents, FHFA did not specify the work steps examiners were expected to
follow to assess the sufficiency of MRA remediation.

As discussed, DER represented to us that it expects its examiners to independently assess
MRA closure packages. According to DER, this independent analysis would include some
independent review or assessment of documentation provided by the Enterprise’s business
unit and/or IA to support an examiner’s recommendation to close an MRA.

The Deputy Director of DER acknowledged to us that guidance in DER’s current OPB is
“sparse” on examiners’ use of IA’s validation work. She stressed that the EIC, not individual
DER examiners, determines whether an MRA has been satisfactorily addressed, based on
review of the examiners’ workpapers. DER requires its EICs to review and approve certain
workpapers prior to closing an MRA, including procedures documents and analysis or


11
  Three of the MRAs in our sample were closed after FHFA rescinded and replaced AB 2012-01 with AB
2017-01.
12
   When FHFA rescinded AB 2012-01, guidance related to remediation follow-up by examiners was moved to
a 2017 DER OPB; however, the 2017 OPB does not mention testing as part of remediation follow-up.
13
   DER does not define “assess” or “review” in the OPB and is silent on the examiner’s responsibilities with
regard to IA’s validation work.



                                   OIG • EVL-2018-003 • March 28, 2018                                         10
summary memoranda. The Deputy Director of DER advised us that, in the event that an EIC
found no evidence of examiners’ independent assessment in the review of workpapers prior to
closing an MRA, she would expect the EIC to send back the closure recommendation to the
examiners. In 2016, DER issued guidance that directs independent quality control reviews to
be conducted of MRA remediation letters to provide reasonable assurance that examination
work performed by examiners met DER standards and FHFA guidance. The Deputy Director
of DER advised that she expects the closure recommendation to be sent back if it does not
pass quality control review. 14

Unlike Other Federal Financial Regulators, FHFA Has Not Concluded on the Overall
Effectiveness or Strength of the IA Functions

As we discussed in our companion report, 15 the Board of Governors of the Federal Reserve
System (Federal Reserve) and the Office of the Comptroller of the Currency (OCC) require
their examiners to reach conclusions on the overall effectiveness or strength of the IA
function of a regulated entity. We explained that the Federal Reserve permits its examiners
to rely on MRA follow-up work performed by the IA function of a regulated entity, provided
that prior examination work has rated the entity’s IA program as effective.

Unlike the Federal Reserve and the OCC, FHFA does not require its examiners to reach
conclusions on the overall effectiveness or strength of either Enterprise’s IA function. DER’s
Office of the Chief Accountant, which leads examinations of the IA functions of Fannie Mae,
Freddie Mac, and the Federal Home Loan Banks, advised us that DER examiners have not
conducted sufficient examination work to conclude on the overall effectiveness of the IA
function for either Enterprise, and DER’s Deputy Director informed us that DER had no plan
to do so when we interviewed her in January 2018. Because DER has not reached
conclusions on the overall effectiveness or strength of either Enterprise’s IA functions, its
examiners do not have that information to assist them when considering whether or to what
extent to use an Enterprise’s IA MRA validation work. However, FHFA, unlike the Federal
Reserve, has imposed no limitations on the use of IA validation work by DER examiners in
their assessments of MRA remediation.

The Chief Accountant and other FHFA officials reported to us that the Agency intends
to issue an updated examination module on internal audit in 2018, and the Chief Accountant




14
  DER issued guidance in 2016 for conducting independent quality control reviews “of certain supervisory
written products [including MRA remediation letters] to provide reasonable assurance that examination work
performed by examiners” met DER standards and FHFA guidance.
15
     See OIG, EVL-2018-002, supra note 6, at 18.



                                    OIG • EVL-2018-003 • March 28, 2018                                      11
advised that he expects the revised module to contain a work program sufficient to permit
examiners to reach a conclusion as to the overall effectiveness of IA.

DER Expects its Examiners to Review the Validation Work Performed by an
Enterprise’s IA but Provides No Guidance on the Use of the Validation Work by its
Examiners

According to the Basel Committee on Banking Supervision (of which OCC and the Federal
Reserve are members), “[b]ecause of the crucial role played by internal audit in assessing the
effectiveness of a bank’s overall control systems and processes, supervisors should assess the
internal audit function. This will influence their overall assessment of the bank and enable
them to determine the extent to which they will use the work of the internal audit function.” 16
The Basel Committee on Banking Supervision is a global standard setter for the prudential
regulation of banks.

In our companion report, 17 we explained that DER officials represented in writing to us
that DER examiners “[g]enerally . . . [do]not accept or rely [on] IA’s assessments of MRA
remediation without independent examiner work to support conclusions.” As discussed in
that report, FHFA provides no guidance to examiners on what they must do, or the conditions
that must exist, to rely on, accept, leverage, or otherwise use the data, analyses, or conclusions
contained in IA’s validation work. Accordingly, we asked examination managers and
examiners whether they relied on or used IA’s validation work, including testing, to close
MRAs.

Nearly all of the DER examination managers and examiners we interviewed reported that
they did not rely on but rather “leverage[d]” the validation work of IA when assessing the
adequacy of MRA remediation. Because DER had not concluded on the overall effectiveness
or strength of the Enterprise’s IA functions, DER examiners lacked assurance of the
competency, objectivity, reliability, and quality of the Enterprise IA functions in order to
“leverage” IA’s work.

The examiners with whom we spoke did not cite any particular policy or guidance that
provides a consistent process for them to use to gain assurance of the competency, objectivity,
reliability, and quality of IA’s validation work. Similarly, none invoked policy or guidance
that defined “leverage.” Several examination managers sought to explain the difference
between leveraging and relying on IA’s validation work. One examination manager
explained that reliance removes the examiner’s own independent judgment of IA’s validation

16
  See Basel Committee on Banking Supervision, The internal audit function in banks, at 17 (June 2012) (online at
www.bis.org/publ/bcbs223.pdf).
17
     See OIG, EVL-2018-002, supra note 6, at 17.



                                    OIG • EVL-2018-003 • March 28, 2018                                       12
work and implies that the examiners are taking IA’s word and not performing their own
assessment. “Leveraging,” according to this manager, is “taking IA’s work . . . within the
course of [the examiner’s] work.” In the view of another examiner, leveraging enables
examiners to assess the quality of IA’s work and verify the work by the business units before
using it.

Two examination managers explained that “reliance” on IA’s work meant that an examiner,
even after reviewing or assessing IA’s work, did not exercise his or her own independent
judgment. The same two examination managers and one other defined reliance to mean a
substitution of their own work with IA’s work. This third examination manager reported that,
absent guidance, he and other examiners conduct their own independent assessments. In his
view, examiners never used IA’s work as part of their own review of the MRA remediation.
Another examiner was not sure whether existing FHFA guidance permitted examiners to rely
on IA’s work.

These various explanations provide no clear distinction between reliance and leverage. As we
now discuss, our review of the workpapers found that examiners, in many instances, relied
solely on IA’s testing instead of performing their own testing.

Where DER Examiners Reported in Their Workpapers that Testing Was Conducted for
MRAs In Our Sample, Our Review Found that These Examiners Generally Relied on the
Validation Testing Conducted by IA

As noted, DER has not issued guidance on the permissible use of IA validation work by
its examiners in assessing MRA remediation. In its 2015 comments to a draft OIG report,
however, FHFA stated that examiners “can leverage the work of internal audit but cannot
rely on testing performed by [the Enterprise’s] Internal Audit.” Despite this position, the
examiners with whom we spoke offered a wide range of opinions on whether to use IA’s
testing. While some examiners explained to us that testing was part of IA’s validation work,
others expressed the view that the nature of the MRA drove whether testing by IA would
be appropriate. According to one examination manager, IA is expected to test in some form
and IA’s testing is considered to be validation. Some examination managers and examiners
explained to us that certain conditions must be met to leverage IA’s work, including IA’s
testing, to validate MRA remediation. For example, one examination manager said examiners
“are permitted to accept” IA’s testing if it makes sense and in many cases, they review IA’s
testing to determine its reasonableness before using IA’s results. That examination manager
also reported that DER examiners were not required to replicate IA’s testing. 18 Another
examination manager stated that examiners can “agree with” IA testing only if there is

18
  During an interview with OIG for a separate evaluation, FHFA’s Chief Accountant explained that there is
some expectation of independent performance or testing when relying on components of IA’s work.



                                  OIG • EVL-2018-003 • March 28, 2018                                       13
evidence that the examiner reviewed and is satisfied with IA’s testing. One examiner said he
would not replicate IA’s testing but instead look at the reasonableness of IA’s testing results.
Another examiner said DER would conduct its own testing during review of remediation of
an MRA for closure this year because, in that examiner’s view, IA does not have the expertise
to perform that testing.

We reviewed examiners’ key workpapers for the 22 MRAs in our sample to determine the
extent to which examiners conducted any testing when assessing remediation or relied on IA’s
testing of MRA remediation. 19 Our review found:

     •   For 9 of the 22 sampled MRAs (41 percent), the examiners’ workpapers cite IA’s
         testing and do not reflect that the examiners performed their own testing.

     •   For 11 of 22 sampled MRAs (50 percent), the examiners’ workpapers contain a
         review of IA’s validation, but did not specifically identify testing performed either by
         examiners or IA; for 2 of the 11 MRAs within this group, the workpapers explicitly
         state that no testing was performed by either IA or the examiners.

     •   For 2 of 22 sampled MRAs (9 percent), the workpapers show that the examiners
         performed their own testing, and it appears that in one of these instances the examiner
         conducted testing for a component of MRA remediation and relied on IA’s testing for
         another component of remediation.

In short, our review of the key DER workpapers for the 22 MRAs in our sample found that
when the workpapers specifically documented that testing was performed, the workpapers
reflected that the examiners generally relied on IA’s testing as part of DER’s assessment of
MRA remediation instead of performing their own testing.

Review of Key Workpapers for the 22 MRAs in the Sample Found Evidence of
Independent Assessments by DER Examiners of the Enterprise’s Closure Packages for
Nearly All of the Sampled MRAs

As discussed earlier, beyond FHFA’s advisory bulletin and DER’s OPB, neither FHFA nor
DER has specified the work steps examiners were expected to follow to assess the sufficiency
of MRA remediation. Based on the interviews and examiner workpapers, we concluded that
DER examination managers and examiners had broad discretion in determining the scope of

19
  DER requires its examiners to prepare two key workpapers for MRA remediation: (1) the procedures
document, which provides the “decision path for the work performed to support the conclusions and
examination findings” and (2) analysis memorandum or summary memorandum, which describe the
examiners’ work and conclusions. For almost all 22 sampled MRAs, the examiners prepared both required
documents. DER did not prepare – and could not provide to OIG upon request – the procedures document for
one MRA.



                                  OIG • EVL-2018-003 • March 28, 2018                                      14
their independent assessment of the adequacy of the remedial actions. For example, one
examination manager reported that some examiners perform two reviews – a review of IA’s
work and then a review of the effectiveness of the completed remediation. Another stated that
examiners should leverage the IA workpapers, look at deliverables, and assess what was
received in comparison to what was required.

They expressed no uniform view on whether they were expected to conduct any testing as part
of that assessment. For those examiners who explained that they conducted their own testing
as part of the assessment, they offered different explanations about the meaning of testing to
assess the adequacy of MRA remediation. One examination manager reported to us that
testing is a broad term that could include getting people together to determine whether they
are following a policy. Another examination manager noted that testing could include
sampling and data analysis, or it could be a walk-through.

In a previous OIG report, we reviewed a fairly small sample of MRA closures and found that
some DER examiners appeared to have accepted an Enterprise’s IA work to close MRAs
without documented evidence of independent review of the adequacy of remediation. In this
evaluation, we reviewed the workpapers for a larger sample of MRAs (22) for evidence that
examiners independently assessed Enterprise management’s closure packages and/or IA’s
validation work to support closure of the MRAs. 20 We observed variability in the work steps
taken and variations in the level of detail provided in examiners’ workpapers for our sample
of 22 MRAs, which is not surprising in light of the lack of guidance on the necessary work
steps.

Although the scope of the examiners’ assessment of MRA remediation varied, our workpaper
review found some evidence of independent assessment of the sufficiency of management’s
remediation activities and/or IA’s validation work for nearly all of the 22 MRAs in our
sample. 21 We caution that the examiners’ broad discretion to determine the scope of their
assessment work and the variability in the documentation in their workpapers constrained

20
   We used the same standards for this review as we did in our 2016 evaluation report. In that report, we
reviewed “documentation made available to us by FHFA to determine whether DER examiners performed
independent analyses or assessments, or merely recorded information that the Enterprises provided. Where
we found no documentation, or where the documentation recited information from an Enterprise without
any analysis, or where documentation reflected that DER agreed with an Enterprise’s assertions without any
supporting analysis, we concluded that no independent analysis or assessment had been performed by DER
examiners. Conversely, we credited DER with performing the independent assessment required by FHFA
where the documentation reflected some independent analysis or assessment by the DER examiner, however
limited.” See OIG, FHFA’s Inconsistent Practices in Assessing Enterprise Remediation of Serious
Deficiencies and Weaknesses in its Tracking Systems Limit the Effectiveness of FHFA’s Supervision of the
Enterprises, at 17-21 (July 14, 2016) (EVL-2016-007) (online at www.fhfaoig.gov/Content/Files/EVL-2016-
007.pdf).
21
   We observed in one instance that the examiner documented IA’s work without evidence of independent
assessment.



                                  OIG • EVL-2018-003 • March 28, 2018                                        15
our assessment of whether the workpapers met expectations for an independent assessment
required by FHFA examination guidance.


FINDINGS .................................................................................

   1. When testing of the sufficiency of MRA remediation was conducted for the MRAs in
      our sample, DER examiners generally relied on the validation testing conducted by IA
      as part of their assessment of the Enterprises’ remedial actions.

   2. FHFA has not issued guidance on the level of assurance of the competency,
      objectivity, reliability, and quality of IA’s validation work needed before the IA work
      can be used.

   3. Because FHFA has not specified the work steps examiners are expected to follow
      to assess the sufficiency of MRA remediation, examiners have broad discretion in
      determining the scope of their independent assessment of the adequacy of the remedial
      actions, including whether to conduct any testing as part of that assessment.

   4. For nearly all of the 22 MRAs in our sample, we found some evidence of independent
      assessment of the sufficiency of management’s remediation activities and/or IA’s
      validation work, although we found the scope of that assessment among examiners
      and the level of detail in the workpapers varied. The examiners’ broad discretion to
      determine the scope of their assessment work and the variability in the documentation
      in their workpapers constrained our assessment of whether the workpapers met
      expectations for an independent assessment required by FHFA examination guidance.


CONCLUSIONS ..........................................................................

Current FHFA guidance directs its examiners to independently review and assess the
documents in the Enterprise’s closure package, including some independent review or
assessment of documentation provided by the Enterprise’s business unit and/or IA. As FHFA
does not identify the steps that examiners should undertake to assess the sufficiency of MRA
remediation, we found that DER examination managers and examiners have broad discretion
in determining the scope of their independent assessment of the adequacy of the remedial
actions. We determined, from our review of key examiner workpapers for the 22 MRAs in
our sample, that the workpapers reflected some independent assessment of the sufficiency of
management’s remediation activities and/or IA’s validation work for nearly all of the 22
MRAs in our sample, although the scope of that assessment varied among examiners.


                             OIG • EVL-2018-003 • March 28, 2018                                16
When DER examiners specifically reported that testing of the sufficiency of MRA
remediation was conducted for the 22 MRAs in our sample as part of their assessment of the
Enterprises’ remedial actions, we found that the examiners generally relied on the validation
testing conducted by the Enterprise’s IA function. We determined, from our review of
examiners’ key workpapers for the 22 MRAs in our sample, that almost half (9 of 22) cited
testing completed by the IA function and did not reflect that the examiners performed their
own testing. For 11 of the 22 sampled MRAs, we found that the examiners’ key workpapers
reflected a review of IA validation work, but did not specifically identify testing performed
either by IA or by the examiners. For 2 of the 22 sampled MRAs, we found that the
examiners performed their own testing; in one of these instances it appears the examiner
conducted testing for a component of MRA remediation and relied on IA’s testing for another
component.

In a companion evaluation issuing today, we found that FHFA, unlike the Federal Reserve,
does not require its examiners to conclude on the overall effectiveness of the Enterprises’ IA
functions as a predicate to use of IA work. As a result, its examiners may lack a sufficient
basis to determine whether, or to what extent, to use IA’s validation work, and FHFA has not
issued guidance on the level of assurance of the competency, objectivity, reliability, and
quality of IA’s validation work needed before the IA work can be used.


RECOMMENDATION .................................................................

We recommend that FHFA adopt clear guidance for examiners to follow when assessing the
sufficiency of MRA remediation by the Enterprises that identifies the work steps that should
be included in examiners’ independent assessments of IA’s work and specifies the conditions
under which examiner testing is expected.


FHFA COMMENTS AND OIG RESPONSE .....................................

We provided FHFA an opportunity to respond to a draft report of this evaluation. FHFA
provided technical comments on the draft report, which we incorporated as appropriate. In its
management response, which is reprinted in its entirety in the Appendix, FHFA agreed with
OIG’s recommendation.




                              OIG • EVL-2018-003 • March 28, 2018                                17
OBJECTIVE, SCOPE, AND METHODOLOGY .................................

We conducted this evaluation to assess DER’s follow-up practices for closing MRAs.
Our objective was to understand the extent to which (1) examiners accepted, relied on, or
otherwise used IA’s validation work in their assessment of the adequacy of MRA remediation
and (2) whether they conducted independent assessments of the adequacy of the remediation.

To achieve this objective, we selected a sample of 22 MRAs from a population of 78 MRAs
that were closed by DER between January 2015 and October 2017; 14 of these MRAs were
issued to Fannie Mae and 8 to Freddie Mac. We reviewed key workpapers prepared by the
examiners and Enterprise documents used by the examiners to assess remediation of the
MRAs. We also reviewed FHFA guidance regarding MRA remediation in effect during the
period in which our sample of MRAs were closed.

We also interviewed the Deputy Director of DER, FHFA’s Chief Accountant, and the
examination managers and examiners for both the Fannie Mae and Freddie Mac examination
teams responsible for closing the MRAs in our sample.

The field work for this report was completed between October 2017 and January 2018.

This evaluation was conducted under the authority of the Inspector General Act and in
accordance with the Council of the Inspectors General on Integrity and Efficiency’s Quality
Standards for Inspection and Evaluation (January 2012). These standards require us to plan
and perform an evaluation based upon evidence sufficient to provide a reasonable basis to
support its findings and recommendations. We believe that the findings and recommendation
discussed in this report meet those standards.




                             OIG • EVL-2018-003 • March 28, 2018                              18
APPENDIX: FHFA MANAGEMENT RESPONSE .............................




                   OIG • EVL-2018-003 • March 28, 2018        19
ADDITIONAL INFORMATION AND COPIES .................................


For additional copies of this report:

   •   Call: 202-730-0880

   •   Fax: 202-318-0239

   •   Visit: www.fhfaoig.gov



To report potential fraud, waste, abuse, mismanagement, or any other kind of criminal or
noncriminal misconduct relative to FHFA’s programs or operations:

   •   Call: 1-800-793-7724

   •   Fax: 202-318-0358

   •   Visit: www.fhfaoig.gov/ReportFraud

   •   Write:

                FHFA Office of Inspector General
                Attn: Office of Investigations – Hotline
                400 Seventh Street SW
                Washington, DC 20219




                               OIG • EVL-2018-003 • March 28, 2018                         20