oversight

FHFA's Use of Inconsistent Criteria Materially Affected its Reporting of Remediation of Serious Deficiencies in its 2015 Performance and Accountability Report

Published by the Federal Housing Finance Agency, Office of Inspector General on 2016-11-09.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

            Federal Housing Finance Agency
                Office of Inspector General




FHFA’s Use of Inconsistent Criteria
Materially Affected its Reporting of
Remediation of Serious Deficiencies
   in its 2015 Performance and
       Accountability Report




Evaluation Report  EVL-2017-001  November 9, 2016
               Executive Summary
               As the federal regulator of Fannie Mae and Freddie Mac (the Enterprises) and
               of the Federal Home Loan Banks (FHLBanks), the Federal Housing Finance
               Agency (FHFA or the Agency) is tasked by statute with ensuring that these
               entities operate safely and soundly so that they serve as a reliable source of
               liquidity and funding for housing finance and community investment. FHFA
EVL-2017-001   has directed its Division of Federal Home Loan Bank Regulation (DBR) to
               conduct supervisory activities for the FHLBanks and its Division of Enterprise
November 9,    Regulation (DER) to conduct these activities of the Enterprises.
   2016
               The Government Performance and Results Act of 1993, as amended by the
               GPRA Modernization Act of 2010 (GPRA), requires FHFA (and other federal
               agencies) to establish strategic plans, develop performance goals aligned with
               those strategic plans, and set performance indicators to measure whether those
               goals are met. Pursuant to GPRA, each federal agency must report, after the
               end of each fiscal year, whether it has met its performance goals.

               To meet its GPRA obligations for fiscal year (FY) 2015, FHFA established
               three strategic goals and identified three performance goals for each strategic
               goal. For its first strategic goal, “Ensure Safe and Sound Regulated Entities,”
               FHFA set three performance goals tied to its supervisory activities: “assess
               the safety and soundness of regulated entity operations”; “identify risks to
               the regulated entities and set expectations for strong risk management”; and
               “require timely remediation of risk management weaknesses.” Risk
               management weaknesses, and other deficiencies, at a regulated entity are
               identified by FHFA during its supervisory activities. Where FHFA finds a
               risk management weakness or other deficiency, it will classify the weakness
               or other deficiency as a Matter Requiring Attention (MRA), a violation, or
               a recommendation. According to FHFA, MRAs are reserved for “the most
               serious supervisory matters” and require “prompt remediation” by the affected
               regulated entity.

               For FY 2015, FHFA determined that it would measure success in achieving
               its stated performance goal – “require timely remediation of risk management
               weaknesses” – by measuring whether its “[r]egulated entities complete
               remedial action for MRAs within agreed upon timeframes.” FHFA established
               a 90% target for this performance measure. FHFA sought uniform criteria
               to be used by DBR and DER to measure performance during FY 2015.
               According to FHFA’s 2015 Performance and Accountability Report (PAR),
               published in November 2015, its regulated entities exceeded this target during
               FY 2015: the “FHLBanks reported a 97% compliance rate and Fannie Mae
               and Freddie Mac both reported a 100% compliance rate.”
               Over the past year, we issued several reports regarding FHFA’s oversight of
               the Enterprises’ remediation of MRAs, in which we identified a number of
               shortcomings. In light of the outstanding performance results for this
               performance measure reported by FHFA in its 2015 PAR, we undertook this
               evaluation to assess FHFA’s bases for those reported results.

               Contrary to FHFA’s expectations, we found that DER and DBR used different
EVL-2017-001   criteria to calculate compliance rates, which materially affected the reported
               compliance rates. Moreover, DER and DBR did not fully disclose their
November 9,    differing criteria to the FHFA office that coordinated and developed the PAR.
   2016        Within DBR, senior officials were vested with complete discretion to
               determine whether 20 of 80 MRAs were “on track” or “off track” to meet
               agreed upon timetables and exercised that discretion to find that 17 of the 20
               were “on track.” Absent the exercise of such discretion, DBR could have
               reported a compliance rate as low as 75%. From our review of internal
               documents and interviews with DER officials, it appears that DER initially set
               out to report on all MRAs open during FY 2015, as DBR did. However, DER
               developed no methodology during the year to capture the data to calculate a
               compliance rate. When asked by FHFA several weeks after the close of FY
               2015 to report a compliance rate for this performance measure for the 2015
               PAR, DER determined that it would report only on those MRAs it closed
               during FY 2015 – or only 29% of the total MRAs open at one point during FY
               2015. As a consequence, DER reported a 100% compliance rate.

               GPRA requires each federal agency to report any limitations to data it reports,
               including inconsistencies with data collection procedures. Because the office
               responsible for coordinating the development and publication of FHFA’s PAR
               was not made aware of the different criteria used by DER and DBR, it did not
               report the inconsistencies with internal data collection procedures nor did it
               qualify or otherwise caveat FHFA’s reported compliance rates in the PAR for
               this performance measure.

               This report was prepared by Howard Klein, Attorney Advisor, Brian Stief,
               Investigative Counsel, and Timothy Callahan, Attorney Advisor. We
               appreciate the cooperation of FHFA staff, as well as the assistance of all those
               who contributed to the preparation of this report.

               This report has been distributed to Congress, the Office of Management and
               Budget, and others and will be posted on our website, www.fhfaoig.gov.




               Angela Choy
               Assistant Inspector General for Evaluations
TABLE OF CONTENTS ................................................................
EXECUTIVE SUMMARY .............................................................................................................2

ABBREVIATIONS .........................................................................................................................5

BACKGROUND .............................................................................................................................6
      Measuring Government Performance .......................................................................................6
      FHFA’s Process to Develop the APP and PAR........................................................................7
      FHFA’s Goals and Performance Measures for the 2015 PAR .................................................8
      The 2015 APP Published FHFA’s Goal for Completing Remediation of MRAs, and
      the 2015 PAR Reported that FHFA Had Achieved that Goal ..................................................9

FACTS AND ANALYSIS.............................................................................................................11
      Development of Performance Goal 1.3 and Performance Measure 1.3.1 ..............................11
      DBR’s Process for Collecting and Reporting Data for Performance Measure 1.3.1 .............13
             DBR’s Data Collection Process for Performance Measure 1.3.1 ...................................13
             DBR’s Criteria for Assessing Whether Remedial Actions to Correct MRAs
                Were Completed Within Agreed Upon Timeframes ...............................................14
             Exercise of Discretion in Reporting Performance Data for Performance Measure
                 1.3.1 .........................................................................................................................15
      DER Data Collection and Reporting for Performance Measure 1.3.1 ...................................17
             DER’s Data Collection Process for Performance Measure 1.3.1 ...................................17
             DER’s Reporting for Performance Measure 1.3.1 ..........................................................19

CONCLUDING OBSERVATIONS ..............................................................................................21

OBJECTIVE, SCOPE, AND METHODOLOGY .........................................................................23

APPENDIX A ................................................................................................................................24
      FHFA’s Management Response .............................................................................................24

ADDITIONAL INFORMATION AND COPIES .........................................................................25




                                          OIG  EVL-2017-001  November 9, 2016                                                                4
ABBREVIATIONS .......................................................................

APP                Annual Performance Plan

CFO                Chief Financial Officer

DBR                Division of Federal Home Loan Bank Regulation

DER                Division of Enterprise Regulation

Enterprises        Fannie Mae and Freddie Mac, collectively

FHFA or Agency     Federal Housing Finance Agency

FHLBanks           Federal Home Loan Banks

FY                 Fiscal Year

GPRA               GPRA Modernization Act of 2010

MRA                Matter Requiring Attention

OBFM               Office of Budget and Financial Management

PAR                Performance and Accountability Report

ROE                Report of Examination




                         OIG  EVL-2017-001  November 9, 2016                      5
BACKGROUND ..........................................................................

Since 2008, FHFA has operated as both regulator and conservator of Fannie Mae and Freddie
Mac and regulator of the Federal Home Loan Bank system to ensure that these entities operate
safely and soundly so that they serve as a reliable source of liquidity and funding for housing
finance and community investment.

DBR is responsible for supervising the FHLBanks and the Office of Finance. DER is
responsible for supervising Fannie Mae and Freddie Mac (collectively, the Enterprises). DER
uses two core examination teams to supervise the Enterprises; one team examines Fannie
Mae, and the other examines Freddie Mac.

Like other federal agencies, FHFA publishes an annual PAR, assessing the Agency’s
performance in meeting specific goals it establishes earlier in the year. The requirements and
structure of the PAR are governed by the Government Performance and Results Act of 1993,1
as amended by the GPRA Modernization Act of 2010 (GPRA).2

Measuring Government Performance

When Congress passed the Government Performance and Results Act of 1993, it was hailed
as “a watershed moment for the federal government.”3 The Act was the first time Congress
established statutory requirements for most agencies to set goals, measure performance, and
submit related plans and reports to Congress. The 1993 Act was intended to assist Congress
in its policy making, oversight, and budget functions.4 GPRA, passed in 2011, built on and
updated the framework of the Act, drawing on the experience of multiple administrations, the
views of the authors of the 1993 Act, and other sources.

GPRA requires federal agencies to establish strategic plans, to develop performance goals
aligned with those strategic plans, and to establish performance indicators to assess whether
those goals are met. An agency’s performance goals for each fiscal year are published in an
annual performance plan (APP), and must (unless otherwise authorized by the Office of
Management and Budget) be “objective, quantifiable, and measurable.” Each APP must also

1
    Pub. L. No. 103-62, 107 Stat. 285 (1993).
2
    Pub. L. No. 111-352, 124 Stat. 3866 (2011).
3
 Cong. Research Serv., Changes to the Government Performance and Results Act (GPRA): Overview of the
New Framework of Products and Processes, at 1(Feb. 29, 2012) (R42379).
4
  Senate Committee on Governmental Affairs, Government Performance and Results Act of 1993, at 3, report
to accompany S. 20, 103rd Cong. (June 16, 1993) (S. Rept. 103-58).




                                    OIG  EVL-2017-001  November 9, 2016                                  6
contain performance indicators, which are the specific benchmarks against which an agency’s
performance is assessed.5

Further, each APP must describe “how the agency will ensure the accuracy and reliability
of the data used to measure progress toward its performance goals.” Agency leaders are
“accountable for choosing goals and indicators wisely and for setting ambitious, yet realistic
targets.”6 Agencies are required to post the APP for each fiscal year on their websites and to
notify the President and Congress when it is available.

After the end of the fiscal year, GPRA requires each federal agency to publish a PAR on its
public website stating whether it has met each performance goal.7 If a performance goal is
not met, the agency must explain the reasons why the goal was not met and its plans to
achieve that goal in the future. According to GPRA, the PAR must also describe how the
agency ensures the accuracy and reliability of its performance data and must identify any
limitations to the data, including inconsistencies in an agency’s data collection procedures.8
The agency head is ultimately responsible for the accuracy of performance data published in
the PAR.

FHFA’s Process to Develop the APP and PAR

FHFA’s Office of Budget and Financial Management (OBFM) coordinates the development
and publication of FHFA’s APP and PAR for each fiscal year. Performance goals and their
benchmarks (or measures) for FHFA are developed by the office or division responsible for
meeting them, in collaboration with OBFM. Once FHFA offices and divisions finalize their
goals and measures and OBFM assembles a complete draft of the APP, the draft APP is
reviewed and approved by the FHFA Director. After the APP is published by FHFA on its
website, the performance goals and measures are established for the fiscal year and are not
subject to change. FHFA’s Chief Financial Officer (CFO), who is in charge of OBFM,
considers the APP to set forth “stretch goals.”



5
  GPRA recognizes two different types of performance indicators: “output measures,” which assess the
agency’s effort in working towards its goals, and “outcome measures,” which assess the ultimate success or
failure of the agency’s program. OMB guidance encourages agencies to use outcome measures, which
measures actual results, whenever feasible.
6
  OMB, Circular A-11, Preparation, Submission, and Execution of the Budget, Part 6, Executive Summary, at
2 (June 30, 2015).
7
  GPRA also authorizes agencies to report this information in an annual performance report. For purposes of
this evaluation, all such reports mandated by GPRA are referred to as a PAR.
8
 OMB, Circular A-11, Preparation, Submission, and Execution of the Budget, Part 6, Section 260, at 4 (June
30, 2015).



                                 OIG  EVL-2017-001  November 9, 2016                                        7
During each fiscal year, FHFA expects that its individual offices and divisions will collect
the designated performance data, which is then entered into OBFM’s electronic performance
tracking system (OBFM tracking system). The head of each office or division is responsible
for the accuracy of the data reported to OBFM. At the end of each fiscal year, OBFM
compiles the performance data that has been entered into the tracking system to generate the
reportable results for the PAR. A draft PAR prepared by OBFM is circulated within FHFA
for review and comment. Once finalized, the PAR is provided to the FHFA Director for
approval, and published and posted on FHFA’s website.

Because some of the goals announced by FHFA in its APP are “stretch goals,” FHFA’s CFO
reported to us that, in his view, it would not be unacceptable for FHFA to occasionally miss
an APP goal, especially if attainment of that goal is not entirely within the agency’s control.
He acknowledged, however, that some FHFA officials were not comfortable in setting
ambitious goals that might not be met.

FHFA’s Goals and Performance Measures for the 2015 PAR

On November 21, 2014, FHFA published its strategic plan for fiscal years 2015-2019 and that
plan set forth three strategic goals and identified three performance goals for each strategic
goal. Pursuant to GPRA, FHFA incorporated these three strategic and nine performance goals
into its APP and PAR for FY 2015. See Figure 1.

                      FIGURE 1. FHFA 2015 STRATEGIC AND PERFORMANCE GOALS




Source: FHFA, Fiscal Year 2015 Performance and Accountability, at 14.




                                OIG  EVL-2017-001  November 9, 2016                             8
The 2015 APP Published FHFA’s Goal for Completing Remediation of MRAs, and the
2015 PAR Reported that FHFA Had Achieved that Goal

In its APP for FY 2015 (2015 APP), published on February 9, 2015, FHFA’s first strategic
goal, Ensure Safe and Sound Regulated Entities, set forth three performance goals.
Performance Goal 1.3 announced that FHFA sought to “require timely remediation of risk
management weaknesses” by the regulated entities. FHFA’s published performance measure
for that goal – the benchmark against which the Agency’s success or failure would be judged
– was set forth in Performance Measure 1.3.1: that the “regulated entities complete remedial
action for MRAs [Matters Requiring Attention] within the agreed upon timeframes” at least
90% of the time.9

In its 2015 PAR, published on November 16, 2015, FHFA reported on the results of its
performance measures for each of its performance goals.10 With respect to Performance
Measure 1.3.1, FHFA reported:

          The regulated entities met the target of completing remedial action for the
          MRAs within the agreed-upon time frames at least 90 percent of the time.
          The FHLBanks reported a 97% compliance rate and Fannie Mae and Freddie
          Mac both reported a 100% compliance rate.

Because Performance Goal 1.3 focused on “timely remediation of risk management
weaknesses” by the regulated entities, which DBR and DER are charged with supervising, the
Deputy Directors of DBR and DER were responsible for collecting and accurately reporting
the information for Performance Goal 1.3 and the performance measure under it—
Performance Measure 1.3.1. In the Validation and Verification of Performance Data section
of its 2015 PAR, FHFA represented that it had verified its “sources of data” to ensure the
completeness and reliability of the information reported.

Over the past year, we have issued several reports regarding FHFA’s oversight of the
Enterprises’ remediation of MRAs, in which we identified a number of shortcomings.11

9
 FHFA considers MRAs to be “the most serious supervisory matters,” and it issues them for “non-compliance
with laws or regulations that result or may result in significant risk of financial loss or damage,” “repeat
deficiencies that have escalated due to insufficient action or attention,” “unsafe or unsound practices,” and
“breakdowns in risk management, significant control weaknesses, or inappropriate risk-taking.” FHFA,
Advisory Bulletin 2012-01, at 2 (Apr. 2, 2012).
10
     FHFA represented that the PAR met the requirements of GPRA.
11
  See OIG, FHFA’s Examiners Did Not Meet Requirements and Guidance for Oversight of an Enterprise’s
Remediation of Serious Deficiencies (Mar. 29, 2016) (EVL-2016-004) (www.fhfaoig.gov/Content/Files/EVL-
2016-004.pdf) [hereinafter OIG, Remediation Report]; OIG, FHFA’s Supervisory Standards for
Communication of Serious Deficiencies to Enterprise Boards and for Board Oversight of Management’s



                                  OIG  EVL-2017-001  November 9, 2016                                         9
These shortcomings included: FHFA’s failure to establish agreed upon timetables for
elements of an MRA; and FHFA’s failure to review the adequacy and timeliness of the
Enterprises’ MRA remediation efforts to determine whether the underlying deficiency
had been corrected, resulting in a substantial backlog of open MRAs. In light of these
shortcomings and the outstanding performance results for Performance Measure 1.3.1
reported by FHFA in the 2015 PAR, we undertook this evaluation to examine FHFA’s basis
for the results it reported for Performance Measure 1.3.1. We have not examined FHFA’s
basis for the results it reported for any other performance measures in this PAR.




Remediation Efforts are Inadequate (Mar. 31, 2016) (EVL-2016-005) (www.fhfaoig.gov/Content/Files/EVL-
2016-005.pdf); OIG, FHFA’s Inconsistent Practices in Assessing Enterprise Remediation of Serious
Deficiencies and Weaknesses in its Tracking Systems Limit the Effectiveness of FHFA’s Supervision of the
Enterprises (July 14, 2016) (EVL-2016-007) (www.fhfaoig.gov/Content/Files/EVL-2016-007.pdf) [hereinafter
OIG, Tracking Report]; OIG, FHFA’s Failure to Consistently Identify Specific Deficiencies and Their Root
Causes in Its Reports of Examination Constrains the Ability of the Enterprises Boards to Exercise Effective
Oversight of Management’s Remediation of Supervisory Concerns (July 14, 2016) (EVL-2016-008)
(www.fhfaoig.gov/Content/Files/EVL-2016-008.pdf); OIG, FHFA’s Implementation of Its Automated System
to Track Deficiencies Identified in Federal Home Loan Bank Examinations (May 26, 2016) (COM-2016-003)
(www.fhfaoig.gov/Content/Files/COM-2016-003.pdf); OIG, DBR’s Unwritten Procedures and Practices for
Oversight of Efforts by Federal Home Loan Banks to Correct Deficiencies Underlying the Most Serious
Supervisory Matters Are Inconsistent with the Written Oversight Requirements Promulgated by FHFA (Sept.
30, 2016) (COM-2016-006) (www.fhfaoig.gov/Content/Files/COM-2016-006.pdf) [hereinafter OIG, OCom
Report].



                                 OIG  EVL-2017-001  November 9, 2016                                        10
FACTS AND ANALYSIS ...............................................................

Development of Performance Goal 1.3 and Performance Measure 1.3.1

Both DBR and DER developed, approved, and reported on Performance Goal 1.3 and
Performance Measure 1.3.1 for FY 2015. While this performance goal – to “require timely
remediation of risk management weaknesses” by the regulated entities – was carried over
from FY 2014 to FY 2015,12 FHFA sought uniform criteria to be used by DBR and DER to
measure performance against that goal during FY 2015.

For FY 2014, the relevant performance measure stated: “[i]n response to examination
findings, evaluate the effectiveness of remedial actions taken by the Enterprises and
FHLBanks.”13 DBR understood this performance measure to focus on outcomes: it reported
the percentage of MRAs that the FHLBanks had completely remediated or were on-plan to
remediate. DER understood this performance measure to focus on outputs: it reported the
percentage of Enterprise MRA remediation plans that DER had responded to within 60 days
of submission, without reporting on efforts by the Enterprises to remediate the MRAs, in
whole or in part.14

For FY 2015, DER and DBR agreed to use the same outcome-based performance measure.
See Figure 2 for Performance Measure 1.3.1, as set forth in FHFA’s APP for FY 2015.

                            FIGURE 2. PERFORMANCE MEASURE 1.3.1 FOR FY 2015




Source: FHFA, Annual Performance Plan for Fiscal Year 2015, at 10.




12
  In both years, the goal remained the same but the number of that goal changed. In the 2014 APP, this goal
was labeled Performance Goal 1.1 and in the 2015 APP, this goal was labeled Performance Goal 1.3.
13
     In the 2014 APP, this was Performance Measure 1.2.2.
14
  FHFA’s Advisory Bulletin 2012-01 directs that “[c]orrective action for MRAs must be articulated in written
remediation plans, prepared by the regulated entity, acceptable to the FHFA.”



                                   OIG  EVL-2017-001  November 9, 2016                                       11
The 2015 APP also sets forth the criteria to be used by FHFA to validate whether
Performance Measure 1.3.1 has been met:

         Data Validation and Verification for Performance Goal 1.3

         Measure 1.3.1 - FHFA discusses MRAs with Enterprise management and
         communicates them in writing. The Enterprises address MRAs through
         preparation and execution of remediation plans. FHFA reviews proposed
         plans and completed work to determine whether identified deficiencies have
         been addressed. FHFA tracks remediation and closure of MRAs.

         All FHLBanks’ Reports of Examination contain a summary of examination
         issues identified and MRAs, which include follow-up dates by which the
         FHLBanks are to resolve the identified issues. For the FHLBanks, DBR
         discusses each MRA with the board of the relevant FHLBank or the Office of
         Finance. When resolved to FHFA’s satisfaction, DBR informs the FHLBank
         and closes the MRA.

         The target is considered met when FHFA confirms that identified weaknesses
         have been remediated within the agreed upon timeframe.15

Senior DBR and DER officials reviewed and provided feedback on both Performance
Measure 1.3.1 and the data validation and verification for Performance Measure 1.3.1. The
Deputy Directors of DBR and DER signed off on the APP in February 2015.

Despite DBR’s and DER’s agreement on the definition for Performance Measure 1.3.1, DBR
and DER used different criteria to report their performance. DBR reported on the adequacy
and timeliness of remediation for open MRAs issued during previous examination cycles for
each FHLBank that the division examined in FY 2015. DER reported only on the timeliness
of those 20 MRAs it closed during FY 2015. This difference was not explained in the 2015
PAR. Because OBFM was not made aware of the different criteria used by DER and DBR, it
did not report the inconsistencies with internal data collection procedures nor did it qualify or
otherwise caveat FHFA’s reported compliance rates in the PAR for this performance measure.




15
  FHFA, Annual Performance Plan for Fiscal Year 2015, at 10. DBR and DER officials separately
acknowledged to us that they understood the word “confirms,” as used in the APP Data Validation and
Verification section, to mean that their respective staff would review the remedial actions taken by the
regulated entity to determine the sufficiency and timeliness of those actions. A statement by a regulated entity
that it remediated an MRA would not meet the validation criteria.



                                  OIG  EVL-2017-001  November 9, 2016                                            12
DBR’s Process for Collecting and Reporting Data for Performance Measure 1.3.1

DBR established a process for tracking and updating quarterly its compliance rate for
Performance Measure 1.3.1 based on the results of its on-site examinations of the FHLBanks.
The division reported a 97% compliance rate in the 2015 PAR. Seventy-five percent of the
MRAs DBR included in its reporting were deemed fully remediated. The remaining 25%
were not fully remediated and were reported as “on track” or “off track” based solely on the
discretionary determinations of senior DBR officials.

      DBR’s Data Collection Process for Performance Measure 1.3.1

DBR’s process for gathering performance data for Performance Measure 1.3.1 was tied to
its supervisory schedule. As supervisor of the 11 regional FHLBanks and the Office of
Finance,16 DBR assesses their safety and soundness primarily through annual, on-site
examinations.17 Three associate directors in DBR manage DBR’s safety and soundness
examinations of the FHLBanks and Office of Finance.18 DBR examination teams spend five
to six weeks on-site at each entity it regulates reviewing its risk management and operations.19
When conducting their supervisory activities, DBR examiners may identify deficiencies that
result in findings, including MRAs. The affected FHLBank is usually given one year or less
within which to completely remediate the MRA.20 DBR expects its examiners to assess the
adequacy and timeliness of remediation of an MRA by a regulated entity during the next
scheduled annual examination and record their determinations (e.g., “Resolved,” “Partially
Remediated,” “Re-issued,” “Repeated,” or “Closed”) in the Report of Examination (ROE)
issued to the FHLBank’s board of directors.21


16
   Each FHLBank is cooperatively owned by its respective member financial institutions, including banks,
thrifts, and insurance companies. The primary business activity of each FHLBank is to provide loans, known
as advances, to their members to support housing finance and for other purposes. FHLBanks may also invest
in U.S. Treasury securities, certain mortgage assets, and other types of assets as permitted by law and FHFA
regulations.
17
     DBR also conducts periodic visits, special reviews, and off-site monitoring of FHLBank financial data.
18
     Each associate director was responsible for four regulated entities.
19
  According to a DBR official, a typical DBR examiner participates in four annual FHLBank examinations
per year and spends approximately five months on travel doing so.
20
     Throughout this report, references to the FHLBanks include the Office of Finance.
21
   FHFA, like other federal financial regulators, produces an ROE in conjunction with its supervision of each
regulated entity. According to FHFA, the ROE communicates to the board of directors of a regulated entity
substantive examination results and conclusions. DER issues an ROE to each Enterprise at the end of each
annual supervisory cycle, and DBR issues an ROE to each FHLBank after completing the bank’s annual on-
site examination.




                                     OIG  EVL-2017-001  November 9, 2016                                      13
DBR’s Deputy Director approved the APP on February 2, 2015. DBR tasked its executive
advisor, an official who reports directly to the Deputy Director, with responsibility for
gathering and reporting quarterly performance data for Performance Measure 1.3.1. DBR’s
executive advisor documented in the OBFM tracking system the process used to gather that
data during FY 2015:

        To determine our actual success rate, we will poll the Associate Directors . . .
        to determine what proportion of MRAs were fully addressed or are on
        schedule for completion. Quarterly, each Associate Director will determine
        whether outstanding MRAs reviewed during the examination were sufficiently
        addressed. We will keep a running total and by year-end have our total
        success rate.

To obtain the required performance data for Performance Measure 1.3.1, DBR’s executive
advisor sent the following written request to DBR’s three associate directors:

        At the [FHLBank or Office of Finance] examiners reviewed MRAs from
        the prior examination. What proportion of MRAs were remediated by the
        FHLBank or are on-plan to be remediated. . . [A]nswers should be like “4 out
        of 4” or “3 out of 5.”

Each associate director obtained that information and sent it by email to the executive advisor,
who recorded the data in the OBFM tracking system. DBR’s Deputy Director then reviewed
and approved each quarterly entry. We found no evidence that DBR performed any
additional validation of the data’s integrity.

DBR gathered and reported data on the status of all open MRAs that had been issued during
(or before) the calendar year 2014 examination cycle for the regulated entities it reviewed
during the calendar year 2015 examination cycle.

    DBR’s Criteria for Assessing Whether Remedial Actions to Correct MRAs Were
    Completed Within Agreed Upon Timeframes

DBR determined that Performance Measure 1.3.1 had been met when examiners either:
(1) confirmed that the FHLBank completely remediated the MRA within the timeframe
initially established or any extension that DBR had granted; or (2) determined that
remediation of the MRA by the FHLBank was not complete but the FHLBank’s remedial



For a fulsome discussion of the assessments made by DBR examiners regarding the timeliness and adequacy of
remedial actions taken by an FHLBank to address an MRA, see OIG, OCom Report, supra note 11.




                                OIG  EVL-2017-001  November 9, 2016                                        14
actions were “on track” and remediation was expected to be complete by the agreed upon
deadline, including any extensions.

In a recent special project report,22 we assessed DBR’s oversight of the FHLBanks’
remediation of MRAs by reviewing a sample of nine MRAs issued by DBR from January
2014 through September 2015. For four of the nine MRAs in our sample, we found that DBR
examiners determined, based on their assessment of corrective actions during the subsequent
annual examination of the affected FHLBank, that the corrective actions were inadequate to
sufficiently remediate the MRA. In each of these four instances, the DBR examiners closed
the MRA and reissued it in whole or in part and extended the remediation timetable for at
least one year from the date of reissuance. For purposes of reporting performance data for
Performance Measure 1.3.1 for the PAR, each associate director had discretion to determine
whether a re-issued MRA, or a partially remediated MRA for which an extension was
granted, was “on track.” As we now discuss, the associate directors’ exercise of discretion
was material to DBR’s report of a 97% compliance rate by its regulated entities in meeting
Performance Measure 1.3.1.

      Exercise of Discretion in Reporting Performance Data for Performance Measure 1.3.1

The 2015 quarterly totals that the executive advisor entered into the OBFM tracking system
are set forth in Figure 3.

                 FIGURE 3. DBR PERFORMANCE DATA FOR PERFORMANCE MEASURE 1.3.1
                             AS REPORTED IN THE OBFM TRACKING SYSTEM

      Reporting Quarter        1Q2015          2Q2015            3Q2015           4Q2015            Total
 Percentage of MRAs         22 of 22        15 of 15          34 of 36         13 of 14         84 of 87
 Fully Remediated or
                            (100%)          (100%)            (94%)            (93%)            (97%)
 On Track
                                            Office of
                            Indianapolis,                     New York,        Indianapolis,
 Regulated* Entities                        Finance,
                            Chicago,                          Dallas,          Boston,
 Examined                                   San Francisco,
                            Topeka                            Pittsburgh       Cincinnati
                                            Atlanta
 * DBR’s examination schedule is based on a calendar year while, the PAR is based on the fiscal year. As a
 result, not every FHLBank may be examined during a fiscal year. Similarly, an FHLBank could be examined
 twice in one fiscal year and, thus, DBR would provide Performance Measure 1.3.1 data twice for one
 FHLBank, as it did for the Indianapolis FHLBank in 2015.




22
     OIG, OCom Report, supra note 11.



                                 OIG  EVL-2017-001  November 9, 2016                                       15
Following the fourth quarter entry, the executive advisor entered a final statement in the
OBFM tracking system confirming that DBR had successfully met the performance measure
for Performance Measure 1.3.1:

        [T]he running total for the year was . . . a 97% success rate. As our target was
        “90 percent of the time” for the year, we have met this Performance Measure
        for FY 2015.

The Deputy Director of DBR approved each quarterly update and the final performance
results through the tracking system.23

We found that DBR’s calculation of a 97% compliance rate for Performance Measure 1.3.1
was based, in material part, on discretionary decisions by the associate directors as to whether
partially remediated MRAs were on track or off track. Of the 80 MRAs reported by the
associate directors as subject to Performance Measure 1.3.1, 60 MRAs had been fully
remediated and DBR determined that remediation for each had been completed within the
agreed upon timeframe. For the remaining 20, DBR: extended the timetable for 4; was
awaiting final validation by the FHLBank for 1; assessed the adequacy of ongoing
remediation efforts where the original deadline had not occurred for 5; and reissued 10 in
whole or in part. The associate directors were vested with discretion to determine whether
Performance Measure 1.3.1 was met for each of these 20 but the criteria to be used to exercise
that discretion was not defined: the associate directors determined that 17 were on track and
met the performance measure and 3 were “off track” and did not meet the performance
measure.24

Without defined criteria, there is no basis for us, or anyone other than the DBR associate
directors, to assess whether incomplete remediation efforts for 17 of these 20 open MRAs
were “on track.” We can only determine that the associate directors’ exercise of discretion
enabled DBR to achieve the 97% reported success rate and, absent such discretion, DBR’s
success rate could have been as low as 75%.


23
   While the final entry in the OBFM tracking system reported that the 97% compliance rate for Performance
Measure 1.3.1 was based on the disposition of 87 MRAs, we found, based on documents provided by DBR,
that the 97% compliance rate was based on a review of 80, rather than 87, MRAs reported by the associate
directors to the executive advisor. The corrected compliance rate would be 96%. The executive advisor
attributed the difference to human error.
24
   During our fieldwork, DBR informed us that 3 of the 20 discretionary MRAs were reported as meeting
Performance Goal 1.3.1 when they did not. In two instances, the associate director who misreported those
MRAs attributed the error to his reliance on a draft ROE that indicated that these MRAs had been adequately
remediated. The final ROE, which issued later, concluded that the two MRAs had not been completely
remediated. In the third instance, a miscommunication between DBR and management resulted in an MRA
being reported on track when it was not. After accounting for the three misreported MRAs, DBR’s compliance
rate for Performance Measure 1.3.1 would be approximately 93%.



                                 OIG  EVL-2017-001  November 9, 2016                                        16
DER Data Collection and Reporting for Performance Measure 1.3.1

DER did not establish any criteria to gauge compliance for Performance Measure 1.3.1 during
FY 2015. Based on our review of DER’s internal documents and interviews with DER
officials, it appears that DER senior officials originally understood Performance Measure
1.3.1 to require an assessment of whether all open MRAs were being remediated in
accordance with agreed upon timetables. That understanding, however, was not
communicated to the core examination teams for each Enterprise. Although DER recorded
identical updates each quarter in the OBFM tracking system, it did not report a compliance
rate or assess the status of any individual MRA in support of those updates for any quarter.

DER closes an MRA after it determines that the deficiency giving rise to the MRA has been
adequately corrected by the affected Enterprise. DER closed a total of 20 MRAs for both
Enterprises during FY 2015. At the end of FY 2015, a total of 48 MRAs remained open for
both Enterprises. When OBFM asked DER to calculate its compliance rate for Performance
Measure 1.3.1 on October 30, 2015, approximately two weeks before the PAR was published,
DER directed its examination staff to assess the timeliness of remediation for the 20 MRAs
closed during FY 2015. Not surprisingly, the examination staff found a 100% compliance
rate.

     DER’s Data Collection Process for Performance Measure 1.3.1

DER supervises the Enterprises through targeted examinations and ongoing monitoring
during the year. When conducting these supervisory activities, DER examiners may identify
deficiencies that result in one or more MRAs. After DER issues an MRA, FHFA’s internal
guidance requires the affected Enterprise to submit a remediation plan with a timetable for
remedial actions, including interim milestones, to DER within 60 days.25 Advisory Bulletin
2012-01 directs DER examiners to check on the progress of remediation during the remedial
period and document their findings.26 When the Enterprise completes its remediation efforts,
Enterprise management submits a closure package to its internal audit department for
validation.27 Once the internal audit department completes its validation efforts, the
Enterprise submits the closure package to DER for review. DER examiners are expected to

25
   As we demonstrated in an earlier report this year, Enterprise remediation plans do not always propose
timetables for all required remediation and DER has accepted those plans without requiring a complete
timetable. See OIG, Remediation Report, supra note 11, at 23-24.
26
   In a report issued earlier this year, we found that DER examiners did not consistently follow these
requirements and did not assess and track Enterprise progress against remediation plans. See OIG, Tracking
Report, supra note 11, at 21.
27
   DER considers an MRA to be “validated” when the internal audit department concludes that the remedial
action is implemented, effective, and sustainable. For more information on the internal audit department’s role
in validating Enterprise remedial action, see id., at 17-18.



                                  OIG  EVL-2017-001  November 9, 2016                                           17
review the internal audit department’s validation and close the MRA if they determine that it
has been satisfactorily remediated. DER imposes no timeframe on examiners for completing
this review.

DER’s Deputy Director approved the APP on February 2, 2015. Based on our review of
internal documents and interviews with DER officials, it appears that DER originally
understood Performance Measure 1.3.1 to require an assessment of timeliness and adequacy
of remedial efforts for all open MRAs. However, when DER reported its compliance rate for
the 2015 PAR, it only considered those 20 MRAs it closed during FY 2015 and did not
consider the timeliness of remediation for the remaining 48 MRAs. An acting associate
director was tasked with gathering and reporting the quarterly performance data for
Performance Measure 1.3.1. Unlike DBR, however, DER did not provide its examination
staff with a written explanation of the method to be used to track and calculate the compliance
rate for the performance measure. Instead, DER provided the following explanation of the
measure for the OBFM tracking system:

        This performance measure is designed to ensure that the on-site examination
        teams follow up with Fannie Mae and Freddie Mac regarding examination
        findings and conclusions that identify weaknesses that should be remediated
        to strengthen Enterprise risk management. Remediation by the Enterprises
        should take place within a timeframe to minimize any ongoing risk to the
        Enterprise operations.

The DER acting associate director advised us that he did not explain to the Fannie Mae and
Freddie Mac examination staff responsible for tracking the progress of MRA remediation the
type of performance data that should be collected for Performance Measure 1.3.1. While he
maintained that he gathered performance data for this measure by speaking with examination
staff, he did not document the information he gathered during those conversations.28 The
acting associate director, or an employee in his office, updated the OBFM tracking system
each quarter with the following identical reports:

        DER is on track to meet this annual measure for the year ending September
        30, 2015. Neither Fannie Mae nor Freddie Mac have advised DER of changes
        to remediation plans for MRAs. DER has not identified any MRAs for which




28
 The examination staff responsible for tracking MRAs did not recall any such oral requests for Performance
Measure 1.3.1 performance data during FY 2015.




                                 OIG  EVL-2017-001  November 9, 2016                                       18
           the remedial action is expected to be incomplete or outside the agreed upon
           timeframes.29

These updates contained no other information. The DER Deputy Director reviewed and
approved each of these quarterly reports.30

At the direction of DER’s Deputy Director, OBFM entered the following information into its
tracking system on October 28, 2015:

           DER met this annual measure for the year ending September 30, 2015.
           Neither Fannie Mae nor Freddie Mac have advised DER of changes to
           remediation plans for MRAs. DER has not identified any MRAs for which
           remedial action is expected to be incomplete or outside of agreed upon
           timeframe.

      DER’s Reporting for Performance Measure 1.3.1

On October 30, 2015, approximately two weeks before FHFA published the 2015 PAR,
OBFM asked DER and DBR to provide the exact percentages each achieved for Performance
Measure 1.3.1’s target of completing remedial action for MRAs within agreed-upon
timeframes. Within one hour, DBR responded, reporting a 97% compliance rate. The DER
acting associate director forwarded OBFM’s request to each examination team. The Fannie
Mae examiner assigned to respond to OBFM’s request reported to us that he spoke with an
OBFM analyst about which MRAs should be considered for this performance measure and
that analyst sought only MRAs closed during FY 2015. That analyst advised us that he did
not recall that conversation and lacked sufficient technical knowledge to interpret the
performance measure. The Freddie Mac compliance analyst assigned to respond to OBFM’s
request reported to us that the Freddie Mac examiner-in-charge instructed her to limit her
review to MRAs closed during FY 2015. The Fannie Mae examiner identified a total of 12
closed MRAs closed during FY 2015 and determined that remediation for each was
completed within agreed-upon timeframes. The Freddie Mac compliance analyst identified
a total of 8 MRAs closed during FY 2015 and concluded that remediation for each one was
completed within agreed-upon timeframes.31 Limiting Performance Measure 1.3.1 to only

29
     The fourth quarter update contained a minor change – “is on track to meet” was replaced by “met.”
30
  OBFM expects FHFA program offices and divisions to enter performance data into its tracking system
within two weeks after the close of each quarter. For several quarters during fiscal year 2015, DER failed to
enter the quarterly update within this time frame, even though its quarterly reports were identical.
31
   Within DER, the examination staff who conducted these retrospective assessments of timeliness did not
follow the same approach. The Fannie Mae examiner assumed that Fannie Mae had timely remediated each of
the 12 MRAs closed during FY 2015 unless he found clear documentation to the contrary. The Freddie Mac
compliance analyst determined timeliness by relying on the examiners who closed each of the 8 MRAs during



                                    OIG  EVL-2017-001  November 9, 2016                                       19
those MRAs closed during FY 2015 meant that the timeliness and adequacy of remediation
for the 48 open MRAs at September 30, 2015, were not considered. As a consequence, the
timeliness and adequacy of 71% of open MRAs were not included in DER’s report, but the
PAR did not disclose these limitations.

Unlike DBR, which reviewed the timeliness and adequacy of remediation for all open and
closed MRAs for FY 2015, DER restricted the applicability of Performance Measure 1.3.1
to those 20 MRAs it closed during fiscal year 2015. According to DER’s acting associate
director, the PAR accurately reported that the Enterprises had a perfect record in remediating
MRAs. DER acknowledged that the 100% compliance rate was made possible by DER’s
decision that only MRAs closed by DER would be covered by this performance measure, and
by its practice of granting undocumented extensions to agreed upon timetables, which meant
that all MRAs closed by DER would have been timely.




FY 2015. DER officials made clear to us that “agreed upon timeframes” for MRA remediation include
extensions, and DER has previously acknowledged that it regularly grants extensions orally and does not
document them. Neither reporting examiner could recall any instance in which DER denied an extension to
Fannie Mae or Freddie Mac.



                                OIG  EVL-2017-001  November 9, 2016                                     20
CONCLUDING OBSERVATIONS ..................................................

Congress passed GPRA to assist in its policy making, oversight, and budget functions. One
of the ways GPRA accomplishes those goals is by requiring agencies to set performance goals
for each fiscal year that are “objective, quantifiable, and measurable” and by directing them to
identify limitations in reported data, including inconsistencies with data collection procedures.
Based on our assessment of FHFA’s reporting for Performance Measure 1.3.1 in the 2015
PAR, we found that FHFA sought uniform criteria to be used by DBR and DER to measure
performance but that DER and DBR used different criteria, which were not fully disclosed
to OBFM, to calculate compliance rates and those criteria materially affected their reported
compliance rates. Because OBFM was not made aware of the different criteria used by DER
and DBR, it did not report the inconsistencies with internal data collection procedures nor did
it qualify or otherwise caveat its reported compliance rates in the PAR for this performance
measure.

We make two observations that the Agency should be aware of and address in order to meet
the requirements of GPRA.

      DBR reported all open MRAs at the regulated entities it examined during FY 2015,
       regardless of whether or not they were completely remediated. DBR vested senior
       DBR officials with discretion to determine whether 25% of the open MRAs – 20 out
       of 80 – were “on track” to meet agreed upon timetables and these officials determined
       that 85% – 17 out of 20 – were “on track.” Without defined criteria, there is no basis
       for us, or anyone other than the DBR associate directors, to assess whether incomplete
       remediation efforts for 17 of these 20 open MRAs were “on track.” We can only
       determine that the associate directors’ exercise of discretion enabled DBR to achieve
       the 97% reported success rate and, absent such discretion, DBR’s success rate could
       have been as low as 75%.

      DER did not establish criteria for calculating Performance Measure 1.3.1 during FY
       2015. Although it appears from internal documents and interviews with DER officials
       that the division originally understood Performance Measure 1.3.1 to require an
       assessment of whether all open MRAs were being remediated in accordance with
       agreed upon timeframes, DER did not report based on that approach. Instead,
       approximately two weeks before FHFA published the 2015 PAR, DER calculated its
       compliance rate based on only the 20 MRAs it closed during FY 2015. This limited
       scope resulted in DER reporting a 100% compliance rate. By limiting the
       performance measure to closed MRAs, DER reported on only 29% of MRAs that had
       reached their original estimated remediation date. The timeliness and adequacy of



                             OIG  EVL-2017-001  November 9, 2016                                  21
remediation efforts for 71% of open MRAs were not included in DER’s reported
compliance rate.




                   OIG  EVL-2017-001  November 9, 2016                       22
OBJECTIVE, SCOPE, AND METHODOLOGY .................................

The objective of this report was to evaluate the basis upon which FHFA reported in its 2015
PAR that DBR and DER met Performance Measure 1.3.1: “Regulated entities complete
remedial action for Matters Requiring Attention within agreed upon timeframes” at least 90
percent of the time.

To achieve this objective, we reviewed FHFA guidance, publicly available documents, and
internal DBR, DER, and OBFM documents. We conducted interviews with DBR, DER, and
OBFM personnel. We also reviewed relevant federal regulations and guidance published by
the Office of Management and Budget.

Our work was conducted under the authority of the Inspector General Act and in accordance
with the Council of the Inspectors General on Integrity and Efficiency’s Quality Standards for
Inspection and Evaluations (January 2012). These standards require us to plan and perform
an evaluation based upon evidence sufficient to provide reasonable bases to support its
conclusions. OIG believes that this review meets these standards.

The field work for this report was completed between May and August 2016.

We provided FHFA an opportunity to respond to a draft report of this evaluation. FHFA
provided technical comments on the draft report, which we incorporated as appropriate.
FHFA’s management response to this report is reprinted in its entirety in Appendix A.




                            OIG  EVL-2017-001  November 9, 2016                                23
APPENDIX A .............................................................................

FHFA’s Management Response




                          OIG  EVL-2017-001  November 9, 2016                      24
ADDITIONAL INFORMATION AND COPIES .................................


For additional copies of this report:

      Call: 202-730-0880

      Fax: 202-318-0239

      Visit: www.fhfaoig.gov



To report potential fraud, waste, abuse, mismanagement, or any other kind of criminal or
noncriminal misconduct relative to FHFA’s programs or operations:

      Call: 1-800-793-7724

      Fax: 202-318-0358

      Visit: www.fhfaoig.gov/ReportFraud

      Write:

                FHFA Office of Inspector General
                Attn: Office of Investigations – Hotline
                400 Seventh Street SW
                Washington, DC 20219




                              OIG  EVL-2017-001  November 9, 2016                        25