Federal Housing Finance Agency Office of Inspector General FHFA’s Use of Inconsistent Criteria Materially Affected its Reporting of Remediation of Serious Deficiencies in its 2015 Performance and Accountability Report Evaluation Report EVL-2017-001 November 9, 2016 Executive Summary As the federal regulator of Fannie Mae and Freddie Mac (the Enterprises) and of the Federal Home Loan Banks (FHLBanks), the Federal Housing Finance Agency (FHFA or the Agency) is tasked by statute with ensuring that these entities operate safely and soundly so that they serve as a reliable source of liquidity and funding for housing finance and community investment. FHFA EVL-2017-001 has directed its Division of Federal Home Loan Bank Regulation (DBR) to conduct supervisory activities for the FHLBanks and its Division of Enterprise November 9, Regulation (DER) to conduct these activities of the Enterprises. 2016 The Government Performance and Results Act of 1993, as amended by the GPRA Modernization Act of 2010 (GPRA), requires FHFA (and other federal agencies) to establish strategic plans, develop performance goals aligned with those strategic plans, and set performance indicators to measure whether those goals are met. Pursuant to GPRA, each federal agency must report, after the end of each fiscal year, whether it has met its performance goals. To meet its GPRA obligations for fiscal year (FY) 2015, FHFA established three strategic goals and identified three performance goals for each strategic goal. For its first strategic goal, “Ensure Safe and Sound Regulated Entities,” FHFA set three performance goals tied to its supervisory activities: “assess the safety and soundness of regulated entity operations”; “identify risks to the regulated entities and set expectations for strong risk management”; and “require timely remediation of risk management weaknesses.” Risk management weaknesses, and other deficiencies, at a regulated entity are identified by FHFA during its supervisory activities. Where FHFA finds a risk management weakness or other deficiency, it will classify the weakness or other deficiency as a Matter Requiring Attention (MRA), a violation, or a recommendation. According to FHFA, MRAs are reserved for “the most serious supervisory matters” and require “prompt remediation” by the affected regulated entity. For FY 2015, FHFA determined that it would measure success in achieving its stated performance goal – “require timely remediation of risk management weaknesses” – by measuring whether its “[r]egulated entities complete remedial action for MRAs within agreed upon timeframes.” FHFA established a 90% target for this performance measure. FHFA sought uniform criteria to be used by DBR and DER to measure performance during FY 2015. According to FHFA’s 2015 Performance and Accountability Report (PAR), published in November 2015, its regulated entities exceeded this target during FY 2015: the “FHLBanks reported a 97% compliance rate and Fannie Mae and Freddie Mac both reported a 100% compliance rate.” Over the past year, we issued several reports regarding FHFA’s oversight of the Enterprises’ remediation of MRAs, in which we identified a number of shortcomings. In light of the outstanding performance results for this performance measure reported by FHFA in its 2015 PAR, we undertook this evaluation to assess FHFA’s bases for those reported results. Contrary to FHFA’s expectations, we found that DER and DBR used different EVL-2017-001 criteria to calculate compliance rates, which materially affected the reported compliance rates. Moreover, DER and DBR did not fully disclose their November 9, differing criteria to the FHFA office that coordinated and developed the PAR. 2016 Within DBR, senior officials were vested with complete discretion to determine whether 20 of 80 MRAs were “on track” or “off track” to meet agreed upon timetables and exercised that discretion to find that 17 of the 20 were “on track.” Absent the exercise of such discretion, DBR could have reported a compliance rate as low as 75%. From our review of internal documents and interviews with DER officials, it appears that DER initially set out to report on all MRAs open during FY 2015, as DBR did. However, DER developed no methodology during the year to capture the data to calculate a compliance rate. When asked by FHFA several weeks after the close of FY 2015 to report a compliance rate for this performance measure for the 2015 PAR, DER determined that it would report only on those MRAs it closed during FY 2015 – or only 29% of the total MRAs open at one point during FY 2015. As a consequence, DER reported a 100% compliance rate. GPRA requires each federal agency to report any limitations to data it reports, including inconsistencies with data collection procedures. Because the office responsible for coordinating the development and publication of FHFA’s PAR was not made aware of the different criteria used by DER and DBR, it did not report the inconsistencies with internal data collection procedures nor did it qualify or otherwise caveat FHFA’s reported compliance rates in the PAR for this performance measure. This report was prepared by Howard Klein, Attorney Advisor, Brian Stief, Investigative Counsel, and Timothy Callahan, Attorney Advisor. We appreciate the cooperation of FHFA staff, as well as the assistance of all those who contributed to the preparation of this report. This report has been distributed to Congress, the Office of Management and Budget, and others and will be posted on our website, www.fhfaoig.gov. Angela Choy Assistant Inspector General for Evaluations TABLE OF CONTENTS ................................................................ EXECUTIVE SUMMARY .............................................................................................................2 ABBREVIATIONS .........................................................................................................................5 BACKGROUND .............................................................................................................................6 Measuring Government Performance .......................................................................................6 FHFA’s Process to Develop the APP and PAR........................................................................7 FHFA’s Goals and Performance Measures for the 2015 PAR .................................................8 The 2015 APP Published FHFA’s Goal for Completing Remediation of MRAs, and the 2015 PAR Reported that FHFA Had Achieved that Goal ..................................................9 FACTS AND ANALYSIS.............................................................................................................11 Development of Performance Goal 1.3 and Performance Measure 1.3.1 ..............................11 DBR’s Process for Collecting and Reporting Data for Performance Measure 1.3.1 .............13 DBR’s Data Collection Process for Performance Measure 1.3.1 ...................................13 DBR’s Criteria for Assessing Whether Remedial Actions to Correct MRAs Were Completed Within Agreed Upon Timeframes ...............................................14 Exercise of Discretion in Reporting Performance Data for Performance Measure 1.3.1 .........................................................................................................................15 DER Data Collection and Reporting for Performance Measure 1.3.1 ...................................17 DER’s Data Collection Process for Performance Measure 1.3.1 ...................................17 DER’s Reporting for Performance Measure 1.3.1 ..........................................................19 CONCLUDING OBSERVATIONS ..............................................................................................21 OBJECTIVE, SCOPE, AND METHODOLOGY .........................................................................23 APPENDIX A ................................................................................................................................24 FHFA’s Management Response .............................................................................................24 ADDITIONAL INFORMATION AND COPIES .........................................................................25 OIG EVL-2017-001 November 9, 2016 4 ABBREVIATIONS ....................................................................... APP Annual Performance Plan CFO Chief Financial Officer DBR Division of Federal Home Loan Bank Regulation DER Division of Enterprise Regulation Enterprises Fannie Mae and Freddie Mac, collectively FHFA or Agency Federal Housing Finance Agency FHLBanks Federal Home Loan Banks FY Fiscal Year GPRA GPRA Modernization Act of 2010 MRA Matter Requiring Attention OBFM Office of Budget and Financial Management PAR Performance and Accountability Report ROE Report of Examination OIG EVL-2017-001 November 9, 2016 5 BACKGROUND .......................................................................... Since 2008, FHFA has operated as both regulator and conservator of Fannie Mae and Freddie Mac and regulator of the Federal Home Loan Bank system to ensure that these entities operate safely and soundly so that they serve as a reliable source of liquidity and funding for housing finance and community investment. DBR is responsible for supervising the FHLBanks and the Office of Finance. DER is responsible for supervising Fannie Mae and Freddie Mac (collectively, the Enterprises). DER uses two core examination teams to supervise the Enterprises; one team examines Fannie Mae, and the other examines Freddie Mac. Like other federal agencies, FHFA publishes an annual PAR, assessing the Agency’s performance in meeting specific goals it establishes earlier in the year. The requirements and structure of the PAR are governed by the Government Performance and Results Act of 1993,1 as amended by the GPRA Modernization Act of 2010 (GPRA).2 Measuring Government Performance When Congress passed the Government Performance and Results Act of 1993, it was hailed as “a watershed moment for the federal government.”3 The Act was the first time Congress established statutory requirements for most agencies to set goals, measure performance, and submit related plans and reports to Congress. The 1993 Act was intended to assist Congress in its policy making, oversight, and budget functions.4 GPRA, passed in 2011, built on and updated the framework of the Act, drawing on the experience of multiple administrations, the views of the authors of the 1993 Act, and other sources. GPRA requires federal agencies to establish strategic plans, to develop performance goals aligned with those strategic plans, and to establish performance indicators to assess whether those goals are met. An agency’s performance goals for each fiscal year are published in an annual performance plan (APP), and must (unless otherwise authorized by the Office of Management and Budget) be “objective, quantifiable, and measurable.” Each APP must also 1 Pub. L. No. 103-62, 107 Stat. 285 (1993). 2 Pub. L. No. 111-352, 124 Stat. 3866 (2011). 3 Cong. Research Serv., Changes to the Government Performance and Results Act (GPRA): Overview of the New Framework of Products and Processes, at 1(Feb. 29, 2012) (R42379). 4 Senate Committee on Governmental Affairs, Government Performance and Results Act of 1993, at 3, report to accompany S. 20, 103rd Cong. (June 16, 1993) (S. Rept. 103-58). OIG EVL-2017-001 November 9, 2016 6 contain performance indicators, which are the specific benchmarks against which an agency’s performance is assessed.5 Further, each APP must describe “how the agency will ensure the accuracy and reliability of the data used to measure progress toward its performance goals.” Agency leaders are “accountable for choosing goals and indicators wisely and for setting ambitious, yet realistic targets.”6 Agencies are required to post the APP for each fiscal year on their websites and to notify the President and Congress when it is available. After the end of the fiscal year, GPRA requires each federal agency to publish a PAR on its public website stating whether it has met each performance goal.7 If a performance goal is not met, the agency must explain the reasons why the goal was not met and its plans to achieve that goal in the future. According to GPRA, the PAR must also describe how the agency ensures the accuracy and reliability of its performance data and must identify any limitations to the data, including inconsistencies in an agency’s data collection procedures.8 The agency head is ultimately responsible for the accuracy of performance data published in the PAR. FHFA’s Process to Develop the APP and PAR FHFA’s Office of Budget and Financial Management (OBFM) coordinates the development and publication of FHFA’s APP and PAR for each fiscal year. Performance goals and their benchmarks (or measures) for FHFA are developed by the office or division responsible for meeting them, in collaboration with OBFM. Once FHFA offices and divisions finalize their goals and measures and OBFM assembles a complete draft of the APP, the draft APP is reviewed and approved by the FHFA Director. After the APP is published by FHFA on its website, the performance goals and measures are established for the fiscal year and are not subject to change. FHFA’s Chief Financial Officer (CFO), who is in charge of OBFM, considers the APP to set forth “stretch goals.” 5 GPRA recognizes two different types of performance indicators: “output measures,” which assess the agency’s effort in working towards its goals, and “outcome measures,” which assess the ultimate success or failure of the agency’s program. OMB guidance encourages agencies to use outcome measures, which measures actual results, whenever feasible. 6 OMB, Circular A-11, Preparation, Submission, and Execution of the Budget, Part 6, Executive Summary, at 2 (June 30, 2015). 7 GPRA also authorizes agencies to report this information in an annual performance report. For purposes of this evaluation, all such reports mandated by GPRA are referred to as a PAR. 8 OMB, Circular A-11, Preparation, Submission, and Execution of the Budget, Part 6, Section 260, at 4 (June 30, 2015). OIG EVL-2017-001 November 9, 2016 7 During each fiscal year, FHFA expects that its individual offices and divisions will collect the designated performance data, which is then entered into OBFM’s electronic performance tracking system (OBFM tracking system). The head of each office or division is responsible for the accuracy of the data reported to OBFM. At the end of each fiscal year, OBFM compiles the performance data that has been entered into the tracking system to generate the reportable results for the PAR. A draft PAR prepared by OBFM is circulated within FHFA for review and comment. Once finalized, the PAR is provided to the FHFA Director for approval, and published and posted on FHFA’s website. Because some of the goals announced by FHFA in its APP are “stretch goals,” FHFA’s CFO reported to us that, in his view, it would not be unacceptable for FHFA to occasionally miss an APP goal, especially if attainment of that goal is not entirely within the agency’s control. He acknowledged, however, that some FHFA officials were not comfortable in setting ambitious goals that might not be met. FHFA’s Goals and Performance Measures for the 2015 PAR On November 21, 2014, FHFA published its strategic plan for fiscal years 2015-2019 and that plan set forth three strategic goals and identified three performance goals for each strategic goal. Pursuant to GPRA, FHFA incorporated these three strategic and nine performance goals into its APP and PAR for FY 2015. See Figure 1. FIGURE 1. FHFA 2015 STRATEGIC AND PERFORMANCE GOALS Source: FHFA, Fiscal Year 2015 Performance and Accountability, at 14. OIG EVL-2017-001 November 9, 2016 8 The 2015 APP Published FHFA’s Goal for Completing Remediation of MRAs, and the 2015 PAR Reported that FHFA Had Achieved that Goal In its APP for FY 2015 (2015 APP), published on February 9, 2015, FHFA’s first strategic goal, Ensure Safe and Sound Regulated Entities, set forth three performance goals. Performance Goal 1.3 announced that FHFA sought to “require timely remediation of risk management weaknesses” by the regulated entities. FHFA’s published performance measure for that goal – the benchmark against which the Agency’s success or failure would be judged – was set forth in Performance Measure 1.3.1: that the “regulated entities complete remedial action for MRAs [Matters Requiring Attention] within the agreed upon timeframes” at least 90% of the time.9 In its 2015 PAR, published on November 16, 2015, FHFA reported on the results of its performance measures for each of its performance goals.10 With respect to Performance Measure 1.3.1, FHFA reported: The regulated entities met the target of completing remedial action for the MRAs within the agreed-upon time frames at least 90 percent of the time. The FHLBanks reported a 97% compliance rate and Fannie Mae and Freddie Mac both reported a 100% compliance rate. Because Performance Goal 1.3 focused on “timely remediation of risk management weaknesses” by the regulated entities, which DBR and DER are charged with supervising, the Deputy Directors of DBR and DER were responsible for collecting and accurately reporting the information for Performance Goal 1.3 and the performance measure under it— Performance Measure 1.3.1. In the Validation and Verification of Performance Data section of its 2015 PAR, FHFA represented that it had verified its “sources of data” to ensure the completeness and reliability of the information reported. Over the past year, we have issued several reports regarding FHFA’s oversight of the Enterprises’ remediation of MRAs, in which we identified a number of shortcomings.11 9 FHFA considers MRAs to be “the most serious supervisory matters,” and it issues them for “non-compliance with laws or regulations that result or may result in significant risk of financial loss or damage,” “repeat deficiencies that have escalated due to insufficient action or attention,” “unsafe or unsound practices,” and “breakdowns in risk management, significant control weaknesses, or inappropriate risk-taking.” FHFA, Advisory Bulletin 2012-01, at 2 (Apr. 2, 2012). 10 FHFA represented that the PAR met the requirements of GPRA. 11 See OIG, FHFA’s Examiners Did Not Meet Requirements and Guidance for Oversight of an Enterprise’s Remediation of Serious Deficiencies (Mar. 29, 2016) (EVL-2016-004) (www.fhfaoig.gov/Content/Files/EVL- 2016-004.pdf) [hereinafter OIG, Remediation Report]; OIG, FHFA’s Supervisory Standards for Communication of Serious Deficiencies to Enterprise Boards and for Board Oversight of Management’s OIG EVL-2017-001 November 9, 2016 9 These shortcomings included: FHFA’s failure to establish agreed upon timetables for elements of an MRA; and FHFA’s failure to review the adequacy and timeliness of the Enterprises’ MRA remediation efforts to determine whether the underlying deficiency had been corrected, resulting in a substantial backlog of open MRAs. In light of these shortcomings and the outstanding performance results for Performance Measure 1.3.1 reported by FHFA in the 2015 PAR, we undertook this evaluation to examine FHFA’s basis for the results it reported for Performance Measure 1.3.1. We have not examined FHFA’s basis for the results it reported for any other performance measures in this PAR. Remediation Efforts are Inadequate (Mar. 31, 2016) (EVL-2016-005) (www.fhfaoig.gov/Content/Files/EVL- 2016-005.pdf); OIG, FHFA’s Inconsistent Practices in Assessing Enterprise Remediation of Serious Deficiencies and Weaknesses in its Tracking Systems Limit the Effectiveness of FHFA’s Supervision of the Enterprises (July 14, 2016) (EVL-2016-007) (www.fhfaoig.gov/Content/Files/EVL-2016-007.pdf) [hereinafter OIG, Tracking Report]; OIG, FHFA’s Failure to Consistently Identify Specific Deficiencies and Their Root Causes in Its Reports of Examination Constrains the Ability of the Enterprises Boards to Exercise Effective Oversight of Management’s Remediation of Supervisory Concerns (July 14, 2016) (EVL-2016-008) (www.fhfaoig.gov/Content/Files/EVL-2016-008.pdf); OIG, FHFA’s Implementation of Its Automated System to Track Deficiencies Identified in Federal Home Loan Bank Examinations (May 26, 2016) (COM-2016-003) (www.fhfaoig.gov/Content/Files/COM-2016-003.pdf); OIG, DBR’s Unwritten Procedures and Practices for Oversight of Efforts by Federal Home Loan Banks to Correct Deficiencies Underlying the Most Serious Supervisory Matters Are Inconsistent with the Written Oversight Requirements Promulgated by FHFA (Sept. 30, 2016) (COM-2016-006) (www.fhfaoig.gov/Content/Files/COM-2016-006.pdf) [hereinafter OIG, OCom Report]. OIG EVL-2017-001 November 9, 2016 10 FACTS AND ANALYSIS ............................................................... Development of Performance Goal 1.3 and Performance Measure 1.3.1 Both DBR and DER developed, approved, and reported on Performance Goal 1.3 and Performance Measure 1.3.1 for FY 2015. While this performance goal – to “require timely remediation of risk management weaknesses” by the regulated entities – was carried over from FY 2014 to FY 2015,12 FHFA sought uniform criteria to be used by DBR and DER to measure performance against that goal during FY 2015. For FY 2014, the relevant performance measure stated: “[i]n response to examination findings, evaluate the effectiveness of remedial actions taken by the Enterprises and FHLBanks.”13 DBR understood this performance measure to focus on outcomes: it reported the percentage of MRAs that the FHLBanks had completely remediated or were on-plan to remediate. DER understood this performance measure to focus on outputs: it reported the percentage of Enterprise MRA remediation plans that DER had responded to within 60 days of submission, without reporting on efforts by the Enterprises to remediate the MRAs, in whole or in part.14 For FY 2015, DER and DBR agreed to use the same outcome-based performance measure. See Figure 2 for Performance Measure 1.3.1, as set forth in FHFA’s APP for FY 2015. FIGURE 2. PERFORMANCE MEASURE 1.3.1 FOR FY 2015 Source: FHFA, Annual Performance Plan for Fiscal Year 2015, at 10. 12 In both years, the goal remained the same but the number of that goal changed. In the 2014 APP, this goal was labeled Performance Goal 1.1 and in the 2015 APP, this goal was labeled Performance Goal 1.3. 13 In the 2014 APP, this was Performance Measure 1.2.2. 14 FHFA’s Advisory Bulletin 2012-01 directs that “[c]orrective action for MRAs must be articulated in written remediation plans, prepared by the regulated entity, acceptable to the FHFA.” OIG EVL-2017-001 November 9, 2016 11 The 2015 APP also sets forth the criteria to be used by FHFA to validate whether Performance Measure 1.3.1 has been met: Data Validation and Verification for Performance Goal 1.3 Measure 1.3.1 - FHFA discusses MRAs with Enterprise management and communicates them in writing. The Enterprises address MRAs through preparation and execution of remediation plans. FHFA reviews proposed plans and completed work to determine whether identified deficiencies have been addressed. FHFA tracks remediation and closure of MRAs. All FHLBanks’ Reports of Examination contain a summary of examination issues identified and MRAs, which include follow-up dates by which the FHLBanks are to resolve the identified issues. For the FHLBanks, DBR discusses each MRA with the board of the relevant FHLBank or the Office of Finance. When resolved to FHFA’s satisfaction, DBR informs the FHLBank and closes the MRA. The target is considered met when FHFA confirms that identified weaknesses have been remediated within the agreed upon timeframe.15 Senior DBR and DER officials reviewed and provided feedback on both Performance Measure 1.3.1 and the data validation and verification for Performance Measure 1.3.1. The Deputy Directors of DBR and DER signed off on the APP in February 2015. Despite DBR’s and DER’s agreement on the definition for Performance Measure 1.3.1, DBR and DER used different criteria to report their performance. DBR reported on the adequacy and timeliness of remediation for open MRAs issued during previous examination cycles for each FHLBank that the division examined in FY 2015. DER reported only on the timeliness of those 20 MRAs it closed during FY 2015. This difference was not explained in the 2015 PAR. Because OBFM was not made aware of the different criteria used by DER and DBR, it did not report the inconsistencies with internal data collection procedures nor did it qualify or otherwise caveat FHFA’s reported compliance rates in the PAR for this performance measure. 15 FHFA, Annual Performance Plan for Fiscal Year 2015, at 10. DBR and DER officials separately acknowledged to us that they understood the word “confirms,” as used in the APP Data Validation and Verification section, to mean that their respective staff would review the remedial actions taken by the regulated entity to determine the sufficiency and timeliness of those actions. A statement by a regulated entity that it remediated an MRA would not meet the validation criteria. OIG EVL-2017-001 November 9, 2016 12 DBR’s Process for Collecting and Reporting Data for Performance Measure 1.3.1 DBR established a process for tracking and updating quarterly its compliance rate for Performance Measure 1.3.1 based on the results of its on-site examinations of the FHLBanks. The division reported a 97% compliance rate in the 2015 PAR. Seventy-five percent of the MRAs DBR included in its reporting were deemed fully remediated. The remaining 25% were not fully remediated and were reported as “on track” or “off track” based solely on the discretionary determinations of senior DBR officials. DBR’s Data Collection Process for Performance Measure 1.3.1 DBR’s process for gathering performance data for Performance Measure 1.3.1 was tied to its supervisory schedule. As supervisor of the 11 regional FHLBanks and the Office of Finance,16 DBR assesses their safety and soundness primarily through annual, on-site examinations.17 Three associate directors in DBR manage DBR’s safety and soundness examinations of the FHLBanks and Office of Finance.18 DBR examination teams spend five to six weeks on-site at each entity it regulates reviewing its risk management and operations.19 When conducting their supervisory activities, DBR examiners may identify deficiencies that result in findings, including MRAs. The affected FHLBank is usually given one year or less within which to completely remediate the MRA.20 DBR expects its examiners to assess the adequacy and timeliness of remediation of an MRA by a regulated entity during the next scheduled annual examination and record their determinations (e.g., “Resolved,” “Partially Remediated,” “Re-issued,” “Repeated,” or “Closed”) in the Report of Examination (ROE) issued to the FHLBank’s board of directors.21 16 Each FHLBank is cooperatively owned by its respective member financial institutions, including banks, thrifts, and insurance companies. The primary business activity of each FHLBank is to provide loans, known as advances, to their members to support housing finance and for other purposes. FHLBanks may also invest in U.S. Treasury securities, certain mortgage assets, and other types of assets as permitted by law and FHFA regulations. 17 DBR also conducts periodic visits, special reviews, and off-site monitoring of FHLBank financial data. 18 Each associate director was responsible for four regulated entities. 19 According to a DBR official, a typical DBR examiner participates in four annual FHLBank examinations per year and spends approximately five months on travel doing so. 20 Throughout this report, references to the FHLBanks include the Office of Finance. 21 FHFA, like other federal financial regulators, produces an ROE in conjunction with its supervision of each regulated entity. According to FHFA, the ROE communicates to the board of directors of a regulated entity substantive examination results and conclusions. DER issues an ROE to each Enterprise at the end of each annual supervisory cycle, and DBR issues an ROE to each FHLBank after completing the bank’s annual on- site examination. OIG EVL-2017-001 November 9, 2016 13 DBR’s Deputy Director approved the APP on February 2, 2015. DBR tasked its executive advisor, an official who reports directly to the Deputy Director, with responsibility for gathering and reporting quarterly performance data for Performance Measure 1.3.1. DBR’s executive advisor documented in the OBFM tracking system the process used to gather that data during FY 2015: To determine our actual success rate, we will poll the Associate Directors . . . to determine what proportion of MRAs were fully addressed or are on schedule for completion. Quarterly, each Associate Director will determine whether outstanding MRAs reviewed during the examination were sufficiently addressed. We will keep a running total and by year-end have our total success rate. To obtain the required performance data for Performance Measure 1.3.1, DBR’s executive advisor sent the following written request to DBR’s three associate directors: At the [FHLBank or Office of Finance] examiners reviewed MRAs from the prior examination. What proportion of MRAs were remediated by the FHLBank or are on-plan to be remediated. . . [A]nswers should be like “4 out of 4” or “3 out of 5.” Each associate director obtained that information and sent it by email to the executive advisor, who recorded the data in the OBFM tracking system. DBR’s Deputy Director then reviewed and approved each quarterly entry. We found no evidence that DBR performed any additional validation of the data’s integrity. DBR gathered and reported data on the status of all open MRAs that had been issued during (or before) the calendar year 2014 examination cycle for the regulated entities it reviewed during the calendar year 2015 examination cycle. DBR’s Criteria for Assessing Whether Remedial Actions to Correct MRAs Were Completed Within Agreed Upon Timeframes DBR determined that Performance Measure 1.3.1 had been met when examiners either: (1) confirmed that the FHLBank completely remediated the MRA within the timeframe initially established or any extension that DBR had granted; or (2) determined that remediation of the MRA by the FHLBank was not complete but the FHLBank’s remedial For a fulsome discussion of the assessments made by DBR examiners regarding the timeliness and adequacy of remedial actions taken by an FHLBank to address an MRA, see OIG, OCom Report, supra note 11. OIG EVL-2017-001 November 9, 2016 14 actions were “on track” and remediation was expected to be complete by the agreed upon deadline, including any extensions. In a recent special project report,22 we assessed DBR’s oversight of the FHLBanks’ remediation of MRAs by reviewing a sample of nine MRAs issued by DBR from January 2014 through September 2015. For four of the nine MRAs in our sample, we found that DBR examiners determined, based on their assessment of corrective actions during the subsequent annual examination of the affected FHLBank, that the corrective actions were inadequate to sufficiently remediate the MRA. In each of these four instances, the DBR examiners closed the MRA and reissued it in whole or in part and extended the remediation timetable for at least one year from the date of reissuance. For purposes of reporting performance data for Performance Measure 1.3.1 for the PAR, each associate director had discretion to determine whether a re-issued MRA, or a partially remediated MRA for which an extension was granted, was “on track.” As we now discuss, the associate directors’ exercise of discretion was material to DBR’s report of a 97% compliance rate by its regulated entities in meeting Performance Measure 1.3.1. Exercise of Discretion in Reporting Performance Data for Performance Measure 1.3.1 The 2015 quarterly totals that the executive advisor entered into the OBFM tracking system are set forth in Figure 3. FIGURE 3. DBR PERFORMANCE DATA FOR PERFORMANCE MEASURE 1.3.1 AS REPORTED IN THE OBFM TRACKING SYSTEM Reporting Quarter 1Q2015 2Q2015 3Q2015 4Q2015 Total Percentage of MRAs 22 of 22 15 of 15 34 of 36 13 of 14 84 of 87 Fully Remediated or (100%) (100%) (94%) (93%) (97%) On Track Office of Indianapolis, New York, Indianapolis, Regulated* Entities Finance, Chicago, Dallas, Boston, Examined San Francisco, Topeka Pittsburgh Cincinnati Atlanta * DBR’s examination schedule is based on a calendar year while, the PAR is based on the fiscal year. As a result, not every FHLBank may be examined during a fiscal year. Similarly, an FHLBank could be examined twice in one fiscal year and, thus, DBR would provide Performance Measure 1.3.1 data twice for one FHLBank, as it did for the Indianapolis FHLBank in 2015. 22 OIG, OCom Report, supra note 11. OIG EVL-2017-001 November 9, 2016 15 Following the fourth quarter entry, the executive advisor entered a final statement in the OBFM tracking system confirming that DBR had successfully met the performance measure for Performance Measure 1.3.1: [T]he running total for the year was . . . a 97% success rate. As our target was “90 percent of the time” for the year, we have met this Performance Measure for FY 2015. The Deputy Director of DBR approved each quarterly update and the final performance results through the tracking system.23 We found that DBR’s calculation of a 97% compliance rate for Performance Measure 1.3.1 was based, in material part, on discretionary decisions by the associate directors as to whether partially remediated MRAs were on track or off track. Of the 80 MRAs reported by the associate directors as subject to Performance Measure 1.3.1, 60 MRAs had been fully remediated and DBR determined that remediation for each had been completed within the agreed upon timeframe. For the remaining 20, DBR: extended the timetable for 4; was awaiting final validation by the FHLBank for 1; assessed the adequacy of ongoing remediation efforts where the original deadline had not occurred for 5; and reissued 10 in whole or in part. The associate directors were vested with discretion to determine whether Performance Measure 1.3.1 was met for each of these 20 but the criteria to be used to exercise that discretion was not defined: the associate directors determined that 17 were on track and met the performance measure and 3 were “off track” and did not meet the performance measure.24 Without defined criteria, there is no basis for us, or anyone other than the DBR associate directors, to assess whether incomplete remediation efforts for 17 of these 20 open MRAs were “on track.” We can only determine that the associate directors’ exercise of discretion enabled DBR to achieve the 97% reported success rate and, absent such discretion, DBR’s success rate could have been as low as 75%. 23 While the final entry in the OBFM tracking system reported that the 97% compliance rate for Performance Measure 1.3.1 was based on the disposition of 87 MRAs, we found, based on documents provided by DBR, that the 97% compliance rate was based on a review of 80, rather than 87, MRAs reported by the associate directors to the executive advisor. The corrected compliance rate would be 96%. The executive advisor attributed the difference to human error. 24 During our fieldwork, DBR informed us that 3 of the 20 discretionary MRAs were reported as meeting Performance Goal 1.3.1 when they did not. In two instances, the associate director who misreported those MRAs attributed the error to his reliance on a draft ROE that indicated that these MRAs had been adequately remediated. The final ROE, which issued later, concluded that the two MRAs had not been completely remediated. In the third instance, a miscommunication between DBR and management resulted in an MRA being reported on track when it was not. After accounting for the three misreported MRAs, DBR’s compliance rate for Performance Measure 1.3.1 would be approximately 93%. OIG EVL-2017-001 November 9, 2016 16 DER Data Collection and Reporting for Performance Measure 1.3.1 DER did not establish any criteria to gauge compliance for Performance Measure 1.3.1 during FY 2015. Based on our review of DER’s internal documents and interviews with DER officials, it appears that DER senior officials originally understood Performance Measure 1.3.1 to require an assessment of whether all open MRAs were being remediated in accordance with agreed upon timetables. That understanding, however, was not communicated to the core examination teams for each Enterprise. Although DER recorded identical updates each quarter in the OBFM tracking system, it did not report a compliance rate or assess the status of any individual MRA in support of those updates for any quarter. DER closes an MRA after it determines that the deficiency giving rise to the MRA has been adequately corrected by the affected Enterprise. DER closed a total of 20 MRAs for both Enterprises during FY 2015. At the end of FY 2015, a total of 48 MRAs remained open for both Enterprises. When OBFM asked DER to calculate its compliance rate for Performance Measure 1.3.1 on October 30, 2015, approximately two weeks before the PAR was published, DER directed its examination staff to assess the timeliness of remediation for the 20 MRAs closed during FY 2015. Not surprisingly, the examination staff found a 100% compliance rate. DER’s Data Collection Process for Performance Measure 1.3.1 DER supervises the Enterprises through targeted examinations and ongoing monitoring during the year. When conducting these supervisory activities, DER examiners may identify deficiencies that result in one or more MRAs. After DER issues an MRA, FHFA’s internal guidance requires the affected Enterprise to submit a remediation plan with a timetable for remedial actions, including interim milestones, to DER within 60 days.25 Advisory Bulletin 2012-01 directs DER examiners to check on the progress of remediation during the remedial period and document their findings.26 When the Enterprise completes its remediation efforts, Enterprise management submits a closure package to its internal audit department for validation.27 Once the internal audit department completes its validation efforts, the Enterprise submits the closure package to DER for review. DER examiners are expected to 25 As we demonstrated in an earlier report this year, Enterprise remediation plans do not always propose timetables for all required remediation and DER has accepted those plans without requiring a complete timetable. See OIG, Remediation Report, supra note 11, at 23-24. 26 In a report issued earlier this year, we found that DER examiners did not consistently follow these requirements and did not assess and track Enterprise progress against remediation plans. See OIG, Tracking Report, supra note 11, at 21. 27 DER considers an MRA to be “validated” when the internal audit department concludes that the remedial action is implemented, effective, and sustainable. For more information on the internal audit department’s role in validating Enterprise remedial action, see id., at 17-18. OIG EVL-2017-001 November 9, 2016 17 review the internal audit department’s validation and close the MRA if they determine that it has been satisfactorily remediated. DER imposes no timeframe on examiners for completing this review. DER’s Deputy Director approved the APP on February 2, 2015. Based on our review of internal documents and interviews with DER officials, it appears that DER originally understood Performance Measure 1.3.1 to require an assessment of timeliness and adequacy of remedial efforts for all open MRAs. However, when DER reported its compliance rate for the 2015 PAR, it only considered those 20 MRAs it closed during FY 2015 and did not consider the timeliness of remediation for the remaining 48 MRAs. An acting associate director was tasked with gathering and reporting the quarterly performance data for Performance Measure 1.3.1. Unlike DBR, however, DER did not provide its examination staff with a written explanation of the method to be used to track and calculate the compliance rate for the performance measure. Instead, DER provided the following explanation of the measure for the OBFM tracking system: This performance measure is designed to ensure that the on-site examination teams follow up with Fannie Mae and Freddie Mac regarding examination findings and conclusions that identify weaknesses that should be remediated to strengthen Enterprise risk management. Remediation by the Enterprises should take place within a timeframe to minimize any ongoing risk to the Enterprise operations. The DER acting associate director advised us that he did not explain to the Fannie Mae and Freddie Mac examination staff responsible for tracking the progress of MRA remediation the type of performance data that should be collected for Performance Measure 1.3.1. While he maintained that he gathered performance data for this measure by speaking with examination staff, he did not document the information he gathered during those conversations.28 The acting associate director, or an employee in his office, updated the OBFM tracking system each quarter with the following identical reports: DER is on track to meet this annual measure for the year ending September 30, 2015. Neither Fannie Mae nor Freddie Mac have advised DER of changes to remediation plans for MRAs. DER has not identified any MRAs for which 28 The examination staff responsible for tracking MRAs did not recall any such oral requests for Performance Measure 1.3.1 performance data during FY 2015. OIG EVL-2017-001 November 9, 2016 18 the remedial action is expected to be incomplete or outside the agreed upon timeframes.29 These updates contained no other information. The DER Deputy Director reviewed and approved each of these quarterly reports.30 At the direction of DER’s Deputy Director, OBFM entered the following information into its tracking system on October 28, 2015: DER met this annual measure for the year ending September 30, 2015. Neither Fannie Mae nor Freddie Mac have advised DER of changes to remediation plans for MRAs. DER has not identified any MRAs for which remedial action is expected to be incomplete or outside of agreed upon timeframe. DER’s Reporting for Performance Measure 1.3.1 On October 30, 2015, approximately two weeks before FHFA published the 2015 PAR, OBFM asked DER and DBR to provide the exact percentages each achieved for Performance Measure 1.3.1’s target of completing remedial action for MRAs within agreed-upon timeframes. Within one hour, DBR responded, reporting a 97% compliance rate. The DER acting associate director forwarded OBFM’s request to each examination team. The Fannie Mae examiner assigned to respond to OBFM’s request reported to us that he spoke with an OBFM analyst about which MRAs should be considered for this performance measure and that analyst sought only MRAs closed during FY 2015. That analyst advised us that he did not recall that conversation and lacked sufficient technical knowledge to interpret the performance measure. The Freddie Mac compliance analyst assigned to respond to OBFM’s request reported to us that the Freddie Mac examiner-in-charge instructed her to limit her review to MRAs closed during FY 2015. The Fannie Mae examiner identified a total of 12 closed MRAs closed during FY 2015 and determined that remediation for each was completed within agreed-upon timeframes. The Freddie Mac compliance analyst identified a total of 8 MRAs closed during FY 2015 and concluded that remediation for each one was completed within agreed-upon timeframes.31 Limiting Performance Measure 1.3.1 to only 29 The fourth quarter update contained a minor change – “is on track to meet” was replaced by “met.” 30 OBFM expects FHFA program offices and divisions to enter performance data into its tracking system within two weeks after the close of each quarter. For several quarters during fiscal year 2015, DER failed to enter the quarterly update within this time frame, even though its quarterly reports were identical. 31 Within DER, the examination staff who conducted these retrospective assessments of timeliness did not follow the same approach. The Fannie Mae examiner assumed that Fannie Mae had timely remediated each of the 12 MRAs closed during FY 2015 unless he found clear documentation to the contrary. The Freddie Mac compliance analyst determined timeliness by relying on the examiners who closed each of the 8 MRAs during OIG EVL-2017-001 November 9, 2016 19 those MRAs closed during FY 2015 meant that the timeliness and adequacy of remediation for the 48 open MRAs at September 30, 2015, were not considered. As a consequence, the timeliness and adequacy of 71% of open MRAs were not included in DER’s report, but the PAR did not disclose these limitations. Unlike DBR, which reviewed the timeliness and adequacy of remediation for all open and closed MRAs for FY 2015, DER restricted the applicability of Performance Measure 1.3.1 to those 20 MRAs it closed during fiscal year 2015. According to DER’s acting associate director, the PAR accurately reported that the Enterprises had a perfect record in remediating MRAs. DER acknowledged that the 100% compliance rate was made possible by DER’s decision that only MRAs closed by DER would be covered by this performance measure, and by its practice of granting undocumented extensions to agreed upon timetables, which meant that all MRAs closed by DER would have been timely. FY 2015. DER officials made clear to us that “agreed upon timeframes” for MRA remediation include extensions, and DER has previously acknowledged that it regularly grants extensions orally and does not document them. Neither reporting examiner could recall any instance in which DER denied an extension to Fannie Mae or Freddie Mac. OIG EVL-2017-001 November 9, 2016 20 CONCLUDING OBSERVATIONS .................................................. Congress passed GPRA to assist in its policy making, oversight, and budget functions. One of the ways GPRA accomplishes those goals is by requiring agencies to set performance goals for each fiscal year that are “objective, quantifiable, and measurable” and by directing them to identify limitations in reported data, including inconsistencies with data collection procedures. Based on our assessment of FHFA’s reporting for Performance Measure 1.3.1 in the 2015 PAR, we found that FHFA sought uniform criteria to be used by DBR and DER to measure performance but that DER and DBR used different criteria, which were not fully disclosed to OBFM, to calculate compliance rates and those criteria materially affected their reported compliance rates. Because OBFM was not made aware of the different criteria used by DER and DBR, it did not report the inconsistencies with internal data collection procedures nor did it qualify or otherwise caveat its reported compliance rates in the PAR for this performance measure. We make two observations that the Agency should be aware of and address in order to meet the requirements of GPRA. DBR reported all open MRAs at the regulated entities it examined during FY 2015, regardless of whether or not they were completely remediated. DBR vested senior DBR officials with discretion to determine whether 25% of the open MRAs – 20 out of 80 – were “on track” to meet agreed upon timetables and these officials determined that 85% – 17 out of 20 – were “on track.” Without defined criteria, there is no basis for us, or anyone other than the DBR associate directors, to assess whether incomplete remediation efforts for 17 of these 20 open MRAs were “on track.” We can only determine that the associate directors’ exercise of discretion enabled DBR to achieve the 97% reported success rate and, absent such discretion, DBR’s success rate could have been as low as 75%. DER did not establish criteria for calculating Performance Measure 1.3.1 during FY 2015. Although it appears from internal documents and interviews with DER officials that the division originally understood Performance Measure 1.3.1 to require an assessment of whether all open MRAs were being remediated in accordance with agreed upon timeframes, DER did not report based on that approach. Instead, approximately two weeks before FHFA published the 2015 PAR, DER calculated its compliance rate based on only the 20 MRAs it closed during FY 2015. This limited scope resulted in DER reporting a 100% compliance rate. By limiting the performance measure to closed MRAs, DER reported on only 29% of MRAs that had reached their original estimated remediation date. The timeliness and adequacy of OIG EVL-2017-001 November 9, 2016 21 remediation efforts for 71% of open MRAs were not included in DER’s reported compliance rate. OIG EVL-2017-001 November 9, 2016 22 OBJECTIVE, SCOPE, AND METHODOLOGY ................................. The objective of this report was to evaluate the basis upon which FHFA reported in its 2015 PAR that DBR and DER met Performance Measure 1.3.1: “Regulated entities complete remedial action for Matters Requiring Attention within agreed upon timeframes” at least 90 percent of the time. To achieve this objective, we reviewed FHFA guidance, publicly available documents, and internal DBR, DER, and OBFM documents. We conducted interviews with DBR, DER, and OBFM personnel. We also reviewed relevant federal regulations and guidance published by the Office of Management and Budget. Our work was conducted under the authority of the Inspector General Act and in accordance with the Council of the Inspectors General on Integrity and Efficiency’s Quality Standards for Inspection and Evaluations (January 2012). These standards require us to plan and perform an evaluation based upon evidence sufficient to provide reasonable bases to support its conclusions. OIG believes that this review meets these standards. The field work for this report was completed between May and August 2016. We provided FHFA an opportunity to respond to a draft report of this evaluation. FHFA provided technical comments on the draft report, which we incorporated as appropriate. FHFA’s management response to this report is reprinted in its entirety in Appendix A. OIG EVL-2017-001 November 9, 2016 23 APPENDIX A ............................................................................. FHFA’s Management Response OIG EVL-2017-001 November 9, 2016 24 ADDITIONAL INFORMATION AND COPIES ................................. For additional copies of this report: Call: 202-730-0880 Fax: 202-318-0239 Visit: www.fhfaoig.gov To report potential fraud, waste, abuse, mismanagement, or any other kind of criminal or noncriminal misconduct relative to FHFA’s programs or operations: Call: 1-800-793-7724 Fax: 202-318-0358 Visit: www.fhfaoig.gov/ReportFraud Write: FHFA Office of Inspector General Attn: Office of Investigations – Hotline 400 Seventh Street SW Washington, DC 20219 OIG EVL-2017-001 November 9, 2016 25
FHFA's Use of Inconsistent Criteria Materially Affected its Reporting of Remediation of Serious Deficiencies in its 2015 Performance and Accountability Report
Published by the Federal Housing Finance Agency, Office of Inspector General on 2016-11-09.
Below is a raw (and likely hideous) rendition of the original report. (PDF)