oversight

The Gap in FHFA's Quality Control Review Program Increases the Risk of Inaccurate Conclusions in its Reports of Examination of Fannie Mae and Freddie Mac

Published by the Federal Housing Finance Agency, Office of Inspector General on 2017-08-17.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

           Federal Housing Finance Agency
               Office of Inspector General




The Gap in FHFA’s Quality Control
   Review Program Increases the
Risk of Inaccurate Conclusions in its
     Reports of Examination of
   Fannie Mae and Freddie Mac




Evaluation Report • EVL-2017-006 • August 17, 2017
                  Executive Summary
                  Since 2008, FHFA has operated as both regulator and conservator of Fannie
                  Mae and Freddie Mac (collectively, the Enterprises) and regulator of the
                  Federal Home Loan Banks (FHLBanks) to ensure that they operate safely and
                  soundly so that they serve as a reliable source of liquidity and funding for
                  housing finance and community investment. FHFA’s supervision program is
 EVL-2017-006     carried out by two divisions: the Division of Enterprise Regulation (DER),
                  which is responsible for supervising the Enterprises, and the Division of
August 17, 2017   Federal Home Loan Bank Regulation (DBR), which is responsible for
                  supervising the FHLBanks.

                  Each year, DER supervises the Enterprises through a series of targeted
                  examinations and ongoing monitoring activities. Since 2014, DER has relied
                  largely on its ongoing monitoring activities to examine the Enterprises. At the
                  conclusion of each annual examination cycle, FHFA prepares and transmits
                  a report of examination (ROE) to the board of directors (Board) for each
                  Enterprise. The annual ROE constitutes DER’s “primary work product that
                  communicates . . . the cumulative results of [DER’s] supervisory activities
                  conducted during the annual examination cycle.” Each ROE also contains
                  numerical ratings that FHFA assigns for seven component areas, a rating
                  system known as CAMELSO. In addition, FHFA assigns a composite rating
                  for each Enterprise’s overall safety, soundness, and risk management practices.

                  The FHFA Office of Inspector General (OIG) has identified FHFA’s
                  supervision program as a significant risk area and assessed the program in
                  numerous audit and evaluation reports and compliance reviews. In this
                  evaluation, we reviewed DER’s processes for assigning CAMELSO ratings to
                  the Enterprises and documenting the bases for those ratings. We found that
                  DER examination managers prepare a draft ROE narrative that contains a
                  proposed rating for each CAMELSO component within their purview. The
                  examination managers then submit their draft narratives to the examiner-in-
                  charge (EIC), who edits the narratives and compiles them into a draft ROE for
                  the Deputy Director’s approval. Based on a review of documentation for two
                  CAMELSO ratings assigned to Fannie Mae and one CAMELSO rating
                  assigned to Freddie Mac for the 2015 examination cycle, we determined that
                  DER’s ratings and the corresponding language in the 2015 ROEs were
                  traceable to supporting DER examination workpapers.

                  During the course of our review of DER’s processes for assigning CAMELSO
                  ratings, we learned that, contrary to a 2013 FHFA supervision directive,
                  DER’s independent quality control review program—which is intended to
                  confirm that examination findings and conclusions are adequately supported
                  before DER communicates them to the Enterprises—does not review the
                  ROEs or the CAMELSO ratings that DER assigns to the Enterprises. Instead,
                  as a proxy, DER performs a quality control review of examination findings
                  and conclusions that it sends to the Enterprises in certain supervisory
                  correspondence throughout the year. According to a DER official, these
                  reviews make it unnecessary to perform quality control reviews of the ROEs
                  and the CAMELSO ratings because the pertinent information has been
                  assessed in previous quality control reviews. However, ongoing monitoring
 EVL-2017-006     activities—unlike targeted examinations—do not necessarily result in formal
                  supervisory correspondence to the Enterprises that trigger a quality control
August 17, 2017
                  review. Unless such correspondence is sent, DER does not perform a quality
                  control review of the conclusions of the ongoing monitoring activity.

                  Had all findings and conclusions in ROEs issued by DER been subject to a
                  quality control review, DER’s position—that a quality control review of the
                  ROE itself and the CAMELSO ratings would be redundant—would be
                  reasonable. Based on our review of DER documents and discussions with
                  DER officials, we determined that the ROEs issued for the 2015 supervisory
                  cycle contained conclusions derived from ongoing monitoring activities
                  that had not been previously communicated to the Enterprises in formal
                  supervisory correspondence. As a result, those conclusions did not undergo
                  a quality control review, revelaing a gap in DER’s quality control review
                  program. This gap presents a risk that an ROE will assure an Enterprise’s
                  board of directors that management is meeting FHFA’s supervisory
                  expectations or making progress in addressing weaknesses, when it is not.

                  We make one recommendation to FHFA to address this shortcoming. FHFA
                  agreed with our recommendation.

                  This report was prepared by Howard Klein, Attorney-Advisor; Minh-Tu
                  Greenburg, Investigative Counsel; and Philip Noyovitz, Senior Auditor; with
                  assistance from Jacob Kennedy, Senior Investigative Evaluator; and Shane
                  Hammond, Investigative Counsel. We appreciate the cooperation of FHFA
                  staff, as well as the assistance of all those who contributed to the preparation
                  of this report.

                  This report has been distributed to Congress, the Office of Management and
                  Budget, and others and will be posted on our website, www.fhfaoig.gov.




                  Angela Choy
                  Assistant Inspector General for Evaluations
TABLE OF CONTENTS ................................................................
EXECUTIVE SUMMARY .............................................................................................................2

ABBREVIATIONS .........................................................................................................................5

BACKGROUND .............................................................................................................................6

FACTS AND ANALYSIS...............................................................................................................8
      DER’s Support for its CAMELSO Ratings of the Enterprises .................................................8
      Despite the Clear Requirements in Supervision Directive 2013-01, DER’s Quality
      Control Review Program Did Not Include Review of CAMELSO Ratings and ROEs ...........9
      DER’s Quality Control Program Does Not Review All Examination Conclusions
      from Ongoing Monitoring Activities ......................................................................................11
             DER’s 2015 ROEs Communicated Examination Conclusions that Did Not
             Undergo Quality Control Reviews..................................................................................14

FINDINGS .....................................................................................................................................16
      1. The three 2015 CAMELSO component ratings and corresponding ROE
      language that we reviewed were traceable to supporting examination workpapers. ..............16
      2. A gap exists in the design of DER’s quality control review process, resulting
      in an increased risk that an ROE may inaccurately report that an Enterprise
      is meeting supervisory expectations or making progress in addressing weaknesses. ............16
      3. DER is aware of the gap in its quality control review program, but has not taken
      action to rectify it. ...................................................................................................................17

CONCLUSION ..............................................................................................................................17

RECOMMENDATION .................................................................................................................18

OBJECTIVE, SCOPE AND METHODOLOGY ..........................................................................19

APPENDIX: FHFA MANAGEMENT RESPONSE ....................................................................20

ADDITIONAL INFORMATION AND COPIES .........................................................................22




                                            OIG • EVL-2017-006 • August 17, 2017                                                              4
ABBREVIATIONS .......................................................................

DBR                   Division of Federal Home Loan Bank Regulation

DER                   Division of Enterprise Regulation

EIC                   Examiner-in-charge

Enterprises           Fannie Mae and Freddie Mac, collectively

FHFA or Agency        Federal Housing Finance Agency

FHLBanks              Federal Home Loan Banks

MRA                   Matter Requiring Attention

OIG                   Federal Housing Finance Agency Office of Inspector General

OQA                   Office of Quality Assurance

OSAB                  Oversight & Supervision Administration Branch

ROE                   Report of Examination

SD                    Supervision Directive




                          OIG • EVL-2017-006 • August 17, 2017                      5
BACKGROUND ..........................................................................

Created by Congress in 2008, the Federal Housing Finance Agency (FHFA) is charged by
the Housing and Economic Recovery Act of 2008 with oversight of the Enterprises and the
Federal Home Loan Bank System. Since September 2008, FHFA has also been the
conservator of the Enterprises. FHFA’s mission is to ensure that its regulated entities operate
safely and soundly so that they serve as a reliable source of liquidity and funding for housing
finance and community investment. FHFA strives to meet this responsibility, in part, through
its supervision program. DBR is responsible for supervising the Federal Home Loan Banks
and DER is responsible for supervising the Enterprises.

DER supervises the Enterprises through targeted examinations and ongoing monitoring
activities. According to FHFA, targeted examinations enable examiners to conduct a deep or
comprehensive assessment of selected areas of high importance or risk, while the purpose of
ongoing monitoring is to analyze real-time information and to use those analyses to identify
Enterprise practices and changes in an Enterprise’s risk profile that may warrant supervisory
attention. DER may conduct ongoing monitoring or targeted examinations to assess the
Enterprises’ remediation of serious deficiencies and the Enterprises’ adherence to supervisory
guidance and conservatorship directives.

During the course of an ongoing monitoring activity or a targeted examination, DER may
identify significant deficiencies related to risk management, risk exposure, or violations of
laws, regulations, or orders affecting the performance or condition of a regulated entity.
These identified deficiencies are known as “adverse examination findings” (findings). FHFA
uses three different types of findings: Matters Requiring Attention (MRAs), Violations, and
Recommendations.

Throughout the annual examination cycle, DER communicates safety and soundness
concerns to the Enterprises through supervisory correspondence. At the close of each targeted
examination, DER issues a conclusion letter to Enterprise management, regardless of whether
findings are made. At the conclusion of an ongoing monitoring activity that results in
findings, DER issues a supervisory letter to Enterprise management. 1 When DER undertakes
an ongoing monitoring activity to assess an Enterprise’s efforts to remediate an MRA and


1
  Prior to March 2016, DER addressed conclusion and supervisory letters to Enterprise management, not to the
board of directors or a board committee of an Enterprise. In response to an OIG recommendation, FHFA now
requires that any conclusion or supervisory letter that includes an MRA also be sent to the chair of the board’s
audit committee of the affected Enterprise. See OIG, FHFA’s Supervisory Standards for Communication of
Serious Deficiencies to Enterprise Boards and for Board Oversight of Management’s Remediation Efforts are
Inadequate, at 20 (Mar. 31, 2016) (EVL-2016-005) (online at www.fhfaoig.gov/Content/Files/EVL-2016-
005.pdf).



                                    OIG • EVL-2017-006 • August 17, 2017                                           6
determines that remediation satisfactorily addresses the deficiency identified in the MRA, it
sends a remediation letter to the Enterprise. 2 DER may also identify supervisory concerns
during ongoing monitoring that do not rise to the level of findings, or may conclude that an
Enterprise has met supervisory expectations in a particular area. In those instances, however,
DER is not required to, and generally does not, send a supervisory letter to the affected
Enterprise.

At the conclusion of each annual examination cycle, FHFA prepares and transmits an ROE
to the Board for each Enterprise. The annual ROE constitutes DER’s “primary work product
that communicates . . . the cumulative results of [DER’s] supervisory activities conducted
during the annual examination cycle.” The ROE rolls up the substantive examination results
from DER’s targeted examinations and ongoing monitoring activities. In addition, the ROE
contains numerical ratings that FHFA assigns to each Enterprise for seven component areas,
a rating system known as CAMELSO, 3 and a composite rating reflecting FHFA’s overall
assessment of each Enterprise’s safety, soundness, and risk management practices. 4

FHFA has issued an Examination Manual to guide its examiners in conducting examinations.
From time to time, it supplements its Manual with supervision directives that impose
additional requirements on DER and DBR. In March 2013, FHFA issued Supervision
Directive 2013-01 (SD 2013-01), directing DER and DBR to perform quality control reviews
of “examination findings, conclusions, ratings, supporting workpapers, and related
documents” and the ROEs, prior to finalizing ROEs. 5 The then-Deputy Directors of DBR and
DER provided input into the content of SD 2013-01 and formally approved it. Previously, the
then-Deputy Director of DER had committed in writing to develop and implement a quality
control review program by December 2012.




2
 In technical comments, DER stated that if it concludes that an Enterprise’s efforts do not satisfactorily
address an MRA, it will also send a remediation letter.
3
  The seven CAMELSO components are: Capital, Asset Quality, Management, Earnings, Liquidity,
Sensitivity to Market Risk, and Operational Risk. FHFA rates each component on a scale of 1 to 5. A rating
of 1 represents the lowest level of supervisory concern and a rating of 5 represents the highest level.
CAMELSO is similar to the “CAMELS” rating system used by federal banking regulators for depository
institutions.
4
  A composite rating of 1 or 2 reflects FHFA’s conclusion that the regulated entity is generally sound; a
composite rating of 3 indicates that the entity has moderate to severe weaknesses and needs improvement; and
a composite rating of 4 or 5 reflects the need for supervisory intervention. CAMELSO ratings constitute
confidential supervisory information and are not publicly disclosed.
5
 FHFA, Supervisory Directive 2013-01, Quality Control Program for Examinations Conducted by the
Division of Bank Regulation and the Division of Enterprise Regulation (Mar. 25, 2013), which was rescinded
and replaced by Supervision Directive 2017-01, Quality Control Program (Apr. 28, 2017).



                                    OIG • EVL-2017-006 • August 17, 2017                                       7
In a prior evaluation, we catalogued the difficulties and delays in establishing a quality control
program within DER. 6 Shortly before we issued that evaluation, and more than two years
after SD 2013-01 issued, DER announced it was implementing a program for quality control
reviews. In December 2015, DER informed OIG that the scope of its quality control review
program included quality control reviews of ongoing monitoring activities, regardless of
whether they resulted in findings or conclusions. 7

In our Audit and Evaluation Plans for 2015, 2016, and 2017, OIG identified FHFA’s
supervision of its regulated entities as a significant risk area. We have assessed FHFA’s
supervisory program in numerous audit and evaluation reports, and compliance reviews. We
initiated this evaluation to review DER’s policies and procedures for assigning CAMELSO
ratings to the Enterprises. We also reviewed the extent to which DER conducted independent
quality control reviews of ratings and certain conclusions in its ROEs for the 2015
examination cycle before it issued the ROEs to the Enterprises.


FACTS AND ANALYSIS ...............................................................

DER’s Support for its CAMELSO Ratings of the Enterprises

To determine how DER assigned CAMELSO ratings to the Enterprises, we examined the
documentation that DER examiners prepared to support their proposed CAMELSO ratings for
the 2013, 2014, and 2015 ROEs. We also interviewed DER officials and reviewed applicable
guidance. Under Advisory Bulletin 2012-03, FHFA Examination Rating System, examination
managers are expected to consider certain enumerated factors for each rating component.

We learned that DER examination managers (or their designees) for each Enterprise prepare
draft ROE narratives for each CAMELSO component within their purview. According to
DER, the examination managers then submit those drafts, along with proposed ratings for
the CAMELSO components for which they are responsible, 8 to the EIC. The EIC, in turn,

6
  See OIG, Intermittent Efforts Over Almost Four Years to Develop a Quality Control Review Process
Deprived FHFA of Assurance of the Adequacy and Quality of Enterprise Examinations, at 13 (Sept. 30, 2015)
(EVL-2015-007) (online at www.fhfaoig.gov/Content/Files/EVL-2015-007.pdf).
7
  DER has not defined the term “conclusions” in its guidance. DER officials advised us that “it is reasonable
to consider [conclusions] as broader than findings,” and that its use of the term “conclusion” includes both
adverse findings and any determination that a particular condition or practice is adequate or meets supervisory
expectations.
8
  The team of Freddie Mac examiners prepares risk assessments each quarter and its semiannual risk
assessment, prepared for the quarter ending June 30, customarily contains a recommended rating for each
CAMELSO component. Although Freddie Mac examiners usually rely on the third quarter risk assessments in
preparing the draft ROE (since fourth quarter risk assessments are not typically completed until the first quarter
of the following year), those assessments may or may not contain recommended CAMELSO ratings. The


                                    OIG • EVL-2017-006 • August 17, 2017                                             8
compiles these narratives into one draft, revises the draft as needed, and submits the draft
ROE for the DER Deputy Director’s approval. Once approved, the ROE is issued to each
Enterprise.

We sought to determine whether the CAMELSO ratings were supported by examination work
during the 2015 examination cycle by testing whether three 2015 CAMELSO component
ratings—the operational ratings for both Enterprises and the management rating for Fannie
Mae—were traceable to DER’s underlying examination work. 9 To perform our testing, we
requested and reviewed the documents that DER relied on in assigning those three 2015
component ratings and interviewed the DER examination managers responsible for proposing
those ratings. Based on that information, we concluded that these three component ratings
and the corresponding examination language in the 2015 ROEs were traceable to supporting
examination workpapers. 10

Despite the Clear Requirements in Supervision Directive 2013-01, DER’s Quality
Control Review Program Did Not Include Review of CAMELSO Ratings and ROEs

In our prior evaluation, we reported that federal financial institution regulators, including
FHFA, recognize that internal quality control reviews provide an important internal control
for their examination programs. 11 These regulators require that independent internal quality
control reviews be conducted prior to the issuance of examination reports to ensure that
examinations adhere to the regulator’s quality standards, are being performed with
consistency, and include workpapers that support the findings and conclusions of the
examination under review. 12



Fannie Mae examination team prepared only one risk assessment per year: a detailed semiannual risk
assessment as of July or August that customarily contains recommendations regarding CAMELSO component
ratings.
9
 DER issued the ROE for the 2015 supervision cycle to Freddie Mac on March 11, 2016, and to Fannie Mae
on March 23, 2016.
10
   Although the 2015 ratings we reviewed were traceable to supporting workpapers, we observed significant
variability in the types of internal workpapers that Fannie Mae examiners relied on for these proposed ratings.
Those workpapers included mid-year risk assessments, procedures documents, close-out memos, and memos to
file. The Freddie Mac examination team showed less variability; the team relied primarily on quarterly risk
assessments.
11
  See OIG, Intermittent Efforts Over Almost Four Years to Develop a Quality Control Review Process
Deprived FHFA of Assurance of the Adequacy and Quality of Enterprise Examinations, at 13 (Sept. 30, 2015)
(EVL-2015-007) (online at www.fhfaoig.gov/Content/Files/EVL-2015-007.pdf).
12
  DER’s Operating Procedures Bulletin 2014-DER-OPB-01, Guidelines for Preparing Supervisory Products
and Examination Workpapers, also identifies the “quality control memorandum” as a workpaper that
documents the results of an independent quality control review, as referenced in SD 2013-01, and shows the
evidence that final examination findings, conclusions, and ratings underwent a quality control review.



                                   OIG • EVL-2017-006 • August 17, 2017                                           9
We found that, almost four years after FHFA’s Office of Quality Assurance (OQA) issued
its recommendation that DER establish and implement formal quality control reviews for
examinations and more than two years after FHFA issued a supervision directive requiring
such reviews, DER had not established and implemented a comprehensive internal quality
control review program. We recommended that FHFA “[e]nsure that DER’s recently adopted
procedures for quality control reviews meet the requirements of Supervision Directive 2013-
01 and require DER to document in detail the results and findings of each quality control
review in examination workpapers, including any shortcomings found during the quality
control review.” In its written response, FHFA stated that it “agrees with this
recommendation,” and acknowledged that “a process for independent quality control of
examination documentation is important to the supervision of Fannie Mae and Freddie Mac.”

When we initiated this evaluation, FHFA’s SD 2013-01 instructed both DER and DBR to
implement a quality control review program to “assess examination findings, conclusions,
ratings, supporting workpapers, and related documents” and “reports of examination.” 13
During fieldwork for this evaluation, however, we learned that DER’s quality control review
program, implemented in July 2015 in response to SD 2013-01, did not include quality
control reviews of the ROEs or CAMELSO ratings assigned to the Enterprises.
Notwithstanding the plain requirements of SD 2013-01, a DER official reported to us that
DER determined that quality control reviews of the ROEs were not needed because quality
control reviews are conducted of all examination findings and conclusions before they are
incorporated in the annual ROEs.

DER codified its practice in June 2016 with Operating Procedures Bulletin DER-OPB-02,
Quality Control Review. DER-OPB-02 announced that quality control reviews of ROEs
fall outside the scope of DER’s quality control review program because an ROE “does
not communicate conclusions, findings, or closures of MRAs, but reference [sic] prior
communications to the Enterprises. . . .” 14 In addition, a DER official advised us that
DER’s CAMELSO ratings are not subject to a quality control review because the ratings
are contained in the ROEs, and the ROEs are based, in part, on work previously assessed in
quality control reviews.

In July 2016, OQA assessed the rigor of DER’s year-old quality control program. (Quality
control reviews for DER are performed by the Oversight & Supervision Administration
Branch (OSAB), a group within DER.) As part of OQA’s assessment, it examined whether
13
   Specifically, SD 2013-01 directed that the quality control reviews include “the report of examination,
findings memoranda, and conclusion letters.”
14
   The OPB asserts that ROEs “are vetted through a separate process.” According to DER’s Deputy Director,
all DER executives review the draft ROEs, and she explained that DER employees also review the ROEs for
formatting. We found no evidence that either of these reviews constituted quality control reviews as
contemplated by SD 2013-01.



                                    OIG • EVL-2017-006 • August 17, 2017                                    10
DER-OPB-02 fully implemented the instructions in SD 2013-01. In its written report, OQA
found that DER-OPB-02 failed to conform to SD 2013-01 in that it did not require quality
control reviews of ROEs and CAMELSO ratings. OQA recommended that DER align the
requirements of its OPB to those in SD 2013-01.

In response, DER reported that it had agreed with DBR to revise SD 2013-01 “to retain
general QC requirements applicable to both DER and DBR” and “omit detailed references
not applicable in the same way to both divisions.” On April 28, 2017, shortly before the
conclusion of our field work, FHFA rescinded and replaced SD-2013-01 with SD-2017-01,
Quality Control Program. This new supervision directive eliminates two requirements that
DER had not met: pursuant to SD 2017-01, neither ROEs nor CAMELSO ratings are subject
to a quality control review.

In sum, DER provided input to FHFA for SD 2013-01 but did not comply with it for several
years. FHFA agreed with our 2015 recommendation that FHFA “[e]nsure that DER’s
recently adopted procedures for quality control reviews meet the requirements of Supervision
Directive 2013-01. . ..” DER’s Operating Procedures Bulletin DER-OPB-02, which it
adopted in June 2016 (more than three years after FHFA issued SD 2013-01), did not fully
implement the supervision directive’s instructions. When alerted to this inconsistency, DER
proposed to reduce the requirements in SD 2013-01 to mirror its OPB, rather than enhance its
OPB to meet the requirements of SD 2013-01, and FHFA’s other approving stakeholders
concurred.

DER’s Quality Control Program Does Not Review All Examination Conclusions from
Ongoing Monitoring Activities

Had DER subjected all findings and conclusions in its ROEs to a quality control review, its
position—that a quality control review of the ROE itself and the CAMELSO ratings would be
redundant—would be reasonable. However, we found that ROEs issued by DER contained
conclusions from ongoing monitoring activities which were not subject to a quality control
review due to a gap in DER’s quality control review program.

Ongoing monitoring activities have grown to comprise the majority of DER’s supervisory
work. In 2013, FHFA modified its supervisory strategy for the Enterprises, announcing that
“2013 will be a transformative year for DER as it shifts it [sic] supervisory strategy to a more
coordinated and effective ongoing monitoring approach.” According to FHFA’s 2013
supervisory strategy, DER’s “comprehensive and structured ongoing monitoring program”
would result in “fewer [targeted] examinations than in past years.” That modification to
FHFA’s supervisory strategy was repeated in 2014. For 2015, 2016, and 2017, FHFA’s




                              OIG • EVL-2017-006 • August 17, 2017                                 11
supervisory strategies appeared to be more nuanced, stating that “DER will continue to
augment targeted examinations with ongoing monitoring activities.” 15

Since 2014, ongoing monitoring activities have consistently comprised more than half of
DER’s planned supervisory activities.

     •   In 2014, 61.6% (85/138) of DER’s planned supervisory activities and 54.7%
         of estimated examiner staff hours for Fannie Mae were designated as ongoing
         monitoring. For Freddie Mac, 62.8% (61/97) of planned supervisory activities and
         73.7% of estimated examiner staff hours were designated as ongoing monitoring.

     •   In 2015, 92.2% (131/142) of the planned supervisory activities and 83.1% of estimated
         examiner staff hours for Fannie Mae were designated as ongoing monitoring. For
         Freddie Mac, 73.5% (50/68) of DER’s planned supervisory activities and 70.2% of
         estimated examiner staff hours were designated as ongoing monitoring.

     •   In 2016, 79.3% (69/87) of DER’s planned supervisory activities and 76% of estimated
         examiner staff hours for Fannie Mae were designated as ongoing monitoring. For
         Freddie Mac, 69.5% (41/59) of planned supervisory activities and 73.9% of estimated
         examiner staff hours were designated as ongoing monitoring. 16

In December 2015, DER submitted documentation to FHFA’s Acting Chief Operating Officer
representing that DER had implemented the recommendations in our September 2015
evaluation. DER stated, in writing, that “the QC procedures DER has followed since July
2015 meet all requirements of SD 2013-01. . ..” It also stated that the scope of its quality
control program included a review of documentation of ongoing monitoring activities,
regardless of whether they resulted in findings or conclusions.

Six months later, DER issued DER-OPB-02, which excluded from any quality control review
the ROEs and CAMELSO ratings, as well as ongoing monitoring activities that did not result
in either a finding or an MRA remediation letter. DER officials confirmed to us that DER-
OPB-02 only requires a quality control review of examination work that results in written


15
  In prior reports, OIG found a significant decrease in the number and percent of targeted examinations that
DER completed from 2012 through 2015. See OIG, FHFA’s Targeted Examinations of Fannie Mae: Less than
Half of the Targeted Examinations Planned for 2012 through 2015 Were Completed and No Examinations
Planned for 2015 Were Completed Before the Report of Examination Issued, at 3 and 14 (Sept. 30, 2016)
(AUD-2016-006) (online at www.fhfaoig.gov/Content/Files/AUD-2016-006.pdf); and FHFA’s Targeted
Examinations of Freddie Mac: Just Over Half of the Targeted Examinations Planned for 2012 through 2015
Were Completed, at 3 and 14 (Sept. 30, 2016) (AUD-2016-007) (online at
www.fhfaoig.gov/Content/Files/AUD-2016-007.pdf).
16
  DER was unable to provide us with a breakdown of actual examination staff hours devoted to ongoing
monitoring and targeted examination for the 2013 through 2016 examination cycles.



                                  OIG • EVL-2017-006 • August 17, 2017                                         12
findings and conclusions that are communicated to the Enterprises in certain supervisory
correspondence. 17 Where ongoing monitoring activities do not result in either findings or
remediation letters, DER is not required to communicate the conclusions of such ongoing
monitoring activities in writing to the Enterprises. DER officials acknowledged to us that,
absent such supervisory correspondence, DER’s underlying examination work is not subject
to a quality control review.

OQA’s July 2016 report setting forth its assessment of DER’s quality control review program
also found that, contrary to DER’s December 2015 written representation to OIG, the OPB
did not require quality control reviews of ongoing monitoring activities that did not result
in findings. The OQA report explained that, between August and December 2015, OSAB
sought to perform quality control reviews of ongoing monitoring activities that did not result
in findings, but determined that DER’s standards and requirements for those activities were
insufficient and thus discontinued the review. 18 According to an OSAB manager quoted in
this report, DER’s “standards and requirements . . . needed to be strengthened before a quality
control process would be effective for [ongoing monitoring] examinations that did not result
in findings or conclusions.” 19 The OQA report recommended that DER reconcile the
inconsistency between its OPB and its representation to OIG, and DER did so by informing
us that it no longer performs quality control reviews of conclusions from ongoing monitoring
activities unless they result in findings or remediation letters.

After the conclusion of our fieldwork, a DER official reported to us that DER now requires its
examiners to draft summary memoranda documenting the results of all ongoing monitoring



17
  The OPB provides that correspondence expressing either an objection or a non-objection to an MRA
remediation plan is not subject to the quality control process, unless specifically directed by the DER Deputy
Director. In technical comments and follow up communications, DER notified us that OSAB started
performing a quality control review of these letters after October 12, 2016.
In technical comments, DER also stated that it performs a quality control review of correspondence notifying
an Enterprise that it has not satisfactorily addressed an MRA. However, our review of OSAB’s completed
quality control reviews for 2015 and 2016 found no evidence that OSAB reviewed such letters, assuming any
were sent.
18
  Our review of 2015 OSAB documents confirmed that OSAB unsuccessfully attempted to conduct
substantive quality control reviews of ongoing monitoring activities that did not result in findings or
remediation letters.
19
   DER has issued two operating procedures bulletins that discuss the expected workpapers for ongoing
monitoring activities. DER’s Operating Procedures Bulletin 2013-DER-OPB-04, DER Supervisory Activities,
requires examiners to prepare a procedures document and at least one of the following in connection with all
ongoing monitoring activities: a periodic status memorandum, an analysis memorandum, an input to the
business profile, or an input to a periodic risk assessment memorandum. Its Operating Procedures Bulletin
2014-DER-OPB-01, Guidelines for Preparing Supervisory Products and Examination Workpapers, provides
general guidance for preparing examination workpapers for ongoing monitoring and advises that a summary
memorandum “may be used to report the results of . . . [o]ngoing monitoring on a periodic basis.”



                                    OIG • EVL-2017-006 • August 17, 2017                                         13
activities, regardless of whether findings are made, 20 and indicated that DER may, at some
point in the future, update its guidance to reflect that requirement. We asked whether DER
intended to conduct quality control reviews of these summary memoranda. In response, DER
asserted that it does not intend to perform quality control reviews of the summary memoranda
unless the specific ongoing monitoring activity results in formal supervisory correspondence
to the affected Enterprise, such as a supervisory letter or an MRA remediation letter. 21

     DER’s 2015 ROEs Communicated Examination Conclusions that Did Not Undergo
     Quality Control Reviews

The annual ROE issued to the Board of each Enterprise constitutes DER’s “primary work
product that communicates . . . the cumulative results of supervisory activities conducted
during the annual examination cycle.” The ROEs—and the CAMELSO ratings they report—
are based on results from both targeted examinations and ongoing monitoring activities.
ROEs identify those areas in which the Enterprises have and have not met supervisory
expectations.

DER explained to us that quality control reviews of the ROEs and CAMELSO ratings were
unnecessary because all the findings and conclusions on which they were based had been
subject to quality control reviews. As we have shown, DER’s supervision of the Enterprises
since 2014 has relied, in significant part, on ongoing monitoring and DER has acknowledged
that it does not conduct quality control reviews for ongoing monitoring activities that do not
result in findings or remediation letters. We sought to determine whether conclusions from
ongoing monitoring activities were reported in the 2015 ROEs without undergoing a quality
control review. 22 Because DER first implemented its quality control review program in July
2015, our review was limited to conclusions from ongoing monitoring workpapers dated after
July 2015.

We first identified a number of conclusions in the 2015 ROEs that appeared to be derived
from ongoing monitoring activities that did not result in findings or remediation letters. We

20
   On March 7, 2017, DER’s associate director for the Office of Enterprise Supervision Operations sent
an email to DER staff stating that all ongoing monitoring activities must be closed out with a summary
memorandum. The email stated further that all 2016 ongoing monitoring activities should be closed by
March 31, 2017, if practical.
21
  In the same written response, DER asserted that, notwithstanding its current position, this issue remains
under review.
22
   In the course of another review, our Office of Audits identified conclusions from a targeted examination that
did not undergo a quality control review before they were reported in an ROE for the 2015 supervisory cycle
issued to an Enterprise on March 23, 2016. In that instance, DER initiated the targeted examination in
September 2015, but did not complete it or conduct the quality control review prior to issuing the ROE. In this
evaluation, we did not assess whether other conclusions from targeted examinations were also reported in the
ROE without undergoing a quality control review.



                                   OIG • EVL-2017-006 • August 17, 2017                                            14
traced those conclusions back to the available 2015 examination workpapers and determined
that at least 10 ROE conclusions, contained in 7 different workpapers, had been derived from
such ongoing monitoring activities. We found that, as a general matter, those conclusions
reported that the affected Enterprise’s practices or condition was acceptable, or that the
affected Enterprise made significant progress to address a previously identified weakness but
some supervisory concern remained. The Deputy Director of DER has taken the position that
some of these conclusions are instead “statements of an Enterprise initiative” or “statements
of fact.”

We then compared 10 conclusions to the available 2015 OSAB documents and found no
evidence that OSAB had performed a quality control review of any of them. Based on our
comparison, we asked DER to confirm that OSAB did not perform a quality control review
of these 10 conclusions. DER responded that it could not provide such confirmation without
reviewing all documentation from 2015 targeted examinations and ongoing monitoring
activities subject to OSAB’s quality control review. However, DER acknowledged that, due
to the structure of its quality control review program, conclusions from ongoing monitoring
activities that did not result in findings or remediation letters could be reported in ROEs
without undergoing a quality control review. Based on our review of the 2015 workpapers
made available to us, we found that at least 10 conclusions from ongoing monitoring activities
that did not result in findings or remediation letters were reported in the 2015 ROEs without
undergoing a quality control review.




                             OIG • EVL-2017-006 • August 17, 2017                                15
FINDINGS .................................................................................

   1. The three 2015 CAMELSO component ratings and corresponding ROE language
      that we reviewed were traceable to supporting examination workpapers.

We tested three 2015 CAMELSO component ratings—the operational ratings for both
Enterprises and the management rating for Fannie Mae—to determine whether they were
traceable to DER’s underlying examination work. After reviewing the documents that DER
relied on in assigning those three 2015 component ratings and interviewing the DER
examination managers responsible for proposing those ratings, we concluded that those
component ratings and the corresponding examination language in the 2015 ROEs were
traceable to supporting examination workpapers.

   2. A gap exists in the design of DER’s quality control review process, resulting
      in an increased risk that an ROE may inaccurately report that an Enterprise
      is meeting supervisory expectations or making progress in addressing
      weaknesses.

DER informed us that quality control reviews of ROEs are unnecessary because all
examination findings and conclusions undergo a quality control review before they are
incorporated in the annual ROEs. However, this assertion is inaccurate. Our review found
that the 2015 ROEs communicated at least 10 examination conclusions from ongoing
monitoring activities that did not undergo a quality control review. We traced those
conclusions to workpapers from ongoing monitoring activities that did not result in findings
or remediation letters. The lack of a quality control review of these ROE conclusions
demonstrates a gap in the design of DER’s quality control program.

This gap involves certain examination conclusions that are reported in the Enterprises’ ROEs,
but are not subject to a quality control review. These conclusions, derived from ongoing
monitoring activities, generally consist of determinations that the affected Enterprise is
meeting supervisory expectations or has made significant progress in addressing a previously
identified weakness. DER’s decision not to perform a quality control review of these
conclusions has resulted in an increased risk that an ROE will assure an Enterprise’s Board
that management is meeting supervisory expectations or making significant progress, when it
is not.

The magnitude of this risk depends, in part, on the extent to which the ROEs contain
conclusions from ongoing monitoring activities that do not result in findings, such as
conclusions that an Enterprise’s practices are acceptable or that it has made significant
progress in addressing previously identified weaknesses. As the number of those conclusions


                             OIG • EVL-2017-006 • August 17, 2017                               16
in the ROEs increases, so does the risk of inaccurate or unsupported statements due to a gap
in the quality control program.

   3. DER is aware of the gap in its quality control review program, but has not
      taken action to rectify it.

DER is aware of the gap in its quality control program. However, it has not strengthened its
guidance or practices to include quality control reviews of ongoing monitoring activities that
do not result in findings or remediation letters. DER has recently enhanced its documentation
standards to require examiners to prepare a summary memorandum to close out each ongoing
monitoring activity, and a quality control review of this workpaper (if it is timely prepared)
could rectify the gap in the existing program. Nonetheless, DER informed us that it does not
currently intend to require a quality control review of those memoranda, although the matter
remains under consideration.


CONCLUSION ............................................................................

Although DER’s annual ROEs constitute the “primary work product that communicates . . .
the cumulative results of supervisory activities conducted during the annual examination
cycle,” DER does not perform a quality control review of the ROE or the CAMELSO ratings
reported therein. Instead, as a proxy, DER performs a quality control review of examination
findings and conclusions contained in certain supervisory correspondence that it sends to the
Enterprises throughout the year. According to DER, these reviews make it unnecessary to
perform quality control reviews of the ROEs and the CAMELSO ratings because the pertinent
information has been assessed in previous quality control reviews. However, we determined
in this evaluation that at least 10 conclusions from ongoing monitoring activities that did not
result in findings or remediation letters were reported in the 2015 ROEs without undergoing
a quality control review. Thus, a gap exists in the design of DER’s quality control program.
Until this gap is filled, there will be an increased risk that an FHFA ROE may assure an
Enterprise’s Board that management is meeting supervisory expectations, when it is not.

Filling this gap in the quality control program appears neither onerous nor impractical. For
example, DER could conduct quality control reviews of the newly required summary
memoranda that close out ongoing monitoring activities. By enhancing the design of its quality
control review program, DER could better assure itself that conclusions communicated to the
Enterprises in the ROEs are accurate and adequately supported.




                              OIG • EVL-2017-006 • August 17, 2017                                17
RECOMMENDATION .................................................................

We recommend that:

       DER enhance its quality control review program so that examination conclusions
       from ongoing monitoring activities which do not result in findings or remediation
       letters are subject to a quality control review prior to being communicated to the
       Enterprises in ROEs.



We provided FHFA an opportunity to respond to a draft report of this evaluation. FHFA
provided technical comments on the draft report, which we incorporated as appropriate. In its
management response, which is reprinted in its entirety in the Appendix, FHFA agreed with
OIG’s recommendation.




                             OIG • EVL-2017-006 • August 17, 2017                               18
OBJECTIVE, SCOPE AND METHODOLOGY ..................................

We conducted this evaluation to review DER’s processes for assigning CAMELSO ratings to
the Enterprises and documenting the bases for those ratings. Based on information that came
to our attention during this evaluation, we also sought to determine the extent to which DER
conducted independent quality control reviews of the CAMELSO ratings and certain
conclusions in its ROEs for the 2015 examination cycle.

In reviewing DER’s processes for assigning CAMELSO ratings to the Enterprises, we examined
the documentation that DER examiners prepared to support their proposed CAMELSO ratings
for the 2013, 2014, and 2015 ROEs, reviewed applicable guidance, and interviewed DER
officials. We then tested whether three 2015 CAMELSO component ratings—the operational
ratings for both Enterprises and the management rating for Fannie Mae—were traceable to DER’s
underlying examination work. For this testing, we requested and reviewed the documents upon
which DER relied in assigning those three 2015 component ratings. We also interviewed the
DER examination managers responsible for proposing those ratings.

With respect to DER’s quality control process, we sought to determine whether conclusions
from ongoing monitoring activities that did not result in findings or remediation letters were
reported in the 2015 ROEs without being subject to a quality control review. Because DER
implemented its quality control review program in July 2015, we limited our review to
conclusions from ongoing monitoring workpapers dated after July 2015. We first identified
a number of conclusions in the 2015 ROEs that appeared to be derived from ongoing
monitoring activities that did not result in findings or remediation letters. We traced those
conclusions back to the available 2015 examination workpapers and determined that at least
10 ROE conclusions, contained in at least 7 different workpapers, had been derived from such
ongoing monitoring activities. We then compared 10 conclusions to the available 2015
OSAB documents to determine whether OSAB had performed a quality control review of any
of them. After finding no evidence that those 10 conclusions had undergone a quality control
review, we provided the list of 10 conclusions to DER for confirmation that they were not
subject to quality control review.

The field work for this report was completed in two phases between May 2016 and June 2017.

This evaluation was conducted under the authority of the Inspector General Act and in
accordance with the Council of the Inspectors General on Integrity and Efficiency’s Quality
Standards for Inspection and Evaluation (January 2012). These standards require us to plan
and perform an evaluation based upon evidence sufficient to provide a reasonable basis to
support its findings and recommendations. We believe that the findings and recommendation
discussed in this report meet those standards.


                             OIG • EVL-2017-006 • August 17, 2017                                19
APPENDIX: FHFA MANAGEMENT RESPONSE .............................




                   OIG • EVL-2017-006 • August 17, 2017       20
OIG • EVL-2017-006 • August 17, 2017   21
ADDITIONAL INFORMATION AND COPIES .................................


For additional copies of this report:

   •   Call: 202-730-0880

   •   Fax: 202-318-0239

   •   Visit: www.fhfaoig.gov



To report potential fraud, waste, abuse, mismanagement, or any other kind of criminal or
noncriminal misconduct relative to FHFA’s programs or operations:

   •   Call: 1-800-793-7724

   •   Fax: 202-318-0358

   •   Visit: www.fhfaoig.gov/ReportFraud

   •   Write:

                FHFA Office of Inspector General
                Attn: Office of Investigations – Hotline
                400 Seventh Street SW
                Washington, DC 20219




                               OIG • EVL-2017-006 • August 17, 2017                        22