Issue Date September 4, 2008 Audit Report Number 2008-LA-0003 TO: Brian D. Montgomery, Assistant Secretary for Housing-Federal Housing Commissioner, H FROM: Joan S. Hobbs, Regional Inspector General for Audit, 9DGA SUBJECT: Implementation Weaknesses Existed in All Major Phases of the FHA Appraiser Review Process HIGHLIGHTS What We Audited and Why We audited the U.S. Department of Housing and Urban Development‘s (HUD) appraiser review process as part of our annual plan. The audit was proposed as the result of a single-family loan origination audit that raised concerns regarding HUD‘s oversight of Federal Housing Administration (FHA) appraisers. This is the second of two audits covering HUD‘s controls over the appraiser review process. The first audit report1 focused on HUD‘s oversight of the FHA appraiser roster (see Followup on Prior Audits). This audit report focuses on the appraiser review procedures conducted by the homeownership centers and HUD‘s oversight of the appraiser review process. 1 ―HUD Did Not Have Adequate Internal Controls over Its FHA Appraiser Roster,‖ OIG report number 2008-LA- 0002, issued on March 4, 2008. What We Found As implemented, the appraiser review process was not adequate to reliably and consistently identify and remedy deficiencies associated with an appraiser. Additionally, HUD did not maintain information necessary to assess the effectiveness of its review process. More specifically, Homeownership centers did not always follow the risk-based targeting review procedures, An estimated one-third of the desk review reports completed by homeownership centers during the audit period contained obvious errors and/or omissions, The homeownership centers did not enforce the requirement for interior inspections during field reviews, and Homeownership centers did not always follow the sanction guidance pertaining to an appraiser‘s prior appraisal deficiencies. What We Recommend We recommend that the Assistant Secretary for Housing-Federal Housing Commissioner develop and implement adequate oversight and controls over the appraiser review process to address the weaknesses identified in this report and to ensure that headquarters continuously evaluates the efficiency and effectiveness of the process. For each recommendation without a management decision, please respond and provide status reports in accordance with HUD Handbook 2000.06, REV-3. Please furnish us copies of any correspondence or directives issued because of the audit. Auditee’s Response We provided HUD with a copy of the report on July 25, 2008, requesting a response by August 24, 2008. HUD requested an extension to the response date to September 2, 2008. However, HUD did not provide a written response to the report by that date, therefore, there are no comments from HUD included in this report. At the exit conference held on August 7, 2008 HUD generally agreed with the report recommendations. 2 TABLE OF CONTENTS Background and Objectives 4 Results of Audit Finding 1: Implementation Weaknesses Existed in All Major Phases of the FHA 6 Appraiser Review Process Scope and Methodology 19 Internal Controls 21 Followup on Prior Audits 22 Appendixes A. Criteria 24 B. Schedule of Desk Review Errors and/or Omissions by Case Number 29 3 BACKGROUND AND OBJECTIVES The Federal Housing Administration (FHA) provides mortgage insurance on loans made by FHA-approved lenders throughout the United States and its territories. The lenders must use appraisers on the FHA appraiser roster to perform the required appraisals on properties that will serve as security for FHA-insured single-family mortgages. FHA insurance reduces a lender‘s risk because FHA will pay a claim to the lender in the event of a homeowner‘s default. As of April 2008, FHA had $382 billion in mortgage insurance in force on single-family properties. FHA‘s Mutual Mortgage Insurance Fund is intended to operate on self-generated income from mortgage insurance premium proceeds without the need for appropriated funds. The U.S. Department of Housing and Urban Development‘s (HUD) appraiser review process is a critical quality control and monitoring mechanism for appraiser performance that helps HUD ensure the reliability of the appraisals that support FHA financing. Over time, HUD has changed the way in which it selects FHA appraisers and who monitors the appraisers‘ performance. Before 1994, HUD maintained an appraiser fee panel, which assigned appraisers to FHA cases on a rotational basis. By March 1996, HUD required lenders to select FHA appraisers from a roster of appraisers maintained by HUD. HUD‘s Real Estate Assessment Center monitored the roster appraisers‘ performance until fiscal year 2002. Since fiscal year 2002, HUD officials at the homeownership centers2 have monitored roster appraisers using revised procedures now called the appraiser review process. Officials execute and document the process primarily through an online database that is part of FHA Connection. HUD‘s appraiser review process consists of the following seven major phases that repeat quarterly. Appraiser review process phases 1. 2. 3. 4. Targeting Desk Field Scoring review review 5. 6. 7. Sanctioning Appeals Reporting process 2 HUD executes the FHA insurance program through four homeownership centers. Each homeownership center insures the loans and oversees the selling of HUD homes for a specified geographic area. 4 Our audit objectives were to determine whether HUD‘s controls over the appraiser review process were adequate to identify and remedy deficiencies associated with an appraiser and to assess the overall effectiveness of the appraiser review process. 5 RESULTS OF AUDIT Finding 1: Implementation Weaknesses Existed in All Major Phases of the FHA Appraiser Review Process Homeownership centers did not fully and consistently implement headquarters‘ guidance in every major phase of HUD‘s FHA appraiser review process. From the initial procedures used to target appraisers for review to the imposition of sanctions, our review noted weaknesses at each major process phase. For example, homeownership center officials did not always perform the minimum number of required desk reviews for a targeted appraiser. Desk reviews completed by homeownership officials contained errors and/or omissions for about one-third of the appraisals reviewed. Field reviewers failed to perform mandatory interior inspections. These weaknesses occurred because HUD lacked adequate oversight and controls for the appraiser review process. As a result, the risk of failure to notify or appropriately sanction appraisers whose appraisals are determined to be deficient increases—even when the appraisers were targeted for review based upon risk indicators. Consequently, the FHA insurance fund is susceptible to an increased risk associated with improperly valued properties used to secure FHA-insured mortgages. Risk-Based Targeting Review Procedures Were Not Always Followed We could not readily analyze the extent to which the homeownership centers completed reviews for all targeted appraisers. However, we did analyze reviews for appraisers targeted by headquarters and determined that, although homeownership centers reviewed the required number of appraisers, in many instances, officials did not conduct the minimum number of desk and field reviews required each quarter for these appraisers. Appraiser Targeting To select appraisers for review, HUD developed a risk-based approach to identify appraisers associated with certain high-risk indicators such as FHA loans that have large gifts or early defaults. Every quarter, headquarters provided each homeownership center with two lists of appraisers and required them to select 15 appraisers for review. In each quarter, the homeownership centers generally reviewed the required 15 appraisers (see criterion (1) in appendix A). In addition, every quarter, HUD required each homeownership center to select and review an additional 30 appraisers using specific risk indicators. Although the homeownership centers maintained a record of each quarter‘s targeting, this information was not captured in the appraiser review process database system (online database). As a result, we could not readily analyze the extent to which reviews were completed for those appraisers. Likewise, without the homeownership 6 centers‘ targeting data, HUD cannot evaluate the effectiveness of the risk indicators used by the homeownership centers and continue to improve the targeting approach. Desk Reviews The review procedures required the homeownership centers to perform desk reviews on 10 appraisal reports for each appraiser and follow up two to five of these with field reviews unless the requirements were waived because the appraiser was targeted due to a prior sanction or complaint (see criterion (1) in appendix A). We reviewed all of the headquarters targeting lists available online for our audit period, which consisted of quarterly data covering April 1, 2006, to September 30, 2007. During this period, the homeownership centers reviewed one or more FHA appraisals submitted by about 4493 of the targeted appraisers. Because we could not readily identify appraisers who were not subject to the minimum review requirements, we evaluated the homeownership centers‘ compliance by (1) analyzing the database for overall trends and (2) reviewing randomly selected appraisers (see the next section on desk reviews). To evaluate overall trends, we compared the headquarters targeting lists to the actual appraisers and cases reviewed by the homeownership centers and determined that in 193 instances, the homeownership centers did not review the minimum number of appraisals for these targeted appraisers. The table below summarizes, by homeownership center, the 193 instances in which the minimum of 10 appraisals per appraiser selected from the headquarters targeting lists were not reviewed. Homeownership Number of instances in which the required minimum of 10 center appraisals per appraiser was not met Atlanta 49 of 111 appraisers or 44 percent Denver 74 of 112 appraisers or 66 percent Philadelphia 62 of 107 appraisers or 58 percent Santa Ana 8 of 119 appraisers or 7 percent Total instances 193 of 449 appraisers or 43 percent Field Reviews We reviewed headquarters gifts and concessions targeting lists further by comparing them to the actual number of field reviews performed per appraiser and determined that in 109 instances, the homeownership centers did not perform the minimum number of field reviews per targeted appraiser. The table below summarizes, by homeownership center, the 109 instances in which the minimum of two field reviews per appraiser selected from the headquarters targeting lists was not conducted. 3 The analysis excluded targeted appraisers with fewer than 10 FHA appraisals identified by headquarters for review. 7 Homeownership Number of instances in which the required minimum of center two field reviews per appraiser was not met Atlanta 48 of 60 appraisers or 80 percent Denver 25 of 59 appraisers or 42 percent Philadelphia 13 of 58 appraisers or 22 percent Santa Ana 23 of 59 appraisers or 39 percent Total instances 109 of 2364 appraisers or 46 percent To the extent the homeownership centers did not desk or field review the minimum number of appraisals, each targeted appraiser may have received a less comprehensive evaluation than intended by headquarters. A HUD official verified that there was no means to ensure that the homeownership centers met the appraiser and appraisal targeting review requirements each quarter without an in-depth analysis. Officials at one homeownership center informed us that not all of the required field reviews were performed due to resource or other constraints. Since the homeownership centers did not fully implement the targeting review procedures as prescribed, HUD could not accurately assess the effectiveness of its targeting review procedures within the appraiser review process. Desk Reviews Contained Errors and/or Omissions At least 37.5 percent of the homeownership centers‘ desk reviews contained clearly identifiable errors and/or omissions. We randomly sampled 64 of 18,159 desk reviews completed for the two-year audit period and found 24 instances in which the homeownership centers‘ desk review checklists contained one or more obvious errors and/or omissions (see the Scope and Methodology for a description of the sampling plan). The following numbers of deficiencies per desk review were identified5: 12 reviews had one or two errors and/or omissions. 10 reviews had three to five errors and/or omissions. Two reviews had six or more errors and/or omissions. The following are examples of the desk review checklist questions for which we disagreed with the reviewer‘s yes/no answer. An Inspector General state-certified 4 Of the 449 appraisers, 213 were not subject to the field review requirements because headquarters targeted them based on an algorithm. The algorithm relies on three risk-based indicators: (1) early loan default, (2) large claim amounts, and (3) severity of net dollar loss to the FHA insurance fund. 5 We identified the errors and/or omissions by using the review checklists completed by the homeownership centers and FHA appraisal packages (the appraisal package includes the uniform residential appraisal report, attachments, and addendums) used by the homeownership center staff to complete their desk reviews. We then compared our results to the responses on homeownership centers‘ checklists. Although the appraiser review process used the checklist results to identify appraisals that needed a field review evaluation, we did not attempt to agree or disagree with the desk reviewer‘s overall conclusion in that regard. To be conservative, we instead focused on the questions that a reviewer could answer with the most objectivity (we excluded the checklist questions that were subjective in nature; for example, ―Excessive number of adjustments‖). 8 appraiser verified the errors and/or omissions. Appendix B contains the complete list of errors and/or omissions identified for the 24 cases. ―Comparable(s) is/are not within defined neighborhood boundaries and the appraiser does not provide adequate explanation.‖ ―Photos and/or maps are either omitted or are of poor quality or the building sketch is insufficient.‖ ―Appraiser indicates generic descriptions (i.e. ―equal,‖ ―similar,‖ ―same,‖ ―typical‖) as opposed to specific descriptions required for items listed in the sales grid.‖ Based on the 37.5 percent (24 of 64) deficiency rate of desk reviews, an estimated 6,810 desk reviews performed by HUD‘s homeownership centers for the two-year audit period may have had similar deficiencies. Such a high deficiency rate raises questions about the efficiency/effectiveness of the checklist procedures as they were implemented. Ineffective desk reviews may result in the homeownership centers‘ failure to impose needed actions/sanctions. In four of the sampled cases, the desk reviewer failed to identify errors and/or omissions, and no field reviews were performed during the review cycle for these appraisers. (All four appraisers were targeted for risk indicators and were subject to the minimum field review requirement.) According to the guidance for the appraiser review process, the ―focus of the desk reviews is to identify deficiencies in the process and format of the reported data….If the desk reviewer concludes that the appraisal report is inconsistent or unacceptable then a field review is warranted.‖ Additionally, officials confirmed that the purpose of performing desk reviews on 10 appraisals for each appraiser is to identify trends and select the most deficient appraisals for field review. Any sanctions were generally based on cases that involved field reviews, and sanctions were rarely based on desk reviews alone. By failing to properly implement appraiser review procedures for these four cases, the homeownership center(s) may have missed opportunities to notify the appraisers of the problems indicated by the desk reviews and possibly impose needed sanctions. HUD had retargeted two of these appraisers for the same risk indicators in quarters that postdated our audit period. The Field Review Requirement for Interior Inspections Was Not Enforced The homeownership centers did not enforce the requirements for mandatory interior field review inspections. According to HUD‘s guidance, the field review process entailed inspections of the subject property‘s interior and exterior. Generally, the homeownership centers contracted out the field reviews, and according to an appendix in HUD‘s guidance, an interior inspection is a mandatory part of the field review. However, the supplemental guidance seems to contradict the requirement by stating ―[i]t is anticipated 9 that HUD will ask the contractor to conduct interior property reviews of 100% of the subject properties assigned for review.‖ (See criterion 3 in appendix A.) Based on an analysis of all field reviews performed,6 the rate of exterior-only field review inspections for the audit period, by homeownership center, ranged from 30 to 73 percent. Officials at two homeownership centers maintained that, despite the written guidance, interior field review inspections were not required. Homeownership center Rate of exterior- only reviews performed Atlanta 45 percent Denver 35 percent Philadelphia 73 percent Santa Ana 30 percent The homeownership centers‘ quality control reviews of the field review contractors‘ performance also reported noncompliance with the interior inspection requirements. In selected quality control reviews we obtained, HUD noted issues regarding the interior field review inspections. In one instance, the reviewing official stated, ―[o]ne of the most serious issues is claiming to have done an interior review when the appraiser had not. One of the homeowners stated flatly that [nobody] contacted him, or his wife, to request permission to do an interior inspection.‖ In another case, under a different homeownership center, the reviewing official noted that in some cases the contractor charged for the interior field review, when exterior-only reviews had been conducted. Accordingly, HUD‘s quality control results indicated that the exterior field review rates noted above may have underestimated the extent to which field reviews were not performed. If HUD‘s field reviewer or the contractor does not conduct an interior field inspection, issues that may affect the continued marketability of the property will remain unknown to HUD. HUD‘s guidance (see criterion 3 in appendix A) discussed the importance of the interior field review inspections. Specifically, ―[t]he interior review shall include examination for any unusual items or serious oversights by the authorized appraiser of noticeable defects in the property, which could affect the health and safety of the occupants or continued marketability of the property.‖ Appraisal Field Review Reports Were Not Consistently Scored Homeownership center officials did not consistently score the appraisal review reports submitted by the field reviewers. According to guidance for the appraisal review process, after field reviewers electronically submitted their results, a qualified official at the homeownership center compared the original appraisal to the field review report and 6 We analyzed field review cases from the universe of desk reviews performed from October 1, 2005, to September 30, 2007. 10 scored the report based upon specific instructions. Using the appraisals selected in our statistical random sample for analysis of the desk review process, we identified field reviews associated with the same appraisers and compared the scores awarded to the guidance. We performed a limited review and did not fully assess whether the rater scored the overall appraisal review report appropriately because we did not have the FHA case files available to compare the field reviewer comments to the original appraisals. However, we found the following inconsistencies in the scoring, which may indicate more widespread problems. For example, Scoring instructions for one question, regarding whether the comparables were truly comparable to the subject property, prescribed that one point should be assessed for each comparable that was not: truly comparable, representative of subject market, and best as of effective date of appraisal. However, two field review reports had identical responses, yet raters assessed a score of three points to one case and two points to the other. In another appraisal review report, the field reviewer comments were as follows: ―Insufficient adjustment on Sale #1 for detached guest house or apartment; Condition adjustment needed on Sale #3 for complete remodel; Adjustment needed on Sales #1 and #3 for VIEW due to the inferior subject location along a thru-street.‖ The scoring methodology said to add one point for each condition not met for each comparable (1, 2, and 3). However, the rater assessed a total of one point for all of the comments. Inconsistent scoring of the field review reports could lead to inconsistent disciplinary actions/sanctions. According to headquarters guidance, the quantified scoring methodology was developed to ensure uniformity (inter- and intra-homeownership center) in recommending actions taken against appraisers. The guidance states further that ―[t]his uniformity will ultimately lead to a fair and defendable appraisal review process for the appraisers on the FHA roster.‖ To the extent to which raters failed to follow the scoring methodology, they potentially undermined the purpose of HUD‘s quantified approach. Sanctioning Guidance for Prior Deficiencies Was Not Always Followed The homeownership center officials did not always incorporate HUD‘s guidance into their appraiser-sanctioning decisions. HUD‘s guidance states, ―[a]long with scoring individual appraisals, the Rater will also be responsible for monitoring the past performance of the appraiser….The Rater will take into account any recommended or imposed actions within the last 12 months.‖ Specifically, raters should choose the most severe action recommended for the appraiser within the past 12 months.7 Also, when an 7 The four primary actions/sanctions that can be imposed on an appraiser are a notice of deficiency, education, removal, or removal with education. 11 appraiser has received three or more recommendations for notices of deficiency within the past 12 months, the rater should escalate the sanction to education. Likewise, three or more recommendations for education should be escalated to a removal. Our analyses of the two years of data provided by HUD showed many instances in which sanctioning guidance was not followed. For the 2,665 appraisers reviewed within the audit period, 491 (18 percent) had one or more desk or field reviews that resulted in a recommended action/sanction. We analyzed the online database history of recommended and imposed actions/sanctions for the 491 appraisers and found the following instances in which the raters did not follow the sanctioning guidance: For 132 appraisers, the homeownership center officials did not impose the most severe action/sanction recommendation in the past 12 months. For 52 appraisers, the homeownership center officials did not escalate an action/sanction when three or more appraisals had the same recommendation in the past 12 months. Failure to consistently escalate sanctions for repeated appraisal deficiencies reduced the credibility of the appraiser review process. According to HUD‘s guidance on management control, when program participants do not comply with recommended actions, appropriate administrative or programmatic sanctions should be applied. Sanctions are used to establish the credibility for enforcing the monitoring system. Without escalation and enforcement, appraisers have less incentive to avoid a repeat sanction. If appraisers who remained on FHA‘s roster failed to correct their deficient practices, then HUD failed to reduce the risk these appraisers posed to the insurance fund. Appeal Timeframes Were Inconsistent Officials allowed appraisers varying timeframes in which to appeal sanctions and/or complete corrective actions. A complete review of the appeals process was beyond the scope of our review. However, we performed a limited analysis to evaluate compliance with guidance regarding timeframes allotted for completion of corrective actions or appeals. We reviewed all 76 instances within our audit period in which, upon appeal, officials reduced recommended sanctions for education, removal, or removal with education (see Scope and Methodology). We noted the following inconsistencies that occurred among and within the homeownership centers. When the homeownership center officials issued an education sanction to an appraiser, HUD‘s guidance stipulated that the training must occur within 60 days of the date of the sanction letter. However, the length of time allowed for the appraiser to complete his or her education varied from one to four months as follows: 12 o Seven appraisers were given one month, o 13 were given two months, o 17 were given three months, and o Two appraisers were given four months.8 For the 76 cases examined, officials granted the appeal request after the appeal deadline in 21 cases. In all cases, the appraisers sent in their appeal request after the appeal deadline date. The distribution of appeals by homeownership center was as follows: o Atlanta: one o Denver: 19 o Philadelphia: zero o Santa Ana: one In addition, it appeared that officials inconsistently updated the database fields for the estimated deadline date and the action deadline when they granted extensions. As a result, it was not possible to perform a reliable analysis of how often (or for how long) officials granted such extensions. Similar to following guidance regarding the escalation of actions/sanctions for repeated offenses (discussed above), compliance with timeframes allotted for disciplinary actions helps to build credibility into the monitoring process. In some cases, good reasons may exist to extend timeframes. However, tracking such extensions could provide management with an opportunity to identify problems or improve the sanctioning and appeals process. Online Database System Reports Were Limited HUD could not readily obtain and analyze pertinent data required to monitor the homeownership centers‘ implementation of the appraiser review process. HUD officials confirmed our observations that queries were not available in the online databases9 to verify whether the homeownership center officials complied with prescribed review policies, such as reviewing the minimum number of appraisers targeted by headquarters each quarter. To verify that the homeownership centers complied with the review policies, officials would have needed to download the information on a case-by-case or appraiser-by-appraiser basis. Summary information and reports on review process implementation are necessary for management to monitor efficiency and effectiveness and to propose improvements to the process when needed. For example, officials could evaluate the effectiveness of the 8 Data were missing for 3 of the 42 cases reviewed. 9 FHA Connection or Computerized Homes Underwriting Management Systems. 13 targeting process by comparing summary information regarding the reasons appraisers were selected for review with the resulting quarterly actions/sanctions. Field Review Contractor Quality Control Guidance Was Limited HUD‘s quality control guidance pertaining to its field review contractors lacked specific instructions regarding follow-through and coordination with its contracting officers. In response to a U.S. Government Accountability Office recommendation (see Followup on Prior Audits section of the report), HUD issued guidance10 in 2006 for on-site monitoring of field review contractor performance. Generally, for a quarterly review cycle, each homeownership center must review two case file reviews completed by one contract appraiser. The two-page guidance requires summary reports to headquarters ―for informational purposes only‖ and states that the report format, which is a summary of findings, must include (a) the number of contractors reviewed and (b) the result of the review. The guidance does not ensure that any deficiencies will be communicated to the contractor or to the government contract representative (responsible for oversight of the contractor‘s performance). The guidance instead states, ―HUD staff will document any pattern of problems, instances of significant non-compliance, or poor performance, which may serve as the basis for termination of the contractor, when warranted,‖ but does not designate the official(s) responsible for tracking the information. For each homeownership center, we performed limited reviews of two contractor quality control reviews that occurred within our audit period and noted the following: Three of the four homeownership centers did not provide a reason for the quality control contractor reviews (although the guidance required homeownership centers to maintain a record of the data used to target the contractors). One staff reviewer provided a formal contractor report to the homeownership center chief indicating trends in wrong photographs, wrong site and property descriptions, and inaccurate data for comparables in two contractor quarterly reviews. The formal report contained findings and an explanation that the appraisal review panel voted to mitigate to an education sanction instead of a removal. However, we were unable to verify in the online database or in the quality control documents whether this sanction was imposed. One staff reviewer provided a formal memorandum to the homeownership center chief and noted deficiencies relating to data verification on the subject, inadequate comparables, failure to analyze and properly adjust comparables, failure to confirm sales data and verification source, and failure to address 10 On-Site Monitoring and Oversight of Federal Housing Administration (FHA) Appraiser Field Review Contractors. Effective March 1, 2006. 14 downpayment assistance and the impact it had on sales price of subject and comparables. According to the quality control review report, the homeownership center removed the appraiser from the roster and then rescinded the removal after accepting a late appeal from the appraiser. However, we were unable to verify this through the FHA Connection appraiser review function or the quality control documents. The quality control reports and supporting documentation did not show how the homeownership centers reported or otherwise addressed these significant deficiencies. Resolution of problems identified during quality control monitoring is a key requirement in the Standards for Internal Control in the Federal Government.11 Without tracking followup actions/sanctions, the contractor monitoring process cannot be effective and credible. Headquarters’ Oversight Was Inadequate Headquarters‘ oversight of the appraiser review process was inadequate to identify and track noncompliance with process guidance or to evaluate the efficiency and effectiveness of the process, for example, by comparing outcomes (actions and sanctions) to the homeownership centers‘ implementation of targeting and review methodologies over time. The appraiser review process is HUD‘s critical control over the quality of work performed by FHA roster appraisers. According to the Office of Management and Budget, ―[m]anagement is responsible for developing and maintaining effective internal control. Effective internal control provides assurance that significant weaknesses in the design operation of internal control, that could adversely affect the agency‘s ability to meet its objectives, would be prevented in a timely manner….Continuous monitoring and testing should help to identify poorly designed or ineffective controls and should be reported upon periodically. Management is then responsible for redesigning or improving upon those controls.‖ We identified the following significant areas in which HUD failed to provide appropriate oversight of the appraiser review process or establish internal controls to ensure that the process was properly implemented. Headquarters Did Not Evaluate Data or Reports and Provide Feedback to Homeownership Centers Officials at all four homeownership centers informed us that they received no feedback from headquarters regarding their performance or the effectiveness of the appraiser review process. Generally the only reports they submitted to headquarters were the quarterly summaries containing (1) statistics regarding the targeting quotas and the number of sanctions each quarter and (2) basic information on the homeownership centers‘ quality control results on the field review contractor(s). The information needed to gauge the homeownership centers‘ compliance with HUD‘s review process 11 U.S. Government Accountability Office/Accounting and Information Management Division (GAO/AIMD)-00- 21.3.1, page 20. 15 requirements was neither fully reported by the centers nor readily obtainable from the online database, as discussed above. In addition, headquarters officials apparently failed to evaluate the accuracy of summary reports that were available online or otherwise note significant differences in the statistics reported by the homeownership centers. For example, they failed to question why online summary reports showed that, compared to the other homeownership centers, Philadelphia had performed almost twice as many desk reviews. Number of appraisers desk reviewed, by homeownership center, from October 1, 2005, to September 30, 2007 Santa Ana Homeownership center Philadelphia Denver Atlanta 0 200 400 600 800 1000 1200 Number of appraisers reviewed During our review, the Philadelphia homeownership center confirmed that, until recently, their staff had erroneously entered reviews from the postendorsement technical review process as part of the appraiser review process, thus inflating the number of appraisers that were desk reviewed. After this error had occurred for more than a year, Philadelphia homeownership center officials noticed that the number of appraisers reviewed in their summary reports was much higher than the numbers reported by the other homeownership centers. Officials stated that around April 2008, they changed their procedures to exclude postendorsement technical reviews unless the appraisers were later targeted for the appraiser review process.12 Headquarters failed to question the homeownership center regarding the significant discrepancies in its quarterly numbers. Notably, if headquarters had used the inflated summary reports to evaluate the efficiency and effectiveness of the appraiser review process, its conclusions would likely have been invalid.13 12 We did not verify whether the Philadelphia homeownership center had changed its reporting method because the change purportedly occurred in about April of 2008, which was after our audit period. 13 We estimated the number of appraisal reviews improperly included in the database and, where appropriate, excluded these from our data analyses. 16 Written Policies and Procedures Were Out of Date The policies and procedures available to the homeownership center officials were outdated. For instance, HUD Handbook 4150.2, chapter 6, Appraisal and Appraiser Monitoring, still included procedures that referred to Real Estate Assessment Center performance although that organization had not been part of the process since 2002. Additionally, headquarters officials noted that HUD guidance available to the homeownership center officials excluded the most recent changes and related to the old system. According to the director, a workgroup was recently assembled to create one comprehensive appraiser review process guide. Headquarters Did Not Systematically Evaluate the Efficiency and Effectiveness of the Appraiser Review Process Headquarters officials did not provide data to demonstrate that they had developed and evaluated information—such as trend analyses, correlations between targeting methods and sanctions, or data comparisons among homeownership centers—that could have been used to find implementation problems with the appraisal review process or to identify best practices. While it appeared that the online database incorporated many beneficial features, it lacked controls to monitor compliance with process requirements. Specifically, as discussed above, the online database system did not readily provide reports that could have been used to monitor how closely the homeownership centers followed the targeting and review guidelines. Conclusion Our review noted weaknesses at each major phase of the appraiser review process. For each phase, we noted problems such as inadequate or incomplete HUD guidance, weak quality controls over implementation of review procedures, and inconsistent application of rating standards and sanctioning timeframes. HUD‘s Handbook on Valuation Analysis for Home Mortgage Insurance14 states that the [appraisal/appraiser] ―review process is a critical quality control and performance monitoring mechanism for HUD….By performing statistical analysis as well as field reviews, HUD maintains the capabilities to broadly track its portfolio and investigate it in greater depth. The reviews will be used to determine the reliability of the appraisal supporting FHA financing as well as the performance of the appraiser.‖ However, as implemented, the appraiser review process was not adequate to reliably and consistently identify and remedy deficiencies associated with an appraiser. Additionally, HUD did not maintain information necessary to assess the effectiveness of its review process. 14 Handbook 4150.2, Valuation Analysis for Home Mortgage Insurance for Single Family One- to Four- Unit Dwellings, sections 6-0 and 6-3. 17 These weaknesses occurred because headquarters lacked adequate oversight and control of the appraiser review process. As a result, the risk of failure to notify or appropriately sanction appraisers whose FHA appraisals are determined to be deficient is increased. Consequently, the FHA insurance fund is susceptible to an increased risk associated with improperly valued properties used to secure FHA-insured mortgages. Recommendations We recommend that the Assistant Secretary for Housing-Federal Housing Commissioner implement adequate oversight and controls over the appraiser review process by 1A. Developing and implementing a written quality control plan for the appraiser review process that includes procedures to address the implementation weaknesses identified in this report. 1B. Developing and implementing a systematic process to evaluate the efficiency and effectiveness of the appraiser review process on a regular and recurring basis. 1C. Updating and consolidating the policies and procedures for the appraiser review process. 1D. Evaluating the appraiser review process data and reports on a regular and recurring basis. 1E. Providing feedback to the homeownership centers regarding their implementation of the appraiser review process to promote consistency and best practices. 18 SCOPE AND METHODOLOGY We performed the audit work between January 14 and June 17, 2008. Our review generally covered appraiser reviews conducted from October 1, 2005, through September 30, 2007. We focused on HUD‘s implementation of its appraiser review process and whether homeownership center officials could adequately identify appraisal deficiencies and issue appropriate remedies to the associated appraiser. We were unable to assess the effectiveness of the process at the department level because of data constraints. To accomplish our audit objective, we Reviewed relevant HUD regulations, Office of Management and Budget circulars, and other pertinent information; Reviewed HUD‘s policies and procedures associated with the appraiser review process; Interviewed appropriate HUD personnel at headquarters and the homeownership centers to obtain an understanding of the appraisal review procedures and controls; Reviewed data and other information provided by HUD headquarters and the homeownership centers; Reviewed a random sample of desk reviews to evaluate whether desk review checklists were correctly and completely completed; and Reviewed appraiser review process data contained in FHA Connection. Headquarters officials provided a database of FHA case numbers and associated appraiser review process data for all desk reviews completed by the four homeownership centers for the audit period (covering eight quarterly appraiser review cycles starting October 1, 2005, and ending September 30, 2007). This database contained 19,304 desk review records. We later estimated that officials improperly included a number of cases that did not go through the appraiser review process. Based on our sample testing described below, we estimated that a universe of 18,159 desk reviews were applicable to our desk review analysis. To review compliance with risk-based targeting review procedures, we obtained headquarters targeting lists for the six quarters that were available in the online database for our audit period— April 1, 2006, to September 30, 2007. We then analyzed the headquarters database records associated with appraisers who were subject to the minimum desk review and, when applicable, field review requirements. For our analysis of desk reviews, we selected a random attribute sample of 64 desk review cases from the universe of 18,159 in the headquarters database of desk reviews. We reviewed HUD‘s desk review checklists as described in the report and determined that checklists for 24 of the 19 sample cases contained one or more errors and/or omissions regarding objective information required in an FHA appraisal (see appendix B for a schedule of desk review errors and/or omissions by case number). An Inspector General state-certified appraiser verified our conclusions. Based on the 24 desk reviews that contained one or more errors or omissions, we computed an attribute error rate of 37.5 percent. We projected this error rate to the universe and estimated that 6,180 desk review checklists for the two-year period contained similar errors and/or omissions. The projected results have a confidence level of 90 percent and a sampling error rate of 9.91 percent. We completed a full review for 26 of the 64 desk review cases in our sample. We prioritized our reviews on the premise that if a desk review proceeded to field review, there was another opportunity for the appraiser to be notified/sanctioned for the appraisal‘s deficiencies. However, if a desk review contained errors and/or omissions and did not proceed to field review, HUD generally did not have the additional opportunity to sanction the appraiser for those items. Therefore, deficient desk reviews with no field review posed the greatest risk that the appraiser would not be notified of the problems and continue to produce defective FHA appraisals. For the same 26 sample cases, we analyzed field review, reselection, and scoring information for the associated appraiser. We did not project any statistical sample result to the universe of appraisers because, although chosen at random, each appraiser did not have an equal chance of being selected (some appraisers had as many as 28 desk reviews in the universe, and many had only one). To analyze data related to sanctions and appeals, we used information in HUD‘s online database for the period October 1, 2005, to September 30, 2007, because the database of desk reviews provided by headquarters did not contain information needed to conduct the analysis. We performed our review in accordance with generally accepted government auditing standards. 20 INTERNAL CONTROLS Internal control is an integral component of an organization‘s management that provides reasonable assurance that the following objectives are being achieved: Effectiveness and efficiency of operations, Reliability of financial reporting, and Compliance with applicable laws and regulations. Internal controls relate to management‘s plans, methods, and procedures used to meet its mission, goals, and objectives. Internal controls include the processes and procedures for planning, organizing, directing, and controlling program operations. They include the systems for measuring, reporting, and monitoring program performance. Relevant Internal Controls We determined the following internal controls were relevant to our audit objectives: Effectiveness of operations and Compliance with applicable laws and regulations. We assessed the relevant controls identified above. A significant weakness exists if management controls do not provide reasonable assurance that the process for planning, organizing, directing, and controlling program operations will meet the organization‘s objectives. Significant Weaknesses Based on our review, we believe the following item is a significant weakness: HUD‘s lack of quality control over the implementation of its appraiser review process. 21 FOLLOWUP ON PRIOR AUDITS HUD Did Not Have Adequate Internal Controls over Its FHA Appraiser Roster (2008-LA-0002) The U.S. Department of Housing and Urban Development (HUD) Office of Inspector General (OIG) for Audit recently issued the audit report entitled ―HUD Did Not Have Adequate Internal Controls over Its FHA Appraiser Roster,‖ (2008-LA-0002). This report reviewed the FHA appraiser roster to determine whether HUD‘s controls over the roster were adequate to ensure that only qualified/eligible appraisers were placed on the roster and whether the oversight and maintenance of the roster were sufficient to ensure that only currently eligible appraisers remained on the roster. The report noted significant weaknesses in HUD‘s internal controls used to maintain the roster and recommended that HUD implement stronger internal controls to ensure that only eligible appraisers are placed on its roster and that oversight and maintenance of the roster ensure that only eligible appraisers remain on the roster. At this time, HUD plans to address the recommendations, and our current audit excluded any followup review of HUD‘s appraiser roster. Single-Family Housing: HUD’s Risk-Based Oversight of Appraisers Could Be Enhanced (GAO 05-14) In 2005 the U.S. Government Accountability Office (GAO) issued the report entitled ―Single-Family Housing: HUD‘s Risk-Based Oversight of Appraisers Could Be Enhanced,‖ (GAO 05-14). GAO conducted the review as a followup to its 1999 report on the need for improvements in HUD‘s oversight of appraisers, which has historically been a challenge for HUD. To ―reduce the financial risks assumed by HUD and to further enhance its oversight of appraisers participating in HUD‘s Single Family Mortgage Insurance programs,‖ GAO recommended that the Secretary of HUD direct the Assistant Secretary for Housing-Federal Housing Commissioner to Institute reasonable controls on the process of placing appraisers on the appraiser roster to ensure that applicants‘ conformance to eligibility criteria is verified; Consider a requirement to include, when targeting appraisers for review, those appraisers who have recently completed a sanction period to ensure that these appraisers have corrected their relevant deficiencies; Maintain the historical information, particularly early loan default information, used to target appraisers for review to ensure that the homeownership centers target and review appraisers based on the criteria in HUD guidance; and 22 Implement a cost-effective field review contractor oversight process that includes on- site monitoring. OIG‘s evaluation of HUD‘s Corrective Actions Taken in Response to GAO‘s Recommendations HUD had taken steps to strengthen the process of placing appraisers on its roster; however, as noted above, OIG recently completed a review that found significant control weaknesses persisted. OIG‘s evaluation of HUD‘s corrective actions remains in progress, and was not part of our current review. On May 31, 2005, headquarters issued a memorandum to homeownership center officials that contained new procedures for targeting FHA roster appraisers ―designed to improve upon the successful risk-based targeting methodology implemented in FY [fiscal year] 2002 and to respond to GAO audit recommendations.‖ The ―Revised Procedures for Targeting Federal Housing Administrations (FHA) Roster Appraisers for Review‖ were effective July 1, 2005. This revised targeting policy included a provision for homeownership center officials to generate targeting lists based, in part, on previously sanctioned appraisers. Our current review looked at compliance with HUD‘s review requirements for targeted appraisers but did not evaluate the effectiveness of the revised targeting policy. The above-mentioned ―Revised Procedures for Targeting Federal Housing Administrations (FHA) Roster Appraisers for Review‖ require homeownership center officials to maintain permanent records of the data used to target appraisers each quarter, listing of appraisers targeted for review, and criteria under which each appraiser was identified for review. During our audit, we did not find any instances in which the homeownership centers deviated from this policy. Additionally, the online system maintained headquarters‘ targeting lists for the most recent nine quarters. However, the online system did not capture the homeownership centers‘ appraiser and appraisal quarterly target lists. In the ―Revised Procedures for Targeting Federal Housing Administrations (FHA) Roster Appraisers for Review‖ policy, effective July 1, 2005, and similarly in the ―Updated Revised Procedures for On-Site Monitoring and Oversight of Federal Housing Administration (FHA) Appraiser Field Review Contractors‖ policy, effective March 1, 2006, HUD outlines procedures for contractor oversight. In short, officials at each homeownership center will review two FHA contractor field reviews for at least one field review contractor per quarter, subject to the availability of adequate resources. Our audit included a limited review of the homeownership centers‘ compliance with the new procedures. 23 APPENDIXES Appendix A CRITERIA 1. Internal HUD memorandum, ―Revised Procedures for Targeting Federal Housing Administration (FHA) Roster Appraisers for Review,‖ effective July 1, 2005. Targeting Threshold The homeownership centers are to follow the requirements set forth below as funding allows: Select 15 appraisers from the headquarters-generated targeting lists. Select 30 additional appraisers based on risk-based indicators. Review Procedures ―For each appraiser identified for review, the HOC [home ownership center] must complete desk reviews of 10 cases except for appraisers associated with prior sanctions and complaints.‖ ―For each targeted appraiser, the HOC must perform on-site reviews of 2 to 5 cases except for appraisers associated with prior sanctions, complaints, and appraiser algorithm.‖ 2.Internal HUD memorandum, ―Revised Procedures for Targeting Federal Housing Administration (FHA) Roster Appraisers for Review,‖ effective July 1, 2005, and similarly ―Updated Revised Procedures for On-site Monitoring and Oversight of Federal Housing Administration (FHA) Appraiser Field Review Contractors,‖ effective March 1, 2006, set forth the requirements for the contractor oversight of the field reviewers. Targeting Threshold Each homeownership center must review at least one contractor per quarter. Each homeownership center will review a minimum of four contractors per year, subject to the availability of adequate resources to perform the review function as specified. 24 Review Procedures Upon receipt of the completed review documents from the contractor, the Technical Support Branch Chief will select two of the appraisal field review cases performed by one contractor to evaluate the contractor‘s performance. HUD staff will document any pattern or problems, instances of significant noncompliance, or poor performance, which may serve as the basis for termination of the contractor, when warranted. Reporting Requirements (specific to the memorandum, effective March 1, 2006) ―These reports are for informational purposes only. The report format, which is a summary of findings, must include (a) the number of contractors reviewed and (b) the results of the review.‖ 3.Internal Desktop Guide (Appraiser Review Process) III. Review Process Desk Reviews ―The Desk Reviewer analyzes the appraisal package. The focus of the desk review is to identify deficiencies in the process and format of the reported data. Ultimately, it is incumbent upon the Desk Reviewer to carefully analyze the appraisal report for reasonable and logical conclusions of value. The Desk Reviewer will determine if the appraisal data is reported consistent with the requirements set forth by HUD Handbook 4150.2 and USPAP (Uniform Standards of Professional Appraisal Practice). The comprehensive valuation package (CVP) must also be reviewed for completeness and compliance with the cited guidelines. If the desk reviewer concludes that the appraisal report is inconsistent or unacceptable then a field review is warranted.‖ Field Reviews ―The value, process, data reported and ultimate conclusion as to the acceptability of the appraisal must be reported in an unbiased manner. The review process involves inspection of the subject‘s interior and exterior. It is a mandatory requirement per the Statement of Work (SOW) that an interior inspection is conducted. (See SOW in Appendix B, Attachment 2.) This inspection is completed in a way that allows the reporting of any readily observable defective conditions that do not meet the Minimum Property Standards (MPS) or HUD Handbook 4905.l.‖ 25 V. Scoring and Rating Methodology ―To quantify the information in the ARR [appraisal review report], a point value was assigned to each element of each question. A summary of the points/critical elements assigned to each scored question off the ARR form can be found in Appendix C, Attachment 1. The information was quantified to ensure uniformity (inter and intra HOC) in recommending actions taken against appraisers. This uniformity will ultimately lead to a fair and defendable appraisal review process for the appraisers on the FHA roster. This scoring method provides the rater with a systematic way of thoroughly examining the appraisal and the corresponding field review. The results from the scoring model give the Rater a recommended action to pursue.‖ Scoring Several Appraisals for One Appraiser ―Along with scoring individual appraisals, the Rater will also be responsible for monitoring the past performance of the appraiser. Under our current targeting system several appraisals will be targeted for one appraiser. The Rater will take into account any recommended or imposed actions within the last 12 months. The Rater can check in CHUMS [Computerized Homes Underwriting Management Systems] and the access database to determine if actions were imposed against an appraiser. If an appraiser has 3 or more appraisals that scored a recommended ‗NOD‘ [notice of deficiency] in the last 12 months, then an ‗Education‘ action is recommended. If an individual appraiser has 3 or more appraisals that scored a recommended ‗Education‘ in the last 12 months, then a ‗Removal‘ action is recommended. The Rater should choose the most severe action recommended. If 5 field reviews were performed on one appraiser within the past 12 months and 1 scored a recommended ‗Removal‘ and the other 4 scored a recommended ‗NOD,‘ the outcome would be a recommended ‗Removal.‘ If 3 field reviews were performed on one appraiser within the past 12 months and all 3 scored a recommended ‗NOD,‘ then the outcome would be a recommended ‗Education.‘ If 4 field reviews were performed on one appraiser within the last 12 months and 2 scored a recommended ‗NOD‘ and 2 scored a recommended ‗Education,‘ the outcome would be a recommended ‗Education.‘ If 5 field reviews were performed on one appraiser within the last 12 months and 2 scored a recommended ‗NOD‘ and 3 scored a recommended ‗Education,‘ the outcome would be a recommended ‗Removal.‘‖ Appendix B, Attachment 2, Statement of Work, C-1(3) ―As instructed by HUD when the cases are assigned, the contractor shall perform an interior property review. It is anticipated that HUD will ask the contractor to conduct interior property reviews of 100% of the subject properties assigned for review. The interior review shall include examination for any unusual items or serious oversights by the authorized appraiser of noticeable defects in the property, which could affect the health and safety of the occupants or continued marketability of the property.‖ 26 Appendix E, Attachment 1 ―After an analysis of the deficiencies listed above, we have determined that you must undergo professional training to improve your performance in order to remain on the FHA Appraiser Roster....The training must occur after the date of this letter and must be completed within 60 days of the date of this letter.‖ 4.HUD Handbook 1840.1, REV-3, Departmental Management Control Program Handbook, dated February 1999, 7-9E states, ―Sanctions are an essential component of any effective monitoring process. There will be cases where participants fail or refuse to comply with recommended actions. Appropriate administrative or programmatic sanctions should be applied in these cases. Sanctions are used to establish credibility for enforcing the monitoring system.‖ 5.Office of Management and Budget (OMB) Circular A-123, Management‘s Responsibly for Internal Control, effective fiscal year 2006. I. Introduction ―Management is responsible for developing and maintaining effective internal control. Effective internal control provides assurances that significant weaknesses in the design or operation of internal control, that could adversely affect the agency‘s ability to meet its objectives, would be prevented or detected in a timely manner.‖ I. Introduction, A. Agency Implementation ―Internal control guarantees neither the success of agency programs, nor the absence of waste, fraud, and mismanagement, but is a means of managing the risk associated with Federal programs and operations. Managers should define the control environment (e.g., programs, operations, or financial reporting) and then perform risk assessments to identify the most significant areas within that environment in which to place or enhance internal control. The risk assessment is a critical step in the process to determine the extent of controls. Once significant areas have been identified, control activities should be implemented. Continuous monitoring and testing should help to identify poorly designed or ineffective controls and should be reported upon periodically. Management is then responsible for redesigning or improving upon those controls. Management is also responsible for communicating the objectives of internal control and ensuring the organization is committed to sustaining an effective internal control environment.‖ 6.HUD Handbook 4150.2, chapter 6, Appraiser and Appraisal Monitoring, dated June 24, 1999. 6-0 Introduction ―The review process is a critical quality control and performance monitoring mechanism for HUD. FHA will monitor appraisals and appraisers using statistical analysis and field reviews. Through analysis of performance measures, FHA will identify candidates for 27 field reviews. By performing statistical analysis as well as field reviews, HUD maintains the capability to broadly track its portfolio and investigate it in greater depth.‖ 6-3 Appraisal Review Process ―The oversight process includes statistical analysis of appraisals and field reviews. The reviews will be used to determine the reliability of the appraisals supporting FHA financing as well as the performance of the appraiser. To gauge an appraiser‘s performance, REAC [the Real Estate Assessment Center] will review a sample of appraisals performed for FHA over a specified time period and/or specified number of appraisals performed.‖ 28 Appendix B SCHEDULE OF DESK REVIEW ERRORS AND/OR OMISSIONS BY CASE NUMBER We compared HUD‘s desk review checklist reports15 to the supporting appraisal packages and, following the same guidance provided to HUD‘s desk reviewers, identified obvious errors and/or omissions in the following 24 cases. An Inspector General state-certified appraiser verified the results shown below. Obvious errors and/or omissions found in HUD's field review feasibility checklist answers FHA case numbers A B C D E F G H I J K L M N Totals 011-5497620 x x x x x x 6 352-5391904 x x x x x x 6 105-2596255 x x x x 4 197-3441495 x x x x 4 332-4441435 x x x x 4 052-4008623 x x x 3 061-2889866 x x x 3 094-5174144 x x x 3 095-0230735 x x x 3 137-3335740 x x x 3 197-3610011 x x x 3 381-7680288 x x x 3 052-3464429 x x 2 052-3723298 x x 2 121-2260716 x x 2 201-3529869 x x 2 263-3809705 x x 2 351-4726939 x x 2 431-4202474 x x 2 461-3997954 x x 2 495-7449556 x x 2 372-3538299 x 1 332-4360233 x 1 197-3612875 x 1 Total errors and/or omissions 13 11 11 6 4 4 3 3 3 2 2 2 1 1 66 15 Field review feasibility checklist. 29 Legend A. ―Photos and/or maps are either omitted or are of poor quality or the building sketch is insufficient.‖ B. ―Comparable(s) is/are not within defined neighborhood boundaries and the appraiser does not provide adequate explanation.‖ C. ―Appraiser indicates generic descriptions (i.e. ―equal,‖ ―similar,‖ ―same,‖ ―typical‖) as opposed to specific descriptions required for items listed in the sales grid.‖ D. ―Well and/or septic separation distances are not sketched; or there are no comments as to whether public services are available.‖ E. ―The subject‘s building sketch of room counts, gross living area, etc. do not match the description provided on the URAR [uniform residential appraisal report].‖ F. ―Recent prior sales – strong indicator of fraud‖ G. ―Zoning code is missing or incomplete; or zoning description is vague or misleading; or zoning compliance (i.e., status as to conforming or non-conforming) is not properly qualified,‖ or ―Specific zoning classification is missing or zoning description is missing, vague; or zoning compliance is not properly qualified.‖ H. ―Failure to report type of financing and/or sales concessions.‖ I. ―Adjustments are inconsistent, not logical, or not market supported.‖ J. ―Gross adjustments exceed 25% overall, 15% net, and 10% line without explanation,‖ or ―Adjustment ratios exceed 25% gross, 15% net, and/or 10% line without adequate explanation.‖ K. ―Data and/or Verification Sources were not identified,‖ or "Data source(s), offering price(s), and date(s) of prior or current listings are not reported for the subject property currently offered for sale or offered for sale within the last twelve months.‖ L. ―Remaining Economic Life is not reported or adequate explanation is not provided.‖ M. ―FEMA [Federal Emergency Management Agency] Flood information is missing or inaccurate.‖ N. ―After comparing the price range described by the appraiser in the range of property values in the Neighborhood Section to the sales price and the adjusted sales price of the comparables, the subject indicated value appears to be outside the ranges indicated by the appraiser.‖ 30
Implementation Weaknesses Existed in All Major Phases of the FHA Appraiser Review Process
Published by the Department of Housing and Urban Development, Office of Inspector General on 2008-09-04.
Below is a raw (and likely hideous) rendition of the original report. (PDF)