U.S. Department of Housing and Urban Development Office of Inspector General Region IX 611 West Sixth Street, Suite 1160 Los Angeles, California 90017-3101 Voice (213) 894-8016 Fax (213) 894-8115 Issue Date January 25, 2010 Audit Memorandum Number 2010-LA-0803 MEMORANDUM FOR: Deborah Lalancette, Director, Office of Grants Management, PNP FROM: Joan S. Hobbs, Regional Inspector General for Audit, Region IX, 9DGA SUBJECT: Review Results for Hotline Case Number HL-09-0756 Kaibab Band of Paiute Indians, Pipe Spring, AZ INTRODUCTION In response to an April 3, 2009, hotline request from the Kaibab Band of Paiute Indians, Pipe Spring, AZ (Kaibab), the U.S. Department of Housing and Urban Development’s (HUD) Office of Inspector General (OIG) reviewed HUD’s evaluation of Kaibab’s application for an Indian Community Development Block Grant (Indian Block Grant) under HUD’s 2008 notice of funding availability. The review objective was to evaluate the merits of the complainant’s allegation that the HUD Southwest Office of Native American Programs (Southwest Office) treated Kaibab’s application prejudicially without providing for a fair review. For each recommendation without a management decision, please respond and provide status reports in accordance with HUD Handbook 2000.06, REV-3. Please furnish us copies of any correspondence or directives issued because of the audit. SCOPE AND METHODOLOGY Our review was limited to an examination of the Southwest Office’s evaluation of Kaibab’s 2008 Indian Block Grant application, as well as elements of Kaibab’s 2007 grant application and scoring that were relevant to the 2008 application. The complainant also submitted information regarding how the Southwest Office rated grant applications from other tribes for various years. However, this memorandum does not discuss other tribal applications because that information did not have an effect on the outcome of the 2008 decision related to the subject application. To conduct our review, we interviewed the complainant’s representative for the 2008 application and HUD officials in the Office of Native American Programs including the Grants Management Director, the Southwest Office Administrator, and the individuals who evaluated Kaibab’s 2007 and 2008 applications. We reviewed the relevant parts of HUD’s 2008 notice of funding availability (2008 notice); HUD laws and regulations for Indian Block Grants; and Office of Native American Programs guidance for the evaluation, rating, and ranking of Indian Block Grant applications. We also reviewed documentation submitted by Kaibab for its 2007 and 2008 Indian Block Grant applications, along with the evaluations prepared by HUD review officials. Finally, we reviewed letters from the complainant and its representative that detailed the allegations and provided arguments regarding why it believed the support in the application warranted passing the threshold criteria. Our review of Kaibab’s grant applications was limited to an evaluation of whether the Southwest Office’s review of the applications complied with applicable requirements and guidance and provided a fair review. To accomplish this objective, we considered the following questions: Were the review processes consistent with Indian Block Grant program requirements? Was the reviewer’s determination that the proposal failed to meet certain threshold requirements reasonably supported? Was the reviewer’s overall conclusion consistent with the applicable threshold criteria for economic development projects? We limited our review to work necessary to address the complaint's central issue. Accordingly, we did not conduct the review in accordance with generally accepted government auditing standards but obtained sufficient evidence to provide a reasonable basis for our conclusions. BACKGROUND The initial complaint regarding the Kaibab 2008 Indian Block Grant application was made to OIG by the consultant/developer that represented Kaibab for the 2008 grant application process and assisted with the preparation of the grant application. The OIG hotline accepted the complaint after the Kaibab tribe followed up with its own signed complaint letter. Kaibab, a small Native American tribe located in Northwest Arizona, submitted an application to the Southwest Office for an Indian Block Grant economic development grant of $605,000 for competition under HUD’s 2008 notice. The project was described as an RV park campground to be located on tribal land donated by Kaibab. Southwest Office grants management reviewer(s) rated the application acceptable under the initial application screening criteria and passed it under the general threshold requirements of 24 CFR (Code of Federal Regulations) 1003.301(a). In accordance with the 2008 notice requirements, reviewers then evaluated whether the 2 application met the project-specific threshold requirements for economic development projects.1 The reviewers determined that the Kaibab 2008 Indian Block Grant application did not meet the following two of the five project-specific threshold requirements: Criterion (1): In accordance with 24 CFR 1003.302, for economic development assistance, the application must include a financial analysis. The financial analysis must demonstrate that the project is financially feasible and the project has a reasonable chance of success. The analysis must also demonstrate the public benefit resulting from the ICDBG [Indian Block Grant] assistance. The more funds the application requests, the greater the public benefit that must be demonstrated. Criterion (4): Not more than a reasonable rate of return on investment is provided to the owner. In accordance with the application review procedures, because it did not meet the threshold screening requirements, the Kaibab 2008 Indian Block Grant application was ineligible to be further considered under the rating and ranking review process. The Southwest Office proceeded to issue a letter, dated November 24, 2008, notifying Kaibab that its 2008 application did not meet threshold requirements and would not receive further funding consideration. According to a letter, dated January 16, 2009, from HUD’s Deputy Assistant Secretary for Native American Programs to the chairperson of the Kaibab Band of Paiute Indians, HUD denied a request for reconsideration of the funding determination. HUD’s basis for the denial was section V(B)(10) of the 2008 notice, which stated that judgments made within the provisions of the notice and program regulations are not subject to claims of error. Kaibab had also submitted an Indian Block Grant application in 2007 for basically the same project, which Southwest Office officials also declined to rate or rank based upon failure to meet two project-specific threshold criteria. The complainants updated the 2007 proposal and felt that the 2008 application adequately addressed the reasons why the 2007 application had failed. Therefore, the complainant believed that the 2008 application should have passed the project- specific threshold criteria and at least have been rated and ranked. RESULTS OF REVIEW Summary Our review did not find evidence that the Southwest Office treated Kaibab’s 2008 Indian Block Grant economic development grant application with prejudice or failed to provide a fair review. The grant application review procedures—as related to project-specific threshold screening—were performed in compliance with program requirements. However, we noted that the review processes were not standardized in a way that easily precluded 1 Fiscal Year 2008 SuperNOFA [notice of funding availability] for HUD’s Discretionary Programs; Community Development Block Grant Program for Indian Tribes and Alaska Native Villages; Section III.(C)( 3)( c) – Economic Development Project Thresholds 3 any perception of unfairness. Specifically, the Southwest Office assigned applications to grants management specialists without regard to ensuring impartial treatment in appearance and in fact. Although economic development proposals tended to be uncommon and more complicated than other Indian Block Grant proposals, application reviewers did not receive training or guidance regarding the evaluation of such proposals. Finally, the Southwest Office did not require reviewers to clearly and thoroughly document the reasons for their determination that an application failed to meet threshold requirements and, therefore, would not be rated or ranked. We determined that the Southwest Office reviewers’ determinations regarding specific threshold criteria were generally defensible and made in good faith. We noted that, according to the 2008 notice, if an application failed even one threshold criterion, it could not proceed to rating and ranking. Because we noted that the reviewers’ written comments were unclear, we also relied on interviews and consulted program criteria to evaluate the reasonableness of the reviewers’ conclusions for Kaibab’s 2007 and 2008 applications. We agreed that some of the concerns raised by the reviewers regarding criteria for financial feasibility, reasonable chance of success, and public benefit were defensible. We also identified significant weaknesses in the proposals that were not directly addressed by the reviewers. We concluded that the Southwest Office’s overall decision that Kaibab’s application did not meet all threshold requirements in 2008 (and in 2007) was within the parameters of applicable program criteria. We noted that the Indian Block Grant project-specific threshold criteria and related guidance were nonspecific regarding key requirements for economic development proposals. As a result, an economic development proposal had to be persuasive in all respects to pass threshold screening. The Kaibab application did not include an independent evaluation of the feasibility study, which is listed in the 2008 notice as an appropriate document to include. As a result of this omission, we concluded that the Kaibab application was less persuasive to reviewers, who were not expected to be underwriters or industry experts. In addition, program guidance was unclear regarding the depth of review required under the threshold determination for financial feasibility because the 2008 notice also required reviewers to consider financial feasibility under one of the rating and ranking factors. Accordingly, reviewers had substantial discretion regarding their approach to evaluating financial feasibility for the threshold screening. Finally, economic development proposals represented less than 7 percent (in 2008) of the Indian Block Grant applications to the Southwest Office, and the projects were generally perceived to entail greater risk of failure to succeed in the long run. As a result, all economic development proposals—not just the Kaibab proposal—faced similar barriers to approval and funding. We recommend that the Office of Native American Programs Director of Grants Management require the Southwest Office Administrator to establish a consistent process for assignment of grant applications to reviewers, develop standards to ensure that written review comments are 4 clear and complete, and develop a consistent evaluation approach for certain nonspecific project eligibility criteria. Discussion The Southwest Office Grants Management’s Indian Block Grant Application Review Processes Had Weaknesses Although the procedures for screening, rating, and ranking applications complied with program requirements and were well documented, the Southwest Office practices were not standardized regarding (1) assignment and reassignment of applications to grants management specialists, (2) application of certain nonspecific threshold criteria, and (3) documentation of review evaluations/comments. As a result, the application review process was not transparent to outsiders, and allegations of unequal treatment were not easily dispelled. Assignment of Reviewers Was Not Consistent In 2007 and 2008, the Southwest Office generally assigned grants management specialists to review applications from the tribes for which they regularly tracked grants. According to Southwest Office officials, the grants management supervisor reassigned applications to other reviewers in some cases when the designated specialist was unable to complete the review within required timeframes due to workload, absence, or other reasons. There was apparently no protocol for application reassignments. Likewise, although each application also received a secondary review (usually by a ―team‖ member), there was no protocol for designating who would perform the secondary review. In 2007 and 2008, the grants management supervisor reassigned the Kaibab application before the Kaibab specialist completed the review. However, we found no indication that this reassignment improperly influenced the review outcome. According to officials, other offices under the Office of Native American Programs used a panel approach for rating and ranking applications. However, the Grants Management Director of the Office of Native American Programs stated that the panel approach was preferable but not required. We noted that use of a panel or some pre-established rotation of application assignments to reviewers would preclude the impression that a particular application was directed to a certain reviewer to facilitate favorable or unfavorable treatment. Guidance for Evaluation of Nonspecific Criteria Was Lacking The Southwest Office had not provided training or other guidance to specialists that would address the unique features of economic development applications (which comprised less than 4 percent of the Indian Block Grant applications to the Southwest Office). Because the project-specific threshold criteria provided little guidance regarding how to assess financial feasibility or chance of success, each reviewer applied his or her own level of knowledge and experience when evaluating these criteria for economic development projects. As a result, different reviewers could reasonably apply very different benchmarks to the financial projections in the applications and, thus, could provide different feedback to applicants with similar proposals. In the case of the 2007 5 and 2008 Kaibab applications that we reviewed, the reviewers had very different levels of training and experience on these issues, yet they arrived at similar conclusions. Written Review Comments Were Not Clear The reviewers’ written comments related to the Kaibab application in 2008 were generally unclear and unconvincing. For example, the comments consisted largely of quotes from various parts of the applications; however, it was not always clear whether the quote was meant to demonstrate that the application met the requirement or fell short. Also, some quotations from the application seemed to have been taken out of context without an explanation of why other material was deemed less relevant. The comments for the 2007 application were also primarily quotations, but the reviewer’s conclusions were in italics. Nevertheless, the distinction between quotations and comments was unclear until the 2007 reviewer explained this during our interview. Because the written comments constituted the primary documentation to support a negative threshold decision that disqualified the application from further review, we believe that the rationale should have been carefully documented, even if it required a longer explanation. Further, if it is the Southwest Office’s or Office of Native American Programs’ policy to provide the comments to the applicant, we question why the comments were written in an abbreviated manner that made it difficult for an outsider to interpret. Reviewers’ Overall Conclusions Were Defensible Based Upon Information in the Applications The reviewers of record generally based their conclusions on information contained in the applications for both the 2007 and 2008 Kaibab grant applications. Further, given the non- specificity of some guidance (discussed in the next section of this memorandum), we concluded that each reviewer arrived at a defensible conclusion in good faith. It should be noted that the grant application must pass every element of the five project-specific threshold criteria for economic development; therefore, even one deficiency will cause the application to fail the overall threshold review. Further, the first project-specific threshold criterion for economic development entailed five separate tests. We acknowledge that the written review comments were unclear and did not completely support all of the reviewers’ conclusions. However, we held discussions with the reviewers and evaluated relevant portions of the grant applications to clarify the basis on which each application did not meet particular threshold criteria. Criterion (1) In both years, reviewers concluded that the applications did not adequately demonstrate that the project was financially feasible or had a reasonable chance of success—two of the five tests under the first criterion. As discussed in the next section, the program guidance for determining financial feasibility was general. As a result, much was left to the discretion of the reviewers who had to rely solely on the information provided in the application. Both reviewers stated that the applications failed to convince them of the project’s financial feasibility and reasonable chance of success. We agreed that some of the reviewers’ concerns were defensible and questioned whether applications complied with the Office of Native American Programs’ guidelines regarding the public benefit criterion. 6 The 2008 reviewer explained that the primary concerns were that the application showed negative projected net income in the first 5 years2 and did not demonstrate how the project would provide management with enough experience to successfully operate the proposed business. In addition, she questioned the marketing plan and thought that the application had insufficient industry data to make up for the absence of historical financial data. We determined that the reviewer’s concerns were defensible regarding the proposal’s (1) negative net income for 4 to 5 years and (2) lack of a specific plan to ensure management expertise. The reviewer considered these to be significant indicators of the likelihood of success, based upon her training and experience. The 2007 reviewer noted that the application contained conflicting opinions regarding the financial feasibility of the project and questioned the long payback period and the marketing plan. In 2007, the reviewer also concluded in the written comments that the application did not demonstrate an adequate public benefit—another test under the first criterion. Our review of the application in this regard also concluded that this was a clear weakness in the 2007 proposal. We also noted that, because the application had to pass all of the threshold criteria, the 2007 reviewer may have curtailed further discussion of the proposal’s financial feasibility once it was determined that the proposal did not meet the public benefit requirements. The 2007 application’s deficiencies related to public benefit were addressed to some extent in the 2008 proposal, and the 2008 reviewer did not note any concerns regarding public benefit. However, we considered the limited support for demonstrable public benefit to be a deficiency in the application both years. We reviewed Office of Native American Programs guidance that stated ―when considering the sufficiency of the proposed public benefit [for economic development project threshold], the reviewer should consider the common indicator used in the CDBG [Community Development Block Grant] program, that no more than $35,000 should be spent for each job created.‖ 3,4 In 2007, the application proposed two jobs ($605,000/2 = $302,500 per job), and in 2008, the application proposed five jobs ($605,000/5 = $121,000 per job). We questioned why Southwest Office reviewers did not discuss (either in written comments or during interviews) how this indicator did or did not apply to the Kaibab proposal each year. 2 The 2008 written comments stated, ―Year 15 reflects only a net profit of $10,989.‖ We examined the referenced page in the application and determined that the statement was correct; however, further scrutiny of the financial projections revealed that the correct net profit for year 15 should have been $53,964. It is possible that the error would have been found by the Southwest Office if the more in-depth review of the financials during the ranking and rating process had been performed. However, the review in question was performed at threshold, and the reviewer relied on the data without recomputation. 3 Business Process Guide, chapter 5 4 The CDBG program is the Community Development Block Grant program under the HUD Office of Community Planning and Development. 7 Criterion (4) In both years, the reviewers concluded that the proposals did not meet the criterion that the activity should not provide more than a reasonable rate of return on investment. For both applications, reviewer comments did not specifically state that the rate of return on investment appeared too high. Comments from both reviewers indicated confusion or imprecision regarding the intent of criterion (4). In both 2007 and 2008, the reviewer’s comments discussed profitability, payback period, and ―not more than a reasonable ROI [return on investment]‖ as if these terms were interchangeable. Further, the complainant’s representative stated several times that Southwest Office officials would not say whether the return on investment was too high or too low. To be fair, both reviewers’ written comments regarding criterion (4) focused on the apparent insufficiency of the return on investment data provided in the applications. However, given the reviewers’ apparent lack of understanding/clarity about the intent of criterion (4), if the applications had failed on this criterion alone, it could be appropriate to consider whether they were improperly failed at threshold. However, in both cases, the applications also failed under the first criterion. The 2008 reviewer stated that the return on investment criterion was meant to screen out projects for which the developer reaped large profits from a public investment. The reviewer for the 2008 application had also reversed her initial conclusion regarding return on investment after a briefing with the applicant. Apparently the reviewer had failed the application on this point because the projected rate of return was so low in the first few years. However, this response seemed more applicable to the first criterion regarding financial feasibility. Regarding the return on investment criterion, the 2007 reviewer stated that the payback period of 182 years was not reasonable. This response raised the question of whether the reviewer thought that a long payback period, which might indicate that the project was not financially feasible, also meant that the return on investment was too high. We noted that, because a long payback period would more likely be accompanied by a low return on investment, the statement did not support the criterion that the rate of return was not more than reasonable. Indian Block Grant Economic Development Criteria and Guidance Were General The project-specific threshold criteria for Indian Block Grant economic development projects provided no measures or standards and were not supplemented by more specific guidance. The threshold criteria language was almost identical among the Office of Native American Programs review forms, the applicable notice, the regulations, and the United States Code (42 U.S.C. 5305). Office of Native American Programs officials stated that there was no guidance available to supplement this language, which stated, ―The financial analysis must demonstrate that the project is financially feasible and the project has a reasonable chance of success.‖ Our own research identified minimal guidance in HUD training materials. Again, the specific guidance primarily repeated the general language in the notice. 8 The Application Must Be Persuasive Because the criteria were imprecise, each economic development application had to supply the HUD reviewer with persuasive information showing that the proposed project would succeed. Accordingly, an application must demonstrate that the project will be financially feasible by furnishing its own financial estimates and industry standards for comparison. Because Southwest Office officials received no supplemental training or guidance regarding how to evaluate a project’s financial feasibility or whether it had a reasonable chance of success, they stated that the information in the application had to be ―convincing.‖ The complainant noted that the Kaibab application contained a detailed feasibility study that included the financial statements and projections. However, we noted that the application did not contain any evaluations or opinions of the feasibility study by an independent underwriter or other expert on financial projections for economic development projects (the 2008 application contained an independent cost certification). The feasibility study was prepared by the tribal consultant/developer that submitted the application and was also a complainant. According to information in the application, this individual appeared to be well qualified. Nevertheless, the 2008 notice stated that the application should appropriately include a ―Financial analysis and feasibility study no more than two years old which indicates how the proposed business will capture a fair share of the market, and which has been conducted by an independent entity‖ (rating factor 3: 5.C.(1)(d)). In this instance of an economic development proposal for a very specialized industry, we believe that the absence of an independent review of the feasibility study was a significant omission from the application. Without an outside expert’s opinion to rely on, the HUD reviewers—who were not expected to be underwriters or industry experts— appropriately relied on their own judgments and experience to assess the adequacy of items such as projected revenues and resulting net incomes. The complainant has requested an independent outside review of the application, yet we noted that the original application could have included a review or certification of the feasibility study and financial projections by an independent professional. The 2008 Notice Rating Factors Also Addressed Financial Feasibility According to the 2008 notice, the financial feasibility of an economic development proposal was to be evaluated under both the threshold screening phase and the rating and ranking phase. However, there was no guidance regarding how in-depth the threshold screening for financial feasibility should be, as compared to the rating and ranking review for the same item. For example, the 2008 notice’s rating and ranking requirements covered five rating factors including one that addressed the ―soundness of the approach‖ (number 3). Under this, subfactor 5, Commitment to Sustain Activities, also required analysis of the financial viability for an economic development proposal. The requirements outlined somewhat specific criteria, including the statement: ―(d) Does the business plan or cash flow analysis indicate that cash flow will be positive within the first year?‖ Office of Native American Programs guidance for rating this notice requirement further suggested that if the business plan or cash flow analysis indicated that the cash flow will be positive in 2 to 3 years, the application could still be awarded points under this subfactor. Neither reviewer referred to this criterion for their threshold evaluation. 9 Economic Development Projects Are Viewed as Having More Risk Than Other Indian Block Grant Activities Economic development projects are generally considered by SWONAP staff to be more high-risk ventures when compared to other Indian Block Grant project activities. We could not find any language in the laws or regulations indicating that a higher-risk level should be afforded to the economic development projects. The 2008 notice appeared to address the risk by requiring financial feasibility and other related tests for these projects. In contrast, the affordable housing projects had no similar ―financial feasibility‖ threshold tests. As a result, it was more difficult for economic development proposals to persuade review officials that the project would be as viable over time as the more traditional housing projects. Also, a small percentage of competitive Indian Block Grant applications are for economic development projects (4 percent of 2007 and 2008 applications combined). Because Southwest Office staff had limited experience with the track record of such projects, it had not developed its own formal or informal indicators or standards to predict the success of economic development proposals. We concluded that all economic development proposals would have faced similar barriers to approval and funding. RECOMMENDATIONS We recommend that the Office of Native American Programs Director of Grants Management require the Southwest Office Administrator to 1A. Develop and implement an application review assignment process to ensure that all applications receive equal consideration for threshold screening in appearance and in fact. 1B. Develop protocols for written review comments that require reviewers to clearly state the basis of their conclusions. The comments should be understandable to an outsider or grantee. 1C. Provide training and develop internal guidelines for evaluation of economic development grants that will improve consistency in the evaluation approach for key nonspecific eligibility criteria. AUDITEE’S RESPONSE We provided a discussion draft memorandum report to the auditee on November 17, 2009, and held an exit conference with its staff on December 2, 2009. The auditee provided written comments on January 4, 2010. It generally agreed with our results, but had some disagreements with two of the recommendations. The complete text of the auditee’s response, along with our evaluation of that response, can be found in appendix A of this report. 10 Appendix A AUDITEE COMMENTS Comment 1 Comment 2 11 Comment 3 Comment 4 Comment 5 Comment 6 12 Comment 7 Comment 8 13 OIG Evaluation of Auditee Comments Ref to OIG Evaluation Auditee Comments Comment 1 We agree that Rating Factor 3, subfactor 5(c) of the 2008 Notice states that an independent financial analysis and feasibility study is an appropriate document to include, but that the project threshold provision does not specify the financial analysis must be independent. However, the SWONAP reviewer—who was the grants management supervisor at the time—clearly deemed the lack of independence on the part of the individual who prepared the proposal’s feasibility study to be a major deficiency for the threshold determination. If SWONAP believes this criterion was improperly applied at threshold, then the reviewer’s argument for questioning the feasibility study’s conclusion is significantly weakened. The reviewer was very clear that she relied on the proposal to be convincing, and that the feasibility study’s lack of independence diminished its credibility to her. The supervisor’s written comments demonstrated this by noting that the feasibility study was prepared by an entity that was ―qualified but not independent since also the application grant writer.‖ Comment 2 As the auditee response states, the report recognized that reviewers must rely on the arguments presented in the application to determine the likelihood of project success. During interviews, both reviewers were adamant that the information in the application had to convince them. Although, as acknowledged under comment 1, an independent feasibility study was not required by the project threshold determination criteria, it is reasonable to conclude that a feasibility study prepared by a qualified independent entity would be more convincing than one prepared by an individual representing the applicant. Accordingly, we stand by our conclusion that the lack of independence on the part of the feasibility study author, in the absence of an independent review or certification, significantly impaired the credibility of the feasibility study conclusions presented in the proposal. As a result, in both application years, reviewers disregarded the study’s conclusion (that the project was financially feasible), and relied on their own judgment/analyses regarding whether the feasibility study data supported a conclusion that the project was financially feasible and likely to succeed. Comment 3 We revised the report to state that panels are used for rating and ranking applications. We cited this practice as an example of how ONAP already has some operating procedures in place that could help to reduce the effect of any individual reviewer bias (in appearance or in fact) and facilitate equal consideration of applications. Furthermore, the Indian Block Grant regulations do not preclude the use of panels for the threshold determinations. The report also suggested that other practices such as pre-established rotation of reviewer assignments would also help to reduce any impression of unequal treatment from different reviewers. 14 Comment 4 We agree the proper comparison should be to the other Indian Block Grant (ICDBG) projects, and revised the report accordingly. Comment 5 The auditee’s response does not address the recommendation that the review assignment process should ensure that all applications receive equal consideration for threshold screening in appearance and in fact. The report did not find that SWONAP’s processing of the 2008 Indian Block Grant application was inconsistent with ONAP requirements. Rather, the report stated that because the process to assign applications to [different] reviewers was not standardized, the application review process was not transparent to outsiders, and allegations of unequal treatment were not easily dispelled. If SWONAP continues to arbitrarily assign (and change assignments for) applications to reviewers with varying qualifications and experience, it will perpetuate the appearance of unequal consideration among applicants. As a result, applicants who are denied funds will submit further complaints and FOIA requests. A more transparent review process would likely reduce such complaints. Comment 6 We acknowledge the auditee’s commitment to improve written documentation of reviewers’ decisions. We changed the recommendation wording to state ―protocols‖ in place of ―standards‖ to avoid the use of a term already employed in formal ONAP guidance. However, during our review, SWONAP reviewers could not identify any writing protocols or guidance that they used. Further, the only specific writing protocol we could find in the ONAP business process (BP) guide pertained to the rating and ranking process: It is important for GM Specialists to clearly document their decision- making process when rating applications. It is not enough to simply indicate a page number on the rating sheet. Instead, GM Specialists must provide some level of factual discussion on the rating sheet so that a clear path can be shown between the rating of the information in the application and the ultimate decision that is made. GM Specialists should be aware that applicants could request to see the rating sheet during the debriefing process. Our recommendation pertains to documentation of the reasons an application was failed at the project threshold stage. We are recommending that SWONAP require its reviewers to clearly state the basis of their conclusions in a manner that is understandable to an outsider or grantee. Comment 7 The auditee’s response does not address the recommendation that SWONAP develop internal guidelines that will improve consistency in the evaluation approach for certain criteria. The report noted that the threshold criteria language for economic development activities was almost identical among the Office of Native American Programs review forms, the applicable notice, the regulations, and the United States Code (42 U.S.C. 5305). The detailed discussion and review checklist in the ONAP business process protocols that the auditee refers to is 15 nearly identical to the language in the applicable law and regulations. As a result, the reviewers must apply their own judgment regarding benchmarks used by the application to conclude the feasibility study demonstrated the proposal’s financial feasibility and reasonable chance of success. We further note that, according to the 2008 NOFA, ―Applicants within an Area ONAP’s geographic jurisdiction compete only against each other for that Area ONAP’s allocation of funds‖. Therefore SWONAP could appropriately augment ONAP business protocols with its own internal guidelines to address significant inconsistencies in evaluation approaches taken by its reviewers. Comment 8 We acknowledge the auditee’s commitment to staff training regarding the ICDBG NOFA requirements. However, the auditee’s response does not address specific training for evaluation of economic development grants that will improve consistency among SWONAP reviewers. As we noted under comment 7, SWONAP’s funding allocation is dedicated to a defined group of potential applicants. Therefore SWONAP could appropriately provide its staff with additional training to promote consistency in the application of ONAP business protocols regarding the nonspecific criteria for economic development activities. 16
Review Results for Hotline Case Number HL-09-0756 Kaibab Band of Paiute Indians, Pipe Spring, AZ
Published by the Department of Housing and Urban Development, Office of Inspector General on 2010-01-25.
Below is a raw (and likely hideous) rendition of the original report. (PDF)