U.S. Department of Housing and Urban Development
Office of Inspector General
Region IX
611 West Sixth Street, Suite 1160
Los Angeles, California 90017-3101
Voice (213) 894-8016
Fax (213) 894-8115
Issue Date
January 25, 2010
Audit Memorandum Number
2010-LA-0803
MEMORANDUM FOR: Deborah Lalancette, Director, Office of Grants Management, PNP
FROM: Joan S. Hobbs, Regional Inspector General for Audit, Region IX,
9DGA
SUBJECT: Review Results for Hotline Case Number HL-09-0756
Kaibab Band of Paiute Indians, Pipe Spring, AZ
INTRODUCTION
In response to an April 3, 2009, hotline request from the Kaibab Band of Paiute Indians, Pipe
Spring, AZ (Kaibab), the U.S. Department of Housing and Urban Development’s (HUD) Office
of Inspector General (OIG) reviewed HUD’s evaluation of Kaibab’s application for an Indian
Community Development Block Grant (Indian Block Grant) under HUD’s 2008 notice of
funding availability. The review objective was to evaluate the merits of the complainant’s
allegation that the HUD Southwest Office of Native American Programs (Southwest Office)
treated Kaibab’s application prejudicially without providing for a fair review.
For each recommendation without a management decision, please respond and provide status
reports in accordance with HUD Handbook 2000.06, REV-3. Please furnish us copies of any
correspondence or directives issued because of the audit.
SCOPE AND METHODOLOGY
Our review was limited to an examination of the Southwest Office’s evaluation of Kaibab’s 2008
Indian Block Grant application, as well as elements of Kaibab’s 2007 grant application and
scoring that were relevant to the 2008 application. The complainant also submitted information
regarding how the Southwest Office rated grant applications from other tribes for various years.
However, this memorandum does not discuss other tribal applications because that information
did not have an effect on the outcome of the 2008 decision related to the subject application.
To conduct our review, we interviewed the complainant’s representative for the 2008 application
and HUD officials in the Office of Native American Programs including the Grants Management
Director, the Southwest Office Administrator, and the individuals who evaluated Kaibab’s 2007
and 2008 applications. We reviewed the relevant parts of HUD’s 2008 notice of funding
availability (2008 notice); HUD laws and regulations for Indian Block Grants; and Office of
Native American Programs guidance for the evaluation, rating, and ranking of Indian Block
Grant applications. We also reviewed documentation submitted by Kaibab for its 2007 and 2008
Indian Block Grant applications, along with the evaluations prepared by HUD review officials.
Finally, we reviewed letters from the complainant and its representative that detailed the
allegations and provided arguments regarding why it believed the support in the application
warranted passing the threshold criteria.
Our review of Kaibab’s grant applications was limited to an evaluation of whether the Southwest
Office’s review of the applications complied with applicable requirements and guidance and
provided a fair review. To accomplish this objective, we considered the following questions:
Were the review processes consistent with Indian Block Grant program requirements?
Was the reviewer’s determination that the proposal failed to meet certain threshold
requirements reasonably supported?
Was the reviewer’s overall conclusion consistent with the applicable threshold criteria
for economic development projects?
We limited our review to work necessary to address the complaint's central issue. Accordingly,
we did not conduct the review in accordance with generally accepted government auditing
standards but obtained sufficient evidence to provide a reasonable basis for our conclusions.
BACKGROUND
The initial complaint regarding the Kaibab 2008 Indian Block Grant application was made to
OIG by the consultant/developer that represented Kaibab for the 2008 grant application process
and assisted with the preparation of the grant application. The OIG hotline accepted the
complaint after the Kaibab tribe followed up with its own signed complaint letter.
Kaibab, a small Native American tribe located in Northwest Arizona, submitted an application to
the Southwest Office for an Indian Block Grant economic development grant of $605,000 for
competition under HUD’s 2008 notice. The project was described as an RV park campground to
be located on tribal land donated by Kaibab. Southwest Office grants management reviewer(s)
rated the application acceptable under the initial application screening criteria and passed it
under the general threshold requirements of 24 CFR (Code of Federal Regulations) 1003.301(a).
In accordance with the 2008 notice requirements, reviewers then evaluated whether the
2
application met the project-specific threshold requirements for economic development projects.1
The reviewers determined that the Kaibab 2008 Indian Block Grant application did not meet the
following two of the five project-specific threshold requirements:
Criterion (1): In accordance with 24 CFR 1003.302, for economic development
assistance, the application must include a financial analysis. The financial analysis must
demonstrate that the project is financially feasible and the project has a reasonable chance
of success. The analysis must also demonstrate the public benefit resulting from the
ICDBG [Indian Block Grant] assistance. The more funds the application requests, the
greater the public benefit that must be demonstrated.
Criterion (4): Not more than a reasonable rate of return on investment is provided to the
owner.
In accordance with the application review procedures, because it did not meet the threshold
screening requirements, the Kaibab 2008 Indian Block Grant application was ineligible to be
further considered under the rating and ranking review process. The Southwest Office proceeded
to issue a letter, dated November 24, 2008, notifying Kaibab that its 2008 application did not
meet threshold requirements and would not receive further funding consideration.
According to a letter, dated January 16, 2009, from HUD’s Deputy Assistant Secretary for
Native American Programs to the chairperson of the Kaibab Band of Paiute Indians, HUD
denied a request for reconsideration of the funding determination. HUD’s basis for the denial
was section V(B)(10) of the 2008 notice, which stated that judgments made within the provisions
of the notice and program regulations are not subject to claims of error.
Kaibab had also submitted an Indian Block Grant application in 2007 for basically the same
project, which Southwest Office officials also declined to rate or rank based upon failure to meet
two project-specific threshold criteria. The complainants updated the 2007 proposal and felt that
the 2008 application adequately addressed the reasons why the 2007 application had failed.
Therefore, the complainant believed that the 2008 application should have passed the project-
specific threshold criteria and at least have been rated and ranked.
RESULTS OF REVIEW
Summary
Our review did not find evidence that the Southwest Office treated Kaibab’s 2008 Indian Block
Grant economic development grant application with prejudice or failed to provide a fair review.
The grant application review procedures—as related to project-specific threshold
screening—were performed in compliance with program requirements. However, we
noted that the review processes were not standardized in a way that easily precluded
1
Fiscal Year 2008 SuperNOFA [notice of funding availability] for HUD’s Discretionary Programs; Community
Development Block Grant Program for Indian Tribes and Alaska Native Villages; Section III.(C)( 3)( c) – Economic
Development Project Thresholds
3
any perception of unfairness. Specifically, the Southwest Office assigned
applications to grants management specialists without regard to ensuring impartial
treatment in appearance and in fact. Although economic development proposals
tended to be uncommon and more complicated than other Indian Block Grant
proposals, application reviewers did not receive training or guidance regarding the
evaluation of such proposals. Finally, the Southwest Office did not require reviewers
to clearly and thoroughly document the reasons for their determination that an
application failed to meet threshold requirements and, therefore, would not be rated or
ranked.
We determined that the Southwest Office reviewers’ determinations regarding
specific threshold criteria were generally defensible and made in good faith. We
noted that, according to the 2008 notice, if an application failed even one threshold
criterion, it could not proceed to rating and ranking. Because we noted that the
reviewers’ written comments were unclear, we also relied on interviews and
consulted program criteria to evaluate the reasonableness of the reviewers’
conclusions for Kaibab’s 2007 and 2008 applications. We agreed that some of the
concerns raised by the reviewers regarding criteria for financial feasibility, reasonable
chance of success, and public benefit were defensible. We also identified significant
weaknesses in the proposals that were not directly addressed by the reviewers.
We concluded that the Southwest Office’s overall decision that Kaibab’s application
did not meet all threshold requirements in 2008 (and in 2007) was within the
parameters of applicable program criteria. We noted that the Indian Block Grant
project-specific threshold criteria and related guidance were nonspecific regarding
key requirements for economic development proposals. As a result, an economic
development proposal had to be persuasive in all respects to pass threshold screening.
The Kaibab application did not include an independent evaluation of the feasibility
study, which is listed in the 2008 notice as an appropriate document to include. As a
result of this omission, we concluded that the Kaibab application was less persuasive
to reviewers, who were not expected to be underwriters or industry experts. In
addition, program guidance was unclear regarding the depth of review required under
the threshold determination for financial feasibility because the 2008 notice also
required reviewers to consider financial feasibility under one of the rating and ranking
factors. Accordingly, reviewers had substantial discretion regarding their approach to
evaluating financial feasibility for the threshold screening. Finally, economic
development proposals represented less than 7 percent (in 2008) of the Indian Block
Grant applications to the Southwest Office, and the projects were generally perceived
to entail greater risk of failure to succeed in the long run. As a result, all economic
development proposals—not just the Kaibab proposal—faced similar barriers to
approval and funding.
We recommend that the Office of Native American Programs Director of Grants Management
require the Southwest Office Administrator to establish a consistent process for assignment of
grant applications to reviewers, develop standards to ensure that written review comments are
4
clear and complete, and develop a consistent evaluation approach for certain nonspecific project
eligibility criteria.
Discussion
The Southwest Office Grants Management’s Indian Block Grant Application Review Processes
Had Weaknesses
Although the procedures for screening, rating, and ranking applications complied with program
requirements and were well documented, the Southwest Office practices were not standardized
regarding (1) assignment and reassignment of applications to grants management specialists, (2)
application of certain nonspecific threshold criteria, and (3) documentation of review
evaluations/comments. As a result, the application review process was not transparent to
outsiders, and allegations of unequal treatment were not easily dispelled.
Assignment of Reviewers Was Not Consistent
In 2007 and 2008, the Southwest Office generally assigned grants management
specialists to review applications from the tribes for which they regularly tracked grants.
According to Southwest Office officials, the grants management supervisor reassigned
applications to other reviewers in some cases when the designated specialist was unable
to complete the review within required timeframes due to workload, absence, or other
reasons. There was apparently no protocol for application reassignments. Likewise,
although each application also received a secondary review (usually by a ―team‖
member), there was no protocol for designating who would perform the secondary
review. In 2007 and 2008, the grants management supervisor reassigned the Kaibab
application before the Kaibab specialist completed the review. However, we found no
indication that this reassignment improperly influenced the review outcome.
According to officials, other offices under the Office of Native American Programs used
a panel approach for rating and ranking applications. However, the Grants Management
Director of the Office of Native American Programs stated that the panel approach was
preferable but not required. We noted that use of a panel or some pre-established rotation
of application assignments to reviewers would preclude the impression that a particular
application was directed to a certain reviewer to facilitate favorable or unfavorable
treatment.
Guidance for Evaluation of Nonspecific Criteria Was Lacking
The Southwest Office had not provided training or other guidance to specialists that
would address the unique features of economic development applications (which
comprised less than 4 percent of the Indian Block Grant applications to the Southwest
Office). Because the project-specific threshold criteria provided little guidance regarding
how to assess financial feasibility or chance of success, each reviewer applied his or her
own level of knowledge and experience when evaluating these criteria for economic
development projects. As a result, different reviewers could reasonably apply very
different benchmarks to the financial projections in the applications and, thus, could
provide different feedback to applicants with similar proposals. In the case of the 2007
5
and 2008 Kaibab applications that we reviewed, the reviewers had very different levels of
training and experience on these issues, yet they arrived at similar conclusions.
Written Review Comments Were Not Clear
The reviewers’ written comments related to the Kaibab application in 2008 were
generally unclear and unconvincing. For example, the comments consisted largely of
quotes from various parts of the applications; however, it was not always clear whether
the quote was meant to demonstrate that the application met the requirement or fell short.
Also, some quotations from the application seemed to have been taken out of context
without an explanation of why other material was deemed less relevant. The comments
for the 2007 application were also primarily quotations, but the reviewer’s conclusions
were in italics. Nevertheless, the distinction between quotations and comments was
unclear until the 2007 reviewer explained this during our interview. Because the written
comments constituted the primary documentation to support a negative threshold decision
that disqualified the application from further review, we believe that the rationale should
have been carefully documented, even if it required a longer explanation. Further, if it is
the Southwest Office’s or Office of Native American Programs’ policy to provide the
comments to the applicant, we question why the comments were written in an
abbreviated manner that made it difficult for an outsider to interpret.
Reviewers’ Overall Conclusions Were Defensible Based Upon Information in the Applications
The reviewers of record generally based their conclusions on information contained in the
applications for both the 2007 and 2008 Kaibab grant applications. Further, given the non-
specificity of some guidance (discussed in the next section of this memorandum), we concluded
that each reviewer arrived at a defensible conclusion in good faith. It should be noted that the
grant application must pass every element of the five project-specific threshold criteria for
economic development; therefore, even one deficiency will cause the application to fail the
overall threshold review. Further, the first project-specific threshold criterion for economic
development entailed five separate tests. We acknowledge that the written review comments
were unclear and did not completely support all of the reviewers’ conclusions. However, we
held discussions with the reviewers and evaluated relevant portions of the grant applications to
clarify the basis on which each application did not meet particular threshold criteria.
Criterion (1)
In both years, reviewers concluded that the applications did not adequately demonstrate
that the project was financially feasible or had a reasonable chance of success—two of
the five tests under the first criterion. As discussed in the next section, the program
guidance for determining financial feasibility was general. As a result, much was left to
the discretion of the reviewers who had to rely solely on the information provided in the
application. Both reviewers stated that the applications failed to convince them of the
project’s financial feasibility and reasonable chance of success. We agreed that some of
the reviewers’ concerns were defensible and questioned whether applications complied
with the Office of Native American Programs’ guidelines regarding the public benefit
criterion.
6
The 2008 reviewer explained that the primary concerns were that the application
showed negative projected net income in the first 5 years2 and did not
demonstrate how the project would provide management with enough experience
to successfully operate the proposed business. In addition, she questioned the
marketing plan and thought that the application had insufficient industry data to
make up for the absence of historical financial data. We determined that the
reviewer’s concerns were defensible regarding the proposal’s (1) negative net
income for 4 to 5 years and (2) lack of a specific plan to ensure management
expertise. The reviewer considered these to be significant indicators of the
likelihood of success, based upon her training and experience.
The 2007 reviewer noted that the application contained conflicting opinions
regarding the financial feasibility of the project and questioned the long payback
period and the marketing plan. In 2007, the reviewer also concluded in the
written comments that the application did not demonstrate an adequate public
benefit—another test under the first criterion. Our review of the application in
this regard also concluded that this was a clear weakness in the 2007 proposal.
We also noted that, because the application had to pass all of the threshold
criteria, the 2007 reviewer may have curtailed further discussion of the proposal’s
financial feasibility once it was determined that the proposal did not meet the
public benefit requirements.
The 2007 application’s deficiencies related to public benefit were addressed to some
extent in the 2008 proposal, and the 2008 reviewer did not note any concerns regarding
public benefit. However, we considered the limited support for demonstrable public
benefit to be a deficiency in the application both years. We reviewed Office of Native
American Programs guidance that stated ―when considering the sufficiency of the
proposed public benefit [for economic development project threshold], the reviewer
should consider the common indicator used in the CDBG [Community Development
Block Grant] program, that no more than $35,000 should be spent for each job
created.‖ 3,4 In 2007, the application proposed two jobs ($605,000/2 = $302,500 per job),
and in 2008, the application proposed five jobs ($605,000/5 = $121,000 per job). We
questioned why Southwest Office reviewers did not discuss (either in written comments
or during interviews) how this indicator did or did not apply to the Kaibab proposal each
year.
2
The 2008 written comments stated, ―Year 15 reflects only a net profit of $10,989.‖ We examined the referenced
page in the application and determined that the statement was correct; however, further scrutiny of the financial
projections revealed that the correct net profit for year 15 should have been $53,964. It is possible that the error
would have been found by the Southwest Office if the more in-depth review of the financials during the ranking and
rating process had been performed. However, the review in question was performed at threshold, and the reviewer
relied on the data without recomputation.
3
Business Process Guide, chapter 5
4
The CDBG program is the Community Development Block Grant program under the HUD Office of Community
Planning and Development.
7
Criterion (4)
In both years, the reviewers concluded that the proposals did not meet the criterion that
the activity should not provide more than a reasonable rate of return on investment. For
both applications, reviewer comments did not specifically state that the rate of return on
investment appeared too high. Comments from both reviewers indicated confusion or
imprecision regarding the intent of criterion (4). In both 2007 and 2008, the reviewer’s
comments discussed profitability, payback period, and ―not more than a reasonable ROI
[return on investment]‖ as if these terms were interchangeable. Further, the
complainant’s representative stated several times that Southwest Office officials would
not say whether the return on investment was too high or too low. To be fair, both
reviewers’ written comments regarding criterion (4) focused on the apparent
insufficiency of the return on investment data provided in the applications. However,
given the reviewers’ apparent lack of understanding/clarity about the intent of criterion
(4), if the applications had failed on this criterion alone, it could be appropriate to
consider whether they were improperly failed at threshold. However, in both cases, the
applications also failed under the first criterion.
The 2008 reviewer stated that the return on investment criterion was meant to
screen out projects for which the developer reaped large profits from a public
investment. The reviewer for the 2008 application had also reversed her initial
conclusion regarding return on investment after a briefing with the applicant.
Apparently the reviewer had failed the application on this point because the
projected rate of return was so low in the first few years. However, this response
seemed more applicable to the first criterion regarding financial feasibility.
Regarding the return on investment criterion, the 2007 reviewer stated that the
payback period of 182 years was not reasonable. This response raised the
question of whether the reviewer thought that a long payback period, which might
indicate that the project was not financially feasible, also meant that the return on
investment was too high. We noted that, because a long payback period would
more likely be accompanied by a low return on investment, the statement did not
support the criterion that the rate of return was not more than reasonable.
Indian Block Grant Economic Development Criteria and Guidance Were General
The project-specific threshold criteria for Indian Block Grant economic development projects
provided no measures or standards and were not supplemented by more specific guidance. The
threshold criteria language was almost identical among the Office of Native American Programs
review forms, the applicable notice, the regulations, and the United States Code (42 U.S.C.
5305). Office of Native American Programs officials stated that there was no guidance available
to supplement this language, which stated, ―The financial analysis must demonstrate that the
project is financially feasible and the project has a reasonable chance of success.‖ Our own
research identified minimal guidance in HUD training materials. Again, the specific guidance
primarily repeated the general language in the notice.
8
The Application Must Be Persuasive
Because the criteria were imprecise, each economic development application had to
supply the HUD reviewer with persuasive information showing that the proposed project
would succeed. Accordingly, an application must demonstrate that the project will be
financially feasible by furnishing its own financial estimates and industry standards for
comparison. Because Southwest Office officials received no supplemental training or
guidance regarding how to evaluate a project’s financial feasibility or whether it had a
reasonable chance of success, they stated that the information in the application had to be
―convincing.‖ The complainant noted that the Kaibab application contained a detailed
feasibility study that included the financial statements and projections. However, we
noted that the application did not contain any evaluations or opinions of the feasibility
study by an independent underwriter or other expert on financial projections for
economic development projects (the 2008 application contained an independent cost
certification). The feasibility study was prepared by the tribal consultant/developer that
submitted the application and was also a complainant. According to information in the
application, this individual appeared to be well qualified. Nevertheless, the 2008 notice
stated that the application should appropriately include a ―Financial analysis and
feasibility study no more than two years old which indicates how the proposed business
will capture a fair share of the market, and which has been conducted by an independent
entity‖ (rating factor 3: 5.C.(1)(d)).
In this instance of an economic development proposal for a very specialized industry, we
believe that the absence of an independent review of the feasibility study was a
significant omission from the application. Without an outside expert’s opinion to rely on,
the HUD reviewers—who were not expected to be underwriters or industry experts—
appropriately relied on their own judgments and experience to assess the adequacy of
items such as projected revenues and resulting net incomes. The complainant has
requested an independent outside review of the application, yet we noted that the original
application could have included a review or certification of the feasibility study and
financial projections by an independent professional.
The 2008 Notice Rating Factors Also Addressed Financial Feasibility
According to the 2008 notice, the financial feasibility of an economic development
proposal was to be evaluated under both the threshold screening phase and the rating and
ranking phase. However, there was no guidance regarding how in-depth the threshold
screening for financial feasibility should be, as compared to the rating and ranking review
for the same item. For example, the 2008 notice’s rating and ranking requirements
covered five rating factors including one that addressed the ―soundness of the approach‖
(number 3). Under this, subfactor 5, Commitment to Sustain Activities, also required
analysis of the financial viability for an economic development proposal. The
requirements outlined somewhat specific criteria, including the statement: ―(d) Does the
business plan or cash flow analysis indicate that cash flow will be positive within the first
year?‖ Office of Native American Programs guidance for rating this notice requirement
further suggested that if the business plan or cash flow analysis indicated that the cash
flow will be positive in 2 to 3 years, the application could still be awarded points under
this subfactor. Neither reviewer referred to this criterion for their threshold evaluation.
9
Economic Development Projects Are Viewed as Having More Risk Than Other Indian
Block Grant Activities
Economic development projects are generally considered by SWONAP staff to be more
high-risk ventures when compared to other Indian Block Grant project activities. We
could not find any language in the laws or regulations indicating that a higher-risk level
should be afforded to the economic development projects. The 2008 notice appeared to
address the risk by requiring financial feasibility and other related tests for these projects.
In contrast, the affordable housing projects had no similar ―financial feasibility‖ threshold
tests. As a result, it was more difficult for economic development proposals to persuade
review officials that the project would be as viable over time as the more traditional
housing projects. Also, a small percentage of competitive Indian Block Grant
applications are for economic development projects (4 percent of 2007 and 2008
applications combined). Because Southwest Office staff had limited experience with the
track record of such projects, it had not developed its own formal or informal indicators
or standards to predict the success of economic development proposals. We concluded
that all economic development proposals would have faced similar barriers to approval
and funding.
RECOMMENDATIONS
We recommend that the Office of Native American Programs Director of Grants Management
require the Southwest Office Administrator to
1A. Develop and implement an application review assignment process to ensure that all
applications receive equal consideration for threshold screening in appearance and in
fact.
1B. Develop protocols for written review comments that require reviewers to clearly
state the basis of their conclusions. The comments should be understandable to an
outsider or grantee.
1C. Provide training and develop internal guidelines for evaluation of economic
development grants that will improve consistency in the evaluation approach for key
nonspecific eligibility criteria.
AUDITEE’S RESPONSE
We provided a discussion draft memorandum report to the auditee on November 17, 2009, and
held an exit conference with its staff on December 2, 2009. The auditee provided written
comments on January 4, 2010. It generally agreed with our results, but had some disagreements
with two of the recommendations.
The complete text of the auditee’s response, along with our evaluation of that response, can be
found in appendix A of this report.
10
Appendix A
AUDITEE COMMENTS
Comment 1
Comment 2
11
Comment 3
Comment 4
Comment 5
Comment 6
12
Comment 7
Comment 8
13
OIG Evaluation of Auditee Comments
Ref to OIG Evaluation Auditee Comments
Comment 1 We agree that Rating Factor 3, subfactor 5(c) of the 2008 Notice states that an
independent financial analysis and feasibility study is an appropriate document to
include, but that the project threshold provision does not specify the financial
analysis must be independent. However, the SWONAP reviewer—who was the
grants management supervisor at the time—clearly deemed the lack of
independence on the part of the individual who prepared the proposal’s feasibility
study to be a major deficiency for the threshold determination. If SWONAP
believes this criterion was improperly applied at threshold, then the reviewer’s
argument for questioning the feasibility study’s conclusion is significantly
weakened. The reviewer was very clear that she relied on the proposal to be
convincing, and that the feasibility study’s lack of independence diminished its
credibility to her. The supervisor’s written comments demonstrated this by noting
that the feasibility study was prepared by an entity that was ―qualified but not
independent since also the application grant writer.‖
Comment 2 As the auditee response states, the report recognized that reviewers must rely on
the arguments presented in the application to determine the likelihood of project
success. During interviews, both reviewers were adamant that the information in
the application had to convince them. Although, as acknowledged under
comment 1, an independent feasibility study was not required by the project
threshold determination criteria, it is reasonable to conclude that a feasibility
study prepared by a qualified independent entity would be more convincing than
one prepared by an individual representing the applicant. Accordingly, we stand
by our conclusion that the lack of independence on the part of the feasibility study
author, in the absence of an independent review or certification, significantly
impaired the credibility of the feasibility study conclusions presented in the
proposal. As a result, in both application years, reviewers disregarded the study’s
conclusion (that the project was financially feasible), and relied on their own
judgment/analyses regarding whether the feasibility study data supported a
conclusion that the project was financially feasible and likely to succeed.
Comment 3 We revised the report to state that panels are used for rating and ranking
applications. We cited this practice as an example of how ONAP already has
some operating procedures in place that could help to reduce the effect of any
individual reviewer bias (in appearance or in fact) and facilitate equal
consideration of applications. Furthermore, the Indian Block Grant regulations do
not preclude the use of panels for the threshold determinations. The report also
suggested that other practices such as pre-established rotation of reviewer
assignments would also help to reduce any impression of unequal treatment from
different reviewers.
14
Comment 4 We agree the proper comparison should be to the other Indian Block Grant
(ICDBG) projects, and revised the report accordingly.
Comment 5 The auditee’s response does not address the recommendation that the review
assignment process should ensure that all applications receive equal consideration
for threshold screening in appearance and in fact. The report did not find that
SWONAP’s processing of the 2008 Indian Block Grant application was
inconsistent with ONAP requirements. Rather, the report stated that because the
process to assign applications to [different] reviewers was not standardized, the
application review process was not transparent to outsiders, and allegations of
unequal treatment were not easily dispelled. If SWONAP continues to arbitrarily
assign (and change assignments for) applications to reviewers with varying
qualifications and experience, it will perpetuate the appearance of unequal
consideration among applicants. As a result, applicants who are denied funds will
submit further complaints and FOIA requests. A more transparent review process
would likely reduce such complaints.
Comment 6 We acknowledge the auditee’s commitment to improve written documentation of
reviewers’ decisions. We changed the recommendation wording to state
―protocols‖ in place of ―standards‖ to avoid the use of a term already employed in
formal ONAP guidance. However, during our review, SWONAP reviewers could
not identify any writing protocols or guidance that they used. Further, the only
specific writing protocol we could find in the ONAP business process (BP) guide
pertained to the rating and ranking process:
It is important for GM Specialists to clearly document their decision-
making process when rating applications. It is not enough to simply
indicate a page number on the rating sheet. Instead, GM Specialists must
provide some level of factual discussion on the rating sheet so that a clear
path can be shown between the rating of the information in the application
and the ultimate decision that is made. GM Specialists should be aware
that applicants could request to see the rating sheet during the debriefing
process.
Our recommendation pertains to documentation of the reasons an application was
failed at the project threshold stage. We are recommending that SWONAP
require its reviewers to clearly state the basis of their conclusions in a manner that
is understandable to an outsider or grantee.
Comment 7 The auditee’s response does not address the recommendation that SWONAP
develop internal guidelines that will improve consistency in the evaluation
approach for certain criteria. The report noted that the threshold criteria language
for economic development activities was almost identical among the Office of
Native American Programs review forms, the applicable notice, the regulations,
and the United States Code (42 U.S.C. 5305). The detailed discussion and review
checklist in the ONAP business process protocols that the auditee refers to is
15
nearly identical to the language in the applicable law and regulations. As a result,
the reviewers must apply their own judgment regarding benchmarks used by the
application to conclude the feasibility study demonstrated the proposal’s financial
feasibility and reasonable chance of success. We further note that, according to
the 2008 NOFA, ―Applicants within an Area ONAP’s geographic jurisdiction
compete only against each other for that Area ONAP’s allocation of funds‖.
Therefore SWONAP could appropriately augment ONAP business protocols with
its own internal guidelines to address significant inconsistencies in evaluation
approaches taken by its reviewers.
Comment 8 We acknowledge the auditee’s commitment to staff training regarding the ICDBG
NOFA requirements. However, the auditee’s response does not address specific
training for evaluation of economic development grants that will improve
consistency among SWONAP reviewers. As we noted under comment 7,
SWONAP’s funding allocation is dedicated to a defined group of potential
applicants. Therefore SWONAP could appropriately provide its staff with
additional training to promote consistency in the application of ONAP business
protocols regarding the nonspecific criteria for economic development activities.
16
Review Results for Hotline Case Number HL-09-0756 Kaibab Band of Paiute Indians, Pipe Spring, AZ
Published by the Department of Housing and Urban Development, Office of Inspector General on 2010-01-25.
Below is a raw (and likely hideous) rendition of the original report. (PDF)