oversight

Small Business Administration: Enhancements Needed for Loan Monitoring System Benchmark Study

Published by the Government Accountability Office on 1999-05-14.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                  United States General Accounting Office

GAO               Report to Congressional Requesters




May 1999
                  SMALL BUSINESS
                  ADMINISTRATION

                  Enhancements
                  Needed for Loan
                  Monitoring System
                  Benchmark Study




GAO/AIMD-99-165
                   United States
GAO                General Accounting Office
                   Washington, D.C. 20548                                                        Leter




                   Accounting Information and
                   Management Division

                   B-280910                                                              Letter

                   May 14, 1999

                   The Honorable James M. Talent
                   Chairman
                   Committee on Small Business
                   House of Representatives

                   The Honorable Roscoe G. Bartlett
                   Chairman
                   Subcommittee on Government
                    Programs and Oversight
                   Committee on Small Business
                   House of Representatives

                   The Small Business Reauthorization Act of 1997 required the Small
                   Business Administration (SBA) to complete eight planning actions to serve
                   as a basis for funding the development and implementation of a proposed
                   loan monitoring system. At your request, we evaluated the report produced
                   for SBA’s first planning action—a benchmark study by a contractor that
                   compared SBA’s loan monitoring business processes to those of selected
                   public and private sector organizations. On April 6, 1999, we briefed your
                   offices on the results of our work. The briefing slides are included in
                   appendix I.

                   This report provides a high-level summary of the information presented in
                   the briefing, including the background, methodology and findings of the
                   benchmark study, and our observations on the benchmark report. This
                   report also presents the suggestions we made to enhance the usefulness of
                   the benchmark study as well as SBA’s comments.



Results in Brief   The benchmark study was an important first step in SBA’s efforts to
                   develop a loan monitoring system. It identified wide gaps between SBA’s
                   practices and its benchmark partners. However, the study would be more
                   useful if it included a better definition of processes associated with best
                   practices, addressed monitoring processes for important activities
                   delegated to lenders, collected measurement data on SBA’s and the
                   benchmark partners’ processes, identified goals for reengineering, and
                   identified potential outsourcing and candidate systems for purchase. SBA
                   agreed with this analysis and stated that it plans to act on each of these
                   items.




                   Page 1                                   GAO/AIMD-99-165 SBA Benchmark Study
                  B-280910




Background        The purpose of SBA’s proposed loan monitoring systems is to use
                  technology and new processes to manage portfolios of small business
                  loans, identify and effectively mitigate risks incurred through loans
                  guaranteed by SBA, implement oversight of internal and external
                  operations, and calculate subsidy rates.

                  The Small Business Reauthorization Act of 1997 required SBA to complete
                  eight planning actions to serve as a basis for funding the development and
                  implementation of the loan monitoring system. One of these requirements
                  was that SBA benchmark loan monitoring business processes and systems
                  against comparable industry processes and, if appropriate, simplify and/or
                  redefine its work processes based on these benchmarks.

                  Benchmarking is the comparison of core process performance with other
                  components of the agency (internal benchmarking) and/or with leading
                  organizations (external benchmarking). Through this comparison, best
                  practices are identified for adaptation into the agency’s operations. Best
                  practices include the processes, practices, and systems that perform
                  exceptionally well in specific areas of public and private organizations.
                  Benchmarking provides a means of establishing a compelling business case
                  for change. It should identify more efficient and effective processes for
                  achieving intended results and suggest goals for program output, product
                  and service quality, and process improvement.



Methodology and   SBA’s contractor used a seven-step benchmarking process to evaluate SBA
                  business gaps with similar organizations for five loan management
Findings of the   functions. The functions benchmarked included risk management, lender
Benchmark Study   oversight, guaranty procedures, subsidy rate calculation, and asset sales.
                  These functions were benchmarked against the practices at 11 federal and
                  private sector organizations.

                  The benchmark report, issued in December 1998, showed a significant gap
                  between SBA’s and benchmark partners’ practices for each of the
                  management functions. The report also suggested that senior management
                  needs to buy into the reengineering process, communications plans need to
                  be developed, system requirements should be preliminary defined, and
                  training plans should be examined.




                  Page 2                                  GAO/AIMD-99-165 SBA Benchmark Study
                        B-280910




Observations on the     In general, the benchmarking methods used were consistent with accepted
                        practices and the benchmarking methodology was followed at a high level.
Benchmark Report        The benchmark report identified standard industry or ‘good’ practices. It
                        also pointed out wide gaps between SBA’s practices and those of and its
                        benchmark partners.

                        The benchmark study was an important first step in SBA’s efforts to
                        develop a loan monitoring system. However, there were areas where
                        enhancements would make the benchmark study report more useful as
                        SBA decides which processes it will reengineer and proceeds with the
                        reengineering. The report would be more useful if it included

                        • the criteria used to determine and validate “best-in-class” processes, 1
                        • the monitoring processes, practices, and systems for activities delegated
                          to lenders for preventing delinquencies, mitigating losses on
                          delinquencies, and liquidating defaulted loans,
                        • candidates for the outsourcing of functions and the extent of
                          outsourcing by benchmark partners, and
                        • quantative measurements of SBA’s and benchmark partners’ processes,
                          measurable improvement goals, and guidelines and parameters for the
                          reengineering of processes.



Suggested Actions and   We suggested that SBA

Agency Comments         • define processes associated with best practices and relate these to SBA’s
                          current processes,
                        • address activities delegated to lenders and develop monitoring
                          processes concerning lenders to prevent delinquencies, mitigate losses
                          on delinquencies, and liquidate defaulted loans,
                        • collect measurement data to allow comparisons between SBA’s current
                          processes and the processes of benchmark partners,
                        • set “stretch” goals for reengineering, and
                        • identify from benchmark partners potential outsourcing and candidate
                          systems for purchase.



                        1
                          According to SBA, the contractor used criteria to select benchmark partners and determine best
                        practices but did not present this information in the benchmark report. SBA plans to have the report
                        modified to specify these criteria.




                        Page 3                                                 GAO/AIMD-99-165 SBA Benchmark Study
                        B-280910




                        SBA agreed with our analysis of the benchmark study and stated that it
                        plans to implement these suggested actions. Appendix II contains the
                        agency comments.



Objective, Scope, and   Our objective was to determine if SBA had completed the benchmarking
                        activities required by the Small Business Reauthorization Act of 1997 in
Methodology             accordance with generally accepted practices. We analyzed SBA’s
                        benchmark report to determine if it was completed in accordance with the
                        principles specified by the Council for Continuous Improvement2 and our
                        business process reengineering (BPR) guidance. We performed our work
                        between January 1999 and March 1999 in accordance with generally
                        accepted government auditing standards.


                        We are sending copies of this report to The Honorable Nydia M. Velazquez,
                        Ranking Minority Member of the Committee on Small Business; The
                        Honorable Aida Alvarez, Administrator, Small Business Administration;
                        The Honorable Jacob J. Lew, Director, Office of Management and Budget;
                        and other interested parties. Copies will also be made available to others
                        upon request.

                        If you have any questions on matters discussed in this report, please
                        contact me at (202) 512-6253 or James Hamilton, Assistant Director, at
                        (202) 512-6271. Major contributors to this report are listed in appendix III.




                        Joel C. Willemssen
                        Director, Civil Agencies Information Systems




                        2
                          The Council is a nonprofit consortium of companies whose purpose is to develop a comprehensive
                        system for implementing continuous product and process improvement using state-of-the-industry
                        methods and tools.




                        Page 4                                                GAO/AIMD-99-165 SBA Benchmark Study
Page 5   GAO/AIMD-99-165 SBA Benchmark Study
Contents



Letter                                                                                        1


Appendix I                                                                                    8
Briefing on the
Benchmark Study for
SBA’s Loan Monitoring
System

Appendix II                                                                                  38
Comments From the
Small Business
Administration

Appendix III                                                                                 48
Major Contributors to
This Report




                        Abbreviations

                        BPR      business process reengineering
                        SBA      Small Business Administration




                        Page 6                               GAO/AIMD-99-165 SBA Benchmark Study
Page 7   GAO/AIMD-99-165 SBA Benchmark Study
Appendix I

Briefing on the Benchmark Study for SBA’s
Loan Monitoring System                                                                                       AppIexndi




               Accounting and Information Management
               Division

             Small Business Administration’s Loan
                      Monitoring System

                  Briefing on the Benchmark Study
                                         April 6, 1999




             Presented to the House Committee on Small Business and its Subcommittee on
             Government Programs and Oversight.




                              Page 8                                       GAO/AIMD-99-165 SBA Benchmark Study
                 Appendix I
                 Briefing on the Benchmark Study for SBA’s
                 Loan Monitoring System




      Briefing Purpose & Outline

    • Briefing purpose is to present results of our review of
      a benchmark report on the development of Small
      Business Administration’s (SBA) loan monitoring
      system (LMS).

    • Briefing outline:
       •   Objective & Scope
       •   Background
       •   Benchmarking methodology and findings
       •   Suggestions
       •   SBA comments


2




                 Page 9                                      GAO/AIMD-99-165 SBA Benchmark Study
                 Appendix I
                 Briefing on the Benchmark Study for SBA’s
                 Loan Monitoring System




      Objective & Scope

    • Our objective is to determine if SBA is completing
      planning actions required by the Small Business
      Reauthorization Act of 1997 in accordance with required
      and generally accepted system development practices.
    • This part of the review pertains to our evaluation of SBA’s
      benchmark report.
    • We analyzed the benchmark report to determine if it was
      completed in accordance with the principles specified by
      the Council for Continuous Improvement and GAO’s
      Business Process Reengineering (BPR) guidance. We
      performed our work between January 1999 and March
      1999 in accordance with generally accepted government
      audit standards.

3




                 Page 10                                     GAO/AIMD-99-165 SBA Benchmark Study
                 Appendix I
                 Briefing on the Benchmark Study for SBA’s
                 Loan Monitoring System




     Background
     Purpose of LMS

    • Use of technology and new processes to
      • manage portfolios of small business loans
      • identify and effectively mitigate risks incurred through
        loans guaranteed by SBA
      • implement oversight of internal and external operations
      • calculate subsidy rates




4




                 Page 11                                     GAO/AIMD-99-165 SBA Benchmark Study
                       Appendix I
                       Briefing on the Benchmark Study for SBA’s
                       Loan Monitoring System




           Background
           Planning actions for LMS
    The Small Business Reauthorization Act of 1997
    required SBA to complete eight planning actions to
    serve as a basis for funding the development and
    implementation of the loan monitoring system
    • Benchmark loan monitoring business processes and systems
      against comparable industry processes and, if appropriate,
      simplify and/or redefine work processes based on these
      benchmarks
    • Define system requirements using on-line, automated capabilities
      to the extent feasible
    • Identify all data inputs and outputs necessary for timely report
      generation


5




                       Page 12                                     GAO/AIMD-99-165 SBA Benchmark Study
                      Appendix I
                      Briefing on the Benchmark Study for SBA’s
                      Loan Monitoring System




          Background
          Planning actions for LMS (continued)

    • Determine data quality standards and control systems for ensuring
      information accuracy
    • Identify an acquisition strategy and work increments to completion
    • Analyze the benefits and costs of alternatives and use to
      demonstrate the advantage of the final project
    • Ensure that the proposed information system is consistent with the
      agency’s information architecture
    • Estimate the cost to system completion




6




                      Page 13                                     GAO/AIMD-99-165 SBA Benchmark Study
                   Appendix I
                   Briefing on the Benchmark Study for SBA’s
                   Loan Monitoring System




      Background
      Benchmarking definition

    • Benchmarking is the comparison of core process performance
      with other components of the agency (internal benchmarking)
      or with leading organizations (external benchmarking).
      Through this comparison, best practices are identified for
      adaptation into the agency’s operations.
    • Best practices include the processes, practices, and systems
      that perform exceptionally well in specific areas of public and
      private organizations.
    • Benchmarking provides a means of establishing a compelling
      business case for change. It should
       • identify more efficient and effective processes for achieving
         intended results, and
       • suggest goals for program output, product and service
         quality, and process improvement.
7




                   Page 14                                     GAO/AIMD-99-165 SBA Benchmark Study
                                                    Appendix I
                                                    Briefing on the Benchmark Study for SBA’s
                                                    Loan Monitoring System




                     Contractor’s Benchmarking
                     Methodology
               • SBA’s contractor used a seven-step benchmarking
                 process to evaluate SBA business gaps with similar
                 organizations.


                                                                                                      4
                                                                                                     Measure
                                                                                                  performance of
                                                                                                   benchmarked
                                                                                                     company




                                                        Identify Key                                                                 Specify
                                                                                      Identify                    Compare


                               1                           2                           3                           6                 7
                            Determine                                                                                             improvement
                                                        Performance                 Companies to                benchmarks/
                          which functions                                                                                           goals and
                                                        Variables to
                          to benchmark                                               Benchmark                 pinpoint causes
                                                          Measure                                                                 monitor results




                                                                                                      5
                                                                                                     Measure
                                                                                                  performance of
                                                                                                   own company




Source: Benchmarking Capital Access/Financial Assistance Program, Final Benchmark Report, Booz Allen & Hamilton, December 1998



8




                                                    Page 15                                                                      GAO/AIMD-99-165 SBA Benchmark Study
                                                        Appendix I
                                                        Briefing on the Benchmark Study for SBA’s
                                                        Loan Monitoring System




                     Lender and SBA Activities for
                     Guaranteed Loans

                        PROGRAM
                                                                                                                         LIQUIDATION
                                                                                                                                                      Lender and SBA
                        DEVELOPMENT                               ORIGINATING                      SERVICING
                        & ADVOCACY
                                                                                                                            PAYMENT
                                                                                                                                                      loan activities
                                                                                                                             IN FULL
                                                                                                                                                      that were to be
                       Lender Activities               Lender Activities               Lender Activities            Lender Activities
                                                                                                                                                      addressed in the
                       • Direct potentially eligible   • Evaluate eligibility          • Bill recipients            • Liquidate and collect on
                         borrowers to programs         • Evaluate credit risk
                                                       • Underwrite loans
                                                                                       • Receive and process
                                                                                         payments
                                                                                                                      bad loans
                                                                                                                    • Clean-up liquidation for
                                                                                                                                                      benchmark
                                                       • Disburse loans                • Update loan status info.     resale
                                                                                       • Send delinquency notices
                                                                                       • Sell/purchase loans
                                                                                                                    • Close loans paid in-full
                                                                                                                    • Perform work-out agree-
                                                                                                                                                      study
                                                                                                                      ments and refinance loans

                       Guarantor Activities            Guarantor Activities            Guarantor Activities         Guarantor Activities
                       • Determine Credit Policy       • Monitor/approve eligibility   • Monitor subsidy rate(s)    • Monitor liquidation
                       • Design lending programs       • Capture credit risk             against repurchases        • Ensure fair treatment
                       • Advocate programs to            information and update        • Monitor portfolio status   • Re-evaluate credit policy
                         lenders and recipients          portfolio risk management       through loan updates       • Monitor program results
                       • Monitor program results       • Capture underwriting          • Take corrective action
                                                         and eligibility information     where needed
                                                       • Track disbursements




Source: Benchmarking Capital Access/Financial Assistance Program, Final Benchmark Report, Booz Allen & Hamilton, December 1998



9




                                                        Page 16                                                                                   GAO/AIMD-99-165 SBA Benchmark Study
                                                    Appendix I
                                                    Briefing on the Benchmark Study for SBA’s
                                                    Loan Monitoring System




                     Loan Management Functions
                     Benchmarked
               • Risk management -- use captured information to determine
                 optimal underwriting criteria and portfolio exposure

               • Lender oversight -- use captured information to ensure lenders
                 are acting in the best interest of the SBA

               • Guaranty procedures -- approve loan guaranties and collect
                 origination and servicing data

               • Subsidy rate calculation -- use captured information to predict
                 accurately future cash flows

               • Asset sales -- use captured information to determine optimal
                 value for a portfolio of loans and guaranties
Source: Benchmarking Capital Access/Financial Assistance Program, Final Benchmark Report, Booz Allen & Hamilton, December 1998



10




                                                    Page 17                                                                      GAO/AIMD-99-165 SBA Benchmark Study
                                                    Appendix I
                                                    Briefing on the Benchmark Study for SBA’s
                                                    Loan Monitoring System




                     Organizations Selected as
                     Benchmarking Partners
                           ORGANIZATION                    LENDER                     RISK                  LENDING              SUBSIDY   ASSET
                                                         OVERSIGHT             MANAGEMENT                PROCEDURES               RATE     SALES
                       Freddie Mac                             XX                       XX                       XX
                       Fannie Mae                              XX                       XX                       XX
                       NationsBank Corp                                                 XX                       XX
                       First Union Corp.                                                XX                       XX
                       Citibank Corp.                                                   XX                       XX
                       Hibernia Corp.                                                   XX                       XX
                       The Money Store                                                  XX

                       Federal Housing                         XX                                                                           XX
                       Authority
                       Veterans Administration                 XX                       XX                                                  XX
                       Comptroller of Currency                 XX                                                                  XX
                       Office of Management                                                                                        XX
                       And Budget

                                                               XX             Denotes Best Practices


Source: Benchmarking Capital Access/Financial Assistance Program, Final Benchmark Report, Booz Allen & Hamilton, December 1998



11




                                                    Page 18                                                                          GAO/AIMD-99-165 SBA Benchmark Study
                                                     Appendix I
                                                     Briefing on the Benchmark Study for SBA’s
                                                     Loan Monitoring System




                     Gap Between SBA and Benchmark Partners’
                     Practices for Risk Management
                              DIMENSION                                                                                                       COMMENTS
                                                              Below                       Average                        Above
                                                             Average                                                    Average
                   Definition of Risk Management                                     ●                                     ▲       •   No definition has been agreed
                                                                                                                                       upon within the organization

                            Understanding Different                                  ●                                      ▲      •   Risk are understood only by
                            Risks                                                                                                      responsible parties

                            Strategies For Defined                         ●                                                ▲      •   No Strategies exist to measure,
                            Risks                                                                                                      monitor or mitigate risks

                            Point at which risk is                                   ●                                      ▲      •   Risk is incurred when guaranties
                            Incurred                                                                                                   are purchased from lenders

                   Organizational Placement of Risk                                  ●                                      ▲      •   Risk Management group has
                   Management Function                                                                                                 been loosely defined

                            Risk Management                                          ●                                      ▲      •   A coordinator has been assigned
                            Coordinator                                                                                                but risks have not been defined

                            Power to Induce Change                         ●                                                ▲      •   Management has authorized
                                                                                                                                       power but it has not been tested

                   How Much Information is                                           ●                                      ▲      •   Information is gathered at time of
                   Collected & When?                                                                                                   guaranty purchase

                   Is Information Used to Mitigate                         ●                                                ▲      •   Possible risks are not
                   Risk?                                                                                                               understood until they materialize




                                                             ● Current Level           ▲ Best in Class                           Gap



Source: Benchmarking Capital Access/Financial Assistance Program, Final Benchmark Report, Booz Allen & Hamilton, December 1998



12




                                                     Page 19                                                                           GAO/AIMD-99-165 SBA Benchmark Study
                                                    Appendix I
                                                    Briefing on the Benchmark Study for SBA’s
                                                    Loan Monitoring System




                     Gap Between SBA and Benchmark Partners’
                     Practices for Risk Management (continued)
                    DIMENSION                                                                                                           COMMENTS
                                                   Below                        Average                       Above
                                                  Average                                                    Average
         Risk Information Used in                                ●                                              ▲    •           Risk Information is not
         Program Development
                                                                                                                                 available for use
                  Program Eligibility                            ●                                                ▲         •    Comprehensive data on
                                                                                                                                 eligibility is not collected
                  Underwriting Criteria                          ●                                                ▲         •    Origination data is not
                                                                                                                                 collected from lenders
                  Guaranty Fee Calculations                      ●                                                ▲         •    Fees are based on public
                                                                                                                                 policy only
                  Program Subsidy Rate                           ●                                                ▲         •    Subsidy Rate is based on
                                                                                                                                 historical data
         Information Used in Developing                          ●                                                ▲         •    Lack of information makes it
         Program Strategy
                                                                                                                                 difficult to develop strategies
                  Additional Risks in Regions                    ●                                                ▲         •    Regional risks are understood
                  Explained
                                                                                                                                 but not quantified
                  Overall Effectiveness of                                 ●                                      ▲         •    Indicators are not managed
                  Program
                                                                                                                                 but are used for performance
         Seamless and Consistent                                           ●                                      ▲         •    Information is mainly paper
         Transfer of Information                                                                                                 based with some use of on-
                                                                                                                                 line transfers


                                                             ● Current Level           ▲ Best in Class                            Gap

Source: Benchmarking Capital Access/Financial Assistance Program, Final Benchmark Report, Booz Allen & Hamilton, December 1998


13




                                                    Page 20                                                                               GAO/AIMD-99-165 SBA Benchmark Study
                                                    Appendix I
                                                    Briefing on the Benchmark Study for SBA’s
                                                    Loan Monitoring System




                     Gap Between SBA and Benchmark Partners’
                     Practices for Lender Oversight
                        DIMENSION                                                                                                          COMMENTS
                                                        Below                       Average                         Above
                                                      Average                                                     Average
             Communication to Lenders                                                                   ●            ▲           •   The lender review process
                                                                                                                                     has been communicated
                                                                                                                                     effectively to the lenders
                     Program Goals                                                                      ●              ▲         •   Program goals and
                                                                                                                                     objectives are communicated
                                                                                                                                     regularly
                     Program Procedures                                                                 ●              ▲         •   Procedures can be easily
                                                                                                                                     found on the Internet
             Effective Lender Review Process                              ●                                            ▲         •   Lender review process does
                                                                                                                                     not review all aspects of a
                                                                                                                                     lenders operations
                     Process Communicated                                                ●                             ▲         •   Lenders have no say in
                                                                                                                                     determining optimal schedule
                     Understanding &                                                                    ●              ▲         •   Lenders comply with the
                     Compliance
                                                                                                                                     process but do not provide
                                                                                                                                     extra assistance
                     Includes All Lenders                                                ●                             ▲         •   Only PLP lenders are
                                                                                                                                     included in the process


                                                             ● Current Level           ▲ Best in Class                           Gap
Source: Benchmarking Capital Access/Financial Assistance Program, Final Benchmark Report, Booz Allen & Hamilton, December 1998



14




                                                    Page 21                                                                              GAO/AIMD-99-165 SBA Benchmark Study
                                                    Appendix I
                                                    Briefing on the Benchmark Study for SBA’s
                                                    Loan Monitoring System




                     Gap Between SBA and Benchmark Partners’
                     Practices for Lender Oversight (continued)
                          DIMENSION                                                                                                             COMMENTS
                                                            Below                         Average                          Above
                                                          Average                                                        Average
              Ensuring SBA’s Best Interests                                                    ●                                 ▲   •    Lender Oversight attempts to
                                                                                                                                          balance SBA needs while
                                                                                                                                          protecting lender operations
                       Timely & Efficient                                                      ●                                 ▲   •    Reviews are done efficiently
                                                                                                                                          by an independent team on
                                                                                                                                          an annual basis for PLP only
                       Lenders’ Treatment of                                   ●                                                 ▲   •    Only SBA loans are reviewed
                       Loans
                       Procedure Review                                                        ●                                 ▲   •    Underwriting, servicing and
                                                                                                                                          liquidating are reviewed
                       Credit Analysis Review                             ●                                                      ▲   •    Reviewers rely on
                                                                                                                                          examination of historical data
              Reason For Lender Inclusion                                      ●                                                 ▲   •    Only PLP lenders are
                                                                                                                                          included in review process
                       Portfolio Analysis                                      ●                                                 ▲   •    Past portfolio performance is
                                                                                                                                          used to determine if an on-
                                                                                                                                          site review will occur



                                                                      ● Current Level           ▲ Best in Class                          Gap

Source: Benchmarking Capital Access/Financial Assistance Program, Final Benchmark Report, Booz Allen & Hamilton, December 1998



15




                                                    Page 22                                                                          GAO/AIMD-99-165 SBA Benchmark Study
                                                    Appendix I
                                                    Briefing on the Benchmark Study for SBA’s
                                                    Loan Monitoring System




                     Gap Between SBA and Benchmark Partners’
                     Practices for Lender Oversight (continued)
                         DIMENSION                                                                                                           COMMENTS
                                                          Below                         Average                         Above
                                                         Average                                                       Average
                      Current Lending Process
                                                                             ●                                             ▲       •   Loan information is only
                                                                                                                                       reviewed at the annual review
                                                                                                                                       for PLP lenders
               Frequency of Lender Review                                                   ●                              ▲       •   PLP lenders are reviewed on
                                                                                                                                       an annual basis. Non-PLP
                                                                                                                                       are not reviewed at all
                      Selection of Lenders for
                                                                             ●                                             ▲       •   Only PLP lenders are chosen
                      review
                                                                                                                                       and special consideration is
                                                                                                                                       given to lenders with high
                                                                                                                                       levels of defaults
                      Responsibility for Lender                                             ●                              ▲       •   The Lender Oversight group
                      selection
                                                                                                                                       determines which lenders will
                                                                                                                                       be reviewed but only examine
                                                                                                                                       PLP lenders
                      Use of Review Information              ●                                                             ▲       •   There is no central repository
                      throughout the program
                                                                                                                                       for information obtained
                                                                                                                                       during lender reviews

                                                             ● Current Level           ▲ Best in Class                           Gap
Source: Benchmarking Capital Access/Financial Assistance Program, Final Benchmark Report, Booz Allen & Hamilton, December 1998



16




                                                    Page 23                                                                            GAO/AIMD-99-165 SBA Benchmark Study
                                                    Appendix I
                                                    Briefing on the Benchmark Study for SBA’s
                                                    Loan Monitoring System




                     Gap Between SBA and Benchmark Partners’
                     Practices for Guaranty Procedures
                   DIMENSION                                                                                                                COMMENTS
                                                     Below                         Average                          Above
                                                    Average                                                        Average
       Effective Guaranty Underwriting                                                                                           •   The program is effective in
                                                                        ●                                              ▲             determining the ability to pay but
                                                                                                                                     not the ability to collect
                                                                                                                                     information on the borrowers
                 Automated for large lenders                                                                                     •   The main form of communication
                                                                        ●                                              ▲             between the program and
                                                                                                                                     lenders is the fax
                 Simple for small lenders                                                                                        •   Small program lenders are
                                                                        ●                                              ▲             required to perform the same
                                                                                                                                     tasks as large, high exposure
                                                                                                                                     lenders
       Effective Underwriting Criteria                                                                                           •   SOPs require that ability to pay
                                                                                        ●                              ▲             is analyzed but no specifications
                                                                                                                                     are found for determining ability
                                                                                                                                     to pay
                 Repayment Ability                                                                                               •   Income of the business is
                                                                        ●                                              ▲             analyzed but the overall ability to
                                                                                                                                     pay is not necessarily
                                                                                                                                     determined
                 Credit Scoring                                                                                                  •   No Credit scores are being
                                                                   ●                                                   ▲             tested in the LowDoc Program
                 Other Measures &                                                                                                •   Some additional information is
                 Indicators                                             ●                                              ▲             captured to indicate which
                                                                                                                                     industries and ethnic groups are
                                                                                                                                     being served by the program


                                                             ● Current Level           ▲ Best in Class                           Gap
Source: Benchmarking Capital Access/Financial Assistance Program, Final Benchmark Report, Booz Allen & Hamilton, December 1998



17




                                                    Page 24                                                                            GAO/AIMD-99-165 SBA Benchmark Study
                                                    Appendix I
                                                    Briefing on the Benchmark Study for SBA’s
                                                    Loan Monitoring System




                     Gap Between SBA and Benchmark Partners’
                     Practices for Guaranty Procedures (cont.)
             DIMENSION                                                                                                                      COMMENTS
                                                 Below                          Average                            Above
                                               Average                                                            Average
 Relevant Information Captured                                                                                                   •   Inadequate information is
                                                               ●                                                      ▲              captured during the origination
                                                                                                                                     and loan approval process
          Proper Data Collected                                                                                                  •   Relevant risk data from the
                                                               ●                                                      ▲              origination process is not
                                                                                                                                     captured for future use
          Information Used for                                                                                                   •   Loan information is entered into
          Tracking                                                        ●                                           ▲              the accounting system for
                                                                                                                                     tracking purposes
          Data Used for Risk                                                                                                     •   No relevant risk management
          Management                                           ●                                                      ▲              information is captured
          Data Used for Lender                                                                                                   •   No data is captured to review
          Oversight                                            ●                                                      ▲              the lenders' underwriting
                                                                                                                                     effectiveness
 Centralization of Approval &                                                                                                    •   The PLP, FA$TRAK and
 Underwriting Processes                                                                               ●               ▲              LowDoc processes centralized
         Credit Criteria Sets                                                                                                    •   Guaranty levels are set through
         Guaranty Level                                              ●                                                ▲              public policy
         Credit Criteria Sets                                                                                                    •   Guaranty fees are set through
         Guaranty Fees                                               ●                                                ▲              public policy
 Efficient Approval System                                                                                                       •   The loan approval system
                                                                     ●                                                ▲              provides timely approval with
                                                                                                                                     little underwriting of loans



                                                             ● Current Level           ▲ Best in Class                           Gap
Source: Benchmarking Capital Access/Financial Assistance Program, Final Benchmark Report, Booz Allen & Hamilton, December 1998



18




                                                    Page 25                                                                            GAO/AIMD-99-165 SBA Benchmark Study
                                                    Appendix I
                                                    Briefing on the Benchmark Study for SBA’s
                                                    Loan Monitoring System




                     Gap Between SBA and Benchmark Partners’
                     Practices for Subsidy Rate
                   DIMENSION                                                                                                           COMMENTS
                                                   Below                        Average                        Above
                                                  Average                                                     Average
       Subsidy Rate Model                                            ●                                              ▲        •   Current model uses 10-year
                                                                                                                                 cohort with estimated cash flows
                Structural Changes Are                               ●                                              ▲        •   Changes are only made with
                Considered
                                                                                                                                 large scale program adjustments
                Macro-Economic Changes                               ●                                              ▲        •   Macro-economic changes can
                Are Accounted For
                                                                                                                                 not be added into the model
       Risk Management & Origination                                                 ●                              ▲        •   Any risk management
       Data is Used in Calculation
                                                                                                                                 information is used to determine
                                                                                                                                 validity of the calculation
       Frequent Subsidy Rate                                                         ●                              ▲        •   Quarterly analysis of the
       Calculation
                                                                                                                                 calculation is possible
       Comparison of Subsidy Rate to                                                 ●                              ▲        •   Examination of actual program
       Actual Program Cash Flows
                                                                                                                                 expenditures is done often
                Actions Are Taken if                                 ●                                              ▲        •   Program changes must be
                Program Exceeds Budget
                                                                                                                                 approved by governing bodies
                Program Changes Are                                                  ●                              ▲        •   Program changes can be made
                Timely
                                                                                                                                 but not frequently



                                                             ● Current Level           ▲ Best in Class                           Gap
Source: Benchmarking Capital Access/Financial Assistance Program, Final Benchmark Report, Booz Allen & Hamilton, December 1998


19




                                                    Page 26                                                                              GAO/AIMD-99-165 SBA Benchmark Study
                                                    Appendix I
                                                    Briefing on the Benchmark Study for SBA’s
                                                    Loan Monitoring System




                     Gap Between SBA and Benchmark Partners’
                     Practices for Asset Sales

                DIMENSION                                                                                                                        COMMENTS
                                                    Below                             Average                             Above
                                                   Average                                                               Average
  Primary Purpose                                                                                                                      •   The Asset Sales mission has
                                                                                           ●                                     ▲         not been clearly defined
             Revenue Enhancer                                                                                                          •   SBA is attempting to reduce
                                                                                           ●                                     ▲         servicing and liquidating
                                                                                                                                           responsibilities
                                                                                                                                       •   No attempt is made to mitigate
             Mitigate Credit Risk                                         ●                                                      ▲         credit risk
  Underlying Assets Fairly Valued                                                                                                      •   SBA does attempt to value the
                                                                                           ●                                     ▲         loans using an independent
                                                                                                                                           contractor
  Types of Assets Sold                                                                                                                 •   Only Loan Assets are currently
                                                                          ●                                                      ▲         in the sale process (Disaster
                                                                                                                                           Loans Only)
             High or Low Risk                                                                                                          •   Risk is not examined prior to the
                                                                    ●                                                            ▲         sale of the assets
             Performing or Non-                                                                                                        •   Loans are not examined to
             Performing                                             ●                                                            ▲         determine if they are performing
                                                                                                                                           or non-performing prior to sale
  Asset Sales Process an Efficient                                                                                                     •   The overall value creation can
  Use of SBA Resources                                                                     ●                                     ▲         not be determined until an actual
                                                                                                                                           sale takes place


                                                             ● Current Level           ▲ Best in Class                           Gap
Source: Benchmarking Capital Access/Financial Assistance Program, Final Benchmark Report, Booz Allen & Hamilton, December 1998



20




                                                    Page 27                                                                            GAO/AIMD-99-165 SBA Benchmark Study
                                                    Appendix I
                                                    Briefing on the Benchmark Study for SBA’s
                                                    Loan Monitoring System




                     Actions Suggested by Contractor

               • Senior management needs to buy-in to the process

               • Communications plans need to be developed

               • System requirements should be preliminarily defined

               • Training plans should be examined




Source: Benchmarking Capital Access/Financial Assistance Program, Final Benchmark Report, Booz Allen & Hamilton, December 1998



21




                                                    Page 28                                                                      GAO/AIMD-99-165 SBA Benchmark Study
                 Appendix I
                 Briefing on the Benchmark Study for SBA’s
                 Loan Monitoring System




       Observations on Report

     • In general, the benchmarking methods were
       consistent with accepted practices.
     • The benchmarking methodology was followed at a
       high level.
     • The benchmark report identified standard industry or
       ‘good’ practices.
     • The report pointed out wide gaps between SBA’s
       practices and its benchmark partners.




22




                 Page 29                                     GAO/AIMD-99-165 SBA Benchmark Study
                   Appendix I
                   Briefing on the Benchmark Study for SBA’s
                   Loan Monitoring System




       Observations on Report (continued)

     This benchmarking effort was an important first step.
     There are a number of areas where enhancements
     would have made the report more useful.
     • Selecting benchmark partners
        • No evidence on how partners were selected
        • No validation that partners had “best-in-class” processes
     • Determining “best practices”
        • No criteria specified
        • No criteria cited to compare between practices--only a
          determination of whether or not SBA uses the practices



23




                   Page 30                                     GAO/AIMD-99-165 SBA Benchmark Study
                  Appendix I
                  Briefing on the Benchmark Study for SBA’s
                  Loan Monitoring System




       Observations on Report (continued)

     • Addressing activities delegated to lenders - the study
       did not address the processes, practices, and
       information systems for
        • preventing delinquencies
        • mitigating losses such as follow-up actions on
          delinquencies
        • liquidating defaulted loans

     • Outsourcing functions
        • criteria for outsourcing were not specified
        • candidates for outsourcing functions were not identified
        • the extent of outsourcing by benchmark partners was not
          identified

24




                  Page 31                                     GAO/AIMD-99-165 SBA Benchmark Study
                  Appendix I
                  Briefing on the Benchmark Study for SBA’s
                  Loan Monitoring System




       Observations on Report (continued)

     • Using measurements
       • quantitative measurements of SBA and benchmark partners
         were not presented
       • measurable improvement goals for operational services and
         products were not cited
       • guidelines and parameters for reengineering were not
         defined




25




                  Page 32                                     GAO/AIMD-99-165 SBA Benchmark Study
                 Appendix I
                 Briefing on the Benchmark Study for SBA’s
                 Loan Monitoring System




       Suggested Actions

     • SBA should define processes associated with best
       practices and relate to SBA’s current processes.

     • In its benchmarking and reengineering, SBA should
       address activities delegated to lenders and develop
       monitoring processes concerning lender actions to
       • prevent delinquencies,
       • mitigate losses such as follow-up actions on delinquencies,
         and
       • liquidate defaulted loans.




26




                 Page 33                                     GAO/AIMD-99-165 SBA Benchmark Study
                  Appendix I
                  Briefing on the Benchmark Study for SBA’s
                  Loan Monitoring System




       Suggested Actions (continued)

     • SBA should collect measurement data to allow
       comparisons between SBA’s current processes and
       the processes of benchmark partners.
        • Measurement of current SBA operations can enable a clear
          demonstration of the benefits of adopting best practices and
          reengineering
        • Measurement of benchmark partners’ processes to
          compare and select among alternatives

     • SBA should set “stretch” goals for reengineering.
        • Goals for each loan monitoring activity
        • Goals to serve as objectives of reengineering
        • Goals based on measurement data

27




                  Page 34                                     GAO/AIMD-99-165 SBA Benchmark Study
                 Appendix I
                 Briefing on the Benchmark Study for SBA’s
                 Loan Monitoring System




       Suggested Actions (continued)

     • SBA should identify from benchmark partners
       potential outsourcing and candidate systems for
       purchase.
       • business activities
       • information processing services
       • commercial-off-the-shelf software




28




                 Page 35                                     GAO/AIMD-99-165 SBA Benchmark Study
                    Appendix I
                    Briefing on the Benchmark Study for SBA’s
                    Loan Monitoring System




       Agency Comments

     SBA agreed with our analysis of the benchmark study
     and plans to implement the suggested actions. SBA
     plans to
     • Review the benchmark project information to identify those
       organizations with best practices for critical loan monitoring
       processes and contact them to request a continuation of the
       benchmark process to collect measurement data.
     • Identify improvement goals for loan guaranty procedures and
       lender oversight processes.
     • Define and document processes of benchmark partners to
       assist in comparing and selecting new processes that best
       meet SBA’s business needs.


29




                    Page 36                                     GAO/AIMD-99-165 SBA Benchmark Study
                   Appendix I
                   Briefing on the Benchmark Study for SBA’s
                   Loan Monitoring System




       Agency Comments (continued)

     • Specifically address outsourcing as SBA performs
       quantitative analysis of the benchmark information. SBA
       plans to identify and evaluate information systems for
       purchase while completing the benchmark and business
       process reengineering.

     • Contact other government agencies to identify their lender
       oversight procedures for loan servicing, loss mitigation, and
       liquidations actions, and address these during business
       process reengineering efforts.




30




                   Page 37                                     GAO/AIMD-99-165 SBA Benchmark Study
Appendix II

Comments From the Small Business
Administration                                                ApIpexndi




              Page 38       GAO/AIMD-99-165 SBA Benchmark Study
Appendix II
Comments From the Small Business
Administration




Page 39                            GAO/AIMD-99-165 SBA Benchmark Study
Appendix II
Comments From the Small Business
Administration




Page 40                            GAO/AIMD-99-165 SBA Benchmark Study
Appendix II
Comments From the Small Business
Administration




Page 41                            GAO/AIMD-99-165 SBA Benchmark Study
Appendix II
Comments From the Small Business
Administration




Page 42                            GAO/AIMD-99-165 SBA Benchmark Study
Appendix II
Comments From the Small Business
Administration




Page 43                            GAO/AIMD-99-165 SBA Benchmark Study
Appendix II
Comments From the Small Business
Administration




Page 44                            GAO/AIMD-99-165 SBA Benchmark Study
Appendix II
Comments From the Small Business
Administration




Page 45                            GAO/AIMD-99-165 SBA Benchmark Study
Appendix II
Comments From the Small Business
Administration




Page 46                            GAO/AIMD-99-165 SBA Benchmark Study
Appendix II
Comments From the Small Business
Administration




Page 47                            GAO/AIMD-99-165 SBA Benchmark Study
Appendix III

Major Contributors to This Report                                                               AIpIexndi




Accounting and         E. Randolph Tekeley, Technical Assistant Director
                       Anh Q. Le, Senior ADP/Telecommunications Analyst
Information            John T. Christian, Senior Business Process Analyst
Management Division,
Washington, D.C.




(511471)       eL
                rtet   Page 48                                GAO/AIMD-99-165 SBA Benchmark Study
Ordering Information

The first copy of each GAO report and testimony is free.
Additional copies are $2 each. Orders should be sent to the
following address, accompanied by a check or money order made
out to the Superintendent of Documents, when necessary, VISA and
MasterCard credit cards are accepted, also.

Orders for 100 or more copies to be mailed to a single address are
discounted 25 percent.

Orders by mail:

U.S. General Accounting Office
P.O. Box 37050
Washington, DC 20013

or visit:

Room 1100
700 4th St. NW (corner of 4th and G Sts. NW)
U.S. General Accounting Office
Washington, DC

Orders may also be placed by calling (202) 512-6000
or by using fax number (202) 512-6061, or TDD (202) 512-2537.

Each day, GAO issues a list of newly available reports and
testimony. To receive facsimile copies of the daily list or any list
from the past 30 days, please call (202) 512-6000 using a touchtone
phone. A recorded menu will provide information on how to obtain
these lists.

For information on how to access GAO reports on the INTERNET,
send an e-mail message with “info” in the body to:

info@www.gao.gov

or visit GAO’s World Wide Web Home Page at:

http://www.gao.gov
United States                       Bulk Rate
General Accounting Office      Postage & Fees Paid
Washington, D.C. 20548-0001           GAO
                                 Permit No. GI00
Official Business
Penalty for Private Use $300

Address Correction Requested