oversight

Federal Research: DOD's Small Business Innovation Research Program

Published by the Government Accountability Office on 1997-04-30.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                  United States General Accounting Office

GAO               Report to Congressional Committees




April 1997
                  FEDERAL RESEARCH
                  DOD’s Small Business
                  Innovation Research
                  Program




GAO/RCED-97-122
                   United States
GAO                General Accounting Office
                   Washington, D.C. 20548

                   Resources, Community, and
                   Economic Development Division

                   B-275265

                   April 30, 1997

                   The Honorable Strom Thurmond
                   Chairman
                   The Honorable Carl Levin
                   Ranking Minority Member
                   Committee on Armed Services
                   United States Senate

                   The Honorable Floyd D. Spence
                   Chairman
                   The Honorable Ronald V. Dellums
                   Ranking Minority Member
                   Committee on National Security
                   House of Representatives

                   The Department of Defense (DOD) and certain other federal agencies are
                   required to set aside a certain percentage of their research and
                   development (R&D) budgets for the Small Business Innovation Research
                   (SBIR) Program. In fiscal year 1997, this will amount to about $1 billion
                   governmentwide, of which about $500 million is expected to be funded by
                   DOD.


                   The National Defense Authorization Act for Fiscal Year 1997 required GAO
                   to review certain aspects of DOD’s SBIR program and report the results to
                   the Congress.1 To meet these objectives and by agreement with your
                   offices, we reviewed DOD’s policies and procedures to determine if they
                   establish processes that help ensure that (1) quality research is performed
                   under the funding agreements, (2) competitive procedures are being
                   followed, and (3) technologies developed through the program are likely
                   to be used in military programs or projects.


                   DOD  has implemented policies and procedures and has established an
Results in Brief   organizational structure to manage its SBIR program. On the basis of these
                   factors and the results of DOD’s research under the program to date, we
                   believe that processes are in place to help ensure that quality research is
                   being conducted, that competitive procedures are being followed, and that
                   the resulting products and processes are being used in military projects
                   and programs.



                   1
                    P.L. 104-201, Sept. 23, 1996.



                   Page 1                   GAO/RCED-97-122 DOD’s Small Business Innovation Research Program
             B-275265




             The Small Business Innovation Development Act of 1982 was intended to
Background   (1) stimulate technological innovation, (2) use small businesses to meet
             federal R&D needs, (3) foster participation by minority and disadvantaged
             persons in technological innovation, and (4) increase private sector
             commercialization of innovations derived from federal R&D.2 The act
             required that agencies with “extramural” (external) R&D budgets of
             $100 million or more set aside a certain percentage of those budgets for
             the SBIR program. Specifically, the act required that these agencies set
             aside not less than 0.2 percent of their extramural R&D budgets and
             provided for annual increases up to a ceiling of not less than 1.25 percent
             of their budgets. The act provided for a three-phased program: Phase I to
             determine the scientific and technical merit and feasibility of a proposed
             research idea; phase II to further develop the idea, taking into
             consideration such things as the commercialization potential; and phase III
             to commercialize the resulting product or process with no further SBIR
             funding.

             The act gave the Small Business Administration (SBA) responsibility for
             (1) issuing guidance to the participating agencies on managing their SBIR
             programs, (2) surveying and monitoring the agencies’ programs, and
             (3) reporting at least annually to pertinent committees of the Congress on
             the status of the programs and the results of the monitoring. SBA’s policy
             directive to the participating agencies recommended that each phase-I and
             phase-II proposal be funded at $50,000 or less and $500,000 or less,
             respectively.

             The Small Business Research and Development Enhancement Act of 1992
             reauthorized the SBIR program through fiscal year 2000.3 The act
             emphasized the program’s goal of increasing private sector
             commercialization. The act provided for further incremental increases in
             SBIR funding up to not less than 2.5 percent of agencies’ extramural R&D
             budgets by fiscal year 1997. In addition, the act directed SBA to modify its
             policy directive to increase funding for phase-I and phase-II proposals to
             $100,000 and $750,000, respectively, with adjustments once every 5 years
             for inflation and changes in the program.

             Funding for DOD’s SBIR program since fiscal year 1992 is shown in figure 1.




             2
              P.L. 97-219, July 22, 1982.
             3
              P.L. 102-564, Oct. 28, 1992.



             Page 2                    GAO/RCED-97-122 DOD’s Small Business Innovation Research Program
                                               B-275265




Figure 1: DOD’s SBIR Expenditures,
Current-Year Dollars (i.e., Not Adjusted       600    Dollars in millions
for Inflation)
                                                                                                              520
                                               500
                                                                                     445     453


                                               400                 385

                                                                             336

                                               300
                                                       241

                                               200



                                               100



                                                 0
                                                        92



                                                                   93



                                                                              94



                                                                                      95



                                                                                              96




                                                                                                               )
                                                                                                              ate
                                                                                     19



                                                                                             19
                                                      19



                                                                   19



                                                                             19




                                                                                                         tim
                                                                                                        (es
                                                                                                   97
                                                                                                   19



                                                     Fiscal year



                                               Note: Percentage of extramural R&D budget—1992 (1.25), 1993-94 (1.5), 1995-96 (2.0), and
                                               1997 (2.5).



                                               Following SBA’s guidance, DOD manages its program through its SBIR
Ensuring Quality                               Program Manager in the Office of the Director, Small and Disadvantaged
Research                                       Business Utilization, which reports to the Under Secretary of Defense for
                                               Acquisition and Technology. There are eight participating military
                                               departments and defense agencies, referred to as “components.” The
                                               components include the Air Force; Army; Ballistic Missile Defense
                                               Organization; Defense Advanced Research Projects Agency; Office of the
                                               Director, Defense Research and Engineering; Defense Special Weapons
                                               Agency; Navy; and Special Operations Command. The components use the
                                               following criteria approved by the Under Secretary to develop their own
                                               prospective SBIR R&D solicitation topics. The topics are to

                                           •   solicit R&D, not procurement; fall within one of DOD’s key technology areas;
                                               allow the performing company significant flexibility in formulating its
                                               approach to meeting the R&D requirement; include examples of possible




                                               Page 3                       GAO/RCED-97-122 DOD’s Small Business Innovation Research Program
    B-275265




    phase III “dual-use” applications;4 and not duplicate each other by
    soliciting identical or very similar sets of proposals.

    The components use their own procedures for evaluating and ranking
    their topics and then submit them to the DOD SBIR Program Manager.

    DOD  has a formal process for reviewing the topics, resolving any areas of
    concern, and incorporating the approved topics into a DOD-wide
    solicitation for proposals from small businesses (see app. I). This process
    is done twice each year. The proposals are received and reviewed by the
    cognizant DOD components. Each component uses the following criteria by
    DOD for selecting research proposals. The reviewers are to consider the


•   soundness and technical merit of the proposed approach; potential for
    commercial (government or private sector) application and the expected
    benefits from commercialization; adequacy of the proposed effort for
    fulfilling requirements of the research topic; and qualifications of the
    proposed principal/key researchers, supporting staff, and consultants.
    Qualifications include the ability to perform the R&D and to commercialize
    the results.

    However, each component uses its own procedures for evaluating and
    ranking the proposals and for determining which proposals will be funded.
    The components provide oversight of the ongoing R&D.

    Measuring the value or quality of any R&D effort has always proven to be
    difficult. In a previous report, we used certain statistics to try to measure
    whether the quality of SBIR proposals governmentwide had kept pace with
    the program’s expansion.5 We looked at trends in the level of competition
    (the number of proposals submitted vs. the number funded) and the
    quality of competition (the number of proposals funded vs. the number
    deemed worthy of funding but not funded). These statistics indicated that
    (1) the level and quality of competition had kept pace with the increases in
    funding and (2) the level of research quality was being maintained. In
    reporting to the Congress on the quality of its SBIR research, DOD used the
    same statistics.6


    4
     Dual use means that the research product or process can be used in military projects or programs and
    can be used commercially.
    5
     Federal Research: Interim Report on the Small Business Innovation Research Program
    (GAO/RCED-95-59, Mar. 8, 1995).
    6
     Quality of Research Under the DOD Small Business Innovation Research (SBIR) Program, Office of
    the Under Secretary of Defense (Acquisition & Technology), May 30, 1996.



    Page 4                 GAO/RCED-97-122 DOD’s Small Business Innovation Research Program
                                  B-275265




                                  For this review, we updated these statistics through fiscal year 1996. The
                                  percentage of phase-I proposals that were funded (fig. 2) and the ratio of
                                  phase-I proposals that were funded to the worthy proposals not funded
                                  (fig. 3) have remained fairly constant.


Figure 2: Percentage of Phase-I
Proposals Funded                  16   Percent
                                                                     14.3
                                  14                                         13.6

                                                   12.5
                                       11.9                  11.8
                                  12


                                  10


                                   8


                                   6


                                   4


                                   2


                                   0

                                           1992      1993     1994    1995    1996
                                           Fiscal year



                                  Note: Percentage of extramural R&D budget—1992 (1.25), 1993-94 (1.5), and 1995-96 (2.0).




                                  Page 5                    GAO/RCED-97-122 DOD’s Small Business Innovation Research Program
                                         B-275265




Figure 3: Phase-I Proposals Deemed
Worthy of Funding—Funded Vs. Not         30   Percent
Funded
                                                                                                  26.0
                                                                                  25.4
                                         25

                                                       21.7                                                       21.4

                                         20



                                         15                                              14.3
                                                                                                         13.6
                                              12.5
                                                                        11.8

                                         10



                                          5



                                          0

                                                  1993                     1994            1995            1996
                                                  Fiscal year



                                                              Funded

                                                              Not funded




                                         As additional potential measures of research quality, we also obtained
                                         available statistics on the percentage of phase-I proposals that went on to
                                         phase II (see app. II) and on the extent of commercialization of the results
                                         of the research. On the basis of the preliminary results of a survey by DOD,
                                         we determined that

                                     •   about one-half of the respondents have had sales of a product, process, or
                                         service; average sales amount per project has been about $756,000; and
                                         about 52 percent of the total sales have been made to DOD or its prime
                                         contractors, about 35 percent to the private sector, and the remaining
                                         percent to other customers.

                                         These data suggest that quality projects are being funded, that there is
                                         adequate competition for the projects, and that the research results are
                                         being used in military projects and programs.




                                         Page 6                        GAO/RCED-97-122 DOD’s Small Business Innovation Research Program
                      B-275265




                      DOD has implemented procedures to help ensure that competition for
Ensuring Full, Open   research funding under the SBIR program is fair. The solicitations for
Competition           research proposals are published widely through trade publications,
                      regularly established information outlets for DOD’s contracting, and the
                      Internet. The solicitations are generally open to any individual or
                      organization that qualifies as a small business. The solicitations specify the
                      requirements that small businesses must meet to be eligible to apply.
                      Furthermore, in their proposals, the small businesses must certify under
                      penalty of law that they meet all of the requirements. The DOD Inspector
                      General conducts investigations, as warranted, to ensure compliance with
                      these requirements.

                      To promote more competition for its research projects, DOD conducts
                      three national SBIR conferences each year and participates in many
                      state-organized conferences. These conferences are used to provide small
                      businesses with information on R&D opportunities available to them and on
                      how to submit research proposals or how to obtain technical assistance in
                      developing proposals. Technical assistance is available from the Defense
                      Technology Information Center through the mail, walk-in offices, and the
                      Internet. DOD also has a prerelease program, whereby small businesses
                      learn about upcoming research solicitations in advance so that they can
                      contact the appropriate DOD component to get specific information.
                      According to DOD, this results in higher-quality proposals. To better gauge
                      the extent of competition, DOD’s solicitations now require small businesses
                      to provide information on their prior SBIR proposals that have been funded.

                      The DOD components have their own programs to encourage more
                      competition. The components issue brochures and advertisements in trade
                      journals, informing small businesses how to get detailed information on
                      R&D opportunities and how to apply for them.



                      Several factors help ensure that the research efforts under the SBIR
Ensuring Military     program have military applicability. The research topics are developed by
Applicability         the components’ technical organizations to support their missions, all of
                      which are military-related. DOD’s guidance requires that the research topics
                      address one of the Department’s key technology areas and that they have
                      dual-use application.7 There is required linkage between the topics and
                      military planning documents. Furthermore, the preliminary results of DOD’s
                      recent survey on commercialization show that about one-half of the sales

                      7
                       For example, a primary success story of DOD’s SBIR program is the “SaviTag” system of tracking
                      shipping containers. Combined military and commercial sales of this system were anticipated to be
                      $30 million in 1996.



                      Page 7                 GAO/RCED-97-122 DOD’s Small Business Innovation Research Program
                  B-275265




                  from the program’s research efforts have been made to DOD or its prime
                  contractors for use in military programs.


                  We provided copies of a draft of this report to DOD for review and
Agency Comments   comment. We discussed the draft with DOD officials, including the SBIR
                  Program Manager, who said that the Department concurs with the facts in
                  the report. The manager also provided information to update certain
                  statistics. That information has been incorporated into the final report.


                  We interviewed officials in SBA, DOD, and the participating DOD
Scope and         components. We obtained and reviewed implementing instructions,
Methodology       guidelines, and other pertinent documents. However, we did not evaluate
                  the effectiveness or implementation of DOD’s policies or procedures. Our
                  verification was limited to reviewing statistics on the number of research
                  proposals funded; the ratio of funded proposals to worthy but unfunded
                  proposals; the number of phase-I proposals that go on to phase II; and the
                  extent of commercialization of the research results. Our planned
                  work—mandated by the Small Business Research and Development
                  Enhancement Act of 1992—will examine in greater detail the management
                  and implementation of the SBIR programs of all participating federal
                  agencies, including DOD. We performed our review from January through
                  March 1997 in accordance with generally accepted government auditing
                  standards.


                  We are sending copies of this report to the Secretary of Defense and the
                  SBA Administrator. We will make copies available to others on request. If
                  you have any questions or need additional information, please call me at
                  (202) 512-3841. Major contributors to this report are listed in appendix III.




                  Victor S. Rezendes
                  Director, Energy, Resources,
                    and Science Issues




                  Page 8            GAO/RCED-97-122 DOD’s Small Business Innovation Research Program
Page 9   GAO/RCED-97-122 DOD’s Small Business Innovation Research Program
Appendix I

The Department of Defense’s Topic
Identification and Approval Process



                  comment and clarification                                   unresolved




  Component    Component              Component                DOD                          Final              Final
    Topic        Topic                  Topic                 Review                       Review             Decision
 Development   Evaluation              Ranking

                                                                     approve
                        resubmittal




                                                                                                                       Topic
                                                               Solicitation
                                                                                                                      Rejection




                                        Note: The Department of Defense’s (DOD) review is performed by the Office of the Director,
                                        Defense Research and Engineering, which interacts with the components to resolve any
                                        concerns. If the concerns cannot be resolved, the topic is submitted to the Integrated Review
                                        Committee for final review (this is a 1-year pilot program). The Committee makes its
                                        recommendation to the Director, Office of Small and Disadvantaged Business Utilization. The
                                        Director makes the final decision in consultation with the Director, Defense Research and
                                        Engineering. The review process (from DOD’s review to topic approval) takes about 6 weeks.




                                        Page 10                GAO/RCED-97-122 DOD’s Small Business Innovation Research Program
Appendix II

Phase-I Proposals That Went on to Phase II,
Fiscal Years 1983 Through 1996


                                                                                        Percent of
                                                                                phase-I proposals
                                            Phase-I proposals          Phase-II   that went on to
                                                       funded proposals funded            phase II
               Army                                       2,681                 1,041                     38.8
               Air Force                                  3,865                 1,448                     37.5
               Director, Defense                              59                   22                     37.3
               Research and Engineering
               Navy                                       3,190                 1,005                     31.5
               Defense Special Weapons                      248                    70                     28.2
               Agency
               Defense Advanced                           1,360                   382                     28.1
               Research
               Projects Agency
               Ballistic Missile Defense                  1,555                   418                     26.9
               Organization
               Special Operations                             30                     7                    23.3
               Command
               Totals                                    12,988                 4,393                     33.8
               Note: The Special Operations Command and the Director, Defense Research and Engineering,
               have been participating in the program since fiscal years 1993 and 1995, respectively.




               Page 11              GAO/RCED-97-122 DOD’s Small Business Innovation Research Program
Appendix III

Major Contributors to This Report


                        Allen Li
Resources,              Robin M. Nazzaro
Community, and          Katherine L. Hale
Economic                Michael F. Duffy
                        John C. Johnson
Development
Division, Washington,
D.C.
                        Mindi G. Weisenbloom
Office of the General
Counsel




(141014)                Page 12         GAO/RCED-97-122 DOD’s Small Business Innovation Research Program
Ordering Information

The first copy of each GAO report and testimony is free.
Additional copies are $2 each. Orders should be sent to the
following address, accompanied by a check or money order
made out to the Superintendent of Documents, when
necessary. VISA and MasterCard credit cards are accepted, also.
Orders for 100 or more copies to be mailed to a single address
are discounted 25 percent.

Orders by mail:

U.S. General Accounting Office
P.O. Box 6015
Gaithersburg, MD 20884-6015

or visit:

Room 1100
700 4th St. NW (corner of 4th and G Sts. NW)
U.S. General Accounting Office
Washington, DC

Orders may also be placed by calling (202) 512-6000
or by using fax number (301) 258-4066, or TDD (301) 413-0006.

Each day, GAO issues a list of newly available reports and
testimony. To receive facsimile copies of the daily list or any
list from the past 30 days, please call (202) 512-6000 using a
touchtone phone. A recorded menu will provide information on
how to obtain these lists.

For information on how to access GAO reports on the INTERNET,
send an e-mail message with "info" in the body to:

info@www.gao.gov

or visit GAO’s World Wide Web Home Page at:

http://www.gao.gov




PRINTED ON    RECYCLED PAPER
United States                       Bulk Rate
General Accounting Office      Postage & Fees Paid
Washington, D.C. 20548-0001           GAO
                                 Permit No. G100
Official Business
Penalty for Private Use $300

Address Correction Requested