oversight

Women-Owned Small Business Program: Actions Needed to Address Ongoing Oversight Issues

Published by the Government Accountability Office on 2019-04-15.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

             United States Government Accountability Office
             Report to Congressional Requesters




             WOMEN-OWNED
March 2019




             SMALL BUSINESS
             PROGRAM

             Actions Needed to
             Address Ongoing
             Oversight Issues




GAO-19-168
                                             March 2019

                                             WOMEN-OWNED SMALL BUSINESS PROGRAM
                                             Actions Needed to Address Ongoing Oversight
                                             Issues
Highlights of GAO-19-168, a report to
congressional requesters




Why GAO Did This Study                       What GAO Found
In 2000, Congress authorized the             The Small Business Administration (SBA) has implemented one of the three
WOSB program, allowing contracting           changes to the Women-Owned Small Business (WOSB) program authorized in
officers to set aside procurements to        the National Defense Authorization Act of 2015 (2015 NDAA). Specifically, in
women-owned small businesses in              September 2015 SBA published a final rule to implement sole-source authority,
industries in which they are                 effective October 2015. As of February 2019, SBA had not eliminated the option
substantially underrepresented. To be        for program participants to self-certify that they are eligible to participate, as
eligible to participate in the WOSB          required by 2015 NDAA. SBA officials stated that this requirement would be
program, firms have the option to self-      addressed as part of the new certification process for the WOSB program, which
certify or be certified by a third-party
                                             they expect to implement by January 1, 2020.
certifier. However, the 2015 NDAA
changed the WOSB program by (1)              SBA has not addressed WOSB program oversight deficiencies identified in
authorizing SBA to implement sole-           GAO’s 2014 review (GAO-15-54). For example, GAO previously recommended
source authority, (2) eliminating the        that SBA establish procedures to assess the performance of four third-party
option for firms to self-certify as being    certifiers—private entities approved by SBA to certify the eligibility of WOSB
eligible for the program and (3)             firms. While SBA conducted a compliance review of the certifiers in 2016, it has
allowing SBA to implement a new              no plans to regularly monitor them. By not improving its oversight of the WOSB
certification process.                       program, SBA is limiting its ability to ensure third-party certifiers are following
GAO was asked to review the WOSB             program requirements. In addition, the implementation of sole-source authority in
program. This report discusses (1) the       light of these continued oversight deficiencies can increase program risk.
extent to which SBA has addressed            Consequently, GAO maintains that its prior recommendations should be
the 2015 NDAA changes, (2) SBA’s             addressed. In addition, similar to previous findings from SBA’s Office of Inspector
efforts to address previously identified     General, GAO found that about 3.5 percent of contracts using a WOSB set-aside
deficiencies, and (3) use of the WOSB        were awarded for ineligible goods or services from April 2011 through June
program. GAO reviewed relevant               2018. SBA does not review contracting data that could identify this problem and
laws, regulations, and program               inform SBA which agencies making awards may need targeted outreach or
documents; analyzed federal                  training. As a result, SBA cannot provide reasonable assurance that WOSB
contracting data from April 2011             program requirements are being met and that the program is meeting its goals.
through June 2018; and interviewed
SBA officials, officials from contracting    While federal contract obligations to all women-owned small businesses and
agencies selected to obtain a range of       WOSB program set-asides have increased since fiscal year 2012, WOSB
experience with the WOSB program,            program set-asides remain a small percentage (see figure).
and three of the four private third-party
certifiers.                                  Obligations for the Women-Owned Small Business Program and to All Women-Owned Small
                                             Businesses in Similar Industries, Fiscal Years 2012–2017
What GAO Recommends
GAO recommends that SBA develop a
process for periodically reviewing the
extent to which WOSB program set-
asides are awarded for ineligible goods
or services and use the results to
address identified issues, such as
through targeted outreach or training
on the WOSB program. SBA agreed
with the recommendation.

View GAO-19-168. For more information,       Note: Obligations to women-owned small businesses represent contract obligations to women-owned
contact William Shear at (202) 512-8678 or   small businesses under WOSB-program-eligible North American Industry Classification System
ShearW@gao.gov.                              codes. FPDS-NG obligation amounts have been adjusted for inflation.

                                                                                           United States Government Accountability Office
Contents


Letter                                                                                 1
               Background                                                              5
               SBA Has Implemented One of the Three Changes Made by the
                 2015 NDAA                                                           10
               SBA Has Not Fully Addressed Deficiencies in Oversight and
                 Program Implementation                                              14
               Federal Contracts to WOSB Set-Asides Remain Relatively Small,
                 and Stakeholders Discussed Various Aspects of Program Use           26
               Conclusions                                                           35
               Recommendation for Executive Action                                   36
               Agency Comments                                                       36

Appendix I     Objectives, Scope, and Methodology                                    38



Appendix II    Comments from the U.S. Small Business Administration                  43



Appendix III   GAO Contact and Staff Acknowledgments                                 46


Table
               Table 1: Top Contracting Departments and Agencies under the
                       WOSB Program, from Third Quarter of Fiscal Year 2011
                       through Third Quarter of Fiscal Year 2018                     29

Figures
               Figure 1: Obligations for the Women-Owned Small Business
                        Program and to All Women-Owned Small Businesses in
                        Similar Industries, Fiscal Years 2012–2017                   27
               Figure 2: Percentage of Contract Obligations to Women-Owned
                        Small Businesses by Set-Aside Type in Similar
                        Industries, Fiscal Years 2012–2017                           28
               Figure 3: Sole-Source Obligations as a Percentage of All WOSB
                        Program Obligations, January 2016–June 2018                  30




               Page i                       GAO-19-168 Women-Owned Small Business Program
Abbreviations

8(a)                       8(a) Business Development Program
2015 NDAA                  National Defense Authorization Act of 2015
DHS                        Department of Homeland Security
DOD                        Department of Defense
EDWOSB                     economically disadvantaged women-owned small
                           business
FAR                        Federal Acquisition Regulation
FPDS-NG                    Federal Procurement Data System-Next
                           Generation
GSA                        General Services Administration
HUBZone                    Historically Underutilized Business Zone
NAICS                      North American Industry Classification System
OIG                        Office of Inspector General
SBA                        Small Business Administration
SDVOSB                     Service-Disabled Veteran-Owned Small Business
                           Program
SOP                        Standard Operating Procedures
WOSB                       women-owned small business
WOSB program               Women-Owned Small Business Federal
                           Contracting Program


This is a work of the U.S. government and is not subject to copyright protection in the
United States. The published product may be reproduced and distributed in its entirety
without further permission from GAO. However, because this work may contain
copyrighted images or other material, permission from the copyright holder may be
necessary if you wish to reproduce this material separately.




Page ii                                GAO-19-168 Women-Owned Small Business Program
                       Letter




441 G St. N.W.
Washington, DC 20548




                       March 14, 2019

                       The Honorable Nydia Velázquez
                       Chairwoman
                       The Honorable Steve Chabot
                       Ranking Member
                       Committee on Small Business
                       House of Representatives

                       The Honorable Alma Adams
                       House of Representatives

                       Women-owned businesses play an increasingly significant role in the U.S.
                       economy. According to the most recent U.S. Census Survey of Business
                       Owners, the number of women-owned businesses grew by almost 27
                       percent from 2007 through 2012. 1 However, the federal goal of awarding
                       5 percent of procurements government-wide to women-owned small
                       businesses has only been met once since fiscal year 2013. 2 In 2000,
                       Congress authorized the Women-Owned Small Business Federal
                       Contracting Program (WOSB program) to, among other things, reserve
                       contracts for qualified women-owned small businesses in industries




                       1
                        In 2012, there were 9.9 million women-owned firms, an increase of more than 2 million
                       (or 26.9 percent) since 2007, according to the U.S. Census Survey of Business Owners
                       2007 and 2012.
                       2
                        For women-owned small businesses, the prime contracting goal for each federal agency
                       is 5 percent of the agency’s prime contracting dollars awarded. A prime contract is
                       awarded directly to a contractor by the federal government. According to the Small
                       Business Administration’s 2017 Small Business Procurement scorecard, in fiscal years
                       2013 through 2017 the federal government’s women-owned small business prime
                       contracting goal of 5 percent was met only in fiscal year 2015. See Small Business
                       Administration, “The Federal Government Achieves Small Business Contracting Goal for
                       the Fifth Consecutive Year with Record-Breaking $105 Billion to Small Businesses,” press
                       release no. 18-39, May 22, 2018, https://www.sba.gov/about-sba/sba-newsroom/press-
                       releases-media-advisories/federal-government-achieves-small-business-contracting-goal-
                       fifth-consecutive-year-record-breaking. Also see GAO, Small Business Administration:
                       Actions Needed to Improve Confidence in Small Business Procurement Scorecard, GAO-
                       18-672 (Washington, D.C.: Sept. 27, 2018).




                       Page 1                                GAO-19-168 Women-Owned Small Business Program
where these businesses are substantially underrepresented. 3 On October
7, 2010, the Small Business Administration (SBA) issued a final rule to
implement the WOSB program, and the program began operating in
2011.

SBA allows businesses to establish program eligibility through self-
certification or certification by third-party certifiers. 4 In 2014, we reviewed
the WOSB program and found a number of deficiencies in SBA’s
oversight of the four SBA-approved third-party certifiers and in SBA’s
eligibility examination processes. 5 In addition, in 2015 and 2018 the SBA
Office of Inspector General (OIG) reviewed the WOSB program and also
found several oversight deficiencies. 6


3
 See Consolidated Appropriation Act, 2001, Pub. L. No. 106-554, §811, 114 Stat. 2763,
2763A708 (2000).The Small Business Act defines various socioeconomic categories of
businesses, and the Small Business Administration administers contracting programs
targeted to these groups. Small businesses are those that are independently owned and
operated and are not dominant in their field of operations. 15 U.S.C. § 632(a). Women-
owned small businesses must have at least 51 percent women ownership and must be
controlled by one or more women who are U.S. citizens. The ownership must be direct
and not subject to conditions, and the woman must manage the day-to-day operations of
the business and make its long-term decisions. See 15 U.S.C. § 632(n); 13 C.F.R. §
127.102; 13 C.F.R. § 127.201; 13 C.F.R. § 127.202.
4
 WOSB program provisions were formally added to the Federal Acquisition Regulation on
April 1, 2011. Program regulations allow third-party eligibility certification by SBA-
approved private entities; by states through the Department of Transportation’s
Disadvantaged Business Enterprise program; or through SBA’s 8(a) Business
Development program. See 13 C.F.R. § 127.300(d)(1). According to SBA officials, the
Disadvantaged Business Enterprise program is not used to certify eligible WOSB firms,
and certifications through the 8(a) Business Development program represent a small
percentage of all WOSB program certifications. SBA officials could not provide data on
how many WOSB program firms are certified through self-certification or through the 8(a)
Business Development program.
5
 GAO, Women-Owned Small Business Program: Certifier Oversight and Additional
Eligibility Controls Are Needed, GAO-15-54 (Washington, D.C.: Oct. 8, 2014). We made
two recommendations to improve SBA’s oversight of the WOSB program. As of February
2019, these recommendations remained open.
6
 SBA Office of Inspector General, Improvements Needed in SBA’s Management of the
Women Owned Small Business Federal Contracting Program, SBA OIG Report 15-10
(Washington, D.C.: May 14, 2015). The report’s objectives were to determine whether (1)
WOSB program awards complied with the program’s set-aside requirements and (2) firms
that received WOSB set-aside awards conformed to the program’s self-certification
requirements. SBA Office of Inspector General, SBA’s Women-Owned Small Business
Contracting Program, SBA OIG Report 18-18 (Washington, D.C.: June 20, 2018). The
report’s objectives were to determine whether (1) contracts awarded on a sole-source
basis complied with requirements of the WOSB program and (2) firms that received
contracts on a sole-source basis conformed to the self-certification requirements.




Page 2                                GAO-19-168 Women-Owned Small Business Program
In December 2014, the National Defense Authorization Act of 2015 (2015
NDAA) amended the WOSB program by, among other things, granting
contracting officers the authority to award sole-source contracts to eligible
women-owned small businesses (WOSB) and economically
disadvantaged women-owned small businesses (EDWOSB). 7 The act
also removed firms’ ability to self-certify as eligible for the WOSB program
and allowed SBA to certify the eligibility of program participants as an
additional alternative to third-party certification. 8

You requested that we evaluate SBA’s WOSB program, including any
actions SBA has taken in response to the 2015 NDAA and to previously
identified deficiencies, as well as usage of the program. This report
examines (1) the extent to which SBA has implemented changes to the
WOSB program made by the 2015 NDAA; (2) the extent to which SBA
has implemented changes to address previously identified oversight
deficiencies; and (3) changes in WOSB program use since 2011 and
stakeholder views on its use, including since the 2015 implementation of
sole-source authority.

To address the first objective, we reviewed the 2015 NDAA, proposed
regulations, and SBA documentation, and we interviewed SBA officials.
To respond to the second and third objectives, we interviewed SBA
officials, three of the WOSB program’s four private third-party certifiers
(referred to in the report as “third-party certifiers”), three selected federal
agencies (and three agency components within two of the agencies), and
a total of eight selected contracting offices within the six selected
agencies or components. 9 Using data from the Federal Procurement Data
System-Next Generation (FPDS-NG), we judgmentally selected the three
federal agencies and three components (for a total of six selected
agencies and components) because their WOSB dollar obligations
(including competed and sole-source) were among the largest or because

7
 Throughout this report, we use the abbreviation “WOSB” to refer specifically to women-
owned small businesses that have been certified as eligible for the WOSB program.
8
 The Carl Levin and Howard P. ‘Buck’ McKeon National Defense Authorization Act for
Fiscal Year 2015, Pub. L. No. 113-291, § 825, 128 Stat. 3292, 3437 (2014).
9
 The four SBA-approved private third-party certifiers under the WOSB program are the El
Paso Hispanic Chamber of Commerce, NWBOC (previously known as the National
Women Business Owners Corporation), U.S. Women’s Chamber of Commerce, and the
Women’s Business Enterprise National Council. We did not speak with NWBOC, as
NWBOC officials did not make themselves available for an interview with us within the
time period necessary for our review.




Page 3                                GAO-19-168 Women-Owned Small Business Program
we had interviewed them for our prior work. 10 Specifically, we selected the
following six agencies or agency components: the Department of
Homeland Security (DHS) and, within DHS, the Coast Guard; the
Department of Defense (DOD) and, within DOD, the U.S. Army and U.S.
Navy; and the General Services Administration (GSA). 11 Within the
components and GSA, we judgmentally selected eight contracting offices
(two each from Coast Guard, U.S. Army, U.S. Navy, and GSA) based on
whether they had a relatively large amount of obligations and had used
multiple types of set-asides (competed or sole-source) to WOSBs or
EDWOSBs.

To address the second objective, we reviewed SBA policies and
procedures for the WOSB program and SBA oversight activities, such as
SBA compliance reviews of the four third-party certifiers. We analyzed
FPDS-NG data on obligations for WOSB program set-aside contracts and
the goods and services for which they were awarded—represented by
North American Industry Classification System (NAICS) codes—against
NAICS codes approved for WOSB program eligibility to determine if these
contracts were made in eligible industries. We compared SBA’s activities
against federal internal control standards, GAO’s framework for managing
fraud risk, and our prior recommendations to SBA. 12 We also reviewed
relevant SBA OIG reports and interviewed SBA OIG officials.

To address the third objective, we analyzed FPDS-NG data from April
2011 through June 2018 (the most recent data available during the period
of our analysis) and identified trends in total obligation amounts of
competed and sole-source contracts awarded under the WOSB program
(including both WOSB and EDWOSB contracts). To determine the
relative usage of the WOSB program, we compared data on obligations

10
  FPDS-NG contains data for most federal contract actions that have an estimated value
over the micropurchase threshold (which is currently $3,500). The federal government
uses its data to create recurring and special reports to the President, the Congress,
federal executive agencies, and the general public.
11
  We spoke with officials from DHS, DOD, and GSA for this review and for our 2014
review. We selected DHS and DOD because they had the largest WOSB program
obligations. Although GSA’s WOSB program obligations were not the largest after DHS
and DOD, we selected GSA to have some continuity between the reviews and because it
maintains FPDS-NG.
12
   GAO, Standards for Internal Control in the Federal Government, GAO-14-704G
(Washington, D.C.: September 2014); A Framework for Managing Fraud Risks in Federal
Programs, GAO-15-593SP (Washington, D.C.: July 2015); and GAO-15-54.




Page 4                               GAO-19-168 Women-Owned Small Business Program
             for set-asides under the WOSB program with federal contract obligations
             for WOSB-program-eligible goods and services to all woman-owned small
             businesses. We assessed the reliability of FPDS-NG data by reviewing
             available documentation and prior GAO data reliability assessments and
             by electronically testing for missing data, outliers, and inconsistent
             coding. 13 We found the data to be reliable for the purposes of reporting on
             trends in the WOSB program and the usage of sole-source authority
             under the program. Appendix I provides more detail on our scope and
             methodology.

             We conducted this performance audit from October 2017 to March 2019
             in accordance with generally accepted government auditing standards.
             Those standards require that we plan and perform the audit to obtain
             sufficient, appropriate evidence to provide a reasonable basis for our
             findings and conclusions based on our audit objectives. We believe that
             the evidence obtained provides a reasonable basis for our findings and
             conclusions based on our audit objectives.


             Federal agencies conduct a variety of procurements that are reserved for
Background   small business participation through small business set-asides. The set-
             asides can be for small businesses in general, or they can be specific to
             small businesses that meet additional eligibility requirements in the
             Service-Disabled Veteran-Owned Small Business (SDVOSB), Historically
             Underutilized Business Zone (HUBZone), 8(a) Business Development
             (8(a)), and WOSB programs.

             The WOSB program enables federal contracting officers to identify and
             establish a sheltered market, or set-aside, for competition among WOSBs




             13
               We pulled data from FPDS-NG on July 25, 2018, for analysis for our third objective, as
             well as for our analysis of NAICS codes for the second objective. FPDS-NG data are
             entered by agency contracting officers on a real-time basis and, thus, they change every
             day. FPDS-NG data undergo a certification process each January for the prior fiscal year.
             The data we included in our analysis from three quarters of fiscal year 2018 had not been
             certified at the time of our analysis. However, based on the data quality summary reports
             from prior years, we believe these data to be reliable for the purposes of our reporting
             objectives.




             Page 5                                GAO-19-168 Women-Owned Small Business Program
and EDWOSBs in certain industries. 14 To determine the industries eligible
under the WOSB program, SBA is required to conduct a study to
determine which NAICS codes are eligible under the program and to
report on such studies every 5 years. WOSBs can receive set-asides in
industries in which SBA has determined that women-owned small
businesses are substantially underrepresented. EDWOSBs can receive
set-asides in WOSB-eligible industries as well as in an additional set of
industries in which SBA has determined that women-owned small
businesses are underrepresented but not substantially so. 15 As of
February 2019, there were a total of 113 four-digit NAICS codes
(representing NAICS industry groups) eligible under the WOSB
program—92 eligible NAICS codes for WOSBs and 21 for EDWOSBs. 16

Additionally, businesses must be at least 51 percent owned and
controlled by one or more women who are U.S. citizens to participate in
the WOSB program. The owner must provide documents demonstrating

14
   Economically disadvantaged is a determination made if a woman can demonstrate that
her ability to compete in the free enterprise system is impaired due to diminished capital
and credit opportunities as compared with others in the same or similar business. In order
to be economically disadvantaged, a woman must have a net worth of less than $750,000
(with certain regulatory exclusions). Generally, a woman with an adjusted gross income
for the prior 3 years averaging $350,000 or more or with assets of $6 million or more is not
deemed economically disadvantaged. Additionally, a spouse’s financial situation can be
considered. See 13 C.F.R. § 127.203.
15
   WOSBs are not eligible to receive awards within industries solely designated for
EDWOSBs.
16
   NAICS uses a six-digit coding system to identify particular industries and their placement
in the hierarchical structure of the classification system. A four-digit code designates the
sector, subsector, and industry group. Additional digits (the fifth and sixth) provide
additional information on the NAICS industry and the national industry. WOSB program
industry eligibility determinations were made at the four-digit NAICS industry group level.
Specifically, two studies were conducted to assist SBA in determining eligible NAICS
codes. SBA commissioned the RAND Corporation to conduct the first such study. Based
on the results of the RAND study, SBA designated 45 four-digit WOSB NAICS codes and
38 EDWOSB NAICS codes, for a total of 83. See Kauffman-RAND Institute for
Entrepreneurship Public Policy, The Utilization of Women-Owned Small Businesses in
Federal Contracting (Santa Monica, Calif.: 2007). In December 2015, the Department of
Commerce issued the next study, based on which SBA increased the total number of
NAICS codes under the program to 113, with 92 WOSB codes and 21 EDWOSB codes
(which became effective March 2016). See David N. Beede and Robert N. Rubinovitz,
Utilization of Women-Owned Small Businesses in Federal Prime Contracting, a report
prepared for the Small Business Administration (Washington, D.C.: Economics and
Statistics Administration, Department of Commerce, Dec. 31, 2015). Other more technical
changes were made to the WOSB program NAICS codes in 2012 and 2017 to align them
with broader changes in NAICS codes.




Page 6                                  GAO-19-168 Women-Owned Small Business Program
that the business meets program requirements, including a document in
which the owner attests to the business’s status as a WOSB or
EDWOSB. EDWOSBs are WOSBs that are controlled by one or more
women who are citizens and who are economically disadvantaged in
accordance with SBA regulations. According to SBA, as of early October
2018, there were 13,224 WOSBs and 4,488 EDWOSBs registered in
SBA’s online certification database.

SBA’s Office of Government Contracting administers the WOSB program
by promulgating regulations, conducting eligibility examinations of
businesses that receive contracts under a WOSB or EDWOSB set-aside,
deciding protests related to eligibility for a WOSB set-aside, conducting
studies to determine eligible industries, and working with other federal
agencies in assisting WOSBs and EDWOSBs. According to SBA officials,
the Office of Government Contracting also works at the regional and local
levels with SBA’s Small Business Development Centers and district
offices, and with other organizations (such as Procurement Technical
Assistance Centers), to help WOSBs and EDWOSBs obtain contracts
with federal agencies. The services SBA coordinates include training,
counseling, mentoring, facilitating access to information about federal
contracting opportunities, and business financing. According to SBA, as
of October 2018, there were two full-time staff within the Office of
Government Contracting whose primary responsibility was the WOSB
program.

Initially, the program’s statutory authority allowed WOSBs to be self-
certified by the business owner or certified by an approved third-party
national certifying entity as eligible for the program. Self-certification is
free, but some third-party certification options require businesses to pay a
fee. Each certification process requires businesses to provide signed
representations attesting to their WOSB or EDWOSB eligibility.
Businesses must provide documents supporting their status before
submitting an offer to perform the requirements of a WOSB set-aside




Page 7                           GAO-19-168 Women-Owned Small Business Program
contract. 17 In August 2016, SBA launched certify.sba.gov, which is an
online portal that allows firms to upload required documents and track
their submission and also enables contracting officers to review firms’
eligibility documentation. 18 According to the Federal Acquisition
Regulation (FAR), contracting officers are required to verify that all
required documentation is present in the online portal when selecting a
business for an award. In addition, businesses must register and attest to
being a WOSB in the System for Award Management, the primary
database of vendors doing business with the federal government. 19

In 2011, SBA approved four organizations to act as third-party certifiers:

•    El Paso Hispanic Chamber of Commerce,
•    NWBOC (previously known as the National Women Business Owners
     Corporation),
•    U.S. Women’s Chamber of Commerce, and
•    Women’s Business Enterprise National Council.
These organizations have been the WOSB program’s third-party certifiers
since 2011. 20 According to SBA data, the Women’s Business Enterprise
National Council was the most active third-party certifier in fiscal year
2017—performing 2,638 WOSB certification examinations. The U.S.

17
  According to WOSB program regulations at 13 C.F.R. § 127.300(e), documents required
of self-certified WOSBs include copies of birth certificates, naturalization papers, or
unexpired passports; joint venture agreement, if applicable; for a sole proprietorship, an
assumed/fictitious name certificate; signed WOSB program certification (an attestation by
the business of its program eligibility); if a limited liability company, articles of organization
and operating agreement; if a corporation, articles of incorporation, bylaws, and any
amendments, and all issued stock certificates, stock ledger, and voting agreements, if
any; and if a partnership, partnership agreements and amendments. Self-certified
EDWOSBs also must provide a personal financial statement for each woman claiming
economic disadvantage (SBA Form 413) and a signed EDWOSB program certification.
18
  According to SBA, the agency plans to move other certifications, such as those of the
8(a) and HUBZone programs, into the new system and allow documents uploaded for one
program to be reused for another.
19
   Federal acquisition regulations generally require prospective vendors to be registered in
the System for Award Management—which is maintained by GSA—before the award of a
contract, basic agreement, basic ordering agreement, or blanket purchase agreement.
20
  To become an SBA-approved third-party certifier, interested organizations submitted an
application to SBA that contained information on the organization’s structure and staff,
policies and procedures for certification, and attestations that they will adhere to program
requirements.




Page 8                                    GAO-19-168 Women-Owned Small Business Program
Women’s Chamber of Commerce, NWBOC, and El Paso Hispanic
Chamber of Commerce—completed 644, 105, and 12 certifications,
respectively.

As discussed previously, in 2014 we reviewed the WOSB program and
found a number of deficiencies in SBA’s oversight of the four SBA-
approved third-party certifiers and in SBA’s eligibility examination
processes and we made related recommendations for SBA. 21 In addition,
in 2015 and 2018 the SBA OIG reviewed the WOSB program and also
found oversight deficiencies, including evidence of WOSB contracts set
aside for ineligible firms. In both reports, the SBA OIG also made
recommendations for SBA. 22 Further, in July 2015, we issued GAO’s
fraud risk framework, which provides a comprehensive set of key
components and leading practices that serve as a guide for agency
managers to use when developing efforts to combat fraud in a strategic,
risk-based way. 23 In July 2016, the Office of Management and Budget
issued guidelines requiring executive agencies to create controls to
identify and respond to fraud risks. These guidelines also affirm that
managers should adhere to the leading practices identified in GAO’s
fraud risk framework. 24




21
   GAO-15-54. We made two recommendations to improve SBA’s oversight of the WOSB
program. As of February 2019, these recommendations remained open.
22
   SBA OIG Report 15-10 and SBA OIG Report 18-18. The SBA OIG’s recommendations
are discussed in further detail later in the report.
23
     GAO-15-593SP.
24
 Office of Management and Budget, Management’s Responsibility for Enterprise Risk
Management and Internal Control, Circular No. A-123 (Washington, D.C.: July 15, 2016).
The Fraud Reduction and Data Analytics Act of 2015 required the Office of Management
and Budget to establish guidelines for federal agencies to create controls to identify and
assess fraud risks and design and implement antifraud control activities. Pub.L. No. 114-
186, § 3(a), 130 Stat. 546 (2016).




Page 9                                 GAO-19-168 Women-Owned Small Business Program
                            As of February 2019, SBA had implemented one of the three changes
SBA Has                     that the 2015 NDAA made to the WOSB program—sole-source authority.
Implemented One of          The two other changes—authorizing SBA to implement its own
                            certification process for WOSBs and requiring SBA to eliminate the
the Three Changes           WOSB self-certification option—have not been implemented. 25 The 2015
Made by the 2015            NDAA did not require a specific time frame for SBA to update its
                            regulations. SBA officials have stated that they will not eliminate self-
NDAA                        certification until the new certification process for the WOSB program is in
                            place, which they expect to be completed by January 1, 2020.


Sole-Source Authority Has   In September 2015, SBA published a final rule to implement sole-source
Been Implemented            authority for the WOSB program (effective October 2015). 26 Among other
                            things, the rule authorized contracting officers to award a contract to a
                            WOSB or EDWOSB without competition, provided that the contracting
                            officer’s market research cannot identify two or more WOSBs or
                            EDWOSBs in eligible industries that can perform the requirements of the
                            contract at a fair and reasonable price. In the final rule, SBA explained
                            that it promulgated the sole-source rule before the WOSB certification
                            requirements for two reasons. First, the sole-source rule could be
                            accomplished by simply incorporating the statutory language into the
                            regulations, whereas the WOSB certification requirements would instead
                            require a prolonged rulemaking process. Second, SBA said that
                            addressing all three regulatory changes at the same time would delay the
                            implementation of sole-source authority. SBA described the sole-source
                            mechanism as an additional tool for federal agencies to ensure that
                            women-owned small businesses have an equal opportunity to participate
                            in federal contracting and to ensure consistency among SBA’s
                            socioeconomic small business procurement programs.

                            According to SBA, most of the 495 comments submitted about the sole-
                            source rule supported the agency’s decision to implement the authority

                            25
                             The 2015 NDAA expanded the WOSB certification process to include the SBA
                            Administrator, in addition to the previously approved methods of a federal agency, a state
                            government, or a national certifying entity approved by SBA.
                            26
                               See Women-Owned Small Business Federal Contract Program, 80 Fed. Reg. 55019
                            (Sept. 14, 2015). The value of a WOSB sole-source contract, including options, cannot
                            exceed $6.5 million for manufacturing contracts and $4 million for all other contracts. A
                            related interim rule updating the FAR was issued in December 2015 and was followed by
                            a final rule in September 2016.




                            Page 10                                GAO-19-168 Women-Owned Small Business Program
                            quickly. However, the SBA OIG’s June 2018 audit report cautioned that
                            allowing sole-source contracting authority while firms can still self-certify
                            exposes the WOSB program to unnecessary risk of fraud and abuse, and
                            the report recommended that SBA implement a new certification process
                            for the WOSB program per the 2015 NDAA. 27 In addition, our previous
                            report identified risks of program participation by ineligible firms
                            associated with deficiencies in SBA’s oversight structure. 28 As we discuss
                            in detail later, SBA has still not addressed these risks, which may be
                            exacerbated by the implementation of sole-source authority without
                            addressing the other changes made by the 2015 NDAA, including
                            eliminating the self-certification option.


A New WOSB Program          As of February 2019, SBA had not published a proposed rule for public
Certification Process Has   comment to establish a new certification process for the WOSB program.
                            Previously, in October 2017, an SBA official stated that SBA was about
Not Been Implemented
                            1–2 months away from publishing a proposed rule. However, in June
                            2018, SBA officials stated that a cost analysis would be necessary before
                            the draft could be sent to the Office of Management and Budget for
                            review. Certain stages of the rulemaking process have mandated time
                            periods, such as the required interagency review process for certain
                            rules. In June 2017, we reported that SBA officials said that an increase
                            in the number of statutorily mandated rules in recent years had
                            contributed to delays in the agency’s ability to promulgate rules in a more
                            timely fashion. 29 As of February 2019, SBA had not provided
                            documentation or time frames for issuing a proposed rule or completing
                            the rulemaking process. However, in response to the SBA OIG
                            recommendation that SBA implement the new certification process, SBA
                            stated that it would fulfill the recommendation (meaning implement a new
                            certification process) by January 1, 2020. 30

                            In December 2015, SBA published an advance notice of proposed
                            rulemaking to solicit public comments to assist the agency with drafting a

                            27
                                 SBA OIG Report 18-18.
                            28
                                 GAO-15-54.
                            29
                             GAO, Small Business Administration: Government Contracting and Business
                            Development Processes and Rule-Making Activities, GAO-17-573 (Washington, D.C.:
                            June 30, 2017).
                            30
                                 SBA OIG Report 18-18.




                            Page 11                            GAO-19-168 Women-Owned Small Business Program
proposed rule to implement a new WOSB certification program. 31 In the
notice, SBA stated that it intends to address the 2015 NDAA changes,
including eliminating the self-certification option, through drafting
regulations to implement a new certification process. Previously, in its
September 2015 final rule implementing sole-source authority, SBA
stated that there was no evidence that Congress intended that the
existing WOSB program, including self-certification, be halted before
establishing the infrastructure and new regulations for a new certification
program. The advance notice requested comments on various topics,
such as how well the current certification processes were working, which
of the certification options were feasible and should be pursued, whether
there should be a grace period for self-certified WOSB firms to complete
the new certification process, and what documentation should be
required.

Three third-party certifiers submitted comments in response to the
advance notice of proposed rulemaking, and none supported the option of
SBA acting as a WOSB certifier. 32 One third-party certifier commented
that such an arrangement is a conflict of interest given that SBA is also
responsible for oversight of the WOSB program, and two certifiers
commented that SBA lacked the required resources. The three third-party
certifiers also asserted in their comments that no other federal agency
should be allowed to become an authorized WOSB certifier, with one
commenting that federal agencies should instead focus on providing
contracting opportunities for women-owned businesses. All three certifiers
also proposed ways to improve the current system of third-party
certification—for example, by strengthening oversight of certifiers or
expanding their number. The three certifiers also suggested that SBA
move to a process that better leverages existing programs with
certification requirements similar to those of the WOSB program, such as
the 8(a) program. In the advance notice, SBA asked for comments on
alternative certification options, such as SBA acting as a certifier or
limiting WOSB program certifications to the 8(a) program and otherwise




31
   Women-Owned Small Business and Economically Disadvantaged Women-Owned Small
Business—Certification, 80 Fed. Reg. 78984 (Advance notice of proposed rulemaking,
Dec. 18, 2015).
32
 Based on our review of the publically available comment letters, one of the four third-
party certifiers did not comment on the advance notice of proposed rulemaking.




Page 12                                GAO-19-168 Women-Owned Small Business Program
relying on state or third-party certifiers. 33 Further, in June 2018, SBA
officials told us that they were evaluating the potential costs of a new
certification program as part of their development of the new certification
rule. 34




33
 WOSB program regulations currently allow WOSB certification through the 8(a) program;
however, SBA officials said that this method of certification currently represents a small
percentage of all WOSB program certifications.
34
   Executive Order 12866 and Office of Management and Budget Circular A-4 require that
certain agencies—including SBA—generally assess the potential costs and benefits of
actions, including proposed rules that are deemed to be a “significant regulatory action.” A
“significant regulatory action” is defined by the order as one that is likely to result in a rule
that may (1) have an annual effect on the economy of $100 million or more, or adversely
affect in a material way the economy or a sector of the economy, among other things
(generally referred to as “economically significant” regulations); (2) create a serious
inconsistency or otherwise interfere with another agency’s taken or planned actions; (3)
materially alter the budgetary impact of entitlements, grants, user fees, or loan programs
or the rights and obligations of recipients thereof; or (4) raise novel legal or policy issues
arising from legal mandates, the President’s priorities, or the principles set forth in the
executive order. Under Executive Order 12866, the cost-benefit analysis for “economically
significant” regulatory actions should generally include, among other things, assessments
of (1) benefits anticipated from the regulatory action (such as, but not limited to, the
promotion of the efficient functioning of the economy and private markets), (2) costs
anticipated from the regulatory action (such as, but not limited to, the direct cost both to
the government in administering the regulation and to businesses and others in complying
with the regulation, and any adverse effects on the efficient functioning of the economy
and private markets), and (3) costs and benefits of potentially effective and reasonably
feasible alternatives to the planned regulation, identified by the agencies or the public
(including improving the current regulation and reasonably viable nonregulatory actions).
See Exec. Order No. 12866 § 6(a)(3)(C), 3 C.F.R. § 12866 (1994). It has not yet been
determined if the new certification rule will be a “significant regulatory action.”




Page 13                                   GAO-19-168 Women-Owned Small Business Program
SBA Has Not Fully
Addressed
Deficiencies in
Oversight and
Program
Implementation

SBA Has Not Implemented   SBA has not fully addressed deficiencies in its oversight of third-party
Procedures to Regularly   certifiers that we identified in our October 2014 report. 35 We reported that
Monitor and Assess the    SBA did not have formal policies for reviewing the performance of its four
                          approved third-party certifiers, including their compliance with their
Performance of Third-     agreements with SBA. Further, we found that SBA had not developed
Party Certifiers          formal policies and procedures for, among other things, reviewing the
                          monthly reports that certifiers submit to SBA. 36 As a result, we
                          recommended that SBA establish comprehensive procedures to monitor
                          and assess the performance of the third-party certifiers in accordance
                          with their agreements with SBA and program regulations. While SBA has
                          taken some steps to address the recommendation, as of February 2019 it
                          remained open.

                          In response to our October 2014 recommendation, in 2016 SBA
                          conducted compliance reviews of the four SBA-approved third-party
                          certifiers. According to SBA, the purpose of the compliance reviews was
                          to ensure the certifiers’ compliance with regulations, their signed third-
                          party certifier certification form (or agreement) with SBA, and other
                          program requirements. The compliance reviews included an assessment
                          of the third-party certifiers’ internal certification procedures and
                          processes, an examination of a sample of applications from businesses
                          that the certifiers deemed eligible and ineligible for certification, and an
                          interview with management staff.



                          35
                               GAO-15-54.
                          36
                            Per their written agreements, the third-party certifiers are required to submit monthly
                          reports to SBA describing WOSB program performance information.




                          Page 14                                 GAO-19-168 Women-Owned Small Business Program
SBA officials said that SBA’s review team did not identify significant
deficiencies in any of the four certifiers’ processes and found that all were
generally complying with their agreements. However, one compliance
review report described “grave concerns” that a third-party certifier had
arbitrarily established eligibility requirements that did not align with WOSB
program regulations and used them to decline firms’ applications. 37 SBA
noted in the report that if the third-party certifier failed to correct this
practice SBA could terminate the agreement. As directed by SBA, the
third-party certifier submitted a letter to SBA outlining actions it had taken
to address this issue, among others. 38 The final compliance review
reports for the other third-party certifiers also recommended areas for
improvement, including providing staff with additional training on how to
conduct eligibility examinations and reviewing certification files to ensure
they contain complete documentation. In addition, two of the three
compliance review reports with recommendations (including the
compliance review report for the certifier discussed above) required the
certifier to provide a written response within 30 days outlining plans to
correct the areas. SBA officials said that they reviewed the written
responses and determined that no further action was required. 39

In January 2017, SBA’s Office of Government Contracting updated its
written Standard Operating Procedures (SOP) to include policies and
procedures for the WOSB program, in part to address our October 2014
recommendation. The 2017 SOP discusses what a third-party-certifier
compliance review entails, how often the reviews are to be conducted,

37
  Specifically, SBA found that the third-party certifier denied firms solely on the grounds
that the women owner was not the highest compensated officer, which SBA’s report states
was outside the parameters of eligibility for the WOSB program. According to the report,
SBA provided guidance to the third-party certifier stating that it should not deny firms
based on compensation issues alone and that the third-party certifier should take into
account the totality of the circumstances, including whether decisions concerning
compensation make business sense.
38
  In its response, the third-party certifier disagreed with several of SBA’s compliance
review findings, but with respect to the highest compensation issue agreed to be more
detailed in documenting descriptions of the indicators included in its analysis. More
specifically, the third-party certifier agreed to direct its examiners to take the following
steps: (1) ensure that SBA guidance is considered; (2) request additional information from
the applicant that explains the business reasons for the highest officer not receiving the
highest compensation, and (3) explain, if a denial is made in part based on highest
compensation, how this issue was considered in the examiner’s broader assessment so it
is clear to the applicant and SBA.
39
 We reviewed the response letters that SBA received from the two third-party certifiers
and found that they appeared to address the identified areas for improvement.




Page 15                                GAO-19-168 Women-Owned Small Business Program
and how findings are to be reported. The 2017 SOP notes that SBA may
initiate a compliance review “at any time and as frequently as the agency
determines is necessary.” In September 2018, SBA officials told us that
they were again updating the SOP, in part to address deficiencies we
identified in our prior work and during this review. However, as of
February 2019, SBA had not provided an updated SOP.

In addition, in April 2018, SBA finalized a WOSB Program Desk Guide
that, according to SBA, is designed to provide program staff with detailed
guidance for conducting oversight procedures, including compliance
reviews of third-party certifiers. For example, the Desk Guide discusses
how staff should prepare for a compliance review of a third-party certifier,
review certification documents, and prepare a final report. However, the
Desk Guide does not describe specific activities designed to oversee
third-party certifiers on an ongoing basis. In November 2017, SBA
officials told us that they planned to conduct additional compliance
reviews of the third-party certifiers. However, in June 2018, officials said
there were no plans to conduct further compliance reviews until the final
rule implementing the new certification process was completed. Further,
SBA officials said that the 2016 certifier compliance reviews did not result
in significant deficiencies. However, as noted previously, one of the
compliance review reports described a potential violation of the third-party
certifier’s agreement with SBA.

Per written agreements with SBA, third-party certifiers are required to
submit monthly reports that include

•   the number of WOSB and EDWOSB applications received, approved,
    and denied;
•   identifying information for each certified business, such as the
    business name;
•   concerns about fraud, waste, and abuse; and
•   a description of any changes to the procedures the organizations
    used to certify businesses as WOSBs or EDWOSBs.
In our October 2014 report, we noted that SBA had not followed up on
issues raised in the monthly reports and had not developed written




Page 16                          GAO-19-168 Women-Owned Small Business Program
procedures for reviewing them. 40 At that time, SBA officials said that they
were unaware of the issues identified in the certifiers’ reports and that the
agency was developing procedures for reviewing the monthly reports but
could not estimate a completion date.

In our interviews for this report, SBA officials stated that SBA still does not
use the third-party certifiers’ monthly reports to regularly monitor the
program. Specifically, SBA does not review the reports to identify any
trends in certification deficiencies that could inform program oversight.
Officials said the reports generally do not contain information that SBA
considers helpful for overseeing the WOSB program, although staff
sometimes use the reports to obtain firms’ contact information. SBA
officials also said that staff very rarely receive information about
potentially fraudulent WOSB firms from the third-party certifiers—maybe
three firms per year—and that this information is generally received via
email and not as part of the monthly reports. SBA officials said that when
they receive information about potentially fraudulent firms, WOSB
program staff conduct an examination to determine the firm’s eligibility
and report the results back to the certifier. However, a third-party certifier
told us it has regularly reported firms it suspected of submitting potentially
fraudulent applications in its monthly reports and that SBA has not
followed up with them. In addition, two third-party certifiers said that if
SBA is not cross-checking the list of firms included in their monthly
reports, a firm deemed ineligible by one certifier may submit an
application to another certifier and obtain approval.

The three third-party certifiers we spoke with said that SBA generally had
not communicated with them about their implementation of the program
since the 2016 compliance reviews. However, SBA officials noted that
three of the four third-party certifiers attended an SBA roundtable in

40
   GAO-15-54. For our 2014 report, we reviewed all of the third-party certifiers’ monthly
reports from August 2011 through May 2014, for a total of 135 reports. We found that not
all reports contained consistent information. Some monthly reports were missing the
owner names and contact information for businesses that had applied for certification.
One certifier regularly identified potential fraud among businesses to which it had denied
certification (about one or two instances per month for 16 of the 34 reporting months we
reviewed). The reporting format and level of detail reported also varied among certifiers.
Finally, we found issues involving two businesses that were denied certification by one
third-party certifier and approved shortly after by another. At the time, SBA officials told us
that they were unaware of these firms but that fraudulently entering into a set-aside
contract was illegal and the businesses would be subject to prosecution. However, we
concluded that if SBA does not follow up on these types of issues, it is unclear how
businesses committing fraud in the program would be prosecuted.




Page 17                                  GAO-19-168 Women-Owned Small Business Program
March 2017 to discuss comments on the proposed rulemaking. In
addition, SBA officials said that the third-party certifiers may contact them
with questions about implementing the WOSB program, but SBA
generally does not reach out to them.

Although SBA has taken steps to enhance its written policies and
procedures for oversight of third-party certifiers, it does not have plans to
conduct further compliance reviews of the certifiers and does not intend to
review certifiers’ monthly reports on a regular basis. SBA officials said
that third-party certifier oversight procedures would be updated, if
necessary, after certification options have been clarified in the final
WOSB certification rule. However, ongoing oversight activities, such as
regular compliance reviews, could help SBA better understand the steps
certifiers have taken in response to previous compliance review findings
and whether those steps have been effective. In addition, leading fraud
risk management practices include identifying specific tools, methods,
and sources for gathering information about fraud risks, including data on
fraud schemes and trends from monitoring and detection activities, as
well as involving relevant stakeholders in the risk assessment process. 41
Without procedures to regularly monitor and oversee third-party certifiers,
SBA cannot provide reasonable assurance that certifiers are complying
with program requirements and cannot improve its efforts to identify
ineligible firms or potential fraud. Further, it is unclear when SBA’s final
rule will be implemented. As a result, we maintain that our previous
recommendation should be addressed—that is, that the Administrator of
SBA should establish and implement comprehensive procedures to
monitor and assess the performance of certifiers in accordance with the
requirements of the third-party certifier agreement and program
regulations.




41
     GAO-15-593SP.




Page 18                          GAO-19-168 Women-Owned Small Business Program
SBA Has Not Implemented       SBA also has not fully addressed deficiencies found in our 2014 review
Procedures to Improve Its     related specifically to eligibility examinations. 42 We found that SBA lacked
Eligibility Examinations of   formalized guidance for its eligibility examination processes and that the
                              examinations continued to identify high rates of potentially ineligible
WOSB Program                  businesses. As a result, we recommended that SBA enhance its
Participants                  examination of businesses that register for the WOSB program to ensure
                              that only eligible businesses obtain WOSB set-asides. Specifically, we
                              suggested that SBA consider (1) completing the development of
                              procedures to conduct annual eligibility examinations and implementing
                              such procedures; (2) analyzing examination results and individual
                              businesses found to be ineligible to better understand the cause of the
                              high rate of ineligibility in annual reviews and determine what actions are
                              needed to address the causes, and (3) implementing ongoing reviews of
                              a sample of all businesses that have represented their eligibility to
                              participate in the program.

                              SBA has taken some steps to implement our recommendation—such as
                              by completing its 2017 SOP and its Desk Guide, both of which include
                              written policies and procedures for WOSB program eligibility
                              examinations. The 2017 SOP includes a brief description of what
                              activities are entailed in the examinations, the staff responsible for
                              conducting them, and how firms are selected. In addition, as noted
                              previously, SBA officials told us in September 2018 that a forthcoming
                              update to the SOP would address deficiencies we identified regarding
                              WOSB eligibility examinations. However, as of February 2019, SBA had
                              not provided an updated SOP. The Desk Guide contains more detailed
                              information on eligibility examinations. It notes that a sample of firms is to
                              be examined annually and it provides selection criteria, which can include
                              whether the agency has received information challenging the firm’s
                              eligibility for the program. The Desk Guide also provides specific
                              instructions on how to determine whether a firm meets the WOSB
                              program’s ownership, control, and financial requirements and what
                              documentation should be consulted or requested.

                              SBA does not collect reliable information on the results of its annual
                              eligibility examinations. According to SBA officials, SBA has conducted
                              eligibility examinations of a sample of businesses that received WOSB

                              42
                                 GAO-15-54. We reported that SBA conducts eligibility examinations for a sample of
                              businesses with a current attestation in SBA’s System for Award Management and that
                              received a contract during SBA’s examination year, regardless of the certification method.




                              Page 19                                GAO-19-168 Women-Owned Small Business Program
program set-aside contracts each year since fiscal year 2012. However,
SBA officials told us that the results of annual eligibility examinations—
such as the number of businesses found eligible or ineligible—are
generally not documented. As a result, we obtained conflicting data from
SBA on the number of examinations completed and the percentage of
businesses found to be ineligible in fiscal years 2012 through 2018. For
example, based on previous information provided by SBA, we reported in
October 2014 that in fiscal year 2012, 113 eligibility examinations were
conducted and 42 percent of businesses were found to be ineligible for
the WOSB program. However, during this review, we received information
from SBA that 78 eligibility examinations were conducted and 37 percent
of businesses were found ineligible in fiscal year 2012. We found similar
disparities when we compared fiscal year 2016 data provided by SBA for
this report with a performance memorandum summarizing that fiscal
year’s statistics. 43 Regardless of the disparity between the data sources,
the rate of ineligible businesses has remained significant. For example,
according to documentation SBA provided during this review, in fiscal
year 2017, SBA found that about 40 percent of the businesses in its
sample were not eligible.

In addition, SBA continues to have no mechanism for evaluating
examination results in aggregate to inform the WOSB program. In 2014,
we reported that SBA officials told us that most businesses that were
deemed ineligible did not understand the documentation requirements for
establishing eligibility. However, we also reported that SBA officials could
not explain how they knew a lack of understanding was the cause of
ineligibility among businesses and had not made efforts to confirm that
this was the cause. In June 2018, SBA officials told us they did not
analyze the annual examinations in aggregate for common eligibility
issues because the examination results are unique to each WOSB firm.
They noted that this was not necessary as WOSB program staff are
familiar with common eligibility issues through the annual eligibility
examinations. As we noted in 2014, by not analyzing aggregate
examination results, the agency is missing opportunities to obtain
meaningful insights into the program, such as the reasons many
businesses are deemed ineligible.

43
   In December 2016, SBA officials created an internal performance memorandum, from
the Director of SBA’s Office of Government Contracting, that summarized the results of
fiscal year 2016 eligibility examinations. Officials said that no subsequent performance
memorandums have been created because the Director position has been vacant since
December 2017.




Page 20                                GAO-19-168 Women-Owned Small Business Program
Also, SBA still conducts eligibility examinations only of firms that have
already received a WOSB award. In 2014, we concluded that this
sampling practice restricts SBA’s ability to identify potentially ineligible
businesses prior to a contract award. Similarly, during this review, SBA
officials said that while some aspects of the sample characteristics have
changed since 2012, the samples still generally consist only of firms that
have been awarded a WOSB set-aside. In addition, officials said that the
sample size of the eligibility examinations has varied over time and is
largely based on the workload of WOSB program staff. Restricting the
samples in this way limits SBA’s ability to better understand the eligibility
of businesses before they apply for and are awarded contracts, as well as
its ability to detect and prevent potential fraud. 44

SBA officials said that their other means of reducing participation by
ineligible firms and mitigating potential fraud is through WOSB or
EDWOSB status protests—that is, allegations that a business receiving
an award does not meet program eligibility requirements. 45 A federal
contractor can file a status protest against any firm receiving an award
that represents itself as a WOSB in the System for Award Management
for grounds that include failure to provide all required supporting
documentation. The penalties for misrepresenting a firm’s status, per
regulation, include debarment or suspension. However, one third-party
certifier expressed in its comments to the advance notice of proposed
rulemaking on certification that status protests alone are not a viable
option for protecting the integrity of the WOSB program. The certifier
questioned how a firm could have sufficient information about a
competitor firm to raise questions about its eligibility. According to SBA
officials, 11 status protests were filed under the WOSB program in fiscal
year 2018. Of these, four firms were deemed ineligible for the WOSB
program, four were deemed eligible, and three status protests were
dismissed. In fiscal year 2017, 9 status protests were filed; of these, three
firms were found ineligible, two were found eligible, and four status
protests were dismissed.



44
     We made similar findings in our 2014 review of the WOSB program. See GAO-15-54.
45
   WOSB status protests may only be initiated in connection with WOSB or EDWOSB
procurement. For sole-source procurements, SBA or the contracting officer may protest
the proposed awardee’s WOSB or EDWOSB status. For all other WOSB or EDWOSB
procurements, an interested party (including contracting officers, SBA, and another firm
submitting an offer) may make the protest. SBA adjudicates these protests.




Page 21                                GAO-19-168 Women-Owned Small Business Program
                            We recognize that SBA has made some effort to address our previous
                            recommendation by documenting procedures for conducting annual
                            eligibility examinations of WOSB firms. However, leading fraud risk
                            management practices state that federal program managers should
                            design control activities that focus on fraud prevention over detection and
                            response, to the extent possible. 46 Without maintaining reliable
                            information on the results of eligibility examinations, developing
                            procedures for analyzing results, and expanding the sample of
                            businesses to be examined to include those that did not receive
                            contracts, SBA limits the value of its eligibility examinations and its ability
                            to reduce ineligibility among businesses registered to participate in the
                            WOSB program. These deficiencies also limit SBA’s ability to identify
                            potential fraud risks and develop any additional control activities needed
                            to address these risks. As a result, the program may continue to be
                            exposed to the risk of ineligible businesses receiving set-aside contracts.
                            In addition, in light of these continued oversight deficiencies, the
                            implementation of sole-source authority without addressing the other
                            changes made by the 2015 NDAA could increase program risk. For these
                            reasons, we maintain that our previous recommendation that SBA
                            enhance its WOSB eligibility examination procedures should be
                            addressed.


SBA Has Not Addressed       In 2015 and 2018, the SBA OIG reported instances in which WOSB set-
Previously Identified       asides were awarded using NAICS codes that were not eligible under the
                            WOSB program, and our analysis indicates that this problem persists. In
Issues with WOSB Set-
                            2015, the SBA OIG reported on its analysis of a sample of 34 WOSB set-
Asides Awarded Under        aside awards and found that 10 awards were set aside using an ineligible
Ineligible Industry Codes   NAICS code. 47 The SBA OIG concluded that this may have been due to
                            contracting officers’ uncertainty about NAICS code requirements under
                            the program and recommended that SBA provide additional, updated
                            training and outreach to federal agencies’ contracting officers on the
                            program’s NAICS code requirements. 48 In response, SBA updated WOSB
                            46
                                 GAO-15-593SP.
                            47
                             In addition, four of these awards that were set aside using ineligible NAICS codes had
                            no documentation in the WOSB program database. SBA OIG 15-10.
                            48
                               SBA OIG Report 15-10. The SBA OIG made five recommendations in total. The other
                            recommendations directed SBA to provide additional training on WOSB program
                            documentation requirements to WOSB program firms, potential WOSB program firms, and
                            contracting officers; update two separate WOSB-specific forms; and conduct eligibility
                            examinations of the firms that SBA OIG identified as potentially ineligible for the program.




                            Page 22                                GAO-19-168 Women-Owned Small Business Program
program training and outreach documents in March 2016 to include
information about the program’s NAICS code requirements.

In 2018, the SBA OIG issued another report evaluating the WOSB
program, with a focus on the use of the program’s sole-source contract
authority. Here, the SBA OIG identified additional instances of contracting
officers using inaccurate NAICS codes to set aside WOSB contracts. 49
Specifically, the SBA OIG reviewed a sample of 56 awards and found that
4 were awarded under ineligible NAICS codes. The report included two
recommendations for SBA aimed at preventing and correcting improper
NAICS code data in FPDS-NG: (1) conduct quarterly reviews of FPDS-
NG data to ensure contracting officers used the appropriate NAICS codes
and (2) in coordination with the Office of Federal Procurement Policy and
GSA, strengthen controls in FPDS-NG to prevent contracting officers from
using ineligible NAICS codes. 50

SBA disagreed with both of these recommendations. In its response to
the first recommendation, SBA stated that it is not responsible for the
oversight of other agencies’ contracting officers and therefore is not in a
position to implement the corrective actions. With respect to the second
recommendation, SBA stated that adding such controls to FPDS-NG
would further complicate the WOSB program and increase contracting
officers’ reluctance to use it. SBA also stated its preference for focusing
its efforts on ensuring that contracting officers select the appropriate
NAICS code at the beginning of the award process.

In our review, we also found several issues with WOSB program set-
asides being awarded under ineligible NAICS codes. Our analysis of
FPDS-NG data on all obligations to WOSB program set-asides from the
third quarter of fiscal year 2011 through the third quarter of fiscal year
2018 found the following:


49
     SBA OIG Report 18-18.
50
   SBA OIG Report 18-18. The SBA OIG made six recommendations in total. The other
remaining recommendations addressed to SBA were to (1) conduct eligibility reviews for
firms that the SBA OIG identified as lacking required documentation and require those
firms to remove their designation in the System for Award Management; (2) initiate
debarment proceedings, if necessary, against the firms that the SBA OIG found to be
lacking the required documentation for WOSB eligibility; (3) implement a new WOSB
certification process as stated in the 2015 NDAA; and (4) conduct quarterly reviews of
firms with newly obtained WOSB or EDWOSB status to ensure they have submitted the
required documentation, until SBA implements its new certification process.




Page 23                               GAO-19-168 Women-Owned Small Business Program
•    3.5 percent (or about $76 million) of WOSB program obligations were
     awarded under NAICS codes that were never eligible for the WOSB
     program; 51
•    10.5 percent (or about $232 million) of WOSB program obligations
     made under an EDWOSB NAICS code went to women-owned
     businesses that were not eligible to receive awards in EDWOSB-
     eligible industries; 52 and
•    17 of the 47 federal agencies that obligated dollars to WOSB program
     set-asides during the period used inaccurate NAICS codes in at least
     5 percent of their WOSB set-asides (representing about $25 million).
According to SBA officials we spoke with during this review, WOSB
program set-asides may be awarded under ineligible NAICS codes
because of human error when contracting officers are inputting data in
FPDS-NG or because a small business contract was misclassified as a
WOSB program set-aside. They characterized the extent of the issue as
“small” relative to the size of the FPDS-NG database and said that such
issues do not affect the program’s purpose. Rather than review FPDS-NG
data that are inputted after the contract is awarded, SBA officials said that
they have discussed options for working with GSA to add controls
defining eligible NAICS codes for WOSB program set-aside opportunities
on FedBizOpps.gov—the website that contracting officers use to post
announcements about available federal contracting opportunities. 53
Adding controls to this system, officials said, would help contracting
officers realize as they are writing the contract requirements that they

51
   As discussed previously, the WOSB program’s eligible NAICS codes have changed
since the program was implemented in 2011. In October 2016, SBA made substantive
changes to the program’s eligible industries. Two other changes occurred in October 2012
and October 2017 to align the WOSB program with broader changes made to NAICS.
Based on our analysis of FPDS-NG data, we also found that, from the third quarter of
fiscal year 2011 through the third quarter of fiscal year 2018, 0.9 percent (about $20.5
million) of WOSB program obligation dollars were awarded under NAICS codes that were
initially ineligible but eventually became eligible. Often, there is a time lag between the
effective date of NAICS codes and when they are entered in FPDS-NG. Therefore, we did
not classify a contract as having an ineligible NAICS code if the code eventually became
eligible under the WOSB program. See app. I for more information on our methodology.
52
 FPDS-NG data also showed that another 9.4 percent (or about $208 million) of WOSB
obligations from the third quarter of fiscal year 2011 through the third quarter of fiscal year
2018 were coded using an EDWOSB NAICS code and awarded to an EDWOSB firm but
were classified as a WOSB set-aside (rather than an EDWOSB set-aside).
53
  FedBizOpps.gov (www.fbo.gov) is the federal government’s website used to post all
federal procurement opportunities with a value over $25,000.




Page 24                                  GAO-19-168 Women-Owned Small Business Program
should not set aside contracts under the WOSB program without
reviewing the proper NAICS codes. However, SBA officials said that the
feasibility of this option was still being discussed and that the issue was
not a high priority. For these reasons, according to officials, SBA’s
updated oversight procedures described in the 2017 SOP and the Desk
Guide do not include a process for reviewing WOSB program set-aside
data in FPDS-NG to determine whether they were awarded under the
appropriate NAICS codes.

Further, as of November 2018, the WOSB program did not have targeted
outreach or training that focused on specific agencies’ use of NAICS
codes. As noted previously, in March 2016, SBA updated its WOSB
program training materials to address NAICS code requirements in
response to a 2015 SBA OIG recommendation. In fiscal year 2018, SBA
conducted three WOSB program training sessions for federal contracting
officers, including (1) a virtual learning session, (2) a session conducted
during WOSB Industry Day at the Department of Housing and Urban
Development, and (3) a session conducted during a Department of
Defense Small Business Training Conference. However, with the
exception of the virtual learning session, these training sessions were
requested by the agencies. SBA officials did not identify any targeted
outreach or training provided to specific agencies to improve
understanding of WOSB NAICS code requirements (or other issues
related to the WOSB program).

Congress authorized SBA to develop a contract set-aside program
specifically for WOSBs and EDWOSBs to address the
underrepresentation of such businesses in specific industries. In addition,
federal standards for internal control state that management should
design control activities to achieve objectives and respond to risks and to
establish and operate monitoring activities to monitor and evaluate the
results. 54 Because SBA does not review whether contracts are being
awarded under the appropriate NAICS codes, it cannot provide
reasonable assurance that WOSB program requirements are being met
or identify agencies that may require targeted outreach or additional
training on eligible NAICS codes. As a result, WOSB contracts may
continue to be awarded to groups other than those intended, which can
undermine the goals of and confidence in the program.


54
     GAO-14-704G.




Page 25                          GAO-19-168 Women-Owned Small Business Program
Federal Contracts to
WOSB Set-Asides
Remain Relatively
Small, and
Stakeholders
Discussed Various
Aspects of Program
Use
The Percentage of          Federal dollars obligated for contracts to all women-owned small
Obligations to Women-      businesses increased from $18.2 billion in fiscal year 2012 to $21.4 billion
                           in fiscal year 2017. These figures include contracts for any type of good
Owned Small Businesses
                           or service awarded under the WOSB program, under other federal
under the WOSB Program     programs, or through full and open competition. Contracts awarded to all
Increased Slightly since   women-owned small businesses within WOSB-program-eligible industries
2012                       also increased during this period—from about $15 billion to $18.8 billion,
                           as shown in figure 1. However, obligations under the WOSB program
                           represented only a small share of this increase. In fiscal year 2012,
                           WOSB program contract obligations were 0.5 percent of contract
                           obligations to all women-owned small businesses for WOSB-program-
                           eligible goods or services (about $73.5 million), and in fiscal year 2017
                           this percentage had grown to 3.8 percent (about $713.3 million) (see fig.
                           1).




                           Page 26                         GAO-19-168 Women-Owned Small Business Program
Figure 1: Obligations for the Women-Owned Small Business Program and to All Women-Owned Small Businesses in Similar
Industries, Fiscal Years 2012–2017




                                       Note: These Federal Procurement Data System-Next Generation (FPDS-NG) data represent contract
                                       obligations to women-owned small businesses whose North American Industry Classification System
                                       (NAICS) codes were among those eligible for the WOSB program. FPDS-NG obligation amounts
                                       have been adjusted for inflation using the Fiscal Year Gross Domestic Product price index. All data
                                       exclude actions in FPDS-NG that are coded other than as a small business and actions under a non–
                                       WOSB program NAICS code. Fiscal year 2018 data are not included in the graphic because data for
                                       the full fiscal year were not available at the time of our analysis. In addition, fiscal year 2011 data are
                                       not included because the WOSB program was implemented in April 2011. For comparative purposes,
                                       we only include full fiscal years 2012 through 2017.




                                       From fiscal years 2012 through 2017, 98 percent of total dollars obligated
                                       for contracts to all women-owned small businesses in WOSB-program-
                                       eligible industries were not awarded under the WOSB program. Instead,
                                       these contracts were awarded without a set-aside or under other, longer-
                                       established socioeconomic contracting programs, such as HUBZone, the
                                       SDVOSB, and 8(a). 55 For example, during this period, dollars obligated to

                                       55
                                          A women-owned small business can receive a contracting award under the 8(a),
                                       HUBZone, or SDVOSB programs, through a general small business set-aside, or without
                                       a set-aside.




                                       Page 27                                       GAO-19-168 Women-Owned Small Business Program
contracts awarded to women-owned small businesses without a set-aside
represented about 34 percent of dollars obligated for contracts to all
women-owned small businesses in these industries (see fig. 2).

Figure 2: Percentage of Contract Obligations to Women-Owned Small Businesses
by Set-Aside Type in Similar Industries, Fiscal Years 2012–2017




Note: These Federal Procurement Data System-Next Generation (FPDS-NG) data on obligations to
women-owned small businesses represent contract obligations to women-owned small businesses
under WOSB-program-eligible North American Industry Classification System (NAICS) codes. All
data exclude actions in FPDS-NG coded other than as a small business and actions under a non–
WOSB program NAICS code.
a
 The “no set-asides” category includes dollar obligations for contracts awarded to women-owned
small businesses through full and open competition.
b
 The “other set-aside or sole source” category predominantly consists of contracts awarded under a
general small business set-aside.


As shown in table 1, six federal agencies—DOD, DHS, Department of
Commerce, Department of Agriculture, Department of Health and Human
Services, and GSA—collectively accounted for nearly 83 percent of the
obligations awarded under the WOSB program from the third quarter of
fiscal year 2011 through the third quarter of fiscal year 2018, with DOD
accounting for about 49 percent of the total.




Page 28                                    GAO-19-168 Women-Owned Small Business Program
Table 1: Top Contracting Departments and Agencies under the WOSB Program,
from Third Quarter of Fiscal Year 2011 through Third Quarter of Fiscal Year 2018

                                                  Total WOSB program set-                     Percentage of
                                                         aside obligations                   WOSB program
 Agency                                                        (in dollars)                     obligations
 Department of Defense                                              1,035,876,037                      48.6
 Department of Homeland                                                263,672,796                     12.4
 Security
 Department of Commerce                                                170,318,456                      8.0
 Department of Agriculture                                             134,363,401                      6.3
 Department of Health and                                                86,226,448                     4.0
 Human Services
 General Services Administration                                         84,220,466                     4.0
 All other agencies                                                    358,079,002                     16.8
 Total                                                             $2,132,756,606                     100%
Source: GAO analysis of Federal Procurement Data System-Next Generation data. | GAO-19-168

Note: These Federal Procurement Data System-Next Generation (FPDS-NG) data on obligations to
women-owned small businesses represent contract obligations to women-owned small businesses
under North American Industry Classification System (NAICS) codes eligible under the Women-
Owned Small Business Program (WOSB program). FPDS-NG obligation amounts have been
adjusted for inflation using the Fiscal Year Gross Domestic Product price index. All data exclude
actions in FPDS-NG coded other than as a small business and actions under a non–WOSB program
NAICS code.


Contracting officers’ use of sole-source authority was relatively limited,
representing about 12 percent of WOSB program obligations from
January 2016 through June 2018. In fiscal year 2017—the only full fiscal
year for which we have data on sole-source authority—about $77 million
were obligated using sole-source authority. 56 The share of sole-source
awards as a percentage of total WOSB program set-asides also varied
considerably by quarter—from as low as 5 percent in the third quarter of
2016 to as high as 21 percent in the first quarter of 2017 (see fig. 3).




56
   In addition, total dollars obligated to WOSB program sole-source awards were
approximately $44.2 million for the last three quarters of fiscal year 2016 and were about
$48.7 million for the first three quarters of 2018.




Page 29                                              GAO-19-168 Women-Owned Small Business Program
                            Figure 3: Sole-Source Obligations as a Percentage of All WOSB Program
                            Obligations, January 2016–June 2018




                            Note: The Small Business Administration implemented sole-source authority for the Women-Owned
                            Small Business Program (WOSB program) effective October 2015. The Federal Procurement Data
                            System-Next Generation (FPDS-NG) data dictionary states that data on WOSB sole-source awards
                            and economically disadvantaged WOSB sole-source awards were added to the system on November
                            13, 2015. Therefore, our analysis begins with the first full quarter in which these data became
                            available—the second quarter of fiscal year 2016. These FPDS-NG data on obligations to women-
                            owned small businesses represent contract obligations to women-owned small businesses under
                            WOSB-program-eligible North American Industry Classification System (NAICS) codes. All data
                            exclude actions in FPDS-NG coded other than as a small business and actions under a non–WOSB
                            program NAICS code. FPDS-NG obligation amounts have been adjusted for inflation using the Fiscal
                            Year Gross Domestic Product price index.




Stakeholders Discussed      We spoke with 14 stakeholder groups to obtain their views on usage of
Various Issues Related to   the WOSB program. 57 These groups consisted of staff within three federal
WOSB Program Usage          agencies (DHS, DOD, and GSA), eight contracting offices within these
                            agencies, and three third-party certifiers. Issues stakeholders discussed
                            included the impact of sole-source authority and program-specific NAICS
                            codes on program usage. Stakeholders also noted the potential effect of
                            other program requirements on contracting officers’ willingness to use the
                            program, and some suggested that SBA provide additional guidance and
                            training to contracting officers.
                            57
                                 See app. I for more information on these groups.




                            Page 30                                  GAO-19-168 Women-Owned Small Business Program
Sole-source authority. Participants in 12 of the 14 stakeholder groups
commented on the effect of sole-source authority on WOSB program
usage. 58 Staff from 4 of the 12 stakeholder groups—including three
contracting offices—said that sole-source authority generally had no
effect on the use of the WOSB program. One of these stakeholders
believed contracting officers seldom use the authority because they lack
an understanding of how and when to use it; therefore, in this
stakeholder’s opinion, use of the WOSB program has not generally
changed since the authority was implemented. However, staff from two
contracting offices and one third-party certifier said that sole-source
authority was a positive addition because, for example, it can significantly
reduce the lead time before a contracting officer can offer a contract
award to a firm. Staff from one of these two contracting offices stated that
the award process can take between 60 to 90 days using sole-source
authority, compared to 6 to 12 months using a competitive WOSB
program set-aside. These staff also said that negotiating the terms of a
sole-source contract is easier, from a contracting officer’s perspective,
because they can communicate directly with the firm. As discussed
previously, SBA officials we interviewed said that adding sole-source
authority to the WOSB program made the program more consistent with
other existing socioeconomic set-aside programs, such as 8(a) and
HUBZone.

The remaining five stakeholder groups that discussed the effects of
WOSB sole-source authority described difficulties with implementing it.
Specifically, representatives from DHS, DOD, and one third-party certifier
said that executing sole-source authority under the WOSB program is
difficult for contracting officers because rules for sole-source authority
under WOSB are different from those under other SBA programs, such as
8(a) and HUBZone. For example, the FAR’s requirement that contracting
officers justify, in writing, why they do not expect other WOSBs or
EDWOSBs to submit offers on a contract is stricter under the WOSB
program than it is for the 8(a) program. Further, staff from one contracting
office noted that justifications for WOSB set-asides must then be
published on a federal website. In contrast, contracting officers generally
do not need to prepare and publish a justification under the 8(a)




58
 While we asked all 14 stakeholder groups to comment on the effect of sole-source
authority on the WOSB program, 2 did not respond to the question.




Page 31                              GAO-19-168 Women-Owned Small Business Program
program. 59 According to staff from another contracting office, it may be
difficult to find more than one firm qualified to do the work under some
WOSB-eligible NAICS codes, but contracting officers would still have to
conduct market research and explain why they do not expect additional
offers in order to set the contract aside for a WOSB.

Program-specific NAICS codes. Participants in 13 of the 14 stakeholder
groups we interviewed commented on the requirement that WOSB
program set-asides be awarded within certain industries, represented by
NAICS codes. 60 For example, two third-party certifiers we interviewed
recommended that the NAICS codes be expanded or eliminated to
provide greater opportunities for WOSBs to win contracts under the
program. Another third-party certifier said that some of its members focus
their businesses’ marketing efforts on industries specific to the WOSB
program to help them compete for such contracts.

Representatives from GSA and DHS made comments about limitations
with respect to the WOSB program’s NAICS code requirement. Staff we
interviewed from three contracting offices made similar statements,
adding that the NAICS codes limit opportunities to award a contract to a
WOSB or EDWOSB because they are sufficient in some industry areas
but not others. All five of these stakeholder groups suggested that NAICS
codes be removed from the program’s requirements to increase
opportunities for WOSBs.

Conversely, staff from five other contracting offices we interviewed
generally expressed positive views about the WOSB program’s NAICS
code requirements and stated that eligible codes line up well with the
services for which they generally contract. Finally, SBA officials noted that
there are no plans to reassess the NAICS codes until about 2020.
However, SBA officials also stated that the NAICS code requirements
complicate the WOSB program and add confusion for contracting officers

59
   Sole-source awards under the 8(a) program for contracts less than or equal to $22
million do not generally require a written justification and approval. See 48 C.F.R. § 6.302-
5(c)(2)(iii). If a justification is prepared, it must be published on FedBizOpps.gov in
accordance with the FAR. For more information on the justification and approval for sole-
source awards in the 8(a) program, see GAO, DOD Small Business Contracting: Use of
Sole-Source 8(a) Contracts over $20 Million Continues to Decline, GAO-16-557
(Washington, D.C.: June 8, 2016).
60
  We asked all 14 stakeholder groups to comment on the effect of the NAICS code
requirements on the use of the program, and 1 did not have any comments.




Page 32                                 GAO-19-168 Women-Owned Small Business Program
who use program, as compared to other socioeconomic programs that do
not have such requirements, such as HUBZone or 8(a).

Requirement to verify eligibility documentation. Staff from 7 of the 14
stakeholder groups we interviewed discussed the requirement for the
contracting officer to review program eligibility documentation and how
this requirement affects their decision to use the program. 61 For example,
staff from one contracting office said that using the 8(a) or HUBZone
programs is easier because 8(a) and HUBZone applicants are already
certified by SBA; therefore, the additional step to verify documentation for
eligibility is not needed. GSA officials noted that eliminating the need for
contracting officers to take additional steps to review eligibility
documentation for WOSB-program set-asides—in addition to checking
the System for Award Management—could create more opportunities for
WOSBs by reducing burden on contracting officers. However, staff from
two contracting offices said it is not more difficult to award contracts under
the WOSB program versus other socioeconomic programs.

WOSB program guidance. Staff from 13 of the14 stakeholder groups we
interviewed discussed guidance available to contracting officers under the
WOSB program. 62 Most generally said that the program requirements
outlined in the FAR are fairly detailed and help contracting officers
implement the program. According to SBA officials, SBA provides training
on WOSB program requirements to contracting officers in federal
agencies by request, through outreach events, and through an annual
webinar. SBA officials also said that the training materials include all the
regulatory issues that contracting officers must address.

However, representatives from two third-party certifiers described
feedback received from their members about the need to provide

61
  In our interviews with officials from GSA, DHS, DOD, and three third-party certifiers, all
interviewees described their experience with the WOSB program as compared to other
socioeconomic contracting programs. During interviews with the eight contracting offices,
we specifically asked contracting office staff to discuss whether awarding contracts under
the WOSB program was more or less difficult than other socioeconomic contracting
programs. Of the 14 stakeholder groups, 7 groups discussed the impact of the WOSB
program’s requirement for contracting officers to verify eligibility documentation on their
decision to use the program, while the remaining 7 did not discuss the requirement in this
context.
62
   We asked stakeholders about the extent of their interaction with SBA and any guidance
provided by SBA to assist with their implementation of the WOSB program. One
stakeholder group did not respond to the question directly.




Page 33                                 GAO-19-168 Women-Owned Small Business Program
additional training and guidance for contracting officers to better
understand and implement the WOSB program. Staff from two
contracting offices also expressed the need for SBA to provide additional
training and guidance. Staff from one of these contracting offices said that
the last time they received training on the WOSB program was in 2011,
when the program was first implemented. Staff in the other contracting
office added that the most recent version of a WOSB compliance guide
they could locate online was at least 6 years old. 63 SBA officials estimated
that the WOSB compliance guide was removed from their public website
in March 2016 because it was difficult to keep the document current and
officials did not want to risk publishing a guide that was out-of-date. SBA
officials also said that there are no plans to issue an updated guide as the
FAR is sufficient.

The stakeholder groups also identified positive aspects of the WOSB
program. Specifically, staff from seven stakeholder groups believed that
the program provided greater opportunities for women-owned small
businesses to obtain contracts in industries in which they are
underrepresented. 64 In addition, staff from three stakeholder groups
mentioned that SBA-led initiatives, such as the Small Business
Procurement Advisory Council and SBA’s co-sponsorship of the
ChallengeHER program, help improve collaboration between federal
agencies and the small business community and overall government
contracting opportunities for women-owned small businesses. 65




63
   In October 2010, SBA published a notice in the Federal Register announcing the
availability of the WOSB compliance guide. See Small Entity Compliance Guide: Women-
Owned Small Business Program, 75 Fed. Reg. 66411 (Oct. 28, 2010). The guide was
designed, in part, to provide a reference for contracting officers and a summary of the
purpose and requirements of the WOSB program.
64
  We reported similar perspectives from stakeholders we interviewed in our October 2014
report; see GAO-15-54. During interviews for this review, we asked all 14 groups to
comment on positive aspects of the WOSB program or best practices; 4 stakeholder
groups did not respond to these questions directly.
65
  The Small Business Procurement Advisory Council consists of members from the other
23 Chief Financial Officers Act agencies. Among other things, the council helps educate
the public and agencies on the WOSB program. The ChallengeHER program is an
educational outreach program that provides women-owned small businesses with
procurement education, best practices for working with the government, and resources to
learn how to apply and obtain a federal contract under the WOSB program.




Page 34                               GAO-19-168 Women-Owned Small Business Program
              The WOSB program aims to enhance federal contracting opportunities for
Conclusions   women-owned small businesses. However, weaknesses in SBA’s
              management of the program continue to hinder its effectiveness. As of
              February 2019, SBA had not fully implemented comprehensive
              procedures to monitor the performance of the WOSB program’s third-
              party certifiers and had not taken steps to provide reasonable assurance
              that only eligible businesses obtain WOSB set-aside contracts, as
              recommended in our 2014 report. 66 Without ongoing monitoring and
              reviews of third-party certifier reports, SBA cannot ensure that the
              certifiers are fulfilling the requirements of their agreements with SBA, and
              it is missing opportunities to gain information that could help improve the
              program’s processes. Further, limitations in SBA’s procedures for
              conducting, documenting, and analyzing eligibility examinations inhibit its
              ability to better understand the eligibility of businesses before they apply
              for and potentially receive contracts, which exposes the program to
              unnecessary risk of fraud. In addition, given that SBA does not expect to
              finish implementing the changes in the 2015 NDAA until January 1, 2020,
              these continued oversight deficiencies increase program risk. As a result,
              we maintain that our previous recommendations should be addressed.

              In addition, SBA has not addressed deficiencies that the SBA OIG
              identified previously—and that we also identified during this review—
              related to WOSB set-asides being awarded under ineligible industry
              codes. Although SBA has updated its training and outreach materials for
              the WOSB program to address NAICS code requirements, it has not
              developed plans to review FPDS-NG data or provide targeted outreach or
              training to agencies that may be using ineligible codes. As a result, SBA
              is not aware of the extent to which individual agencies are following
              program requirements and which agencies may require targeted outreach
              or additional training. Reviewing FPDS-NG data would allow SBA to
              identify those agencies (and contracting offices within them) that could
              benefit from such training. Without taking these additional steps, SBA
              cannot provide reasonable assurance that WOSB program requirements
              are being met.




              66
                   GAO-15-54.




              Page 35                         GAO-19-168 Women-Owned Small Business Program
                     The SBA Administrator or her designee should (1) develop a process for
Recommendation for   periodically reviewing FPDS-NG data to determine the extent to which
Executive Action     agencies are awarding WOSB program set-asides under ineligible NAICS
                     codes and (2) take steps to address any issues identified, such as
                     providing targeted outreach or training to agencies making awards under
                     ineligible codes. (Recommendation 1)


                     We provided a draft of this report to DHS, DOD, GSA, and SBA for review
Agency Comments      and comment. DHS, DOD, and GSA indicated that they did not have
                     comments. SBA provided a written response, reproduced in appendix II,
                     in which it agreed with our recommendation. SBA stated that it will
                     implement a process to review WOSB program data extracted from
                     FPDS-NG and certified by each agency. Specifically, through the
                     government-wide Small Business Procurement Advisory Council, SBA
                     plans to provide quarterly presentations to contracting agencies’ staff that
                     would include training and an analysis and review of the data. The
                     response also reiterated that SBA has contacted GSA to implement a
                     system change to FedBizOpps.gov that would prevent contracting officers
                     from entering an invalid NAICS code for a WOSB program set-aside.




                     Page 36                         GAO-19-168 Women-Owned Small Business Program
As agreed with your offices, unless you publicly announce the contents of
this report earlier, we plan no further distribution until 30 days from the
report date. At that time we will send copies of this report to appropriate
congressional committees and members, the Acting Secretary of DOD,
the Secretary of DHS, the Administrator of GSA, the Administrator of
SBA, and other interested parties. This report will also be available at no
charge on our website at http://www.gao.gov.

If you or your staff have any questions concerning this report, please
contact me at (202) 512-8678 or shearw@gao.gov. Contact points for our
Offices of Congressional Relations and Public Affairs may be found on
the last page of this report. GAO staff who made major contributions to
this report are listed in appendix III.




William B. Shear
Director, Financial Markets and
  Community Investment




Page 37                           GAO-19-168 Women-Owned Small Business Program
Appendix I: Objectives, Scope, and
              Appendix I: Objectives, Scope, and
              Methodology



Methodology

              This report examines (1) the extent to which the Small Business
              Administration (SBA) has implemented changes to the Women-Owned
              Small Business Program (WOSB program) made by the 2015 National
              Defense Authorization Act (2015 NDAA); (2) the extent to which SBA has
              implemented changes to address previously identified oversight
              deficiencies; and (3) changes in WOSB program use since 2011 and
              stakeholder views on its use, including since the 2015 implementation of
              sole-source authority.

              To describe the extent to which SBA has implemented changes to the
              WOSB program made by the 2015 NDAA, we reviewed relevant
              legislation, including the 2015 NDAA; related proposed regulations; and
              SBA documentation. We reviewed comment letters on the advance notice
              of proposed rulemaking for the new WOSB program certification process
              from three of the four SBA-approved third-party certifiers: the El Paso
              Hispanic Chamber of Commerce, the U.S. Women’s Chamber of
              Commerce, and the Women’s Business Enterprise National Council. 1 To
              ensure the accuracy of our characterization of the comment letters, one
              staff member independently summarized the third-party certifiers’
              comments on the advance notice, and a second staff member then
              reviewed the results. We also interviewed SBA officials, including officials
              from SBA’s Office of Government Contracting and Business
              Development.

              To respond to the second and third objectives, we conducted interviews
              on SBA’s implementation and oversight of the WOSB program and its use
              with SBA officials, three of the WOSB program’s four third-party certifiers,
              three selected agencies (and three agency components within two of the
              agencies), and a total of eight selected contracting offices within six
              selected agencies or components. 2 Using data from the Federal
              Procurement Data System-Next Generation (FPDS-NG), we judgmentally
              selected the three federal agencies and three components (for a total of
              six federal agencies and components) because their WOSB program
              dollar obligations (including competed and sole-source) were among the

              1
               See Women-Owned Small Business and Economically Disadvantaged Women-Owned
              Small Business—Certification, 80 Fed. Reg. 78984 (Advance notice of proposed
              rulemaking, Dec. 18, 2015). Based on our review of the publically available comment
              letters, the fourth third-party certifier—NWBOC, formerly known as the National Women
              Business Owners Corporation—did not submit a comment letter.
              2
               We did not speak with NWBOC because NWBOC officials did not make themselves
              available for an interview with us within the time period necessary for our review.




              Page 38                               GAO-19-168 Women-Owned Small Business Program
Appendix I: Objectives, Scope, and
Methodology




largest or because we had interviewed them for our prior work. 3
Specifically, we selected the following six agencies or agency
components: the Department of Homeland Security (DHS) and, within
DHS, the Coast Guard; the Department of Defense (DOD) and, within
DOD, the U.S. Army and U.S. Navy; and the General Services
Administration (GSA). Within the components and GSA, we judgmentally
selected eight contracting offices (two each from Coast Guard, U.S.
Army, U.S. Navy, and GSA) based on whether they had a relatively large
amount of obligations and had used multiple types of WOSB program set-
asides (competed or sole-source) to WOSBs or economically
disadvantaged women-owned small businesses (EDWOSB).

To address our second objective, we reviewed the findings and
recommendations in our October 2014 report and in audit reports issued
by the SBA Office of Inspector General (OIG) in May 2015 and June
2018. 4 We also reviewed SBA documentation on the WOSB program,
including SBA’s 2017 Standard Operating Procedures and 2018 WOSB
Program Desk Guide, results from 2016 compliance reviews of the four
third-party certifiers, and SBA eligibility examinations from fiscal years
2012 through 2018. In addition, we analyzed FPDS-NG data on contract
obligations to WOSB program set-asides from the third quarter of fiscal
year 2011 through the third quarter of fiscal year 2018 to determine
whether set-asides were made using eligible program-specific North
American Industry Classification System (NAICS) codes. 5 To conduct this

3
 FPDS-NG contains data for most federal contract actions that have an estimated value
over the micropurchase threshold (which is currently $3,500). The federal government
uses its data to create recurring and special reports to the President, the Congress,
federal executive agencies, and the general public.
4
  GAO, Women-Owned Small Business Program: Certifier Oversight and Additional
Eligibility Controls Are Needed, GAO-15-54 (Washington, D.C.: Oct. 8, 2014). SBA Office
of Inspector General, Improvements Needed in SBA’s Management of the Women Owned
Small Business Federal Contracting Program, SBA OIG Report 15-10, (Washington, D.C.:
May 14, 2015) and SBA’s Women-Owned Small Business Federal Contracting Program,
SBA OIG Report 18-18 (Washington, D.C.: June 20, 2018).
5
 SBA uses industry studies to determine in which industries and areas of work
(represented by NAICS codes) WOSBs and EDWOSBs are underrepresented. We pulled
data from FPDS-NG on July 25, 2018, for this analysis and for analysis for our third
objective. FPDS-NG data are entered by agency contracting officers on a real-time basis
and, thus, they change every day. FPDS-NG data undergo a certification process each
January for the prior fiscal year. The data we included in our analysis from three quarters
of fiscal year 2018 had not been certified at the time of our analysis. However, based on
the data quality summary reports from prior years, we believe these data to be reliable for
the purposes of our reporting objectives.




Page 39                                GAO-19-168 Women-Owned Small Business Program
Appendix I: Objectives, Scope, and
Methodology




analysis, we compared contract obligations in FPDS-NG with the NAICS
codes eligible under the WOSB program at the time of the award for the
time frame under review. The WOSB program’s eligible NAICS codes
have changed three times since the program was implemented in 2011,
but the eligible industries have changed once. 6 SBA commissioned the
RAND Corporation to conduct the first study to assist SBA in determining
eligible NAICS codes under the WOSB program. Based on the results of
the RAND study, SBA identified 45 four-digit WOSB NAICS codes and 38
four-digit EDWOSB NAICS codes, for a total of 83 four-digit NAICS
codes. 7 WOSB and EDWOSB NAICS codes are different and do not
overlap. In December 2015, the Department of Commerce issued the
next study, which increased the total NAICS codes under the program to
113 four-digit codes, with 92 WOSB NAICS codes and 21 EDWOSB
NAICS codes (which became effective March 2016). 8 Often, there is a
time lag between the effective date of NAICS codes and when they are
entered in FPDS-NG. Therefore, we did not classify a contract as having
an ineligible NAICS code if the code eventually became eligible under the
WOSB program. We also excluded actions in FPDS-NG coded other than
as a small business. These actions represented a small amount of
contract obligations—approximately $125,000. We compared SBA
information on its oversight activities and responses to previously
identified deficiencies, federal internal control standards, and GAO’s fraud
risk framework. 9



6
 SBA’s Office of Government and Business Development updated the eligible six-digit
NAICS codes for the WOSB program in October 2012 to reflect the NAICS 2012 revision
and again in October 2017 to reflect the NAICS 2017 revision.
7
 NAICS uses a six-digit coding system to identify particular industries and their placement
in the hierarchical structure of the classification system. A four-digit code designates the
sector, subsector, and industry group. Additional digits (the fifth and sixth) provide
additional information on the NAICS industry and the national industry. WOSB program
industry eligibility determinations were made at the four-digit NAICS industry group level.
See Kauffman-RAND Institute for Entrepreneurship Public Policy, The Utilization of
Women-Owned Small Businesses in Federal Contracting (Santa Monica, Calif.: 2007).
8
 See David N. Beede and Robert N. Rubinovitz, Utilization of Women-Owned Small
Businesses in Federal Prime Contracting, a report prepared for the Small Business
Administration (Washington, D.C.: Economics and Statistics Administration, Department of
Commerce, Dec. 31, 2015).
9
  GAO, Standards for Internal Control in the Federal Government, GAO-14-704G
(Washington, D.C.: September 2014) and A Framework for Managing Fraud Risks in
Federal Programs, GAO-15-593SP (Washington, D.C.: July 2015).




Page 40                                 GAO-19-168 Women-Owned Small Business Program
Appendix I: Objectives, Scope, and
Methodology




We assessed the reliability of FPDS-NG data by considering their known
strengths and weaknesses, based on our past work and through
electronic testing for missing data, outliers, and inconsistent coding in the
data elements we used for our analysis. We also reviewed FPDS-NG
documentation, including the FPDS-NG data dictionary, FPDS-NG data
validation rules, FPDS-NG user manual, prior GAO reliability
assessments, and relevant SBA OIG audit reports. Based on these steps,
we concluded that the data were sufficiently reliable for the purposes of
reporting on trends in the WOSB program and the use of sole-source
authority under the program.

To describe how participation in the WOSB program has changed since
2011, including since the 2015 implementation of sole-source authority,
we analyzed FPDS-NG data from the third quarter of fiscal year 2011
through the third quarter of fiscal year 2018. We identified any trends in
WOSB program participation using total obligation dollars set aside for
competitive and sole-source contracts awarded to WOSBs and
EDWOSBs under the program. We also compared data on obligations for
set-asides under the WOSB program with federal contract obligations for
WOSB-program-eligible goods and services to all women-owned small
businesses, including those made under different set-aside programs or
with no set-asides, to determine the relative usage of the WOSB program.
In our analysis, we excluded from WOSB program set-aside data actions
in FPDS-NG coded other than as a small business (representing
approximately $125,000) or coded under ineligible NAICS codes that
were never eligible under the WOSB program (representing
approximately $76.3 million).

To describe stakeholder views on WOSB program use, we conducted
semistructured interviews to gather responses from 14 stakeholder
groups. These groups consisted of staff within three federal agencies
(DHS, DOD, and GSA), eight contracting offices within these agencies,
and three third-party certifiers (selection criteria described above). One
person summarized the results of the interviews, and another person
reviewed the summary of the interviews to ensure an accurate depiction
of the comments. In addition, a third person then reviewed the
summarized results.

We conducted this performance audit from October 2017 to March 2019
in accordance with generally accepted government auditing standards.
Those standards require that we plan and perform the audit to obtain
sufficient, appropriate evidence to provide a reasonable basis for our
findings and conclusions based on our audit objectives. We believe that


Page 41                              GAO-19-168 Women-Owned Small Business Program
Appendix I: Objectives, Scope, and
Methodology




the evidence obtained provides a reasonable basis for our findings and
conclusions based on our audit objectives.




Page 42                              GAO-19-168 Women-Owned Small Business Program
Appendix II: Comments from the U.S. Small
              Appendix II: Comments from the U.S. Small
              Business Administration



Business Administration




              Page 43                               GAO-19-168 Women-Owned Small Business Program
Appendix II: Comments from the U.S. Small
Business Administration




Page 44                               GAO-19-168 Women-Owned Small Business Program
Appendix II: Comments from the U.S. Small
Business Administration




Page 45                               GAO-19-168 Women-Owned Small Business Program
Appendix III: GAO Contact and Staff
                  Appendix III: GAO Contact and Staff
                  Acknowledgments



Acknowledgments


                  William B. Shear, (202) 512-8678, or shearw@gao.gov
GAO Contact
                  In addition to the contact named above, Allison Abrams (Assistant
Staff             Director), Tiffani Humble (Analyst-in-Charge), Pamela Davidson,
Acknowledgments   Jonathan Harmatz, Julia Kennon, Jennifer Schwartz, Rebecca Shea,
                  Jena Sinkfield, Tyler Spunaugle, and Tatiana Winger made key
                  contributions to this report.




(102412)
                  Page 46                               GAO-19-168 Women-Owned Small Business Program
                         The Government Accountability Office, the audit, evaluation, and investigative
GAO’s Mission            arm of Congress, exists to support Congress in meeting its constitutional
                         responsibilities and to help improve the performance and accountability of the
                         federal government for the American people. GAO examines the use of public
                         funds; evaluates federal programs and policies; and provides analyses,
                         recommendations, and other assistance to help Congress make informed
                         oversight, policy, and funding decisions. GAO’s commitment to good government
                         is reflected in its core values of accountability, integrity, and reliability.

                         The fastest and easiest way to obtain copies of GAO documents at no cost is
Obtaining Copies of      through GAO’s website (https://www.gao.gov). Each weekday afternoon, GAO
GAO Reports and          posts on its website newly released reports, testimony, and correspondence. To
                         have GAO e-mail you a list of newly posted products, go to https://www.gao.gov
Testimony                and select “E-mail Updates.”

Order by Phone           The price of each GAO publication reflects GAO’s actual cost of production and
                         distribution and depends on the number of pages in the publication and whether
                         the publication is printed in color or black and white. Pricing and ordering
                         information is posted on GAO’s website, https://www.gao.gov/ordering.htm.
                         Place orders by calling (202) 512-6000, toll free (866) 801-7077, or
                         TDD (202) 512-2537.
                         Orders may be paid for using American Express, Discover Card, MasterCard,
                         Visa, check, or money order. Call for additional information.

                         Connect with GAO on Facebook, Flickr, Twitter, and YouTube.
Connect with GAO         Subscribe to our RSS Feeds or E-mail Updates. Listen to our Podcasts.
                         Visit GAO on the web at https://www.gao.gov.

                         Contact FraudNet:
To Report Fraud,
                         Website: https://www.gao.gov/fraudnet/fraudnet.htm
Waste, and Abuse in
                         Automated answering system: (800) 424-5454 or (202) 512-7700
Federal Programs
                         Orice Williams Brown, Managing Director, WilliamsO@gao.gov, (202) 512-4400,
Congressional            U.S. Government Accountability Office, 441 G Street NW, Room 7125,
Relations                Washington, DC 20548

                         Chuck Young, Managing Director, youngc1@gao.gov, (202) 512-4800
Public Affairs           U.S. Government Accountability Office, 441 G Street NW, Room 7149
                         Washington, DC 20548

                         James-Christian Blockwood, Managing Director, spel@gao.gov, (202) 512-4707
Strategic Planning and   U.S. Government Accountability Office, 441 G Street NW, Room 7814,
External Liaison         Washington, DC 20548




                            Please Print on Recycled Paper.