oversight

Electronic Procurement: Business Strategy Needed for GSA's Advantage System

Published by the Government Accountability Office on 2003-02-19.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                United States General Accounting Office

GAO             Report to the Chairman, Committee on
                Government Reform, U.S. House of
                Representatives


February 2003
                ELECTRONIC
                PROCUREMENT
                Business Strategy
                Needed for GSA's
                Advantage System




GAO-03-328
                                               February 2003


                                               ELECTRONIC PROCUREMENT

                                               Business Strategy Needed for GSA’s
Highlights of GAO-03-328, a report to the      Advantage System
Chairman, Committee on Government
Reform, U.S. House of Representatives




The Congress has been seeking to               GSA Advantage has had only limited success as an on-line market research
increase federal agency purchases              and ordering tool. Market research has been limited primarily to off-the-
of equipment and services on line.             shelf office products, and sales through Advantage have never exceeded
The General Services                           one–half of 1 percent of overall schedule sales. Because of initial design
Administration’s (GSA) Advantage               limitations, Advantage has not been effective in acquiring complex products
Internet-based ordering system is
meant for conducting market
                                               and services, particularly information technology services that make up
research and ordering all types of             most of the growth in schedule sales. Recent enhancements may facilitate
products and services on line.                 use of the system for buying complex services, but vendor participation has
About $84 million has already been             not been strong. For example, for almost half the requests for quotes,
invested in this endeavor. GAO                 agencies received no responses from vendors. Advantage does not contain
was asked to assess how                        all schedule products and services, as required, and some schedule vendors
effectively GSA has managed this               complain about the cost and complexity involved in participating.
investment.
                                               While GSA has modified the Advantage system to provide better and more
                                               information on products and services, the system is unlikely to succeed
                                               without a business strategy. To date, GSA has not yet answered basic
GAO recommends that GSA
                                               questions surrounding Advantage, including (1) if it is technically feasible to
develop a business strategy that
reexamines its objectives for on-              make purchases of all products and services on line given the range of
line purchasing of products and                activities involved with making more complex acquisitions, (2) if so, whether
services, assesses what solutions              Advantage is the right system to do this, and (3) if not, whether Advantage
are most appropriate for meeting               could be used for smaller, simpler purchases and as an information source
these objectives, and develops                 for making larger, more complicated acquisitions.
measures for assessing success.
GSA concurs with this report’s                 Moreover, GSA has not thoroughly analyzed other alternatives to Advantage
recommendations.                               or developed effective measures to assess return on investment and user
                                               satisfaction. A business strategy would require GSA to not only reexamine
                                               its objectives for on-line purchasing and various solutions, but also develop
                                               measures appropriate for gauging success. As such, it would help GSA
                                               assess whether continued investment in Advantage is worthwhile and make
                                               modifications to the system that will substantially enhance its performance.

                                               Sales under Federal Supply System Schedule Contracts—Fiscal Years 1998 to 2002




www.gao.gov/cgi-bin/getrpt?GAO-03-328.

To view the full report, including the scope
and methodology, click on the link above.
For more information, contact William Woods
at (202) 512-4841, WoodsW@gao.gov.             Note: Numbers on top of columns are the percentages of schedule sales through Advantage.
Contents


Letter                                                                                               1
           Results in Brief                                                                          1
           Background                                                                                2
           Agencies Use Advantage More for Market Research than for
             Ordering, and Mostly for Standard Office Products                                       3
           Accommodating Buys of Complex Services Has Been Problematic                               4
           Lack of Full Vendor Participation Due to Concerns About Cost and
             Use                                                                                     6
           Performance Reflects the Lack of a Business Strategy                                      6
           Conclusion                                                                                7
           Recommendations                                                                           7
           Agency Comments                                                                           7
           Scope and Methodology                                                                     7

Appendix   Comments From the General Services
           Administration                                                                            9



Figure
           Figure 1: Sales under FSS Schedule Contracts—Fiscal Years 1998
                    to 2002                                                                          4




           This is a work of the U.S. Government and is not subject to copyright protection in the
           United States. It may be reproduced and distributed in its entirety without further
           permission from GAO. It may contain copyrighted graphics, images or other materials.
           Permission from the copyright holder may be necessary should you wish to reproduce
           copyrighted materials separately from GAO’s product.




           Page i                         GAO-03-328 Business Strategy Needed For GSA Advantage
United States General Accounting Office
Washington, DC 20548




                                   February 19, 2003

                                   The Honorable Tom Davis
                                   Chairman
                                   Committee on Government Reform
                                   House of Representatives

                                   Dear Mr. Chairman:

                                   Federal agencies ordered about $22 billion worth of products and services
                                   from commercial firms during fiscal year 2002 using the General Services
                                   Administration’s (GSA) Multiple Award Schedules program. To help
                                   federal agencies use the program more efficiently, GSA developed a
                                   system called GSA Advantage, which is an Internet-based system for
                                   conducting market research and ordering schedule products and services
                                   on line. Because of concerns about the effectiveness of GSA Advantage,
                                   you asked that we review the performance of the system. In response to
                                   your request, we assessed how effective Advantage has been as an on-line
                                   system for federal agencies to shop for and order schedule products and
                                   services.


                                   GSA Advantage has had only limited success as an on-line market research
Results in Brief                   and ordering tool. Market research has been limited primarily to off-the-
                                   shelf office products, and sales through Advantage have never exceeded
                                   one-half of 1 percent of overall schedule sales. Because of initial design
                                   limitations, Advantage has not been an effective means for acquiring
                                   complex products and services, particularly information technology (IT)
                                   services that make up most of the growth in schedule sales. Recent
                                   enhancements may facilitate use of the system for buying complex
                                   services, but vendor participation has not been strong. For example, for
                                   almost half the requests for quotes, agencies received no responses from
                                   vendors. Advantage does not contain all schedule products and services,
                                   as required, and some schedule vendors complain about the cost and
                                   complexity involved in participating.

                                   The limited success of Advantage reflects the lack of a business strategy
                                   for the system. Such a strategy normally would examine such issues as the
                                   type of schedule products and services best suited for on-line market
                                   research and ordering, whether Advantage is the best system to
                                   accomplish these functions in a manner consistent with the needs of


                                   Page 1                   GAO-03-328 Business Strategy Needed For GSA Advantage
             customer agencies and the capabilities of the vendors, and how the
             performance of that system could be measured. After developing such a
             strategy, GSA would have greater assurance that its system is the most
             feasible solution to accomplish its objectives, and would have a sound
             basis for setting policies for Advantage. We are recommending that GSA
             develop a comprehensive business strategy for the Advantage program.
             GSA agrees with this report’s recommendations.


             GSA’s mission includes providing policy leadership and expertise in the
Background   acquisition of products and services for use by federal agencies. To help
             carry out its mission, GSA’s Federal Supply Service (FSS) assists federal
             agencies in acquiring supplies—such as furniture, computers, tools, and
             equipment—and a variety of services through its commercial acquisition
             business line. This GSA business line gives agencies access to several
             million commercial products and services available either through
             schedule contracts or as stock items from a GSA warehouse.

             GSA launched Advantage in October 1995. The system was envisioned as
             an on-line system for agencies to order stock items from GSA’s
             warehouses with a government payment card. Initially, Advantage offered
             only stock items. In February 1996, Congress passed the Clinger-Cohen
             Act, which required the Administrator of GSA to develop a system capable
             of providing governmentwide, on-line Internet access to information on all
             FSS schedule products and services. GSA decided that Advantage,
             although in its infancy, was the only Internet-based, on-line ordering
             system that potentially could meet the requirements of the act,1 and began
             using Advantage to satisfy this requirement. GSA began adding schedule
             services to Advantage in February 2000.

             Despite efforts to include more products and services on Advantage, only
             35 percent of all vendors participated in the program at the beginning of
             fiscal year 2001. GSA therefore issued a mandate that all schedule vendors
             participate in Advantage and offer all their products and services through
             the Advantage system. For off-the-shelf products, vendors typically upload



             1
              The Clinger-Cohen Act called for the economic and efficient procurement of IT and other
             commercial items using a system that provides on-line access to price, features, and
             performance information on all products and services found in FSS schedules, and that
             provides on-line ordering capability. Although GSA Advantage continues, the statutory
             requirement for the systemwas repealed by Section 825(c)(1) of the Bob Stump National
             Defense Authorization Act for Fiscal Year 2003, Pub. L. No. 107-314, 116 Stat. 2616.




             Page 2                        GAO-03-328 Business Strategy Needed For GSA Advantage
                       three types of information: product price, features, and performance. In
                       some cases, they offer quantity discounts. For services, vendors typically
                       upload the labor rates contained in their schedule contracts, along with
                       other information.

                       GSA has added various functions and enhancements to Advantage during
                       the last 7 years, such as reference files to increase the amount of
                       information available to customer agencies. In total, GSA has spent about
                       $84 million to develop, implement, and maintain its Advantage system.


                       Customer agencies do a great deal more comparative shopping than
Agencies Use           ordering through Advantage. During a 7-month period in fiscal year 2002,
Advantage More for     comparative shopping on Advantage ranged between 27,000 and 50,000
                       site searches each day.2 Most of this market research through Advantage
Market Research than   involved office products and equipment. For example, during the same 7-
for Ordering, and      month period, the top searched-for products included office supplies like
                       binders, pens, copier paper, and envelopes, and office equipment like
Mostly for Standard    laptops, fax machines, chairs, and shredders. Searches for services were
Office Products        far less frequent than for products, and most often were for training,
                       security, maintenance, and consulting.

                       Sales through Advantage represent a very small portion of total schedule
                       sales. As shown in the figure, overall schedule sales have grown
                       considerably in the past 5 years—from about $8.5 billion to almost $22
                       billion. Sales through Advantage during this period, however, have never
                       exceeded one-half of 1 percent of total schedule sales. For fiscal year 2002,
                       for example, there were 129,370 schedule orders through Advantage for a
                       total of $81,579,970, representing 0.37 percent of total schedule sales. Most
                       sales through Advantage are for standard office products. For example, for
                       fiscal year 2002, the top 10 schedule items in order of sales frequency
                       included file folders, self-stick pads, multiuse paper, double pocket
                       portfolio, inkjet toner, copy paper, binder clips, and 3-ring binders. Our
                       review of Advantage orders from 1997 through 2002 shows a similar




                       2
                           This calculation does not include holidays and weekends.




                       Page 3                           GAO-03-328 Business Strategy Needed For GSA Advantage
                       pattern of purchases of standard office products. According to GSA,
                       schedule sales through Advantage have averaged about $500 per order.3

                       Figure 1: Sales under FSS Schedule Contracts—Fiscal Years 1998 to 2002




                       Note: Numbers on top of columns are the percentages of schedule sales through Advantage.


                       As shown in the figure, most of the growth in overall schedule sales has
Accommodating Buys     been in spending for IT under GSA’s schedule 70 contracts. Most of this
of Complex Services    growth in IT spending has been for IT services, which now account for
                       about 65 percent of IT spending. Yet an inherent limitation in using
Has Been Problematic   Advantage is that the system was not designed for making complex
                       purchases, such as procurements of IT services.

                       The information available on Advantage regarding complex services
                       typically consists of general descriptions of labor categories, such as a



                       3
                        Advantage sales also include purchases of stock items from GSA’s two remaining
                       warehouses. GSA’s stock program, which comprised about 55 percent of the Advantage
                       sales (in fiscal year 2002), has had a 4-year decline in sales overall due to a changing
                       customer base and the increased use of governmentwide purchase cards. In fact, FSS has
                       closed all but two warehouses and transitioned from a government-owned and operated
                       supply distribution network to a system of alliances with private companies to provide the
                       goods and services. Without alternative strategies such as a virtual store through
                       Advantage that would allow GSA to provide the products to the agencies using direct
                       delivery rather than storing and shipping them from a warehouse, agency officials told us
                       the stock program sales might continue to decline.




                       Page 4                           GAO-03-328 Business Strategy Needed For GSA Advantage
program manager or network analyst, and loaded labor rates.4 In a prior
report, we pointed out that this information alone is not sufficient to
enable agencies using GSA’s schedules to select the best value provider of
IT services.5 Rather, agencies must consider the number of hours and mix
of labor categories proposed by the schedule contractors to perform
specific tasks. Special ordering procedures issued by GSA require agencies
acquiring complex services to obtain this information by providing a
statement of work to at least three schedule contractors asking for either
fixed or ceiling prices for the work specified.

In 2001, GSA added a capability to Advantage, known as E-buy, which
enables agencies to provide a statement of work to vendors and request
responses on-line. This modification provides a means to share
information between customer agencies and vendors on a range of
products and services, including the more complex IT services. During
fiscal year 2002, the first full fiscal year of operation, agencies used E-buy
to place a total of 64 orders. By comparison, customers placed an average
of about 1,240 non-E-buy orders each day through Advantage during the
same period.

Due to the limited initial use of E-buy, GSA revised the function in July
2002 to make it more user-friendly. During the next 2 months of operation
(July-August 2002), agency usage of the revised system increased, but
there were other implementation problems. About 12 percent of requests
for quotations (RFQs) were sent to fewer than three vendors. For each
RFQ issued, agencies received an average of only 1.3 responses. For
almost half the RFQs (about 46 percent), the agencies received no
responsive quote from any vendor.

GSA officials have expressed disappointment in the low usage of E-buy in
its first year of operation, and are concerned about the lack of vendor
responses following revisions to the system. The officials also pointed out,
however, that Advantage and its E-buy enhancement should improve
agency buyers’ market research capabilities.




4
    Loaded labor rates include salary, fringe benefits, overhead, and profit.
5
 U.S. General Accounting Office, Contract Management: Not Following Procedures
Undermines Best Pricing Under GSA’s Schedule, GAO-01-125 (Washington D.C.: Nov. 28,
2000).




Page 5                             GAO-03-328 Business Strategy Needed For GSA Advantage
                         Despite GSA’s mandatory vendor participation policy, not all vendors list
Lack of Full Vendor      all their products and services on Advantage. Out of approximately 9,800
Participation Due to     schedule contracts, about 1,200 contracts (12 percent) were not on
                         Advantage as of September 2002. Even some vendors who supply off-the-
Concerns About Cost      shelf products—which are most likely to be ordered on line—decline to
and Use                  participate. For example, one vendor who sells scales and butcher
                         supplies is not on Advantage because the vendor’s primary customers do
                         not use the system. Another vendor indicated that it uploaded only certain
                         products from its catalog; it has not uploaded the services it offers
                         because its principal service customers do not use Advantage. Other
                         vendors report that it can cost several thousand dollars to upload data on
                         Advantage, particularly for complex products and services. Such
                         expenditures do not make sense, they say, if customers are not using
                         Advantage to make purchases.

                         An underlying reason why some vendors are reluctant to list all of their
                         complex products and services on Advantage is because the system does
                         not contain sufficient descriptive information to permit agencies to make
                         informed buying decisions. To address this issue, GSA added a capability
                         called e-catalog reference files, which allows vendors to supplement the
                         information about their products or services by providing links to the
                         vendors’ own Web pages. Although this capability provides more
                         information about vendor products and services, the descriptions vary
                         widely in quality.


                         GSA has not conducted the type of analysis central to developing a
Performance Reflects     comprehensive business strategy for Advantage. Specifically, GSA has not
the Lack of a Business   analyzed customer requirements, identified the products and services
                         most likely to be purchased on line by agency customers, assessed the
Strategy                 technological feasibility of meeting customer requirements through an on-
                         line system, or considered alternatives to Advantage. In addition, GSA has
                         not established performance goals for Advantage or a system for
                         measuring results. Each of these is an important element of a
                         comprehensive business strategy.

                         GSA officials acknowledge the lack of a business strategy for Advantage.
                         According to these officials, GSA initiated the Advantage project before
                         the agency began performing the kind of rigorous analysis it undertakes
                         before initiating a major system today. They recognized that a business
                         strategy would have enabled GSA management to answer two important
                         questions: (1) how much and what kind of products and services could be



                         Page 6                   GAO-03-328 Business Strategy Needed For GSA Advantage
                  expected to be purchased by federal customers on line and (2) whether
                  Advantage is the right system to facilitate such on-line purchases.


                  GSA has spent about $84 million on Advantage, but it has not developed a
Conclusion        comprehensive business strategy that would justify that level of
                  investment. In the absence of a business strategy, GSA has made changes
                  to enhance the system, but has yet to target what products and services
                  are best suited for on-line market research and ordering. GSA has made
                  these modifications without integrating them into an overall strategy. A
                  rigorously developed business strategy, including performance measures,
                  would help GSA assess whether its continued investment in Advantage is
                  worthwhile and, if so, what additional changes to enhance its performance
                  may be needed.


                  We recommend that the Administrator of General Services develop a
Recommendations   comprehensive business strategy that identifies GSA’s objectives for an
                  on-line market research and ordering system, the desired capabilities of
                  that system, the resources required to implement the system, and
                  measures to track performance. After developing the business strategy,
                  GSA should determine whether Advantage is the most appropriate system
                  for achieving its objectives.


                  GSA concurs with the report’s recommendations. GSA’s comments appear
Agency Comments   in appendix I.


                  To address our objective, we interviewed GSA and industry officials,
Scope and         analyzed electronic files of past market research and on-line sales,
Methodology       attended a GSA training course on how vendors electronically transfer
                  information about their products and services into Advantage, and
                  received various demonstrations of Advantage operations and marketing
                  techniques. We also visited GSA’s two largest acquisition centers
                  (Arlington, Virginia and Ft. Worth, Texas) to learn more about GSA’s
                  efforts to address vendor concerns and encourage vendor participation in
                  Advantage. We conducted our review in accordance with generally
                  accepted government auditing standards from March through December
                  2002.




                  Page 7                   GAO-03-328 Business Strategy Needed For GSA Advantage
Unless you publicly announce the contents of this report earlier, we plan
no further distribution of it until 30 days from the date of this letter. At that
time, we will send copies to other interested congressional committees,
the Administrator of General Services, and the Director, Office of
Management and Budget. We will also make copies available to others
upon request. In addition, the report will be available at no charge on the
GAO Web site at http://www.gao.gov.

Please contact me at (202) 512-8214 or Ralph Dawn at (202) 512-4544 if
you have any questions regarding this report. Major contributors to this
report were John E. Clary, Ronald J. Salo, James B. Smoak, and Ralph O.
White.

Sincerely yours,




William T. Woods
Director
Acquisition and Sourcing Management




Page 8                      GAO-03-328 Business Strategy Needed For GSA Advantage
             Appendix: Comments From the General
Appendix: Comments From the General
             Services Administration



Services Administration




(120140)
             Page 9                      GAO-03-328 Business Strategy Needed For GSA Advantage
                         The General Accounting Office, the audit, evaluation and investigative arm of
GAO’s Mission            Congress, exists to support Congress in meeting its constitutional responsibilities
                         and to help improve the performance and accountability of the federal
                         government for the American people. GAO examines the use of public funds;
                         evaluates federal programs and policies; and provides analyses,
                         recommendations, and other assistance to help Congress make informed
                         oversight, policy, and funding decisions. GAO’s commitment to good government
                         is reflected in its core values of accountability, integrity, and reliability.


                         The fastest and easiest way to obtain copies of GAO documents at no cost is
Obtaining Copies of      through the Internet. GAO’s Web site (www.gao.gov) contains abstracts and full-
GAO Reports and          text files of current reports and testimony and an expanding archive of older
                         products. The Web site features a search engine to help you locate documents
Testimony                using key words and phrases. You can print these documents in their entirety,
                         including charts and other graphics.
                         Each day, GAO issues a list of newly released reports, testimony, and
                         correspondence. GAO posts this list, known as “Today’s Reports,” on its Web site
                         daily. The list contains links to the full-text document files. To have GAO e-mail
                         this list to you every afternoon, go to www.gao.gov and select “Subscribe to daily
                         E-mail alert for newly released products” under the GAO Reports heading.


Order by Mail or Phone   The first copy of each printed report is free. Additional copies are $2 each. A
                         check or money order should be made out to the Superintendent of Documents.
                         GAO also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a
                         single address are discounted 25 percent. Orders should be sent to:
                         U.S. General Accounting Office
                         441 G Street NW, Room LM
                         Washington, D.C. 20548
                         To order by Phone:     Voice:    (202) 512-6000
                                                TDD:      (202) 512-2537
                                                Fax:      (202) 512-6061


                         Contact:
To Report Fraud,
                         Web site: www.gao.gov/fraudnet/fraudnet.htm
Waste, and Abuse in      E-mail: fraudnet@gao.gov
Federal Programs         Automated answering system: (800) 424-5454 or (202) 512-7470


                         Jeff Nelligan, managing director, NelliganJ@gao.gov (202) 512-4800
Public Affairs           U.S. General Accounting Office, 441 G Street NW, Room 7149
                         Washington, D.C. 20548