oversight

Telecommunications: GSA Needs to Improve Process for Awarding Task Orders for Local Service

Published by the Government Accountability Office on 2003-04-04.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

             United States General Accounting Office

GAO          Report to the Chairman, Committee
             on Government Reform, House of
             Representatives


April 2003
             TELECOMMUNICATIONS
             GSA Needs to Improve
             Process for Awarding
             Task Orders for Local
             Service




GAO-03-369
             a
                                                April 2003


                                                TELECOMMUNICATIONS

                                                GSA Needs to Improve Process for
Highlights of GAO-03-369, a report to the       Awarding Task Orders for Local Services
Chairman, Committee on Government
Reform, House of Representatives




The Metropolitan Area Acquisition               GSA field offices take different approaches to awarding task orders under
(MAA) program, managed by the                   multiple-award MAA contracts, leading to variations both among cities and
General Services Administration                 within cities. Although the FAR gives contracting officers broad latitude in
(GSA), provides local telecommun-               ensuring that this process offers contractors a fair opportunity to be
ications services to government                 considered, GSA recognizes that consistency is important within the
agencies in selected metropolitan
areas. Of the 25 cities in which
                                                nationwide MAA program. However, GSA headquarters has not developed or
MAA contracts were awarded as of                implemented a uniform fair consideration process. As a result, GAO found
January 2003, 15 were awarded to                variations in the processes used: principally, in the time frames used in
two or more providers. Such                     contractor price comparisons (see table). Such inconsistencies frequently
multiple-award contracts are a                  influenced the choice of contractor. Further, because oversight was not
means of promoting competition.                 provided, in six cases agency preference was used as a criterion for selecting
To ensure equity in the award of                a contractor, which is a violation of the FAR. Because GSA did not
task orders under these contracts,              consistently follow a common process that ensured compliance with the
the Federal Acquisition Regulation              FAR, it cannot ensure the fairness of its decisions.
(FAR) requires that the government
provide contractors a fair                      Further, the documentation for about one-fifth of GSA’s fair consideration
opportunity to be considered. GAO
was asked to review, among other
                                                decisions was not adequate for determining how these decisions were
things, whether GSA’s                           reached. According to the FAR, sufficient documentation of all contractual
implementation of the fair                      actions must be maintained to provide (1) a basis for decisions reached and
consideration process is consistent             (2) information for subsequent reviews. Out of 483 fair consideration
and the effect of any inconsistency,            decisions from regional GSA offices in the 11 cities that GAO assessed, the
as well as the adequacy of GSA’s                documentation furnished for 91 (19 percent) was not adequate. Weaknesses
documentation to support the                    observed include lack of stated rationale for decisions reached, price
decisions reached.                              comparisons that did not support the choice of contractor selected by GSA,
                                                and lack of support for technical factors used in making the decisions. These
                                                weaknesses occurred because GSA did not establish and implement uniform
                                                guidelines for documenting its MAA fair consideration decisions. As a result,
GAO is recommending that GSA
                                                MAA stakeholders (GSA, agencies, and MAA contractors) do not have
follow a consistent fair
consideration process, including                assurance that the fair consideration process was properly administered.
uniform requirements for
documentation. Deviations from                  Variations in Time Frames Used in MAA Contractor Price Comparisons
this common process should be                                                               Time frame used in price analysis
documented and communicated to                                                                                                    Insufficient data to
contractors so that all MAA                         City             1 month      1 year      3 years      4 years   Life cycle             determine
stakeholders can understand the                     Atlanta
                                                              a
                                                                            —          —           —           —            —                      —
process.                                            Boston                   X          X          —            X           —                      —
                                                    Cleveland               —          —           —           —             X                     —
In written comments on a draft of
                                                    Dallas                  —           X          —           —             X                     —
this report, the Administrator of
                                                    Denver                   X         —           —           —             X                     —
General Services agreed with our
recommendations and said that                       Indianapolis            —          —           —           —             X                     —
GSA was acting to implement them.                   Los Angeles             —          —           —           —            —                       X
                                                    Minneapolis             —          —           —           —             X                     —
www.gao.gov/cgi-bin/getrpt?GAO-03-369
                                                    New York                —           X           X          —            —                      —
To view the full report, including the scope        Philadelphia             X         —           —            X           —                      —
and methodology, click on the link above.           St. Louis               —          —           —           —            —                       X
For more information, contact Linda Koontz at
(202) 512-6240 or koontzl@gao.gov.              Source: GAO, GSA.
                                                a
                                                No price comparison was completed in calendar year 2001.
Contents


Letter                                                                                   1
               Results in Brief                                                          2
               Background                                                                4
               GSA Has Not Established a Consistent Process or Provided
                 Adequate Oversight                                                      7
               Numerous Fair Consideration Decisions Were Not Adequately
                 Documented                                                             14
               Use of Requests for Quotations Produced Mixed Results, but
                 Limited Data Preclude a Thorough Evaluation                            16
               Conclusions                                                              18
               Recommendations for Executive Action                                     19
               Agency Comments and Our Evaluation                                       20

Appendix I     Termination Charges and Service Initiation
               Charges                                                                  22



Appendix II    Objectives, Scope, and Methodology                                       24



Appendix III   Comments from the General Services
               Administration                                                           27



Tables
               Table 1: MAA Multiple-Award Cities in Which GSA Made Fair
                        Consideration Decisions in 2001                                  7
               Table 2: Comparison of Time Frames Used in MAA Contractor
                        Price Comparisons                                                9
               Table 3: Comparison of 1-Year and Life-Cycle Price Analyses for
                        Dallas                                                         10
               Table 4: Effect of Including Reconfiguration Charges in MAA
                        Contractor Price Comparisons (Dallas)                           12




               Page i                                        GAO-03-369 Telecommunications
Abbreviations

ASP       Aggregated Switch Procurement
FAR       Federal Acquisition Regulation
FTS       Federal Technology Service
GSA       General Services Administration
MAA       Metropolitan Area Acquisition
RFQ       Request for Quotations
SIC       service initiation charge



This is a work of the U.S. Government and is not subject to copyright protection in the
United States. It may be reproduced and distributed in its entirety without further
permission from GAO. It may contain copyrighted graphics, images, or other materials.
Permission from the copyright holder may be necessary should you wish to reproduce
copyrighted materials separately from GAO’s product.




Page ii                                                 GAO-03-369 Telecommunications
United States General Accounting Office
Washington, DC 20548



                                   April 4, 2003

                                   The Honorable Tom Davis
                                   Chairman
                                   Committee on Government Reform
                                   House of Representatives

                                   Dear Mr. Chairman:

                                   The Metropolitan Area Acquisition (MAA) program provides local
                                   telecommunications services to federal agencies in selected metropolitan
                                   areas. The MAA program manager, the General Services Administration
                                   (GSA), initiated this nationwide program in 1997 to achieve immediate,
                                   substantial, and sustained price reductions for agencies’ local
                                   telecommunications services; to expand agencies’ choices of high-quality
                                   services; and to encourage cross-agency sharing of resources. In 15 of 25
                                   metropolitan areas in which MAA contracts had been awarded as of
                                   January 2003, GSA had awarded contracts to two or more
                                   telecommunications providers. The intent of such multiple-award
                                   contracts is to sustain competition and obtain the best value on task
                                   orders awarded throughout the contract period. For these contracts, the
                                   Federal Acquisition Regulation requires agencies to provide the multiple-
                                   award contractors a fair opportunity to be considered for task orders; GSA
                                   refers to this process as its fair consideration process. Fair consideration
                                   decisions may be based on price alone, or they may be based on
                                   consideration of price plus other factors, such as technical requirements
                                   or the contractors’ past performance. This report responds to your request
                                   that we determine (1) whether GSA’s fair consideration process is
                                   consistent within and among metropolitan areas, and if not, whether or
                                   not inconsistencies affect the process results; (2) whether GSA’s
                                   documentation properly and appropriately supports its fair consideration
                                   decisions; and (3) whether the use of Requests for Quotations1 in the fair


                                   1
                                     Requests for Quotations are used in acquisitions to communicate government
                                   requirements to prospective contractors and to solicit quotations from them regarding
                                   price and other factors. In this report, the term “Request for Quotations” refers to a request
                                   from GSA to MAA contract vendors for price and other information. These Requests for
                                   Quotations are authorized by section G.2.1, “Service Price Quotes,” of the MAA contract, as
                                   follows: “The contractor shall provide price quotes for specific services and features when
                                   requested by the GDR or ADR prior to submitting a service order request. The price quote
                                   shall identify all recurring and nonrecurring charges, the service availability date, the date
                                   when the price quote will become nonbinding, and appropriate technical information that
                                   describes the service.”



                                   Page 1                                                    GAO-03-369 Telecommunications
                   consideration processes followed by GSA is cost-effective. In response to
                   concerns raised at an oversight hearing on the MAA program, you also
                   requested that we determine whether contract termination charges and
                   service initiation charges had affected GSA’s fair consideration decisions;
                   these results are reported in appendix I.

                   This report is based on work we performed at GSA’s Federal Technology
                   Service (FTS) headquarters and at our Washington, D.C., office, using
                   documentation furnished by GSA’s regional FTS offices. We reviewed all
                   fair consideration decisions made by GSA during calendar year 2001 as
                   part of the MAA contracts’ service ordering process, including supporting
                   documentation maintained by GSA, the MAA contracts, applicable federal
                   acquisition guidelines, and any further GSA guidance on this process. We
                   conducted our work from May 2002 through February 2003, in accordance
                   with generally accepted government auditing standards. Appendix II
                   contains a detailed discussion of our objectives, scope, and methodology.


                   Although GSA officials have stated that the nationwide MAA program
Results in Brief   should be consistently administered, GSA did not establish and follow a
                   consistent process when making its fair consideration decisions.2 During
                   calendar year 2001, fair consideration processes varied both within and
                   across MAA cities. Variations occurred in the lengths of time over which
                   contractor prices are compared and in the use of additional estimated
                   costs for changing telecommunications lines or service features over time.
                   Both variations influenced which contractors received task order awards
                   to provide services to agencies under these contracts. Further, because
                   GSA has not provided adequate oversight, regional offices in six instances
                   incorrectly used agency preference as a basis for selecting a contractor—
                   violating the Federal Acquisition Regulation—and selected a higher priced
                   incumbent service provider.

                   Of the fair consideration decisions made by GSA during calendar year
                   2001, 19 percent were not fully supported by documentation. According to
                   the Federal Acquisition Regulation, all contractual actions must be
                   documented in a manner sufficient to provide a basis for decisions
                   reached in the acquisition process, and to provide information for



                   2
                    Consistent with the request letter, we use the term “fair consideration” for the task order
                   award process established by the Federal Acquisition Regulation, Part 16, requiring that
                   each vendor be accorded “a fair opportunity to be considered for each order.”




                   Page 2                                                    GAO-03-369 Telecommunications
subsequent reviews of those decisions. Weaknesses observed included
lack of stated rationale for decisions reached, price comparisons that did
not support the choice of contractor selected by GSA, and lack of support
for technical factors used in making the decisions. These weaknesses were
allowed to occur because GSA did not establish and implement uniform
guidelines for documenting its MAA fair consideration decisions. As a
result, for 91 out of 483 decisions made in 2001, MAA stakeholders (GSA,
agencies, and MAA contractors) do not have assurance that the fair
consideration process was properly administered.

GSA’s use of Requests for Quotations in the fair consideration process
may be cost-effective for some local telecommunications services, but
limited data preclude a comprehensive evaluation. By pursuing this
additional competition, GSA may receive cost proposals from contractors
that could include more favorable terms than those in the published
contract, such as lower monthly prices, or waived or reduced service
initiation charges. Examination of this process in Denver revealed that
although it produced substantial savings when one type of service was
acquired, it did not result in savings when a second type of service was
obtained. GSA did not measure the costs and benefits of this process to
determine where the use of Requests for Quotations was most suitable for
acquiring local telecommunications services or to identify improvements
that could make the process more cost-effective.

In addition, we determined that the inclusion of contract termination
charges may have changed the choice of contractors in 5 of 16 GSA fair
consideration decisions in the one MAA city in which these charges
applied,3 and service initiation charges were a deciding factor in the choice
of contractors in 61 out of 272 decisions in seven cities (details are given in
app. I).

In light of the inconsistencies in the MAA fair consideration process, we
are recommending to the Administrator of General Services that GSA
establish and follow a uniform process for fair consideration. We are also
recommending that GSA develop and implement uniform guidelines for



3
 Three of these decisions were made based on price alone, and therefore would have been
directly affected. The two other decisions were made based on price and technical
considerations; it is unclear from the documentation, however, how the technical
considerations supported those decisions. Therefore, in those two cases, we are unable to
predict whether inclusion of termination charges resulted in a different choice of
contractor.




Page 3                                                  GAO-03-369 Telecommunications
             documenting its fair consideration decisions, and that it establish the
             measures needed to ensure the cost-effectiveness of its process and to
             provide a basis for improvement.

             In written comments on a draft of this report, the Administrator of General
             Services agreed with our recommendations and said that GSA was acting
             to implement them.


             The MAA program comprises contracts offering local voice and certain
Background   data telecommunications services to federal agencies in selected
             metropolitan areas across the country. GSA began planning this program
             just a few months after the passage of the Telecommunications Act of
             1996, which was intended to increase competition and reduce regulations
             in the telecommunications industry, particularly for local services.
             Recognizing that this competition would create an opportunity for the
             government to gain an immediate price reduction in local
             telecommunications services, GSA developed and launched the MAA
             program to take advantage of this emerging competition. Further, it
             envisioned the MAA contracts as a complement to existing local service
             contracts in metropolitan areas, as well as a solution for contracts that
             were expiring. As of January 2003, GSA had awarded MAA contracts in 25
             cities, with a total maximum value of $5.1 billion.

             Each MAA contract is a fixed-price, indefinite-delivery, indefinite-quantity
             contract with a base term of 4 years (48 months) from date of award, with
             four successive 1-year options. In 15 of the 25 MAA cities, GSA awarded
             these contracts to two or more telecommunications providers; such
             contracts are referred to as multiple-award contracts. The Federal
             Acquisition Streamlining Act (FASA) of 1994 established a preference for
             awarding contracts for indefinite quantities to multiple firms rather than to
             a single company. This approach was intended to provide agencies a
             means to procure goods and services quickly, using streamlined
             acquisition procedures, while obtaining the advantage of competition.

             Under multiple-award contracts, the Federal Acquisition Regulation (FAR)
             requires that contractors be afforded “a fair opportunity to be considered”
             in the subsequent award of task orders issued to meet specific agency
             needs under these contracts. The process used to ensure this opportunity
             is referred to as the fair consideration process. In administering this
             process, contracting officers are given broad latitude by the FAR.




             Page 4                                          GAO-03-369 Telecommunications
The MAA contracts give a broad outline of the procedure to be followed by
the government in conducting its fair consideration process.4 The
government- or agency-designated representative is to consult the latest
information about the contractors, including published contract prices,
related analyses that aid decisionmaking, information from contractors
such as price quotes or technical analyses, and other relevant information.
Using available information, the representative selects a contractor by one
of two methods: basing the decision solely on relative prices without
further consideration of other factors, or basing the decision on a
combination of price, technical, and past performance considerations
appropriate to the particular decision. After completing this decision
process, the representative then places a task order with the selected
contractor for the required telecommunications services.

GSA contracting officers making task order decisions can obtain price
information from published contracts, or they can choose to issue a
Request for Quotations (RFQ) as an additional mechanism for lowering
prices. Using an RFQ process to support fair consideration decision-
making can offer additional competitive benefits by allowing contractors
to lower prices beyond their contract offers and to reduce or waive service
initiation and other charges. However, while the RFQ process is being
pursued, agencies must continue to pay pre-MAA prices for
telecommunications services, rather than the lower MAA contract prices.

When making fair consideration decisions under the MAA contracts, in
addition to comparing monthly recurring charges for providing
telecommunications services, GSA may consider additional costs
associated with these services, such as the one-time termination charges
that may be associated with a pre-MAA telecommunications service
contract. GSA may also consider contractors’ service initiation charges5
for implementing service. When a contractor is also the incumbent, pre-
MAA telecommunications provider in a city, it does not generally include a
service initiation charge in its price quotes for existing lines and services
to be transitioned to an MAA contract, because it does not incur new



4
 The MAA service ordering process is outlined in Section G.2 of the MAA contracts,
including the prescribed procedure to give fair consideration to contractors for task order
awards under the contract.
5
 A service initiation charge is a charge to a customer by a telecommunications provider for
the initiation of a new telecommunications service, such as the installation of a new
telephone line.




Page 5                                                   GAO-03-369 Telecommunications
expenses to provide these lines and services. Where volume or types of
services differ from existing services, incumbents may include service
initiation charges in their pricing. All contractors can choose to waive
these charges.

Including these charges in fair consideration price comparisons may give
an advantage to an incumbent contractor (since the incumbent’s price
would generally not include such charges), but this advantage is
permissible. That is, acquisition case law has established that a contractor
may have unique advantages and capabilities (because of its prior
experience, for example), and the government is not required to equalize
competition or compensate for these advantages, unless there is evidence
of preferential treatment or other improper action.

GSA’s FTS headquarters in Fairfax, Virginia, and its regional offices in the
various metropolitan areas share responsibility for administering the MAA
contracts. According to testimony by the FTS Commissioner in June 2001,6
FTS headquarters is responsible for developing solicitations, evaluating
offers, awarding contracts, and supporting implementation activities, and
the FTS regional offices are responsible for developing city-specific
requirements and for contract implementation activities, including
managing the process used to select among MAA contract awardees for
placing task orders (that is, the fair consideration process).

In calendar year 2001, fair consideration processes were conducted in 11
of the 15 MAA cities with multiple-award contracts. Table 1 lists these 11
cities, their MAA contractors, and the number of fair consideration
decisions reached in 2001.




6
 Testimony of Sandra Bates, FTS Commissioner, General Services Administration, before
the Subcommittee on Technology and Procurement Policy (June 13, 2001).




Page 6                                                GAO-03-369 Telecommunications
                        Table 1: MAA Multiple-Award Cities in Which GSA Made Fair Consideration
                        Decisions in 2001

                                                            Number of fair
                            MAA multiple-award              consideration
                            cities                             decisions           Contractors
                            Atlanta                                     1          WinStar
                                                                                   BellSouth
                            Boston                                       21        AT&T
                                                                                   Southwestern Bell
                                                                                   Verizon
                                                                                   Winstar
                            Cleveland                                    71        Ameritech
                                                                                   AT&T
                            Dallas                                       20        AT&T
                                                                                   Southwestern Bell
                                                                                   WinStar
                            Denver                                      128        AT&T
                                                                                   Qwest
                                                                                   WinStar
                            Indianapolis                                 50        WinStar
                                                                                   AT&T
                                                                                   Ameritech
                            Los Angeles                                  44        WinStar
                                                                                   Pacific Bell
                            Minneapolis                                 118        WinStar
                                                                                   Qwest
                            New York                                     11        AT&T
                                                                                   Verizon
                            Philadelphia                                 10        AT&T
                                                                                   WinStar
                            St. Louis                                     9        WinStar
                                                                                   Southwestern Bell
                        Source: GSA.



                        GSA management has stated its desire to ensure consistency throughout
GSA Has Not             the nationwide MAA program, but it has not established a common
Established a           process for fair consideration decisions. Because GSA headquarters has
                        not developed or implemented a uniform method to be followed by its
Consistent Process or   regional offices in conducting the fair consideration process, approaches
Provided Adequate       vary among cities and, in some cases, within cities. Variations occurred in
                        the periods of time over which contractor prices were compared and in
Oversight               the inclusion of reconfiguration costs7 in price comparisons, which


                        7
                          Reconfiguration costs are associated with the need to move, add, or change telephone
                        lines, services, or features after they have been installed.




                        Page 7                                                  GAO-03-369 Telecommunications
affected the choice of contractors. Further, because GSA did not oversee
the application of this process, in some instances regional offices violated
the FAR by incorrectly using agency preference as a basis, in part or as a
whole, for selecting a higher priced contractor for an MAA task order.

Although the FAR gives contracting officers broad latitude in
administering the fair consideration process, GSA recognizes that the MAA
program and its contracts should be consistently managed and
administered. In her June 2001 testimony, the FTS Commissioner stated
that because MAA is a national program, communications and
coordination among GSA staff with MAA program responsibilities (FTS
headquarters, its regional offices, and MAA program management staff)
were essential to ensure program continuity and consistency.8 Thus, GSA
views consistency as an important attribute within the nationwide MAA
program.

The principal variation we identified in the fair consideration process
concerned the period of time selected by regional staff over which to
compare contractor prices. The different time periods that GSA regional
offices used for comparing contractor prices ranged from as short as
1 month to as long as the entire period remaining in the life of the contract
(GSA documentation referred to the latter as a life-cycle analysis). In three
cities—Cleveland, Indianapolis, and Minneapolis—GSA considered
contract life as the evaluation period. GSA’s Denver staff usually used a
1-month evaluation period,9 but it used a life-cycle analysis to justify 17
percent of its decisions. In four other cities, this price comparison varied
from decision to decision: specifically, the Dallas regional office alternated
between use of contract life and a 1-year period in its analyses; the New
York regional office used 1-year and 3-year periods; the Philadelphia staff
used a 1-month period and a 4-year period; and the Boston regional office
used a 1-month period in some cases, and in others considered savings for
both 1-year and 4-year periods. The reason for the specific comparison
period used was not recorded in decision documentation. Table 2
summarizes the different price comparison periods used.




8
 Testimony of Sandra Bates, FTS Commissioner, General Services Administration, before
the Subcommittee on Technology and Procurement Policy (June 13, 2001).
9
 Although Denver used a 1-month price comparison period for 83 percent of its fair
consideration decisions, it also identified life-cycle cost savings to customer agencies.




Page 8                                                     GAO-03-369 Telecommunications
Table 2: Comparison of Time Frames Used in MAA Contractor Price Comparisons

                                           Time frame used in price analysis
                                                                      Life Insufficient data
                                                                                            a
    City                  1 month       1 year 3 years 4 years cycle          to determine
            b
    Atlanta                    —            —        —         —        —                 —
    Boston                      X            X       —          X       —                 —
    Cleveland                  —            —        —         —         X                —
    Dallas                     —             X       —         —         X                —
    Denver                      X           —        —         —         X                —
    Indianapolis               —            —        —         —         X                —
    Los Angeles                —            —        —         —        —                  X
    Minneapolis                —            —        —         —         X                —
    New York                   —             X        X        —        —                 —
    Philadelphia                X           —        —          X       —                 —
    St. Louis                  —            —        —         —        —                  X
Source: GAO, GSA.

Note: GAO analysis of GSA data.
a
    Decision documentation did not identify the time frame used.
b
 One fair consideration decision was reached in Atlanta during calendar year 2001. An RFQ was
issued to the contractors to obtain a price quote for service. However, because a valid cost proposal
was received from only one contractor, a complete price comparison was not necessary.


Although the MAA contracts state that contract price will always be a
factor in GSA’s fair consideration procedure, they do not specify the time
period over which price comparisons should be made. However, a
consistent time period is important, because analyses over different time
periods may lead to different results. For example, one-time costs such as
service initiation charges have a less direct influence on decisions that
compare prices over a longer period of time, because those additional one-
time costs may be offset by lower prices over the life of the contract.
Conversely, when prices are compared over a shorter period of time, such
additional charges form a relatively greater portion of total costs
compared; thus, their inclusion can favor the incumbent service provider
(whose price does not generally include these charges).

The effect of using different time frames in price comparisons is illustrated
in table 3. This table summarizes the decisions reached in Dallas, where
Southwestern Bell was the incumbent telecommunications provider. If
GSA had used a contract life-cycle time frame in its price comparisons,
then the decisions reached might have been different in half of those




Page 9                                                             GAO-03-369 Telecommunications
                                              cases.10 Further, if it had consistently made these MAA task order awards
                                              to the contractors offering lower life-cycle prices, GSA could have realized
                                              an additional estimated $459,000 in savings for customer agencies in
                                              Dallas over the life of the contracts.

Table 3: Comparison of 1-Year and Life-Cycle Price Analyses for Dallas

              Lowest cost contractor according to analysis (checks indicate analysis used)                        Decision would have
                                                                                                                  differed if life-cycle price
 Customer               1st year price analysis     Life-cycle price analysis        Recipient of award           analysis had been used

 1                      3 Southwestern Bell           AT&T                           Southwestern Bell            Yes
 2                      3 Southwestern Bell           AT&T                           Southwestern Bell            Yes
 3                      3 Southwestern Bell           AT&T                           Southwestern Bell            Yes
 4                        Southwestern Bell         3 Southwestern Bell              Southwestern Bell            No
 5                      3 Southwestern Bell           AT&T                           Southwestern Bell            Yes
 6                      3 Southwestern Bell           AT&T                           Southwestern Bell            Yes
 7                      3 WinStar                     WinStar                        WinStar                      No
 8                        Southwestern Bell         3 Southwestern Bell              Southwestern Bell            No
 9                        Southwestern Bell         3 Southwestern Bell              Southwestern Bell            No
 10                       Southwestern Bell         3 Southwestern Bell              Southwestern Bell            No
                                                                        a                                                        a
 11                       AT&T                      3 Southwestern Bell              Southwestern Bell            Cannot predict
 12                       Southwestern Bell         3 AT&T                           AT&T                         No
 13                       WinStar                   3 WinStar                        WinStar                      No
 14                     3 Southwestern Bell           AT&T                           Southwestern Bell            Yes
 15                     3 Southwestern Bell           WinStar                        Southwestern Bell            Yes
 16                     3 Southwestern Bell           WinStar                        Southwestern Bell            Yes
 17                     3 Southwestern Bell           AT&T                           Southwestern Bell            Yes
 18                     3 WinStar                     WinStar                        WinStar                      No
 19                     3 WinStar                     WinStar                        WinStar                      No
 20                     3 Southwestern Bell           AT&T                           Southwestern Bell            Yes
Source: GAO, GSA.

                                              Note: GAO analysis of GSA data.
                                              Southwestern Bell was the incumbent telecommunications service provider.
                                              Check marks indicate the actual time frame used by GSA to compare contractor pricing.
                                              Analyses are ordered chronologically by date of first task order award issued following each
                                              associated fair consideration decision.
                                              a
                                               Fair consideration decision made by GSA based on price and technical consideration (agency
                                              preference).
                                              A secondary inconsistency within fair consideration processes concerned
                                              the use of reconfiguration costs (costs to move, add, or change telephone


                                              10
                                                In 16 of 20 cases, GSA cites price and technical considerations as the basis for its
                                              decisions.




                                              Page 10                                                        GAO-03-369 Telecommunications
lines, services, or features) in contractor price comparisons. Although
these charges are a part of all MAA contracts, only the Dallas regional
office included estimates of these costs in the MAA price comparisons
supporting its fair consideration decisions. Further, Dallas did not use
these charges consistently: they appeared in only half the price
comparisons completed, and they were calculated in two different ways.
(In most cases, GSA staff based estimates on the assumption that
10 percent of the telecommunications lines ordered would move locations,
add services, or change features during the course of a year; in two cases,
GSA staff assumed that services or features associated with 25 percent of
lines ordered would change annually.) The reason for the variations in the
use of these charges and in their estimates was not recorded in decision
documentation.

Table 4 shows that in 5 of 10 decisions made in Dallas, a different
contractor might have been awarded the task orders if reconfiguration
charges had not been included in price comparisons. (Three of these five
decisions were based on price alone. Two of these decisions were based
on both price and technical considerations, and so we cannot predict the
effect of excluding reconfiguration charges.)




Page 11                                       GAO-03-369 Telecommunications
Table 4: Effect of Including Reconfiguration Charges in MAA Contractor Price
Comparisons (Dallas)

                                                   Result if reconfiguration charge were not
                                                                    included
                    Actual result, including                                  Change in
    Customer        reconfiguration charge         Result                     outcome
    1               Southwestern Bell              AT&T                       Yes
    2               Southwestern Bell              AT&T                       Yes
    3               Southwestern Bell              AT&T                       Yes
                                                                                      a
    4               Southwestern Bell              AT&T                       Unknown
    5               AT&T                           AT&T                       No
    6               Winstar                        Winstar                    No
                                                                                      a
    7               Southwestern Bell              AT&T                       Unknown
    8               Southwestern Bell b            Southwestern Bell          No
    9               Southwestern Bell b            Southwestern Bell          No
    10              Southwestern Bell              Southwestern Bell          No
Source: GAO, GSA.

Note: GAO analysis of GSA data.
Southwestern Bell was the incumbent telecommunications service provider. Comparisons are
ordered chronologically by date of first task order award issued following each associated fair
consideration decision.
a
    Task order awarded based on price and technical considerations.
b
 Analysis used reconfiguration estimate based on 25 percent of lines. In all other cases, the basis
was 10 percent of lines.


These variations exist because GSA has not established a common process
for making fair consideration decisions. GSA did provide guidance on the
fair consideration process to be followed in MAA cities following each
contract award. This guidance provided general background, including the
basis of the requirement in the Federal Acquisition Streamlining Act of
1994; the exceptions to this requirement; and the procedure to be followed
for issuing an MAA task order. However, it did not outline a common
process or identify common costs to be considered other than contract
prices. Rather, according to a GSA official, GSA permitted its regional
offices to define and follow their own fair consideration processes in order
to encourage innovation and to learn which process yields the best results.
However, GSA headquarters has taken no action that would permit it to
learn from any such experiences in order to address inconsistencies and
determine the most suitable process. As a result, variations occur in the
application of the fair consideration process, contrary to GSA’s stated
interest in ensuring the consistency of the MAA program.




Page 12                                                         GAO-03-369 Telecommunications
Further, these variations led to a lack of transparency that could hinder
GSA’s ability to obtain full value from the fair consideration process.
Specifically, although the MAA contracts outline the fair consideration
procedure in broad terms, they do not outline specific aspects of the
process that vary across MAA cities, such as costs that may be considered
by the government in addition to the price of services, or the length of time
that may be used to compare contractor prices. Because these aspects of
the process are not disclosed, the contractors may find it difficult to
determine their most competitive offers.

GSA’s lack of oversight also hampered its ability to ensure that its fair
consideration processes always complied with appropriate federal
acquisition guidelines. Specifically, in six instances GSA violated the FAR
by incorrectly using agency preference as a basis, in part or as a whole, for
selecting a contractor for an MAA task order. According to this regulation,
designating preferred awardees is not permissible in the award of task
orders valued at more than $2,500. For orders that exceed that threshold,
all contractors in a multiple-award contract must be given a fair
opportunity to be considered. However, we identified six cases in Boston,
Dallas, Denver, and Indianapolis where decisions explicitly cited agency
preference as a factor for choosing a contractor for task orders above the
$2,500 threshold.11 These violations of the FAR were allowed to occur
because GSA did not provide adequate oversight to ensure that staff were
correctly applying regulations when conducting fair consideration
processes. As a result, the integrity of the fair consideration process could
not be ensured, and potential savings were lost. In all six cases the
incumbent provider was selected, which was also the higher priced service
provider; selecting the lowest priced contractor in these cases would have
yielded $76,000 in additional estimated cost savings to those agencies over
the life of the contracts.




11
  We identified two additional cases in Boston citing agency preference as a factor;
however, because adequate documentation was not maintained, we are unable to
determine what services were ordered and therefore whether those decisions breached the
$2,500 threshold.




Page 13                                               GAO-03-369 Telecommunications
                         About one-fifth of GSA’s fair consideration decisions were not adequately
Numerous Fair            documented. According to the FAR, documentation of all contractual
Consideration            actions must be maintained in a manner sufficient to provide a basis for
                         decisions reached in the acquisition process, and to provide information
Decisions Were Not       for subsequent reviews of those decisions.12 Out of 483 fair consideration
Adequately               decisions from regional GSA offices in the 11 cities that we assessed, the
                         documentation furnished for 91 (19 percent) did not adequately support
Documented               the task order award that was made.

                         One or more of the following four weaknesses were present in
                         documentation for these 91 decisions:

                     •   An explanation of how the fair consideration decision was made was
                         absent. Although contracting offices are required by the FAR to maintain
                         documentation sufficient to constitute a complete history of contracting
                         actions, this documentation was not available in 66 of the 483 decisions
                         assessed. Specifically, the documentation of decisions reached during
                         calendar year 2001 in Boston, Denver, Indianapolis, Los Angeles, New
                         York, and St. Louis did not include a stated rationale for the decisions
                         reached. As a result, in these cases it is not possible to determine whether
                         or not the fair consideration decisions reached by GSA were justified. A
                         GSA program officer stated that a decision rationale was not prepared for
                         53 decisions in Los Angeles and St. Louis that were based on price alone
                         because of a lack of clarity pertaining to documentation requirements for
                         those cases. She also stated that 5 decisions in New York were not
                         documented because of urgency, as those decisions were made shortly
                         after the September 11th terrorist attacks in that city. Further, the GSA
                         program officer stated that a decision rationale was not prepared for the
                         balance of decisions because of administrative oversight.

                     •   The price analysis did not support the decision reached. As part of fair
                         consideration decisionmaking, GSA usually included in each task order
                         file an analysis that compared the service prices offered by each
                         contractor over some defined period of time. In Dallas, Denver, and
                         Indianapolis, eight task order awards were justified by price comparisons
                         that did not support the decisions reached. Although price was the sole
                         factor considered in these cases, the lowest priced contractors, as
                         revealed by the price comparisons, were not awarded task orders. GSA’s


                         12
                           Contract documentation requirements are outlined in section 4.801 of the FAR. The FAR
                         and Office of Federal Procurement Policy guidance require that task order awards be
                         documented adequately in the contract file to provide a history of the transaction and a
                         complete background for informed decisions at each step in the acquisition process.




                         Page 14                                                GAO-03-369 Telecommunications
    decision documentation does not explain why these awards were made to
    higher priced contractors.

•   Technical factors were cited but not supported. According to the
    procedure defined in the MAA contracts, fair consideration decisions may
    be based on price or on a combination of price, technical factors, and past
    performance. How these technical factors are weighted depends on the
    particular circumstances of each decision. Technical factors were cited as
    the reason for fair consideration decisions, either as a whole or in part, in
    four MAA cities: Boston, Dallas, Indianapolis, and New York. For three of
    these cities, Boston, Dallas, and Indianapolis, we were not able to
    determine how these technical factors were applied to support a total of
    20 decisions. Contract documentation for these 20 decisions included a
    statement that both price and technical factors were considered. However,
    in one case, the specific factor considered was not identified. In the other
    19 cases, the specific technical factor was identified, but the
    documentation did not specify how it supported the decision reached.

•   Other documentation weaknesses were identified in Boston.
    Documentation prepared to support fair consideration decisions in Boston
    contained two additional weaknesses. In three cases, the decision
    documentation suggests that not all MAA contractors were included in
    those fair consideration evaluations, but it does not indicate why not all
    contractors were considered. Further, the documentation in three other
    decisions indicates that price was not considered in these cases, although
    price must be considered in all fair consideration decisions.

    In three regional offices, GSA has taken some action to improve its fair
    consideration documentation. Specifically, in response to concerns that
    we initially raised during our prior review of early MAA contract
    implementation efforts,13 GSA improved its decision documentation in
    Cleveland and Indianapolis by including additional analyses and clarifying
    memorandums in those contract files. In addition, Denver staff have also
    taken action to correct their files.

    Nevertheless, GSA has yet to take nationwide action to improve its fair
    consideration documentation. Currently, weak documentation of fair
    consideration decisions makes it difficult to determine the basis upon



    13
      The results of that review were reported earlier: U.S. General Accounting Office,
    Telecommunications: GSA Action Needed to Realize Benefits of Metropolitan Area
    Acquisition Program, GAO-02-325 (Washington, D.C.: Apr. 4, 2002).




    Page 15                                                 GAO-03-369 Telecommunications
                        which a contractor was selected for a task order. These problems were
                        permitted to occur because GSA did not establish uniform guidelines to
                        ensure that all regional offices were documenting fair consideration
                        decisions in a manner consistent with the FAR. As a result, in 19 percent of
                        the cases we reviewed, GSA, customer agencies, MAA contractors, and the
                        Congress do not have assurance that the procedure followed by GSA to
                        award MAA task orders was properly applied.


                        In MAA multiple-award cities, GSA attempted to reduce the cost of
Use of Requests for     telecommunications services by asking contractors to submit price
Quotations Produced     quotations to compete for task orders. However, the process had mixed
                        results in the only MAA city in which we could do a partial evaluation. (We
Mixed Results, but      could not do a comprehensive evaluation because of limitations in
Limited Data Preclude   documentation.) For acquiring some types of services, substantial savings
                        were realized by the use of Requests for Quotations (RFQ), but for others,
a Thorough              the savings were not sufficient to offset the cost of paying pre-MAA prices
Evaluation              during the time taken to complete the process.

                        GSA’s most common approach to selecting contractors for task orders was
                        to issue an RFQ, rather than basing decisions on published contract prices
                        alone: specifically, GSA followed this process in 347 out of the 394
                        decisions reached in the 11 MAA cities in which fair consideration
                        processes were conducted.14 Of the 11 cities, GSA issued RFQs to support
                        fair consideration in 8. (In the 3 other cities—Boston, Los Angeles, and St.
                        Louis—the documentation was not sufficient to determine whether GSA
                        issued RFQs.) Denver was the only city, of the 11 reviewed, where GSA
                        staff documented their actions while reaching most fair consideration
                        decisions. As a result, Denver was the only city where we could make an




                        14
                          Documentation in 89 cases was not sufficient to determine whether or not an RFQ
                        process was used.




                        Page 16                                               GAO-03-369 Telecommunications
assessment (although still partial15) of the cost-effectiveness of the RFQ
process. We were not able to comprehensively evaluate the cost-
effectiveness of the RFQ process across all 11 MAA cities, or to partly
assess processes in any other city, because the documentation maintained
was not sufficient for that purpose.

Despite their limitations, data available on the RFQ process in Denver
indicate substantial differences in savings realized, depending on the types
of local telecommunications services acquired; therefore, this process may
not be cost-effective for acquiring all types of local telecommunications
services. GSA used its MAA contracts in Denver to acquire two types of
services: one type is a large telephone line, known as a trunk, which is
used to interconnect a customer-owned switch, called a private branch
exchange (PBX),16 to the contractor’s network; the second type is a voice
telephone line served by a switch that is owned and operated by the
contractor. According to our evaluation of Denver’s records for 119 fair
consideration decisions17 that preceded award of task orders for these two
types of services, the benefit of the RFQ process (over the life of the
contracts) varied between the two. Specifically, in five of the eight cases
where GSA sought to buy PBX trunks, the MAA contractor waived or
reduced service initiation charges, reduced its monthly recurring cost, or
both (no additional benefits were derived by this process in the three




15
  In general, the complete RFQ process has four segments. In the first segment, GSA
prepares and issues the RFQ to the MAA contractors. In the second, the contractors
prepare and submit their RFQ responses. In the third, GSA reviews and accepts the RFQ
responses. Finally, GSA compares contractor prices and awards task orders. The
documentation in 40 of 121 cases in Denver was not sufficient to allow a complete
assessment, because it did not permit us to determine the time it took to complete all
discrete segments of the process, such as the time to compare contractor prices, which
would be essential to a complete assessment. Because of this limitation, we could not
comprehensively evaluate this process, nor can we present results in precise terms. In 2 of
the 121 decisions, sufficient data were not available to permit any analysis. The available
data for the other 119 decisions do suffice, however, to permit general observations.
Additional information on our methodology is disclosed in appendix II.
16
   A private branch exchange (PBX) is a communications switching system serving an
organization and normally located on the organization’s premises.
17
  Denver made 128 fair consideration decisions, but they issued RFQs to contractors in
only 121 of those decisions. In 2 of those 121 decisions, sufficient data were not available to
permit any analysis.




Page 17                                                    GAO-03-369 Telecommunications
              other cases). For PBX services, the net savings were substantial—an
              estimated $790,000.18

              In contrast, for switched voice services, the benefits of using the RFQ
              process were less substantial, and the available data suggest that they
              were not sufficient to offset the savings deferred while the process was
              completed. In 23 of 111 cases, additional cost or price reductions were
              obtained in the form of waived or reduced service initiation charges that
              did offset the value of savings deferred. However, in 88 cases, the benefit
              realized did not offset the deferred savings. Thus, using the RFQ process
              to acquire all switched voice services instead of taking immediate
              advantage of low MAA contract prices was not cost-effective.

              GSA was not able to maximize the value of the RFQ process for the benefit
              of its customers, however, because it did not institute performance
              measures that would allow it to gauge cost-effectiveness and determine
              where the RFQ process would be most suitable for acquiring local
              telecommunications services, or that would aid in identifying where its
              processes could be improved. Furthermore, because adequate
              documentation of fair consideration decisions was not maintained
              throughout this program, GSA does not have the data it would need to
              evaluate its processes throughout its MAA cities and to make
              improvements. As a result, GSA is unable to direct the most suitable and
              cost-effective use of the RFQ process in the administration of its MAA
              contracts.


              Inconsistencies in GSA’s process and practices for determining how it
Conclusions   awards MAA task orders to its contractors are hampering its ability to


              18
                To estimate net savings, we identified those fair consideration decisions where the
              government benefited either through waived or reduced service initiation charges or
              through further reductions in contract pricing, and we estimated the total value of these
              benefits. In all cases, we then estimated the value of monthly savings deferred until the
              RFQ process was completed. We evaluated the difference between these two figures to
              determine the net cost or net savings associated with the RFQ process. In 40 of the 119
              decisions evaluated, complete data were not available regarding the final segment of this
              process (3 cases pertaining to PBX service and 37 cases pertaining to switched voice
              services). In those cases, we assumed that the process was completed at the same time that
              GSA received the final contractor RFQ response. Because this assumption may reflect a
              shorter time period than use of an actual process completion date for these cases, the
              effect of this assumption is to minimize the value of savings deferred, and as a result, our
              estimates may overstate the potential net savings and may understate any potential net
              loss. The value of net savings is expressed in constant year 2001 dollars.




              Page 18                                                  GAO-03-369 Telecommunications
                      appropriately administer these contracts; because these inconsistent
                      processes are not transparent, contractors may not be able to compete
                      most effectively. In addition, weaknesses in documenting fair
                      consideration decisions and inadequate oversight of the process
                      undermine GSA’s ability to assure customer agencies, MAA contractors,
                      and the Congress that the procedures it followed to award MAA task
                      orders were always appropriately and fairly applied. Further, because it
                      did not establish measures that would enable it to learn from the fair
                      consideration experiences in its regional offices, GSA was unable to gauge
                      the cost-effectiveness of RFQ processes so that it could make the most
                      suitable and effective use of RFQs to acquire local telecommunications
                      services. As a result, GSA cannot provide assurance that its MAA fair
                      consideration processes are sound, and that they provide the government
                      the maximum benefit from the MAA contracts.


                      We recommend that the Administrator of General Services establish a
Recommendations for   common process for GSA to consistently follow in reaching its fair
Executive Action      consideration decisions under the MAA contracts, and that he direct the
                      FTS Commissioner to oversee the proper application of this process. This
                      common process should include a uniform time frame for comparing MAA
                      contractor prices, and it should specify the cost elements (such as
                      reconfiguration costs) to be considered in those comparisons. Should
                      local conditions warrant deviation from this common process, we
                      recommend that GSA document these deviations and communicate them
                      to GSA’s MAA contractors, so that all MAA stakeholders have a clear and
                      consistent understanding of the process being followed. This process
                      should include the management oversight necessary to ensure adherence
                      to the FAR prohibition against the use of agency preference in decisions
                      on task orders valued at more than $2,500.

                      We also recommend that the Administrator of General Services direct the
                      FTS Commissioner to establish and apply uniform guidelines for
                      documenting fair consideration decisions that are sufficient to ensure that
                      GSA appropriately reaches its decisions to award task orders. For each
                      fair consideration decision, this documentation should include

                  •   the rationale for the decision;
                  •   the supporting contractor price comparison; and
                  •   support for other factors considered in reaching the decision, such as
                      technical and past performance considerations, as appropriate.




                      Page 19                                        GAO-03-369 Telecommunications
                     We further recommend that the Administrator of General Services direct
                     the FTS Commissioner to develop performance measures to determine
                     when the RFQ process best achieves program goals. Once GSA has
                     outcomes for these measures, it should evaluate the results of its RFQ
                     process to identify potential improvements and to determine its most
                     suitable and cost-effective use.


                     In written comments on a draft of this report, the Administrator of General
Agency Comments      Services agreed with our recommendations and indicated that GSA was
and Our Evaluation   acting to implement them. Specifically, GSA has created draft guidance on
                     its fair consideration process under MAA procurements, and it plans to
                     disseminate this guidance to all its FTS regional offices by April 11, 2003.
                     According to the Administrator, this guidance addresses all our
                     recommendations and will ensure consistency in the fair consideration
                     process and supporting documentation. GSA also provided technical
                     comments that we have incorporated into this report as appropriate.
                     GSA’s written comments are presented in appendix III.


                     As agreed with your office, unless you publicly announce the contents of
                     this report earlier, we will not distribute it until 30 days from its issue date.
                     At that time, we will send copies of this report to the Ranking Minority
                     Member, Committee on Government Reform, and interested congressional
                     committees. We will also send copies to the Director of the Office of
                     Management and Budget and the Administrator of the General Services
                     Administration. Copies will be made available to others on request. In
                     addition, this report will be available at no charge on our Web site, at
                     http://www.gao.gov.




                     Page 20                                           GAO-03-369 Telecommunications
Should you or your staff have any questions on matters discussed in this
report, please contact me at (202) 512-6240. I can also be reached by E-
mail at koontzl@gao.gov. Other key contributors to this report are Scott
Binder, Harold Brumm, Barbara Collier, Kevin E. Conway, Frank Maguire,
Mary Marshall, Charles Roney, and Michael Stiltner.

Sincerely yours,




Linda D. Koontz
Director, Information Management Issues




Page 21                                      GAO-03-369 Telecommunications
              Appendix I: Termination Charges and Service
Appendix I: Termination Charges and Service
              Initiation Charges



Initiation Charges

              At your request, we evaluated the effect of contract termination charges
              and contract service initiation charges (SIC)19 on the results of fair
              consideration decisions for awarding task orders in cities with multiple-
              award Metropolitan Area Acquisition (MAA) contracts. Under acquisition
              contract law, agencies are permitted to consider termination charges and
              SICs in contract award decisions.

              Our analysis of termination charges showed that they had minimal impact
              on fair consideration decisions. The General Services Administration
              (GSA) included these charges in 16 contract price comparisons in Dallas,
              the only city in which they were relevant.20 Termination charges had a
              direct effect on 5 of 16 fair consideration decisions; in other words, those 5
              decisions might have resulted in an award to another contractor if Dallas
              had not included the termination charges as part of its contract price
              comparisons.21

              Our analysis of SICs showed that they were included as a cost factor in
              GSA’s fair consideration price comparisons in 7 of 11 MAA cities: Boston,
              Cleveland, Dallas, Indianapolis, Minneapolis, New York, and Philadelphia.
              In these 7 cities there were 272 fair consideration decisions that included
              SICs in price comparisons, and SICs were a deciding factor in 61 of these
              decisions. That is, if SICs had not been part of GSA’s analyses, the award
              decision would have been issued to another contractor in 22 percent of
              those decisions. SICs did not affect more decisions primarily because of
              the period of time used in comparing contractor prices. In 90 percent of
              fair consideration decisions where SICs were identified as a cost factor,
              GSA used the contract life-cycle period as the evaluation time frame. When
              SICs were amortized over the life of the contract, these charges were
              usually not large enough to influence fair consideration decisions.



              19
                Service initiation charges are charges to a customer by a contractor for the initiation of a
              new telecommunications service, such as the acquisition of a new line or a new feature.
              20
                Contract termination charges were a factor in Dallas because termination charges were
              part of a Southwestern Bell Aggregated Switch Procurement (ASP) contract that was
              awarded before the MAA contracts. That contract imposes a $25.90 charge to disconnect
              each line from the ASP service when the customer transitions to a contractor other than
              Southwestern Bell. (Specifically, the contract identifies a charge of $19.92 per line. After
              applying its surcharge to that amount, GSA charges the agency a disconnect charge of
              $25.90.)
              21
                Three of these decisions were based on price alone, and two were based on price and
              technical considerations. It is unclear from the decision documentation, however, how
              technical considerations supported these decisions.




              Page 22                                                    GAO-03-369 Telecommunications
    Appendix I: Termination Charges and Service
    Initiation Charges




    For the balance of the decisions, SICs were not included in price
    comparisons, the treatment of SICs could not be determined, or cost
    comparisons were not completed:

•   In all 44 decisions in Los Angeles and in 9 decisions in Boston, no SICs
    were included in fair consideration price comparisons.

•   Contract price comparisons were not completed to support four fair
    consideration decisions in Boston, one in Philadelphia, and one in Atlanta,
    because only one MAA contractor was responsive to their RFQs. A
    contractor price comparison was also not completed to support one fair
    consideration decision in New York City, where the award was based on
    the urgency of the requirement.

•   In all nine decisions in St. Louis, seven decisions in Boston, five decisions
    in New York City, and two decisions in Indianapolis, we could not
    determine whether SICs were used in making fair consideration decisions
    because the documentation was not sufficient to permit us to make that
    determination.

    In the remaining MAA city, Denver, SICs were not included in comparisons
    of contractor prices; only monthly recurring charges were included for
    comparison purposes. However, in 22 of its 128 price comparisons in
    Denver, GSA did cite the value of SICs in recommending that task order
    awards be made to contractors with a higher monthly recurring cost for
    services, because the agency could save more money over the contract
    life; we verified the accuracy of those analyses. To show the total cost of
    the MAA service in those cases where the recommended contractor
    charged SICs, GSA staff in Denver disclosed these charges as a separate
    item in the decision results presented to customer agencies. This
    disclosure was an amortization analysis indicating the time that it would
    take for monthly MAA contract savings to amortize that one-time cost.




    Page 23                                         GAO-03-369 Telecommunications
               Appendix II: Objectives, Scope, and
Appendix II: Objectives, Scope, and
               Methodology



Methodology

               In our review of the Metropolitan Area Acquisition (MAA) contracts
               managed by the General Services Administration (GSA), our objectives
               were to determine (1) whether GSA’s fair consideration process varies
               within or among cities, and if so, whether or not variations affect the
               process results; (2) whether GSA’s documentation properly and
               appropriately supports its fair consideration decisions; and (3) whether
               the use of Requests for Quotations in the fair consideration processes
               followed by GSA is cost-effective. We also determined whether contract
               termination charges and service initiation charges affected GSA’s fair
               consideration decisions.

               We reviewed all fair consideration decisions made by GSA during calendar
               year 2001 as part of the MAA contracts’ service ordering process. To
               determine whether there were variations in the fair consideration process,
               we reviewed the fair consideration procedure outlined in the MAA
               contracts, applicable acquisition guidelines, and GSA Federal Technology
               Service guidance and direction on the fair consideration procedure. Using
               the decision documentation maintained by the GSA regional offices, we
               then reviewed the steps taken by those offices in conducting their fair
               consideration processes. We gathered and assessed documentary and
               testimonial explanations for variations within or among MAA regional
               offices in their fair consideration processes.

               To determine whether GSA’s documentation properly and appropriately
               supports its fair consideration decisions, we reviewed appropriate federal
               contract administration guidelines including the Federal Acquisition
               Regulation (sections 4.801 and 16.505) and Office of Federal Procurement
               Policy guidance (May 4, 1999, Memorandum for Agency Senior
               Procurement Executives regarding “Competition and Multiple Award Task
               and Delivery Order Contracts”), as well as MAA contract language
               outlining the fair consideration procedure. We then reviewed the
               documentation maintained by GSA’s regional offices to support fair
               consideration decisions. This documentation included, where available,
               contractor price information, GSA’s analyses that compared contractor
               prices, and other memorandums supporting and documenting the decision
               process. We also reviewed the task order documentation prepared by GSA
               following its fair consideration decision, in order to match the service
               ordered with the decision reached.

               To determine whether the use of Requests for Quotations (RFQ) in the fair
               consideration processes followed by GSA is cost-effective, we reviewed
               regional offices’ documentation to determine whether data were available
               that would permit the evaluation of time taken to complete the fair


               Page 24                                       GAO-03-369 Telecommunications
Appendix II: Objectives, Scope, and
Methodology




consideration process. Following our review of the documentation, we
selected Denver as a multiple-award MAA city where the documentation
available was sufficient to permit evaluation of key aspects of that
process, such as determining the approximate time taken to complete the
RFQ process and the benefits derived from that additional competition.
Specifically, we used data regarding the RFQ issue date and the date when
Denver completed its analysis to determine the time taken to complete
that process in the 79 decisions where those data were available. In the 40
other decisions where data were not available regarding the complete
process (3 cases pertaining to PBX service and 37 cases pertaining to
switched voice services), we assumed that the process was completed at
the same time that GSA received the final contractor RFQ response.22 We
also examined documentation to determine, for each decision, any
additional benefit realized as a result of the RFQ process, such as lowered
monthly prices or waived or reduced service initiation charges. Where
data were available to determine whether the value of benefits derived
from the RFQ process justified the time required to complete that process,
we compared pre-MAA prices for services with MAA prices, in order to
calculate a baseline savings provided by the MAA contracts. We then
evaluated the cost of the RFQ process exclusively in terms of the monthly
savings that were deferred until the fair consideration process was
completed. We did not include the labor cost expended by GSA or its MAA
contractors in completing this process.

To determine the effect of contract termination charges and service
initiation charges on fair consideration decision results, we reviewed the
price comparisons that were prepared by GSA regional offices. Where
applicable, we calculated prices for services with and without these
additional charges to determine whether the decision supported was
influenced by these charges.



22
  Because this assumption may reflect a shorter time period for these cases, the effect of
this assumption is to minimize the time taken to complete the RFQ process, and therefore
the value of savings deferred during that period. As a result, our estimates may overstate
the potential net savings and may understate any potential net loss.




Page 25                                                  GAO-03-369 Telecommunications
Appendix II: Objectives, Scope, and
Methodology




We conducted our review from May 2002 through February 2003, in
accordance with generally accepted government auditing standards.




Page 26                                     GAO-03-369 Telecommunications
              Appendix III: Comments from the General
Appendix III: Comments from the General
              Services Administration



Services Administration




              Page 27                                   GAO-03-369 Telecommunications
           Appendix III: Comments from the General
           Services Administration




(310341)
           Page 28                                   GAO-03-369 Telecommunications
                         The General Accounting Office, the audit, evaluation and investigative arm of
GAO’s Mission            Congress, exists to support Congress in meeting its constitutional responsibilities
                         and to help improve the performance and accountability of the federal
                         government for the American people. GAO examines the use of public funds;
                         evaluates federal programs and policies; and provides analyses,
                         recommendations, and other assistance to help Congress make informed
                         oversight, policy, and funding decisions. GAO’s commitment to good government
                         is reflected in its core values of accountability, integrity, and reliability.


                         The fastest and easiest way to obtain copies of GAO documents at no cost is
Obtaining Copies of      through the Internet. GAO’s Web site (www.gao.gov) contains abstracts and full-
GAO Reports and          text files of current reports and testimony and an expanding archive of older
                         products. The Web site features a search engine to help you locate documents
Testimony                using key words and phrases. You can print these documents in their entirety,
                         including charts and other graphics.
                         Each day, GAO issues a list of newly released reports, testimony, and
                         correspondence. GAO posts this list, known as “Today’s Reports,” on its Web site
                         daily. The list contains links to the full-text document files. To have GAO e-mail
                         this list to you every afternoon, go to www.gao.gov and select “Subscribe to daily
                         E-mail alert for newly released products” under the GAO Reports heading.


Order by Mail or Phone   The first copy of each printed report is free. Additional copies are $2 each. A
                         check or money order should be made out to the Superintendent of Documents.
                         GAO also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a
                         single address are discounted 25 percent. Orders should be sent to:
                         U.S. General Accounting Office
                         441 G Street NW, Room LM
                         Washington, D.C. 20548
                         To order by Phone:     Voice:    (202) 512-6000
                                                TDD:      (202) 512-2537
                                                Fax:      (202) 512-6061


                         Contact:
To Report Fraud,
                         Web site: www.gao.gov/fraudnet/fraudnet.htm
Waste, and Abuse in      E-mail: fraudnet@gao.gov
Federal Programs         Automated answering system: (800) 424-5454 or (202) 512-7470


                         Jeff Nelligan, managing director, NelliganJ@gao.gov (202) 512-4800
Public Affairs           U.S. General Accounting Office, 441 G Street NW, Room 7149
                         Washington, D.C. 20548