oversight

Acquisition Reform: NASA's Internet Service Improves Access to Contracting Information

Published by the Government Accountability Office on 1999-02-09.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                  United States General Accounting Office

GAO               Report to Congressional Committees




February 1999
                  ACQUISITION
                  REFORM
                  NASA’s Internet Service
                  Improves Access to
                  Contracting
                  Information




GAO/NSIAD-99-37
                   United States
GAO                General Accounting Office
                   Washington, D.C. 20548

                   National Security and
                   International Affairs Division

                   B-280239

                   February 9, 1999

                   Congressional Committees

                   In response to the Federal Acquisition Streamlining Act (FASA) of 1994, we
                   reviewed the National Aeronautics and Space Administration’s (NASA)
                   Acquisition Internet Service (NAIS).1 Specifically, we determined whether
                   NAIS was an effective mechanism for disseminating procurement
                   information to industry, including small businesses. In addition, we
                   identified the status of efforts to develop a governmentwide electronic
                   procurement information system similar to NAIS.

                   In 1993, NASA began an initiative called the Midrange Pilot Program for
                   procurements between $25,000 and $500,000.2 The purpose of the program
                   was to test new processes and tools to streamline the acquisition process
                   while maintaining or improving the quality of procurements. A key
                   objective of the program was to develop and test an alternative to paper
                   documents as the primary means of communicating procurement
                   information to prospective contractors. After researching available
                   options, NASA chose to implement an Internet-based electronic
                   procurement notice and publication system, which ultimately became NAIS.

                   In December 1996, the Department of Commerce, in partnership with the
                   Government Printing Office, introduced CBDNet, an Internet version of
                   the Commerce Business Daily (CBD),3 which is available to the public free
                   of charge. Many other agencies have developed a variety of other
                   electronic posting systems to help improve access to, and the visibility of,
                   their contracting opportunities.


                   NAIS is a simple, effective, and user-friendly system for disseminating
Results in Brief   information on contract opportunities. NAIS has contributed to the
                   development of a more standardized and streamlined acquisition process
                   at NASA and provides a central electronic source of procurement
                   information from NASA’s decentralized facilities. It allows businesses to
                   obtain procurement information immediately, without waiting for mail

                   1
                    P.L. 103-355, section 5062, authorized NASA to conduct a test of electronic notice and publication for
                   procurements and requires GAO to report on the results of NASA’s test.
                   2
                    In fiscal year 1997, NASA increased the threshold to $2 million. NASA officials estimate that 80 to
                   90 percent of all NASA’s annual contract awards are in the midrange category.
                   3
                    The CBD gives a brief description (or synopsis) of federal acquisitions over $25,000. The primary
                   purposes of CBD notices are to improve small businesses’ access to acquisition information and
                   enhance competition by identifying contracting and subcontracting opportunities.



                   Page 1                                                        GAO/NSIAD-99-37 Acquisition Reform
             B-280239




             delivery of printed information. Vendors especially like the electronic mail
             (e-mail) notification service that automatically sends announcements
             about procurements of interest to them. Vendor feedback about NAIS came
             primarily from small businesses and was generally positive. Procurement
             data showed that offers and awards to small businesses did not change
             significantly after NAIS implementation. NASA noted that data limitations
             made it difficult to quantify other NAIS benefits.

             NASA, the General Services Administration (GSA), and other federal
             agencies are working together to develop a single, governmentwide
             Internet entry point for information on federal procurement opportunities.
             But a number of steps must still be taken and many obstacles remain.
             Even if the new system is successfully developed and implemented
             governmentwide, current statutory requirements for publication of
             procurement notices and minimum waiting periods for mail delivery may
             continue to limit the potential benefits of an electronic procurement
             information system. The same legislation that encouraged NASA and others
             to work together also requires the Office of Federal Procurement Policy to
             submit to Congress annual reports assessing compliance with the
             requirement to provide direct access to procurement information through
             a single governmentwide electronic entry point.


             In the early 1990s, federal agencies and Congress recognized that doing
Background   business electronically could save resources, streamline the procurement
             process, and improve the private sector’s access to federal contracting
             opportunities. NASA first used NAIS at its Marshall Space Flight Center in
             1994 to post midrange procurement opportunities on the Internet. NAIS
             services and features expanded rapidly, and the system began to be used
             NASA-wide in 1995. It was the first agencywide procurement information
             system on the Internet.

             With some exceptions, federal agencies must publish notices of
             competitive contracting opportunities above $25,000 in the CBD. To
             accommodate timely delivery of notices through a paper-based mail
             delivery system, the law generally prescribes a 15-day minimum interval
             between publication of a notice in the CBD and the issuance of a
             solicitation by an agency. After a solicitation is issued, agencies generally
             must give contractors a minimum of 30 days to prepare and deliver bids or
             offers. In support of NASA’s test of alternatives to the use of paper
             documents for communicating business opportunities to prospective
             contractors, FASA authorized NASA to waive the CBD notice and waiting



             Page 2                                       GAO/NSIAD-99-37 Acquisition Reform
B-280239




period requirements for 4 years (until Oct. 13, 1998) and for up to
$100 million in procurement life-cycle costs.

Congress has supported the use of federal electronic commerce4 and
electronic access to procurement information. Under FASA, this capability
was to be provided through the Federal Acquisition Computer Network
(FACNET), a governmentwide systems architecture for acquisitions based
on electronic data interchange, which is the computer-to-computer
exchange of routine business documents using standardized data formats.
FACNET implementation proved to be problematic, however, and in 1997,
section 850 of the National Defense Authorization Act for Fiscal Year 1998
replaced the preference for FACNET with a more flexible policy that
promotes the use of electronic commerce techniques whenever
practicable or cost-effective.5

Section 850 also requires that in implementing electronic commerce,
agencies apply standards consistent with nationally and internationally
recognized standards that broaden interoperability and ease of use.
Agency notices of procurement opportunities must be provided in a form
that allows convenient and easy user access through a single
governmentwide point of entry. While CBDNet satisfies the requirement
with respect to notices, neither it nor other currently available electronic
procurement posting systems satisfy the requirement for a single
governmentwide point of entry for access to solicitations and other
procurement information.

The law also requires that the Administrator of the Office of Federal
Procurement Policy (OFPP) submit to Congress yearly reports through 2003
on the progress being made in implementing section 850. The reports must
include a strategic plan for implementation of governmentwide electronic
commerce and an assessment of compliance with the requirement for a
single point of entry.




4
 Electronic commerce is conducted using electronic techniques for business transactions, including
electronic mail or messaging, World Wide Web technology, electronic bulletin boards, purchase cards,
electronic funds transfers, and electronic data interchange.
5
 We reported previously on numerous concerns about FACNET implementation, including FASA’s
requirements. See Acquisition Reform: Obstacles to Implementing the Federal Acquisition Computer
Network (GAO/NSIAD-97-26, Jan. 3, 1997) and Acquisition Reform: Classes of Contracts Not Suitable
for the Federal Acquisition Computer Network (GAO/NSIAD-97-232, Sept. 17, 1997).



Page 3                                                     GAO/NSIAD-99-37 Acquisition Reform
                                B-280239




                                Since 1994, NAIS has evolved into a procurement information system with
NAIS Is an Effective            many operational features that are effective, easy to use, and beneficial to
Information Service             NASA and vendors. By eliminating many of the steps in the paper-based
                                process of publicizing contract opportunities and issuing solicitations, NAIS
                                has contributed to a more streamlined agency acquisition process.
                                Feedback from NAIS users, particularly small businesses, has generally
                                been positive. However, the lack of data makes it very difficult to quantify
                                specific benefits of NAIS, such as the amount of time it actually saves. NASA
                                exceeded the $100-million ceiling authorized by FASA for the test. After NAIS
                                implementation, the number of offers per contract and small businesses’
                                share of contract actions and dollars showed some increases.


NAIS Is User-Friendly and       Since July 1994, NAIS has evolved to provide a broad range of
Works                           procurement-related functions and information. For example, in
                                November 1995, NASA added an on-line procurement reference library. In
                                March 1997, NASA completed the NAIS Electronic Posting System (EPS),
                                which enables procurement staff anywhere in the agency to prepare and
                                post notices and place solicitation files directly onto the NAIS Internet site.
                                In March 1997, NASA also added an e-mail notification service that
                                automatically notifies subscribers of new procurement information
                                available on NAIS.

                                We found that the system’s operational features were easy to use and
                                worked as described. Procurement staff demonstrated how they use the
                                EPS to generate, edit, and post notices and place solicitation files directly
                                onto the Internet. The EPS automatically formats each NAIS posting, inserts
                                links to other relevant documents and external information, updates the
                                business opportunities index library, and constructs and feeds messages to
                                the NAIS e-mail notification service. Appendix I provides more information
                                on the development and evolution of the EPS and the approach NASA used
                                to design, develop, and operate the system.

                                By going to the NAIS Internet site, users can

                            •   identify, at one location, notices of and solicitations for competitive
                                business opportunities and awards over $25,000 at all NASA centers;
                            •   search for business opportunities by type of product and/or service, NASA
                                center, or posting date;
                            •   locate and download documents related to a specific procurement; and




                                Page 4                                          GAO/NSIAD-99-37 Acquisition Reform
                             B-280239




                         •   view summaries of NASA’s current contracts (with contractor name, value,
                             obligations, and description) and search current contracts by state,
                             congressional district, business type, or type of product and/or service.

                             The e-mail notification service automatically transmits announcements to
                             subscribers immediately after new procurement information is posted on
                             NAIS. This allows subscribers to track developments in the solicitation
                             cycle and learn of new releases. Subscribers can register free of charge by
                             providing an e-mail address and selecting the types of contracting
                             opportunities in which they have the greatest interest. Thus, instead of
                             having to search for information, subscribers can have it selected and
                             delivered to their desktop electronically. As of July 1998, there were over
                             7,100 subscribers to the e-mail notification service. According to NASA, user
                             feedback showed that this service was the most popular feature of NAIS.


NAIS Contributes to a        One of NASA’s goals in establishing NAIS was to streamline and standardize
Streamlined and              the agency’s procurement process by using the Internet as the gateway to
Accessible Procurement       contracting opportunities for businesses. NAIS streamlined or eliminated
                             many of the steps required by the paper-based process for publicizing
System                       synopses of contracting opportunities and issuing solicitations.

                             The NAIS process involves fewer steps for procurement staff than the paper
                             process. The EPS, for example, automatically formats notices, performs
                             edit checks, automatically posts notices on the Internet through the NAIS
                             server, and transmits them to the Government Printing Office for
                             publication in the CBD. Using the paper process, procurement staff have to
                             perform analogous functions manually and must also respond to requests
                             for copies of solicitations by preparing mailing labels and mailing the
                             copies. With NAIS, the solicitation files are available for review and
                             retrieval (“downloading”) through the Internet.

                             We believe the NAIS process is also easier for contractors to use. Using the
                             paper-based system, contractors must either scan the CBD or periodically
                             call or visit each of NASA’s procurement offices to identify contracting
                             opportunities. In contrast, using NAIS, contractors can search for and view
                             as many synopses and solicitations as they wish on-line. They can search
                             by specific commodity, service, or location; view and provide comments
                             on draft solicitations; and access the Federal Acquisition Regulation, NASA
                             procurement policies and procedures, and NASA procurement forms.




                             Page 5                                       GAO/NSIAD-99-37 Acquisition Reform
                             B-280239




                             NASAofficials point to the absence of protests6 from contractors
                             concerning NAIS and the lack of complaints about NAIS to the NASA
                             Ombudsman Program7 as evidence that the contractor community
                             approves of NAIS.


User Feedback Was            Both businesses and NASA procurement staff have provided positive
Positive                     feedback about NAIS. Overall, industry responses to specific NAIS
                             operational features were positive. In a tally of on-line comments from NAIS
                             users, 87 percent of respondents said they found NAIS to be beneficial. The
                             respondents identified themselves primarily as small, small disadvantaged,
                             or women-owned businesses. For the most part, feedback from the
                             procurement staff was positive, and comments were often similar to those
                             made by businesses. Federal electronic commerce officials from other
                             agencies and industry representatives told us that systems with the
                             operational features of NAIS can benefit both industry and government,
                             especially when all buying organizations are using the same system.

                             An on-line NAIS feedback system was established in 1995 to obtain input,
                             especially from businesses, about the system. The NAIS team8 also requests
                             feedback from the procurement staff at NASA centers. This feedback was
                             compiled in December 1997 and again in June 1998. The team compiles
                             survey feedback data, develops a consolidated summary of industry
                             suggestions, and uses the summary to determine which changes and
                             enhancements should be made.


Lack of Data Makes It        NASA’s evaluation plan for the Midrange Pilot Program did not include a
Difficult to Quantify NAIS   separate NAIS evaluation plan. In May 1998, NASA published the NAIS
Benefits                     Business Case, which provides details of current NAIS features, along with
                             its history, benefits, and funding alternatives. However, NASA did not
                             collect data on or evaluate individual procurements advertised through
                             NAIS. The information needed to determine how much time NAIS actually
                             saved or to quantify other NAIS benefits, such as cost avoidance for the
                             procurement process, was not available. We did not find any evidence that


                             6
                              A protest is a written objection by an interested party to a solicitation or proposed award or award of
                             a contract. Protests are usually filed either with the procuring agency or with GAO.
                             7
                              The program was established in October 1995 to facilitate communications between NASA and
                             current and potential contractors. It provides offerors, potential offerors, and contractors with a single
                             point of contact to address their concerns.
                             8
                             The NAIS team consists of a small group of procurement and, in some cases, technical staff from each
                             NASA center and headquarters.



                             Page 6                                                        GAO/NSIAD-99-37 Acquisition Reform
                             B-280239




                             NAIS had negative effects, and there is substantial qualitative evidence of
                             the positive impact of NAIS. Appendix II provides more information on our
                             assessment of the NAIS Business Case.


NASA Exceeded Waiver         Although FASA allowed NASA a 4-year test authority to waive CBD
Authority’s Dollar Ceiling   publication requirements and waiting periods up to a limit of $100 million
                             in life-cycle costs, NASA did not establish a mechanism to track compliance
                             with the waiver provision. NASA officials told us that when the waiver
                             authority was originally requested, they anticipated that NAIS, though not
                             yet developed, would be used primarily by the Marshall Space Flight
                             Center, the focal point of the pilot test, and that the $100-million ceiling
                             would be adequate to cover Marshall’s procurements. However, the
                             Internet posting capability was rolled out to all NASA centers by April 1995
                             and was being used agencywide for all competitive procurements over
                             $25,000 1 month before NASA began waiving the CBD notice requirements in
                             October 1995.

                             In early 1997, NASA officials realized that the $100-million ceiling authorized
                             by FASA for the test would not cover agencywide use of NAIS. In May 1997,
                             the Administration submitted a legislative proposal to increase the ceiling
                             for NASA’s use of the waiver authority and extend the test period from 4 to
                             6 years. Congress did not act on the proposal. NASA used the CBD waiver
                             authority through the end of the fiscal year–September 30, 1997. NASA
                             calculated that it awarded approximately 1,585 contracts worth a total of
                             about $177 million using the CBD waiver authority.


No Significant Changes in    In an effort to determine the impact of NAIS on the amount of competition
Offers and Awards            for awards, NASA compared the number of vendors’ proposals received
Reported After NAIS          before and after NAIS implementation. NASA compared the number of offers
                             received per contract action in fiscal year 1994–the year prior to
Implementation               agencywide use of NAIS−to the number of offers per contract action in
                             fiscal year 1997–a year in which all midrange procurements were only
                             advertised through NAIS. As table 1 shows, the average number of offers
                             received per acquisition increased from 6.1 to 7.2 after NAIS
                             implementation. NASA said in the NAIS Business Case that the increase in
                             the average number of offers received per acquisition was especially
                             promising because the majority of these acquisitions were set aside for
                             small businesses.




                             Page 7                                        GAO/NSIAD-99-37 Acquisition Reform
                                        B-280239




Table 1: NAIS’s Impact on Competition
                                        Dollars in thousands
                                                                                                                    Fiscal year
                                                                     a
                                        Procurement activities                                                      1994           1997
                                        Number of contracts awarded                                                   644           396
                                        Offers received from businesses                                             3,959         2,852
                                        Offers per contract                                                           6.1            7.2
                                        Number of contracts to small businesses                                       380           252
                                        Small-business share (percent)                                              59.01         63.64
                                        Dollar value of contract awards                                           $3,296           $930
                                        Dollars awarded to small businesses                                         $431           $283
                                        Small-business dollar share (percent)                                        13.1          30.4
                                        a
                                            Competitive contract actions over $25,000.



                                        We also calculated and compared the percentages of contract actions and
                                        dollars awarded to small businesses in fiscal years 1994 and 1997,
                                        respectively. Table 1 shows these calculations. The percentage share of
                                        awards to small businesses did not change significantly after NAIS
                                        implementation, but the percentage share of dollars awarded to small
                                        businesses increased substantially in 1997. A NASA official said this was
                                        due in large part to a drop in awards to large businesses and not to any
                                        substantial change in dollars awarded to small businesses.


                                        Today most federal agencies use the Internet in some way to advertise
Efforts Are Underway                    their contracting opportunities. Vendors search all these various Internet
to Establish a                          sites for government procurement information. Recent legislation requires
Governmentwide EPS                      the government to provide direct access to notices of agencies’
                                        requirements and solicitations through a single governmentwide electronic
                                        point of entry. NASA, GSA, and other agencies are testing a multiagency
                                        electronic posting system. Agency officials are concerned that the benefits
                                        of providing direct electronic access to contracting opportunities may be
                                        offset by existing statutory requirements for publication of procurement
                                        notices and minimum waiting periods.


Status of Efforts to                    According to the San Antonio Electronic Commerce Resource Center,9
Develop a                               more than 400 different Internet sites provide federal procurement
Governmentwide EPS                      information. This multitude of sites makes it difficult and time-consuming

                                        9
                                         Electronic Commerce Resource Centers are a network of government-sponsored centers that provide
                                        electronic commerce training and support to the contractor community.



                                        Page 8                                                   GAO/NSIAD-99-37 Acquisition Reform
B-280239




for contractors to obtain governmentwide information on contracting
opportunities.

The March 1998 report, Electronic Commerce for Buyers and Sellers, A
Strategic Plan for Electronic Federal Purchasing and Payment,10 described
planned enhancements to the electronic version of the CBD, CBDNet,
which now provides a single point of entry only for procurement notices.
GSA and NASA approached the Department of Commerce and the
Government Printing Office to form an interagency team to implement an
enhanced CBDNet and to make an Internet-based EPS with features
comparable to those of the NAIS system available to the entire federal
sector.11 However, the agencies could not reach agreement on an
enhanced CBDNet implementation approach. Among unresolved issues
were the source and level of funding needed for the project, the costs to
develop and implement the system, and the potential financial impact on
the operating and user agencies.

NASA, GSA, and other members of the Interagency Acquisition Internet
Council12 are developing and testing a pilot multiagency posting system
derived from the NAIS EPS to enable agencies to post notices, solicitations,
and other acquisition-related documentation directly onto the Internet,
where it would be available through a single point of entry. GSA is leading
this effort and spent over $400,000 in the past year developing this
multiagency EPS software. In October 1998, GSA began using the pilot
software as the agency’s single EPS. If testing demonstrates that the
multiagency EPS is capable of providing effective access to notices and
solicitations through a single point of entry, the Administrator of the Office
of Federal Procurement Policy will consider designating it as the single
governmentwide point of entry required by section 850.




10
 Prepared by the Electronic Processes Initiatives Committee of the President’s Management Council
and submitted to Congress in response to the requirement in section 850 of the Fiscal Year 1998
Department of Defense Authorization Act for a strategic plan for the implementation of a
governmentwide electronic commerce capability.
11
  The Department of Commerce is responsible for keeping the business community informed about
federal procurement actions through publication of the CBD. In 1961, the Government Printing Office
assumed CBD publication and distribution functions. However, all editorial functions remained within
the jurisdiction of the Department of Commerce.
12
  The Council was established in May 1996 to seek and promote ways to optimize the use of the
Internet in streamlining the federal acquisition process and increasing communication of federal
acquisition-related information.



Page 9                                                      GAO/NSIAD-99-37 Acquisition Reform
                 B-280239




Challenges to    The difficulty of establishing a single governmentwide electronic point of
Governmentwide   entry for federal procurement opportunities was emphasized by several
Implementation   agency officials, who said they were concerned about the costs involved in
                 developing, operating, and maintaining such a system; cost sharing among
                 different agencies; and the linkages that need to be established with
                 existing agency systems through the Internet. They also questioned how
                 and when decisions would be made concerning who would lead such a
                 project, who would be responsible for operating and maintaining the
                 system, and who would mandate its use by agencies.

                 Agency officials also said that the benefits of a governmentwide electronic
                 notice capability could be enhanced through changes to the current
                 statutory minimum waiting periods for notices and solicitations, which
                 were fashioned around a paper-based process. Acquisition officials noted,
                 in particular, that the law requires a 15-day waiting period after
                 publication of a notice before a solicitation can be issued, even though the
                 Internet enables agencies to publicize notices and provide solicitation files
                 simultaneously, giving contractors immediate access to these documents
                 and saving time in the procurement process. Agency officials said that
                 changing these requirements could increase the benefits of using
                 electronic commerce. OFPP plans to study what type of legislative action
                 may be needed to enhance the benefits of electronic publication of
                 business opportunities.


                 NASA has shown that it can successfully use NAIS to effectively disseminate
Conclusions      solicitations and other procurement information, giving contractors easy
                 and immediate access to business opportunities at each of its
                 geographically dispersed procurement offices. Feedback from users has
                 been positive, and the system has allowed increased standardization of the
                 procurement process agencywide. However, benefits such as the time
                 savings obtained through use of the system cannot be readily quantified
                 because of data limitations. While NASA believes that NAIS has had a
                 positive impact on competition, this conclusion was based on only 2 years
                 of data.

                 NASA’s on-line electronic posting system was the model for the pilot
                 multiagency EPS software that is being tested by GSA, NASA, and several
                 other agencies. If successful, this EPS may be considered for
                 governmentwide expansion and designation as the single governmentwide
                 point of entry for both notices and solicitations. While the March 1998
                 report assessing electronic commerce initiatives discussed enhancements



                 Page 10                                      GAO/NSIAD-99-37 Acquisition Reform
                  B-280239




                  to the CBDNet, it did not discuss the multiagency EPS pilot program. If a
                  governmentwide EPS is successfully implemented, its benefits potentially
                  could be enhanced by changing the statutory notice and waiting periods.


                  We recommend that the Administrator of the Office of Federal
Recommendations   Procurement Policy describe the current plan for implementation of a
                  governmentwide single point of entry to procurement information in its
                  next annual report to Congress. This plan should take into consideration
                  the status of and lessons learned from the multiagency EPS pilot and its
                  potential as the basis for a comprehensive governmentwide electronic
                  posting system.

                  We also recommend that the Administrator of the Office of Federal
                  Procurement Policy determine whether the issues raised by agency
                  officials concerning procurement notice requirements set forth in statute,
                  including waiting periods, require legislative action and make
                  recommendations to Congress on legislative changes, if necessary.


                  In commenting on the draft of this report, NASA and OFPP generally agreed
Agency Comments   with our findings and recommendations. NASA stated that it was proud of
                  the NAIS staff for pioneering the use of the Internet in federal procurement;
                  and our review largely corroborated the agency’s assessment of the value
                  of NAIS.

                  OFPP stated that NAIS has created the foundation for the ongoing
                  multiagency pilot of an electronic posting system. If successful, the EPS
                  will provide easy access to procurement information across the
                  government through a single point of entry. OFPP was encouraged by the
                  promising findings identified in the report. OFPP also commented on our
                  recognition that current statutory requirements for procurement notices
                  and minimum waiting periods may limit the potential benefits of electronic
                  commerce and indicated that the feasibility of legislative action would be
                  studied.

                  The comments from NASA and OFPP are reprinted in their entirety in
                  appendix III and IV, respectively.




                  Page 11                                      GAO/NSIAD-99-37 Acquisition Reform
              B-280239




              To determine whether the NAIS electronic notice and publication system
Scope and     was an effective mechanism for disseminating procurement information,
Methodology   we obtained information about the system’s design, development,
              operation, costs, and benefits. We obtained documents from and
              interviewed officials at NASA Headquarters in Washington, D.C., and at the
              Marshall Space Flight Center (the lead center for NAIS project
              management) in Huntsville, Alabama. We also accessed NASA’s
              procurement Internet site (http://procurement.nasa.gov) and reviewed the
              on-line NAIS operational features available to both public users and NASA
              contracting staff to determine whether the system worked as NASA
              represented. To evaluate NASA’s claim that NAIS usage had a positive impact
              on competition and was beneficial to small businesses, we compared NASA
              procurement data for fiscal year 1994 (the year prior to agencywide use of
              NAIS) with data for fiscal year 1997. We did not independently verify the
              NASA procurement data obtained from the NASA acquisition management
              information system.

              We also reviewed NASA’s evaluation of the system contained in the
              May 1998 NAIS Business Case. We reviewed the data in the NAIS Business
              Case, including calculations of cost avoidance and other benefits
              attributed to NAIS. We examined on-line industry feedback and narrative
              responses from vendors and NASA buyers about NAIS. To determine the
              agency’s compliance with statutory requirements, we reviewed the goals,
              objectives, and time frames established by FASA for NASA’s test of electronic
              notice and publication.

              To assess the status of efforts to establish a governmentwide EPS based on
              the NAIS model, we asked electronic commerce coordinators from several
              federal agencies, federal Electronic Commerce Resource Centers, and two
              industry groups for their observations. We received responses from
              coordinators at five agencies, two resource centers, and one industry
              group. We also interviewed acquisition officials at GSA, the Department of
              Commerce, and OFPP to obtain information on interagency efforts to
              develop a governmentwide electronic procurement posting system. We
              also reviewed the goals, objectives, and milestones established for the
              governmentwide electronic commerce program by the National Defense
              Authorization Act for Fiscal Year 1998.

              We performed our work between January and December 1998 in
              accordance with generally accepted government auditing standards.




              Page 12                                      GAO/NSIAD-99-37 Acquisition Reform
B-280239




We are sending copies of this report to the Administrator, NASA; the
Administrator, OFPP; and the Director, Office of Management and Budget.
We will also make copies available to others upon request.

Please contact me at (202) 512-4841 if you or your staff have any questions
concerning this report. Major contributors to this report are listed in
appendix V.




Allen Li
Associate Director
Defense Acquisitions Issues




Page 13                                     GAO/NSIAD-99-37 Acquisition Reform
B-280239




List of Congressional Committees

The Honorable Fred Thompson
Chairman
The Honorable Joseph I. Lieberman
Ranking Minority Member
Committee on Government Affairs
United States Senate

The Honorable John W. Warner
Chairman
The Honorable Carl Levin
Ranking Minority Member
Committee on Armed Services
United States Senate

The Honorable Christopher S. Bond
Chairman
The Honorable John F. Kerry
Ranking Minority Member
Committee on Small Business
United States Senate

The Honorable Dan Burton
Chairman
The Honorable Henry A. Waxman
Ranking Minority Member
Committee on Government Reform
House of Representatives

The Honorable Floyd Spence
Chairman
The Honorable Ike Skelton
Ranking Minority Member
Committee on Armed Services
House of Representatives

The Honorable James M. Talent
Chairman
The Honorable Nydia M. Velázquez
Ranking Minority Member
Committee on Small Business
House of Representatives


Page 14                             GAO/NSIAD-99-37 Acquisition Reform
Page 15   GAO/NSIAD-99-37 Acquisition Reform
Contents



Letter                                                                                           1


Appendix I                                                                                      18
The NAIS Electronic
Posting System
Appendix II                                                                                     20
NASA’s Business Case
for NAIS
Appendix III                                                                                    22
Comments From the
National Aeronautics
and Space
Administration
Appendix IV                                                                                     23
Comments From the
Office of Federal
Procurement Policy
Appendix V                                                                                      25
Major Contributors to
This Report
Tables                  Table 1: NAIS’s Impact on Competition                                    8
                        Table II.1: Annual NAIS Cost Estimates                                  21




                        Page 16                                  GAO/NSIAD-99-37 Acquisition Reform
Contents




Abbreviations

CBD        Commerce Business Daily
EPS        Electronic Posting System
FACNET     Federal Acquisition Computer Network
FASA       Federal Acquisition Streamlining Act
FTE        full-time equivalent
GSA        General Services Administration
NAIS       NASA Acquisition Internet Service
NASA       National Aeronautics and Space Administration
OFPP       Office of Federal Procurement Policy


Page 17                                  GAO/NSIAD-99-37 Acquisition Reform
Appendix I

The NAIS Electronic Posting System


              Staff at the National Aeronautics and Space Administration (NASA)
              responsible for developing and implementing the NASA Acquisition Internet
              Service (NAIS) consisted of a small group of procurement and, in some
              cases, technical staff from each NASA center and headquarters. They
              adopted an incremental approach to system development and operations.
              This approach included the following steps: (1) general requirements were
              drawn up, (2) commercially available software options were researched,
              (3) a prototype was designed and developed if a commercial solution was
              not available, (4) a prototype for testing was installed, (5) requirements
              were refined and a small pilot initiated, and (6) the pilot was expanded
              and the software solution added to the NAIS system.

              NASA’s development of its Electronic Posting System (EPS) is characteristic
              of the way the NAIS team applied the concept of incremental design,
              development, and implementation of NAIS features. Prior to agencywide
              use of EPS, each NASA center had its own unique process for preparing and
              posting procurement notices and solicitations on the Internet. Some
              centers used word processors while others used some form of automated
              document generation system. The completed files were then provided to a
              Web curator at the center who would post them on an Internet server and
              manually update a readable index of files. Changes to the posting process
              occurred spontaneously at various centers with little central control or
              agreement. The 10 centers’ differing processes required 10 separate efforts
              to implement a single change that had to be agreed on by the larger NAIS
              team. These process differences also made it difficult to maintain a
              standard “look and feel” when searching out business opportunities. A
              vendor was faced with 10 different entry points to search.

              NAIS operated for a year in this fashion. Then an ad-hoc group of NAIS team
              members began to study ways to make the centers’ posting processes
              more uniform. The team concluded that the NAIS posting process should be
              centralized to a single server and that the application for posting and
              maintaining documents should be standardized by using on-line forms to
              be completed by users. NAIS technical team members then designed and
              developed the uniform Web posting tools that were first piloted and then
              deployed agencywide.

              According to NASA, implementation of the EPS eliminated much duplication
              among the centers, standardized posting practices across all NASA centers,
              and lowered NAIS maintenance costs. The EPS application provides a
              common look and feel of acquisition opportunities.




              Page 18                                     GAO/NSIAD-99-37 Acquisition Reform
Appendix I
The NAIS Electronic Posting System




The incremental, evolutionary approach to deploying an electronic posting
system avoided the costs and delays often associated with attempting a
single solution to standardize disparate, decentralized systems.
Additionally, because much of the system is based on commercial
standards, commercially available software, and standard hardware, it was
easily adopted by each of the centers without major disruptions.

All NAIS features have been built using commercial Internet standards and
Web methodology so that even casual Internet users can easily use NAIS.
For internal procurement staff using standard NASA computer hardware
and software, NAIS tools may include user instructions. For example, the
NAIS EPS has an on-line user’s manual linked directly to the application. For
external users with a wide variety of hardware and software, NAIS features
are based on widely accepted Internet protocols and standards, and the
features are designed to be used without any formal training.




Page 19                                      GAO/NSIAD-99-37 Acquisition Reform
Appendix II

NASA’s Business Case for NAIS


              In May 1998, NASA completed and published an evaluation of NAIS in a
              document called the NAIS Business Case. The document sought to justify
              continued funding for NAIS by showing that the system’s quantitative and
              qualitative benefits far exceed NASA’s investment. According to NASA’s
              assessment, the NAIS tools implemented to date had produced outstanding
              results.

              We found weaknesses in the evaluation’s methodology. For example, NASA
              claimed that the Midrange Pilot initiatives, including NAIS, reduced
              procurement leadtimes by 40 percent.1 We examined the data used by NASA
              to support this calculation. In a number of cases, the data showed that the
              official purchase requests arrived at NASA procurement offices after
              solicitations for the requested products or services had already been
              issued. Consequently, we could not verify NASA’s claimed 40-percent
              reduction in leadtime. However, it is obvious that the reduction in mail
              delays and in the manual steps and efforts needed to post a procurement
              notice should contribute to reduced leadtimes.

              We also found that the NAIS Business Case’s discussion of the resources
              used in developing and operating NAIS was incomplete. It described only
              the full-time equivalent (FTE)2 staff years of effort, both funded and
              unfunded, which were devoted to NAIS development and support in fiscal
              year 1997 and did not reflect the additional resources such as technical
              support, hardware, software, and communications services provided by
              various NASA centers.

              Because the evaluation only identified NAIS FTE staffing for 1 year and did
              not identify hardware, software, and other costs associated with NAIS, we
              asked NASA to provide fiscal year estimates of the total costs associated
              with NAIS development and ongoing operations. The table below depicts
              NASA’s estimates of the staffing, hardware, software, and miscellaneous
              costs associated with NAIS for fiscal years 1995 through 1998. Amounts for
              fiscal years 1995 through 1997 are estimates, while the amount for fiscal
              year 1998 is a projection.




              1
               Procurement leadtime, broadly speaking, is the time it takes to accomplish a contracting action.
              Leadtime can be calculated in a variety of ways and is affected by many variables. Leadtime in the
              NAIS evaluation represented the period of time from receipt of a purchase request by the procurement
              office to award of the contract.
              2
              Federal civilian employment is generally stated on an FTE basis. It is the total number of hours
              worked (or to be worked) divided by the number of compensable hours applicable to each fiscal year.



              Page 20                                                    GAO/NSIAD-99-37 Acquisition Reform
                                         Appendix II
                                         NASA’s Business Case for NAIS




Table II.1: Annual NAIS Cost Estimates
                                                                                      Contractor        Hardware/
                                         Fiscal year                     NASA staff         staff   software/misc.      Total
                                         1995                              $268,620      $56,000         $145,000    $469,620
                                         1996                               529,100      108,500          145,000    $782,600
                                         1997                               345,090      248,500          185,900    $788,490
                                         1998                               488,400      280,000          145,000    $913,400



User Feedback on NAIS                    NASA reported in the NAIS Business Case that data from the NAIS on-line
Was Positive                             feedback system showed that over 85 percent of those who responded to
                                         the question, “Is NAIS, as a whole, beneficial?,” answered “yes.” We
                                         reviewed summary reports of responses tallied from the NAIS system and
                                         found that of the 2,310 persons answering this question, 2,007 answered
                                         yes (87 percent). The survey respondents identified themselves primarily
                                         as small, small disadvantaged, or women-owned businesses. Overall, we
                                         found both industry and NASA procurement staff comments about specific
                                         NAIS operational features were positive.


                                         We also reviewed feedback responses submitted from 1995 through
                                         June 1998 to an open-ended question about enhancements that would
                                         improve NAIS. As of June 30, 1998, the NAIS team had completed actions or
                                         had actions underway related to 42 of 89 suggestions.

                                         Federal officials and groups representing industry perspectives told us
                                         that the NAIS Business Case properly identified the areas in which both
                                         government and industry may experience time and cost savings through
                                         an effective agencywide EPS. They noted that electronic commerce
                                         systems with the functionality and user-friendly features of NAIS have clear
                                         potential to benefit both industry and government, particularly if all buying
                                         centers are using the same system.




                                         Page 21                                          GAO/NSIAD-99-37 Acquisition Reform
Appendix III

Comments From the National Aeronautics
and Space Administration




               Page 22        GAO/NSIAD-99-37 Acquisition Reform
Appendix IV

Comments From the Office of Federal
Procurement Policy




              Page 23          GAO/NSIAD-99-37 Acquisition Reform
Appendix IV
Comments From the Office of Federal
Procurement Policy




Page 24                               GAO/NSIAD-99-37 Acquisition Reform
Appendix V

Major Contributors to This Report


                        Delores Cohen
National Security and   Richard J. Herley
International Affairs   Dan Hoagland
Division, Washington,   Stefano Petrucci
                        Patricia D. Slocum
D.C.
                        Bobby Hall
Atlanta Field Office
                        John Carter
Office of General
Counsel, Washington,
D.C.




(707346)                Page 25              GAO/NSIAD-99-37 Acquisition Reform
Ordering Information

The first copy of each GAO report and testimony is free.
Additional copies are $2 each. Orders should be sent to the
following address, accompanied by a check or money order
made out to the Superintendent of Documents, when
necessary. VISA and MasterCard credit cards are accepted, also.
Orders for 100 or more copies to be mailed to a single address
are discounted 25 percent.

Orders by mail:

U.S. General Accounting Office
P.O. Box 37050
Washington, DC 20013

or visit:

Room 1100
700 4th St. NW (corner of 4th and G Sts. NW)
U.S. General Accounting Office
Washington, DC

Orders may also be placed by calling (202) 512-6000
or by using fax number (202) 512-6061, or TDD (202) 512-2537.

Each day, GAO issues a list of newly available reports and
testimony. To receive facsimile copies of the daily list or any
list from the past 30 days, please call (202) 512-6000 using a
touchtone phone. A recorded menu will provide information on
how to obtain these lists.

For information on how to access GAO reports on the INTERNET,
send an e-mail message with "info" in the body to:

info@www.gao.gov

or visit GAO’s World Wide Web Home Page at:

http://www.gao.gov




PRINTED ON    RECYCLED PAPER
United States                       Bulk Rate
General Accounting Office      Postage & Fees Paid
Washington, D.C. 20548-0001           GAO
                                 Permit No. G100
Official Business
Penalty for Private Use $300

Address Correction Requested