oversight

Defense Pilot Programs: DOD Needs to Improve Implementation Process for Pilot Programs

Published by the Government Accountability Office on 2003-07-28.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

             United States General Accounting Office

GAO          Report to the Committee on Armed
             Services, U.S. Senate



July 2003
             DEFENSE PILOT
             PROGRAMS
             DOD Needs
             to Improve
             Implementation
             Process for Pilot
             Programs




GAO-03-861
                                                July 2003


                                                DEFENSE PILOT PROGRAMS

                                                DOD Needs to Improve Implementation
Highlights of GAO-03-861, a report to           Process for Pilot Programs
the Committee on Armed Services,
U.S. Senate




In fiscal years 1999, 2000, and 2003,           The 1999 and 2000 pilot programs have not worked as intended. Since
the Congress authorized pilot                   their inception, 178 initiatives have been proposed by the participating
programs to help the Department                 laboratories and test centers but only 4—or 2 percent—were implemented
of Defense (DOD) laboratories                   under the pilot programs, as shown below. Participants proposed initiatives
and test centers explore innovative             covering a variety of areas, including business-like practices, partnerships,
business partnerships and human
capital strategies. Congressional
                                                and human capital innovations.
concerns about DOD’s
implementation of the pilot                     The pilot programs were not effective because DOD lacked an effective
programs have been growing.                     implementation process and proposed human capital initiatives were
The Congress mandated                           not consistent with statutory provisions. First, DOD did not provide
that GAO review pilot program                   standardized guidance on proposal requirements, coordinate proposals,
implementation. GAO                             or clarify decision-making authority for proposal review and approval.
(1) identified the pilot initiatives            Furthermore, DOD did not designate a strong focal point to provide
proposed and their current status,              assistance and advice to participants and advocate process improvements.
(2) examined factors that affected              The lack of a strong focal point exacerbated other process gaps. Second,
implementation, and (3) assessed                DOD attorneys advised that the pilot programs did not provide authority to
implementation challenges the
2003 pilot program faces.
                                                make most of the proposed human capital changes.

                                                Implementation of the new 2003 pilot program faces several challenges.
                                                First, DOD has not addressed implementation problems. For example,
                                                clear guidance is still lacking and decision-making authority is still unclear.
                                                Second, the 2003 pilot program provides no change in authority concerning
GAO recommends that by
                                                human capital initiatives. Finally, laboratories and test centers may be
March 31, 2004, the Secretary of
Defense (1) inform the Congress of              reluctant to participate. Many participants in the earlier pilots told us
DOD’s objectives regarding human                they were discouraged by their experience and consequently unwilling to
capital and business operations in              repeat it.
the laboratories and test centers;
(2) develop a process for                       Status of Proposed Initiatives
proposing, evaluating, and
implementing human capital and
business operations initiatives,
regardless whether by the pilot
authority or by some other vehicle;
and (3) designate a strong focal
point to coordinate and facilitate
this process. DOD did not concur
with GAO’s recommendations.




www.gao.gov/cgi-bin/getrpt?GAO-03-861.

To view the full product, including the scope
and methodology, click on the link above.
For more information, contact Catherine
Baltzell at (202) 512-8001 or
baltzelld@gao.gov.
Contents


Letter                                                                                   1
              Results in Brief                                                           1
              Background                                                                 3
              Many Initiatives Were Proposed but Few Were Implemented under
                Pilot Programs                                                          5
              The Pilot Programs Were Not Effective for Two Primary Reasons             8
              The 2003 Pilot Program Faces Implementation Challenges                   12
              Conclusions                                                              12
              Recommendations for Executive Action                                     13
              Agency Comments and Our Evaluation                                       13
              Scope and Methodology                                                    15

Appendix I    Fiscal Year 1999 and 2000 Pilot Program Participants 17


Appendix II   Comments from the Department of Defense                                  18



Table
              Table 1: Laboratory and Test Center Pilot Program Proposals                5


Figure
              Figure 1: Status of Proposed Initiatives                                   7




              Page i                                     GAO-03-861 Defense Pilot Programs
Abbreviations

DOD               Department of Defense
DDR&E             Directorate of Defense Research and Engineering
NSPS              National Security Personnel System
OSD               Office of the Secretary of Defense
P&R               Personnel and Readiness




This is a work of the U.S. government and is not subject to copyright protection in the
United States. It may be reproduced and distributed in its entirety without further
permission from GAO. However, because this work may contain copyrighted images or
other material, permission from the copyright holder may be necessary if you wish to
reproduce this material separately.




Page ii                                              GAO-03-861 Defense Pilot Programs
United States General Accounting Office
Washington, DC 20548




                                   July 28, 2003

                                   The Honorable John Warner
                                   Chairman
                                   The Honorable Carl Levin
                                   Ranking Minority Member
                                   Committee on Armed Services
                                   United States Senate

                                   Over the last decade a number of studies have raised concerns that
                                   dwindling budgets and an aging workforce have contributed to serious
                                   shortfalls in the infrastructure and capabilities of Department of Defense
                                   (DOD) laboratories and test centers. In fiscal years 1999 and 2000, the
                                   Congress enacted legislation aimed at helping DOD laboratories and
                                   test centers address problems by undertaking pilot programs to explore
                                   innovative partnerships and human capital strategies.1 In fiscal year 2003,
                                   the Congress extended the 1999 and 2000 pilot programs until 2005 and
                                   enacted a new pilot program that runs until 2006. However, congressional
                                   concerns about pilot program implementation have been growing.
                                   Consequently, the Senate Committee on Armed Services directed us to
                                   review the implementation of the 1999 and 2000 pilot programs.2 In
                                   response, this report (1) identifies initiatives proposed to date and
                                   determines their current status, (2) examines factors that affected
                                   implementation of proposed initiatives, and (3) assesses implementation
                                   challenges the new 2003 pilot program faces.


                                   The 1999 and 2000 pilot programs have not worked as intended. Since
Results in Brief                   their inception, 178 initiatives have been proposed by the participating
                                   laboratories and test centers but only 4—or 2 percent—have been
                                   implemented using the pilot program authorities. Twelve times as many—
                                   24 percent—were implemented using other authorities than those
                                   provided by the pilot programs. Participating laboratories and test centers
                                   proposed initiatives covering a variety of areas, including business-like


                                   1
                                    Public Law 105-261, Strom Thurmond National Defense Authorization Act for Fiscal Year
                                   1999, section 246. Public Law 106-65, National Defense Authorization Act for Fiscal Year
                                   2000, section 245.
                                   2
                                       Senate Report 107-151.



                                   Page 1                                              GAO-03-861 Defense Pilot Programs
practices, partnerships with industry and academia, and human capital
innovations. In general, laboratories tended to propose initiatives dealing
with human capital innovations and test centers focused on business-like
practices and partnerships.

The pilot programs were not effective because DOD lacked an effective
implementation process and proposed human capital initiatives were
not consistent with statutory provisions. First, DOD did not provide
standardized guidance on proposal requirements, coordinate proposals,
or clarify decision-making authority for proposal review and approval.
Furthermore, DOD did not designate a strong focal point to provide
assistance and advice to participants and advocate process improvements.
The lack of a strong focal point exacerbated other process gaps. Second,
DOD attorneys advised participants that the 1999 and 2000 pilot
programs did not provide authority to make most of the proposed human
capital changes.

Implementation of the new 2003 pilot program faces several challenges.
First, DOD has not addressed implementation problems. For example,
clear guidance is still lacking and decision-making authority has not been
clarified. Second, the 2003 pilot program provides no change in authority
concerning human capital initiatives. DOD officials believe that the
human capital management legislation the department recently proposed
to the Congress will provide flexibility throughout DOD to make
necessary human capital changes, thereby eliminating the need for the
pilot programs in this area.3 However, this legislation, if enacted, would
still require an implementation process. Finally, laboratories and test
centers may be reluctant to participate in the new pilot program. Many
participants in the earlier pilots told us they were discouraged by their
experience and consequently unwilling to repeat it.

We are making recommendations aimed at clarifying how DOD plans to
address concerns about the laboratories and test centers and improving
the implementation of initiatives proposed for that purpose. In written
comments on a draft of this report, DOD stated that it did not concur with
our recommendations.




3
 Defense Transformation for the 21st Century Act of 2003, as transmitted by letter,
dated April 10, 2003, from the DOD General Counsel to the Speaker of the House and
the President of the Senate.




Page 2                                               GAO-03-861 Defense Pilot Programs
                 The United States has a long history of military research and development.
Background       To help conduct and manage this research, DOD has a diverse network
                 of 80 in-house laboratories and 26 test centers. Their missions range
                 from basic scientific research to direct technical support to operational
                 commands. The management, operations, and funding for these disparate
                 laboratories and test centers also vary among the services.

                 Over the past decade, several organizations, panels, and commissions
                 have identified significant personnel and resource problems facing the
                 laboratories and test centers. For example, several studies found that
                 the laboratories needed more flexibility in personnel rules governing the
                 scientific workforce in order to attract and retain staff.4 Similarly, several
                 recent studies identified problems with declines in investment and
                 infrastructure, resulting in outdated facilities and technical equipment.5

                 To help the laboratories and test centers with these problems, the
                 Congress enacted legislation in fiscal years 1999 and 2000 establishing
                 pilot programs for laboratories and test centers to propose innovative
                 partnerships, business-like practices, and human capital initiatives.6 The
                 1999 pilot program focused on partnerships and business-like practices,
                 while the 2000 program focused more on human capital initiatives.
                 Together, the two pilot programs authorized the Secretary of Defense to
                 provide one laboratory and one test center in each service the authority to

             •   explore innovative methods for partnering with universities and private
                 sector entities to conduct defense research and development;
             •   attract a workforce balance between permanent and temporary personnel
                 and with an appropriate skill and experience level;
             •   develop or expand innovative methods of operation that provide more
                 defense research for the dollar; and
             •   waive any restrictions on these methods that are not required by law.


                 4
                  Blue Ribbon Panel on Management Options for Air Force Laboratories, January 1994;
                 Improving Federal Laboratories to Meet the Challenges of the 21st Century, National
                 Science and Technology Council, July 1999; Science and Technology Community in
                 Crisis, Naval Research Advisory Committee, May 2002.
                 5
                  Defense Science and Technology Base for the 21st Century, Defense Science Board,
                 June 1998; House Report 105-532, House National Security Committee, May 1998; Science
                 and Technology Community in Crisis, Naval Research Advisory Committee, May 2002.
                 6
                  Public Law 105-261, Strom Thurmond National Defense Authorization Act for Fiscal Year
                 1999, section 246; Public Law 106-65, National Defense Authorization Act for Fiscal Year
                 2000, section 245.




                 Page 3                                              GAO-03-861 Defense Pilot Programs
    A total of 10 laboratories and test centers from all 3 services participated
    in the pilot programs. They are listed in appendix I.

    Both programs were authorized for 3 years. The 1999 pilot expired in
    March 2002; the 2000 pilot, in March 2003. For both programs, DOD
    was required to submit preliminary and final reports to the Congress
    on program activities. The preliminary report for the 1999 program was
    submitted in July 1999. However, as of the date of this report, the three
    other reports have not been submitted.

    In fiscal year 2003, the Congress authorized another 3-year pilot program
    and extended the 1999 and 2000 pilot programs until 2005.7 Under the new
    2003 pilot program, the Secretary of Defense is to provide one laboratory
    and one test center in each service the authority to

•   use innovative personnel management methods to ensure that the
    participants can employ and retain an appropriately balanced workforce,
    and effectively shape the workforce to fulfill the organization mission;
•   develop or expand innovative methods of using cooperative agreements
    with private sector and educational organizations to promote the
    technological industrial base for critical defense technologies and
    facilitate the training of a future scientific and technical workforce; and
•   waive any restrictions not required by law.

    As of May 2003, DOD had not identified any participants for the 2003 pilot
    program.

    The 2003 legislation also requires DOD to issue three reports, including
    a January 2003 report on its experience with the 1999 and 2000 pilot
    programs, barriers to implementation of these programs, and proposed
    solutions to overcome these barriers.8 According to DOD officials, this
    report has been drafted, but as of May 2003, it had not been submitted to
    the Congress.




    7
     Public Law 107-314, Bob Stump National Defense Authorization Act for Fiscal Year 2003,
    section 241.
    8
     The other two reports are a September 2003 report on all three pilot programs and a final
    report on the 2003 pilot at its conclusion.




    Page 4                                                GAO-03-861 Defense Pilot Programs
                        Since the inception of the pilot programs in 1999, 178 initiatives have
Many Initiatives Were   been proposed, but only 4—or 2 percent—have been implemented
Proposed but Few        under the pilot programs. Participating laboratories and test centers
                        proposed initiatives covering a variety of areas, including business-like
Were Implemented        practices, partnerships with industry and academia, and human capital
under Pilot Programs    innovations. We found that laboratories focused many of their proposals
                        on human capital innovations, while test centers tended to concentrate on
                        business-like practices and partnerships.


Range and Volume of     Over the course of the 1999 and 2000 pilot programs, the laboratories
Proposed Initiatives    and test centers proposed 178 human capital, business, and partnership
                        initiatives. As shown in table 1, slightly over half of the initiatives dealt
                        with human capital and the remainder dealt with business-like practices
                        and partnerships.

                        Table 1: Laboratory and Test Center Pilot Program Proposals

                                        Business/partnerships     Human capital             Total
                                           Number     Percent    Number     Percent    Number    Percent
                         Laboratories           49         33         98          67       147       100
                         Test centers           27         87          4          13        31       100
                         Total                  76         43        102          57       178       100
                        Source: GAO.


                        Overall, the laboratories proposed substantially more initiatives than did
                        the test centers. Furthermore, the laboratories and test centers focused
                        on different types of initiatives. The laboratories more often proposed
                        human capital initiatives, while the test centers overwhelmingly focused
                        on business and partnership initiatives. Laboratory officials told us that
                        they are especially concerned about attracting top-quality scientists to
                        replace a retiring workforce. Test center officials told us that they are
                        focused on modernizing their infrastructure and developing new methods
                        of sharing the cost of operations.

                        Proposals for business-like practices included many initiatives to
                        streamline or improve local operations. Some initiatives focused on
                        expanding the use of innovative techniques such as other transactions




                        Page 5                                          GAO-03-861 Defense Pilot Programs
                          or cooperative agreements.9 Several other proposals sought the authority
                          to reinvest fees or revenues into facilities revitalization. For example, one
                          Navy laboratory proposed imposing a surcharge for its services and using
                          that revenue to fund capital investments, and an Air Force laboratory
                          proposed using facility construction as a valid in-kind contribution under
                          cooperative agreements.

                          Partnership proposals included initiatives such as collaborative research
                          agreements with Arnold Engineering Development Center and the
                          University of Tennessee Space Institute to create a formal business bond
                          to pursue research in laser-induced surface improvement technology and
                          university flight research.

                          The Army’s Aberdeen Test Center proposed a limited liability company.
                          Under this concept, industry, academia, and government would form a
                          profit-making company to conduct research and testing at the installation.
                          The test center proposed using its share of the profits to reinvest in the
                          infrastructure at Aberdeen.

                          Several human capital initiatives focused on recruiting and retention
                          flexibilities as well as additional voluntary separation incentives.
                          These proposals included initiatives to streamline hiring of experts
                          and consultants; accelerate promotions for scientists and engineers;
                          provide retention bonuses for key scientists; and hire students directly
                          after graduation. Several participants submitted proposals for direct
                          hire authority to allow faster hiring of scientists, and several submitted
                          proposals for voluntary retirement incentives as a mechanism for
                          reshaping the workforce.


Few Proposals Were        Almost none of the 178 proposed initiatives were approved and
Implemented under Pilot   implemented using the pilot programs’ authorities. As figure 1 shows,
Program Authorities       only 2 percent—or 4 proposals—were implemented under the pilot
                          programs. In contrast, 74 percent were blocked or dropped during the
                          review process or remain on hold awaiting resolution.



                          9
                            “Other transactions” is a term commonly used to refer to 10 U.S.C. 2371 authority to enter
                          into agreements that are not generally covered by federal laws and regulations applicable
                          to standard procurement contracts. Consequently, the arrangements include broader
                          latitude to negotiate terms and conditions than standard procurement contracts under the
                          Federal Acquisition Regulations.




                          Page 6                                                GAO-03-861 Defense Pilot Programs
    Figure 1: Status of Proposed Initiatives




    The four implemented initiatives were

•   donating laboratory equipment directly to local schools,
•   waiving top-level certification of certain service agreements with
    private industry,
•   streamlining cooperative agreements to facilitate collaborative work
    agreements with outside activities,10 and
•   granting temporary relief from some mandatory personnel placement
    reviews.11

    Officials at the laboratories that proposed these initiatives told us that they
    were considered minor changes with little impact on the larger problems
    facing the laboratories.




    10
      This initiative included several closely related but separate waivers that were grouped
    together by the service of the laboratories that proposed them.
    11
      This waiver expired in August 2002. It was extended until February 2003 as part of a
    separate pilot program sponsored by the DOD Business Initiative Council.




    Page 7                                                GAO-03-861 Defense Pilot Programs
                         Twelve times as many initiatives—24 percent—were implemented
                         using different authorities than the pilot programs. For example, several
                         laboratories requested the authority to appoint retired military members
                         to civilian positions without having to wait the required 180 days. This
                         requirement was waived using a different authority than the pilot
                         programs. Another human capital initiative—to appoint senior scientists
                         from private industry—was authorized by subsequent legislation.12 In the
                         business/partnership category, the 46th Test Group at Holloman Air Force
                         Base used other authorities to negotiate a complex leasing arrangement
                         with industry to install a radar test facility at White Sands Missile Range.
                         This effort took several years and overcame many contractual and
                         regulatory barriers. In addition, a Navy laboratory streamlined foreign
                         license applications using another authority.13


                         The low level of implementation of the proposed initiatives occurred for
The Pilot Programs       two primary reasons. First, DOD did not develop an effective process for
Were Not Effective for   implementing the pilot programs. Second, DOD determined that proposed
                         human capital initiatives—for example, requests for the authority to hire
Two Primary Reasons      directly or offer voluntary retirement incentives—were in conflict with
                         statutory provisions.


Lack of an Effective     DOD did not provide standardized guidance on proposal requirements
Implementation Process   or feedback for improving proposals; coordinate or prioritize proposals;
                         or clarify decision-making authority for proposal review and approval.
                         DOD also did not designate a strong focal point to coordinate the pilot
                         programs, advocate process improvements, and provide assistance and
                         advice to participants. The lack of a strong focal point exacerbated other
                         process gaps.

Lack of Guidance and     According to officials at DOD laboratories, test centers, and headquarters,
Coordination             DOD did not provide standardized guidance on proposal requirements or
                         feedback for improving proposals (or, in many cases, information on the
                         status of proposals submitted for approval). Proposals often lacked



                         12
                            Public Law 106-398, National Defense Authorization Act for Fiscal Year 2001,
                         section 1113.
                         13
                           The Office of the Secretary of Defense (OSD) and the Navy disagree on what authority
                         was used to implement this proposal. OSD believes that it was implemented using pilot
                         program authority.




                         Page 8                                                GAO-03-861 Defense Pilot Programs
                          specificity and detail. Many were broadly conceptual or generic in nature
                          and lacked a detailed business case that linked their contribution to
                          overall objectives for the pilot programs. For example, a proposal to
                          permit scientists to serve in a leadership role in professional societies
                          failed to include details of the problems encountered, and the potential to
                          improve operations. Similarly, several proposals for direct hire authority
                          failed to include a business case to explain what specific needs this
                          authority would address or how it would address them. Lack of specificity
                          and business case detail led to the failure of many initiatives to win
                          approval. DOD attorneys told us that many proposals were so vague that it
                          was impossible to determine whether or not the proposed initiatives could
                          meet legal requirements.

                          At a department level, DOD also did not coordinate or prioritize proposals,
                          thereby precluding decisions on how best to pursue common interests
                          and issues such as direct hiring authority or forming partnerships with
                          universities. Instead, each participant submitted proposals individually,
                          and thus multiple independent proposals were often submitted for the
                          same or similar issues. DOD attorneys pointed out that it would have
                          been more effective to group proposals by common theme and prioritize
                          them. They believed a unified approach and prioritized proposals with
                          clearly written, specific plans for solving well-defined problems would
                          have enabled them to more effectively assist participants with resolving
                          legal issues.

Unclear Decision-Making   DOD did not clarify decision-making authority for proposal review and
Authority                 approval. Many organizations and individuals were stakeholders in
                          proposal review and approval, and they often had differing management
                          structures, concerns, and interests. Stakeholders included military and
                          civilian leaders, attorneys, and human capital and personnel staff at
                          several levels: the local installation where participating laboratories and
                          test centers were housed; the individual service; and OSD. The roles
                          and decision-making authority of the various stakeholders were
                          never negotiated and clarified. As a result, many players at multiple
                          organizational levels had—and took—an opportunity to say “no” to a
                          particular proposal, but it remained unclear who had the authority to
                          say “yes.”

                          For example, some participants believed that the pilot program legislation
                          gave the director of a participating laboratory or test center the authority
                          to approve a proposed initiative. OSD officials, however, believed that the
                          proposed initiatives had to be approved at higher levels. The role of the
                          services was also unclear. Some laboratory and test center directors


                          Page 9                                        GAO-03-861 Defense Pilot Programs
                        initially sent proposals directly to OSD’s Directorate of Defense Research
                        and Engineering (DDR&E), bypassing their service headquarters. Others
                        sent proposals to their service headquarters for approval before
                        submitting the proposals to DDR&E. Eventually, however, each of the
                        service headquarters decided to become more heavily involved in the
                        approval process and provide service-level responses to proposals.
                        These service-level responses often came into play after proposals had
                        been sent directly to DDR&E for approval, further complicating the
                        approval process.

                        Within OSD, both DDR&E and Personnel and Readiness (P&R) had
                        substantial stakes in the human capital proposals—DDR&E because it
                        is charged with oversight and management of defense laboratories and
                        P&R because it has the authority within DOD for human capital issues.14
                        However, DDR&E and P&R never agreed on a process for approving
                        proposals. In addition, for the past year P&R’s attention has been focused
                        primarily on developing DOD’s proposed new civilian human capital
                        management system, the National Security Personnel System (NSPS),
                        which the Secretary of Defense recently submitted to the Congress. DOD
                        officials believe that, if enacted, NSPS will provide flexibility to make
                        necessary human capital changes.15 The Undersecretary of Defense P&R
                        directed that implementation of new personnel initiatives be placed on
                        hold during the development of NSPS so that the existing system could be
                        studied to identify needs and best practices. Consequently, P&R officials
                        believed it would be premature for DOD to implement new personnel
                        initiatives during this time.

No Strong Focal Point   DOD did not designate a strong focal point to coordinate the pilot
                        programs, advocate process improvements, and provide assistance and
                        advice to participants. This exacerbated the other process gaps. Without
                        such a focal point, participants found their own individual ways to develop
                        proposals and get them reviewed. Several officials agreed that a strong
                        focal point would be helpful. For example, DOD attorneys stated that the
                        laboratories or someone acting as their focal point needed to define the


                        14
                          Unlike the laboratories, the test centers are not overseen centrally but by the individual
                        service to which they belong.
                        15
                         The Comptroller General recently testified on NSPS. U. S. General Accounting Office.
                        Human Capital: DOD’s Civilian Personnel Strategic Management and the Proposed
                        National Security Personnel System. GAO-03-493T (Washington, D.C.; May 12, 2003).
                        Defense Transformation: DOD’s Proposed Civilian Personnel System and
                        Governmentwide Human Capital Reform. GAO-03-741T (Washington, D.C.; May 1, 2003).




                        Page 10                                                GAO-03-861 Defense Pilot Programs
                          issues they wanted to resolve. The attorneys noted that a focal point
                          could have more successfully drawn upon their expertise and experience
                          with addressing legal challenges in other innovative programs
                          (e.g., demonstration projects). Some pilot program participants also
                          agreed a strong focal point was needed, but they had some concerns
                          regarding the amount of influence and authority he or she should have.


Human Capital Proposals   According to officials at DOD laboratories, test centers, and headquarters,
Were in Conflict with     human capital initiatives were generally in conflict with title 5 of the
Existing Statutory        United States Code. Title 5 provides the framework for standard and
                          equitable personnel practices across the federal government and is the
Provisions                current foundation for management of the DOD civilian workforce. Over
                          time, the Office of Personnel Management has added implementing rules
                          and regulations to the framework. Proposed human capital initiatives
                          often sought relief from these provisions, for example, requests for the
                          authority to hire directly or offer voluntary retirement incentives.

                          However, after reviewing the legislation, the DOD Office of General
                          Counsel advised that the 1999 and 2000 legislation did not provide the
                          authority to waive personnel rules based on title 5 provisions. Rather, the
                          office advised that the pilot programs’ authorities allow only for changes
                          that could already be accomplished under existing DOD regulations. In
                          other words, the pilot programs did not provide any new or additional
                          authority to waive existing personnel rules and regulations grounded in
                          title 5. Consequently, absent statutory authority beyond that provided by
                          the pilot programs, human capital proposals in conflict with title 5 and its
                          implementing rules and regulations could not be implemented.16 Many
                          initiatives fell into this category.




                          16
                           Our attorneys reviewed the pilot program legislation and concurred with the DOD
                          General Counsel’s view.




                          Page 11                                            GAO-03-861 Defense Pilot Programs
                 The 2003 pilot program faces several implementation challenges. First,
The 2003 Pilot   as of May 2003, DOD had not addressed implementation problems. Thus,
Program Faces    proposals made via the 2003 pilot program will face the same obstacles as
                 previous proposals.
Implementation
Challenges       Second, human capital initiatives will continue to face title 5 challenges.
                 Like the earlier legislation, the 2003 legislation does not provide DOD any
                 new authority. Hence, initiatives proposed under the 2003 pilot program
                 will encounter the same statutory restrictions as previous initiatives. P&R
                 officials believe that, if implemented, NSPS will provide the flexibility to
                 make necessary human capital changes, thereby eliminating the need for
                 the pilot programs in this area. However, NSPS has not yet been enacted,
                 and if enacted, it will still require an implementation process.

                 Finally, laboratories and test centers may be reluctant to participate in the
                 new pilot program. Many participants in the earlier pilots told us they were
                 discouraged by their experience and consequently unwilling to repeat it.
                 Some expressed frustration with the lack of guidance and feedback on
                 their proposals; others questioned whether management was really
                 committed to the pilot program. Even those few participants that had
                 proposals approved were wary of expending additional resources on
                 another pilot program.


                 While DOD appears to recognize a need to address human capital and
Conclusions      business operations issues specific to laboratories and test centers, it has
                 not effectively managed the pilot programs. If DOD intends to use the
                 pilot programs to address laboratory and test center issues, it will have to
                 address the factors—both process and statutory—that blunted previous
                 proposals made through the pilot programs. The small volume of approved
                 proposals, coupled with DOD’s not providing status reports required by
                 the Congress, has left the Congress uninformed about what objectives
                 DOD would like to achieve with the laboratories and test centers, how it
                 plans to achieve those objectives, and what vehicles it plans to use. This
                 information will be important to the success of any future actions.




                 Page 12                                      GAO-03-861 Defense Pilot Programs
                         We recommend that by March 31, 2004, the Secretary of Defense inform
Recommendations for      the Congress of DOD’s objectives regarding human capital and business
Executive Action         operations in the laboratories and test centers, how it plans to meet these
                         objectives, and what vehicles it will use to meet them.

                         We also recommend that by March 31, 2004, the Secretary of Defense
                         develop a process for proposing, evaluating, and implementing human
                         capital, business, and partnership initiatives for the laboratories and test
                         centers, regardless whether by the pilot authority or by some other
                         vehicle. Such a process should include

                     •   clear decision-making authority,
                     •   instructions for proposal requirements such as linking to overall goals and
                         measurable objectives and the need for a business case, and
                     •   specification of procedures for proposal submission and review and
                         providing feedback on proposal quality and scope.

                         Finally, we recommend that the Secretary of Defense designate a strong
                         focal point to

                     •   receive, evaluate, and prioritize all proposals and
                     •   work with laboratory and test center directors, legal counsel, personnel
                         and other specialists to develop sound and well-developed business cases
                         and strategies to obtain needed changes.


                         In written comments on a draft of this report, DOD states that it does not
Agency Comments          concur with our recommendations because it has already taken actions
and Our Evaluation       that in effect implement them.17 While the actions DOD cites that it has
                         taken are important to implementing our recommendations, they are not
                         sufficiently specific to address the problems identified in our report.
                         DOD’s written comments are contained in appendix II.

                         Regarding our first recommendation—that DOD inform the Congress of its
                         human capital and business objectives for the laboratories and test centers
                         and the strategies it will employ to meet them—DOD did not concur. DOD
                         discusses various high-level, agencywide initiatives it has taken to address
                         human capital and business issues in general and stated that the Congress


                         17
                           In its letter, DOD refers to the “1999 and 2000 demonstration programs.” We confirmed
                         with DDR&E that these demonstration programs were indeed the 1999 and 2000 pilot
                         programs as described in this report.




                         Page 13                                              GAO-03-861 Defense Pilot Programs
has been made aware of these initiatives, obviating the need for additional
reporting. We continue to believe that additional reporting is necessary.
We recognize that the general initiatives DOD discusses may provide ways
of helping the laboratories and test centers; however, to be effective, they
must be made specific, that is, developed into targeted strategies and plans
that address the particular problems the laboratories and test centers face.
DOD has not provided the Congress sufficient details on how the general
initiatives will be used to address laboratories’ and test centers’ objectives
and problems.

Regarding our second recommendation—that DOD develop a process for
proposing, evaluating, and implementing human capital and business-like
practices initiatives for the laboratories and test centers—DOD did not
concur. DOD states that it has already introduced new agencywide
management processes—the Business Initiative Council and the
submission of the NSPS proposal to the Congress—to address human
capital and business issues in general. However, DOD has not detailed
how these general initiatives will apply to the laboratories and test centers
or address our process concerns. For example, while the Business
Initiative Council may have an effective process for proposing, evaluating,
and implementing laboratory and test center business-like practices
initiatives, DOD has not provided sufficient information for us to make
such a determination. We also recognize that NSPS may address some of
the human capital problems faced by the laboratories and test centers, but
this system is still under consideration by the Congress. Until it becomes
law, we believe it is premature to cite it as an effective management tool.

With regard to our third recommendation—that DOD designate a strong
focal point to work with the laboratories and test centers to develop,
evaluate, prioritize, and coordinate proposed initiatives—DOD did not
concur. DOD states that the recently created position of Undersecretary
for Laboratories and Basic Sciences has oversight responsibility for all
laboratory initiatives and that it is establishing a new Defense Test
Resources Management Center that will oversee the test centers.18 DOD
asserts that these two organizations will perform as focal points. However,
DOD has not detailed how these organizations will fulfill this role and
work with the laboratories and test centers to overcome the many barriers
noted in our report.




18
     This new center is not yet operational.




Page 14                                        GAO-03-861 Defense Pilot Programs
              During our review, we met with officials from the following organizations
Scope and     in the Office of the Secretary of Defense: the Director, Defense Research
Methodology   and Engineering; the Director, Operational Test and Evaluation; the
              General Counsel, and the Deputy Undersecretary of Defense for Personnel
              and Readiness. We also met with officials from the Army Research
              Laboratory, Aberdeen Test Center, Army Medical Research and Materiel
              Command, Naval Research Laboratory, Naval Undersea Warfare Center,
              Air Force Research Laboratory, Air Force Research Laboratory’s Space
              Vehicles Directorate, and 46th Test Wing. We also discussed pilot program
              issues with each participating laboratory or center.

              To determine the initiatives proposed to date and their status, we obtained
              records from OSD and service officials. From these records and from
              discussions with each participant, we compiled a listing of initiatives
              proposed by each participating laboratory and test center. We verified
              the listing and the current status of each initiative with cognizant
              service officials.

              To determine what obstacles inhibited DOD’s implementation of the pilot
              programs, we obtained documentation and data from pilot program
              participants as well as from OSD officials. We also discussed statutory
              obstacles with the officials from DOD’s Office of General Counsel and
              Undersecretary of Defense for Personnel and Readiness. We discussed
              management and procedural obstacles with officials from the Director,
              Operational Test and Evaluation and Defense Research and Engineering.
              In addition, we discussed all obstacles with the participating laboratories
              and test centers.

              The problems facing the laboratories and test centers have been
              documented by many organizations, panels, and commissions. We did
              not independently verify these problems or the findings and conclusions
              of these entities. We conducted our review from July 2002 to April 2003
              in accordance with generally accepted government auditing standards.


              We are sending copies of this report to the Secretary of Defense; the
              Secretaries of the Army, Navy, and Air Force; and interested congressional
              committees. We will also make copies available to others upon request. In
              addition, the report will be available at no charge on the GAO Web site at
              http://www.gao.gov.




              Page 15                                      GAO-03-861 Defense Pilot Programs
Major contributors to this report were Catherine Baltzell, Arthur Cobb,
Christopher Durbin, Rae Ann Sapp, Sylvia Schatz, and Katrina Taylor.
If you have any questions regarding this report, please call me at
(202) 512-4841.




Paul L. Francis
Director
Acquisition and Sourcing Management




Page 16                                     GAO-03-861 Defense Pilot Programs
               Appendix I: Fiscal Year 1999 and 2000 Pilot
Appendix I: Fiscal Year 1999 and 2000 Pilot
               Program Participants



Program Participants


                Laboratory/test center                                     1999 pilot   2000 pilot
                Army Research Laboratory                                                X
                Army Medical Research and Materiel Command                 X
                Aberdeen Test Center                                       X            X
                Naval Research Laboratory                                  X
                Naval Undersea Warfare Center                                           X
                Naval Air Warfare Center–Aircraft and Weapons Divisions    X            X
                Air Force Research Laboratory–Information Directorate                   X
                Air Force Research Laboratory–Space Vehicles Directorate   X
                Air Armament Center, 46th Test Wing                                     X
                Arnold Engineering Development Center                      X
               Source: DOD.




               Page 17                                            GAO-03-861 Defense Pilot Programs
             Appendix II: Comments from the Department
Appendix II: Comments from the Department
             of Defense



of Defense




             Page 18                                     GAO-03-861 Defense Pilot Programs
Appendix II: Comments from the Department
of Defense




Page 19                                     GAO-03-861 Defense Pilot Programs
Appendix II: Comments from the Department
of Defense




Page 20                                     GAO-03-861 Defense Pilot Programs
           Appendix II: Comments from the Department
           of Defense




(120159)
           Page 21                                     GAO-03-861 Defense Pilot Programs
                         The General Accounting Office, the audit, evaluation and investigative arm of
GAO’s Mission            Congress, exists to support Congress in meeting its constitutional responsibilities
                         and to help improve the performance and accountability of the federal
                         government for the American people. GAO examines the use of public funds;
                         evaluates federal programs and policies; and provides analyses,
                         recommendations, and other assistance to help Congress make informed
                         oversight, policy, and funding decisions. GAO’s commitment to good government
                         is reflected in its core values of accountability, integrity, and reliability.


                         The fastest and easiest way to obtain copies of GAO documents at no cost is
Obtaining Copies of      through the Internet. GAO’s Web site (www.gao.gov) contains abstracts and full-
GAO Reports and          text files of current reports and testimony and an expanding archive of older
                         products. The Web site features a search engine to help you locate documents
Testimony                using key words and phrases. You can print these documents in their entirety,
                         including charts and other graphics.
                         Each day, GAO issues a list of newly released reports, testimony, and
                         correspondence. GAO posts this list, known as “Today’s Reports,” on its Web site
                         daily. The list contains links to the full-text document files. To have GAO e-mail
                         this list to you every afternoon, go to www.gao.gov and select “Subscribe to e-mail
                         alerts” under the “Order GAO Products” heading.


Order by Mail or Phone   The first copy of each printed report is free. Additional copies are $2 each. A
                         check or money order should be made out to the Superintendent of Documents.
                         GAO also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a
                         single address are discounted 25 percent. Orders should be sent to:
                         U.S. General Accounting Office
                         441 G Street NW, Room LM
                         Washington, D.C. 20548
                         To order by Phone:     Voice:    (202) 512-6000
                                                TDD:      (202) 512-2537
                                                Fax:      (202) 512-6061


                         Contact:
To Report Fraud,
                         Web site: www.gao.gov/fraudnet/fraudnet.htm
Waste, and Abuse in      E-mail: fraudnet@gao.gov
Federal Programs         Automated answering system: (800) 424-5454 or (202) 512-7470


                         Jeff Nelligan, Managing Director, NelliganJ@gao.gov (202) 512-4800
Public Affairs           U.S. General Accounting Office, 441 G Street NW, Room 7149
                         Washington, D.C. 20548