oversight

Records Management: National Archives and Records Administration's Acquisition of Major System Faces Risks

Published by the Government Accountability Office on 2003-08-22.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

              United States General Accounting Office

GAO           Report to the Chairman, Subcommittee
              on Transportation, Treasury and
              Independent Agencies, Committee on
              Appropriations, House of
              Representatives
August 2003
              RECORDS
              MANAGEMENT
              National Archives and
              Records
              Administration’s
              Acquisition of Major
              System Faces Risks




GAO-03-880
              a
                                                August 2003


                                                RECORDS MANAGEMENT

                                                National Archives and Records
Highlights of GAO-03-880, a report to the       Administration’s Acquisition of Major
Chairman, Subcommittee on
Transportation, Treasury and Independent        System Faces Risks
Agencies, Committee on Appropriations,
House of Representatives




Increasingly, government records                The ERA program’s acquisition policies, plans, and practices do not
involve documents that are                      consistently conform to industry standards. In developing the plans and
electronically created and stored.              policies to guide its acquisition of the ERA system, NARA elected to
In support of its mission to manage
and archive these records and
                                                follow recognized industry standards set forth by the Institute of
ensure access to the “essential                 Electrical and Electronics Engineers (IEEE). However, key policy and
evidence” that they contain, the                planning documents are missing elements that are required by the
National Archives and Records                   standards. For example, one key document is the concept of operations,
Administration (NARA) is acquiring              which should describe the characteristics of a proposed system from the
an advanced Electronic Records                  users’ viewpoint. The ERA Concept of Operations does not include
Archives (ERA). GAO was asked to                several key elements required by the IEEE standard, including a
determine, among other things,
                                                complete description of the proposed system. Because these policy and
how the ERA program’s system
acquisition policies, plans, and                planning documents form the basis of the acquisition, such shortcomings
practices conform to industry                   could result in serious long-term risks to the cost, schedule, and
standards and how well NARA is                  performance of the ERA program.
meeting the ERA program’s cost
and schedule.                                   NARA cannot adequately track the cost and schedule of the ERA
                                                program. A comprehensive schedule with an appropriate work
                                                breakdown structure is a prerequisite to program tracking, as it allows
To reduce the risks associated with             managers to measure how well the program is achieving its cost and
NARA’s efforts to design and                    schedule goals. To achieve upcoming major milestones (some of which
acquire ERA, GAO recommends                     are shown in the figure), the program must successfully complete a
that the U.S. Archivist direct the              complex series of tasks. However, the program schedule omits
NARA Chief Information Officer to               significant tasks and activities; for example, it does not include the
take a range of actions, including
revising key planning documents
                                                process to reengineer the agency’s life cycle business processes, which
and developing a schedule that is               will be crucial to defining requirements. In addition, the schedule lacks a
based on a comprehensive work                   work breakdown structure, which would allow accurate estimates of the
breakdown structure (including                  resources and time required for each work activity. If NARA cannot track
associated costs and other                      how well the program is meeting cost and schedule, the risk is increased
resources).                                     that funds may not be used efficiently or effectively, quality problems
                                                may limit the usefulness of the resulting system, and the system may not
In comments on the draft report,
                                                be delivered according to established milestones.
the Archivist of the United States
accepted our recommendations
and provided an update on NARA’s                Major Milestones in ERA Acquisition
efforts to implement them. The
Archivist also provided additional
information on the ERA acquisition
schedule.




www.gao.gov/cgi-bin/getrpt?GAO-03-880.

To view the full product, including the scope
and methodology, click on the link above.
For more information, contact Linda Koontz at
(202) 512-6240 or koontzl@gao.gov.
Contents



Letter                                                                                                                 1
                            Recommendations for Executive Action                                                       4
                            Agency Comments                                                                            4


Appendixes
             Appendix I:    National Archives and Records Administration’s Acquisition
                            of Electronic Records Archives                                                             6
             Appendix II:   Comments from the National Archives and Records
                            Administration                                                                         49




                            Abbreviations

                            ERA          Electronic Records Archives
                            IEEE         Institute of Electrical and Electronics Engineers
                            IT           information technology
                            NARA         National Archives and Records Administration


                             This is a work of the U.S. government and is not subject to copyright protection in the
                             United States. It may be reproduced and distributed in its entirety without further
                             permission from GAO. However, because this work may contain copyrighted images or
                             other material, permission from the copyright holder may be necessary if you wish to
                             reproduce this material separately.




                            Page i                                          GAO-03-880 Electronic Archive Acquisition
A
United States General Accounting Office
Washington, D.C. 20548



                                    August 22, 2003                                                                       Leter




                                    The Honorable Ernest J. Istook, Jr.
                                    Chairman
                                    Subcommittee on Transportation, Treasury and Independent Agencies
                                    Committee on Appropriations
                                    House of Representatives

                                    Dear Mr. Chairman:

                                    The National Archives and Records Administration (NARA) has initiated
                                    the Electronic Records Archives (ERA), a project to acquire a major
                                    information system to maintain and provide access to permanent federal
                                    records independent of the technological state of the art and the varieties
                                    of record formats. NARA’s goal is for this system to preserve and provide
                                    access to any kind of electronic record, so that the agency can carry out its
                                    mission into the future. However, as we have reported previously,1
                                    acquiring a major information technology (IT) system like ERA is a
                                    significant challenge for a relatively small organization such as NARA,
                                    which has no previous experience in acquiring major information systems.

                                    Our objectives were to determine

                                    1. the status of NARA’s efforts to establish organizational capabilities for
                                       acquiring major information systems,

                                    2. how the ERA project’s system acquisition policies, plans, and practices
                                       conform to industry standards, and

                                    3. how well NARA is meeting the ERA project’s cost and schedule goals.

                                    To achieve these objectives, we reviewed agency information technology
                                    policies and practices, and we obtained and analyzed ERA program
                                    documents on system acquisition, project management, and cost and
                                    schedule. We evaluated ERA documents and practices by the standards
                                    selected by the program to guide the ERA acquisition, including specifically
                                    those of the Institute of Electrical and Electronics Engineers (IEEE). We
                                    also interviewed NARA information resources management and ERA


                                    1
                                     U.S. General Accounting Office, Information Management: Challenges in Managing and
                                    Preserving Electronic Records, GAO-02-586 (Washington, D.C.: June 17, 2002).




                                    Page 1                                      GAO-03-880 Electronic Archive Acquisition
program officials. We performed our work from July 2002 to May 2003 in
accordance with generally accepted government auditing standards.

On May 15, 2003, we provided your staff with a briefing on the results of
our study, which included procurement-sensitive information. The slides
from that briefing—with procurement-sensitive information removed—are
included as appendix I to this report. The purpose of this report is to
provide the published briefing slides to you and to officially transmit our
recommendations to the Archivist of the United States.

In summary, our briefing made three points:

• To establish its capabilities for acquiring major information systems,
  NARA has made progress in implementing the key management areas of
  IT investment management, enterprise architecture, and IT security.
  However, these capabilities are not yet completely established, and
  NARA has more work to do to implement our prior recommendations in
  this area.2 Specifically, while NARA continues to develop an enterprise
  architecture, it does not plan to complete its target architecture3 in time
  to influence the ERA system definition and requirements. Furthermore,
  while NARA has completed some elements of an information security
  program, several key areas have not yet been addressed, such as
  (1) individual system security plans and (2) security certification and
  accreditation of its information systems.4 Without strong IT
  management capabilities, NARA increases its risk of failing to achieve



2
 U.S. General Accounting Office, Information Management: Challenges in Managing and
Preserving Electronic Records, GAO-02-586 (Washington, D.C.: June 17, 2002).
3
 A target architecture is one aspect of an overall enterprise architecture. An enterprise
architecture describes (in useful models, diagrams, and narrative) the mode of operation for
an enterprise, such as an agency or mission area. It provides a perspective on enterprise
operations both for the current (“as is”) operating environment and for the target (“to be”)
environment. More specifically, the target environment is the business and technology
environment that is planned to result from aligning technology investments with the
strategic goals of the enterprise (including requisite changes to the operations, organization,
and management of both the automated and manual processes of the enterprise). An
enterprise architecture also includes a transition plan for sequencing from the current to the
target environment.
4
 Under OMB policy, responsible federal officials are required to make a security
determination (called accreditation) to authorize placing IT systems into operation. In order
for these officials to make sound, risk-based decisions, a security evaluation (known as
certification) of the IT system is needed.




Page 2                                            GAO-03-880 Electronic Archive Acquisition
   cost, schedule, and performance objectives for its information systems,
   including ERA.

• The ERA program has developed policies, plans, and practices to guide
  and manage its acquisition of the ERA system. In many cases, however,
  these do not conform to the chosen standards or to applicable federal
  acquisition guidance. In developing its plans and policies, NARA elected
  to follow recognized industry standards set forth by IEEE. However, key
  policy and planning documents are missing elements that are required
  by the standards. For example, one key document is the concept of
  operations, which should describe the characteristics of a proposed
  system from the users’ viewpoint. The ERA Concept of Operations does
  not include several key elements required by the IEEE standard,
  including a complete description of the proposed systems. In addition,
  key ERA staff positions are unfilled, including positions that NARA
  determined are needed to carry out system acquisition tasks. Without
  adequate policy and planning documents—which form the basis of the
  acquisition—and adequate staff to carry out these policies and plans,
  NARA increases the long-term risks to the acquisition.

• Finally, NARA cannot adequately track the cost and schedule of the ERA
  project because the schedule does not include all program tasks and
  lacks a work breakdown structure.5 In addition, NARA has not used
  earned value management—a performance-based technique that allows
  managers to track the budget against the schedule—to track the ERA
  cost and schedule programwide. Without the ability to track cost and
  schedule effectively, NARA increases the risk that ERA funds will not be
  used efficiently or effectively, quality problems will limit the usefulness
  of the ERA system, and the ERA system will not be delivered according
  to established milestones.

In light of the challenges NARA faces in acquiring ERA, NARA will face
significant difficulties unless it addresses the weaknesses described above.



5
 A work breakdown structure provides descriptions of all work activities for a given project
that are detailed enough to expose risk factors and allow accurate estimates of resource
requirements and schedule duration for each work activity. Each major work activity should
include standard elements such as assigned personnel, resource budgets, estimated task
duration, and dependencies among work activities. An adequate work breakdown structure
is a prerequisite to program tracking, allowing managers to measure how well a program is
achieving its cost and schedule goals.




Page 3                                           GAO-03-880 Electronic Archive Acquisition
Recommendations for   To reduce the risks associated with NARA’s efforts to design and acquire
                      the Electronic Records Archives, we recommend that the U.S. Archivist
Executive Action      direct the NARA Chief Information Officer to address weaknesses in the
                      acquisition policies, plans, and practices by

                      • revising the ERA Life Cycle document and associated procedures and
                        practices to conform to IEEE standards;

                      • revising the ERA Concept of Operations to conform to IEEE standards,
                        including a complete description of the current and proposed systems;

                      • revising the ERA Acquisition Strategy to conform to IEEE standards
                        and the Federal Acquisition Regulation;

                      • revising the ERA Risk Management Plan to provide processes and
                        procedures specific to the ERA program;

                      • revising the ERA Quality Assurance Plan to conform to appropriate
                        industry standards, establishing a vigorous, independent ERA quality
                        assessment process, and providing the staffing resources necessary to
                        ensure that quality assessment duties are performed effectively; and

                      • filling key vacant ERA positions.

                      Further, we recommend that the U.S. Archivist direct the NARA Chief
                      Information Officer to immediately address weaknesses in tracking cost
                      and schedule by

                      • developing an ERA schedule that is based on a comprehensive work
                        breakdown structure (including associated costs and other resources)
                        and establishes dependencies between successor and predecessor
                        tasks; and

                      • using earned value management to capture and monitor progress for the
                        entire ERA program.



Agency Comments       In providing written comments on a draft of this report (which are
                      reprinted in app. II), the Archivist of the United States indicated that NARA
                      is acting to implement our recommendations and provided an update on
                      the status of the agency’s efforts to do so. In addition, the Archivist



                      Page 4                                   GAO-03-880 Electronic Archive Acquisition
provided a clarification regarding the ERA acquisition schedule, stating
that there will be two to three releases for each of the increments in the
schedule.


We are sending copies of this report to the Chairmen and Ranking Minority
Members of the Subcommittee on Transportation, Treasury and
Independent Agencies, House Committee on Appropriations; the
Subcommittee on Transportation, Treasury and General Government,
Senate Committee on Appropriations; the Subcommittee on Technology,
Information Policy, Intergovernmental Relations and the Census, House
Committee on Government Reform; and the Subcommittee on Oversight of
Government Management, the Federal Workforce and the District of
Columbia, the Senate Committee on Governmental Affairs. We are also
sending copies to the Archivist of the United States. We will also make
copies available to others upon request. In addition, the report will be
available at no charge on the GAO Web site at www.gao.gov.

Should you have any question on matters contained in this report, please
contact me at (202) 512-6240 or by E-mail at koontzl@gao.gov. Other key
contributors to this report were Timothy Case, Barbara Collier, Mirko
Dolak, and Elena Epps.

Sincerely yours,




Linda D. Koontz
Director, Information Management Issues




Page 5                                   GAO-03-880 Electronic Archive Acquisition
Appendix I

National Archives and Records                                                         Appendx
                                                                                            ies




Administration’s Acquisition of Electronic
Records Archives                                                                       Append
                                                                                            x
                                                                                            Ii




     National Archives and Records Administration’s
     Acquisition of Electronic Records Archives

     Briefing for the Staff of the
     Subcommittee on Transportation,
     Treasury and Independent Agencies


     House Appropriations Committee


     May 15, 2003




                                                                                 1




                                     Page 6   GAO-03-880 Electronic Archive Acquisition
                              Appendix I
                              National Archives and Records
                              Administration’s Acquisition of Electronic
                              Records Archives




                                                                                    Outline of Briefing


Introduction
Objectives, Scope, and Methodology
Results in Brief
Background
Results
  • IT Organizational Capabilities
  • Systems Acquisition Policies, Plans, and Practices
  • Cost and Schedule
Conclusions
Recommendations
Agency Comments




                                                                                                             2




                              Page 7                                       GAO-03-880 Electronic Archive Acquisition
                              Appendix I
                              National Archives and Records
                              Administration’s Acquisition of Electronic
                              Records Archives




                                                                                              Introduction


The mission of the National Archives and Records Administration (NARA) is to ensure “ready
access to essential evidence” for the public, the President, the Congress, and the Courts.
NARA is responsible for oversight of records management and archiving, which increasingly
involves dealing with documents that are electronically created and stored. Accordingly,
NARA is acquiring an advanced Electronic Records Archives (ERA) system.
  • NARA’s goals are for this system to preserve and provide access to any kind of
    electronic record, free from dependency on any specific hardware or software, so that
    the agency can carry out its mission into the future.
  • According to NARA, the ERA system will be a distributed system, allowing storage and
    management of massive record collections at a variety of installations, with accessibility
    provided via the Internet.




                                                                                                             3




                              Page 8                                       GAO-03-880 Electronic Archive Acquisition
                                       Appendix I
                                       National Archives and Records
                                       Administration’s Acquisition of Electronic
                                       Records Archives




                                                                                                       Introduction


In June 2002, we issued a report that assessed the ERA program and NARA’s key
organizational capabilities for acquiring a major information system.1 As we noted in that
report, acquiring a major information technology (IT) system such as ERA is a significant
challenge for a relatively small organization like NARA, whose IT management capabilities
are relatively limited. NARA has no previous experience in acquiring major information
systems.
In our 2002 report we recommended that the Archivist
    • revise the ERA project schedule so that it reflected estimates of the amount of work and
      resources required to complete each task, and schedule enough time for NARA to
      complete essential planning tasks, and
    • strengthen IT management capabilities by
        (1) implementing an IT investment management process,
        (2) developing an enterprise architecture, and
        (3) improving information security.



1
 U.S. General Accounting Office, Information Management: Challenges in Managing and Preserving Electronic Records,
GAO-02-586 (Washington, D.C.: June 17, 2002).

                                                                                                                      4




                                       Page 9                                       GAO-03-880 Electronic Archive Acquisition
                               Appendix I
                               National Archives and Records
                               Administration’s Acquisition of Electronic
                               Records Archives




                                                             Objectives, Scope, and Methodology


Objectives
As agreed with the Subcommittee on Transportation, Treasury and Independent Agencies,
House Appropriations Committee, our work focused on three objectives:
  • What is the status of NARA’s efforts to establish organizational capabilities for acquiring
    major information systems?
  • How do the ERA project’s system acquisition policies, plans, and practices conform to
    industry standards?
  • How well is NARA meeting the ERA project’s cost and schedule goals?




                                                                                                              5




                               Page 10                                      GAO-03-880 Electronic Archive Acquisition
                              Appendix I
                              National Archives and Records
                              Administration’s Acquisition of Electronic
                              Records Archives




                                                            Objectives, Scope, and Methodology


Scope and Methodology
To accomplish our objectives, we
  • obtained and evaluated policies and conducted interviews to determine the status of
    NARA’s efforts to establish organizational capabilities in IT investment management,
    enterprise architecture, and IT security;
  • analyzed ERA system acquisition and project management documents, including the
    Acquisition Strategy, Life Cycle, and Concept of Operations.
  • analyzed the project’s cost and schedule documents;
  • analyzed ERA risk management, program assessment, and verification and validation
    reports; and
  • interviewed NARA information resource management and ERA program officials and
    contractor staff to understand the scope and contents of project documents and plans
    and to clarify information in supporting documentation.
In our review of the ERA program’s policies, plans, and practices, we evaluated the
implementation of life cycle processes against the Institute of Electrical and Electronics
Engineers (IEEE) standards selected by the program to guide the ERA acquisition, as well
as applicable federal acquisition regulations and guidance.

                                                                                                             6




                              Page 11                                      GAO-03-880 Electronic Archive Acquisition
                              Appendix I
                              National Archives and Records
                              Administration’s Acquisition of Electronic
                              Records Archives




                                                            Objectives, Scope, and Methodology


We also reviewed NARA’s efforts to staff the ERA acquisition program.
We have requested the revised life cycle cost estimates for the ERA system, but these have
not yet been provided. Consequently, we are currently unable to assess the current
estimated life cycle costs and benefits of the proposed system.
We performed our work from July 2002 to May 2003 in accordance with generally accepted
government auditing standards.




                                                                                                             7




                              Page 12                                      GAO-03-880 Electronic Archive Acquisition
                                Appendix I
                                National Archives and Records
                                Administration’s Acquisition of Electronic
                                Records Archives




                                                                    Results in Brief: Objective 1
                                                         Information Technology Management

NARA has made some progress in establishing capabilities for acquiring major information
systems, but key IT organizational capabilities are not yet completely established. NARA is
still working to establish its information technology investment management, enterprise
architecture, and information security capabilities:
  • NARA has strengthened its information technology investment management.
  • NARA is continuing to develop an enterprise architecture, but it does not plan to
    complete its target architecture in time to influence the ERA system definition and
    requirements.
  • NARA completed some elements of an information security program, but several key
    areas have not yet been addressed, such as individual system security plans and
    certification and accreditation of its information systems.
Without strong IT organizational capabilities in place, the risk is increased that NARA will not
be able to achieve cost, schedule, and performance objectives for information systems,
including ERA.




                                                                                                               8




                                Page 13                                      GAO-03-880 Electronic Archive Acquisition
                               Appendix I
                               National Archives and Records
                               Administration’s Acquisition of Electronic
                               Records Archives




                                                                            Results in Brief: Objective 2
                                                                                 Systems Acquisition

The ERA program has developed policies, plans, and practices to guide and manage its
acquisition of the ERA system, and has elected to follow Institute of Electrical and
Electronics Engineers (IEEE) standards in this effort. The ERA program intends to apply
these standards to the development of the plans and policies that will guide the practices to
be followed in acquiring the ERA system. However, in many cases, its policies, plans, and
practices do not conform to the chosen standards or to applicable federal acquisition
guidance. For example:
  • Key policy and planning documents (such as the ERA Risk Management Plan, Quality
    Assurance Plan, and Acquisition Strategy) are missing elements that are required by the
    standards and federal acquisition guidance.
Such shortcomings in ERA policies, plans, and practices could result in serious long-term
risks to the cost, schedule, and performance of the ERA program.




                                                                                                                 9




                               Page 14                                         GAO-03-880 Electronic Archive Acquisition
                               Appendix I
                               National Archives and Records
                               Administration’s Acquisition of Electronic
                               Records Archives




                                                                         Results in Brief: Objective 3
                                                                       Tracking Cost and Schedule

NARA is unable to objectively track the cost and schedule of the ERA project. The ERA
schedule does not include all program tasks and lacks a work breakdown structure.
If NARA cannot track how well the ERA program is meeting cost and schedule, the likelihood
is increased that ERA funds will not be used efficiently or effectively, quality problems will
limit the usefulness of the ERA system, and the ERA system will not be delivered according
to established milestones.




To reduce the risks associated with NARA’s efforts to design and acquire the Electronic
Records Archive, we are making several recommendations to the U.S. Archivist.
In oral comments on a draft of this briefing, NARA generally agreed with our findings,
conclusions, and recommendations.




                                                                                                             10




                               Page 15                                      GAO-03-880 Electronic Archive Acquisition
                                        Appendix I
                                        National Archives and Records
                                        Administration’s Acquisition of Electronic
                                        Records Archives




                                                                                                    Background
                                                                                           Acquisition Strategy

Developing the ERA Acquisition Strategy
NARA envisions ERA to be a major information system with the ability to authentically
preserve and provide access to massive volumes of all types and formats of electronic
records, free from dependency on any specific hardware or software. The system may be
based on persistent object preservation, a method of accommodating a variety of file formats
that is the subject of research sponsored by NARA and other organizations. A leading
candidate for capturing the necessary information is the Extensible Markup Language (XML),
which provides a means for “tagging” (annotating) information in a meaningful fashion that
can be readily interpreted by disparate computer systems.3




3
 For more information on XML, see our earlier report: U.S. General Accounting Office, Electronic Government: Challenges to
Effective Adoption of the Extensible Markup Language, GAO-02-327 (Washington, D.C.: Apr. 5, 2002).


                                                                                                                         11




                                        Page 16                                        GAO-03-880 Electronic Archive Acquisition
                                        Appendix I
                                        National Archives and Records
                                        Administration’s Acquisition of Electronic
                                        Records Archives




                                                                                                     Background
                                                                                            Acquisition Strategy

However, as pointed out in our earlier report,4 the solution to the electronic records archiving
challenge has yet to be developed. No electronic archive system exists that is comparable in
complexity or scale to NARA’s vision.5 Further, some key technologies associated with
electronic document archiving and storage are not yet available commercially. As a result,
ERA is technically and managerially complex and challenging, requiring the development of
an advanced architecture for the conversion and preservation of electronic records.
To guide its acquisition of the ERA system, the ERA program has adopted IEEE standards
for software life cycle processes.6 The standards establish a common framework for the
acquisition of software products and services, defines processes and tasks that are to be
tailored and applied during the acquisition of a system, and identifies specific guidance
applicable to the acquisition process. One standard identifies five primary processes:
acquisition, supply, development, operations, and maintenance.

4
 U.S. General Accounting Office, Information Management: Challenges in Managing and Preserving Electronic Records,
GAO-02-586 (Washington, D.C.: June 17, 2002).
5
 NARA officials believe that many relevant hardware and software components are available in the marketplace, and that
potential integrators have both an understanding of NARA’s needs and appropriate strategies for addressing them.
6
  The Institute of Electrical and Electronics Engineers, 12207.0 Standard for Information Technology—Software Life Cycle
Processes; 12207.1 Standard for Information Technology—Software Life Cycle Processes—Life Cycle Data; and 12207.2
Standard for Information Technology—Software Life Cycle Processes—Implementation Considerations.


                                                                                                                           12




                                        Page 17                                         GAO-03-880 Electronic Archive Acquisition
                               Appendix I
                               National Archives and Records
                               Administration’s Acquisition of Electronic
                               Records Archives




                                                                                         Background
                                                                                Acquisition Strategy

The ERA program is currently in the acquisition life cycle process. It plans to take the
following acquisition approach:
  • define the ERA concept,
  • develop ERA requirements,
  • release a draft and final Request for Proposal,
  • award two contracts for a design competition, and
  • based on the results of the competition, select a single contractor who will develop the
    ERA system in five increments, beginning in January 2005.
ERA also needs to follow applicable federal acquisition requirements for major information
systems. Specifically, it was required to submit to OMB an evaluation of alternatives to be
considered and to develop life cycle cost and benefit estimates of the proposed ERA system.
According to the acquisition schedule, shown on the following slide, development of the first
increment will take 2 years to complete and will encompass key ERA capabilities related to
the records management process, while subsequent increments will add additional
functionality and record types.
The expected completion date for the development of the ERA system is December 2010.


                                                                                                             13




                               Page 18                                      GAO-03-880 Electronic Archive Acquisition
                           Appendix I
                           National Archives and Records
                           Administration’s Acquisition of Electronic
                           Records Archives




                                                                                     Background
                                                                            Acquisition Strategy

ERA Acquisition Schedule




                                                                                                         14




                           Page 19                                      GAO-03-880 Electronic Archive Acquisition
                              Appendix I
                              National Archives and Records
                              Administration’s Acquisition of Electronic
                              Records Archives




                                                                                    Background
                                                                           Other NARA Initiatives

Other NARA Initiatives Related to Electronic Records
Besides the ERA acquisition, NARA’s strategy for archiving electronic records includes,
among other things, three other initiatives:
  • Expanding the capability of its current systems for archiving electronic records by
    accommodating additional electronic record formats and volumes.
  • Through the Records Management Initiatives, revising NARA’s records disposition
    policies and processes, including reengineering of NARA’s life cycle management of
    records.
  • Through the Electronic Records Management initiative, providing the guidance and
    tools that agencies need to manage their records in electronic form, including policy
    guidance on new electronic record formats accepted by NARA.




                                                                                                            15




                              Page 20                                      GAO-03-880 Electronic Archive Acquisition
                               Appendix I
                               National Archives and Records
                               Administration’s Acquisition of Electronic
                               Records Archives




                                                                                        Background
                                                                            Technical Assessments

National Academy of Sciences Assessment of ERA
NARA has funded two independent assessments of ERA by the National Academy of
Sciences.
The first assessment, due in May 2003, is to assess the viability of an advanced architecture
for file format conversion and encapsulation being researched by the National Partnership for
Advanced Computational Infrastructure.
The second assessment is to
  • identify and evaluate alternative methods for digital preservation of records,
  • examine the operational use of the Internet for digital archiving,
  • identify aspects of the preservation of electronic records that cannot be adequately
    addressed either by state-of-the art information technology or by technologies under
    development, and
  • determine the feasibility of commercializing new ideas from research.
It is to be completed from 6 to 9 months after the first assessment.




                                                                                                              16




                               Page 21                                       GAO-03-880 Electronic Archive Acquisition
                                        Appendix I
                                        National Archives and Records
                                        Administration’s Acquisition of Electronic
                                        Records Archives




                                                                                                  Background
                                                                                     ERA Program Management

ERA Program Management
The ERA program management office is responsible for the development of ERA policies,
plans, guidance, and procedures.
    • NARA hired a contractor, Integrated Computer Engineering (ICE), Inc.,7 to assist in
      developing the capability to design, acquire, and manage the ERA system.
    • ICE is responsible for developing many ERA acquisition documents and for validating
      and verifying that these documents conform to industry standards for content and
      structure.
The following slide shows the organizational chart for the program management office,
showing government staff positions.8




7
 On January 15, 2002, American Systems Corporation (ASC) announced its acquisition of ICE, Inc. According to the
ERA project manager, this change does not affect the status of NARA’s contract with ICE, Inc.
8
 According to NARA, it is considering organizational changes that are under review based on findings of its most recent
assessment of the ERA program and other lessons learned.


                                                                                                                          17




                                        Page 22                                         GAO-03-880 Electronic Archive Acquisition
                         Appendix I
                         National Archives and Records
                         Administration’s Acquisition of Electronic
                         Records Archives




                                                                                   Background
                                                                      ERA Program Management

ERA Program Organization Chart




                                                                                                          18




                         Page 23                                         GAO-03-880 Electronic Archive Acquisition
                               Appendix I
                               National Archives and Records
                               Administration’s Acquisition of Electronic
                               Records Archives




                                                     Objective 1: IT Organizational Capabilities


In our earlier report, we recommended that the Archivist strengthen key IT organizational
capabilities by
  (1) implementing an IT investment management process,
  (2) developing an enterprise architecture, and
  (3) improving information security.
NARA has made some progress in strengthening its capabilities in these areas, but they are
not yet completely established.




                                                                                                             19




                               Page 24                                      GAO-03-880 Electronic Archive Acquisition
                                      Appendix I
                                      National Archives and Records
                                      Administration’s Acquisition of Electronic
                                      Records Archives




                                                            Objective 1: IT Organizational Capabilities
                                                                          IT Investment Management

IT Investment Management
An IT investment management process is an integrated approach to managing IT
investments that provides for the continuous identification, selection, control, life cycle
management, and evaluation of IT investments. The Clinger-Cohen Act and OMB guidance
emphasize the need to have investment management processes and information to help
ensure that IT projects are being implemented at acceptable costs and within reasonable
and expected time frames and that they are contributing to tangible, observable
improvements in mission performance (i.e., that projects are meeting the cost, schedule, and
performance commitments upon which their approval was justified).
GAO’s Information Technology Investment Management (ITIM) maturity framework9 defines
critical processes pertaining to IT investment management and oversight. Among other
things these processes provide for establishing investment decision-making bodies
responsible for selecting and controlling IT investments by (1) understanding, for example,
each project’s expected return on investment and associated costs, schedule, and
performance commitments; (2) regularly determining each project’s progress toward these
expectations and commitments; and (3) taking corrective actions to address deviations.


9
 U.S. General Accounting Office, Information Technology Investment Management: A Framework for Assessing and
Improving Process Maturity (Exposure Draft), GAO/AIMD-10.1.23 (Washington, D.C.: May 2000).

                                                                                                                    20




                                      Page 25                                      GAO-03-880 Electronic Archive Acquisition
                              Appendix I
                              National Archives and Records
                              Administration’s Acquisition of Electronic
                              Records Archives




                                                    Objective 1: IT Organizational Capabilities
                                                                  IT Investment Management

NARA has taken steps to strengthen its IT investment management processes. Among the
actions it has completed are the following:
  • NARA has established an IT investment review board, which receives monthly briefings
    on strategic projects covering schedule and risks.
  • NARA has updated its directive addressing IT investment management.
  • Initial versions of policy documentation have been completed for two of the three phases
    of IT investment management: control and evaluation.
  • NARA has completed interim guidance for analyzing the life cycle benefits, costs, and
    risks of IT.




                                                                                                            21




                              Page 26                                      GAO-03-880 Electronic Archive Acquisition
                                Appendix I
                                National Archives and Records
                                Administration’s Acquisition of Electronic
                                Records Archives




                                                      Objective 1: IT Organizational Capabilities
                                                                       Enterprise Architecture

Enterprise Architecture
An enterprise architecture provides a description—in useful models, diagrams, and
narrative—of the mode of operation for an agency. It describes the agency in both
    (1) logical terms, such as interrelated business locations and users; and
    (2) technical terms, such as hardware, software, data, communications, and security
        attributes and standards.
An enterprise architecture provides these perspectives both for the “as is” environment
(baseline) and for the “to be” environment (target). It also consists of technical reference
model and technical standards used by the agency, as well as a plan to transition from the
baseline to the target environment.
NARA has ongoing efforts to complete an enterprise architecture, which are expected to
continue over the next 2 years. It developed initial versions of its enterprise architecture,
including a baseline and target architecture, a technical reference model, and standards
profile. According to NARA’s Chief Technology Officer, while this initial effort provided a
basis for an architecture, it was not complete and NARA is continuing to develop a more
comprehensive version.




                                                                                                              22




                                Page 27                                      GAO-03-880 Electronic Archive Acquisition
                               Appendix I
                               National Archives and Records
                               Administration’s Acquisition of Electronic
                               Records Archives




                                                     Objective 1: IT Organizational Capabilities
                                                                      Enterprise Architecture

Beginning in July 2003, NARA plans to address a number of activities associated with its
enterprise architecture, including
  • procuring and installing software tools to manage the development of its enterprise
    architecture,
  • continuing to refine data management processes and definitions, and
  • establishing an information security architecture.
NARA has not completed the definition of its “to be” architecture, which will provide a
framework for defining the ERA system’s requirements. The ERA program currently plans to
complete the definition of requirements by the end of June 2003.




                                                                                                             23




                               Page 28                                      GAO-03-880 Electronic Archive Acquisition
                               Appendix I
                               National Archives and Records
                               Administration’s Acquisition of Electronic
                               Records Archives




                                                     Objective 1: IT Organizational Capabilities
                                                                         Information Security

Information Security
Federal legislation and guidance for information security require organizations, among other
things, to establish an information security program, including the following activities:
  • Develop information security policy and procedures.
  • Develop system security plans for networks, facilities, and systems or groups of
    information systems.
  • Perform risk assessments.
  • Determine the sensitivity and criticality of systems.
  • Establish certification and accreditation programs for information systems.
As a result of security assessments conducted by NARA and its Inspector General, the
Archivist declared NARA’s information security a material weakness in fiscal year 2000; it
remains a material weakness.
In response to the recommendations in our earlier report, NARA has taken steps to establish
an information security program. In fiscal year 2002, NARA
  • completed an information security policy and security control handbooks,
  • addressed several information security vulnerabilities, and
  • conducted a limited vulnerability assessment and risk analysis of NARA’s network.

                                                                                                             24




                               Page 29                                      GAO-03-880 Electronic Archive Acquisition
                               Appendix I
                               National Archives and Records
                               Administration’s Acquisition of Electronic
                               Records Archives




                                                     Objective 1: IT Organizational Capabilities
                                                                         Information Security

However, NARA’s IT security program is incomplete in several areas, including the following:
  • Security plans for all NARA systems have not been developed.
  • NARA has not implemented a security risk assessment program.
  • It has not determined the sensitivity and criticality of systems.
NARA indicated that in fiscal year 2003, it plans to
  • complete certification and accreditation of 41 IT systems, and
  • address outstanding security risks and remove security as a material weakness.
Nevertheless, without a fully established information security program, the risk to the planned
ERA system and its archived data is increased.




                                                                                                             25




                               Page 30                                      GAO-03-880 Electronic Archive Acquisition
                               Appendix I
                               National Archives and Records
                               Administration’s Acquisition of Electronic
                               Records Archives




                                                                   Objective 2: Systems Acquisition
                                                                    Policies, Plans, and Practices

To guide its acquisition of the ERA system, the program has adopted an IEEE standard for
software life cycle processes (IEEE 12207 and associated standards). These standards were
expected to be used in developing the life cycle plan and processes that the program is
following to acquire the ERA system:
  • developing the ERA Life Cycle document,
  • defining a concept of operations,
  • developing requirements,
  • creating an acquisition strategy,
  • performing risk management,
  • performing configuration management, and
  • conducting quality assurance.




                                                                                                             26




                               Page 31                                      GAO-03-880 Electronic Archive Acquisition
                               Appendix I
                               National Archives and Records
                               Administration’s Acquisition of Electronic
                               Records Archives




                                                                   Objective 2: Systems Acquisition
                                                                                    ERA Life Cycle

ERA Life Cycle
The ERA Life Cycle document describes the life cycle of the ERA system from
conceptualization of ideas through retirement, as well as processes for acquiring and
supplying software products and services for the ERA system.
However, the ERA Life Cycle document does not meet all applicable IEEE standards:
  • It states that it was tailored to the IEEE 12207 standards but fails to identify which
    elements were excluded or modified.
  • It does not document key activities, such as the processes, activities, and tasks needed
    for contract preparation, including all tasks associated with development of the Request
    for Proposal.
  • It does not show the proposed approach, including the design competition and the
    incremental approach.
Without a complete, well-documented life cycle that meets applicable standards, NARA and
the ERA program are impeded in their ability to manage and track key acquisition processes
and activities.




                                                                                                             27




                               Page 32                                      GAO-03-880 Electronic Archive Acquisition
                              Appendix I
                              National Archives and Records
                              Administration’s Acquisition of Electronic
                              Records Archives




                                                                  Objective 2: Systems Acquisition
                                                                          Concept of Operations

Concept of Operations
According to IEEE standards, the Concept of Operations is a user-oriented document that
describes the characteristics of a proposed system from the users' viewpoint. The document
becomes the framework for all subsequent activities leading to system deployment. The
IEEE standard describes key elements that should be included in this document, including
major system components, interfaces to external systems, and performance characteristics
such as speed, throughput, and volume.
The ERA Concept of Operations describes the high-level operational characteristics of the
ERA system from the viewpoint of the users, developer, and other stakeholders. The
Concept provides justification for the proposed system, identifies classes of ERA users, and
describes operational scenarios. It does not include several key elements required by the
IEEE standard, including
  • a complete description of the proposed system,
  • the system’s modes of operation under different circumstances (for example, regular,
    degraded, emergency, backup, etc).
The ERA program is planning to update its Concept of Operations to address these elements
by August 18, 2003.




                                                                                                            28




                              Page 33                                      GAO-03-880 Electronic Archive Acquisition
                               Appendix I
                               National Archives and Records
                               Administration’s Acquisition of Electronic
                               Records Archives




                                                                   Objective 2: Systems Acquisition
                                                                                ERA Requirements

ERA Requirements
Requirements provide the blueprint that system developers and program managers use to
design, develop, and acquire a system. It is critical that requirements be carefully defined
and that they flow directly from the organization’s concept of operations (how the
organization’s day-to-day operations are or will be carried out to meet mission needs).
ERA requirements are currently being defined; NARA expects them to be completed by June
30, 2003.
Accordingly, we did not review them for their conformance to industry standards.




                                                                                                             29




                               Page 34                                      GAO-03-880 Electronic Archive Acquisition
                               Appendix I
                               National Archives and Records
                               Administration’s Acquisition of Electronic
                               Records Archives




                                                                   Objective 2: Systems Acquisition
                                                                              Acquisition Strategy

Acquisition Strategy
An acquisition strategy is a description of how an organization plans to acquire a system. It
establishes the framework by which detailed acquisition planning and program execution will
be accomplished and communicated to key stakeholders. For ERA, the acquisition strategy
should conform to IEEE standards and must conform to the Federal Acquisition Regulation
(FAR) for acquisition planning.
The ERA Acquisition Strategy does not fully conform to applicable standards, according to
an assessment by the ERA program:
  • It does not satisfy all FAR requirements: 28 of 39 content requirements are not satisfied,
    including the identification of acquisition risks, the description of the ERA cost model and
    life cycle costs, and the establishment of detailed milestones for the acquisition process.
  • It does not meet all IEEE standards: 15 of 32 content requirements are not satisfied,
    including the establishment and implementation of an acquisition planning strategy,
    standards for the preparation of contract requirements, and criteria for the selection of a
    qualified supplier.
Unless it adheres to these standards and requirements, NARA increases the risk that its
strategy will not be clearly understood by key stakeholders.



                                                                                                             30




                               Page 35                                      GAO-03-880 Electronic Archive Acquisition
                               Appendix I
                               National Archives and Records
                               Administration’s Acquisition of Electronic
                               Records Archives




                                                                   Objective 2: Systems Acquisition
                                                                                 Risk Management

Risk Management
In acquisition, risk management is a process for identifying potential problems before they
occur and adjusting the acquisition to mitigate problems and to decrease the chance of their
occurring. Risk management includes developing a project risk management plan; identifying
and prioritizing potential problems; implementing risk mitigation strategies, as required; and
tracking and reporting progress against the plans.
NARA has developed an ERA Risk Management Plan. However, the ERA program has not
fully established a risk management process to comply with IEEE requirements:
  • In the ERA Risk Management Plan, the risk planning policy is incomplete. For example,
    the plan does not discuss how resources are to be made available to treat risks or how
    risks are to be communicated to and reviewed by stakeholders.
  • The plan is written at a high level of generality and has not been defined specifically for
    the ERA program. For example, procedures for risk identification and risk analysis are
    not specific to the ERA program.
In the absence of a fully established risk management process, NARA may miss
opportunities to avoid or mitigate predictable risks in its ERA acquisition.




                                                                                                             31




                               Page 36                                      GAO-03-880 Electronic Archive Acquisition
                                Appendix I
                                National Archives and Records
                                Administration’s Acquisition of Electronic
                                Records Archives




                                                                    Objective 2: Systems Acquisition
                                                                       Configuration Management

Configuration Management Plan
The ERA Configuration Management Plan, based on the IEEE standard, is intended to
establish and maintain the integrity and control of the products of the ERA program through
its life cycle. The IEEE standard for software configuration plans describes classes of
information that should be included in the plan. These include identifying
  • who is responsible for accomplishing planned activities,
  • what configuration management project activities are to be performed,
  • when, in relation to other project activities, the activities are to be scheduled, and
  • what tools and human resources are required for executing the plan.
A verification and validation assessment of the Configuration Management Plan by ERA
contractors found that while the plan did not satisfy all of the above IEEE requirements, the
amount of information included in the plan was appropriate for the point in the acquisition
process at which it was written.




                                                                                                              32




                                Page 37                                      GAO-03-880 Electronic Archive Acquisition
                               Appendix I
                               National Archives and Records
                               Administration’s Acquisition of Electronic
                               Records Archives




                                                                   Objective 2: Systems Acquisition
                                                                                Quality Assurance

Quality Assurance
In IT acquisition, quality assurance describes processes for providing independent
assessment of the requirements and processes for developing and producing a system or
software. Quality assurance includes developing a quality assurance plan, determining
applicable processes and product standards to be followed, and conducting reviews to
ensure that the product and process standards are followed. Quality assessments should
take place throughout the life cycle of an acquisition.
The ERA program has developed the ERA Quality Assurance Plan, which references IEEE
standards for software quality assurance. However, elements required by the standard are
absent. For example, the plan
  • does not include ERA documentation to be reviewed,
  • lacks criteria by which to assess documentation,
  • does not specify quality assurance tasks to be performed, and
  • lacks estimates of resources to be expended.
Without a fully established quality assurance plan, the ERA program is at risk of generating
products that do not meet requirements.



                                                                                                             33




                               Page 38                                      GAO-03-880 Electronic Archive Acquisition
                                      Appendix I
                                      National Archives and Records
                                      Administration’s Acquisition of Electronic
                                      Records Archives




                                                                          Objective 2: Systems Acquisition
                                                                           Policies, Plans, and Practices

In addition to the life cycle processes described by IEEE standards, ERA is subject to federal
acquisition requirements.10 Among other things, these call for
     • an evaluation of alternatives to be considered and
     • the development of life cycle cost and benefit estimates of proposed systems.
Also associated with the ERA acquisition process are NARA’s efforts to staff the ERA
acquisition program.




10
  Office of Management and Budget Circular A-109, Major System Acquisitions; OMB Circular A-11, Preparing and
Submitting Budget Estimates.

                                                                                                                    34




                                      Page 39                                      GAO-03-880 Electronic Archive Acquisition
                               Appendix I
                               National Archives and Records
                               Administration’s Acquisition of Electronic
                               Records Archives




                                                                   Objective 2: Systems Acquisition
                                                                                     ERA Staffing

ERA Staffing
NARA has made progress in staffing the ERA program, but staffing may pose a risk to ERA.
NARA has determined that to carry out the acquisition tasks discussed earlier, it will need 46
staff: 24 government staff and 22 contractor staff. According to program officials, funding
shortfalls have prevented them from hiring the needed ERA staff. Currently, all 22 contractor
staff are onboard, but only 15 of the 24 projected ERA government staff have been hired.
Although we have not evaluated the appropriate staffing level, some staffing shortages are
significant. Three key government positions—the business manager and contract and
financial management specialists—are yet to be hired. In addition, staff for other key
positions were only recently hired: the chief system engineer and the risk management and
quality assurance specialists. All these positions are important to the quality and
completeness of key program processes.




                                                                                                             35




                               Page 40                                      GAO-03-880 Electronic Archive Acquisition
                                          Appendix I
                                          National Archives and Records
                                          Administration’s Acquisition of Electronic
                                          Records Archives




                                                                  Objective 3: Tracking Cost and Schedule


A project schedule with an appropriate work breakdown structure is a prerequisite to
program tracking, allowing managers to measure how well the program is achieving its cost
and schedule goals. The work breakdown structure should be detailed enough to expose all
risk factors and allow accurate estimates of resource requirements and schedule duration for
each work activity. Each major work activity should include standard elements such as
assigned personnel, resource budgets, estimated task duration, and dependencies among
work activities.
Work activities can be measured through performance-based techniques such as earned
value management, which allows managers to track the budget against the schedule.
“Earned value” may be used to determine if work is being done at a higher or lower
cost/performance rate than was planned.
OMB requires the use of such techniques in the monitoring of the cost, schedule, and
performance of system acquisition projects.14




14
     Office of Management and Budget Circular A-11.

                                                                                                                        36




                                          Page 41                                      GAO-03-880 Electronic Archive Acquisition
                               Appendix I
                               National Archives and Records
                               Administration’s Acquisition of Electronic
                               Records Archives




                                                       Objective 3: Tracking Cost and Schedule


In our 2002 report, we recommended that the ERA project schedule be revised to reflect
estimates of the amount of work and resources required to complete each task, and that
NARA schedule enough time to complete essential planning tasks.
While the ERA program has developed a more detailed project schedule that includes
standard elements, NARA has not yet fully carried out our recommendation, as these
elements are not yet complete:
  • The project schedule omits significant tasks and activities, including
      • the process to reengineer the agency’s life cycle business processes, upon which
        ERA requirements are being developed, and
      • the activity by ERA contractors to verify and validate ERA project management and
        acquisition documentation for conformance to industry standards.




                                                                                                             37




                               Page 42                                      GAO-03-880 Electronic Archive Acquisition
                              Appendix I
                              National Archives and Records
                              Administration’s Acquisition of Electronic
                              Records Archives




                                                      Objective 3: Tracking Cost and Schedule


NARA has indicated that it plans to use earned value management to assess performance
programwide, but this plan has not yet been implemented.
  • In the absence of a program schedule that includes all program tasks and a work
    breakdown structure with associated costs, the ERA program cannot use earned value
    management to track ERA cost and schedule.
The lack of earned value management impairs NARA’s ability to track and meet future
project milestones.
The ERA program is in an early phase of acquisition planning and has not yet been required
to meet many significant milestones. To date, NARA has met a major ERA milestone—the
definition of the ERA concept.
Meeting major milestones will become more important and more challenging in later phases
of the acquisition.




                                                                                                            38




                              Page 43                                      GAO-03-880 Electronic Archive Acquisition
                                Appendix I
                                National Archives and Records
                                Administration’s Acquisition of Electronic
                                Records Archives




                                                                                               Conclusions


In acquiring ERA, NARA faces significant challenges. ERA will be a major information
system; NARA has no previous experience in acquiring major information systems. Further,
no comparable electronic archive system is now in existence, in terms of either complexity or
scale. Finally, technology necessary to address some key requirements of ERA is not
commercially available and will have to be developed.
In light of these challenges, NARA will face significant difficulties in its ERA acquisition
unless it addresses
  • its IT organizational capabilities;
  • ERA system acquisition policies, plans, and practices; and
  • its ability to control ERA’s cost and schedule.
NARA has made progress implementing the key management areas of IT investment
management, enterprise architecture, and IT security, but NARA has more work to do to
implement our prior recommendations in this area.




                                                                                                              39




                                Page 44                                      GAO-03-880 Electronic Archive Acquisition
                               Appendix I
                               National Archives and Records
                               Administration’s Acquisition of Electronic
                               Records Archives




                                                                                              Conclusions


Without strong acquisition policies, plans, and practices, the risk is increased that the ERA
system will fail to meet user expectations, will cost more than currently estimated, and will be
delivered later than currently planned. Specifically, NARA and the ERA program have not
  • followed the IEEE 12207 standards and associated guidance in defining and managing
    the ERA acquisition process,
  • developed ERA acquisition processes and associated documentation in sufficient detail
    to allow program managers and NARA executives to manage and oversee the program,
    or
  • implemented the defined acquisition processes and practices according to these
    documented policies and plans.
Further, NARA is unable to effectively track and assess the status of the ERA program
because its program schedule does not include all program tasks and lacks a work
breakdown structure.




                                                                                                             40




                               Page 45                                      GAO-03-880 Electronic Archive Acquisition
                               Appendix I
                               National Archives and Records
                               Administration’s Acquisition of Electronic
                               Records Archives




                                                                                    Recommendations


To reduce the risks associated with NARA’s efforts to design and acquire the Electronic
Records Archives, we recommend that the U.S. Archivist direct the NARA Chief Information
Officer to do the following:
  • Address weaknesses in the acquisition policies, plans, and practices by
      • revising the ERA Life Cycle document and associated procedures and practices to
        conform to IEEE standards;
      • revising the ERA Concept of Operations to conform to IEEE standards, including a
        complete description of the current and proposed systems;
      • revising the ERA Acquisition Strategy to conform to IEEE standards and the Federal
        Acquisition Regulation;
      • revising the ERA Risk Management Plan to provide processes and procedures
        specific to the ERA program;
      • revising the ERA Quality Assurance Plan to conform to appropriate industry
        standards, establishing a vigorous, independent ERA quality assessment process,
        and providing the staffing resources necessary to ensure that quality assessment
        duties are performed effectively; and
      • filling key vacant ERA positions.

                                                                                                             41




                               Page 46                                      GAO-03-880 Electronic Archive Acquisition
                          Appendix I
                          National Archives and Records
                          Administration’s Acquisition of Electronic
                          Records Archives




                                                                               Recommendations


• Immediately address weaknesses in tracking cost and schedule by
   • developing an ERA schedule that is based on a comprehensive work breakdown
     structure (including associated costs and other resources) and establishes
     dependencies between successor and predecessor tasks; and
   • Using earned value management to capture and monitor progress for the entire ERA
     program.




                                                                                                        42




                          Page 47                                      GAO-03-880 Electronic Archive Acquisition
                               Appendix I
                               National Archives and Records
                               Administration’s Acquisition of Electronic
                               Records Archives




                                                                                    Agency Comments


NARA provided us with oral comments on a draft of this briefing during a meeting held on
April 24, 2003, attended by the Deputy Archivist, the ERA program manager, and other
agency officials. These officials generally agreed with our findings, conclusions, and
recommendations. They indicated that NARA is already addressing the weaknesses in
ERA’s acquisition policies, plans, and practices identified through a verification and validation
activity. NARA officials also provided clarifications and additional technical comments, which
we have incorporated into the briefing as appropriate.




                                                                                                             43




                               Page 48                                      GAO-03-880 Electronic Archive Acquisition
Appendix II

Comments from the National Archives and
Records Administration                                               Appendx
                                                                           Ii




              Page 49        GAO-03-880 Electronic Archive Acquisition
           Appendix II
           Comments from the National Archives and
           Records Administration




(310371)   Page 50                                   GAO-03-880 Electronic Archive Acquisition
GAO’s Mission            The General Accounting Office, the audit, evaluation and investigative arm of
                         Congress, exists to support Congress in meeting its constitutional responsibilities
                         and to help improve the performance and accountability of the federal government
                         for the American people. GAO examines the use of public funds; evaluates federal
                         programs and policies; and provides analyses, recommendations, and other
                         assistance to help Congress make informed oversight, policy, and funding
                         decisions. GAO’s commitment to good government is reflected in its core values of
                         accountability, integrity, and reliability.


Obtaining Copies of      The fastest and easiest way to obtain copies of GAO documents at no cost is
                         through the Internet. GAO’s Web site (www.gao.gov) contains abstracts and full-
GAO Reports and          text files of current reports and testimony and an expanding archive of older
                         products. The Web site features a search engine to help you locate documents
Testimony                using key words and phrases. You can print these documents in their entirety,
                         including charts and other graphics.
                         Each day, GAO issues a list of newly released reports, testimony, and
                         correspondence. GAO posts this list, known as “Today’s Reports,” on its Web site
                         daily. The list contains links to the full-text document files. To have GAO e-mail this
                         list to you every afternoon, go to www.gao.gov and select “Subscribe to
                         e-mail alerts” under the “Order GAO Products” heading.


Order by Mail or Phone   The first copy of each printed report is free. Additional copies are $2 each. A check
                         or money order should be made out to the Superintendent of Documents. GAO
                         also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a single
                         address are discounted 25 percent. Orders should be sent to:
                         U.S. General Accounting Office
                         441 G Street NW, Room LM
                         Washington, D.C. 20548
                         To order by Phone:     Voice: (202) 512-6000
                                                TDD: (202) 512-2537
                                                Fax: (202) 512-6061


To Report Fraud,         Contact:
                         Web site: www.gao.gov/fraudnet/fraudnet.htm
Waste, and Abuse in      E-mail: fraudnet@gao.gov
Federal Programs         Automated answering system: (800) 424-5454 or (202) 512-7470



Public Affairs           Jeff Nelligan, Managing Director, NelliganJ@gao.gov (202) 512-4800
                         U.S. General Accounting Office, 441 G Street NW, Room 7149
                         Washington, D.C. 20548
United States                  Presorted Standard
General Accounting Office      Postage & Fees Paid
Washington, D.C. 20548-0001           GAO
                                 Permit No. GI00
Official Business
Penalty for Private Use $300
Address Service Requested