oversight

Information Technology: Architecture Needed to Guide NASA's Financial Management Modernization

Published by the Government Accountability Office on 2003-11-21.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                United States General Accounting Office

GAO             Report to Congressional Committees




November 2003
                INFORMATION
                TECHNOLOGY
                Architecture Needed
                to Guide NASA’s
                Financial Management
                Modernization




GAO-04-43
                a
                                                November 2003

                                                INFORMATION TECHNOLOGY

                                                Architecture Needed to Guide NASA’s
Highlights of GAO-04-43, a report to            Financial Management Modernization
congressional committees




The National Aeronautics and                    To date, NASA has acquired and implemented significant components of
Space Administration (NASA) is in               IFMP without an enterprise architecture to guide and constrain the program.
the process of modernizing its                  An enterprise architecture is an organizational blueprint that defines—in
financial management operations                 both business and technology terms—how an organization operates today
and supporting information                      and how it intends to operate in the future; it also provides a plan for
technology systems. This
modernization, known as the
                                                transitioning to this future state. Using an enterprise architecture to guide
Integrated Financial Management                 and constrain systems modernization programs is a federal requirement and
Program (IFMP), is intended to                  a recognized best practice of successful public and private organizations. In
provide NASA with an agencywide,                addition, GAO’s research has shown that attempting major modernization
integrated approach to performing               programs such as IFMP without a well-defined enterprise architecture risks,
critical business functions, such as            among other things, building systems that are duplicative, are not
contract management—an area                     interoperable, and do not effectively and efficiently support mission
that GAO first designated as high               operations and performance.
risk in 1990 and continues to do so
today. GAO was requested to                     During the course of GAO’s work, NASA recognized the need for an
review various aspects of IFMP,                 enterprise architecture and has taken steps to develop one. For example, it
and this report is one in a series on
the program. The objective of this
                                                has established an architecture program office, designated a chief architect,
review was to determine whether                 and selected an architecture framework to use. In addition, after GAO
NASA has been acquiring and                     completed its audit work, NASA released an initial version of an enterprise
implementing IFMP in the context                architecture, which the chief technology officer stated was not yet complete
of an enterprise architecture.                  and would be improved upon in future versions. However, the agency has
                                                yet to establish other key architecture management capabilities, such as
                                                designating an accountable corporate entity to lead the architecture effort,
                                                having an approved policy for developing and maintaining the architecture,
GAO is making recommendations
                                                and implementing an independent verification and validation function to
to the NASA Administrator for
establishing an effective enterprise            provide needed assurance that architecture products and architecture
architecture management                         management processes are effective. Moreover, the architecture products
capability, ensuring the                        used to date to manage NASA’s investment in IFMP did not provide
completeness of future releases of              sufficient context (depth and scope of agencywide operational and technical
NASA’s enterprise architecture,                 requirements) to effectively guide and constrain the program.
and minimizing its exposure to risk
on IFMP caused by system                        The chief technology officer agreed that NASA needs an effective enterprise
component acquisition and                       architecture program and stated that efforts are under way to establish one.
implementation efforts that have                GAO’s experience in reviewing other agencies has shown that not having an
proceeded to date in the absence of             effective enterprise architecture program can be attributed to, among other
an enterprise architecture. NASA
                                                things, an absence of senior management understanding and support, as well
concurred with GAO’s
recommendations and described                   as cultural resistance.
completed, ongoing, and planned
actions to address them.                        NASA’s current approach to acquiring and implementing IFMP outside the
                                                context of an architecture unnecessarily increases the risk that the
                                                program’s system components will not effectively and efficiently support
                                                agencywide operations. The result will be costly system rework. It is critical
www.gao.gov/cgi-bin/getrpt?GAO-04-43.
                                                for NASA to discontinue this approach and adopt the best practice of
To view the full product, including the scope   managing its IFMP system investments within the context of a well-defined
and methodology, click on the link above.       enterprise architecture.
For more information, contact Randolph C.
Hite at (202) 512-3439 or hiter@gao.gov.
Contents



Letter                                                                                                   1
                             Results in Brief                                                            3
                             Background                                                                  4
                             IFMP Has Proceeded without an Enterprise Architecture, and
                               NASA’s Ongoing Architecture Management Efforts Are Missing
                               Key Elements                                                             16
                             Conclusions                                                                30
                             Recommendations for Executive Action                                       30
                             Agency Comments and Our Evaluation                                         32


Appendixes
              Appendix I:    Objective, Scope and Methodology                                           34
             Appendix II:    Detailed Results of GAO’s Analyses of Architecture Products
                             Used to Date by NASA to Acquire and Implement IFMP                         36
             Appendix III:   Assessment of NASA’s EA Management Efforts against GAO’s
                             Architecture Management Maturity Framework                                 46
             Appendix IV:    Comments from the National Aeronautics and Space
                             Administration                                                             50
              Appendix V:    GAO Contacts and Staff Acknowledgments                                     58
                             GAO Contact                                                                58
                             Acknowledgments                                                            58


Tables                       Table 1: Overview of NASA’s Organizational Components                       5
                             Table 2: Overview of NASA’s Strategic Enterprises and the
                                      Contributing Centers                                               6
                             Table 3: Description and Status of NASA’s IFMP System
                                      Modules                                                           10
                             Table 4: Summary of IFMP Management Structure                              11
                             Table 5: Summary of the Maturity Stages and Core Elements of
                                      GAO’s Enterprise Architecture (EA) Management
                                      Framework                                                         28


Figure                       Figure 1: Summary of Extent to Which NASA’s Architecture
                                       Products Satisfy Key Elements Governing Architectural
                                       Content                                                          19




                             Page i                                GAO-04-43 NASA’s Enterprise Architecture
Contents




Abbreviations

ARC          Ames Research Center
CIO          chief information officer
CTO          chief technology officer
CRUD         create, read, update, and/or delete
DFRC         Dryden Flight Research Center
EAI          enterprise application integration
EA           enterprise architecture
FEAF         Federal Enterprise Architecture Framework
GRC          Glenn Research Center
GSFC         Goddard Space Flight Center
IFMP         Integrated Financial Management Program
IT           information technology
JPL          Jet Propulsion Laboratory
JSC          Johnson Space Center
KSC          Kennedy Space Center
LaRC         Langley Research Center
MSFC         Marshall Space Flight Center
NISSU        NASA’s Information System Services Utility
NASA         National Aeronautics and Space Administration
OIG          Office of the Inspector General
OMB          Office of Management and Budget
SSC          Stennis Space Center
TRM          technical reference model

 This is a work of the U.S. government and is not subject to copyright protection in the
 United States. It may be reproduced and distributed in its entirety without further
 permission from GAO. However, because this work may contain copyrighted images or
 other material, permission from the copyright holder may be necessary if you wish to
 reproduce this material separately.




Page ii                                         GAO-04-43 NASA’s Enterprise Architecture
A
United States General Accounting Office
Washington, D.C. 20548



                                    November 21, 2003                                                                             Leter




                                    The Honorable John McCain
                                    Chairman
                                    The Honorable Ernest F. Hollings
                                    Ranking Minority Member
                                    Committee on Commerce, Science
                                     and Transportation
                                    United States Senate

                                    The Honorable Sherwood L. Boehlert
                                    Chairman
                                    The Honorable Ralph M. Hall
                                    Ranking Minority Member
                                    Committee on Science
                                    House of Representatives

                                    To improve its ability to manage its contractors, which is an area that we
                                    designated as high risk in 1990, and continue to do so today,1 the National
                                    Aeronautics and Space Administration (NASA) began its third attempt at
                                    modernizing its financial management systems in April 2000. This
                                    modernization effort, known as the Integrated Financial Management
                                    Program (IFMP), is expected to produce an integrated, NASA-wide
                                    business systems environment by acquiring and incrementally
                                    implementing commercial hardware and software components. Our
                                    research of successful public- and private-sector organizations shows that
                                    attempting a modernization program, like IFMP, without having and using a
                                    well-defined modernization blueprint, commonly called an enterprise
                                    architecture,2 results in operations and systems that are duplicative, are not


                                    1
                                     In 1990, we began a special effort to review and report on the federal program areas that
                                    our work had identified as high risk because of vulnerabilities to waste, fraud, abuse, and
                                    mismanagement. We first issued our High-Risk Series in December 1992 and have since
                                    continued to include NASA’s contract management as an area of high risk. See U.S. General
                                    Accounting Office, High-Risk Series: NASA Contract Management, GAO/HR-93-11
                                    (Washington, D.C.: December 1992) and High-Risk Series: NASA Contract Management,
                                    GAO-03-119 (Washington, D.C.: January 2003).
                                    2
                                     An enterprise architecture is a blueprint that defines, both in logical terms (including
                                    integrated functions, applications, systems, users, work locations, and information needs
                                    and flows) and in technical terms (including hardware, software, data, communications, and
                                    security), how an organization’s information technology systems operate today, how they
                                    are to operate in the future, and a road map for the transition.




                                    Page 1                                           GAO-04-43 NASA’s Enterprise Architecture
well integrated, are unnecessarily costly to maintain and interface, and do
not effectively optimize mission performance.

In April 2003, we issued the first in a series of reports on the program, in
which we concluded that NASA’s approach to acquiring and implementing
IFMP components had and would continue to introduce risk and increase
the chances that the agency would fall short of meeting its program goal.3
Because of the importance of IFMP to overall mission performance, you
asked us to continue our review. Specifically, you requested that we
determine whether (1) NASA has been acquiring and implementing IFMP in
the context of an enterprise architecture, (2) the core financial module as
implemented in June 2003 would satisfy NASA’s key external reporting
requirements, and (3) NASA’s life-cycle cost estimate, program schedule,
and funding reserves for IFMP were reasonable.

We are responding to the second two issues in separate reports,4 as well as
summarizing our findings on all three areas in an additional report.5 This
report addresses the first issue—whether NASA had and was using an
enterprise architecture to acquire and implement IFMP. To accomplish this,
we compared the architecture documents that NASA provided us, and
represented as being used to manage IFMP, against published guidance
governing the content of a well-defined architecture.6 We also compared
NASA’s architecture development, maintenance, and implementation
practices against our enterprise architecture management maturity



3
 U.S. General Accounting Office, Business Modernization: Improvements Needed in
Management of NASA’s Integrated Financial Management Program, GAO-03-507
(Washington, D.C.: Apr. 30, 2003).
4
 U.S. General Accounting Office, Business Modernization: NASA’s Integrated Financial
Management Program Does Not Fully Address Agency’s External Reporting Issues, GAO-
04-151 (Washington, D.C.: Nov. 21, 2003) and Business Modernization: Disciplined
Processes Needed to Better Manage NASA’s Integrated Financial Management Program,
GAO-04-118 (Washington, D.C.: Nov. 21, 2003).
5
 U.S. General Accounting Office, Business Modernization: NASA Challenges in Managing
Its Integrated Financial Management Program, GAO-04-255 (Washington, D.C.: Nov. 21,
2003).
6
 See, for example, Office of Management and Budget, Federal Enterprise Architecture
Business Reference Model, Version 1.0 (2002); Chief Information Officer Council, A
Practical Guide to Federal Enterprise Architecture, Version 1.0 (February 2001); and Office
of Management and Budget, Management of Federal Information Resources, Circular No.
A-130 (Nov. 28, 2000).




Page 2                                          GAO-04-43 NASA’s Enterprise Architecture
                   framework.7 Details on our objective, scope, and methodology are in
                   appendix I.



Results in Brief   To date, NASA has acquired and implemented significant components of
                   IFMP without an enterprise architecture, or modernization blueprint, to
                   guide and constrain the program.8 During the course of our review of this
                   program, the agency recognized the need for an enterprise architecture and
                   began efforts to develop one. For example, NASA established some
                   important architecture management structures and process controls
                   advocated by best practices and federal guidance, such as having an
                   enterprise architecture program office; designating a chief architect; and
                   using an architecture development methodology, framework, and
                   automated tool. In addition, after we completed our audit work, the agency
                   released an initial version of an enterprise architecture, which the chief
                   technology officer stated was not yet complete and would be improved
                   upon in future versions. However, NASA has yet to establish other key
                   architecture management capabilities that are essential to having a mature,
                   effective enterprise architecture program. Moreover, the architecture
                   products that the agency has used to date in managing its $983 million
                   IFMP investment9 did not provide sufficient context (depth and scope of
                   agencywide operational and technical requirements) to effectively guide
                   and constrain the program. NASA’s chief technology officer agreed that
                   NASA needs an effective enterprise architecture program and stated that
                   efforts are under way to establish one.

                   Our experience in reviewing other agencies has shown that not having an
                   effective enterprise architecture program can be attributed to, among other
                   things, an absence of senior management understanding and support, and
                   cultural resistance to having and using one. Our experience also shows that
                   attempting major modernization programs such as IFMP without having
                   and using an enterprise architecture often results in system
                   implementations that are duplicative, are not well integrated, and require


                   7
                    U.S. General Accounting Office, Information Technology: A Framework for Assessing and
                   Improving Enterprise Architecture Management, Version 1.1, GAO-03-584G (Washington,
                   D.C.: April 2003).
                   8
                    NASA has acquired and implemented five of the nine planned major components of IFMP
                   and is in the process of implementing the sixth component.
                   9
                   GAO-04-118.




                   Page 3                                        GAO-04-43 NASA’s Enterprise Architecture
             costly rework to interface. In the case of IFMP, this is occurring.
             Specifically, NASA’s Office of the Inspector General (OIG) recently
             reported10 that the agency would need to resolve several accounting and
             costing issues before the IFMP core system component provides full cost
             accounting capabilities. This means that the agency will now have to
             reconfigure the software for this component to reflect the issues’
             resolution. According to the chief technology officer, determining the need
             for additional rework of already implemented IFMP system components
             will be based on a future assessment of these components’ alignment to an
             initial version of the agency’s enterprise architecture that NASA first
             provided to us on September 24, 2003, which was after we had completed
             our audit work.

             To assist NASA in its efforts to effectively and efficiently acquire and
             implement IFMP, as well as its recently launched efforts to develop and use
             a well-defined enterprise architecture, we are making recommendations to
             the Administrator related to establishing an effective enterprise
             architecture management capability, ensuring the completeness of planned
             future releases of its enterprise architecture, and minimizing its exposure
             to risk on IFMP caused by system component acquisition and
             implementation efforts that have proceeded to date without an enterprise
             architecture. NASA concurred with our recommendations, and described
             actions recently completed, ongoing, or planned to implement them.



Background   NASA’s mission encompasses human exploration and development of
             space, the advancement and communication of scientific knowledge, and
             research and development of aeronautics and space technologies. Its
             activities span a broad range of complex and technical endeavors—from
             investigating the composition, evaluation, and resources of Mars; to
             working with the agency’s international partners to complete and operate
             the International Space Station; to providing satellite and aircraft
             observations of Earth for scientific and weather forecasting purposes; to
             developing new technologies designed to improve air safety. NASA’s
             workforce comprises over 19,000 civil service employees, primarily located
             at its headquarters and 10 major field centers, and more than 40,000
             contractors and grantees, who collectively perform a wide range of roles


             10
              National Aeronautics and Space Administration Office of Inspector General, Integrated
             Financial Management Program Core Financial Module Conversion to Full Cost
             Accounting, IG-03-015 (Washington, D.C.: May 30, 2003).




             Page 4                                         GAO-04-43 NASA’s Enterprise Architecture
                                               and responsibilities. Table 1 describes the roles and responsibilities of
                                               NASA’s headquarters and field centers.



Table 1: Overview of NASA’s Organizational Components

NASA headquarters and
field centers               Location                    Roles/responsibilities
NASA Headquarters           Washington, D.C.            NASA headquarters manages the space flight centers, research centers, and
                                                        other installations. It is responsible for determining programs and projects;
                                                        establishing management policies, procedures, and performance criteria;
                                                        evaluating progress; and reviewing and analyzing all phases of the aerospace
                                                        program.
Ames Research Center        Moffett Field, Calif.       ARC is a principal center for computational fluid dynamics, rotorcraft and
(ARC)                                                   powered-lift technology, artificial intelligence, and airborne sciences.
Dryden Flight Research      Edwards Air Force           DFRC is the premier installation for aeronautical flight research.
Center (DFRC)               Base, Calif.
Glenn Research Center       Cleveland, Ohio             GRC, the lead center for aeropropulsion, is responsible for developing, verifying,
(GRC)                                                   and transferring aeropropulsion technologies to U.S. industry.
Goddard Space Flight        Greenbelt, Md.              GSFC is a major U.S. laboratory for developing and operating unmanned
Center (GSFC)                                           scientific spacecraft.
Jet Propulsion Laboratory   Pasadena, Calif.            JPL is the lead U.S. center for robotic exploration of the solar system, with a
(JPL)                                                   primary focus on planetary exploration (e.g., missions to Mars) and
                                                        environmental research (e.g., Shuttle Imaging Radar). The California Institute of
                                                        Technology manages JPL for NASA.
Johnson Space Center        Houston, Tex.               JSC is the primary center for designing, developing, and testing spacecraft and
(JSC)                                                   associated systems for human flight, selecting and training astronauts, and
                                                        planning and conducting human space flight missions.
Kennedy Space Center        Cape Canaveral, Fla.        KSC is primarily responsible for ground turnaround and support operations,
(KSC)                                                   prelaunch checkout, and launching of the space shuttle and its payloads,
                                                        including NASA's International Space Station. KSC is the nation's spaceport—the
                                                        liftoff site for all manned missions into space.
Langley Research Center     Hampton, Va.                LaRC is primarily responsible for basic research in aeronautics and space
(LaRC)                                                  technology. It is the lead center for managing NASA’s technology development
                                                        programs for future high-speed civil transport, hypersonic vehicle concepts, and
                                                        general aviation.
Marshall Space Flight       Huntsville, Ala.            MSFC is the premier organization for developing space transportation and
Center (MSFC)                                           propulsion systems and for conducting microgravity research.
Stennis Space Center        Hancock County, Miss.       SSC is the primary center for testing large rocket propulsion systems for the
(SSC)                                                   space shuttle and future generation space vehicles.
Source: NASA.


                                               Transcending NASA’s organizational components are six strategic mission
                                               enterprises or business areas, each with a unique set of strategic goals,
                                               objectives, and implementation strategies focused on the requirements of
                                               the agency’s customers. Each enterprise draws on the capabilities of



                                               Page 5                                            GAO-04-43 NASA’s Enterprise Architecture
                                              several centers so that each center contributes to multiple enterprises.
                                              Table 2 summarizes NASA’s strategic enterprises and the contributing
                                              centers.



Table 2: Overview of NASA’s Strategic Enterprises and the Contributing Centers

Strategic enterprise               Primary goal                                                          Contributing NASA centers
Aerospace technology               Pioneer and develop advanced technologies that, in turn, improve      ARC, DFRC, GRC, and LaRC
                                   the air transportation system, access to space, and science
                                   missions. Includes helping others use NASA technology for
                                   nonaerospace commercial purposes and developing technology
                                   partnerships with those in industry and academia that are outside
                                   of traditional aerospace fields.
Biological and physical research   Offer a unique laboratory in which to study biological and physical   ARC, GRC, JPL, JSC, KSC, and
                                   processes. Experiments that take advantage of this environment        MSFC
                                   extend from basic biology to quantum mechanics and from
                                   fundamental research to research with near-term applications in
                                   medicine and industry.
Earth science                      Seek to understand and protect our planet by advancing Earth-         ARC, DFRC, GSFC, JPL, LaRC,
                                   system science with a near-term emphasis on global climate            MSFC, and SSC
                                   change through the use of Earth remote-sensing spacecraft,
                                   airborne observations, space shuttle missions, and ground-based
                                   measurements.
Education                          Inspire students to pursue the study of science and engineering,      ARC, DFRC, GRC, GSFC, JPL,
                                   with the ultimate goal of having them choose careers in               JSC, KSC, LaRC, MSFC, and
                                   aeronautics and space at NASA.                                        SSC

Space flight                       Provide critical enabling capabilities that make possible the      ARC, GRC, JPL, JSC, KSC, and
                                   science, research, and exploration achievements of the rest of the MSFC
                                   agency.
Space science                      Seek to answer fundamental questions about life in the universe:      ARC, GSFC, JPL, KSC, and
                                   how it arose, what its mechanisms are, where in the solar system      MSFC
                                   life may have originated or may exist today, and whether there are
                                   similar planetary environments around other stars where the
                                   signature of life can be found.
Source: NASA.


                                              To execute its mission responsibilities, NASA performs numerous
                                              management functions, such as contract management, financial
                                              management, and human capital management, relying heavily on
                                              information technology (IT) to assist it in performing these functions. For
                                              fiscal year 2003, the agency estimated that it would spend approximately
                                              $2.3 billion on IT systems and services. Of this amount, NASA anticipated
                                              spending $32.5 million on IT security and $11 million on enterprise
                                              architecture.



                                              Page 6                                            GAO-04-43 NASA’s Enterprise Architecture
NASA Continues to Face   In January 2003, we reported11 that NASA faced challenges that threaten its
Challenges in Managing   ability to effectively run its largest programs. We also reported that because
                         these challenges are rooted in NASA’s culture and long-standing ways of
Large Programs           doing business, the agency needed to make a major transformation. In
                         particular, we identified the following four performance and accountability
                         challenges facing the agency:

                         • Strengthening strategic human capital management. NASA is
                           facing shortages in its workforce, which could likely worsen as the
                           workforce continues to age and the pipeline of talent shrinks. This
                           dilemma is more pronounced among areas crucial to NASA’s ability to
                           perform its mission, such as engineering, science, and IT. NASA is
                           addressing this challenge through strategic planning, through a new
                           workforce planning and analysis system, and by requesting additional
                           personnel flexibilities, among other initiatives.

                         • Controlling International Space Station costs. Development costs
                           for this project have soared to the point where NASA has had to cut
                           back the program substantially, including reducing construction, the
                           number of crew members, and scientific research. These cutbacks have
                           raised concern among NASA’s international partners, who have a large
                           stake in the scientific research to be performed on the station. Although
                           NASA is instituting management and cost-estimating reforms, it still
                           needs to reach agreement with its partners on its planned cutbacks.

                         • Reducing costs of space launches. The administration submitted an
                           amendment to NASA’s fiscal year 2003 budget request, which (1)
                           extends the life of the space shuttle and enhances its reliability, (2)
                           funds the development of a new vehicle for ferrying crew to and from
                           the space station, and (3) alters the time frame for a shuttle
                           replacement. Accomplishing these and other goals related to space
                           launches will be difficult and risky in light of the technology advances
                           that NASA would like to pursue and the high degree of communication
                           and coordination required among industry and government partners.




                         11
                          U.S. General Accounting Office, Major Management Challenges and Program Risks:
                         National Aeronautics and Space Administration, GAO-03-114 (Washington, D.C.: January
                         2003).




                         Page 7                                       GAO-04-43 NASA’s Enterprise Architecture
• Improving contract management. NASA spends most of its funds on
  acquisitions.12 Yet, for many years, the agency has been unable to
  oversee contracts effectively, principally because it lacked accurate and
  reliable information on contract spending and placed little emphasis on
  end results, product performance, and cost control. NASA has
  addressed many acquisition-related weaknesses and is beginning to
  tackle one of its most formidable barriers to sound contract
  management—the lack of a modern, integrated financial management
  system. Considerable work remains to be done since NASA is only in the
  early stages of designing and implementing this new system, and NASA
  reported that it is already facing challenges in terms of cost,
  interoperability, and security.

We also reported that NASA’s ability to collect, maintain, and report the full
cost of its projects and programs is weakened by diverse and often
incompatible and nonintegrated center-level accounting systems; uneven
and nonstandard cost-reporting capabilities; decentralized policies,
procedures, and practices that are unique to its field centers; nonstandard
data formats; and online financial information that is not readily available
to program managers. Thus, it is difficult to ensure that contracts are being
efficiently and effectively implemented and that budgets are executed as
planned. This lack of integration and standardization also impedes the
agency’s ability to provide data required for external reporting purposes.

Recognizing the need for change, NASA’s Administrator articulated a new
vision for the agency—one that is science-driven, not destination-driven. To
better enable NASA to fulfill this vision, the agency is taking on a major
transformation aimed at eliminating stovepipes, becoming more integrated
and results-oriented, and reducing risks while working more economically,
efficiently, and effectively.




12
 NASA spends 90 percent or $12.7 billion of its annual budget for aeronautical and space-
related projects on the work performed by its contractors.




Page 8                                          GAO-04-43 NASA’s Enterprise Architecture
NASA Has Initiated a Large,   A key transformation effort is IFMP, which is NASA’s third attempt in more
Complex Systems               than 12 years to modernize its financial management processes and
                              systems. NASA spent about $180 million on its two prior failed efforts, and
Modernization to Address      NASA’s data indicate that the agency will spend approximately $983 million
Financial Management          through 2010 for its current effort, IFMP, which it began in April 2000.13
Concerns                      IFMP is expected to produce an integrated, NASA-wide financial
                              management system by acquiring and incrementally implementing
                              commercial software packages and related hardware and software
                              components. The main objective of IFMP is to improve the financial,
                              physical, and human capital management processes throughout the agency.
                              According to NASA, once fully implemented, IFMP will reengineer NASA’s
                              business operations around industry “best practices” and use enabling
                              technology to provide necessary management information to support
                              implementation of the agency’s strategic plan. To meet this objective and
                              support these crosscutting activities, NASA has identified the following
                              business drivers for the program:

                              • providing timely, consistent, and reliable information for management
                                decisions;

                              • improving NASA’s accountability and enabling full cost management;

                              • achieving increased efficiencies and operating effectively;

                              • exchanging information with customers and stakeholders in a timely
                                and reliable way; and

                              • attracting and retaining a world-class workforce.

                              The IFMP system is to consist of nine modules supporting a range of
                              functionality (see table 3).




                              13
                                   GAO-04-118.




                              Page 9                                  GAO-04-43 NASA’s Enterprise Architecture
Table 3: Description and Status of NASA’s IFMP System Modules

Module                   Description                                                                  Reported status
Core financial           Support full cost management by providing agencywide standards for           Operational as of June 2003.
                         accounting and budget execution processes and financial reporting.
                         Includes eight financial subprocesses: budget execution, purchasing,
                         cost management, accounts payable, accounts receivable, fixed assets,
                         standard general ledger, and federal reporting.
Travel management        Streamline and unify the agency’s employee travel system and improve         Operational as of June 2003.
                         traveler and vendor reimbursement. (According to NASA, this product
                         has been integrated into the core financial module.)
Executive financial      Provide budget, cost, and performance information for all major NASA         Operational as of July 2002.
management information   programs and projects in a standardized format.
system (Erasmus)
Resume management        Enable applicants to search for matching NASA job listings and generate Operational as of December
                         resumes online and allow NASA’s human resources community to            2001.
                         generate job listings while increasing efficiency and effectiveness.
Position description     Enable position descriptions to be rapidly generated and classified and      Operational as of early 2002.
management               associated documents to be automatically generated.
Budget formulation       Enable the formulation of project, program, institutional, enterprise, and   Currently being implemented; to
                         agency-level budget requirements. It will promote full cost management       be operational at all locations in
                         and real-time decision making.                                               February 2004.
Integrated asset         Enable financial reporting, physical inventory, maintenance, and liability Not yet initiated; no milestones
management               reporting for the functional areas of aircraft, environmental, facilities, and set.
                         logistics management.
Procurement              Support the procurement, receiving, invoicing, and payment of materials. Not yet initiated; no milestones
                         Will provide detailed and quantitative data to facilitate, economize, and set.
                         expedite procurement processes.
Human resources          Provide a human resources infrastructure that meets recordkeeping and        Not yet initiated; no milestones
                         process requirements while helping NASA managers fill positions with         set.
                         staff that possess the appropriate skill sets and career goals.
Source: NASA.


                                            As structured, NASA is the IFMP system integrator and thus is responsible
                                            for acquiring and integrating the multiple commercial components and
                                            ensuring that they collectively perform in a manner that meets the defined
                                            requirements. Table 4 describes the key IFMP program management
                                            positions/entities and their respective responsibilities.




                                            Page 10                                           GAO-04-43 NASA’s Enterprise Architecture
Table 4: Summary of IFMP Management Structure

Management position/entity    Responsibilities
Program Executive             Manages, on a corporate level, program rollout, budget, performance, and schedule requirements; has
                              decision authority over all program content, implementation schedules, and budget allocations; and
                              provides leadership and accountability for top-level program requirements, implementation success
                              criteria, overall performance definition, and strategic planning in the direction and operation of the
                              Integrated Financial Management Program (IFMP).
Program Director              Implements IFMP according to specific guidelines (e.g., the program plan); reports to the Program
                              Executive, and is under the oversight of the agency’s Chief Financial Officer and the IFMP Steering
                              Council.
Chief Financial Officer       Ensures that the program meets externally mandated requirements while satisfying internal customer
                              needs in a cost-effective manner.
Steering Council              Acts as a forum for reviewing and approving the agencywide crosscutting facets of the program,
                              including, for example, program strategy and budgets and expanding the scope of projects; resolves
                              functional conflicts and ensures functional integration.
Project Manager               Plans and manages the implementation of each functional module approved by the program office;
(each NASA center has a       coordinates process team activities and supports the selection of software products, including updating
Project Manager)              requirements on the basis of the selected software’s capabilities and the developed gap assessments,
                              which identify differences between NASA’s requirements and the software’s capabilities.
Integration Project Manager   Establishes a viable technical infrastructure and ensures that the various functional module
                              requirements are coordinated, ensures that each IFMP module is appropriately integrated/interfaced,
                              minimizes redundant data, ensures that data definitions are consistent across modules, establishes life-
                              cycle requirements, and performs configuration management.
Source: NASA.




Early Problems with IFMP                    In April 2003, we reported14 that NASA was not following key best practices
Have Been Reported                          for acquiring and implementing IFMP. For example, the agency had not
                                            established an analytical capability to understand and proactively manage
                                            the dependencies among IFMP commercial components. Further, in
                                            implementing the core financial module component, NASA had deferred
                                            addressing the needs of key systems users and had not properly developed
                                            detailed system requirements. We concluded that the agency was at risk of
                                            making a substantial investment in a system that would fall far short of its
                                            stated goal of providing meaningful and reliable information to support
                                            effective program management and congressional oversight.




                                            14
                                             U.S. General Accounting Office, Business Modernization: Improvements Needed in
                                            Management of NASA’s Integrated Financial Management Program, GAO-03-507
                                            (Washington, D.C.: Apr. 30, 2003).




                                            Page 11                                          GAO-04-43 NASA’s Enterprise Architecture
To address its problems, we recommended that NASA (1) develop and
implement a short-term plan to identify and mitigate the risks currently
associated with relying on already deployed IFMP commercial components
and to expeditiously stabilize these components’ operational capability and
performance; (2) as part of the short-term plan, develop and properly
document requirements, reengineer acquisition management processes,
and fully engage stakeholders—including program managers, cost
estimators, and the Congress—in the development of user requirements;
and (3) develop a longer-term strategy for acquiring additional IFMP
components that includes implementing a methodology for analyzing
commercial system component dependencies. NASA concurred with the
need for a short-term plan but disagreed with most of our findings and
recommendations related to user needs and requirements and testing.
NASA also agreed with the importance of having an approach for acquiring
additional IFMP components, but stated that it already has an effective
strategy in place.

In May 2003, NASA’s OIG reported15 that the core financial module
software, which had been deployed at six NASA centers, had the capability
to implement full cost accounting. However, before this implementation
could take place, NASA needed to resolve several complex accounting and
costing issues. These issues involved how to allocate service and general
and administrative costs, civil service costs, and unassigned costs. Once
these accounting and costing issues were resolved, the OIG reported that
NASA would have to configure the IFMP software to reflect the changes.
The OIG recommended that NASA revise the IFMP plans to include
(1) time frames and milestones for completing steps to implement full cost
accounting, including addressing and resolving the cost issues identified
above; (2) identification of the personnel and other resources necessary to
perform the steps within the established time frames; and (3) senior
management approval and support of these additional procedures. IFMP
officials concurred with the recommendations and plan to have all phases
of full cost accounting implemented by October 1, 2003. (NASA reported
that full implementation of the core financial module at all centers was
completed in June 2003.)




15
 National Aeronautics and Space Administration Office of Inspector General, Integrated
Financial Management Program Core Financial Module Conversion to Full Cost
Accounting, IG-03-015 (Washington, D.C.: May 30, 2003).




Page 12                                        GAO-04-43 NASA’s Enterprise Architecture
An Enterprise Architecture   Effective use of enterprise architectures, or modernization blueprints, is a
Is Critical to an            trademark of successful public and private organizations. For a decade, we
                             have promoted the use of architectures to guide and constrain systems
Organization’s Ability to    modernization, recognizing them as a crucial means to a challenging goal:
Effectively Modernize Its    agency operational structures that are optimally defined in both business
Business Operations and      and technological environments. The Congress, the Office of Management
Systems                      and Budget (OMB), and the federal Chief Information Officer (CIO)
                             Council have also recognized the importance of an architecture-centric
                             approach to modernization. The Clinger-Cohen Act of 1996 mandates that
                             an agency’s CIO develop, maintain, and facilitate the implementation of
                             architectures as a means for managing the integration of business
                             processes and supporting systems. Further, OMB has issued guidance that,
                             among other things, requires system investments to be consistent with
                             these architectures.

                             An enterprise architecture provides a clear and comprehensive picture of
                             an entity, whether it is an organization (e.g., federal department or agency)
                             or a functional or mission area that cuts across more than one organization
                             (e.g., financial management). This picture consists of snapshots of both the
                             enterprise’s current or “As Is” operational and technological environment
                             and its target or “To Be” environment, as well as a capital investment road
                             map for transitioning from the current to the target environment. These
                             snapshots further consist of “views,” which are basically one or more
                             architecture products that provide conceptual or logical representations of
                             the enterprise.

                             The suite of products and their content that form a given entity’s enterprise
                             architecture are largely governed by the framework used to develop the
                             architecture. Since the 1980’s, various frameworks have emerged and been
                             applied. For example, John Zachman developed a structure or “framework”
                             for defining and capturing an architecture.16 This framework provides for
                             six windows from which to view the enterprise, which Zachman terms
                             “perspectives” on how a given entity operates: those of (1) the strategic
                             planner, (2) the system user, (3) the system designer, (4) the system
                             developer, (5) the subcontractor, and (6) the system itself. Zachman also
                             proposed six abstractions or models associated with each of these
                             perspectives: these models cover (1) how the entity operates, (2) what the


                             16
                                J.A. Zachman, “A Framework for Information Systems Architecture,” IBM Systems
                             Journal 26, no. 3 (1987).




                             Page 13                                       GAO-04-43 NASA’s Enterprise Architecture
entity uses to operate, (3) where the entity operates, (4) who operates the
entity, (5) when entity operations occur, and (6) why the entity operates.

In September 1999, the federal CIO Council published the Federal
Enterprise Architecture Framework (FEAF), which is intended to provide
federal agencies with a common construct for their respective
architectures, thereby facilitating the coordination of common business
processes, technology insertion, information flows, and system
investments among federal agencies. FEAF describes an approach,
including models and definitions, for developing and documenting
architecture descriptions for multiorganizational functional segments of
the federal government. Similar to most frameworks, FEAF’s proposed
models describe an entity’s business, data necessary to conduct the
business, applications to manage the data, and technology to support the
applications.

More recently, OMB established the Federal Enterprise Architecture
Program Management Office to develop a federated enterprise architecture
according to a collection of five “reference models:”

• The Business Reference Model is intended to describe the business
  operations of the federal government independent of the agencies that
  perform them, including defining the services provided to state and local
  governments.

• The Performance Reference Model is to provide a common set of
  general performance outputs and measures for agencies to use to
  achieve business goals and objectives.

• The Data and Information Reference Model is to describe, at an
  aggregate level, the types of data and information that support program
  and business line operations, and the relationships among these types.

• The Service Component Reference Model is to identify and classify IT
  service (i.e., application) components that support federal agencies and
  promote the reuse of components across agencies.

• The Technical Reference Model is to describe how technology is
  supporting the delivery of service components, including relevant
  standards for implementing the technology.




Page 14                                 GAO-04-43 NASA’s Enterprise Architecture
These various enterprise architecture frameworks differ in their
nomenclatures and modeling approach. However, the frameworks
consistently provide for defining an enterprise’s operations in both
(1) logical terms, such as interrelated business processes and business
rules, information needs and flows, and work locations and users, and
(2) technical terms, such as hardware, software, data, communications,
and security attributes and performance standards. The frameworks also
provide for defining these perspectives for both the enterprise’s current or
“As Is” environment and its target or “To Be” environment, as well as a
transition plan for moving from the “As Is” to the “To Be” environment.

The importance of developing, implementing, and maintaining an
enterprise architecture is a basic tenet of both organizational
transformation and IT management. Managed properly, an enterprise
architecture can clarify and help optimize the interdependencies and
relationships among an organization’s business operations and the
underlying IT infrastructure and applications that support these
operations. Employed in concert with other important management
controls, such as portfolio-based capital planning and investment control
practices, architectures can greatly increase the chances that
organizations’ operational and IT environments will be configured to
optimize mission performance. Our experience with federal agencies has
shown that investing in IT without defining these investments in the
context of an architecture often results in systems that are duplicative, not
well integrated, and unnecessarily costly to maintain and interface.17




17
 See, for example, U.S. General Accounting Office, DOD Business Systems Modernization:
Improvements to Enterprise Architecture Development and Implementation Efforts
Needed, GAO-03-458 (Washington, D.C.: Feb. 28, 2003); Information Technology: DLA
Should Strengthen Business Systems Modernization Architecture and Investment
Activities, GAO-01-631 (Washington, D.C.: June 29, 2001); and Information Technology: INS
Needs to Better Manage the Development of Its Enterprise Architecture, AIMD-00-212
(Washington, D.C.: Aug. 1, 2000).




Page 15                                        GAO-04-43 NASA’s Enterprise Architecture
IFMP Has Proceeded          NASA has thus far acquired and deployed IFMP without a sufficiently
                            complete enterprise architecture to guide and constrain program
without an Enterprise       investment decisions. During the course of our review of IFMP, NASA took
Architecture, and           steps to correct this situation by establishing key architecture management
                            capabilities and undertaking the development of an initial version of an
NASA’s Ongoing              enterprise architecture that, according to the chief technology officer, will
Architecture                provide some missing contextual information (operational and technical).
Management Efforts          However, NASA has not established other key architecture management
                            capabilities, such as designating an accountable corporate entity to lead
Are Missing Key             the architecture effort, having an approved policy for developing and
Elements                    maintaining the architecture, and implementing an independent
                            verification and validation function to provide needed assurance that
                            architecture products and architecture management processes are
                            effective. The chief technology officer agreed that NASA needs an effective
                            enterprise architecture program and stated that efforts are under way to
                            establish one. Based on our experience in reviewing other agencies, not
                            having an effective enterprise architecture program is attributable to,
                            among other things, limited senior management understanding and
                            commitment and cultural resistance to having and using an architecture.
                            The result is an inability to implement modernized systems in a way that
                            minimizes overlap and duplication and maximizes integration and mission
                            support.



The Architecture Products   As previously discussed, the various frameworks used to develop
that NASA Has Used for      architecture products consistently provide for describing a given enterprise
                            in both logical (e.g., business, performance, application, information) and
IFMP Are Limited
                            technical (e.g., hardware, software, data) terms, and for doing so for the
                            enterprise’s current or “As Is” environment and its target or “To Be”
                            environment; these frameworks also provide for defining a capital
                            investment sequencing plan to transition from the “As Is” to the “To Be”
                            environment. However, the frameworks do not prescribe the degree to
                            which the component parts should be described to be considered correct,
                            complete, understandable, and usable—essential attributes of any
                            architecture. This is because the depth and detail of the descriptive content
                            depends on what the architecture is to be used for (i.e., its intended
                            purpose).

                            NASA’s stated intention is to use an architecture as the basis for
                            agencywide business transformation and systems modernization, including
                            IFMP. This purpose necessitates that NASA’s architecture products provide



                            Page 16                                  GAO-04-43 NASA’s Enterprise Architecture
considerable depth and detail, as well as logical and rational structuring
and internal linkages. More specifically, it means that these architecture
products should contain sufficient scope and detail so that, for example,
(1) duplicative business operations and systems are eliminated;
(2) business operations are standardized and integrated, and supporting
systems are interoperable; (3) use of enterprisewide services is maximized;
and (4) related shared solutions are aligned, like OMB’s e-government
initiatives.18 Moreover, this scope and detail should be accomplished in a
way that (1) provides flexibility in adapting to changes in the enterprise’s
internal and external environments; (2) facilitates its usefulness and
comprehension by varying perspectives, users, or stakeholders; and
(3) provides for properly sequencing investments to recognize, for
example, the investments’ respective dependencies and relative business
value.

The architecture artifacts that NASA’s chief technology officer provided to
us and represented as those used to date in acquiring and implementing
IFMP do not contain sufficient context (depth and scope of agencywide
operational and technical requirements) to effectively guide and constrain
agencywide business transformation and systems modernization efforts.
More specifically, these artifacts do not satisfy the most basic
characteristics of architecture content, such as clearly distinguishing
between artifacts that represent the “As Is” and the “To Be” environments.
The agency’s chief technology officer agreed that these existing
architecture products do not clearly distinguish between the two
environments. Therefore, for purposes of our analyses, the chief
technology officer told us to treat the architecture products as descriptive
of the “To Be” environment, and to assume that any “As Is” content in these
products represented capability intended for reuse in the future
environment. This characterization is consistent with NASA contractual
documents associated with developing these architecture products. On the
basis of this characterization, we did not assess these artifacts for their “As
Is” content and accepted the chief technology officer’s acknowledgment


18
 OMB has identified 24 e-government initiatives that are expected to support the goal of the
President’s management agenda and ultimately provide improved government services to
citizens, businesses, and other levels of government. Examples of these initiatives include:
(1) e-Grants, which will provide a single site intended to streamline the federal grants
management process and allow customers of federal grants to find and apply for grants;
(2) e-Payroll, which will simplify and integrate payroll systems across the federal
government; and (3) e-Travel, which will streamline the government’s travel administration
by creating a governmentwide Web-based travel management service.




Page 17                                          GAO-04-43 NASA’s Enterprise Architecture
that this content was missing. Instead, we focused on the “To Be” content
and the transition plan. To assess the “To Be” architecture products, we
divided them into five architectural components similar to those in OMB’s
architecture reference models: the business, information/data,
services/applications, technical, and performance components; we added
security as a sixth component because of its recognized importance and
relevance to the other five. We then compared architecture products NASA
used to date for IFMP to relevant criteria19 governing the content of key
architectural elements for the transition plan and these six components of
the “To Be” architecture. Based on this comparison, we determined
whether the architecture products generally satisfied, did not satisfy,20 or
partially satisfied21 each architectural element.

Overall, we found that NASA’s “To Be” architecture products did not satisfy
18 of 35 (51 percent) key elements and partially satisfied the remaining 17
(49 percent), and its transition plan partially satisfied 1 (25 percent) of 4
elements and did not satisfy the remaining 3 (75 percent) (see fig. 1).




19
 See, for example, Office of Management and Budget, Federal Enterprise Architecture
Business Reference Model, Version 1.0 (2002); Chief Information Officer Council, A
Practical Guide to Federal Enterprise Architecture, Version 1.0 (February 2001); Office of
Management and Budget, Management of Federal Information Resources, Circular No.
A-130 (Nov. 28, 2000); M.A. Cook, Building Enterprise Information Architectures:
Reengineering Information Systems (Prentice Hall Inc.: 1996); and National Institute of
Standards and Technology, Information Management Directions: The Integration
Challenge, Special Publication 500-167 (September 1989).
20
     The architecture does not satisfy any aspects of this key architectural element.
21
 The architecture partially satisfies some aspects of this key architectural element but does
not satisfy at least one significant aspect.




Page 18                                              GAO-04-43 NASA’s Enterprise Architecture
Figure 1: Summary of Extent to Which NASA’s Architecture Products Satisfy Key
Elements Governing Architectural Content




This means that the architecture products that NASA used to date in
acquiring and implementing IFMP have not provided an adequate context
in which to wisely invest in the program. In general, these products were
limited to descriptions of (1) technology characteristics, which is one of
many enterprise architecture elements, and (2) one of nine business
operations (finance and accounting). Our specific analysis of “To Be” and
transition plan products follows.

“To Be” Products: A “To Be” architecture is intended to capture the vision
of future business operations and supporting technology. It should describe
the desired capabilities, structures (e.g., entities, activities, and roles), and
relationships among these structures at a specified point(s) in the future.
The “To Be” architecture should show, for example, future business
processes, information needs, and supporting infrastructure and be fiscally




Page 19                                    GAO-04-43 NASA’s Enterprise Architecture
and technologically achievable. According to relevant guidance,22 the “To
Be” architecture should contain, among other things, a description of
(1) the future business operations that will be performed to support the
organization’s mission, including the entities or people that will perform
the functions, processes, and activities, and the locations where the
functions, processes, and activities will be performed; (2) the logical
database model that is to be used to guide the creation of the physical
databases where information will be stored; (3) the systems to be
developed or acquired to support the business operations; (4) the physical
infrastructure (e.g., hardware and software) that will be needed to support
the business systems; (5) the organizations that will be accountable for
implementing security and the tools to be used to secure and protect
systems and data; and (6) the metrics that will be used to evaluate the
effectiveness of mission operations and supporting system performance in
achieving mission goals and objectives. By including these elements, the
architecture would provide NASA with the necessary frame of reference
for engineering business processes and systems in a manner that supports
agencywide goals and objectives, such as ensuring that decision makers
routinely receive timely, accurate, and reliable information.

The “To Be” architecture products used to date in acquiring and
implementing IFMP provide minimal descriptive content. On the positive
side, they contain a description of one future business operation (i.e.,
finance and accounting). However, they do not describe other future
business operations (e.g., asset management and human resources). In
addition, they do not describe (1) finance and accounting in terms of the
entities or people who will perform the functions, processes, and activities
and the locations where the functions, processes, and activities will be
performed; (2) the logical database model to be used to create the physical
databases; (3) the actual systems to be developed or acquired to support
future business operations; (4) the physical infrastructure (e.g., hardware
and software) that will be needed to support the business systems; (5) the
organizations that will be accountable for implementing security and the


22
 See, for example, Office of Management and Budget, Federal Enterprise Architecture
Business Reference Model, Version 1.0 (2002); Chief Information Officer Council, A
Practical Guide to Federal Enterprise Architecture, Version 1.0 (February 2001); Office of
Management and Budget, Management of Federal Information Resources, Circular No.
A-130 (Nov. 28, 2000); M.A. Cook, Building Enterprise Information Architectures:
Reengineering Information Systems (Prentice Hall Inc.: 1996); and National Institute of
Standards and Technology, Information Management Directions: The Integration
Challenge, Special Publication 500-167 (September 1989).




Page 20                                         GAO-04-43 NASA’s Enterprise Architecture
tools to be used to secure and protect systems and data; and (6) the metrics
that will be used to evaluate the effectiveness of mission operations and
supporting system performance in achieving mission goals and objectives.
Without this information, the organization will not have a common vision
and frame of reference for defining a transition plan to guide and constrain
the transformation of business operations and associated capital
investments and, thus, will be unable to effectively leverage technology to
orchestrate logical and systematic change and optimize enterprisewide
mission performance. Detailed results of our analysis are provided in
appendix II.

Transition Plan Products: According to relevant guidance and best
practices,23 the transition plan should provide a temporal road map for
moving from the “As Is” to the “To Be” environment. An important step in
developing a well-defined transition plan is a gap analysis—comparison of
the “As Is” and “To Be” architectures to identify differences. Other
important steps include analyzing technology opportunities and
marketplace trends, as well as assessing fiscal and budgetary realities and
institutional acquisition and development capabilities. With the use of such
analyses and assessments, options are explored and decisions are made
regarding which legacy systems to retain, modify, or retire and which new
systems to introduce on a tactical (temporary) basis or to pursue as
strategic solutions. Accordingly, transition plans identify legacy, migration,
and new systems and sequence them to show, for example, the phasing out
and termination of systems and capabilities and the timing of the
introduction of new systems and capabilities, and they do so in light of
resource constraints, such as budget, people, acquisition/development
process maturity, and associated time frames.

The transition plan artifacts that NASA relied on in acquiring and
implementing IFMP generally do not possess these attributes. Specifically,
they do not (1) provide a gap analysis identifying the needed changes to
current business processes and systems; (2) identify all of the systems that
will not become part of the “To Be” architecture, as well as the time frames
for phasing out these systems; (3) show a time-based strategy for replacing
legacy systems, including identifying intermediate (i.e., migration) systems

23
 See, for example, Office of Management and Budget, Federal Enterprise Architecture
Business Reference Model, Version 1.0 (2002); Chief Information Officer Council, A
Practical Guide to Federal Enterprise Architecture, Version 1.0 (February 2001); and Office
of Management and Budget, Management of Federal Information Resources, Circular No.
A-130 (Nov. 28, 2000).




Page 21                                         GAO-04-43 NASA’s Enterprise Architecture
that may be temporarily needed; and (4) define the resources (e.g., funding
and staff) needed to transition to the target environment. The result is that
NASA has not had a meaningful and reliable basis for managing the
disposition of its systems or for sequencing the introduction of modernized
business operations and supporting systems. Detailed results of our
analysis appear in appendix II.

The chief technology officer agreed that the architecture products used to
date to acquire and implement IFMP do not provide sufficient scope and
content to constitute a well-defined enterprise architecture. Based on our
experience in reviewing other agencies, not having an effective enterprise
architecture program is attributable to, among other things, limited senior
management understanding and commitment and cultural resistance to
having and using an architecture.

Our experience with federal agencies has shown that attempting to define
and build major IT systems without first completing an enterprise
architecture often results in IT systems that are duplicative, are not well
integrated, are unnecessarily costly to maintain and interface, and do not
effectively optimize mission performance. In fact, NASA’s OIG recently
reported24 that the agency would need to resolve several accounting and
costing issues before the IFMP core financial module, which is to
implement NASA’s finance and accounting business process, would be able
to provide full cost-accounting capabilities as envisioned. To accomplish
this, the agency will have to reconfigure the IFMP software to reflect these
changes, resulting in system rework and additional associated costs that
could have been prevented.

Beyond this known rework, additional corrective action could be
necessary to address any misalignment between already implemented
IFMP system components and NASA’s just-released initial version of an
enterprise architecture. Specifically, the chief technology officer provided
us with an initial version of a NASA enterprise architecture on September
24, 2003, which was after we completed our audit work. According to this
official, although this initial version of the architecture is incomplete, it
does provide some of the missing contextual information (operational and
technical) that we had identified during our review. The official also stated


24
 National Aeronautics and Space Administration Office of Inspector General, Integrated
Financial Management Program Core Financial Module Conversion to Full Cost
Accounting, IG-03-015 (Washington, D.C.: May 30, 2003).




Page 22                                        GAO-04-43 NASA’s Enterprise Architecture
                            that future versions of the architecture are to be issued quarterly through
                            June 2004 and semiannually thereafter, and that plans are currently being
                            developed for assessing IFMP’s alignment with the architecture. The IFMP
                            deputy program manager affirmed these plans for assessing IFMP’s
                            alignment. In the likely event that any misalignment is found, NASA will be
                            faced with additional system rework demands.



NASA Does Not Have Key      As NASA proceeds with its enterprise architecture effort, it is critical that it
Capabilities in Place for   employs rigorous and disciplined management practices in doing so. Such
                            practices form the basis of our architecture management maturity
Effectively Managing Its
                            framework,25 which specifies by stages the key architecture management
Recently Launched           controls that are embodied in federal guidance and best practices, provides
Enterprise Architecture     an explicit benchmark for gauging the effectiveness of architecture
Effort                      management, and provides a road map for making improvements. Each of
                            the five stages is described below.

                            1. Creating enterprise architecture awareness. The organization does
                               not have plans to develop and use an architecture, or it has plans that
                               do not demonstrate an awareness of the value of having and using an
                               architecture. While stage 1 agencies may have initiated some
                               architecture activity, these agencies’ efforts are ad hoc and
                               unstructured, lack institutional leadership and direction, and do not
                               provide the management foundation necessary for successful
                               architecture development.

                            2. Building the enterprise architecture management foundation. The
                               organization recognizes that the architecture is a corporate asset by
                               vesting accountability for it in an executive body that represents the
                               entire enterprise. At this stage, an organization assigns architecture
                               management roles and responsibilities and establishes plans for
                               developing enterprise architecture products and for measuring
                               program progress and product quality; it also commits the resources
                               necessary for developing an architecture—people, processes, and
                               tools.




                            25
                             U.S. General Accounting Office, Information Technology: A Framework for Assessing
                            and Improving Enterprise Architecture Management, Version 1.1, GAO-03-584G
                            (Washington, D.C.: April 2003).




                            Page 23                                      GAO-04-43 NASA’s Enterprise Architecture
3. Developing the enterprise architecture. The organization focuses on
   developing architecture products according to the selected framework,
   methodology, tool, and established management plans. Roles and
   responsibilities assigned in the previous stage are in place, and
   resources are being applied to develop actual enterprise architecture
   products. The scope of the architecture has been defined to encompass
   the entire enterprise, whether organization-based or function-based.

4. Completing the enterprise architecture. The organization has
   completed its enterprise architecture products, meaning that the
   products have been approved by the architecture steering committee or
   an investment review board, and by the CIO. Further, an independent
   agent has assessed the quality (i.e., completeness and accuracy) of the
   architecture products. Additionally, evolution of the approved products
   is governed by a written architecture maintenance policy approved by
   the head of the organization.

5. Leveraging the enterprise architecture to manage change. The
   organization has secured senior leadership approval of the enterprise
   architecture products and a written institutional policy stating that IT
   investments must comply with the architecture, unless granted an
   explicit compliance waiver. Further, decision makers are using the
   architecture to identify and address ongoing and proposed IT
   investments that are conflicting, overlapping, not strategically linked,
   or redundant. Also, the organization tracks and measures architecture
   benefits or return on investment, and adjustments are continuously
   made to both the architecture management process and the enterprise
   architecture products.

For stage 2, our framework specifies nine key practices or core elements
that are necessary to provide the management foundation for successfully
launching and sustaining an architecture effort. Examples of stage 2 core
elements are described below.

• Establish a committee or group representing the enterprise that is
  responsible for directing, overseeing, or approving the enterprise
  architecture. This committee should include executive-level
  representatives from each line of business, and these representatives
  should have the authority to commit resources and enforce decisions
  within their respective organizational units. By establishing this
  enterprisewide responsibility and accountability, the agency




Page 24                                 GAO-04-43 NASA’s Enterprise Architecture
   demonstrates its commitment to building the management foundation
   and obtaining buy-in from across the organization.

• Appoint a chief architect. The chief architect should be responsible and
  accountable for the enterprise architecture, supported by the
  architecture program office, and overseen by the architecture steering
  committee. The chief architect, in collaboration with the CIO, the
  architecture steering committee, and the organizational head, is
  instrumental in obtaining organizational buy-in for the enterprise
  architecture, including support from the business units, as well as in
  securing resources to support architecture management functions, such
  as risk management, configuration management, quality assurance, and
  security management.

• Use a framework, methodology, and automated tool to develop the
  enterprise architecture. These elements are important because they
  provide the means for developing the architecture in a consistent and
  efficient manner. The framework provides a formal structure for
  representing the enterprise architecture, while the methodology is the
  common set of procedures that the enterprise is to follow in developing
  the architecture products. The automated tool serves as a repository
  where architectural products are captured, stored, and maintained.

• Develop an architecture program management plan. This plan
  specifies how and when the architecture is to be developed. It includes a
  detailed work breakdown structure; resource estimates (e.g., funding,
  staffing, and training); performance measures; and management
  controls for developing and maintaining the architecture. The plan
  demonstrates the organization’s commitment to managing architecture
  development and maintenance as a formal program.

• Allocate adequate resources. An organization needs to have the
  resources (funding, people, tools, and technology) to establish and
  effectively manage its architecture. This includes, among other things,
  identifying and securing adequate funding to support architecture
  activities, hiring and retaining the right people, and selecting and
  acquiring the right tools and technology to support activities.

Our framework similarly identifies key architecture management practices
associated with later stages of architecture management maturity. For
example, at stage 3, the stage at which organizations focus on architecture




Page 25                                 GAO-04-43 NASA’s Enterprise Architecture
development activities, organizations need to satisfy six core elements.
Examples of these core elements are discussed below.

• Issue a written and approved organization policy for development of
  the enterprise architecture. The policy defines the scope of the
  architecture, including the requirement for a description of the baseline
  and target architecture, as well as an investment road map or
  sequencing plan specifying the move between the two. This policy is an
  important means for ensuring enterprisewide commitment to
  developing an enterprise architecture and for clearly assigning
  responsibility for doing so.

• Ensure that enterprise architecture products are under configuration
  management. This involves ensuring that changes to products are
  identified, tracked, monitored, documented, reported, and audited.
  Configuration management maintains the integrity and consistency of
  products, which is key to enabling effective integration among related
  products and for ensuring alignment between architecture artifacts.

At stage 4, during which organizations focus on architecture completion
activities, organizations need to satisfy eight core elements. Examples of
these core elements are described below.

• Ensure that enterprise architecture products and management
  processes undergo independent verification and validation. This core
  element involves having an independent third party—such an as internal
  audit function or contractor that is not involved with any of the
  architecture development activities—verify and validate that the
  products were developed in accordance with architecture processes
  and product standards. Doing so provides organizations with needed
  assurance of the quality of the architecture.

• Ensure that business, performance, information/data,
  application/service, and technology descriptions address security. An
  organization should explicitly and consistently address security in its
  business, performance, information/data, application/service, and
  technology architecture products. Because security permeates every
  aspect of an organization’s operations, the nature and substance of
  institutionalized security requirements, controls, and standards should
  be captured in the enterprise architecture products.




Page 26                                 GAO-04-43 NASA’s Enterprise Architecture
At stage 5, during which the focus is on architecture maintenance and
implementation activities, organizations need to satisfy eight core
elements. Examples of these core elements are described below.

• Make the enterprise architecture an integral component of the IT
  investment management process. Because the road map defines the IT
  systems that an organization plans to invest in as it transitions from the
  “As Is” to the “To Be” environment, the enterprise architecture is a
  critical frame of reference for making IT investment decisions. Using the
  architecture when making such decisions is important because
  organizations should approve only those investments that move the
  organization toward the “To Be” environment, as specified in the road
  map.

• Measure and report return on enterprise architecture investment. Like
  any investment, the enterprise architecture should produce a return on
  investment (i.e., a set of benefits), and this return should be measured
  and reported in relation to costs. Measuring return on investment is
  important to ensure that expected benefits from the architecture are
  realized and to share this information with executive decision makers,
  who can then take corrective action to address deviations from
  expectations.

Table 5 summarizes our framework’s five stages and the associated core
elements for each stage.




Page 27                                 GAO-04-43 NASA’s Enterprise Architecture
Table 5: Summary of the Maturity Stages and Core Elements of GAO’s Enterprise Architecture (EA) Management Framework

Stage                            Core elements
Stage 1:                         • Agency is aware of EA.
Creating EA awareness
Stage 2:                         • Adequate resources exist.
Building the EA management       • Committee or group representing the enterprise is responsible for directing, overseeing, or approving
foundation                         EA.
                                 • Program office responsible for EA development and maintenance exists.
                                 • Chief architect exists.
                                 • EA is being developed using a framework, methodology, and automated tool.
                                 • EA plans call for describing the “As Is” environment, the “To Be” environment, and a sequencing plan.
                                 • EA plans call for describing the enterprise in terms of business, information/data, application/service,
                                   and technology.
                                 • EA plans call for business, performance, information/data, application/service, and technology
                                   descriptions to address security.
                                 • EA plans call for developing metrics for measuring EA progress, quality, compliance, and return on
                                   investment.
Stage 3:                         • Written and approved organization policy exists for EA development.
Developing EA products           • EA products are under configuration management.
(includes all elements from      • EA products describe or will describe the enterprise’s business, performance, information/data,
stage 2)                           application/service, and the technology that supports them.
                                 • EA products describe or will describe the “As Is” environment, the “To Be” environment, and a
                                   sequencing plan.
                                 • Business, performance, information/data, application/service, and technology descriptions address or
                                   will address security.
                                 • Progress against EA plans is measured and reported.
Stage 4:                         •   Written and approved organization policy exists for EA maintenance.
Completing EA products           •   EA products and management processes undergo independent verification and validation.
(includes all elements from      •   EA products describe the “As Is” environment, the “To Be” environment, and a sequencing plan.
stage 3)                         •   EA products describe the enterprise’s business, performance, information/data, application/service,
                                     and the technology that supports them.
                                 •   Business, performance, information/data, application/service, and technology descriptions address
                                     security.
                                 •   Organization chief information officer has approved current version of EA.
                                 •   Committee or group representing the enterprise or the investment review board has approved current
                                     version of EA.
                                 •   Quality of EA products is measured and reported.
Stage 5:                         •   Written and approved policy exists for IT investment compliance with EA.
Leveraging the EA for managing   •   Process exists to formally manage EA change.
change                           •   EA is integeral component of IT investment management process.
(includes all elements from      •   EA products are periodically updated.
stage 4)                         •   IT investments comply with EA.
                                 •   Organization head has approved current version of EA.
                                 •   Return on EA investment is measured and reported.
                                 •   Compliance with EA is measured and reported.
Source: GAO.




                                                Page 28                                         GAO-04-43 NASA’s Enterprise Architecture
The state of NASA’s implementation of key enterprise architecture
management practices, conditions, and structures currently places the
agency at stage 1 of our maturity framework. Specifically, it has satisfied all
but one of the core elements associated with building the architecture
management foundation—stage 2 of our framework—but only about 23
percent (5 of the 22) of the core elements associated with stages 3, 4, and 5.
According to our framework, effective architecture management is
generally not achieved until an enterprise has a completed and approved
architecture that is being effectively maintained and is being used to
leverage organizational change and support investment decision making;
having these characteristics is equivalent to having satisfied all stage 3 core
elements and many stage 4 and 5 elements.

Regarding stage 2 core elements, NASA has, for example, recently
established a program office, assigned a chief architect, and selected a
framework (Zachman) and automated tools (e.g., the Rational Rose by
Rational Software Corporation/IBM Software Group). However, the agency
has not satisfied a stage 2 core element that is critical to effective
architecture management. Specifically, a committee or group representing
the enterprise has not yet been established to guide, direct, or approve the
architecture. Instead, the CIO is guiding the architecture development
effort. Having such a corporate entity is critical to overcoming cultural
resistance to using an enterprise architecture. Without such an entity to
lead and be accountable for the architectural effort, there is increased risk
that the architecture will not represent a corporate decision-making tool
and will not be viewed and endorsed as an agencywide asset.

Concerning stage 3, NASA has not satisfied three of six core elements. For
example, the agency does not have a written and approved policy for
architecture development, which is a stage 3 core element. Without such a
policy that, for example, identifies the major players in the development
process and provides for architecture guidance, direction, and approval,
NASA will be challenged in overcoming cultural resistance to using an
enterprise architecture and achieving agencywide architecture
commitment and support.

The agency also has yet to implement numerous stage 4 and 5 core
elements. For example, NASA has not (1) documented and approved a
policy for architecture maintenance, (2) implemented an independent
verification and validation function that covers architecture products and
architecture management processes, and (3) made the architecture an
integral component of its IT investment management process. (The



Page 29                                   GAO-04-43 NASA’s Enterprise Architecture
                      detailed results of our assessment of NASA’s satisfaction of each of the
                      stages and associated core elements are provided in app. III.)

                      According to the chief technology officer, the agency recognizes the
                      importance of having rigorous and disciplined architecture management
                      controls and is in the process of establishing them. Our research of
                      successful organizations and experience in reviewing other agency
                      enterprise architecture efforts shows that not having these controls is,
                      among other things, a function of limited senior management
                      understanding of and commitment to an enterprise architecture and
                      cultural resistance to having and using one. Until such barriers are
                      addressed, and effective architecture management structures and
                      processes are established, it is unlikely that any agency will be able to
                      produce and maintain a complete and enforceable architecture or
                      implement modernized systems in a way that minimizes overlap and
                      duplication and maximizes integration and mission support.



Conclusions           NASA’s acquisition and implementation of six major IFMP system
                      components outside the context of an enterprise architecture was not a
                      prudent decision. Such a systems modernization approach unnecessarily
                      increases the risk that system components will not effectively and
                      efficiently support agencywide operations, which in turn leads to costly
                      system rework. It is critical for NASA to discontinue this approach and
                      adopt the best practice of managing its IFMP system investments within
                      the context of a well-defined enterprise architecture. In order to do so, it is
                      important for NASA to establish an effective means for developing and
                      implementing an architecture, which includes gaining top management
                      understanding and support to lead the way in overcoming any cultural
                      resistance. It is equally important that the agency ensure that the
                      architecture contains sufficient depth and scope, quickly determine
                      whether existing and planned IFMP component systems align with initial
                      and subsequent versions of the architecture, and limit further investment in
                      these systems until such determinations are made. To do less risks
                      introducing additional system rework to that already facing the agency on
                      already implemented system components.



Recommendations for   To ensure that NASA has the necessary agencywide context within which
                      to make informed IFMP and other systems modernization decisions, we
Executive Action      recommend that the NASA Administrator demonstrate an institutional



                      Page 30                                   GAO-04-43 NASA’s Enterprise Architecture
commitment to developing and using an enterprise architecture by
establishing a NASA enterprise architecture policy and designating a NASA
architecture board, or comparable body, that is made up of agency
executives who are responsible and accountable for developing and
maintaining the architecture.

In carrying out its responsibility, we recommend that the Administrator
direct the architecture board, in collaboration with the CIO, to ensure that
the architecture content requirements identified in this report are satisfied
by first determining the extent to which NASA’s initial release of an
enterprise architecture satisfies these content requirements and then
developing and approving a plan for incorporating any content that is
missing.

We further recommend that the Administrator direct the IFMP Program
Executive Officer to appropriately limit acquisition and implementation
activities until the agency ensures that the program’s plans are aligned with
the initial and subsequent versions of the enterprise architecture. In
addition, we recommend that the Administrator direct the architecture
board, in collaboration with the CIO, to immediately map already
implemented IFMP components to the agency’s enterprise architecture and
report to the Program Executive Officer any instances of misalignment, the
associated risks, and proposed corrective actions. Moreover, we
recommend that the Administrator direct the Program Executive Officer to
develop corrective action plans, as appropriate, that include specific
milestones, cost estimates, and detailed actions to be taken to align the
program with the enterprise architecture.

To further assist NASA, we recommend that the Administrator direct the
board, in collaboration with the CIO, to ensure that the best practices
involved in stages 3 through 5 of our enterprise architecture management
maturity framework are implemented. More specifically, we recommend
that the board and the CIO (1) establish a written and approved policy for
architecture development, (2) place enterprise architecture products under
configuration management, and (3) ensure that progress against
architecture plans is measured and reported.

In completing the architecture, we recommend that the board and CIO
(1) establish a written and approved policy for architecture maintenance;
(2) ensure that enterprise architecture products and management
processes undergo independent verification and validation; (3) ensure that
architecture products describe the enterprise’s business and the data,



Page 31                                  GAO-04-43 NASA’s Enterprise Architecture
                      application, and technology that supports it; (4) ensure that enterprise
                      architecture products describe the “As Is” environment, the “To Be”
                      environment, and a sequencing plan; (5) ensure that business,
                      performance, data, application, and technology descriptions address
                      security; (6) ensure that the CIO approves the enterprise architecture;
                      (7) ensure that the steering committee and/or the investment review board
                      has approved the current version of the enterprise architecture; and
                      (8) measure and report on the quality of enterprise architecture products.

                      In implementing the architecture, we recommend that the board and CIO
                      (1) establish a written and approved policy for IT investment compliance
                      with the enterprise architecture, (2) ensure that the enterprise architecture
                      is an integral component of IT investment management processes,
                      (3) ensure that IT investments comply with the enterprise architecture,
                      (4) obtain Administrator approval of each enterprise architecture version,
                      (5) measure and report enterprise architecture return on investment, and
                      (6) measure and report on enterprise architecture compliance.



Agency Comments and   In written comments on a draft of this report signed by the Deputy
                      Administrator (reprinted in app. IV), NASA concurred with our
Our Evaluation        recommendations and described recently completed, ongoing, or planned
                      efforts to address them. For example, the agency stated that it has
                      developed a 3-year plan for refining the latest version of its architecture, as
                      well as a plan to guide the agency in using the architecture to achieve
                      NASA’s strategic goals. In addition, the agency stated that it has adopted
                      our architecture management maturity framework and is currently working
                      to satisfy the framework’s core elements, including establishing
                      architecture policies and a function for independently verifying and
                      validating architecture artifacts and management practices. Additionally, it
                      stated that it plans to continually validate IFMP against its architecture on a
                      quarterly basis.

                      NASA also stated that its CIO board, which is chaired by NASA’s CIO and
                      composed of the CIOs from the agency’s six major lines of business and its
                      ten field centers, serves as the NASA architecture board or steering
                      committee. While we support CIO representation on an architecture
                      steering committee, recognized best practices and our maturity framework
                      both advocate that architecture ownership and accountability be vested
                      with an enterprise’s business owners. Thus, we state in our framework that
                      the architecture steering committee should be composed of executive-level
                      representatives from each line of business and that these representatives



                      Page 32                                   GAO-04-43 NASA’s Enterprise Architecture
should have the authority to commit resources and enforce decisions for
their respective organizational units. Without such an entity to lead and be
accountable for the architectural effort, there is increased risk that the
architecture will be viewed solely as an IT tool and not represent a
corporate, business-driven decision-making tool and will not be viewed and
endorsed as an agencywide asset. Accordingly, it is important for NASA to
ensure that its architecture board’s membership includes business owner
representation.


As agreed with your offices, unless you announce its contents earlier, we
will not distribute this report further until 30 days from its date. At that
time, we will send copies to interested congressional committees as well as
to the NASA Administrator and the Director of OMB. We will make copies
available to others upon request. In addition, the report will be available at
no charge on the GAO Web site at http://www.gao.gov.

If you or your staff have any questions concerning this report, please
contact me at (202) 512-3439 or hiter@gao.gov. Key contributors to this
report are acknowledged in appendix V.




Randolph C. Hite
Director
Information Technology Architecture
  and Systems Issues




Page 33                                   GAO-04-43 NASA’s Enterprise Architecture
Appendix I

Objective, Scope and Methodology                                                                           AA
                                                                                                            ppp
                                                                                                              ep
                                                                                                               ned
                                                                                                                 n
                                                                                                                 x
                                                                                                                 id
                                                                                                                  e
                                                                                                                  x
                                                                                                                  Iis




              To determine whether the National Aeronautics and Space Administration
              (NASA) had and was using an enterprise architecture to guide and
              constrain its investment in its Integrated Financial Management Program
              (IFMP), we requested all NASA enterprise architecture artifacts and related
              documentation that had been used to date to guide and constrain IFMP
              and, based on what we were provided by NASA’s chief technology officer,1
              compared them to relevant guidance.2

              In doing so, we first segmented our analysis of artifacts and guidance into
              the three primary component parts of any architecture: the “As Is”
              architecture, the “To Be,” and the transition plan. We then further divided
              the “As Is” and “To Be” architectures into five architectural components
              similar to the Office of Management and Budget’s architecture reference
              models: business, information/data, services/applications, technical, and
              performance. We also added security as a sixth component because of its
              recognized importance in the various architecture frameworks and
              relevance to the other five architectural components. Because NASA had
              not clearly distinguished between its “As Is” and “To Be” environments, the
              chief technology officer told us to treat the architecture products provided
              as the “To Be” environment and assume that any “As Is” content would be
              intended for reuse in the future environment. As a result, we did not
              analyze whether NASA’s architecture products satisfied relevant “As Is”
              guidance; instead, we accepted the chief technology officer’s
              acknowledgment that NASA did not have any “As Is” artifacts.

              To augment our documentation reviews and analyses of architecture
              products used to date in acquiring and implementing IFMP, we also
              interviewed various officials, including the chief information officer and
              chief technology officer, to determine, among other things, the agency’s
              plans to develop an enterprise architecture. Specifically, we inquired as to
              NASA’s basis for selecting already acquired IFMP commercial products and


              1
               We reviewed technology architectures and an enterprise architecture for the IFMP core
              financial module.
              2
               See, for example, Office of Management and Budget, Federal Enterprise Architecture
              Business Reference Model, Version 1.0 (2002); Chief Information Officers Council, A
              Practical Guide to Federal Enterprise Architecture, Version 1.0 (February 2001); Office of
              Management and Budget, Management of Federal Information Resources, Circular No.
              A-130 (Nov. 28, 2000); M.A. Cook, Building Enterprise Information Architectures:
              Reengineering Information Systems (Prentice Hall Inc.: 1996); and National Institute of
              Standards and Technology, Information Management Directions: The Integration
              Challenge, Special Publication 500-167 (September 1989).




              Page 34                                         GAO-04-43 NASA’s Enterprise Architecture
Appendix I
Objective, Scope and Methodology




its plans for selecting future IFMP modules, including whether the agency
had developed an enterprise architecture to guide and constrain its future
investment in IFMP.

We also requested information on ongoing efforts to develop the initial
version of NASA’s enterprise architecture, such as detailed program plans,
updated policies and procedures, and the architecture itself, but this
information was not provided until September 24, 2003, which was after we
had completed our audit work. As a result, our review did not include an
assessment of the initial version of NASA’s enterprise architecture, which
the chief technology officer stated addressed some, but not all, of the
limitations discussed in this report.

To determine whether NASA’s initial and subsequent versions of its
enterprise architecture were supported by effective management
structures and processes, we used our Enterprise Architecture
Management Maturity Framework,3 which describes the five stages of
management maturity, and determined the extent to which NASA has
adopted key elements of architecture management best practices
embodied in the framework. To make this determination, we reviewed
program documentation, such as program policies and procedures, an IBM
report4 on the agency’s efforts to implement management processes and
controls over its architecture development activities, and the architecture
products used to date in acquiring IFMP system components, and we
compared them to the elements in our framework. We did not
independently validate the cost and budget information provided by the
chief technology officer.

We conducted our work at NASA headquarters in Washington, D.C. We
performed our work from June to mid-September 2003 in accordance with
generally accepted government auditing standards.




3
 U.S. General Accounting Office, Information Technology: A Framework for Assessing and
Improving Enterprise Architecture Management, Version 1.1, GAO-03-584G (Washington,
D.C.: April 2003).
4
 IBM, NASA Enterprise Architecture: Roadmap for Building and Sustaining Business
Value Through an Integrated EA, Sept. 6, 2002.




Page 35                                       GAO-04-43 NASA’s Enterprise Architecture
Appendix II

Detailed Results of GAO’s Analyses of
Architecture Products Used to Date by NASA
to Acquire and Implement IFMP                                                                                                          Appendx
                                                                                                                                             Ii




Detailed Analysis of NASA’s “To Be” Architecture Products

Key architectural element                Element satisfied?         Explanation of partially satisfied
                                        Yes       No    Partially
Business
A description of the overall                               X        The available architecture products contain a high-level
architectural vision and strategic                                  description of the agency’s OneNASA vision, which focuses on
goals that define what an                                           how technology will be managed to improve services (e.g.,
organization wants to achieve.                                      providing secure and highly interoperable information systems
                                                                    in support of all NASA operations). It lists mission statements
                                                                    for both the agency and the lines of business. The architecture
                                                                    also lists business architecture drivers, which can be
                                                                    considered business goals.

                                                                    However, the available architecture products do not contain a
                                                                    description of the strategic goals, objectives, missions, and
                                                                    implementing strategies established to support NASA lines of
                                                                    business. In addition, the architecture products do not explain
                                                                    what the OneNASA vision encompasses since it appears to be
                                                                    technology-centric, as opposed to business-centric (i.e., it
                                                                    addresses business, information/data, services/applications,
                                                                    and technology).
A business strategy, which defines                         X        The available architecture products list business strategies,
how the enterprise’s strategic goals                                such as implementing the Integrated Financial Management
and objectives will be achieved.                                    Program (IFMP). However, they do not describe how these
                                                                    strategies will be implemented.
Common (standard and                               X
agencywide) policies, procedures,
and business and operational rules
for consistent implementation of the
architecture.
A description of key business                              X        The available architecture products contain a description of the
processes and how they support the                                  finance and accounting processes (i.e., the processes to be
agency’s mission, including the                                     supported by the IFMP core financial module). This description
organizational units responsible for                                also identifies the subprocesses within these processes and
performing the business processes                                   includes detailed diagrams of process flows.
and the locations where the
business processes will be                                          However, these products do not identify the organizational
performed. This description should                                  units responsible for performing the finance and accounting
provide for the consistent alignment                                business processes nor the locations where they will be
of (1) applicable federal laws,                                     performed. Moreover, the architecture products do not contain
regulations, and guidance;                                          a description of other business processes, such as asset
(2) agency policies, procedures, and                                management, human resource management, and budget.
guidance; (3) operational activities;
(4) organizational roles; and
(5) operational events and
information.




                                              Page 36                                     GAO-04-43 NASA’s Enterprise Architecture
                                                Appendix II
                                                Detailed Results of GAO’s Analyses of
                                                Architecture Products Used to Date by NASA
                                                to Acquire and Implement IFMP




(Continued From Previous Page)
Key architectural element                  Element satisfied?            Explanation of partially satisfied
                                          Yes       No     Partially
A description of the operational                     X
management processes to ensure
that the agency’s business
transformation effort remains
compliant with the business rules for
fault, performance, security,
configuration, and account
management.
A listing of opportunities to unify and                        X         The available architecture products recognize the need for an
simplify systems or processes                                            implementing strategy to streamline financial operations and
across the agency.                                                       identify IFMP as that strategy. However, the products do not
                                                                         describe specific opportunities for improving weaknesses in
                                                                         the “As Is” financial systems or processes or how IFMP will be
                                                                         implemented to achieve this unification/simplification.
A description of the organizational                            X         The available architecture products contain a description of the
approach (processes and                                                  management reporting lines for the agency’s chief information
organizational structure) for                                            officer (CIO) organization as it relates to managing the
communications and interactions                                          architecture products and standards. However, they do not
among business lines and program                                         describe the roles and responsibilities of other organizations.
areas for (1) management reporting,                                      For example, these products do not have a model for roles and
(2) operational functions, and                                           an organization chart that shows the lines of communication
(3) architecture development and                                         and reporting responsibilities for financial management
use (i.e., how to develop the                                            operations.
architecture description, implement
the architecture, and
govern/manage the development
and implementation of the
architecture).
Information/data
A description of data management                     X
policies, processes, procedures, and
tools (e.g., CRUD matrixa) for
analyzing, designing, building, and
maintaining databases in an
enterprise architected environment.
A description of the business and                              X         The available architecture products contain a description of
operational rulesb for data                                              technical standards currently being used. However, these
standardization to ensure data                                           products do not state whether these standards are still relevant
consistency, integrity, and accuracy,                                    or will need to be updated to reflect changes to the current
such as business and security rules                                      environment. In addition, the architecture products do not
that govern access, maintenance,                                         identify data standards upon which business rules can later be
and use of data.                                                         developed.c
A data dictionary, which is a                        X
repository of standard data
definitions for applications.




                                                Page 37                                        GAO-04-43 NASA’s Enterprise Architecture
                                               Appendix II
                                               Detailed Results of GAO’s Analyses of
                                               Architecture Products Used to Date by NASA
                                               to Acquire and Implement IFMP




(Continued From Previous Page)
Key architectural element                 Element satisfied?            Explanation of partially satisfied
                                         Yes       No     Partially
A conceptual data model that                        X
describes the fundamental
things/objects (e.g., invoices,
financial statements, inventory) that
make up the business, but without
regard for how they will be physically
stored. It represents the
consolidated structure of business
objects to be used by business
applications.
A logical database model that                       X
provides (1) the data structures that
support information flows and (2) the
basis for developing the schemas for
designing, building, and maintaining
physical databases.d
A metadatae model that specifies the                X
rules and standards for access to
information.f
A description of information flows                  X
and relationships between
organizational units, business
operations, and system elements.
Services/applications
A description of the end-user                       X
services to be provided by the
application systems.
A list of application systems                       X
(acquisition/development and
production portfolio)g and their
relative importance to achieving the
agency’s vision based on business
value and technical performance.
A description of the policies,                      X
procedures, processes, and tools for
selecting, controlling, and evaluating
application systems to enable
effective IT investment management.

A description of the enterprise                               X         The available architecture products contain a description of an
application systems and                                                 enterprise application system that supports finance and
components and their interfaces.g                                       accounting (IFMP core financial module) functions. They also
                                                                        identify legacy systems that interface with this application
                                                                        system.

                                                                        However, this description is limited to this one system.




                                               Page 38                                         GAO-04-43 NASA’s Enterprise Architecture
                                               Appendix II
                                               Detailed Results of GAO’s Analyses of
                                               Architecture Products Used to Date by NASA
                                               to Acquire and Implement IFMP




(Continued From Previous Page)
Key architectural element                 Element satisfied?            Explanation of partially satisfied
                                         Yes       No     Partially
A description of the common                                   X         The available architecture products list several architectural
technical approach, policies, and                                       principles for system development and acquisition (e.g.,
procedures for developing/acquiring                                     modular design, open system approach) and identify a strategy
application systems throughout their                                    (i.e., a hub-spoke configuration) for integrating legacy systems.
life cycle, including requirements                                      They also identify a minimum set of technical standards for
management, design,                                                     hardware and software. In addition, the architecture products
implementation, testing, deployment,                                    contain policies and guidance for implementing systems.
operations, and maintenance. The
common technical approach should                                        However, these products do not describe a common technical
also describe the process for                                           approach. In addition, the products did not state whether the
integrating legacy systems with the                                     existing policies and procedures are common, complete, and
systems to be developed/acquired.                                       sufficient to effectively implement the architecture. They do
                                                                        recognize the need to revise existing policies and procedures.
Technical
A list of infrastructure systems and                X
their relative importance to achieving
the agency’s vision based on
business value and technical
performance.
A description of the policies,                      X
procedures, processes, and tools for
selecting, controlling, and evaluating
infrastructure systems to enable
effective IT investment management.
A description of the technical                                X         The available architecture products recognize the need for a
reference model (TRMh) that                                             TRM and contain a generic description of the TRM. These
describes the enterprise                                                products also note that technology services needed to support
infrastructure services,i including                                     the application portfolio should be defined and identify several
specific details regarding the                                          of these services.
functionality and capabilities that
these services will provide to enable                                   However, according to NASA’s chief technology officer, the
development of application systems.                                     TRM is incomplete and flawed. In addition, the list of
                                                                        technology services identified is incomplete.
A description in the TRM that                                 X         The available architecture products note that these standards
identifies and describes the                                            should be identified and documented. They also contain a list
technical standardsj to be                                              of specific standards.
implemented for each enterprise
service.                                                                However, the architecture products do not state whether these
                                                                        standards are for the current or future environment. In addition,
                                                                        the architecture products do not identify the technical
                                                                        standards to be implemented for specific enterprise services,
                                                                        such as query processing.




                                               Page 39                                         GAO-04-43 NASA’s Enterprise Architecture
                                               Appendix II
                                               Detailed Results of GAO’s Analyses of
                                               Architecture Products Used to Date by NASA
                                               to Acquire and Implement IFMP




(Continued From Previous Page)
Key architectural element                 Element satisfied?            Explanation of partially satisfied
                                         Yes       No     Partially
A description of the physical IT                              X         The available architecture products contain a high-level
infrastructure needed to support the                                    description (i.e., diagrams without supporting narrative) of the
developed and/or acquired systems,                                      IFMP core financial network, OneNASA network backbone,
including the relationships among                                       and NASA’s Information System Services Utility (NISSUk)
hardware, software, and                                                 network architecture.
communications devices.
                                                                        However, these networks do not encompass the entire
                                                                        enterprise, but rather a subset of activities.
Common policies and procedures for                            X         The available architecture products contain a list of policies
developing infrastructure systems                                       and procedures for implementing systems. However, the
throughout their life cycle, including                                  products do not state whether these policies and procedures
requirements management, design,                                        are common, complete, and sufficient to effectively implement
implementation, testing, deployment,                                    the architecture.
operations, and maintenance. These
policies and procedures should also
address the integration of
applications, including legacy
systems.
Security
A description of the policies,                                X         The available architecture products contain a high-level
procedures, goals, strategies, and                                      description of security goals and strategies and identify some
requirements relevant to information                                    security requirements. They also note that an “Information
assurance and security and how                                          Assurance Trust Model” is needed and will be developed.
they (the policies, procedures, goals,
strategies, and requirements) align                                     The architecture products do not contain a description of
and integrate with other elements of                                    security policies and procedures. They also do not identify
the architecture (e.g., security                                        important security requirements (e.g., availability and access
services).                                                              control), nor do they link identified security requirements to
                                                                        security services. Moreover, the architecture products do not
                                                                        define the “Information Assurance Trust Model” or address
                                                                        plans for its completion. Finally, regarding the strategies, they
                                                                        do not identify and summarize the agency’s most significant
                                                                        security risks.

                                                                        According to NASA’s chief technology officer, a clear computer
                                                                        security policy does not exist within the agency, and there is a
                                                                        lack of understanding as to how such a policy could be
                                                                        integrated into the network infrastructure.
Definitions of security and                                   X         The available architecture products contain definitions for some
information assurance related terms.                                    security-related terms (e.g., authentication, confidentiality, and
                                                                        intrusion detection); however, they do not define other key
                                                                        terms listed (e.g., integrity, physical security, and encryption
                                                                        services). In addition, the definitions for these terms are
                                                                        inconsistent with the definitions shown in current standards
                                                                        (e.g., firewalls).




                                               Page 40                                         GAO-04-43 NASA’s Enterprise Architecture
                                                Appendix II
                                                Detailed Results of GAO’s Analyses of
                                                Architecture Products Used to Date by NASA
                                                to Acquire and Implement IFMP




(Continued From Previous Page)
Key architectural element                  Element satisfied?            Explanation of partially satisfied
                                          Yes       No     Partially
A listing of accountable                             X
organizations and their respective
responsibilities for implementing
enterprise security services.
Organizational relationships are
important to show in an operational
view because they illustrate
fundamental roles (e.g., who
conducts operational activities) and
management relationships (e.g.,
what is the command structure or
relationship to other key players),
and how they influence the
operational nodes.
A description of operational security                X
rules that are derived from security
policies.
A description of enterprise security                           X         The available architecture products contain high-level
infrastructure services (e.g.,                                           descriptions of enterprise security services; however, in most
identification and authentication) that                                  instances, these products describe the technology components
will be needed to protect the                                            that will be implemented to provide the security service, and
agency’s assets, and the means for                                       not the security service. For example, the architecture products
implementing such a service (e.g.,                                       classify “Audit Logs” as a security service; however, audit logs
firewalls and intrusion detection                                        are generally the function/component within an “Auditing
software). This description should                                       Service.”
also address how these services will
align and integrate with other                                           The architecture products also do not link the security services
elements of the architecture (e.g.,                                      to security policies, procedures, goals, strategies, and
security policies and requirements).                                     requirements.
A description of the security                                  X         The available architecture products contain a description of
standards to be implemented for                                          various security standards, but it is unclear if these standards
each enterprise service. These                                           are relevant to the “As Is” or “To Be” environment or both.
standards should be derived from
security requirements. This                                              In addition, the architecture products do not contain a
description should also address how                                      traceability matrix that links goals, strategies, requirements,
these services will align and                                            and services to the security standards and security products
integrate with other elements of the                                     (e.g., SmartCard). They also do not clearly state whether the
architecture (e.g., security policies                                    list of security standards for enterprise services is complete.
and requirements).




                                                Page 41                                         GAO-04-43 NASA’s Enterprise Architecture
                                              Appendix II
                                              Detailed Results of GAO’s Analyses of
                                              Architecture Products Used to Date by NASA
                                              to Acquire and Implement IFMP




(Continued From Previous Page)
Key architectural element                Element satisfied?                 Explanation of partially satisfied
                                        Yes        No       Partially
A description of the protection                                 X           The available architecture products contain a high-level
mechanisms that will be                                                     description of protection mechanisms (e.g., firewalls). However,
implemented to secure the agency’s                                          they do not describe the level of protection needed and the
assets, such as firewalls and                                               types of services the protection mechanisms will provide to
intrusion detection software,                                               protect IFMP applications that access information/data,
including a description of the                                              business services/applications, and the various networks.
relationships among these
protection mechanisms.                                                      In addition, the architecture products do not contain a
                                                                            traceability matrix that links goals, strategies, requirements,
                                                                            and services to the security standards, so it is unclear whether
                                                                            this is the definitive list of protection mechanisms.
Performance
A description of the processes for                  X
establishing, measuring, tracking,
evaluating, and predicting business
performance regarding business
functions, baseline data, and service
levels.
A description of customer-focused                   X
measurable business goals and
outcomes for business products and
services through the execution of
financial and financial-related
business activities.
A description of measurable                         X
technical goals and outcomes for
managing technology products and
services for the “To Be” architecture
that enable the achievement of
business goals and outcomes.

Source: GAO analysis of NASA data.
                                              a
                                               A CRUD (create, read, update, and/or delete) matrix shows the specific business functions and
                                              applications that create, read, update, and/or delete specific data elements, which enables the
                                              organization to develop applications.
                                              b
                                               Business and operational rules define specific constraints for the data, such as security needs (e.g.,
                                              confidentiality and accessibility of data) and actions that should or should not occur, such as updating
                                              or deleting data.
                                              c
                                                The framework that NASA is using for architecture development does not identify a work product that
                                              supports the creation of business rules.
                                              d
                                               Although the framework that NASA is using identifies a logical database model as a work product, the
                                              available architecture products do not include such a model, and there was contradictory evidence in
                                              the architecture products that stated that NASA considered this model to be nonessential. As a result,
                                              it is unclear whether the agency plans to produce a logical database model as part of its architecture
                                              description.
                                              e
                                               Metadata is “data about data” that enables automation and consistent management and use of
                                              information, such as rules and standards.




                                              Page 42                                                 GAO-04-43 NASA’s Enterprise Architecture
Appendix II
Detailed Results of GAO’s Analyses of
Architecture Products Used to Date by NASA
to Acquire and Implement IFMP




f
The framework that NASA is using does not identify the metadata model as a type of work product nor
does the agency’s action plan address the development of a metadata model for later inclusion in the
architecture description.
g
 While the framework that NASA is using does identify the application portfolio as a type of work
product, the agency’s action plan does not address the development of this portfolio for later inclusion
in the architecture description.
h
 The technical reference model (TRM) describes how technology is supporting the delivery of service
components, including relevant standards for implementing the technology. The TRM is a generally
accepted representation of the generic components of an information system. It allows designers,
developers, and users to agree on definitions, have a common understanding of the services to be
provided, and identify and resolve issues affecting such requirements as interoperability, portability,
reliability, scalability, and serviceability.
i
  Examples of enterprise services include application services, such as Web services, and collaboration
services, such as instant messaging and video conferencing.
j
 Technical standards are strict rules and protocols governing how a given enterprise service is to be
implemented.
k
  NISSU was initially established to support the deployment of the IFMP core financial module.
However, NASA is now considering using this technical infrastructure to support the deployment of
other enterprise applications that have yet to be identified.




Page 43                                                 GAO-04-43 NASA’s Enterprise Architecture
                                               Appendix II
                                               Detailed Results of GAO’s Analyses of
                                               Architecture Products Used to Date by NASA
                                               to Acquire and Implement IFMP




Detailed Analysis of NASA’s Transition Plan Artifacts

                                          Element satisfied?
Key architectural element                Yes       No     Partially     Explanation of partially satisfied
Transition plan
Analysis of the gaps between the                    X
baseline and target architecture for
business processes,
information/data, and
services/application systems to
define missing and needed
capabilities.
A high-level strategya for
implementing the enterprise
architecture, including specific time-
phased milestones for acquiring and
deploying systems, performance
metrics, and financial and
nonfinancial resource needs.
This strategy should include:
• A listing of the legacy systems that                        X         The transition plan identifies only the core financial legacy
  will not be part of the “To Be”                                       systems that have been or will be retired. It does not identify all
  environment and the schedule for                                      legacy systems or provide a schedule for terminating these
  terminating these systems.                                            systems.
• A description of the training                               X         The transition plan provides a high-level description of a
  strategy/approach that will be                                        training strategy and references a change management
  implemented to address the                                            strategy for IFMP. It also identifies a business driver for
  changes made to the business                                          improving human capital management within the organization.
  operations (processes and
  systems) to promote operational                                       However, the architecture does not (1) address training needs
  efficiency and effectiveness. This                                    for nonfinancial business operations, (2) contain training plans,
  plan should also address any                                          (3) identify changes to existing policies and procedures, or
  changes to existing policies and                                      (4) estimate resource needs. Moreover, this generic strategy
  procedures affecting day-to-day                                       was developed without the benefit of a gap analysis.
  operations, as well as resource
  needs (staffing and funding).
• A list of the systems to be                                 X         The transition plan notes that there is a list of project
  developed/acquired/modified to                                        development initiatives, such as core financial and travel
  achieve business needs and a                                          manager, but it does not provide a complete list of systems to
  description of the relationship                                       be developed or acquired to achieve business needs (e.g.,
  between the system and the                                            human resources and budget).
  business need(s).
• A strategy for employing enterprise                         X         The transition plan contains a description of an EAI strategy for
  application integration (EAI) plans,                                  IFMP applications. However, the transition plan does not state
  methods, and tools to, for example,                                   whether this strategy would be applied to other agencywide
  provide for efficiently reusing                                       application systems.
  applications that already exist
  concurrent with adding new
  applications and databases.




                                               Page 44                                         GAO-04-43 NASA’s Enterprise Architecture
                                              Appendix II
                                              Detailed Results of GAO’s Analyses of
                                              Architecture Products Used to Date by NASA
                                              to Acquire and Implement IFMP




(Continued From Previous Page)
                                         Element satisfied?
Key architectural element               Yes        No       Partially        Explanation of partially satisfied
A technical migration plan (systems,
infrastructure, and data) that shows:
(a) the transition from legacy to                   X
    replacement systems with
    explicit “sunset” dates and
    intermediate systems that may
    be temporarily needed to
    sustain existing functionality
    during the transition period;
(b) an analysis of system                           X
    interdependencies, including
    the level of effort required to
    implement related systems in a
    sequenced portfolio of projects
    that includes milestones,
    timelines, costs, and
    capabilities; and
(c) a cost estimate for the initial                 X
    phase(s) of the transition and
    high-level cost projection for
    transition to the target
    architecture.
A description of the architecture                   X
governance and control structure
and the integrated procedures,
processes, and criteria (e.g.,
investment management, security)
to be followed to ensure that the
agency’s business transformation
effort remains compliant with the
architecture.
Source: GAO analysis of NASA data.
                                              a
                                               Acquisition/business strategy is a plan or action for achieving a specific goal or result through
                                              contracting for software products and services.




                                              Page 45                                                  GAO-04-43 NASA’s Enterprise Architecture
Appendix III

Assessment of NASA’s EA Management
Efforts against GAO’s Architecture
Management Maturity Framework                                                                                                            Appendx
                                                                                                                                               iI




Stage                   Core element                                 Satisfied?   Comments
Stage 1: EA awareness Agency is aware of EA.                         Yes          In December 2002, the CIO issued a
                                                                                  memorandum stating the agency’s intent to
                                                                                  develop and use an EA.
Stage 2: Building the   Adequate resources exist (funding,           Yes          According to the chief technology officer (CTO),
EA management           people, tools, and technology).                           the agency has adequate program funding. NASA
foundation                                                                        estimates that it will cost $750,000 to develop its
                                                                                  EA for fiscal years 2001 through 2003. Further, the
                                                                                  agency reports that it has skilled staff (government
                                                                                  employees and contractors) for its architecture
                                                                                  program. In addition, NASA is using automated
                                                                                  tools and technology, such as Rational Rose by
                                                                                  Rational Software Corporation/IBM Software
                                                                                  Group.
                        Committee or group representing the          No           NASA has not assigned responsibility for directing,
                        enterprise is responsible for directing,                  overseeing, or approving the EA to a committee or
                        overseeing, or approving the EA.                          group comprising representatives from across the
                                                                                  agency.
                        Program office responsible for EA            Yes          In December 2002, NASA established a program
                        development and maintenance exists.                       office that is responsible for EA development and
                                                                                  maintenance.
                        Chief architect exists.                      Yes          In January 2003, NASA designated the CTO as the
                                                                                  chief architect.
                        EA is being developed using a                Yes          The EA is being developed using the Zachman
                        framework, methodology, and automated                     framework. According to the CTO, the agency is
                        tool.                                                     also using a defined methodology to develop the
                                                                                  EA.a In addition, NASA is using automated tools,
                                                                                  such as Rational Rose by Rational Software
                                                                                  Corporation/IBM Software Group, to build the EA.
                        EA plans call for describing both the        Yes          According to the CTO, the plansb for the EA
                        “As Is” and the “To Be” environments of                   provide for describing both the “As Is” and the
                        the enterprise, as well as a sequencing                   “To Be” environments and a sequencing plan.
                        plan for transitioning from the “As Is” to
                        the “To Be.”
                        EA plans call for describing both the      Yes            According to the CTO, the plansb for the EA
                        “As Is” and the “To Be” environments in                   provide for describing both the “As Is” and the
                        terms of business, performance,                           “To Be” environments in terms of business,
                        information/data, application/service, and                performance, information/data, application/service,
                        technology.                                               and technology.
                        EA plans call for business, performance, Yes              According to the CTO, the plansb for the EA
                        information/data, application/service, and                provide for addressing security for the “As Is” and
                        technology descriptions to address                        “To Be” environments.
                        security.
Stage 2: Building the   EA plans call for developing metrics for     Yes          According to the CTO, the plansb for the EA
EA management           measuring EA progress, quality,                           provide for developing metrics to measure
foundation              compliance, and return on investment.                     progress, quality, compliance, and return on
                                                                                  investment.




                                                  Page 46                                   GAO-04-43 NASA’s Enterprise Architecture
                                              Appendix III
                                              Assessment of NASA’s EA Management
                                              Efforts against GAO’s Architecture
                                              Management Maturity Framework




(Continued From Previous Page)
Stage                    Core element                              Satisfied?      Comments
Stage 3: Developing      Written/approved organization policy      No              According to the CTO, the agency is revising its
EA products              exists for EA development.                                existing policy to require the development of an
(includes all elements                                                             EA. As written, the policy requires the CIO to
from stage 2)                                                                      develop an information technology (IT)
                                                                                   architecture, which is one aspect of an EA.
                         EA products are under configuration       No              According to the CTO, EA products are not
                         management.                                               currently under configuration management.
                         EA products describe or will describe      Yes            According to the CTO, the plansb for the EA
                         both the “As Is” and the “To Be”                          products provide for describing the “As Is” and the
                         environments of the enterprise, as well as                “To Be” environments, as well as a sequencing
                         a sequencing plan for transitioning from                  plan.
                         the “As Is” to the “To Be.”
                         Both the “As Is” and the “To Be”          Yes             According to the CTO, the plansb for the EA
                         environments are described or will be                     provide for describing both the “As Is” and “To Be”
                         described in terms of business,                           environments in terms of business, performance,
                         performance, information/data,                            information/data, application/service, and
                         application/service, and technology.                      technology.
                         Business, performance, information/data, Yes              According to the CTO, the plansb for the EA
                         application/service, and technology                       provide for the business, performance,
                         descriptions address or will address                      information/data, application/service, and
                         security.                                                 technology descriptions addressing security for the
                                                                                   “As Is” and “To Be” environments.
                         Progress against EA plans is measured     No              According to the CTO, the agency is measuring
                         and reported.                                             and reporting progress against EA plans; however,
                                                                                   NASA was unable to provide evidence of these
                                                                                   reports.
Stage 4: Completing      Written/approved organization policy      No              There is no written/approved policy for EA
EA products              exists for EA maintenance.                                maintenance.
(includes all elements   EA products and management processes No                   According to the CTO, management processes are
from stage 3)            undergo independent verification and                      independently verified and validated, but EA
                         validation.                                               products do not undergo independent verification
                                                                                   and validation. According to the CTO, EA products
                                                                                   will be subject to independent verification and
                                                                                   validation in the future.
                         EA products describe both the “As Is” and No              According to the CTO, the plansb for the EA
                         the “To Be” environments of the                           provide for describing both the “As Is” and the
                         enterprise, as well as a sequencing plan                  “To Be” environments of the enterprise, as well as
                         for transitioning from the                                a sequencing plan for transitioning from the “As Is”
                         “As Is” to the “To Be.”                                   to the “To Be.” However, the initial version of
                                                                                   NASA’s EA was not provided to us in time to
                                                                                   determine if its products address this core element.
                                                                                   Therefore, our analysis is based on the products
                                                                                   that were used to date to guide and constrain
                                                                                   IFMP.




                                              Page 47                                        GAO-04-43 NASA’s Enterprise Architecture
                                               Appendix III
                                               Assessment of NASA’s EA Management
                                               Efforts against GAO’s Architecture
                                               Management Maturity Framework




(Continued From Previous Page)
Stage                    Core element                               Satisfied?      Comments
Stage 4: Completing      Both the “As Is” and the                   No              According to the CTO, the plansb for the EA
EA products              “To Be” environments are described in                      provide for describing both the “As Is” and “To Be”
(includes all elements   terms of business, performance,                            environments in terms of business, performance,
from stage 3)            information/data, application/service, and                 information/data, application/service, and
                         technology.                                                technology. However, the initial version of NASA’s
                                                                                    EA was not provided to us in time to determine if its
                                                                                    products address this core element. Therefore, our
                                                                                    analysis is based on the products that were used
                                                                                    to date to guide and constrain IFMP.
                         Business, performance, information/data, No                According to the CTO, the plansb for the EA
                         application/service, and technology                        provide for the business, performance,
                         descriptions address security.                             information/data, application/service, and
                                                                                    technology descriptions addressing security.
                                                                                    However, the initial version of NASA’s EA was not
                                                                                    provided to us in time to determine if its products
                                                                                    address this core element. Therefore, our analysis
                                                                                    is based on the products that were used to date to
                                                                                    guide and constrain IFMP.
                         Organization CIO has approved current      No              The CIO approved the initial version of the EA.
                         version of EA.                                             However, the initial version of NASA’s EA was not
                                                                                    provided to us in time to determine if its products
                                                                                    address this core element. Therefore, our analysis
                                                                                    is based on the products that were used to date to
                                                                                    guide and constrain IFMP.
                         Committee or group representing the       No               According to the CTO, the plansb for the EA do not
                         enterprise or the investment review board                  provide for approval by a committee or group
                         has approved current version of EA.                        representing the enterprise or the investment
                                                                                    review board.
                         Quality of EA products is measured and     No              According to the CTO, the quality of EA products is
                         reported.                                                  not measured and reported.
Stage 5: Leveraging      Written/approved organization policy       No              There is no written/approved policy requiring IT
the EA for managing      exists for IT investment compliance with                   investment compliance with the EA. The current
change                   EA.                                                        policy requires the CIO to ensure that new IT
(includes all elements                                                              investments are in alignment with technology
from stage 4)                                                                       architectures, which are one aspect of an EA.
                                                                                    According to the CTO, this policy is being revised.
                         Process exists to formally manage EA       Yes             According to the CTO, there is a process for
                         change.                                                    formally managing EA change.
                         EA is integral component of IT investment No               Since the EA is currently being developed and has
                         management process.                                        not been used to date in acquiring and
                                                                                    implementing IFMP, it is not part of the investment
                                                                                    management process.
                         EA products are periodically updated.      Yes             According to NASA, it plans to update the EA
                                                                                    quarterly through June 2004, and semiannually
                                                                                    thereafter.




                                               Page 48                                        GAO-04-43 NASA’s Enterprise Architecture
                                                         Appendix III
                                                         Assessment of NASA’s EA Management
                                                         Efforts against GAO’s Architecture
                                                         Management Maturity Framework




(Continued From Previous Page)
Stage                            Core element                                   Satisfied?         Comments
Stage 5: Leveraging                  IT investments comply with EA.             No                 Since the EA is currently being developed and has
the EA for managing                                                                                not been used to date in acquiring and
change                                                                                             implementing IFMP, it is not part of the investment
(includes all elements                                                                             management process.
from stage 4)                        Organization head has approved current     No                 The organization head has not approved the
                                     version of EA.                                                current version of the EA.
                                     Return on EA investment is measured        No                 Metrics and processes for measuring EA benefits
                                     and reported.                                                 have not been developed, and an initial version of
                                                                                                   the EA has not been completed, thus precluding
                                                                                                   return-on-investment measurement.
                                     Compliance with EA is measured and         No                 Metrics for measuring EA compliance have not
                                     reported.                                                     been developed, and an initial version of the EA
                                                                                                   has not been completed, thus precluding
                                                                                                   measuring and reporting on compliance.
Source: GAO analysis of NASA data.
                                                         a
                                                          According to NASA, its methodology is from the following: Melissa A. Cook, Building Enterprise
                                                         Information Architectures: Reengineering Information Systems, Prentice Hall Inc. (1996).
                                                         b
                                                          We requested NASA’s architecture program plan, but NASA did not provide it.




                                                         Page 49                                               GAO-04-43 NASA’s Enterprise Architecture
Appendix IV

Comments from the National Aeronautics and
Space Administration                                                Appendx
                                                                          iIV




              Page 50        GAO-04-43 NASA’s Enterprise Architecture
Appendix IV
Comments from the National Aeronautics
and Space Administration




Page 51                                  GAO-04-43 NASA’s Enterprise Architecture
Appendix IV
Comments from the National Aeronautics
and Space Administration




Page 52                                  GAO-04-43 NASA’s Enterprise Architecture
Appendix IV
Comments from the National Aeronautics
and Space Administration




Page 53                                  GAO-04-43 NASA’s Enterprise Architecture
Appendix IV
Comments from the National Aeronautics
and Space Administration




Page 54                                  GAO-04-43 NASA’s Enterprise Architecture
Appendix IV
Comments from the National Aeronautics
and Space Administration




Page 55                                  GAO-04-43 NASA’s Enterprise Architecture
Appendix IV
Comments from the National Aeronautics
and Space Administration




Page 56                                  GAO-04-43 NASA’s Enterprise Architecture
Appendix IV
Comments from the National Aeronautics
and Space Administration




Page 57                                  GAO-04-43 NASA’s Enterprise Architecture
Appendix V

GAO Contacts and Staff Acknowledgments                                                           Append
                                                                                                      x
                                                                                                      i
                                                                                                      V




GAO Contact       Cynthia Jackson, (202) 512-5086



Acknowledgments   Staff making key contributions to this report were Sophia Harrison, Anh Le,
                  Randolph Tekeley, William Wadsworth, and Lillian Whren.




(310256)          Page 58                                 GAO-04-43 NASA’s Enterprise Architecture
GAO’s Mission            The General Accounting Office, the audit, evaluation and investigative arm of
                         Congress, exists to support Congress in meeting its constitutional responsibilities
                         and to help improve the performance and accountability of the federal government
                         for the American people. GAO examines the use of public funds; evaluates federal
                         programs and policies; and provides analyses, recommendations, and other
                         assistance to help Congress make informed oversight, policy, and funding
                         decisions. GAO’s commitment to good government is reflected in its core values of
                         accountability, integrity, and reliability.


Obtaining Copies of      The fastest and easiest way to obtain copies of GAO documents at no cost is
                         through the Internet. GAO’s Web site (www.gao.gov) contains abstracts and full-
GAO Reports and          text files of current reports and testimony and an expanding archive of older
                         products. The Web site features a search engine to help you locate documents
Testimony                using key words and phrases. You can print these documents in their entirety,
                         including charts and other graphics.
                         Each day, GAO issues a list of newly released reports, testimony, and
                         correspondence. GAO posts this list, known as “Today’s Reports,” on its Web site
                         daily. The list contains links to the full-text document files. To have GAO e-mail this
                         list to you every afternoon, go to www.gao.gov and select “Subscribe to
                         e-mail alerts” under the “Order GAO Products” heading.


Order by Mail or Phone   The first copy of each printed report is free. Additional copies are $2 each. A check
                         or money order should be made out to the Superintendent of Documents. GAO
                         also accepts VISA and Mastercard. Orders for 100 or more copies mailed to a single
                         address are discounted 25 percent. Orders should be sent to:
                         U.S. General Accounting Office
                         441 G Street NW, Room LM
                         Washington, D.C. 20548
                         To order by Phone:     Voice: (202) 512-6000
                                                TDD: (202) 512-2537
                                                Fax: (202) 512-6061


To Report Fraud,         Contact:
                         Web site: www.gao.gov/fraudnet/fraudnet.htm
Waste, and Abuse in      E-mail: fraudnet@gao.gov
Federal Programs         Automated answering system: (800) 424-5454 or (202) 512-7470



Public Affairs           Jeff Nelligan, Managing Director, NelliganJ@gao.gov (202) 512-4800
                         U.S. General Accounting Office, 441 G Street NW, Room 7149
                         Washington, D.C. 20548
United States                  Presorted Standard
General Accounting Office      Postage & Fees Paid
Washington, D.C. 20548-0001           GAO
                                 Permit No. GI00
Official Business
Penalty for Private Use $300
Address Service Requested