oversight

Embedded Computers: Navy's Approach to Developing Patrol Aircraft Avionics System Too Risky

Published by the Government Accountability Office on 1990-09-28.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                        United S t a t e s General Accouhtihg'Ofgic'e
                        Report to            Congresmms$^^^0j00:



September 1990
                        EMBEDDED
                        COMPUTERS
                                                                        to
                                >mg
                        Aircraft AMm^
                        System Too I I




(,A()/r(VITF;( »0 7?>
                   ^United. Stites':;;.:• X.;
GAO                 G^^6irafA(>coimtiit^
                    WaBhihgtoii, D.C. 2p{i48

                   Infohnation Management and
                   Technology Division

                   B-238826
                   September 28,1990

                   The Honorable John Ck)nyers, Jr.
                   Chairman, Legislation and National
                     Security Subcommittee
                   Govemment Operations Committee
                   House of Representatives
                    The Honorable Frank Horton
                    Ranking Minority Member, Legislation
                      and National Security Subconunittee
                    Govemment Operations Committee
                    House of Representatives

                    The Navy plans to buy complex avionics computer systems, and related
                    conununications equipment and sensors, for submarine patrol aircraft.
                    This acquisition, designated the Update IV Program, is intended to pro-
                    vide the Navy with the capability to locate, identify, and attack the
                    expected threat of more quiet submarines. Between September 1990 and
                    May 1993, the Navy plans to buy 28 of the avionics systems at a cost of
                    $496 million. Although its plans are uncertain, the Navy's total purchase
                    could be for up to 240 systems at a cost of about $2.1 billion.

                   This report responds to your offices' October 1989 request to review the
                   Update IV Program, and is part of your overall request to review the
                   Department of Defense's acquisition of computer systems embedded in
                   weapon systems. Our objectives were to determine whether (1) the Navy
                   plans to adequately test the avionics computer systems before buying
                   them, and (2) Navy management oversight of these computer systems
                   has occurred. A detailed discussion of our objectives, scope, and method-
                   ology is contained in appendix I.

                    The Navy is taking a high risk approach in acquiring a new and complex
Results in Brief    computer-based avionics system for its patrol aircraft. Although the
                    Navy originally planned to thoroughly test this system before buying
                    more than four, program delays led the Navy to p>ostpone complete
                    testing. This is clearly contrary to (1) Defense policies which, when fol-
                    lowed, should be effective In mitigating computer system development
                    risks and (2) the principle of "fly before you buy."




                    Paget                       OAO/IMTBC«0-Ttt The Navy'* Patrol Alraraft Avionica 8yatem
             B-238828




             Specifically, the Navy plans to:
             Continue developing software (i.e., coding) before it has approved the .
             design specifying what the software should do and how well it should do
             it;
             Buy 28 of the avionics systems before all testing is successfully com-
             pleted; and
             Use a model of one of the system's processors during testing that is not
             an accurate representation of the final version.
             The Navy is planning to follow this high risk approach because it
             believes that any further delays will cause it to missfixed-pricecontract
             option deadlines and increase contract costs. The Navy, however, has
             not prepared any detailed analysis to support its contention that con-
             tract costs will increase. In addition, the Navy's position fails to consider
             the costs of buying 28 systems that may not work as intended and may
             require expensivefixes,assuming they can befixed.As we previously
             reported, it can be six to ten times more costly to correct a software
             problem after a system in placed in operation than it is during early
             system development.

             We recognize that adhering to Defense policies might increase acquisi-
             tion costs. But possible cost increases do not justify spending almost
             $500 million on a system that has not been thoroughly tested. If the
             Navy finds that missing contract option deadlines wiU be prohibitively
             expensive, it must decide whether this avionics system is affordable.


Background   In February 1985, the Navy began the $2.1 billion Update IV Program to
             provide submarine patrol aircraft with modem avionics computer tech-
             nology and sensors. These long range, land-based patrol aircraft are
             deployed globally to find, identify, and attack new classes of very quiet
             enemy submarines. The Navy planned to install the new avionics com-
             puter system on 108 existing P-3 aircraft and an estimated 125 new P-7
             aircraft.' However, the Navy has since tenninated the P-7 aircraft pro-
             gram, and is now considering other altematives such as buying more P-3
             aircraft or reducing the number of avionics systems to be bought.
             The Update IV Program includes computer systems that process and dis-
             play sensor data, and control aircraft sensor, communication, naviga-
             tion, and armament subsystems. In 1987, the Navy awarded an Update
             ' The remaining 7 of the 240 total avionics systems are for engineering development modeling.


             Pagea                             OAO/IMraOMKT* Tha Nav)r*a Patrol Aircraft Avionica Syataa
                                         B-238826




                                         IV system integration contract having multiple contract production
                                         options, including an initial four that are fixed-price. The system's
                                         software is currently being developed (i.e., coded) and undergoing early
                                         laboratory testing. The first contract option deadline is September 1990.
                                         Table 1 shows development milestones, the first three contract option
                                         quantities, and funding requirements.

Table 1: Development Milestones, First
Three Contract Options, and Funding      Dollars in millions
Requirements                                                                    Funding Requirements*
                                         Milestones               Quantity      Production     Support                                     Total
                                         Option 1
                                           September
                                           1990                            4           $68.3            $21.2                              $89.5
                                         Option 2
                                           May 1992                       12          $153.9            $41.1                          $ '95.0
                                         Option 3
                                           April 1993                    12          $174.3             $37.6                          $211.9
                                         Totals                          28          $396.5             $99.9                          $496.4
                                         "Funding requirements have not been revised to reflect anynear-term impact of terminating the P-7
                                         program.



                                         The Update IV systen^ integration contract requires the contractor to
Navy Did Not Specify                     prepare detailed fimctional and performance specifications for each
Software                                 software subsystem. According to Defense software development
Requirements Before                      policy,2 such specifications are necessary to establish a requirements
                                         baseline for detailed software design and development. Contrary to this
Development                              policy, however, the Navy is allowing the Update IV contractor to
                                         develop software before subsystem specifications are completed and
                                         approved. The Defense Department has reported'' and we agree that
                                         failure to define complete specifications before developing software may
                                         not only jeopardize software quality, but can also increase development
                                         costs and delay project completion. This concern was raised in a risk
                                         analysis prepared by the Navy laboratory monitoring the contractor's
                                         performance. As stated in the analysis, developing the software that
                                         will perform some of the system's mission functions (e.g., communica-
                                         tion and navigation processing, nonacoustic sensor management, search
                                         stores and weapf ^ns management, and inflight performance monitoring)
                                         at the san>e time that software requirements are being specified affords

                                          ^Military Standard 2167A, Defense System Software Development. Kebniary 1988.
                                         "Proceedlinas of the Joint Loglstlgt Commanders .lolnt Policy Coordinating Oro\ip oi\ Ctm>putcr
                                         Resource Management (Aug~l979).



                                          Pages                             GAO/IMTBOBO-79 The Navy'a Patrol Aircraft Avinnim Syatem
                        B-238826




                        minimal time for much-needed design validation and greatly increases
                        the risk that the software will be unacceptable.

                        As of June 1990, the Navy had not approved most subsystem specifica-
                        tions for the Update IV avionics system, yet the contractor had written
                        about 80 percent of the software and had started testing to determine
                        whether individual units of code performed their functions. Although
                        draft specifications exist, the Navy has approved or conditiona':
                        accepted only a third of th'^m because the remaining two thirds either do
                        not meet the requirements of the system level specifications or have yet
                        to be evaluated. Consequently, the avionics software is being coded
                        against a moving baseline, and the Navy currently lacks firm criteria for
                        testing and contract acceptance. This has not only resulted in an esti-
                        mated 3-month delay in software testing, but more importantly could
                        result in (1) additional coding costs once the software design is finalized
                        or (2) a system which does not meet all requirements set forth in the
                        system specification.

                        We reviewed the status of the Update IV subsystem specifications as of
                        June 1990, and found that the Navy had approved only 3 of 93 specifi-
                        cations^. Of the remaining 90 specifications, 30 had received conditional
                        approval pending incorporation of minor comments, 17 were unaccept-
                        able and rejected because they did not meet requirements in the overall
                        system specification, and 43 were still under review. Examples of
                        rejected specifications include those for the taciicai mission subsystem^
                        and the systems management software,"* without either of which the avi-
                        onics system could not accomplish the antisubmarine warfare mission
                        objective. Although approval of all subsystem specifications was
                        targeted for August 1990, program officials stated that approval will
                        not occur before September 1990.


                        The Navy's planned testing of the Update IV Program will not provide
Adequate Testing Will   reasonable assurance that this computer-based avionics system is ready
Not Occur Prior to      for full rate production. Testing validates that a system meets its func-
Production Decisions    tional and performance requirements and can effectively perform the
                        intended mission. For the Update IV Program, testing is particularly


                        ''This sutmyHtem provides environmental analysis aldi, correlation, tactical execiitlnn, taitlcal and
                        nonacoustic situation management. cla.<tsincattnn, and tactical planning aids.
                        ''This software is reflponslble for recording, data baae management, displays and controls, and real-
                        time executive management functions.



                        Page 4                             OAO/IMTB&80-79 The Navy's Patrol Aircraft Avionica Byateiii
B-238826




important becSiuse the system's software is estimated at about one mil-
lion lines of Ada code," and the Navy laboratory monitoring the Update
IV Program considers Ada's use to be a high risk because relatively little
experience has been gained with Ada in stringent, real-time environ-
ments.^ Additionally, this software performs critical mission functions.
For example, the avionics system's software includes sophisticated algo-
rithms," such as multisensor target tracking and decision aids, to help
crew operators work with the massive number of sensor data inputs
associated with locating, tracking, and targeting submarines. The cor-
rectness and effectiveness of such complex software systems is ascer-
tained by thorough testing. Failure to conduct rigorous testing greatly
increases the possibility of producing and deploying a system that fails
to meet its mission requirements.

Computer system testing is incremental and can be viewed as having
two major components—developmental .;esting and operational testing.
Generally, the purpose of developmental testing is to determine whether
a system meets the requirements in its specifications, while operational
testing determines whether a system that has been designed according
to its specifications meets mission requirements. More specifically, early
developmental tests focus on whether individual software modules per-
form as required in the specification. Later developmental testing
addresses whether and how well the integrated modules perform
required functions (i.e., how fast, how reliable, how accurate, how
often) in a laboratory that realistically simulates an operational setting.
Following this laboratory integration testing, the complete system is
tested with actual users in a true operational setting. This operational
testing is sometimes conducted in two phases, with the first phase
showing the system's "potential" mission effectiveness and justifying
initial rate production quantities, and the second phase demonstrating
the system's ability to meet the mission requirement and justifying full
rate production. The above described testing progression emphasizes the
benefits of finding problems early in the development process, when
they are cheaper to correct.



"Ada Is a relatively new high-order language designed for use In real-time computer systems.

''Such systems must be able to obtain data from an activity or process, perform computations, and
respond quickly enough to affect the outcome of that activity or process. Depending on the applica-
tion, a response may be required In seconds or tn milliseconds. Aircraft avionics usc real-time com-
puter systems.

"Well-defined steps for solving a problem.



PageB                              OAO/IMTBC-90-79 The Navy's Patrol Aircraft Avto.nlca System
                             B-238826




   f""^


Developmental Testing        The contractor is responsible for developing two software integration
                             laboratories, one for testing the avionics system's acoustic capabilities
Hampered by Contractor       and one for testing its nonacoustic capabilities. Tnese laboratories
Laboratory Test Facilities   include computer hardware and software that simulate the avionics
                             system's operational conditions and provide an environment for testing
                             the system.

                             The contractor is currently more than 1 year late in developing the two
                             laboratories. As a result, the extent of laboratory testing that can be
                             performed before the first production options expire has been reduced
                             considerably. Moreover, the Navy's technical review of the contractor's
                             laboratory facilities has raised some doubt as to whether the laborato-
                             ries' simulation programs accurately simulate the mission environment.
                             For example, the review questions whether the laboratory simulation
                             includes a realistic number of targets.


Operational Testing Will     Navy pohcy** requires full operational testing before a system enters full
                             rate production. Additionally, the National Defense Authorization Act
Not Be Completed Before      for Fiscal Years 1990 and 1991, dated November 29,1989, (P.L. 101-
Systems Are Bought           189) defines low-rate initial production of new weapons systems as the
                             minimum quantity necessary to (1) provide production-configured or
                             representative articles for operational tests, (2) establish an initial pro-
                             duction base for the system, and (3) permit an orderly increase in the
                             production rate sufficient to lead to full rate production upon successful
                             completion of operational testing. Navy policy'" uses a similar definition.
                             Consistent with these policies and definitions, the Navy initially planned
                             to (1) approve what it called limited rate production in April 1991 for
                             four avionics systems (i.e., exercise the first contract option), (2) com-
                             plete lull operational testing in Febmary 1992, and (3) approve full rate
                             production in April 1992 to buy as many as 24 more systems (i.e., exer-
                             cise second and third contract options).

                             The Navy revised this initial plan in response to contractor delays in
                             developing the system and govemment delays in providing certain
                             required government-furnished equipment." Under the revised plan, the
                             Navy has delayed full operational testing by more than 2 years until the
                             "Chief of Naval Operations Instruction 3960. lOC, Teat and Evaluation, September 1987.
                             '"Chief of Naval Operations Instruction 5000.42C, Reaearch, Development and Acquisition Proce-
                             dures, May 1986.
                             '' An enhanced acoustic processor Is being separately developed by the Navy and will be provided to
                             the Update IV contractor for Integration with other avionics system components.



                             Page A                            OAO/IMTBGM-Tg The Navy's Patrol Aircraft Avionica Syatoti
first quarter of 1994, but still plans to buy the Initial four systems in
September 1990 (now calling this pilot production instead of limited rate
production) and the additional 24 between May 1992 and April 1993
(now calling this limited rate instead of full rate production). This
means that the Navy plans to buy as many as 28 systems before it
knows whether the system can satisfy the mission requirement. When
questioned. Navy program and oversight officials stated that their pri-
mary reason for increasing their initial rate quantities from four to 28
was the need to exercise favorable, fixed-price contract options on time.
These officials further stated that under different contractual circum-
stances, they would not buy so many systems before completing full
operational testing. In our opinion, the Navy's decision to expand initial
rate production to 28 systems is inconsistent with Navy policy to hold
limited rate production at minimum levels before completing full opera-
tional testing, and consequently creates an unacceptable risk of buying a
large number of systems that may not be mission effective.

We do not support the Navy's revised plan, and are concemed that
without the results of full operational testing, the Navy will not have
the information it needs to make a pmdent management decision on
whether to buy more than the initial four systems. Moreover, given that
the revised plan calls for delivery of twelve avionics systems to an oper-
ational squadron at the same time operational testing is occurring, the
Navy risks sending systems to the field that do not work.

Navy officials also contend that they need to exercise production
options on schedule to maintain price guarantees, adding that costs
could increase significantly if the contract is renegotiated. However,
their estimates of cost increases are not supported by detailed analysis.
Although the Navy began an analysis to better estimate the effect of
contract renegotiation on costs, this analysis was not completed because
of difficulties in obtaining requisite data from the contractor. Addition-
ally, the Navy's estimates fail to recognize the substantial time and
expense that could be incurred if an avionics system is delivered to the
fleet with extensive hardware and software problems. As we previously
reported, "i it can be six to ten times more costly to correct a software
discrepancy after a system is placed in operation than it is during early
system development. Also, even though the Navy program manager
stated that it would be difficult to renegotiate the contract to allow the
Navy to exercise production options after full operational testing, the

' 'Embedded Computers; Navy Not Ready to Buy Avionics Computcns for Ita LAMPS Mic I Helicopters
(dAO/lMTEC-9rf54.May3riBWy



Psg«7                           OAO/IMTBOM>'79 The Navy's Patrol Aircraft Avioiilca Syatam
                      B-238826




                      Navy has recently negotiated extending both the third and fourth pro-
                      duction options in the Update IV contract by 6 months at no additional
                      cost to the govemment.

                      Last, program officials stated that their current plan allows for ade-
                      quate testing before making production decisions, noting that they have
                      extended the initial phases of planned operational testing by several
                      months. However, this extended testing will still not support a full rate
                      production decision because it will only assess "potential" system effec-
                      tiveness. According to Navy policy, such testing only provides an ade-
                      quate basis for producing limited rate production quantities. It does not
                      constitute full operational testing, which is necessary to demonstrate
                      system mission effectiveness and is required to approve full rate pro-
                      duction. Moreover, this extension of initial operational testing will use a
                      model of one of the avionics system processors that is not an accurate
                      representation of the version of this processor to be produced (see next
                      section).


Processor Model Not   An enhanced acoustic processor" is being separately developed and will
Representative        be given to the Update IV contractor for integration with the other avi-
                      onics system components. Originally, the Navy was to provide the
                      processor in April 1990. However, hardware and software problems
                      have delayed the processor's development, causing its delivery to the
                      Update IV contractor to slip to November 1990. According to program
                      officials, acoustic processing is the most critical function that the
                      Update IV Program will perform.

                      Because of delays in developing this processor, initial operational testing
                      to determine whether to exercise the second contract option will use a
                      Navy-provided model of the processor. The validity of the initial opera-
                      tional testing then will depend on how accurately the model simulates
                      the acoustic processor. However, the model the Navy has provided the
                      contractor has less functionality and performance capability than the
                      final version of the processor, and thus is not representative. For
                      example, this model can accept only about one third of the data inputs
                      to the processor. According to the Navy's 1986 risk assessment for the
                      Update IV Program, use of this model is a high risk.


                      '^Thls processor, designated the Enhanced Modular Signal Processor, analyses daU from acoustic
                      sensors to determine the location and Identity of enemy submarines. It otten Improved acoustic
                      processing, Including more Ihpttt/biiitput chaniwis, prooBsslr^ power, and operator aids.



                      Pigil                           ^gihti/nrh^^         TIM Ntvy't Pttnl AUrenft AyHmtai 8k««Mi
Pi


                    8^238826




                    Within the Department of Defense, the management level responsible
Program Oversight   for overseeing a system acquisition is generally determined by how
Has Occurred        much the system costs. For the Update IV Program, the Office of the
                    Secretary of Defense has delegated oversight to the Office of the Secre-
                    tary of the Navy. According to an Office of the Secretary of Defense
                    official, the Update IV Program was delegated because the number of
                    programs the Office of the Secretary of Defense oversees had to be
                    reduced to a manageable level.

                    In light of the importance of computer technology to the Update IV Pro-
                    gram, Navy 'oversight authorities api>ear to have focused on this tech-
                    nology in tht.r program reviews. We found briefing documents involving
                    these oversight authorities and the program office which show that
                    such issues as completion of adequate contractor laboratory test facili-
                    ties, completion of the enhanced acoustic processor, and status of
                    software development have received attention. Additionally, the pro-
                    gram office's current approach to developing the Update IV Program
                    has been reviewed and approved by the Navy's Program Executive
                    Officer (Acquisitions). Office of the Secretary of the Navy review of this
                    plan was scheduled for late August 1990, but has been delayed for sev-
                    eral months. Should this final approval be granted, however. Navy over-
                    sight authorities will be allowing the continued development of software
                    without approved specifications and the planned purchase of 28 sys-
                    tems before full operational testing is completed. Thus, these authorities
                    will not be acting to ensure adherence to Navy software development
                    policies as discussed earlier. According to an official in the Office of the
                    Assistant Chief of Naval Operations (Air Warfare), this software devel-
                    opment and testing approach has thus far been approved because they
                    believe that program costs will significantly increase and jeopardize the
                    program if contract production options are not exercised on time.


                    The Navy is faced with a difficult decision in acquiring a new, techni-
Conclusions         cally challenging avionics computer system for its patrol aircraft. If the
                    Navy proceeds according to its current acquisition plan, it will be unable
                    to perform thorough operational testing, and could therefore buy and
                    deploy an expensive system that does not meet mission requirements.
                    On the other hand, If the Navy renegotiates the contract to delay pro-
                    duction decisions until It has tested the system and assures that it meets
                    operational requirements, acquisition costs might increase. The Navy
                    has done no detailed analysis, however, to assess the extent of these
                    potential cost increaises.



                    figil                     J(}A0/Iiifi64l>>r*;t)to iiiyy'i^ Patiml AlMMt AVUMUnSyatMiii
                  B-238826




                  The Navy plans to (1) continue developing software (i.e., coding) before
                  approving detailed requirements for the system, (2) exercise contract
                  options on time and assume an unknown level of risk that the system it
                  buys may not work, and (3) use a model of an enhanced acoustic
                  processor during testing that is not representative of the final version.
                  In our opinion, the Navy's current acquisition approach is unacceptable,
                  causing the Navy to spend almost $500 million on a system that may not
                  meet mission requirements.


                  We recommend that the Secretary of the Navy direct the Commander,
Recommendations   Naval Air Systems Command to (1) halt further software development
                  (i.e., coding) until system specifications are approved, (2) thoroughly
                  justify the need for initial rate production to exceed the four systems
                  originally planned, (3) conduct*initial operational testing using actual
                  system components or accurate simulations of them, and (4) conduct full
                  operational testing before making a full rate production decision. In light
                  of the possibility that this may preclude the Navy from exercising
                  existing, fixed-price contract options, we also recommend that the Secre-
                  tary direct the Commander to thoroughly analyze the cost impact of
                  contract renegotiation, and based on this analysis decide whether the
                  entire Update IV Program is financially viable.


                  As requested by your offices, we did not obtain official agency com-
                  ments on a draft of this report. However, we discussed its contents with
                  Navy and Office of the Secretary of Defense officials, and have incorpo-
                  rated their comments where appropriate. Our work was performed
                  between December 1989 and July 1990, in accordance with generally
                  accepted government auditing standards.

                  As arranged with your offices, unless you publicly announce the
                  report's contents earlier, we plan no further distribution until 30 days
                  from the date of this letter. At that time, we will send copies to the
                  Chairman, Senate and House Appropriations Committees; the Secre-
                  taries of Defense and Navy; and to other interested parties. We will also
                  make copies available to others upon request. This report was prepared
                  under the direction of Samuel W. Bowlin, Director, Defense and Security




                  Page 10                  aA0/IMn'BC4M)-T9 The Navy's Patrol Aircraft Avionics SystiamI
     Information Systems, who can be reached at (202) 276-4649. Other
     major contributors are Usted in appendix II.


                             muiu^
^    Ralph V. Carione
    ^Assistant Comptroller General




      Pagan                  OAO/DNTKOM-Tt The Navy's Patrol Aircraft Avtonlea Syatam
Contents


Letter                                                                                                 1

Appendix I                                                                                            14
Objectives, Scope, and
Methodology
Appendix II                                                                                           16
Major Contributors to
This Report
Table                    Table 1: Development Milestones, First Three Contract
                             Options, and Funding Requirements




                         Abbreviations

                         (iAO      General Accounting Office
                         mrKC-     Infonnation Management and Technology Division


                         Page IS                 OAO/IMTBOW-79 The Navy's Patrol Aircraft Avionica System
Appendix I

Objectives, Scope, and Methodology


               In October 1989, the Subcommittee on Legislation and National Security,
               House Committee on Govemment Operations, expressed interest in the
               Navy's plans to acquire embedded computer systems for selected anti-
               submarine warfare systems, and asked that we determine whether (1)
               the Navy's Update IV Program calls for adequate testing of a new avi-
               onics computer system for the P-3 and P-7 aircraft before they are
               bought, and (2) Navy management is overseeing the development of
               these embedded systems. This request relates to an overall request from
               the Chairman and the Subcommittee's Ranking Minority Member to
               review computer systems that are embedded in Defense weapon
               systems.

               To accomplish our objectives, we reviewed Defense and Navy instruc-
               tions and standards goveming the development, testing, and manage-
               ment oversight of embedded computer systems. We also reviewed
               Update IV documentation (e.g., acquisition plan, test plans, schedules,
               and funding requirements) as well as the development/production con-
               tract and related documents for the avionics program. Additionally, we
               interviewed both program officials responsible for managing software
               development and laboratory officials responsible for monitoring con-
               tractor performance. We also interviewed officials at the Navy test
               activities participating in development and operational testing, and
               toured contractor software development facilities.

               Further, we interviewed officials in the Office of the Secretary of
               Defense and the Chief of Naval Operations responsible for program
               oversight, and reviewed documentation associated with the discharge of
               this oversight responsibility. This work focused on whether and to what
               extent this oversight specifically addressed the embedded avionics com-
               puter system.

               We performed our work between December 1989 and July 1990, prima-
               rily at the Update IV program office within the Naval Air Systems Com-
               mand, Arlington, Virginia, and the Naval Air Development Center,
               Warminster, Pennsylvania. We also visited the Naval Air Test Center,
               Patuxent River, Maryland; the Navy's Operational Test and Evaluation
               Force, Norfolk, Virginia; and the contractor's system development facili-
               ties in Seattle, Washington.

               As requested by the Chairman's office, we did not obtain official agency
               comments on a draft of the report, However, we discussed Its contents
               with Navy and Office of the Secretary of Defense officials, and have
                Incorporated their c o m m e n t s w h e r e appropriate. We c o n d u c t e d o u r



               Page 14                        OAO/IMTB080.T9 The Navy's Patrol Aircraft Avionics Syatem
 Appendix!
 Objectives, Scope, and Methodology




 review in accordance with generally accepted govemment auditing'
 standards.




                                                  • -.•-'             iy->:;.':..„t'i
Page IS                      OAO/miECM^ro Hw Navy'e PMiM Alraiift Avioaia SlyBtaB
Appendix II

Mgjor Contributors to This Report


                        John B. Stephenson, Assistant Director
Information             Randolph C. Hite, Assignment Manager
Management and
Technology Division,
Washington, D.C.

                        Norman C. Berman, Evaluator-in-Charge
Philadelphia Regional   Amy Ganulin, Evaluator
Office




iltMii)                 Pagaie                 vGAO/lirrBbWTfinM   Navy's Patrol AlTevaft AvIbnieaitKsM* i
    ^   ^




            Ordering Information
                                             -f^             %>}'




            The first five copies of each GAO report are free: Additional
            copies are $2 each. Ordef's should be sent tp the following
            address, accompanied by a check or*noney order made out to
            the Superintendent of Documerits, when necessary. Orders for
            100 or more copies to be mailed to a single addressi are
            discounted 25 percent.

            U.S. General Accounting Office
            P.O. Box 6015
            Gaithersburg, MD 20877           .

            Orders may also be placed by calling (202) 275-6241.

    I




N

                                                         \          - -



              V
  United States                           First-Class Mail
^ General Accounting Office             Postage & Fees Paid
  Washington, D.C. 20548                        GAO
 '                      •   '   .

                                          Permit No. GlOO
     Official Business
     Penalty for Private Use $300




                                    \
                                               ^




       v^