oversight

The Department of Agriculture's Allegation Concerning the General Services Administration's Mandatory Requirements Contract for ITEL Disk Drives

Published by the Government Accountability Office on 1977-06-20.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

 The Honorable      Jack Brooks,
   --L ft-giy,an--Governmen
-Comml
                                  Chairman
                             t Op15td t iOi?S             - --- - -----                               _
 Souse of Rcpcesentatives
 Dear    Mr.   Cha icman:
         This report         is    in response
                                          to an infcrmal       request     from
 your    off ice for      our      assistance
                                        in resolvin!      a dispute       among
 the Department    of                   the General Services
                                Agriculture,                           Admini-
 stration    (GSA) and the ITEL Data Products Cocpocation.                   The
 disDute   concerned an allegation        by the Department         that   disk
 drives   (Model 7338) acquired      from ITEL thcough a GSA manda-
 tocy requirements    contract    wece causing       degradation       of set-
 vice at the Department’s      h’ashington     Computer     Center.

        Kc had p:eviously             reviewed      the Department’s            proposal
 to lease     44 double        density      disk drives         to :eplace        80 I!%
 single    density      disk drives          that   had been leased            on a solc-
 source    basis     foe the Centc. . On April                  16, 1976, we reported
 (LCD-76-120)        that    the Denartment           had not justified            the need
 for double      density       disk b:ives        and that        sinale    density      disk
 drives    available       from a GSA mandatcry               requiiements         contract
 would meet the Center’s                storage needs and save about $339,000
 annually.       tie pcoposed,          and the Department            agreed,      tha:     t$e
 IBM disk drives          be replaced          by those available            from ITEL
 undec the mandatory             requirements         contract.

        Fe have reviewed      the allegation,       with      tecnnical        assjs-
 tance    from the Federal      Computer:   Performance         Evaluation        and
 Simulation    Center    (FEDSX;II),   and discussed        our ccncl usions
 witn   your staff,     GSA, and Agciculture        officials.           As a result,
 the Pepactment      hc? agreed      to complete    its cor.tcaet          with     TTET.,.

         A summacy      of    our    work   follows.
          Tne GSA contract       involved      is GS-OOC-50022.      It was compe-
 titively     awarded     to ITEL on February        5, 1974, and is renewable
 annua1l.y    through     fiscal     year 1979.     The tontract     is for ITEL
 Kodel 7330 disk drives            that   are replacements       for IBM Hodel 3330
 disk drives.         Government      agencies    who have cequicements      foe the


                                                                           LCD-77-115
                                                                      ED
                                                                                                  .
        Y         +



                                 IBM type disk drives        generally    must use the iTEL contract
                                 as their      source  of supply.      3t the presenr     time there     ace
                                 about     3,000 of the ITEL drives       in use throughout       the Govern-
                                 merit.     These drives   cost    the Government     about   $i5 million
                                 annually.
                                  c       The Department          ordered      80 ITEL disk drives            from the
                                 requirements           contract      in April     1976 to replace          80 IBM
__--.   - --...           -   -- drives
                                  -_ _-._-_- that
                                              _.-    were being         used _with
                                                                                _     the Center’s        IBM 370/168 -- - ___. _____.
                                 computer        system.        The ocdec‘ was -for 10 lots -of 8-disk-dci’ves
                                 each to be delivered               in two-let      increments       with     initial
                                 delivery        in May 1976.           The Department       accepted       the first
                                 two lots        in June 1976 and bega- testing                 lots     3 and 4.
                                 Due ing that         test in!,     the Center      encountered       problems          with
                                 the drives         and saia that          they caused degradation              of service
                                 provided        by the I3!! system.           Those Fcotrlems       were resolved            by
                                 ITEL,     but the Centec did not contractually                      accept       lots      3
                                 and 4. The Department                contended     that   they continued             to
                                 degrade       service.

                                        On February        22, 1977,          the Department            notified      ITEL
                                 that    it was curtailing           further         installation           of the ITEL
                                 drives     bxause     the drives           wece degrading             service    more
                                 than the IBM drives            that      the Center         continued         to use.
                                 The Depactment        claimed       th-:       the degradation             was in breach
                                 of the contract         provisio:          that     required        the ITEL drives
                                 to perfccm       equal to oc better               than the IBM drives              and
                                 provided      ITEL with      a study         to support          its claim.        The
                                 study,     which was made in December 1976 and amended in
                                 February      1977, compared          the performance               of the two types
                                 01 drives       using performance              data compiled           during    the
                                 period     June 1976 through            January        1977.

                                         The Department             immediately         brought     the issue of a
                                 Possible       breach      of contract           to the attention           of GSA for
                                 resolution.           On March 4, 1977, the contracting                         officer
                                 issued      his fikings           and determinations              concerning          the
                                 issue.        He concluded           that     (1) ITEL was not in breach                   of the
                                 contract       and (2) the continued                 refusal      by the Depactment
                                 to cecisce        its     installed         IBM disk drives          with     ITEL drives
                                 constituted         a breach        of the contract            by the Government             and
                                 a violation         of Federal          Pcocucement        Regulations.             The Depart-
                                 zent accested          trre finding           concerning       ITEL’s     alleged        breach
                                 of contract         but refused           to install       the remaining            ITEL
                                 drives      because      of its continuing              contgntion        that      the drives
                                 degraded       system ?erfornance.

                                        As a result   of the impasse    reached between GSA,                             the         i
                                 Department,    and ITEL,   your off ice was asKed by ITEL


                                                                                 2


                      -
                        officials        to resolve          the issue.         Subsequently,             we met with
                        your     staff     and the par ties           involved       to discuss           how the issue
                        should       be resolved.          The parties         agreed    that we should               contact
                        other      users of the ITEL drives                  to ascertain           their      experience
                        with     them.      They also agreed              to ou:    using       FEDSIM tQ evaluate
                        the D&artment’s             disk drove          study     and to abide by our findings
                        and conclusions.
_.~_ ..._.__. ..---_.   _. _- ____-._-.-. _.-                                     - _ __- _._                                .__.._._ _ - - - -. --
                                 We contracted         rith       FEDSIW to evaluate              the     depart$nt’s
                        study.         FEDSI.” (1) reviewed             the study      to determine             if its
                        assumptions         and methodology             ware valid,        (2) verified            the
                        e~curacy of the study data,                     (3) assessed          the performance
                        differences         between        the ITEL and IBly disk               drives       to determine
                        if the alleged           performance          degcadatioh        existed          and (4) pre-
                        paced a final          project         report     (see enclosure)             summarizing          its
                        findings        and conclusions.              In bc ief , FEDSIH concluded:
                                    (1)      The conclusions       in the        study          ace    based      on the    ce-
                        suits         of    inappropriate    statistical            methods            and     therefore     ace
                        inval id.
                                    (2)       The use of certain methods              of        averaging         data    to
                        compare             the IBM and ITEL disk drives                   is    inappropriate            and
                        mislead           ing.
                                  (3)  Althcugh      somo.of    the performance      measures   were
                        correctly       reported,     one was inconsistent        with   the Center’s
                        definition        of that.18easure      and others    were (a) meaningless
                        as performance          measures,    (b) inconsequent     ial,   (c) inval idly
                        computed      or (d) erroneous.

                                (4)          Most   of the failures    attributed                     to   ITEL    disk    drives
                        occurred           during     the ITEL acceptance       tests.
                               (5)         Such factors      as acceptance   test  procedures,      the
                        placement           of system     packs and the Center‘s      air conditioning
                        may have           adversely    affected    ITEL disk drives’     performance.
                                (6)    Neither    the study   nor the Center’s                             manual logs
                        supFact     the conclusion      that  ITEL disk drives’                             performance
                        is inferior       to that    of IBM disk drives.
                              We contacted             a number of other    Government      users of
                        ITEL  disk drives             and were told   by some that     performance
                        pcokdems   existed.             However,  they felt   that   their     problems




                                                                         3
           _..-




    f             ‘




                              may have been caused by the unique                    environmental        chacacter-
                              istics     of theic     installations.          Other      users were pleased          with
                              the ITEL pecfocmance             and some stated         that   the ITEL drives
                              performed     as well as, if not better                than,    the IBM disk
                              drives.      The consensus          of the usecs contacted             was that     the
                              ooerating     Environment          greatly    impacts      on the performance          of
                              disk drives      and that        a side-by-side        comparison        of the ITEL
- _.- .._. . ---_. _.-.--_.   dc ives xi-t-h -&h-g- I-BM dc i.cs -using         identical      ..wot&loads    and. _ ___-_______
                              operating     conditions        would be needed to msasuce their
                              respective      performance.

                                      On June 6, 1977, we met with             your staff      and the parties
                              involved     to discuss     our findings        and conclusions.           At that
                              meeting    the Department        admitted     that    its study      was invalid.
                              It agreed to contractually             accept    the disk drives         delivered
                              as lots    3 and 4, which the Center             was using     on a conditional
                              basis,    and to install       the remaining        ITEL drives       that    wece
                              ordered.       The Department      and ITEL also agreed            to negotiate
                              any payments      due ITEL as a result           of this    dispute.         These
                              agreements      should   resolve     the issue to the satisfaction                 of
                              IT&L and th? Government.
                        .
                                      We will  be pleased        to    assist     you   further      in   this   matter
                              if   you so desire.

                                                                  Sincerely        yours,



                                                                  of    ihe     Wited       States
                              Enclosure
                                                                        ..-_ .            - -.-       ..-                    --
       t        1


                    *   .

                                                                         ..                                                  .
                                                              .




-   .-. -_._ ~--. ...- .-_- -_-__.   --   ----_‘-.-.   -_.-       _._     .      _          _     -
                                                                                                            .       --   -        -   _   .   __   ____       _       -..-._   -__.   -_-




                                                                                                                .




                                                        AY-of7-062~GAO                                 ,
                                                                                                                                                                  .




                                                                  PAY 1977




                                                                                                       Washington,                                        D. C.


                                                                                     .’
_   -.   ._.-_-   ..-.   ..-- __..- ---_




                                                              PRE?X?ZD FOR:

                                                   US    GZXS?AL ACCOUiZTil:G OFFICE
                                                          WhSliXNGTOS, DC 20548
                               .
                               .
                                   ??3JECT ?sRso:\3EL:                          APPROVED   '
                                                                          .


                                   Dorr!?dti R. Deem
                                   Thon%s J. Tesso, Jr
                                   Bobby L. !-:c:iemie

                                              .


                                                    TE3EFS.L Cr):tPt'TER PZRFO%KNCZ            ,
                                                  EVXXkTiC1! 23 SI!XW.TIOh' CEXTER              I

                                                        h'AS*iIi:GTO'
                                                            *        r;, DC 20330
                                                     .                             .*
                                                                                  /
                                                                                      .
                                                   .




                                                                                  .
                             This Report is based oh the Federel                Cceputer Perfozzance
                     _ Eveluatfm        e-.2 Simlati-3        Center’s   (FZDSIE;’ s) detailed
                       asulysfs       05 a report      an,G Cata Frovi&d       by the Waskagton
-   -                  computer
        -. ._ ---------results-~cress center   (KC)    ,  US  Depar*%ent    of  k;ricaltxe.         The
                                              XE&?:4!s-.te..c!!iral      walua-,ioq       05 -fhe MCI= --_..-   .- -
                       report      for the US General kccomting             Office      (GAO) .- Be&&e
                        tfre natarc     oi t!ae amlysis        strongly   depends on the report
                       and the data provid&              by XC, ~eneralizxg          the recoxmezxZa!Aons
                       beyond z.he syetm describei!             or extractirq       coxlwinnr       without
                       their     res?ec:ive     cpclifyhg       conditions is not ?ossibre.
                       Qucjs%iozs r:aleted to zhe subj%ct of this &ooxt or to the
                       popsibility        of extendir.g     the stated conclusions           or =eccxxtz-
                       datlons       should be addressed to the Report’s               authors   et
                  , FEDSIN.




                                       .


                .



                                                                                                         .
                                                                          ..          .


                                                                  .
                                                                      9




                                                                                 A3S'ESACT
                                                                                                                              .
                                                                                          .
                          In       res?ozse           to    a GAO recpest,                                FSDSi!-l performed                      a techzical
                   evaluatfor:          of a i's De?aEz~~t   of Agriculture      Kashin~ton
                  Comcter             Ce-*e-
                                        ..- a (FCC) r&rt.     The XC repcrt        cozxared   the
                       ;
                  zier,0rxanc.e  or’ 32 ITT
                                          ,,L Cisk drives with 64 13X di’sk d=ivcbs
                  ‘rArom June 1976 tfiro?-yh January 1977, The.WCC re?ort.cow                       _.____
_ _   - . ._                                 _. - _= ; .._.-.
             --eCllu%e%-m(l)    Cat de ITZL cask drives'               perforzcnce      was i.Q ierior
                   to  that         of tf,e 1%: disk                            drives              and    (2)       that      ITGL 2isk                 drives
                   degraded          perfo~marrce   of                          the           Wlfeshfngon            Cozpztor                 Center.
                          Ff Cs 3:         ZeZiCSC?d        tSe                fOllOWinS               conclusions                    abOUt       t!X     WCC
                   report:

                   r2sLjl;    oThe conclusions                                    irr the KC                report           are         based          on the
                         ’ t    f kappzsp, -4* ate                                stati stical                n?etho5s arrd therefore
                   are invalid.       .

                             (2)      TSe use of               “per               d:ive             ?er month”  average                         data      to
                   coxpare          the I3X and ITEL 2iSk                                          Crivcs is ina?xro;riate                               and                ’
                   misleading.
                             (3)      Xthaush              som             of         the          psrtorcance              masxcs                were
                   correctly            reported,              om xc5 iaconsisteat                                      with           1SCC'r definition
                   of that   cep.sc~e, and others    (1) wer3 meaningless                                                                      a5 perLor-
                   maqce
                     .     measuzss , (2) were inccnsequsntial,       (3)                                                              were      irrvalidly
                   computed,    or (4) were erroneous.

                             (4)      Most     of      t he herd fails                                attributed             to ITLL dirk
              a drives             occzred            dzriz;   the 17X,                              acceptance              tests.
             . .
                         (5)          SC&I    fac:crs     as acceptance    test procedures,                                                         the
                   plac&Tent           cf systez       packs,   and k?CC air conditicniag                                                          Kay have
                   adversely           affected       Et"> disk Srive performence.
                                                                                                                                                     -
                           (6)        h’ei”,her         tSe      WCC report
                                                                    nor tSe KC                                                 ;r,ancal           logs
                   support          the      conclusion  that IPEL disk drives’                                                        pe:fczzance                is
                   inferior           to     that  of IBX disk drives.

                          FfDSIX           recomer.ds       that    the US General Accoatins             Office
                   re-ues' m tSe KC                   to
                                                      reevaluate         tSe performance      of 13X and
                   IT?L disk            drives.       This reevaluation           should   Proceed     onlv if                    1                       e

                   WCC still            believes      tfiat    iT’5L disk dr pive perfo-rmance         is                                                               .
                   ir.r’erior         to tSct Of I3Ii Eisk drives.                 FSDSII4 ft;rti,er     recom-
                   mends that             any zeevaltiatioz           iaclude   such perforaa::ce        t3racta                                                       ses
                   as access            the     perfomance          and that    it exclu5e      acce?tence
                   test       data.             .


                                                                                              ii
                                                  .
                                                                                                                                    .



                                                                                                  .
                                                                                                                                                      .
                                                                                      .


                                                                                           TAB55           OF CONTzN?S                            ,
                                                                                               .
                                                                                                                                              .
                                                                                                                                                                     Paqe
               PIEFACE . . . . . . . . . . . . . . . . . . . . . ..<.....................                                                                                       i
               AaSTRACT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ...**..*.*                                                                     ii
--   -   -   - . -_ -_.__._. .. _.___.._- ___.           -- -- -- _---_-. _.- . _                    _                                                                              _ _ _ _.____._
               TMLZ OF CONTENTS ..‘*.....I..........*...*..........                                                                                                      iii
               TASLES       l   .    .   .   ..~...............~....................#..                                                                                    iv
               FIGUXZ . . . . . . . . . . . ...*......*.......*.-                                                                         .. . . . . . . . . . . .   .          v-
                                             I
               Secf ion
                    I.          IN?RODUCTlON                                               ................................                                                     1
                                A.           ' Sk~lXROUXD ..............................                                                                                        1
                                3.               3RO3xT                                   0B3EcTIv-ES ......................                                                    3
                                C.               STAIE!GZ!:T OF                                       WOPX          .......................                                     4
                                D.               TXfE SCHEDULf ...........................                                                                                      s
                                E.               ?X?'W?Y FXXJSC': CWTACTS ................                                                                                      5
                                F.               hCGNO:fLSDG?l.W? ..........................                                                                                    5

         .           II.        kZTHODOLOGY .................................                                                                                                   7
         .
                                A.               3ZViE3d O? WCC RfPOitT ....................                                                                                    7
                                B.               EXTINGS                                   OITH ITEL M:D KC PE?..SO:~::S
                                                                                                                      L ....                                                    7
                                c.               PZVIfW 0' IJCif SOURCZ D3CUXZWZS ..........                                                                                    8
                                D.               'AMLYSIS OF f?.EP SUXNARI;SS hJtD
                                                  RELIiri3ILITY PLUS OUTPUT .................                                                                                       9
         .         III.         CO;ZSTXXl?TS . . . . . . . ..*.....~.........c..*.*..                                                                                          10
         .
                                A.               TI!E                       SCtr,SDUIZ J,................,........                                                             10
                                                                                                                                                                                               l
                                                                                            .
                                a.               3ROJECT SCOPE . . . . . . . . ..*............**e.                                                                             10
                                C.                ?sPzo?~*s.scE AREAS .,.*.,................*                                                                                  10
                                D.                UTiLITY                                 OF ICC-PROVIDSD D?+                                 ............                     Ii


                                                                                                            iii
r    I


                                                                                           .                 .
                                                                                                                         .
                                                                   TA3LS OP C3":T3STS (Cont'd)
                                                                                                                                                                                   Page
                                    IV.          ?C%??TS . . . . . . . . . . . . . . . ..&...................                                                                        13
--           .-       -       -.    --       _--. A-.1_     VX,IDI'X
                                                            .-.              O? STATISTI%
                                                                -.--- _- .___..                !-!ZTE?OZS
                                                                                                 -__. _._ .. ...*.....
                                                                                    -- -- -- --.-.             _                                                                      13
                                                                                                                       _                                                           .-- -. - -   _ - - -_._
                                                  3.        VhLXDlTY 0’ “PSI DiiIV3 TZ3 XXXR" ASALYSIS
                                                             HZZtOD      l   *.,..........................*....                                                                      14
                                                  C.         KAGKITCX UD ACTiZL SiGh'IFIUTlCZ
                                                             O? DATA '.................................                                                                              16
                                                                                        .
                                                  3.         &tic\-'   x3 C0x?x4ATZOS O? ?E;RFOwzd4cE
                                                             !EASUELpS . . . . . . . . . . . . . . . . . . . . ..*.a........                                                         16
         .
                                                  c
                                                  L.         FEDSIM     AZSALYSIS                                  OF KCC         I*XiiA.L          LOGS           . . . . . . .     22
                                                  ?.         FACTOZS AFFEC'Xh'G DISX D?JVZ PZP.?OR2XCE
                                                             A? WCC . . . . . . . . . . . . . . . . . . . . ...*........*..                                                        ' 23
                                       v.         coNcLcs;o~?s               ~c2i3 PzCO:c’z?;DATIOiZS                                           ..............                       26
                                                  A.          CONCLUSIO~:S                         ..............................                                                    26
                                                  3.          P2xO!QcxviTIO:~S                                     .........           ..*.***.i                 ........            28

                  .       .
                  .       .                                                                         . TA3LES
                                                                                                                                                                 :          --
                              TA3LE
                                                                                                                                .
                                           I-1      WCC Re?o=t ?erfarmnce                                                      Mecsures           Definition                 ..           2
                                           I-2      WCC Repxt                ii6Ztsdu.3 ?erfc,-nmce                                               Pleastze
                                                    Definition                . . ..*..................*..........                                                                        3
                                           I-3      PzLm~zy 3roject                                     Contacts                 ..*...*.............                                     6
                                    II-1            h!".eetiEgS       with                     IX!,                anti 'WCC Pekonnel                            .......,                 8
                                   III-1            I;e; Project                  Dates                           ..*.....*...*..............                                        10



                                    IV-1            kctucl         Perfczzmce                                      ?:ezstzes,            13X Disk Drives                             17
                                          . .




---        --m-3   -P:rr-xi~Zotd.di~2~ly             'Ove,r>ead!"-- -- Caused
                                                                           --._.   by _ -
                                                                              -._ ___           -20   ,-.-_   -   ___
                    SOff ?zd.s . . . . . . . . . ..*...*...e.....*.*...***
            TV-4    Cor;.-,ezison    of Original                  end ?.ecozpted
                    Dotcatine      Yocs         l . . . . . . . . . . . . . . ..*............    22




            IV-5    XT'3L 'riard    Fail3         ,....*.....................*.                  23




       .
      .
                                                                             .
                                                                                     I.         XGTRO3UCTIOh
                                                                                           -.
               A.           B.ACxGRou?:3                                         .                                      .I
                       the L’S De?ar%~er.t cf h~ricultcre          Kashinston     Ccrz?nter
                cezter      (Kc,“) xstallcd     EO I3M sinsle density         disk drives in
                3 2‘c-197:.       The ee:eral   Services Abidstration            (GSA) actk:orize:!
                the iz5tallazioz         of tSese disk drives on an emergency             sole
    _ _. _ .--- sJQ:r.c~.   basis1?,   czdpr--+a*lp     alleviate-    the   pzob-le==s--KC’-vas-   - -----
                         _ ..--..- with its ,‘%a: Systeh/370
                exp,,ar:err:r-                                     (Model 168) coqmtez
                syste.
                            TSe US General                                 A-zo'~z:tins                  Office    (GAO) reviewed            WC's

               disk   drives fro:                                      i<ZS               Cor?orztion.            (GSA had       ?,r&~usly            ,I
               a*;arted ITS 3                                      i’2?.5ZttC-~               Re+rements  Contract  [KmSer                          GS-
               OK-5OC22) that                                        stat4                 that the ITEL Clsk drives were
               f~~d.:hrTjf'..
                    l   e   W,“..““LZ               t?;!Avc?lfmt                             to   tha     1351 single        density    disk          ’
               drives                 2’: WCC xi2 specified
                                        installed            ITSL as the sole
               SO”--“O
                   L.--r o=- SC~?l*f :0= these disk drives.    1 GAO esttiated      that
               re;lacinc;     t:?e L$! disk 2rivas  wit5 17'31 disk drives     would
               s .I\'!? a~y3xk etely $~?9,900 per year.
                     TSe FZC reS.ncec? 32 X3!d disk *rives vith 32 ITfL disk
               drit.es in CX- 1916.         ?'S.e IXC co~fi~~~ration        thus include2
               64 T;?!-: i.isk drives arr? 32 ITX:!, disk drives.              In Decctier
               1976, L!= XC 3zo3are8           a re?z:rt   *et      em3ared      the perfoxance
               of tfit 6: 13?l k&        drives with tf.~t of the 32 IZL disk
               drives.       TSi s reo2rt was bcsee rx data for June through
               h’o-.~exb~r 13i6.      &I 10 February      1977, KC presared            an adLexiw
               to t-3e* .7ee30rz* tS2t aLdenC*c.z reflected           additional       data for
               Dece&er        1?76 and Jamery        1977.A The Chief, Procura?lent
               nivisix,       lezrral   Se,-ices,     US2A fos;zzded        a co?y of tie WCC
               re?OZt t0 1.X4 03 22 Fekcary              197?. The ,YOZWZZding          letter
               advised iTEL that t!7e WC was ZJZtaiiinc M further                    izstallcAon
               cf IT32 disk &ives.
                     The XC r&srt          Pefines five xleas2res cf gerfcmcnce       (see
                Zable I-l)    en5 uses thez to evaluate the disk drives.            The
                adGe3tz.z   defines en s;d3F-Liorhal mcsuzc of pe=Zomzzxe          (see
                Teble I-2) azd 3ses it, along k'ith t.Se origixl            five meas*uresI
                to evaluate      t.h4 disk Crivss.        The XC zzport assrrts  that
                the PTSL e;:ipment         dc;Trt!ied the CCC cczyztar  systen more .
l               tSL”. i3X tqci;merit       ZZCZ=:~rt this de5rrdtition  was a breach of
                the ~antatory       Re;uire;,as-;T    tmt:a~z   %zk*r ZS-OCC-50022.
                                                                                                                                               --




                                                                                                           i

                                                                                                                                                           w-
  I




                                                                                .



                                         .                            1



                                             .




                                                                            .
                                                           h'CC X'O2T
                                                                   .        DZ'IKTIO:;

       IPL                           This    cxc5e&Aec?        13~5 of the o?eratin-;          s~sze-ll
                                     affects      the mtire       “signed-w”     co.xxnity       of
_-__    _     . _ . -_. .-.- _--_-   users;.   .- In &he--case     -of- WCC, an .avkro*         I3L--------       . .-.-__ ..- ----
                                     requires     20 minutes     recovery           time.
       Herd       Fail               This error on a d*Jve
                                                         we results   in the user’s
                                     j 05 a%ormally   terxinating . if a herC leil
                                     oc,ur s on a system pack, *the svstem
                                                                      m     will be
                                                                                                                     I



       Scft       Fail               This error  ox a Chive is ult~a~cly                    recovereC
                                     either  by software or hardware.            The systm
                                     attemts- to coqlete the read or writ 8
                                     activity      t!!a .t sencrated the soft fail     befor
                               .     terminating         the job under a hard fail     condi    #on.
                                     When such te zaination          occ-uzs, the so5 t fa'l CM
                                                                                              -
                                     co'ur,te' reverts        to the nazber it held befcre
                                     the aciivfty          was first  atteqtnd,.  and zhe f.rrE
                                     fail     counter iS incremented by one.
       Xncidmts                      These  are all the errors recorded in LOGXX
                                     thet can be traced to a single cause and
                                     recorded as 0.78 incident      2~03 the ti3e of the
                                     first  error occuzrence until        rfz5edial    action
                                     h%f been taken.     Any error    that    occurs
                                     a-c er the vendor's    custmer     engineer     -infor;-,s
                                     WCC that the drive is available           for use is
                                     recorded as a new incident.
       Downtime           Houzs      These
                                       .     are dorc~.tfsPk  hours at=ibuteSle             to a
                                     seven comn;=snent. Time besins acctzzxlating
                                     wr.en the device      has cr. error     condition.         The
                                     device    is then varied     off-lina,     and the resTon-
                                     sible custozner engineer is nctiZied.                  Down=x~e
                                     ends when the corqonent is deciared                "fixed"
                                     by the responsible       customer engineer.
                                                                 .




                                                  TA3LZ I-l




                                                                                                              l
                                                                     .




      systea
       -. - _ AvaFlability
              . -_..._.__ _ _              TSis perforziance measure is computed
                                --- ------by divi353~-ktuel-Drive     U~tize -by.-. -__-..~ .- _.___. -. -._
                                           Scheduled Drive U?ttie.    Actual Drive
                                           Uptime is co~~sE:ted by subtracting
                                           Drive I+wntine fro3 Scheduled Drive
                                           uptime.




                                           T-L):   I-2

                                      the GSA Contracting       Officer,    AD? Pro-
                                     ssued his findirrgs      an5 deterzin~t~on
                                    I. :-:t? csncluded     (1) that XTZL was riot in
                                     x2xr CS-OOC-50022. a?d (2) that the
                                      t'§X to replace tseir installed           1331
                                     ecci~rxlt    constitcted     a breech of Contract
                                      &d-a violation        of FIR l-4.1107-7     and

           hftz  tX3 xas es%& to reviw         the situation,     GAO
    * rquestec . %*h,at FSDS21.';provide techice      assistance    LO the
      review prozess.                                            ..
      P
      4.      F?.SJzCT   033~CTT~:S

           The objectives     of the lLDSiti project        k'ere (11 to eval!Jate
      the validity      of the KC report,        (2) to ve:ify    the accuracy
      of the data contahed          in the KC report,       and (3) to attemgt
      to ic?er,tify   tke oerfor;;lance   differences      between the ITEL
      aad 13X disk driks        (iI; only the perfo,-rriance areas identified
:     by the WC rep:::)       end detezzine       the. reasons for sezh differences.
              Althou* the GSh Cxtracting     Office:,   hD? ?rocuxzent
      Zi.ViSiCZl, >,a:! daLg-i
                         -- -...,ncZ tier ITZL was r:'it in breach    of
      ccrntra ct, GAO requested that FESSI?: evaluate tfie velility         of
      tSe FCC report.      !ZCC had dccmezted     their belief    *at  ITZL

                                                                                                    f
                                                                                                    \
                                                                                                    1
                                                   3
                                                                                                    i
                                                                                                    &
                                                                                            L---I
                                                   .


      was deszadFz75 the ?erforzence        cf the WCC ccqxter      system.
      GAO wished to determine       xkether   tke netho%   esed in the
      report   were valid.      If the netSods were valid,     relief    zight
      be obtained   eve.-. throqS     ITSL xas not held in breach of
      contract.      _                                                                    .
             GAO requestes         t?zt TH!XI!+: ?:roride tech&Cal          suT?ort to
___ verify      the accuracy           of   the    data
                        - --.- ---- -.. _ _ _ _. ______ iz~ the WCC report.     There
      had    beea. x5..&    cmtrovezsy                       over    Cbhe
                                                                       @‘~e$~~~A+-~ d*,’       even    if- -   -   --       .- I -.- .._. .. _.__
      FE9SI.h; deterzined    that the methods used in the report were
      valid,   tSe:e Eigh t still    be doubt tiaat ‘U?e actual data in
      tSe repsrt    were valid.     GA3 therefore  wanted an urAiaoed
      review  of tSe        source                irte       frcz~ wCicS       the   KC   =e?o:t       data
      were obtaiaed.
                                                                                                                                            I

             GAO reqxeste3         that FE3SIX de=ez&ne the ~erfcmcnce
      differences        between ITSL end I3!{ at the WCC. Sine * WCC
      persome        believed        that perfomance    diZZerences     existed
      betweer! IS% and ITZL disk drives,             GAO wished to deternine
      the basis for this belief--resordless                 of the validity     of the
      WCC report.          ITZL disk drives      azc installed      at meny other
      Federal coquter            sites,    me GAO was aware of no other
      coqxter      site tSat Sad issued a report asserting                that ITSL
      equipcterlz ?erfc,..,--ad        less well than IB!.:. GAO therefare
      reasond      that,      even if 17% ef#pnent          pcrioxed    wozse tSan
      13M egzipient          at WCC, cenCi+ &ions miqtle       to WCC night    be
      resoozsible        for tSe pocr per20,rzance.          GAO wished to ider.tify
      such    conditiox           if        czy          misted.         GAO and fgDSIf4        realized
      thak    t5is   o5;jective              nfzkt            the short
                                                             not    be   met   Seccuse    of
       time allowed  for tSe project    and that the objective    would
    -. not be net xless    cl1 relevant   CCC data were natie eveilejle
       to FEDSIM in a tinely   nsnaer.
      C.     STAZ’E?T!IT    OF WO?X

            To accospLisS tie objectives    cf t.his projest,  FSDSI!4
      identified    fox  se;;arate tasks.   These   tasks are sunarfzed
      below an3 detdiled     in Section II, 14ZTHOiDOLOGY.
           Task 1 - WCC Report P.eview.               FEZSIX was to review tSe
                                                      fEXI!-:                 the
      zesulzs cf the  zhe WCC ze?ort
                                   zegmrt in order to detemine        its validit
                                                                          validity ,Y
      and was to deternine
      and             deterniae      -(l)
                                      (1) if the the data presented were justi:iable
                                                                           justi:i   able
      and relevant,
           releva at,      (2)  L,.a the analysis
                           (2) r-                      nethods were ap?ro?riate,
.     and (2) if-the
                 if 'the correct          coxlusiozs     were derived fron the ’
      data    end methods.



                                                                                                   ,


                                                                     4

                                       ..     .
                                                                                                                        .
                                                I:             .
                                                       .
                                                           .
                                                                   .


                          Tcsk       2 - Veri2v
                                          Acccracv of WCC PeDort Data.   ?ZDSIX WCS
                  t0 review    the ace~racy cf &he deta in de WC report     both by
                  analyzing;   5s actual data presented azd by reviewing    the
                  sc=;rce doctzzezts  fzon which the KCC r-e?ort date were obtained.
                          Tcsk       3 - ?erfcrztxe                    Coz>arison.       FZ,DSI?I was to asalpze
                  lthe so-zzce         dcta ~xwideb        3 KC,
                                               _. __ __. .---5- -- _. attest
                                                                      _- -.--a-- to gsiess Lie.     _                            __ __
    __ __..                             ‘cze=zerences    between    the ITEi, and IS!4 disk drives,
                  per.. =orr;z5ce       c-m-
                  &d      at$eqt         to detezize        the reasons for whatever
                  perfo,-axe             iifferezces               were found.
                      Task 4 - 'Prcject          3e3crt aAd Briefix.               FfDSiM WAS to
                  prepare   a Fxal    Irojecz        3eport     rhat sczza:ized         the   results
                  Of     the
                          study.     This doc*z?lent is the result                 cf Task 4.
                  FEDSI!< hzs clso ?ze?arel            t briefiag      thrt     will be presented
                  to GAO nenzgerrezt       after     tke final      project      report     1s deliver                     ed.

                  D.       Ti!*Z SCEfXJT,E
                      FECSIX be:za thfs         ?:rDject  on 9 Xay 1977 at tSe first
                  GAO/FZ3SLM E;eezin~.         T5e weeks of 9 Xey and 16 Itay were
                  spent      reviec:ir.s the XC zo?ort,     neeting with ITEL and WCC
                  perso~.~iel, an5 zevie;:izq        WCC source docuzqeats.      r"EDSIM
                  d0Wiie,c.t     ed the res~l%s of this project     by writing'this      Final
                  Report the veek of 23 Xay 1977.
                  E.       ?I?I!G~Y ?'?O,'X‘Z' COXACTS
                           DurLas this project,                        ZDSIX       dealt with gersomel          from
              .   three      seaarete cqanizations                           (see    Table I-3).
              .
                  F.       AC~;O>~T~~G~:.~h'T
                                                                                                  .        .

                        ?XSIX tharz.s Xz Kurt Xatlock        (GAO] for the outstanding
                  sup,-or: he provided       deriag this project,     Fx Xatlock    arrangeZ
                  meetings,     acted as LFaFszr, be:wserr the four organizations
                  involved,     acd.?rovided     FS3SIX most of the project     backg-oznd
                  infozrbztion.      Since tfiis   project was very short     and involved
                  50 ur separate    cr:azizations,        the potential      for                      serious
          .       problezzs   ~2s treat.        r"EDSIl4 credits  ?Zz &!atlock’s                         outstanding
                  coozdizazioz             wit:? preventing                 such problems.
                                                                                   .
          .               .      -                                                                                     .




                                            .
                                                                               5
-
                          .

                                                                    .




             ORGA!f;ZATI3N                                   PERSO??hX

US General    Aczrunting       @ffice      ,xr D. Eirich
                                            Pix 3. Xatlock
               -.- ------- -- -_ -_ ___.__     -- - -----                   __   --       -   -.   -.._-_.-
US D&ii&&         of Agriculture            Px T. Kuhn
Washingto;    Cxpter       Center           Kr A. Borough
                                            k?x D. Flym
                                            Dr D. Bearfoot
ITEL Corporation                                          b!! w. Topercer
                                                          b!! J. White
                                                          k!! IL Nebel


               ?RI?!ARY       PROXCT CONTACTS




                                              .       .




                                                  6
                                 .*   l   .
                                                                                      .
                . -                                  .

                                                                         .
                                       .




               The bcsic Formosa of           this yo$ect   was to review whether
          IT32 disk drives dt?Sreds           the KC computer when coqarcd     to
          IB!4 d I Sk ~-rives , 2x KC           rap:=t was cruciel to the KC
          %sser-,iori t,?!%t I'=SL disk      dzfves    degraded      pzfornaace.
        -. Fz2si;*: t~?crefar.e...re~~~ew~    bo2.h .dri~. validitz.      of the .me$hods
           md :ha n~ctlracy          of tie deta in the WCC rqo:t;             but,   becausa      -   --       --
           FZ0Sl.h: did not wish to be cor~trained              to consultiq        only the
           report,    iC, met with bo*th iTZL and WC personnel.                   The
         ~“~poe          of these mestin;s       was to 5ain additional          insight
         int3 Isxkyrouzd           and dezeil t.L,tt rr.isht not be covered in the
         ZCt?ZZ 9 FL3SSZ-l alsc revieweC mazy 0,= the IOU:CQ doxznmts
         2r                 the !CCI: re?ort had hen prepred.                WCC ?lar,red     to
         ;r;;i;;i;;Ls-;:         soyw+Lt’\. repr=s      from EX? end txtw+l tic iron the
         %zlia!Ality          PIUS sof&re,          (~?a is the resort         grovadac
         ire? t?.e :3X SYSl.LOGRX file                oi error records.          Raliability
         ?l~s--sofz4&re          provided by Reliability           Resedizch,    Inc.--analytea
         SYSl, LOG?ZC C~ta.)            FESSIK wcs tc use these salaries                  so
         that it wc~.ild ezdsrstmd             batter     the efiects     of ITEL and IBX
         disk drives          on the KC’s      per,oormnce.

          Ii.   Fzsis:-i+ GP rice   ?Z?OR?
                                                                                                       .
              ?ZXi!< reviewed the ?KC report           to deteraim      (11 whether
         the sti%tisc'c%l   mzhods csed were a~~ro~~ia~~ and coxect.,
          (2) Lf the -i&hod   of u.;.forr,ly    amljiing     the selected      ?srforzance
         measc.res cn a "pc drives per no>;h" bssis was valid,                  and
    .     (3) the ma@tude     end astual     significance       c f the pirformance
    .    va~luss s!?s-e-z. The WC report ~*‘Es also carefully            reviewed      to
          cietezzins     (1) vhct:?er “3x2 co~~utati.0~3          in   the   report   were
          ccczcte      and (2j iL the deta clenmts           within   the report               .
         correlated     with me another      and wf';h dcta oSt&fned              fsom
         other WCC sozrces.       FEZSc; reviewed         the XC regozt durizq
         the rseeks of 9 !'a~' and 16 Kazf 1977. The FS3SIil persoanel
         WhO reviwe2      .ths KCC report    x.ncLuded experts        in zanagmont
         reviews    of data pocessing      installatiorrs         and experts        in
         computer    prfo ZZA3Ce CVAlGAtiOZl.         The report's       statistical
         mefh=rcls we:e, reviewed    Sv two ‘535 ii.; persozael         with doctoral
.        Cbgrees ir: 2rcSESility      kd statistks,           and, in operations
.
         research.


                 FXGX      not  wit:?   iT3L znd KC ?ersmnel      (see Table II-11                         #d
          (11 ts ~iscxss        the backc;rczx? of the KCC re>crt,        (2) to
          solitiz     c;i~:iozs      EZd cl2rifi cation of t!le methods snd validity                        ;
          OI
             c she WC2 reps,           and (3) to s;Et,ier rspmts    Iron  those who
                                                                                         .
                                                                                 .
                                                                                             .        ’
                                                                            .




                    bad personally         obsekdtke perfokanca    of IT2L and IBM
                    disk drives    and nie;ht have noted reasons for pazforzance
                    differexes     between the two.    ffDSIPS felt that the last
                    puqose     of these neetin~ was p&rticclarly     kqortant,       because
                    the real issue wzs whether ITZL disk drives 3erfcrned             less
                    well thnn X3!,! disk drives.     This issue should not necessarily
                    be decided solely on t!x basis of a potentially           deficient
                    re2ort.    If Derfcrr*anca diff erences existed between   ITEL and
                    83."l disk drii*es, WCCxzisht DO: have docmentad    these differaxes
-   -               whh .B technically.sound-zaezhod
                                                  -----A- bw
                                                           -.%---.*Aa ____
                                                                       dicce-ences
                                                                          f?..'  _. ___ would
                    nonetheless         FZDSTM hoped c,t rt the mmtfrrs~ waul;d- ~. -. --
                                      exist.                                                                                 .-   -   -
                    provide            inaisht abxt t.a daily cperational
                              ad SitiOR&l                                                 use
                    of ITEL versus IBM disk drives anC about daily observations
                    Of such use.


                         DATE                                    PERSONT~L
                                                                         IN
                *                                                ATTENDANCE


                    12 May 1977                     FEDSI!?:     D. Deem, T.                     Fasso
                                                    GAOZ         K.   Matlock
                                                    ITEL:        3. Togercer,                    3. White
                    13 ray 1977                     FEDSIM:      D. Deese,       T. Fasso,                  3.   McKer.zio
                                                    GAO:         K. Mntlock
                                                    WCC:         T. Kuhn,       A.           3oroush,        D. Bearfoot,
                                                                 D. Flynn
                    16 May 1977                     FEDSIM:      D. Deese, T.                    Fat50,     3.   hkKenzfe
        -   .                                       ITEL:        M. Nobel
        .   s                                       PEDSI;y
                    18 May 1377                                  3'. McKaatfa
                                                    WCC:         T. Kuhn, A.                 Borough,        D. Flynn
                    20 May 1977                     FEDSIf3: B.       McKenzie
                                                    WCC:         A. Boqoush,                 D. Flynn
                                       .




                                                               TABLE II-1            '
                                                   m
                    c.    R?D'iZ:F c;? WCC SODRCEDOC?f:ERTS
                         FEDSf!4 r evicwed the source documents available    at 6!CCin
                    ozdcz to verify     the reported data for IPL's, ixidents,    and
                    downtime hoczs. These reviews were conducted at WCC cn 18
                    and 20 Kay 1977.       WCCalso provided FE3SUi nancal error
                    logs that were corcailed frcz~ daily ZRZP reports.
                                                                    te
                                                                         --
sumaries as plcxed;     cperatiq     sysc,& proS1t.m~ in4 tata
e:zors in tSe h.X? tqes    prever.tcc! the tbely    dalivcy    of
tsa 3c.zzeries.    These pro51 ezs are   dfscusseC   further   in
Seczion   z:z ) CxG'2TmL3~?S.




                  .




                                          .
             -.




                                  9
                                                                _.- . .   __           -.---    -          .-. .. . .   --

                                                                 .
                                                                                                            .
                                                         .




- _.--_-. - _._   _




                       DATE                                          PR3JXT ACTIVITY

                        9 May     1977           First   F ensm/GAo        meeting
                       12 My 2.977               First   FECSIM/GAO        meeting                  with    IT&L
                       23 Bay 1977               p’w St FEDSI!4/GAO meeting with                            USDA
                       19 Mery 1977              FS3Si>: reCC?iVedEEE? SUELTariCS
                       27 Mey 1977               FEDSI! delivered              Etial           Report        to GAO


                                                         IEY    PROJECT DATES
                                                               TkBL5 XI-1
                       8.   PROJECT SCOPE
                            The scope  G f this   ?rojec t wm limited      to the WCC. NO
                    .
                  ~ m attezpt   WCS made to ex?a- * the project        so that it would
                       include  data Lrom other ,B!4 System/370 ccmp~ter site6 that
                       had 13X and IT3L disk drives.          This constxaint    was reason-
                       able consi2erdq      (1) the lir lted    ti;ae schedule and (2) that
                       the issues had been fomclly         raised o.?ly by WCC.

                                         .
                              This project      exalts-.ed    only those masures       of perfozmance
                       aCv&nced by t!CC (i.e.,            IPL's,  Eard Fails,    Soft ‘ails,    LnciSents,
                       Downtime %ou:s, and Sys%m AvEflaSility).                   klthou5h    FEDS114
                       recognizes       thtt   m8np other areas could have beerr addressed,
                       tiF,@ COnSf$Er8tiOZ?!&        ;;rez=ludcd e~9aaustfve malysis,        and the
                       z:ezs     advcnced    by WCC were ektually         the only ones at Issue.
                       Perfohmnaace       conside:zticns~other        than these are briefly
                       discussed      in Sectiorr      Y, CO3CLXSO::S A3D F.ZCO!G~::DA'=IO:iS.


                                                                                  .            2’
                                    :

                                             :                       10
                                                                                       .. .

                                                                                                     .
                                                            .                           .




                           FZYS”,?! e.... --
                                    ~-~i:slly          ex~~ine5           8cliability          ?ZJs software
                     raparts   anS fzz? Xeao:t3 Cf eke 13 !*iay 1377 3eeting            wits CA0
  c                  and WCC. ‘=sis exL:irration       rwealed       inconsi§tencies    batwaen
                     th’3 d&t& in TdXt t!cc :CQQZt,     ?.e~i&bility      ?lUS SOftWR~t2 repQ:tS,
-- ____.--
       ._.__._     -axi t!??e E?ZP- re ? o=c,s .- -_xx per§on,?~3..wP,re.-~~le~ta~retsont~er_     -.----                                --
                     these difieroncas,      but thy    fait   tkst the XX? &rid R%liabi!ity
                     Plus data perhaps did net cover the exact periods               reqresanted
                     in the KC rap::.         WCC personnel      also felt *&at the ES23
                    snd Reliability           ?i::s daza          mfgkt        be fragm~tczy,               si.xe        they
                    we-e tstracft?~         frc3    tape3         that     contained          a nm!xr          oi ezrcrs.
                    Thiset ez=cz:s r:o:LC Save causal taaa records to have                                          bean
                    rajecta2 or orkexise     2.zp:operly jrocessed.
                           WCCprcposzd to roprocass                       the tap-es in order
                                                                          the                                 to provide
                    nontSly      swzxry                   Reliability
                                                I=XZ? repsrts             Plus software
                                                                         and
           .        raports.     TSis would be done by (1) using
                                                             using identical
                                                                      identiczl  input,
                     (2) covering    tSe eight-rs:tS period represented by ths WCC
                    rap~rt , and (3) providing tke reports in emct aonthly
                    incrmants.            Thcsa dstn            were c,o 5s arovided
                                                                                   I
                                                                                                 to PE3SIbl for
                    znalys;is     0:: 16 Key 1977.
                            WCCdid not provide FEDSIN the data an 26 May as planned.
                     The Sistoricnl       data tapes apparmtly cor.caixd     nmy     errors,
                   . and the KC oacsazin g sysma had been chaqad sinca the
                     tapes were cr;azcd.         TSesa two factcrs   causaC considerable
                     delay     in tha plarkned   da1 iva,ry of EC? scz3aria3   and ReLia-
                     bility      Plus sofzware repxts.
                 * .
                 I .        when CCC fint;lly    did prsvibe the $umariee    on 19 and 20
                             c LI v< ,90tlnd no co rrela9.09
                     May, rr-s                                 between +~keEX? tsux3azfe9,
                     the P&liability        Plus software, and the WCC seport.       Data
                    that      suppose21y        c~vercd         the same periods              were       inconsistent.
                     Close exa:izatFon revealed (1) that the data wcze not always
                     for  the W.XI periods,     (2) that the data were fragztantary      (in
                     that several nonths           represented by only 6 to 10 days of
                                                     were

                     data) , and (3) that Reliability      Plus software repcrts    printed
                     the Zeliability     Plus execukic~ date raffier than the dates
                     represented     by rhe repoxted data. These problem rendered
                   : ?A3  reports    aad suz~aries useless for FS3SIX's cnalysio,
                         The hard- f ail and .i?ort fpil dati in the FCC report   had
                    been compile:! :r- Ozi&mtnual logs prepared fran daily SZZP
                    reaortc,.  Sine e zho HPS? sur.ziaries   axovided FEDSXtSwere of
                    no*valu&, FZ3SX was uza5le to veri,?y the accuracy       of the



                                                                                                                                .

                                                                      11 .
                                                                                                                                    1
                                    .




_~ --   __ _-.--_.-. _.-   -   _   _ -   _ . _- - _._   .       . _..._   _-__-__ -_ -_ __ - ____.   _..-_.---




                                                            .




                                                                               ..




                                               12
                                                .
                           .



                               ‘IV.   3XSZLTS




_   --.- _- _-.--_.-   _




                                       13
  .

                                                                     .
                                           .



                   L,=     q-5

         c           ,2     n1 A, n2
                             r. 1 "2
- - --   -- --.-.-._ __
                      7    - .                           - -. _-._
                                   cn,-u       ;:- i jn2-1) s;;                  - -----___. _ _ -- -_---_ . .   _____
                                                                                                                     _
                                                                     -. -. ---
                            2
                     Khere s =                 n 12+ I:   -
                                                             .     .                 ..



                   IX disk equi-,zezt           rescLts     in 2 0.016         (1~621 “per 2zive.per
                   r.CZt:s”   eve-ace
                                  - e    vz~~~p
                                            B”W,    w:iereas    me    i?c     caused     by ITSL disk
                   e-..;-e-”
                     ‘J-d=“.       l   .   -       a 0.031
                                               resLts   in     (1132)     “?er drive       per month”
                   averece     vzlue.      These evercse        dttt,     w?.ich are used to
                   coq,cge     Zecr2detio2,
                                   _               2:re 5.us bicsd          i.3 cs=2vor of ISi.       The
                    "?er drive       per mnt:?"     method of coqari3s               62ta nekes IT23
                   3”‘=C
                     e-e zzz2ce 233ftr          fez worse fcr events              that equivalently
.- -.---           &-.;=-E-e- -sy5t~--~e~~o~~~e-.-                     - -- -----.-. - _.-          -                       .   ..__

                            The t:CC report       assu.zes      (1) that degradation      ceused                 by
                    e rrcrs    czr. IF uzifozzly          distrihte2      az1on3 all the Usk
                    dzives     ad    (2T) th2r     errors       ~23 be evelua'et     IzdeDenfently                    Of
                    the use?e of the device.                 ?E:EIX belie&;       that   nkther
                    ~sss----*
                          ‘““B  or, is c-w-
                                          ,,,eqf.

                            The first        css*.?ljt:, on could obviocsly                be r;,isleaSinq        if it
                   were use? cs a besis                   for celczleting           tot21     system desrcdction.
               .   The e==ect--         02 total’svste3             perforzicnce       of a 12rTe nuder              of
                   errers        on oze disk drive             s;li;ht     be si@zificcntly           different
                   then she effect               cn sys-,e=l ?erforxcxe               of the su;le nu:zber           of
                   errors        uzi5zrz.ly       distrlbu:eZ           amoz; all       disk drives.           To
                   ;:1..
                   -w-w     s=rzte      this2difiereace,              FfCSiI4    found that        761 of the 840
                   ITSL herd fails                 is Auszst        1576 occurred          on one drive
                   dew.:,’
                         -w. 3 2 40-:i3u=e             period.        The inference         in the KC report
                      ..
                   tkat      ear\“. oc* the 32 iTF'L-4 dzfves                averaged      26.25 h2rd fails            in
                   E%'2;2St      lSi6     is a ccm?l,ctely            ridsleading       conclusion        about
                 . L,otel       syste-,     ZeSreCation.

                           %e seccnd objectiorAle                    assk:.$io,?     ir. the WCC reprt               is
               c    t52t device          err 0~s ere uzrelcted             to device       usage.        FE3SX4
               -    disagrees       with this        ass*uqtion.           Two exzqles         illustrate          k‘hy
                    usa,-e c~“~ot          be ignzre2       when evalceting          device       errors.
                    F1’,SC
                    .-     -, coxii&er        a device        *Aat has little           or no usage;          few,
           .        i5 cy',       exors      would ba essociated               with *his      device       because
                    it ~2s used very II,,-*&l;.                Very lox usage will             therefore
                    reselt     in a lox errcr           rzte.        (This usage charecteristic                  is
                    CC~O~ iz cmp.ker               installations           with many disk drives.)
                    SeccnS,       consider      a device        that    cozpter      cqerztors          know has
                    ezrcrs.        An c?eratx        ‘s naturcl         tendency      is to zut active
                     =; ‘es OS: zzre reliable
                     --a-                                 drives      2nd, if possible;             to avoid
                    usin3 . the      feilizg     drive.         Ysoki     error    conditions         can tkerefore
                    reset       i:? very low usage.               These two exaqles             illustrate
                    TSzSfV-- 1s &lies _ that           both usage and errors               must be enelyted
                    on 2 De: co~?xext              basis--not          on the basis of a unifo-3
                    distr~b~ution.


                               2
                                 FS3SiY 21i0 foxd                  this  count inconsistent         with    th-e
                    KC         re?srt   definition               of harti fails.

                                                                       15
                                                             _..-



                                                        ‘.
                                                                    :



                                                      .,




                           The WCC reoort              sfiowed selected             perfozxxe           zeasuzes
                  2iS tri!xted           rz?ifcrzly         over all drives               a33 p:reser.ted        sratis-
                  d,C&l      results         to eiz!zx           three      cr four decinsl            places.
                   =u”;ZSiX has akezdy                 doru:ented           its cbjeczio::         t4 the basic
                   “per drive          per =5ont.S” approach                  to cnuysis.            No= only is zr?e
                   e?aly+           cp:_r. a-?-“.T - invalid
                                                       .__-_. .. -.c    but   the   data     dimlay
                                                                                               L- -- _..._ techzF?Je
                                                                                                            -_.
                --*we2 tt5adid           to cx=fuse           :ea&fi-%S3zt              th6 accuracy           Xc?- ad~uZ1
                   ugaitu3e!           of t.Se values                shown.      No one fZDSI.hi         ictezvieue2
                   Sad ~uestione:!             zho data.              Evervone      assuzaed that          thy    were
                   C3 Z%CE        six2       5%~ were presezbd                     witi    fox-detiscl-place
                   -e
                   ,’ - ecisicr..        Tev ever; questio,7ed                  dether       tSe de:8 velzss            were
                   laqe        emu,-h      fo be sisnificezt                  or were too larse              to 5e
                   reasonabli!.                   .                I
                                                                  I




    .       .            The ‘Z?;’ s caused by ZTzi, aad 1% disk drives                          wetc
                  za.7czlby      recozEeQ      anil uszE Sy XC as one neasure                    of tke
                  effectiveness          of the insttlle2          equipment.         EDSIX         revieve d
                  the so*uxce data that were used to develo?                        the WCC re?xz.
                  rhe data shr;*#ed that           i?fL    hc:! been charged          with      32 I?L's
                  8ne Z:3:: bed bees chersei             uitS    13.     Ii0 prcblms         or incxsi       .S-
                  zenciaz       were observe3        in the mmner in whicS IX’s                       were
                  :eso:lved      ar.5 chersed      to eacS vendor.
                                       w
                         The so~ce         dats rtvealed       tiat     16 of the I?L’s             charred      to
                  ITZL occcrr%C          Cuzins    a %o-week        period      in late      Segtcrnr         arrd
        .         early      October     aad tSat      %hey b?ere caused by a r,;         4.C-I-1 GnS co~~rcllcr.
                  This controller           was subsequcatly          replace<      by fTPL.           Sizcc
                  ticit    repl$xze5t,         tSe nurzbe~s of i3L'S            charged      to XZZ an5
                  Z3!3 eqipxent          Swe been coc?arable.                 .                   .




                                                                                     --




                                                                        ..
c                                                                         L
                                                                                              .
                                                                .




                              JUZ                   2             1              839                              26
_..   -.__-.-_-        ..--   JUl             -     2". __ - ---__-f _. _~___
                                                                            _- _--_~;   -~ -A;--                  27
                              Aug      --                                                          -_._-.-   _.- .29-     --
                              Se?                                                219
                              oc:                   5              i               13        ::                   22:
                              Nov
                              DQC                   :           1201             fl      j   222                  2
                              J&r:                  0              0            107            5                  19




                                                                mm           SOFT                            lXNXTiXS*
                              r.o:m?              115'S         FAILS        PAILS       ssCIb3STS              H3'JitS

                              Jun                                              237            12
                   .          JUl                                              755            64
                  .           xug                                                             53
                              Se2                                              470           196
                              0 zt                                             137
                              Nov                                              143           12
                              Dee                                                             12
                              J&3                                            10:;             15
                                     wXTZL Dss:nzrz.e 5ours .‘rave b een cor:ecteY            rl -0:
                                                                                                 c     December
                              and J&n;lery.
       1          .




                                                                                            .


                              2.       Sard          Fails

                                       The          >arE            ftil            <eta reported                             in the WC report                           bre 302
                              co.-.siste~t                 with             tfiat           :c?ortts                   defir.ition               cf     2     hare        f2Z.
                              The haA fefl     2e% t.!zak are presea:ed therefcze CCXC~ Se
                              used to ieterr.ine whctSrr ITEL disk dzfves ~ezfc,?~l
                                                                                .    le'ss Gel1
                              tt;2n 1315.
---_             - - -----                     __                       -     -        - - --. -- -.-- --. ...__ -_ __-___                                     -     -        - -- ---.-_-       -.-   -   _.
                                   The WC rccart
                                   I              de fines a hard fail as a device erxr
                              that has the e>fect cf causing a user job on the systm                                                                                            tc
                              be cmcdle”                     -.       A couzlt of                           hard feils,                   as deEned,                     wot;ld Se
                              one nets-ze                         0,c Lie n-x&er                            of        jobs           aborted          beceuse            of        device
                              errors.                if           the       device              havi.zzg errors                          wbs 2 systez~ pack,                               a
                              haze! faiL uxlC :esult                                            in the loqs cf all jobs in tie
                              system ken the systezi                                            uhs cancelled;  an i?L wottld tie-1 be
                              riqzked                to           reini%ialite                    the q&en,    md scxe xzber      c,C job IS
                              wocld           here           to         be reszrrte2.                              k 12rge               COWS          o?     hazd fails,                      a#S
                              defirre:,              wodd re;rescx                                     2 serids                       jwOS1~.
                                  T2bles 3-l as2 n-2 sSow the actucl performance zeasezes
                              fct I3!4 and ST% disk drives.    Tor some cf the aonths shoim,
                              the couzt 05 herd fail,s for 3otb I3H azd I'I'ZL is extreztely
                              lerl;e(IZ!".: Kov=12C; IXL:Jcl=166,    hug=843, Se?=971. FZSI:{
                              questioxd    -,?e re2soxabZeness of these co-ants of herb fails;
                              the STEL Augzs: velre cf 645 translates     to over 27 had
                              tails           per         avercge                   A2)tT         After                exerhing                 the     921          s czz2.r~
                              reports          WCC-provide5, ?33SIM co*zld not ftid 840 hard 5&s
                              ia    kug2st;          in    fact,        FZDSIE       could     find only two ZTZL h2rd
                              fails         i.9 tSe ixcort=le-,e                 Z?.Z? suznsries             an2 36 IZZL hsrd
                          .   fails         iE a reprt            .‘frcm      iktlizbility         213s       s3fttjtere     tS2:t    exe-
             ,
                              cued 7 Se?tecber                     1976 (t?2 pre.smaSly                      psocesse5       Azgcst
                       -      drta).           After cSeckicg detailed,                        daily EFZ? se?rts,                    XC
                              pczso.mer locctod                    761 herd          fails     that occzzed curins one
                              r;C-nincte pried                   z~d were essocieted                    ~5th       a systen ~"~301
             a
                              pzzk.           C' early      , the       systezft     was not encelled                    761 tines in
                              one 40-zLute                 Teriod.             (Tfre WCCregxxt inZic=tes thaz an
                              everage          SPL      after      the      systezn       is car:cclleC            reqires         20 f
                              minutes to recover the systea.)                                  The WCCregmrt list&                      i61
                              “hard ftils"               when only one (by the re>ort's definition)
                      .       sboul& have been i?c?uded.
                      .                After              soae lisccssfcn                                   with             ZDSIM,            WCC i-?~ita&C                       zSeir
                              beli&             z?32t        the coa?zter                              sy>tem                 wo-zlC      ettem?t             t=, zec3’iez
                              fro;=          per=lzkent device ezz ors associated                                                               k,tk        systm                 ;acks
                               (for          so~~         xknom                     ndcr               of         atten?Si)               until         eitker                2    successf*2




             ,                                                                                                   18

           - .
                                                                                              .
                                                   .



                I/O cpe rrticn t:zs cozqleted    cr tSe svster3 uiyzi cencelled.
                TSese ztzezpts   v021d obviously    be n&e reszrdless     of w5ether
                tSe systeiz pxk wcs on 13X or ITiS.
                        XC did not identify    t?ze conditions    under which 1 -se
                mhs2      =,c
                          --we7 n coxts  Nere ettribcted    to I3!1 or ITS.'    Ii0?J ever,
                sixe       (1) tSe mr.tSs cited earlier     ize creditEd   with am-
                tsat,      ‘,
                       in rruaust,- ITS5            wes erroieously    credited  with 761
                "hard Eails;"    PZDSIPI           discogznted the WCC re?orz hard fail
                dtte and reviewed tSe               ner,cel  loss fJCC ha2 pe?ared     fro=r
                dtilr  2X? repczts.                This reviet: is dismssed     elsewfiere in
                t?rs- dot'meat.
                3.      so ft    ?ails                                  II
                           A soft ftil              zesxlts         frox     zm uncoapleted       I/O operation;
                the I/O o?err,tim                     E'GS= be retried.                 A soft fai1      therefore
                esseztlally                   re+res            enotSer I/O zicccs.             The worst case
                the           for both 13~: and ITfL Cisk drives to eerfom                                  this X/O
                operetion                is a~~rcxixtely                   33.3 cilliseconb          (cssuzing one
                complete rczztioa                      to SOS'LcclC3     '       the read/write     head 2nd c
                full-trkck                   reaE) . The software                   werhea     necessery for the
                I/O retry                on the WC scr;?puzer is potentially                        0.5 rilliseconds.
                3acs soft fzil                    thus "degrades perForzxmze"                    bv as much as
                33.8 z+.FLlisecczds.                        Teble IV-3 shows the zx-tiuzz total
                rupb'-l
                    l . a..-_  \. “overhead*             caused         5y   ‘,e     nuder   of IEtl ad 1113 soft
                feils           reported          5~. tSe CCC report,                  As Table XY-3 shows, the .
                co,?tf?lv tozel soft fa;ls                            Zcr all 1313 and XZL drives werages
            s   3.27 e?d 11.12 second;,                             reqectively!           These valires are so
        .
                 snail t.kat it is zct clerr                               wky XC selected         soft fcils      es a
                perfcrnance                   mezsure.
                        Clew-l--_ v 8 2r;y rxder        of measures could be used in en
                evaluztioa           cf ;erfc;,lcace.         The selection     cf specific
                neat-ces        depends on sue:? vcriables             as er,vizoxrtent      (e.g.,
                weight of a de*lice       .      is relatively      tmiqortazt       to nost data
                processing           iaszalla,, *ions but becones critical              with, for
                exclple,        zirborae        conputers)     and  the  mcgnitude       of t%
                I;&ZS'2ZP   3 (s.s.,        soft fails     are unizportant      ES a perfo?cnce
.
    .

                necscre         vhen t5evc occ*x icfrecpeatly              bzt become zn m?ortznt
                3e-corza7ce
                a   -a        .        ineascze    when  the   frequency     is largei.
                                                                              .         .
                                                                                                                                  t -
                     3 Iimever,    soft fefls  my be izportant                              BS zn aid to custoaer                 t,
                engineers     re?zrdless   of their magnitude.
                                                                                  &.-                                             i
                                ..
                                                                                                                          s       !
                                                                                                                                  1
                                             .
                                                               19
                                                                                                                                    .
                                                                                                                          *.rrY ii .
                                         a
                                                                                                       .
                                                             .

                          .




     _-. _-__.   _-_.-.        - --.---              .~ --                         --.-   .__. ...--___ - .--.     _. --   _- _-_,_. -   _._   .   -
._                            Jun    ?6          809             ;-   - 27; j4‘-               237                8.01
                              Jul 76                                     0.30                  75s               25.52
                              hug 76               ;                     0.23                   50                1.69
                              Sep 76             219                     7.40                  187                6.32
                              Ozt 76                 13                  0.44                  237                4.63
                              Nov 76                 23                  0.78                  PC3                4.83
                              Dee 76                                     2.06      .            165               2.20
                              Jan 77             $3                      3.62                 1059               35.79
                                                 -
                              hverage                                    5.27                   329              11.12




                                    The ncder    cf soft    fcils    cssociated    with disk drives at
                              ICC was verv sztall.       35' contrast,      source doclments    showed
                               several hc.&red tims       33ro soft fails        cssocicted  with tape
                              tian wL:h disk dzix*es.         Althznqh FEDSIX believes       that the
                              tape units    repesent     a seriotrs problem et WCC, TfDSLr
                              ccnnot wZerstanS       k'Sy WCC ckose soft fei3          2s a mees*re of
                              c3isk 6tive perfo,xance.
                              4.     Incidents
                                     FECSII3 cokparsd the count of incidents                  in the MCC report
                              to    the  3ar.3al 10,-s r3aintained         ct WC.        T3e sepcrt      w2s consistent
                              with the data in t5e mnuel               logs.      However, fEZjIM for?56
                              several problems with using the nurber cf i=lcidents                           2s 2n
                              indication       of systezz degradation.            First,    there is no cozrelatioz
                              in the XC re?or= between (1) incidents                      and (2) the hard S
                              fcils,     scft 52ils,       or dowzthe        de~r~~atioc      r;ecsures.       It is
                              *Aerefcre       impossible     to dctetznine the severity              or daration
                              Of 83 incident.           Second, there is th= question                of eqdvalezcy
                              i.3 systea cesza<atio3.            Incidents      nay resul:        fron either
                              hard fails        (?e,--,enezt errors)       CT soft fails         (t&;lpxa-q'


                                                     .-                    20
                              .




                          .




            .




m-v--           .   I..

    *   -
       ‘      .
                                                                                                        .-                           -.
                                                                          .
                                                               ’ .




-   - ---- --.- - . -. --. _Dee 1976       __ _.__ -83.       __               _-    - 108.7            12.0
                                                                                                        .---.--_-.-.____.18.2 -. -- --_- ---_-
                                                                                        - . .-. -__- ____
                        Jan 1977                                                       122.3.                           20.4


                                 CO>i?AXSQ’ru’ OF GRIGifi&L              Xi3

                                                                      Th3LE
                         6.       Systezl hveflabilfty--
                                  The cddendm          to the         orlginel      report       added a sixth    ncasure
                         of      systfim      desr adation-‘1          sy ste;11 avtilcbility        . o ln reviwdng
                         the data in Table IIIa                    FSDSI>1found that
                                                                     of the aidendua,
                         the calculations fcr l?ZL system availaSility   erroneously
                         used the nmber      of ixidents fro;zt Table Xa Ah the KC rc$rt
                         rather    then zne I:% ciovn%m hoers.       The WCC rcqaort thctrefos                                 ‘e
                         contaimd the irrcor~ec~ coaclusianthat        there was a sicpificn                                   2 t
                         diffe:ence    betmen 1351an+ ZT3L systea 8Vd,l%bility.


                           EDSIN reviewed the VCCmanual logs to verify                                         the accuracy
                  . . of the WCCreport   hard fcils data and to icicntify                                     ar,y w~csually
                      large counts of hard fails.   FEDSIN foxx? minor                                    Cifferesces
                         between           the      data presented            in    the   WCC report   end the mm%1
                         logs;       the      WCC zeoort      attributed  1173 hard fails                    to ITZL,
                        while        FEDS31          could rind o.-,ly 1142.
                                  FEDS114fo*,xad seven occasions ~:hen more thnn 10 hazd
                         fails    were recorded for a single device i? one day.          These
                         seve3 occasions        accounted for 1056 (or 92.58) of the 1142
                         hard falls.        It sees    unreasoncble &hat these   hard fails
                         acturlly     caused lC56 jobs to be tezzfaated      abormally.     In
                         fsct,    as discxsed earlier in this document, 761 of the hard
                         fails    occnrre2 dx:in5 one 4$-mirxte period.     FZDSIK also
                         observed     that six of these occasions (and 1024 hare fails)
                         occczred             the ITZL acccatance test months of July,
                                            &XinS

                         A'sgut, 2nd Seprezber 15176. To obtain a mere rezsonable
                         assessment of the passiSle degzadatioz       caused by ITZL k=trd
                         fails,    FSDSi3   reduced the number of hard faLls for each cf
                         these seven c3servations to one (althoqh WY reasonable
                         smzll, nz5er would have sufficei).
                                           .               22
                                                                                        _.__-         . .- .-      ._   I
‘   r)

         I




                                                      .




                             F$u:re   iv-1       displays     tSa   hard   fail         co&              FZ3SIX     erxtracted
                  frcs        t5e zaxal          Togs and tsa adjusted              Co'Atts.        IV-1  Figure
                  cleerrly       ~50~s z.Set DOS% (1106 or 96.8%) of the hard iailt                                                      <
                    occ:rref      ckring rse XEL acceptance test ncnths.                      Azter      9
                    accaptazce       tcsz    pericd,     had foils     dlninished      to the poar.t
     --.- - - - - -.
                    t~i.ar- thq~ xwze2xsic;ttiflcant.         - - Tfie ad jcs!zd--hard--failsr---m                               -- ----__-
                     (dashed lize)        tozalad      94 for t!m aisht-month        period.        They
                    renched n ~ontf?ly high cf 33 during August 1976. There were
                        ’ v foe:: hard fbilS         rzcord%d froz October 1976 through
                    ary          1977.
                                                                             ,- *
                   2



                                                   -r
                              900                                                                           ios Data




                                                    I
                   %          800                                                             -----         Adjusted
                   i?         700




                                                  I
                   r=         600
                   =          500
                   ‘A         400
                   0          3to
                   I.
                   E:         200
                   z          100
                    2            0
                    2

                                                          ITkL HA?3 FAILS
                                                            .'fGU3E IV-1
             _-
                        FEDSI! s z.mrized    the htrd feils    for each disk drive in
                  order TV detezrne       wSether errors were unifornly         distributed
                  L?~OZ; tSe IT3L disk drives.       The results     (see Table IV-51
                  shaw thtt Se:! Zails were mt unifonly            distributed.         fn
                  iact,   66.3% cf cl1 hard fails     occured     on one drive and
                  89.9% of ell errors      occurred on four drives.
                  ?.         FACTOSS;ZF"-fYG
                                        tl.,,.              DISI: D?.ft'S PEiL"ORVWJCE
                                                                                     AT WCC

             I           One objective of this project              was to deternine      reisons
                   for   per5 orsccce c,,   ‘; uCerer.ces
                                                c           Serween ITEL and 1BT-ldfsk drives
                   at WCC. _Si.?CE p~~)s3!.{ was unab!.o to identify             maninc;ful
                   perfcrzerice     '4ccerences
                                    c,,   _            betweer. ITZL aad 1324, this objective
                   yas oz:1y pa:zially          tchieved.      FZiSIM did, however, identify
                   three ftczors        ts:Et ~0211 cause perfomance          differences
                    (eider     real or apprent)           between 13M and ITELc disk drives.
                                             .                                      -
                                                                                                                                      I
                                                                    23                                                                I-,*
                                                                                                     .




                                                                PERCEKTOF                                  CUMULATIVZ
                     DISI; mIvE                                TOTAL EilROX                                 ?ET(CEE?

-   - . - _..._._ - .;**--.      .-_--.    -.      _ _                                                          .zib.8--e-e
                     688                                                                                         76.4
                     581                                                                                         84.0
                     56D                                                                                         89.9
                     566                                                                                         93.4
                     56F                                                                                         94.7
                     56E                                                                                         95.8
                     563                                                                                        196.9
                     568                                                                                        *97.5
                     589                                                                                         98.1
                     58F                                                                                        i98.7
    .                587                                                                                         99.1
                     582                                                                                         99.4
                     589                                                                                         99.6


                                                         FRXQUENCYDISTXSUTION OF
                                                             IT&L HAXDFkILS
                                                                    Ti'BLE: IV-S'

                .     1.        iTEX      Acce~ttnce         Test     Proce2ure        at WCC
           .    .
                             The a xeptaxe       test period  is that period curing which a
                      newly installe:!       piece of equipment must meet certain      levels
                      of performance       ior ti prolonged   period of time (usually         .
                      thitty    days). At WCC, the acceptance tests were run in a'
                      axial     mode on each lot or shipment of ITZL disk drives.
                      Each shigent       3ad to ptss accepcaxe tests        and be signed off
                      as accepted by the WCCbefore           the next shipment could begin
                      acceptance                tests.
               .                m Thdm,s Kuhn (Director                       of the KC Data Cecter)              steted
                      at the           13 fray       1977 neetirrc;      that     Mi    acceptance       test   proce&,zre
                      was designed to utilize     heavily  the equ5.pr;rer.t undergoing     .
                      acceptance   tests.   Dzing the test period, hiTh activity
                      files   were placed on the XTZL drives,      and special    ?,ro~z~:s
                      were *an that heavily     accessed the ITSL disk drives.          The
                      purpose of this he&*:, 2% ws to ensure that, if the equi?-
                     ,ment were defective,    ‘;; would fail   &xinS-- the acceptance
                       tests.

                                                     .                   34
.


                                                                          .a
                                                                                                      .
                                                         .       .
                                                                                        v   .


                                                     .
                                                                                        .

                                                             .




                           Lots 1 and 2 of c’  me ITEL disk drives began acceptance
                    rests on 17 !-ky 1936 and were accepted on 1” June 1976.
                    Lots 3 ax? 4 began acceptance tests            03 12 July 1976 and were
                    accepted Sy the KC in early Septez!xr              1976 (exact date no:
                    available).      ti VT-A-& coztrcller    szarted failing     durinu the
                    lest week of September 1976 and was replaced the firs’. week
    - -.- --------- of-October-    >9i6-i -- -%e-WC required     that. the replacement.-.---
                    ccntrollar     under;0 acceptance     tests.      The replacement   con-
                    troller     was accepted on 2 1Joveraber 1976.
                           Much of    the    data    cantaiaed                 in the       WC    report     were Sathered
                   wkile     the ITIL       equipment            was undergoing                 acceptance     testing.

                   ..
                   7       Location     cf S!.stem Packs                                                          II
                         A desice error has its most serious effect       when it
                   causes the system to be cancelled        and results in an XPL.
                   This CEI: occ~: ~+en ‘Ae device error is associate?! with a
                   Ssstcz~ pack.    osviocsly   , the nuz3ez of 1%‘~ attributed     to
                   either    1% o: ITEL CEI; he 2irectly     influenced by the place-
                   men-, of 5s s-•stm
                                  5        packs.
                           ZTZL pers cnnel  indicated     they believe      that system packs
                   we:s located on ITZL disk drives during the rr,oaths when ITZL
                   was cred ired wizh a relatively           l&rge number of IPL’s.           WC
                   pezsoxel      partially    asreed.     WCC personnel       further    described
                   their  policy     of placing    heavy activity     files       (e.g.,  system
                   packs) on the ITZL drives          during the ITS        disk drive acce?t-
                   .&ice period.
               .




                         ICC xovided     E’E!lSi!4 no records of the location    of
               *




                   system files     (such records are not commonly kept hy compu   computer
                   sites).    FEXIX was therefore         amble   to verify  the location
                                                                                 locati
                     f system Dacks.           +he system packs were on ITEL disk
                                                if



                   &ives , hokver      , f53SX     does net believe   that the number of
                   IPL's credited     to ITS could 5e sigr%icaot.
                                  .
                   3. Air Conditioning         at WCC
                        ITZL personnel    expressed concern about the quality            of
           .       air conditioning     at WCC. This concern is apparently            valid.
                   FECSIM personnel     observed that the tczperattzre         near t:le ITEL
                   disk drltrss iscs significantly       higher than the temperature         in
                   other parts of the cczputer       room. FSDSIH did not actually
                   use ternperez-ixc  recordxng    devi:es    to obtain data and thare-
                   fore cannot doc*.zent these tecperature          differences.


                                                                     .

                                                                     25
                                                                                                                                                          _.--                /



rl   ‘*   .

                  .                     ,.’                                                                         .                   .

                                                                                                                                   .




                                  k.    CONC?nSIO::S
                                  1.    VatliPitv                0:     Statistical              E:athods
                                                                                                                                                          ...-.-__ ..--- --
                              -         The-re           is- foZ--Uafsrg-t!x--piSized
                                                                 no -basis               b-ta -t test. t’W-----’
                                  detertim      wharher ox not thure is a 5tatfstically          siszi2icazt                                                     .
                                  difference     berween the petrio,mnces         of ITZL and 13): Dick
                                  drives.      SAX? ccn~lusions in the WC re?cxf are bssezd on
                                  the rao::lfs    cf kaa~~=o~rfsz~     statiszkal      mezhods, they &:QI
                                  invalid.
                                                                                                                                                             I
                                  2.     Validity                Of “?lZ         Drive          Per Mmth”       Analysis               Method,
                                         Usir,gni)er drive 25: mohth” avernge data is in5?po?riata
                                  and misleading.       This method is ifiapproprfate       because it
                                  ,Balse?ly BfS'um?S (1) that degradation       is u~ffor~ly,discri~~tad
                                  amens all disk drivas      and (2) that begra?azion        is   inCspenCent
                                  of device    usage,    The ncrthod is riislmding     bect~ss weraging
                                  so biases              the          deta   that       ITSL      psr,Dormnca       ~~pwzs                  ,dsr WO~SQ
                                  than 13:: p~rfcmexs,                                ever. though      bDeh types of di3k                       drives
                                  cause identfcsl                       degrad5tioa              in oystm   perfomance.

                                  3.
                                      kfter  reviawing     the actual values' in tho WCC report,
                                  FEOSIM coxluP,sd     that several were 50 large thet they WI:Q
                                  unraasoscSlo (Zhay were 2150 found to be                                          inCOnsisteEt                  with
              .           s

                                  KC'5 defizizionj       or were so 5mall they were incocssgwn:inl
                                  2nd shculd be ignored.
                          .




                                  4.      Accurac;,              2nd Corr rslation                of Perforxance             t4easures

                                              :iatthe
                                          drives
                                                       InitLal
                                                           Praq7sm Loads (IPZ’s).
                                                      niaAirs
                                                    have baetn conpreble    since
                                                                                    FZDSIb! concludes
                                                                 of IZL‘s cau5ed by IB!4 and ITEL disk
                                                                                  the reglaccnant    OF a
                                          defective     I?ZL controLler.
                      .                       b.    Hard Fai.15,.   FEZ)SIb: found that hard fails i!ata were
                                              fnconsistont     with the WCC ragout definition.     FSPSIM
                                              colclcludes thet the KC r_eport data cannot be used to
                                              coqare-performnce.
                                                       Soft Fails.                FZDSIM         fcund thet the number of KC disk
                                              Lve        soft falls               was ve:y          small and that the soft ftils'
                                              effect       on systar,             pcx,dcxznce           was inconsequential.
                                                                                                                .
                                                                                                                H
                                                                                                                      _.                I




a   ‘b   c


                                                .      .
                                                                                            .
                                            .




                 gi, ~:c?t.!xi :-:ours. ‘SSI.cl co.?cl.jdes tStt the WC
             - ..=r.w:.: &uadeL-K&s     p:qa.rabzsing         a-diFferant       (aad.-. ----.---                      .. -._.--   __---_.
                 f&lid)    3sthod cf re?ortfng         I TEL dovntise     bows thrcn
                 waz used in the original       report.      The ITS      downtime
                 hours     re~orttsd      wftS this r~ethod                  ware    oi&ffcazatly
                 F;r’,3te:    z.s:r=:: the downtfce   hoers            ra?asted            with     t!ae
                 catSod      used in tsf3 Crf~i.731                 KC report.




                        rhi3 ITEL acce~tar!         #C% rest   3tZZiOd
                       das th3t   the hard          fails    we’re not                                          tad
                       t 96% 0: css h3rd            ,Oail5        occurred



                 tl.     I TZL Acze?ranc%           Test      Trocadure             at WCC.       FE3SiM     Con-
                 cludas =!)a= s:;cr, of tne cata cmt&.zn%d err the WCCrwort
                 were 5rthared oki la tsc ITSL equipe3t W&Bu3d%rgoin;
                 accagtcntx testin+      FE3SiE believes *Aat tSese data
                 should not be used in m evaluation      of ITZL versus 13!C.
                 since  (l)- ths ITEL %qipnont was only recently     installed
                 and needed to “settle     down”, asd (2) the nccegtsnce test
                 ~roc%c?uz’% intentfonelly                   placed      itn unusual            stress     on the

                                                                         FZOSiE:   concludes that
                                                                        on    ITfL em? IBM disk
                                                                        nz!x:     of I?L's aktriSu:ted




                                                             27
                       c. Air C33diticzlnq      at WCC. FEDSIX believes        that the
                       air cont~taonx~y     a: KCC my Save a?verselv       affected    the
                       ITEL disk drives.      FEDS;!< CA,? zearh no delinite      concluriczs
                       abo~~t tSe c;-zality of biz conditionfag,    since no gucatlta-
                       tin  datet were gathered.
- -   __ - - ._         .. - - - ------- -.. -- --. _-____ _   _.._ ----                   __
                  7.   ITEL Disk Drivs           Perfoxmncs
                      FE3S3"1 cor.cludes     that aeither     the WCC resort   r.or WCC
                  m~.~ual loss suppcrt     the belief    tS~t    ITEL disk drive ?erfox-
                  An28 is kf%:iOr       t0 that of IBM disk drives.
                  3,   zx3!LYz:~3?b~ic:~s

                  1. FEDSIM rccomends           that the GAD repest         the WCC to
                  reevaluate     tSe parZormance o,O 1%: and IT2L disk drives.
                  This reevaluaticn        should proceed only if WCC still           believ ‘es
                  that I?Z:I, disk drive perftazm.ace          is ir+ferio=    to that of I 3:;
                  disk   drives.     f,O requested,    FhDSX will       helz, GAO and XC
                  plan knd/cr isl?,lmant        t.Ce evaluation.       ?53SrA further
                  rescxmnds      thst any reeveluation         exclude acceptance test
                  period data.
                  2. FEDEIN recsztxmds                  that ariy effort      to cda.?tify    the net
                  perforzxace         diffexnces           of 131 and IT5L disk drives +a': KC
                  cozsider      the access           time pcrfozzance.          IS!< disk drive s?eci-
                  ficatiox        ir.Zicato         an average access tizne of 30 xrilliseconds.
                  ITZL disk drive s?etifications                     indicate    m average ACCeSS
                  time    of 27 zillisetonds.                 FEDSiX has not verified          these
                  specifications            but has no reason to believe               then incorrect.
                  If these specifications                  are valid,      ITEL disk drives perform
                  better vhtxa seekir,s than do IS?; Cisk drives                       by an average 05
                  3 rilEseconds             per seek.         Although FESSIX did not deterxiae
                  how nanv seek operations                  were executed et WCC, the WCC
                  report inCicbte5             that     a total    of 77,500,OOO I/O o?ezetions
                  WAS executed           in October 1976.            If this nuxber is valid          aad
                   if one-half         the l/O orxrations            required     the execution      of a
                  seek o~exazfon,             38,80b,OOO seek operations             wculd have been
         .        executed.          ITEL disk drives wo~,zld have performd                  these seeks
              ;   an averaGe of 3 r.illiseconds                   per seek faster        than would I3X
          0       disk    drives.          The ITZL disk drives            would berefore        have
                  per f owed    the   seeks     amrox    isately      116,400         seconds   or   ztbout
                  31    hours *faster      durini-the ~03th. (Note                  that the total        X'EL
                  downtime fez October was only 25 fiours.)                          This sigzificazt
                  perforzexe        nd*:a~~c~c sSould be cmsidered                    as il prt    of     any
                  evalustioa       effort.
                                                                           0’




                                                                                .