oversight

Review of the Automated Management Information System of the Veterans Administration

Published by the Government Accountability Office on 1971-09-14.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                            2 ~JNITEDSTATESGENERAL
                                         WASHINGTON,      D.C


CIVIL   DIVISIOM




        B-133044




        Dear Mr.      Johnson:
                The General Accounting        Office    reviewed       the AutqmAted
                                                                            ---
        Management Information         System (AMIS) of the Veterans.Awnis-                              :
        tration      (VA) to determine     if the system was providin.g_VA
        management with reliable         and timely       information.          Our review
        wakmade       at the Central    Office,      Washington,       EC.;     the re-
        gional    offices    in Houston and Waco, Texas;             the Data Processing
        Center     (DPC) in Austin,     Texas;     and the VA hospitals            in Big
        Spring,     Houston,   and Waco, Texas, and in Washington,                   D.C.
                The VA established             AMIS in 1964 because of VA's inability
        to manually        gather     reliable       and timely     information         needed in
        administering         and reporting          on its various        assistance        programs
        for the Nation's          growing       veteran    population        (about 28 million
        at June 30, 1971).             Prior     to the establishment            of AMIS, conso-
        lidated     reports     of comparative          data by station,           activity,       and
        accounting       period    were prepared          manually      at the Central          Office
        from reports        submitted        by VA hospitals,         regional      offices,       and
        other    reporting      stations.
                VA officials    have estimated     that about 8,000 different
        AMIS reports--issued         monthly,    quarterly,      semiannually      and
        annually--will       be processed     in 1972.      On the basis      of information
        available      for 1970, we have estimated          that the annual operat-
        ing costs of AMIS are about $7 million.
                 We believe      that VA has done a commendable         job in develop-
        ing a system for compiling          and reporting      information     on its
        diversified        programs   and operations.       The reports     generated   as
        a result     of AMIS and the opportunity          to have access to informa-
        tion in the AMIS data bank should be of considerable                   value to
        Central     Office     management.
               For several       years VA has been aware that the usefulness
        of AMIS reports        to Central       Office     managers has been impaired
        because of inaccurate           data in the reports.            Because of the
        continued    unreliability          of the AMIS reports,          some Central
        Office    managers,      especially       in the Department        of Medicine  and
        Surgery,    were reviewing,          verifying,       and correcting    AMIS report
        data before     using it,       thereby      precluding    its timely    use.

                                      50TH ANNIVERSARY          1921- 1971
I B-133044.          ,


        Although         VA has initiated     various     actions    to correct
  this  situation,         these actions     have not     been fully    effective
  because:
         1.   Field  station   employees either    have been care-
              less or have not been fully       aware of what they
              are required   to do with respect      to transmitting
              data to DPC.
         2.   Field   station     employees     have not been following
              the procedures        established     to detect   and
              correct    inaccurate       data after   its transmission
              to DPC.

         3.   VA has not been making             periodic    reviews     to
              verify  the implementation              of its internal
              control   procedures.

         4.   VA does not have a system for compiling
              statistics       on the number of errors         found in
              AMIS reports,        thus VA management is deprived
              of perspective         on the magnitude       of the problem
              and the identification            of the stations     consis-
              tently     reporting     inaccurate     data.
  INACCURATE DATA--A           CONTINUING PROBLEM
          During      the period    January     through    April     1968, VA awarded
 contracts         in amounts totaling         $277,000    to a management con-
 sulting      firm to obtain        assistance       in the study,      design,    and
 implementation           of improvements       in the reporting        of AMIS data.
 VA records          showed that    a study was needed because AMIS reports
 for the Department            of Medicine      and Surgery       were untimely,
 inaccurate,          and incomplete.        As a result     of this      study,   certain
 changes in the reports             for the Department           of Medicine     and
 Surgery      were suggested.          It appears      that the study concentrated
 on the type of reporting              desired     by management and did not
 identify        the specific     causes for the unreliability               of the
 data.
        In a report         dated May 1969, the Central      Office  Internal
 Audit    Service       commented on the continuing     problems    concerning
 accuracy,       timeliness,      usefulness of data,   excessive    data,
 and flexibility          of the system and concluded      that the



                                             2
; B-133044'         *


  validity    of AMIS, overall,    was dependent   upon clearing   up
  these problems.     The Internal    Audit Service    recommended that
  high priority    be given to improving     and developing    AMIS to
  the satisfaction    of the users,
         In July 1969     the Chief Medical      Director,     in a memorandum
  to the Chief Data       Management Director,        stated   that there was
  a continuing    need    for accurate     and timely      management reports
  and, in pointing       out his dissatisfaction         with the accuracy    of
  the AMIS reports,       commented as follows:
        "Currently,      there are no meaningful          *** controls     which
        ensure input       completeness      or reasonableness.         Manual
        checking     and correcting       of the reports     after    they
        arrive     at Central    Office    is after   the fact,     is a
        monumental     job, delays      the timely    use of these re-
        ports,     and is never complete."
          As a result    of several    meetings      between officials         of the
  Departments     of Data Management and Medicine              and Surgery,
  actions    were taken in September          1969 to improve the accuracy
  of the data in certain          AMIS reports.       These actions       involved
   (1) additional     use of the computers         to identify      the stations
  that were late in transmitting            data for inclusion         in the
  AMIS reports      and (2) providing       the Central      Office    with a
  consolidated     error   analysis    report     showing the errors         made by
  each station      in transmitting      data to DPC.
         VA procedures      provide  that data input         from field         stations
  be edited       (checked by the DPC computer         for acceptability             for
  input    to the data bank) and that the computer                audit      the rea-
  sonableness       of data before   it is put in the data bank.                   They
  provide     also that,    when erroneous     or questionable            data is detec-
  ted, DPC is to send teletype           messages (referred           to as edit
  and audit      messages)    to the field    stations     requesting          correction
  or clarification        of the data.     VA maintains        statistics         on the
  number of edit        and audit messages sent to field              stations.
          Another'procedure        used by VA to detect         and correct    errors
  in data submitted         by field      stations,      before the data is
  included      in AMIS reports,        provides      for DPC to send all
  field    stations     a monthly     printout      of the data recorded     in the
  data bank for their         stations.         Upon receipt    of the printout,
  the station       is required      to verify      the data and report     any
  corrections       that are needed.
    -                .
B-133044


      Problems with the accuracy      of the data in AMIS reports
have continued,    as evidenced    by the following       excerpts     from an
April  1970 VA teletype     message from the office        of the Controller
to all Department    of Medicine    and Surgery     field    stations:
        "IN ADDITION TO THE RECEIPT OF THE MACHINE-GENERATED
        EDIT AND AUDIT MESSAGES, FOUR CENTRAL OFFICE EMPLOYEES
        SPENT THE MAJOR PART OF THE FIRST SIX WORKDAYS OF
        APRIL, 1970 MAKING TELEPHONE CALLS TO DM&S [DEPART-
        MENT OF MEDICINE AND SURGERY] STATIONS TO ADVISE OF
        IMBALANCES, MISSING SEGMENTS OR OUT-OF-LINE SITUATIONS: ***
                                          *   *   *   *   *

        "THE MISSING SEGMENTS AND OUT-OF-BALANCE ITEMS WILL
        AFFECT OVER 100 AMIS PRINTOUTS WHICH WILL BE RECEIVED
        IN CENTRAL OFFICE DURING APRIL, 1970;   THE AFFECTED
        INDIVIDUAL STATION PRINTOUTS, REGION TOTALS AND NATION
        WIDE TOTALS WILL BE USELESS AS MANAGEMENT INFORMATION."
         (Underscoring supplied.)
IMPROVEMENTS STILL             NEEDED
       We discussed          the problem of inaccurate                    data with Central
Office     officials       in 16 services           in the Departments               of Veterans
Benefits       and Medicine        and Surgery.             About half of the officials,
mostly     in the Department          of Medicine            and Surgery,          told us that
inaccurate         data was still        a problem          and that they found it
necessary        to review and often to verify                    information          in AMIS re-
ports before          they could use the information                      to manage their
operations.           The extent     of such review and verification                         varied
among the services.              The officials           commented, however,              that
their    reviews        of the AMIS reports              generally        would disclose          only
gross inaccuracies             and would not disclose                 all the inaccurate
data.      They stated         that their        verification           frequently        involved
contacting         the appropriate         field       station      and asking         it to
resubmit       the data to DPC.
     One Central      Office    official    informed   us that,    after     receipt
of AMIS reports     for the quarter        ended March 31, 1970, letters
were sent to $80 stations         advising    each of the errors      in the
data that it had reported           and of the need for accuracy         in the
data to be submitted         for the June 30, 1970, year-end          reports.
This official    told us that the letters           had accomplished       little



                                                  4
B-133044


or nothing     and that the year-end        reports    contained      numerous
errors.      He said that correcting        the inaccuracies        required
considerable     time and contributed         to delays in making manage-
ment decisions       and in formulating       his service's      part of the
1972 budget.       The official    pointed      out several    significant
errors   which,    if undetected,     could have distorted          procurement
requirements     and appropriation       requests.
       The continuing        incidences       of inaccurate      data included      in
the AMIS reports        indicate      that field     station     employees   respon-
sible    for preparing       the basic data documents            for submission
to DPC either     do not fully          understand      what they are to do
or are careless       in preparing         the documents.        A teletype    message
dated March 1, 1971, from the Central                   Office   to Department      of
Medicine     and Surgery       field    stations,    illustrates       the continuing
problems    with inaccurate          data and identifies         these two causes
for the problem.          Following       are excerpts       from that message.
       "WE ARE EXPERIENCING CONTINUING PROBLEMS WITH THE
       ACCURANCY OF REPORTING *** THESE PROBLEMS ARE DUE
       TO TWO FACTORS, I.E., THE EMPLOYEES PREPARING THE
       AMIS CODE SHEETS *** ARE MISINTERPRETING HEADING
       INSTRUCTIONS PRINTED ABOVE THE ENTRY BLOCKS ON THE
       CODE SHEETS AND ARE ALSO FAILING TO CHECK THE
       ENTRIES TO ASSURE THAT REPORTED FIGURES ARE MATH-
       EMATICALLY CORRECT.
                                  ******

       "STATION MANAGEMENT WILL ASSURE THAT EMPLOYEES
       RESPONSIBLE FOR PREPARING THIS CODE SHEET FULLY
       UNDERSTAND THESE INSTRUCTIONS AND ALSO ASSURE
       THAT THESE REPORTS ARE MATHEMATICALLY ACCURATE
       PRIOR TO SUBMISSION."
       We found that in some cases employees                responsible        for
implementing       VA's procedures        for ensuring    the accuracy         of
data reported       to DPC were not carrying           out their    prescribed
functions.        Each Department       of Medicine     and Surgery       station
has an AMIS reports         coordinator       who is responsible        for en-
suring     that the monthly printouts            of data,   furnished      by DPC,
are reviewed       for completeness         and accuracy    and that additional
or corrected       data is sent to DPC when omissions              or errors
are noted.        The reports     coordinator      is also responsible           for
ensuring      that appropriate       action    is t                          39 p$lp#j \*E
DPC edit      and audit messages.


                                           5
I   ’
        B-133044         -


               We found a number of errors           in AMIS reports     that should
        have been corrected         by the reports      coordinators    at two
        hospitals.        At one hospital,     where we found 59 errors,           the re-
        ports     coordinator     had not distributed       the month-end     printouts
        for review,       because,   in his opinion,      such a review would have
        required      too much time.       The reports    coordinator    stated      also
        that,     in his opinion,      some of the errors       were not important.
        At the other hospital,          where we found 48 errors,        the reports
        coordinator       had not taken action        to ensure that appropriate
        responses       were being made to all edit         and audit   messages sent
        from DPC.
        CONCLUSIONS
                 We believe       that the incidence   of inaccurate    data in re-
        ports       furnished     for management use should be infrequent        enough
        to permit         management to have reasonable      confidence    in the
        reliability          and accuracy   of the data and to find it not
        necessary         to verify    the accuracy  of the data prior     to using
        it.
               In our opinion,     the basic problem         facing    VA is to eliminate,
        to the maximum extent        possible,    inaccurate       data   before   it is
        reported    by the field     stations   to DPC. We believe           that all VA
        employees    involved    in accumulating,       reporting,      and verifying
        AMIS data should be made aware of the importance                   of accuracy
        in their    work and the need for         them to follow        the prescribed
        procedures.
                Also we believe    that VA should review periodically
        the implementation       of the procedures      for detecting      and
        correcting    inaccurate     data to provide      assurance    that this
        aspect of internal       control  is functioning       as intended.
               VA was compiling       statistics      on the number         of errors
        detected     in the data submitted          by field    stations       to the
        data bank at DPC; however,             these statistics        do   not show
        the full    extent    of the inaccurate         data reported         by stations
        to DPC and give no indication              as to the extent         to which
        the internal      control   procedures       for identifying          and correcting
        inaccurate     data are ineffective.            This fact is        borne out by
        the number of errors        found in the AMIS reports.




                                                   6
                            .
       "   B-,133044


                   We believe    that VA should establish         a system for com-
           piling    statistics     on the inaccurate     data found in AMIS re-
           ports,    because it would give Central         Office    management a more
           complete     picture    of the reporting     of inaccurate      data by
           stations      and a means of identifying       weaknesses     in the pro-
           cedures     for detecting     and correcting    inaccurate      data.
           RECOMMENDATIONS
                   We recommend    that,to     improve   the   reliability       of AMIS
           reports,    you take    action    to:
                 --Establish     a program to educate          and train     VA
                    employees    involved     in AMIS, so that each
                    employee is aware of the importance                of care
                    and accuracy      in following    the prescribed
                    procedures     for accumulating,        reporting,     and
                    verifying    AMIS data and is sufficiently
                    trained    to perform     his function.
                 --Establish       a system for compiling          and report-
                    ing statistics,        by source,       on the number of
                    errors     in AMIS reports        detected    after
                    issuance,     so that Central         Office   management
                    will    be in a better       position      to (1) identify
                    those stations       that consistently         report
                    inaccurate      data and (2) evaluate          the effective-
                    ness of internal         controls     for identifying      and
                    correcting      inaccurate      data sent by stations
                    to the data bank.
                 --Periodically       review the     implementation          of the
                    procedures     for detecting       and correcting
                    inaccurate     data.


              Your attention        is invited      to section    236 of the Legisla-
        tive  Reorganization        Act of 1970 which requires            that you sub-
        mit written     statements       of the action      taken with respect         to
        the above recommendations,,            The statements     are to be sent to
        the House and Senate Committees               on Government     Operations       not'     1.
      > later   than   60  days    after    the  date   of  this  report      and   to the
i 0. .. House and Senate Committees             on Appropriations       in connection         "
-r
        with the first       request     for appropriations       submitted       by your
        agency more than 60 days after              the date of this        report.      We
        shall   appreciate      being furnished        with copies of your state-
        ments to the Committees.

                                                    7
      A        -(   c


"   B-133044


             Copies of this  report    are being sent to the Committees           “> < -d<I _
    mentioned      above, the Senate and House Committees     on Veterans'           --".
    Affairs,      and to the Director,    Office of Management and
    Budget.
         We appreciate       the cooperation     and courtesy     extended   to
    us by VA employees       during  our review.
                                          Sincerely   yours,




                                          Director,    Civil    Division


    The Honorable       Donald E. Johnson
    Administrator       of Veterans Affairs




                                                                                      c




                                              8