oversight

ADP Systems: HCFA's Failure to Follow Guidelines Makes Systems Effectiveness Uncertain

Published by the Government Accountability Office on 1990-07-26.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

              IyttitwI   States   Getwral   Accotttttittg   Office
-.    --d                                                                ‘1

GiO           Report to the Chairman, Subcommittee
              on Labor, Health and Human Services.                   I
                                                                         -
                                                                              ’
              Education, and Related Agencies,
              Comtiittee on Appropriations,
              U.S. Senate
.Jttly i9YO
              ADP SYSTEMS
              HCFA’s Failure to
              Follow Guidelines
              Makes System
              Effectiveness
              Uncertain
_-


GAO
                   United States
                   General Accounting  Office
                   Washington, D.C. 20548

                   Information    Management    and
                   Technology    Division

                   B-239339.1

                   July 26, 1990

                   The Honorable Tom Harkin
                   Chairman, Subcommittee on Labor,
                     Health and Human Services,
                     Education, and Related Agencies
                   Committee on Appropriations
                   United States Senate

                   Dear Mr. Chairman:

                   On January 22, 1990, you asked us to review the development, progress,
                   performance, and cost of a Medicare automated information system
                   known as the Common Working File (CWF). This system is being imple-
                   mented by the Health Care Financing Administration (HCFA), Depart-
                   ment of Health and Human Services, for the stated purpose of
                   improving Medicare claims processing. This report presents our evalua-
                   tion of the CWF system, now being installed nationwide.

                   Our work was performed primarily at HCFA headquarters in Baltimore,
                   Maryland, and at the offices of Medicare contractors, including Blue
                   Cross and Blue Shield of Maryland and Blue Cross and Blue Shield of
                   Texas, the CWF pilot contractors. Details of our objectives, scope, and
                   methodology are discussed in appendix I.


                   HCFA approved and is implementing CWFwithout knowing whether it is a
Results in Brief   sound investment. This uncertainty exists because HCFA did not follow
                   federal and Department of Health and Human Services guidelines on
                   developing new systems.

                   These guidelines specify that before proceeding with a project, an
                   agency should develop a plan that documents the existing program’s
                   problems or needs, identifies alternative solutions, and determines the
                   costs and benefits of the alternatives. Once an alternative is selected, the
                   agency needs a strategy to develop the new system, test it, and identify
                   and minimize the risks of implementing it.

                   HCFA believed it did not have to follow these guidelines and, therefore,
                   did not generate basic information needed during system development.
                   HCFA, for example, never estimated total system costs, did not document




                   Page I                GAO/IMTEG90-63 HCFA Fails to Follow Guidelines on ADP Systems
             B-239339.1




             expected benefits (savings),’ and did not test the system in a way that
             would generate useful information.

             The system is now essentially installed, but will cost an estimated $30
             million per year to operate. If the system cannot generate savings at
             least equal to this amount, then HCFA should consider discontinuing it.


             Medicare is authorized by title XVIII of the Social Security Act. It pro-
Background   vides health insurance to individuals aged 65 and over, and to certain
             disabled people under age 65. Medicareis divided into two parts: part A
             covers inpatient hospital services, skilled nursing facility services, hos-
             pice care, and home health care, while part B covers physician, outpa-
             tient hospital, and various other health services.

             HCFA administers Medicare through contractors (generally insurance
             companies) that process claims, make payments, perform payment over-
             sight, recover erroneous payments, and provide various other services.
             For fiscal year 1989, HCFA estimates (as of May 1990) that Medicare ben-
             efits were about $55.3 billion for part A and $34.6 billion for part B.
             Costs to administer the program in fiscal year 1989 were about $1.3
             billion.

             Prior to CWF, HCFA allowed part A and part B contractors to process and
             determine whether to pay Medicare claims using their own data bases
             and the HCFA master file. The claims information was then processed
             through the HCFA headquarters data base for postpayment review. In
             addition, part A contractors would periodically provide payment data to
             part B contractors so a comparison of payment decisions could identify
             inappropriate payments.

             HCFA began developing CWF in fiscal year 1987 to improve the accuracy
             of Medicare claims processing, reduce overpayments by Medicare con-
             tractors, and capture claims data that can be used to develop a centrally
             managed data base for research and policy development. CWF is essen-
             tially a data base of combined Medicare part A and part B claims histo-
             ries, previously maintained at HCFA headquarters, placed into nine
             regional data bases, called hosts. The hosts are operated by Medicare



              ‘In this report, because the benefits are measurable, the terms benefits and savings are used
              interchangeably



              Page 2                      GAO/IMTEGS@63HCFA Fails to Follow Guidelines on ADP Systems
B-239339.1




part A or part B contractors who maintain CWF data under separate con-
tracts. Thus the hosts are operated by an added layer of contractors
that already have either part A or part B responsibility.

Each host contains information on beneficiaries residing within its
region. HCFA believes CWF'S combined part A and part B data, not previ-
ously available as a prepayment control, will prevent overpayments
involving duplicate bills and inappropriate or medically unnecessary
services. According to HCFA, payment decisions will be improved because
CWF will use more complete and up-to-date information, thereby elimi-
nating overpayments that result when, for example, a contractor is una-
ware that a beneficiary has reached a limit on coverage.

HCFA pilot-tested CWF at two host locations. Blue Cross and Blue Shield of
Maryland helped develop CWF'S software, and began testing the opera-
tion of the system in June 1987. In December 1987 HCFA initiated testing
at Blue Cross and Blue Shield of Texas. Maryland tested whether CWF
would be able to respond to questions from Medicare contractors and
verify their payment decisions. In this test, the only contractors that
queried the data base or sent claims through the host for review and
validation were Blue Cross of Maryland for part A and Blue Shield of
Maryland for part B. The Texas test, however, involved more contrac-
tors and had a dual purpose: (1) to demonstrate that CWF could be physi-
cally moved to and operated at another host location, and (2) to
demonstrate that CWF could function when communicating with Medi-
care contractors located in other states. HCFA considered the tests suc-
cessful Although problems were encountered, such as the need for
additional telecommunications capacity between the Texas host and
HCFA headquarters, HCFA considered the problems to be correctable.

Before part A and part B contractors can communicate with the CWF
host, the contractors must convert their systems’ software to be compat-
ible with the host system. As of December 1989 CWF had been installed
at all nine host contractors, and 38 percent of the Medicare parts A and
B contractors were operational.




Page3               GAO/IMTEG99-53HCFAFaUstiFollowGuidelinesonADPSystems
                       B-239339.1




                       In developing the CWF system, HCFA did not prepare a plan as outlined by
HCFA Has Not           governmentwide guidelines and reinforced in a Department of Health
Followed               and Human Services (HHS) manual.* This is not an isolated incident; HHS
Governmentwide         officials acknowledge that, in the past, HCFA has not followed automated
                       data processing guidance and HHS has not attempted to enforce compli-
Management             ance. According to HHS officials, HCFA believes that because it employs
Procedures             contractors to process claims, rather than operate its own claims-
                       processing facility, it does not have to follow this guidance.

                       In our opinion, however, because HCFA is mandating and funding the
                       system’s development, HCFA should follow federal guidelines. Not doing
                       so has contributed to HCFA'S inability to substantiate project benefits and
                       may have contributed to increased software and telecommunications
                       costs. HHS management supports the need for adequate planning and
                       agrees that HCFA should have prepared a plan and documented benefits
                       for CWF.


Federal Information    The Paperwork Reduction Act of 1980 (P.L. 96-51 l), as amended, pro-
                       motes the effective management of automated data processing resources
Management Standards   in the federal government, emphasizing information as a resource with
                       associated costs and benefits. The HHS manual, which provides guidance
                       to its component agencies, cites this act in emphasizing the importance
                       of proper planning in the successful development of automated systems,
                       such as CWF, and recommends that management thoroughly document
                       all planning and development steps. The manual further states that HHS'
                       organizational components, including HCFA, shall use the National Insti-
                       tute of Standards and Technology Federal Information Processing Stan-
                       dards publications (FIPS PUBS) in planning and developing an automated
                       information system. However, HHS has not actively ensured compliance
                       with this manual and HCFA has, at least with CWF, not followed it.

                       FIPSPUBS 38 and 64 identify three phases in an automated system’s life
                       cycle-initiation, development, and operation -and stress the impor-
                       tance of documenting these phases as a project progresses. The initiation
                       phase establishes the objectives and general definition of the require-
                       ments for the software, including feasibility studies and cost/benefit
                       analyses. During the development phase, the specific requirements for
                       the software are determined and the software is then defined, specified,


                       ‘Department of Health and Human Services Information Resources Management Manual (Nov. 1,
                       1985).



                       Page4                    GAO/JMTEG9O-63HCFAFailstoFollowGuidelinesonADPSyst.em
__-~
                          B-239339.1




                          programmed, and tested. In the operational phase, the software is main-
                          tained, evaluated, and changed as additional requirements are identi-
                          fied. At the beginning of 1990, CWF was in the development phase,
                          moving into the operational phase.


Federal Guidelines Also   We found that HCFA had not adequately documented system initiation
Not Followed During CWF   and development. In starting the project, for example, HCFA did not
                          clearly define the need for CWF, or perform a feasibility study to identify
Development               possible alternative solutions and their costs and benefits.

                          The agency likewise did not prepare a risk analysis during the initiation
                          phase to identify system vulnerabilities that could endanger either the
                          system or its program data. This analysis is to be reviewed and revised,
                          as necessary, during each phase of the system development life cycle to
                          assure that appropriate security measures are installed.

                          Federal standards provide that, at the onset of the development phase,
                          potential users provide input to system planning so that, to the extent
                          possible, the system will serve their needs. We met with officials of
                          HCFA'S Health Standards and Quality Bureau and HCFA'S Bureau of Data
                          Management and Strategy. These organizations use Medicare data in
                          planning or policy-making. According to these users, they had little
                          opportunity to influence decisions about what type of information
                          would be generated by CWF or how that information would be made
                          available to them. These users expressed concern that if CWF did not
                          provide sufficient information in a usable format, they would be unable
                          to fully monitor health care trends.

                          We also met with the Physician Payment Review Commission, which
                           advises the Congress on physician payment under Medicare. Officials
                           were uncertain if information critical to them, such as physicians’ iden-
                          tifier numbers and zip codes, would be available from CWF. These users
                          expressed concern that without such information, some of their statis-
                           tical studies and trend analyses would either be inaccurate or could not
                           be conducted at all. Although they have relayed this concern to CWF pro-
                          ject managers, they do not know how it will be resolved.

                          HCFA officials responsible for managing this project acknowledge that
                          these procedures had not been followed and the recommended docu-
                          ments not prepared. These officials are primarily responsible for man-
                          aging claims processing and are relatively unfamiliar with the standards
                          for developing automated information systems. Once we informed them


                          Page 5               GAO/lMTECXO-53HCFA Fails to Follow Guidelines on ADP Systems
___.   .~
                        Bz39339.1




                        of the standards, they still did not believe that the guidelines applied to
                        this system because the system’s primary purpose was to process
                        claims. The objective of the guidelines, however, is not to distinguish
                        systems according to their purposes, but to ensure that all systems are
                        effectively designed and implemented.


                        HHS guidelines provide that the cost for an automated information
Cost of CWF’ Was Not    system be estimated during the initiation phase of a project’s develop-
Initially Established   ment. HCFA, however, did not estimate project development costs before
and Continues to        initiating CWF. Instead, HCFA budgeted annually for CWF as a productivity
                        improvement item within the Medicare contractor section of the
Increase                agency’s annual budget. Through interviews with HCFA project officials
                        and reviews of the budget documentation, we found that CWF, from ini-
                        tial project funding in fiscal year 1987 through its expected completion
                        in fiscal year 1991, will cost about $103.8 million.

                        CWF costs have continued to increase because of system enhancements
                        and telecommunications. When HCFA submitted its 1990 budget it did not
                        contain reference to additional development costs, and indicated that
                        CWF would be fully implemented in fiscal year 1990. However HCFA cur-
                        rently plans to spend $11 million in fiscal year 1991 on system enhance-
                        ments, which include software changes designed to correct system
                        operational problems and to provide for planned benefits. These
                        software changes were apparently not anticipated by HCFA one year ear-
                        lier. In addition, the expected cost of telecommunications has risen dra-
                        matically. In March 1989 HCFA estimated that CWF telecommunications
                        would cost $2.6 million per year. In fiscal year 1990 HCFA revised this
                        estimate upward. As of May 1990, HCFA estimates CWF telecommunica-
                        tions will cost $5.8 million in fiscal year 1990 and $9 million in fiscal
                        year 1991. Table 1 presents CWF'S costs during fiscal years 1987-1991.




                        Page 6               GAO/IMTECXO-53HCFA FaiJsto Follow Guidelines on ALIP Systems
                                       B-239339.1




Table 1: CWF Costs From Fiscal Years
1987 Through 1991a                     Dollars in millions
                                       Fiscal Year                   cost       Category of Expenditures
                                       1987                          $4.6       Development of CWF software, pilot operations
                                       1988                           7.6       Development of CWF software, pilot operations
                                       1989                                     Host operations, maintenance, system conversions,
                                                                     23.9       telecommunications
                                       1990                          34.6       Host operations, maintenance, system conversions
                                       1991                                     Host operations, enhancements, system conversions,
                                                                      33.1      telecommunications
                                       Total                      $103.8
                                       Tests   for fiscal years 1987-1989 are actual. Costs for fiscal years 1990 and 1991 are HCFA estimates



                                       Although HCFA has justified CWF on the basis of the system’s expected
HCFA Could Not                         financial savings, the agency could not provide any documentation sup
Document Expected                      porting these savings. In addition, HCFA did not develop any actual data
Savings                                on CWF savings during pilot testing. This absence of evidence calls into
                                       question the validity of HCFA'S savings estimates.


Estimated Savings From                 HCFA estimates that CWF will provide about $145 million in annual Medi-
                                       care program savings and administrative cost savings. According to
CWF                                    HCFA, the existing process cannot identify all payment errors. The
                                       largest portion of estimated savings-about $72 million-is expected to
                                       be derived from reducing payment errors by comparing claims sub-
                                       mitted by the contractors with combined Medicare part A and part B
                                       history files. Other projected savings include better identification of pri-
                                       vate insurance coverage to ensure that Medicare pays last in the Medi-
                                       care secondary payer program; elimination of overpayments written off
                                       because of their small size; interest on overpayments to beneficiaries;
                                       improved medical review based on more complete beneficiary data; and
                                       better identification of payments for the duplicate purchase of certain
                                       medical equipment and other services. HCFA officials also project that
                                       CWF will provide $20 million in annual savings because of a reduction in
                                       contractors’ administrative costs. Table 2 summarizes the expected
                                       annual savings attributable to CWF.




                                       Page 7                        GAO/IMTEG90-63 HCFA Fails to Follow Guidelines on ADP Systems
__    ..--
                                         B-239339.1




Table 2: HCFA’s Estimate of Annual CWF
Savings                                  Dollars in millions
                                         Activity                                                                      Amount
                                         Parts A/B data exchange                                                            $72
                                         Improved medical review                                                             12
                                         Eliminated write-off of small overpayments                                          15
                                         Improved secondary payer identification                                             15
                                         Lost interest on overpavments                                                        7
                                         Identifying duplicate payments                                                       4
                                         Program savings                                                                  $125
                                         Administrative   savings                                                           20
                                         Total annual savinas                                                             $145



HCFA Lacks Data to                       HCFA could not provide data to show that combining part A and part B
                                         data, through a common file, will produce the estimated program sav-
Support the Projected                    ings. Although federal guidelines recommend that a system be thor-
o,.,,:“.r(e
LXLV 1rtg;a                              oughly tested, and that test results and findings be documented, HCFA
                                         did not follow these guidelines. HCFA'S tests did not document the
                                         system’s ability to generate savings. As a result, HCFA is unable to accu-
                                         rately estimate what savings will result.

                                         At the time that CWF was pilot-tested in Maryland, the system did not
                                         include software to identify and gather data on actual program savings.
                                         Similarly, HCFA did not gather actual savings data during the pilot test at
                                         Blue Cross and Blue Shield of Texas. HCFA believed that, because Texas
                                         already used a combined part A and part B system before CWF, savings
                                         attributable to CWF may have been difficult to measure if, in fact, they
                                         existed at all. HCFA plans to implement a project this year to gather
                                         national data on CWF savings. As of May 1990, however, the agency had
                                         not yet taken action.

                                         In addition, HCFA estimates that the annual cost of operating CWF will be
                                         about $30 million. This amount more than negates the $20 million in
                                         administrative savings HCFA expects from the system’s implementation.


                                         HCFA decided to implement CWF before it considered alternatives, esti-
Conclusions                              mated costs, or documented savings. This decision is inconsistent with
                                         HHS and governmentwide procedures for developing automated informa-
                                         tion systems, and also with good management practices. These proce-
                                         dures-embodied in federal and HHS guidelines and regulations-are       not



                                         Page 8                     GAO/IMTEG90-53 HCFAFails to Follow Guidelines on ADP Systems
                      5239339.1




                      intended to suggest doing paper exercises or documenting decisions
                      already made. Instead, they should be an integral part of, and influence
                      on, the decision-making process. Ultimately, they should be used to
                      decide whether or not to make an investment.

                      Without proper planning and documentation, the acquisitions of com-
                      plex automated systems may encounter problems. HHS management
                      acknowledges that HCFA should have followed the Department’s gui-
                      dance when developing CWF.

                      HCFA'S failure to follow government procedures for developing informa-
                      tion systems leaves the agency unable to determine whether CWF is cost-
                      effective, or whether it will achieve its objectives. At this time, procure-
                      ment and deployment of CWF has essentially been completed. All that
                      HCFA can do now is gather CWF savings data and assure itself that the
                      system will achieve savings at least equal to its estimated annual oper-
                      ating cost of $30 million. If not, HCFA is faced with the decision of
                      whether to continue CWF.More importantly, HCFA can learn from this
                      experience with CWF, and avoid getting into such situations in the
                      future. Properly following governmentwide and HHS system develop-
                      ment procedures would help ensure that funds for future systems are
                      spent in the most effective and efficient manner.


                      In order to assure that the CWF is cost-effective and to limit the risks
Recommendations       inherent in developing information systems, we recommend that the Sec-
                      retary of Health and Human Services direct the Administrator, Health
                      Care Financing Administration, to

                      evaluate the cost-effectiveness of continuing to operate the CWF system,
                      and if the system is found not to be cost-effective to determine what, if
                      any, alternatives exist; and
                      follow federal information system development standards in all future
                      system modifications and acquisitions, seeing to it that (1) all phases of
                      a project’s development are adequately documented, (2) system costs
                      and benefits are fully identified and justified, and (3) the system is ade-
                      quately tested.


                      The Department of Health and Human Services generally concurred
Agency Comments and   with our recommendation that HCFA needs to establish the cost-effective-
Our Evaluation        ness of CWF or determine what alternatives, if any, exist. The Depart-
                      ment stated, however, that it did not believe that cost-effectiveness


                      Page 9               GAO/IMTEC90-63 HCFA Fails to Follow Guidelines on ADP Systems
-
    BZ39339.1




    should be a sole criterion for building or retaining a system. The Depart-
    ment indicated that HCFA is conducting a study of CWF’S cost-effective-
    ness and that it will use these results to enhance the system. Also, the
    Health and Human Services Inspector General is conducting an evalua-
    tion of alternatives for the future. The Department plans to use this
    study to develop requirements for CWF’Seventual successor.

    We believe that not paying sufficient attention to the cost/benefit anal-
    ysis can result in agencies acquiring ineffective and inefficient auto-
    mated information systems. Federal guidelines, such as the National
    Institute of Standards and Technology Federal Information Processing
    Standards, emphasize the importance of the cost/benefit analysis in
    making the critical decision to begin the development and implementa-
    tion of an automated information system. While not the sole criterion,
    the cost/benefit analysis allows managers to make more informed deci-
    sions on whether to commit scarce resources to a project. The guidelines
    further emphasize the need to consider the cost/benefit of several alter-
    natives before initiating a project.

    The Department further concurred that HCFA should follow federal
    information system development standards in all future automated
    information system modifications and acquisitions. It noted, however,
    that when CWF was initiated in 1986, HCFA viewed CWF as a contractor
    system and, as such, not subject to information resource management
    guidelines. The Department indicated that this position is now being
    reexamined in light of recent events, such as a prior report we issued on
    automated information system cost reporting.”

    We believe that the above federal guidelines should have been applied to
    CWF from the StarL CWF iS HCFA’S SySk!In, not a CO~traCtOI’ SySkIn; HCFA
    has controlled CWF from its initiation and continues to maintain total
    control of the software. Given these facts, we disagree with the conten-
    tion that CWF development was ever exempted from the application of
    federal guidelines.

    Detailed Department of Health and Human Services comments and our
    evaluation are contained in appendix II.




    “information Technology:Health CareFinancingAdministration’s BudgetProcessNeedsImprove-
    ment (GAO/IMTm’;gS-31, Aug. 11,1989).




    Page10                   GAO/IMTEGW        HCFAFailstoFollowGuidelinesonADPSystems
We are sending copies of this report to the Chairmen of the House Com-
mittee on Appropriations, Senate Committee on Governmental Affairs,
and House Committee on Government Operations; the Secretary of
Health and Human Services; and the Administrator, Health Care
Financing Administration. We will also make copies available to other
interested parties upon request.

This report was prepared under the direction of Frank Reilly, Director,
Human Resources Information Systems, who can be reached at (202)
275-3462. Other major contributors are listed in appendix III.

Sincerely yours,




Ralph V. Carlone
Assistant Comptroller General




Page 11             GAO/IMTEC90-53 HCFA Fails to Follow Guidelines on ADP Systems
Contents


Letter                                                                                               1

Appendix I                                                                                          14
Objectives, Scope, and
Methodology
Appendix II                                                                                         16
Agency Comments and GAOco~ents                                                                      22
Our Evaluation
Appendix III                                                                                        24
Major Contributors to
This Report




                         Abbreviations

                         BDMS     Bureau of Data Management and Strategy
                         CWF      Common Working File
                         FIPS PUB Federal Information Processing Standards Publications
                         GAO      General Accounting Office
                         HCFA     Health Care Financing Administration
                         HHS      Department of Health and Human Services
                         IMTEC    Information Management and Technology Division


                         Page 12            GAO/JMTJ3C9O-53
                                                          HCFA Fails to Follow Guidelines on ADP Systems
--




     Page 13   GAO/IlWEG9O-53 HCFA Fails to Follow Guidelines on ADP Systems
Appendix I

Objectives,Scope,and Methodology


              We initiated our review of the CWF project because it represented a
              major change to the processing of Medicare claims. On January 22,1990,
              we received a request to review the progress, performance, and costs of
              CWFfrom the Chairman, Subcommittee on Labor, Health and Human
              Services, Education, and Related Agencies, Senate Committee on Appro-
              priations. Our objectives were to assess (1) CWF'S performance, benefits,
              and costs; and (2) HCFA'S management of the project’s development.

              To obtain information on the CWFproject, we met with Bureau of Pro-
              gram Operations officials at HCFA headquarters in Baltimore, Maryland,
              and reviewed acquisition planning documentation. In addition, we met
              with officials of selected contractors. We interviewed officials of Blue
              Cross and Blue Shield of Maryland and Blue Cross and Blue Shield of
              Texas because these sites were chosen by HCFA to pilot-test the CWF pro-
              ject. We interviewed officials of Empire Blue Cross and Blue Shield in
              New York because Empire is a large host contractor and because HCFA
              chose Empire to be the CWFmaintenance contractor. We met with offi-
              cials of Blue Cross and Blue Shield of Massachusetts to discuss a shared
              processing arrangement that the contractor had entered into with other
               New England Blue Cross plans to process their claims, a preliminary
               step to implementing CWF,and to identify how this arrangement would
               affect CWF.We further discussed this arrangement with officials from
               the HCFA Boston Regional Office. We also met with officials of Aetna Life
               and Casualty Insurance Company in Connecticut because Aetna is a
               multistate contractor covered by five cw~ host sectors, and we wanted
               to determine how CWF would operate in this environment.

              To determine the requirements for effectively managing an automated
              information system, we reviewed Publications 38 and 64 of the National
              Institute of Standards and Technology Federal Information Processing
              Standards, and the Dewrtment of Health and Human Services’ Informa-
              tion Resource Management Manual guidelines for initiating and devel-
              oping an automated system. To evaluate the responsibilities of senior
              information resources management officials, we also reviewed the
              Paperwork Reduction Act of 1980, the Paperwork Reduction
              Reauthorization Act of 1986, and Federal Information Resources Man-
              agement Regulations. In addition, we contacted the Health and Human
              Services’ information resource management official responsible for HCFA
              expenditures on automated information systems to obtain his opinions
              of whether HCFA had adequately followed the Department’s information
              resource manual guidelines.




              Page 14              GAO/IMTECr9o-63HCFA Fails to Follow Guidelines on ADP Systems
Appendix 1
Objectives,   Scope, and Methodology




We also incorporated Department of Health and Human Services and
HCFA’S comments obtained in July 1990. Our review was conducted in
accordance with generally accepted government auditing standards,
from July 1989 through May 1990.




 Page 15                     GAO/lM’R$C~   HCFA Fails to Follow Guidelines on ADP Systems
Agency Comments and Our Evaluation


suppienentmg     those In the
report text appear at the
end of this appendix             DWARTMENT   OF   HEALTH& HUMAN    SERVICES                   Olllc*   01 Yspaan   ouluti




                                                                       JUL 21990

                                 Rr. Ralph V. Carlone
                                 Assistant    Comptroller General
                                 United States General
                                    Accounting office
                                 Washington, D.C. 20584
                                 Dear Rr. Carlone:
                                 Enclosed    are the Department'e   comments  on your draft report,     'ADP
                                 Systema:     ?iCFAgs Failure to Follow Guidelinrs    Rakes System
                                 Effectiveness     Uncertain."   The comments  represent  the tentative
                                 position    of the Department and are subject to reevaluation        when
                                 the final version of this report is received.
                                 The Department appreciates  the opportunity            to comment on this
                                 draft report before its publication.
                                                                   sincerely   yours.



                                                                   Richard P. Xuoserow
                                                                   Inspector General
                                 Enclosure




                                                                  15




                                Page16                    GAO/lMTEG9lM3HCFAFailstoFoUow                  Guidelines         onADPSystems
        AppendixII
        AgencyComments
                    andoUr Evaluation




Comment8
partment of the De
                on the General Accounting Office Draft Report,
            "ADPSystems: Health Care Financing tiini8tration's
              (HCFA’s) Failure to Follow Guidelines Makes System
                            FffeCtiVelM?SS Uncertain”

We are pleased to cement on the subject report.   We \aderstand the GAO
concern for procedural safeguards, but believe the actual result of the
Comon Working File (CWFI initiative  speaks for the system's careful
design, testing and implementation.  We retain our conviction that 134~
represents a major advance in the way HCFA conducts business.
We would first like to make clear why HCFA implemented CWF, and the
benefits and opportunities     it provides in the volatile     Medicare
environment.     In 1987, major components of HCFA's central claims operations
were 15-20 years old, cunberscme and inefficient.        Elaborate and expensive
postpayment error handling processes were required to maintain the system.
Data were incomplete and took months ta assemble. CWFsimplified
operation8 by establishing a standard prepayment claims authorization
process utilizing    complete beneficiary entitlement,     eligibility     and
utilization    data. Thernovementto prepayment authorization           is perhaps the
most significant    change in claim Operation8 since the inception of
Medicare. HCFAcan nw effectively         control claims processing and en8ure
uniform benefit administration.
Beyond this, CWFfacilitates   policy development via its database, increases
service to contractors and beneficiaries   and strategically positions HCFA
to deal with inevitable program changes directed by Congress. The system
was consciously designed not just to deal with known requirements, but also
to be able to hsndle the unknowable. This concept grew out of close
coordination with potential users of the system and a comitment to be
responsive to their needs. We believe its response to the challenges of
Catastro&ic Health Insurance Coversge and physician payment Reform
validates the HCFAapproach.
The report establishes a standard of effectiveness based only on COSt-
savings. While HCFAbelieves significant      savings are obtainable, the
decision to pumue the project looked beyond classical return on investment
calculations  which attempt to recover all expenditures from direct savings.
The capital investment in CWFis viewed as ah investment in better quality,
higher levels of service and greater flexibility    to deal with the future.




         Page17                 GAO/lMTEC9063 HCPAFailstoFoUowGuidelineson ADPSystem
         Appendix II
         Agency Commentsand Our Evaluation




Page 2
GAORecomaendation
In order to ensure that the CWFis cost-effective   and to limit the risks
inherent in developing information systems, we recognnend that the Swretarv
of Health and HumanServices direct the Administrator,   HCFX. to:
--    evaluate the cost-effectiveness   of continuing to onerate the cWF
      system, and if the system is found not to be cost-effective   to
      determine what, if any, alternatives   exist: and
Department Cknmaent
We concur in part with the recommendation. HCPAconducts an on-going
program of cost-effectiveness    reviews and will certainly       include CWPin
those processes. In addition, the HHSInspector General is currently
engaged in a study of CWFwhich includes an evaluation of alternatives            for
the future.    We will use the results of those studies to enhance CWFand
develop requirements for its eventual successor. We note that CWPhas
never exceeded its annual budget. Moreover, CWFis l~lw the linchpin for
all contractor claims processing operations and is the only system’
available which supports the dynamic Medicare benefit administration
environment. As noted above, CWFalready provides HCFAwith significant
benefits which are quantitative,     qualitative   and strategic.      We do not
agree with the premise that administrative       cost-effectiveness     is the sole
criterion   for building or retaining a system.
GAORecomnendation
--    follow Federal information system develounent standards in all fut
      system modifications and acouisitions,   seeing to it that (1) all
      phases of a project’s develorxnent are adeauately documented,
       (2) system costs and benefits are fully identified  and
      justified,  and (3) the system is adeauately tested.
Department &nnent
Weconcur in the GM recommendation to base the future development of cWF
on established formal procedures. When the CWPproject was initiated        in
1986, HCFAwas sure  that, as a contractor benefit administration    system,
CwFwas not subject to Infomtion     Resource Management (IPkl) guidelines.
More recent events, such as the GAOreview of HCFAADPcost reporting,




          Page 18                GAO/IMTEGSO-53HCFAFails to Follow Guidelines on ADP Systems
                         Appendix II
                         Agency Commentsand Our Evaluation




                page3
                planning for drug bill processing under the Medicare Catastrophic
                legislation     and this report , indicate that a clear dichotomy between
                Federal Systems and Medicare contractor systems laay no longer exist.     HCFA
                will,     therefore, reexamine its Medicare systems in relation to the IR4
                requirements to determine to what extent and in which instances IF@l
                guideline8 will apply. Where the guideline8 do apply, we will certainly
                follow the recammndations.
                Other Conrnenta
See comment 1   page3    - (%F was not developed to "create a data base for research and
                           policy development." Rather, HCFAcapture8 claim8 data as they
                           are processed through the CWFhost sites and uses this by-
                           product to develop a centrally  held claims database which
                           supports those functions.

See comment 2   page6-     We disagree that benefit8 were not docueented. We furnished the
                           review team with our docMent.ation and statements of the
                           methodology u8ed to estimate savings where actual data were
                           unobtainable.   The chart on paOe 12 of the report reflects
                           HCFA's conservative estimate of benefits.   The benefits of CWF
                           were certainly identified  and documented a8 best we could. We
                           reiterate our belief that benefits mean much more than cost
                           savings alone.
See comment 3   Pape7-     HCFA's Bureau of Data Msnagement and Strategy (EMI had ample
                           opportunity to influence data decisions for '%P. ~      was
                           actively involved from the outset in the developnt    snd design
                           of c!G. BIBS worked closely with HCPA's Wwsu of program
                           Operations to ensure that the data being gathered was proper,
                           both in terms of its definition   and fotmst. Furthermore, BEM
                           staff is directly   involved in the design, development and
                           maintenance of over forty core progranrning modules now utilized
                           by CWFsites.      BUMSis the gatekeeper for all HCFAprogtwmnatic
                           data and will play a major role in the systems use and
                           evolution.
See comment 4   page8-     'Ihe paraoraph on the pE%Cshould be deleted.  CWFprovides 100
                           percent of all claims data intact to the HCFA statistical
                            systems.




                          Page19                GAO/IMTEG99-63 HCFAFails to Follow Guidelines on ADP Systems
                                  Appendix II
                                  Agency Commentsand Our Evaluation




                       PBge 4
See   ‘. r. f,-’   5   Page 9 -     HCFAha8 been highly successful using competitive processes (for
                                    both CVF maintenance snd host operations) to achieve operational
                                    cost efficiencies.
                                   Telecammmications costs for CWFdo exceed estimates.      This is
                                   not, however, an operational problem in CWF. ‘Ihe combination of
                                   moving from SSAbased telecomnunications     to the HCFAData Center
                                   and the major expansion of the part B record to better support
                                   initiatives   such a8 effectiveness studies 80 changed the HCFA
                                   telecossnunicstions environment that our working asswsptions for
                                   calculating   future costs were flawed.
                                   The report’s position on CWFenhancements seems to be based on a
                                   belief that “operational”   means ” in its final configuration. *’
                                   This wm8never our intent.     The system is already operational,
                                   processing well over 1 million claims daily.      We always planned
                                   to incrementally e.xp8nd CWFfunctionality.      Further, it is
                                   incorrect to report that HCFAdid not anticipate certain
                                   enhancements. Nsjor changes such as host-to-host
                                   telecommunications and database restructuring     have long been
                                   part of the system plan. They are being phased-in to meet
                                   changing needs, such a8 transaction growth, in a msnner that
                                   will not er&nger the system’s overriding priority       of claims
                                   payment. Finally, such enhancements compete with other Medicare
                                   requirements for funding each year.

See comment        6   Page 10 - CWFcost data for FYs 87 snd 88 include costs
                                 for pilot operation8 in Maryland and Texas. HCFAbelieves the
                                 software development and maintenance costs for a system of this
                                 complexity are reasonable based on our experience with other
                                 Medicare claims processing systems. GAOwas provided with
                                 savings estimates from the pilot tests.
See comment        7   Page 12 - The lack of savings data is in no way related to testing
                                 deficiencies.   CWFwa8 elaborately tested; i ,e., tested before
                                 installation  at Harylsnd, tested in pilot production at
                                 Maryland, further tested in beta production at Texas and tested
                                 in a remote environment at Arizona. The problem in docmntine
                                 savings comes from the structure of CWFitself.      It is a
                                 transaction processing system. It authorizes or rejects
                                 payment, but keeps records only of authorized transactions.




                                  Page20                 GAO/IMTEG90-63 HCFA Fails to Follow Guidelines on ADP Systems
                        Agency Comments   and Our Evaluation




                           Savings are actualized at the individual   contrsctors.
                           HCFAbelieved that the case for program savings was so
                           obvious that incurring significant   costs to document
                           them in advance could not bs justified.
See comment 8   Page 13 - The report contends that the $30 million CWFcost "negates" the
                          $20 million administrative    savings, Thus, the current
                          incremental cost of the system is established at approximately
                          $10 million.    We point out that cost/savings is rather volatile
                          in Medicare. The savings would have been $33 million until the
                          repeal of the Catastrophic Coverage tit of 1988 (CCA) eliminated
                          the need to track and inform beneficiaries     of progress toward
                          the Part B cap. 'Ihe physicisn payment reform requirement to
                          track volune perfoxmsnce standards wss inconceivable prior to
                          CWF. We have not even attempted to price the cost of
                          administering payment refono'without     CWF, 80 as to claim
                          savings. Again we assert the real value of benefits which are
                          not quantifiable.    WhenCCA was repealed, the reestablishment of
                          Part A spell of illness processing was so difficult       that it
                          would have taken until tiy 1990 to properly process inpatient
                          bills without CWF. The cost to HCFA had those bills been held
                           up for months is incalculable.    These exsmples reinforce our
                           position, stated above, that the flexibility    built by design
                           into CWFis its great strength.
See comment 9   Psge 14 - We believe that we have a highly functional,   robust claims
                          authorization  system which supplies terabytes of timely,
                          detailed program data for use by Congress, the Administration
                          and the pPRCin directing the future of Medicare and which
                          appears to be a suitable base for sustaining operations into the
                          next century.




                         P8ge 21                  G.40/IMTEG90-62 HCFA Fails to Follow Guidelines on ADP Systems
                                                                                          .-
               Appendix Jl
               Agency Commentsand Our Evaluation




               1. We agree that the language suggested by the Department is more pre-
GAO Comments   cise than the language we used; accordingly, we have modified our
               draft.

               2. We believe that the documentation of CWF'S benefits should have
               included supporting evidence that benefits can be achieved. We
               reviewed HCFA files and interviewed HCFA program officials but were
               unable to obtain this type of support for the purported benefits. HCFA
               officials indicated that they could not measure actual savings resulting
               from CWF. We have no way of knowing, therefore, whether HCFA'S esti-
               mate of benefits is conservative or inflated.

               3. We discussed CWF development with the Bureau of Data Management
               and Strategy (BDMS) information resource management officials and do
               not agree that HCFA'S BDMS was sufficiently involved in the identification
               of CWF requirements and its design and development. While BDMS appar-
               ently had more interaction with BP0 in later stages of CWF's develop-
               ment and implementation, BDMS officials indicated that they had little
               opportunity to provide input during CWF'S design.

               4. The Physician Payment Review Commission is an important user of
               information collected by HCFA from the claims process, and was identi-
               fied by the CWF project staff as a user involved in developing CWF. We
               disagree that this paragraph should be deleted. At issue is not whether
               data are provided, but whether they meet the needs of users.

               5. We believe that continued improvements to CWF are acceptable. How-
               ever, we believe that a significant portion of the cost increase shown in
               HCFA'S fiscal year 1991 budget is for projects to make CWF operational
               rather than to improve it.

               6. Table 1 has been changed to show that the costs HCFA incurred for CWF
               in fiscal years 1987 and 1988 also include CWF pilot operations.

               7. The most basic element of testing is to determine if a system will per-
               form as intended. A major indicator of satisfactory performance for CWF
               is program savings, which was HCFA'S basis for implementing the system.
               HCFA officials told us, however, that the testing performed was intended
               to determine only if CWF would function. They acknowledged that the
               test was not intended to determine if CWF achieved anticipated savings,

               8. We agree that the benefits resulting from a systems development
               effort may not be easily quantified. However, that does not negate the


               Page 22                GAO/IMTEG9@33HCFA Fails to Follow Guidelines on ADP Systems
Appendix II
Agency Comments   and Our Evaluation




need to assess potential benefits to determine the direction and level of
investment in a systems development effort. HCFA'Sjustification for CWF
was limited to cost savings and did not include any discussion of poten-
tial benefits. Therefore, in reviewing HCFA'Sdocumented benefits, we
were limited to a review of that information. We could not assess poten-
tial benefits that HCFAdid not document.

9. We agree with HCFAthat cw is functioning and supplying informa-
tion. However, whether CWFis cost-effective, or whether it will be able
to achieve its objectives, has not been established. Properly following
governmentwide and HHSsystem development procedures would have
provided greater assurances that CWF was the most suitable base for
sustaining the future claims operations of Medicare.




Page 23                   GAO/IMTEG9O43 HCFA Fsils to Follow Guidelines on ADP Systemm
Appendix III

Major Contributors to This Report


                         Douglas Nosik, Assistant Director
Information              Ronald Yucas, Senior Evaluator
Management and           Michael Resser, Evaluator
                         Mary T. Marshall, Reports Analyst
Technology Division,
Washington, D.C.

                         William A. Moffitt, Regional Management Representative
Boston Regional Office   Donald P. Benson, Evaluator-in-Charge
                         Susan A. Fleming, Evaluator




(510450)                 Page 24             GAO/IMTEGSO-63HCFA Fails to Follow Guidelines on ADP Systems
Requests for copies of     GAO   reports   should be sent to:

U.S. General Accounting      Office
Post Office Box 6015
Gaithersburg, Maryland      20877

Telephone   202-275-624 1

The first five copies of each report       are free. hddit ional copies are
$2.00 each.

There is a 254, discount    on orders for 100 or more copies mailed to a
single address.

Orders must be prepaid by cash or bj check or money order made
out to the Superintendent of Documrnt s.