IyttitwI States Getwral Accotttttittg Office -. --d ‘1 GiO Report to the Chairman, Subcommittee on Labor, Health and Human Services. I - ’ Education, and Related Agencies, Comtiittee on Appropriations, U.S. Senate .Jttly i9YO ADP SYSTEMS HCFA’s Failure to Follow Guidelines Makes System Effectiveness Uncertain _- GAO United States General Accounting Office Washington, D.C. 20548 Information Management and Technology Division B-239339.1 July 26, 1990 The Honorable Tom Harkin Chairman, Subcommittee on Labor, Health and Human Services, Education, and Related Agencies Committee on Appropriations United States Senate Dear Mr. Chairman: On January 22, 1990, you asked us to review the development, progress, performance, and cost of a Medicare automated information system known as the Common Working File (CWF). This system is being imple- mented by the Health Care Financing Administration (HCFA), Depart- ment of Health and Human Services, for the stated purpose of improving Medicare claims processing. This report presents our evalua- tion of the CWF system, now being installed nationwide. Our work was performed primarily at HCFA headquarters in Baltimore, Maryland, and at the offices of Medicare contractors, including Blue Cross and Blue Shield of Maryland and Blue Cross and Blue Shield of Texas, the CWF pilot contractors. Details of our objectives, scope, and methodology are discussed in appendix I. HCFA approved and is implementing CWFwithout knowing whether it is a Results in Brief sound investment. This uncertainty exists because HCFA did not follow federal and Department of Health and Human Services guidelines on developing new systems. These guidelines specify that before proceeding with a project, an agency should develop a plan that documents the existing program’s problems or needs, identifies alternative solutions, and determines the costs and benefits of the alternatives. Once an alternative is selected, the agency needs a strategy to develop the new system, test it, and identify and minimize the risks of implementing it. HCFA believed it did not have to follow these guidelines and, therefore, did not generate basic information needed during system development. HCFA, for example, never estimated total system costs, did not document Page I GAO/IMTEG90-63 HCFA Fails to Follow Guidelines on ADP Systems B-239339.1 expected benefits (savings),’ and did not test the system in a way that would generate useful information. The system is now essentially installed, but will cost an estimated $30 million per year to operate. If the system cannot generate savings at least equal to this amount, then HCFA should consider discontinuing it. Medicare is authorized by title XVIII of the Social Security Act. It pro- Background vides health insurance to individuals aged 65 and over, and to certain disabled people under age 65. Medicareis divided into two parts: part A covers inpatient hospital services, skilled nursing facility services, hos- pice care, and home health care, while part B covers physician, outpa- tient hospital, and various other health services. HCFA administers Medicare through contractors (generally insurance companies) that process claims, make payments, perform payment over- sight, recover erroneous payments, and provide various other services. For fiscal year 1989, HCFA estimates (as of May 1990) that Medicare ben- efits were about $55.3 billion for part A and $34.6 billion for part B. Costs to administer the program in fiscal year 1989 were about $1.3 billion. Prior to CWF, HCFA allowed part A and part B contractors to process and determine whether to pay Medicare claims using their own data bases and the HCFA master file. The claims information was then processed through the HCFA headquarters data base for postpayment review. In addition, part A contractors would periodically provide payment data to part B contractors so a comparison of payment decisions could identify inappropriate payments. HCFA began developing CWF in fiscal year 1987 to improve the accuracy of Medicare claims processing, reduce overpayments by Medicare con- tractors, and capture claims data that can be used to develop a centrally managed data base for research and policy development. CWF is essen- tially a data base of combined Medicare part A and part B claims histo- ries, previously maintained at HCFA headquarters, placed into nine regional data bases, called hosts. The hosts are operated by Medicare ‘In this report, because the benefits are measurable, the terms benefits and savings are used interchangeably Page 2 GAO/IMTEGS@63HCFA Fails to Follow Guidelines on ADP Systems B-239339.1 part A or part B contractors who maintain CWF data under separate con- tracts. Thus the hosts are operated by an added layer of contractors that already have either part A or part B responsibility. Each host contains information on beneficiaries residing within its region. HCFA believes CWF'S combined part A and part B data, not previ- ously available as a prepayment control, will prevent overpayments involving duplicate bills and inappropriate or medically unnecessary services. According to HCFA, payment decisions will be improved because CWF will use more complete and up-to-date information, thereby elimi- nating overpayments that result when, for example, a contractor is una- ware that a beneficiary has reached a limit on coverage. HCFA pilot-tested CWF at two host locations. Blue Cross and Blue Shield of Maryland helped develop CWF'S software, and began testing the opera- tion of the system in June 1987. In December 1987 HCFA initiated testing at Blue Cross and Blue Shield of Texas. Maryland tested whether CWF would be able to respond to questions from Medicare contractors and verify their payment decisions. In this test, the only contractors that queried the data base or sent claims through the host for review and validation were Blue Cross of Maryland for part A and Blue Shield of Maryland for part B. The Texas test, however, involved more contrac- tors and had a dual purpose: (1) to demonstrate that CWF could be physi- cally moved to and operated at another host location, and (2) to demonstrate that CWF could function when communicating with Medi- care contractors located in other states. HCFA considered the tests suc- cessful Although problems were encountered, such as the need for additional telecommunications capacity between the Texas host and HCFA headquarters, HCFA considered the problems to be correctable. Before part A and part B contractors can communicate with the CWF host, the contractors must convert their systems’ software to be compat- ible with the host system. As of December 1989 CWF had been installed at all nine host contractors, and 38 percent of the Medicare parts A and B contractors were operational. Page3 GAO/IMTEG99-53HCFAFaUstiFollowGuidelinesonADPSystems B-239339.1 In developing the CWF system, HCFA did not prepare a plan as outlined by HCFA Has Not governmentwide guidelines and reinforced in a Department of Health Followed and Human Services (HHS) manual.* This is not an isolated incident; HHS Governmentwide officials acknowledge that, in the past, HCFA has not followed automated data processing guidance and HHS has not attempted to enforce compli- Management ance. According to HHS officials, HCFA believes that because it employs Procedures contractors to process claims, rather than operate its own claims- processing facility, it does not have to follow this guidance. In our opinion, however, because HCFA is mandating and funding the system’s development, HCFA should follow federal guidelines. Not doing so has contributed to HCFA'S inability to substantiate project benefits and may have contributed to increased software and telecommunications costs. HHS management supports the need for adequate planning and agrees that HCFA should have prepared a plan and documented benefits for CWF. Federal Information The Paperwork Reduction Act of 1980 (P.L. 96-51 l), as amended, pro- motes the effective management of automated data processing resources Management Standards in the federal government, emphasizing information as a resource with associated costs and benefits. The HHS manual, which provides guidance to its component agencies, cites this act in emphasizing the importance of proper planning in the successful development of automated systems, such as CWF, and recommends that management thoroughly document all planning and development steps. The manual further states that HHS' organizational components, including HCFA, shall use the National Insti- tute of Standards and Technology Federal Information Processing Stan- dards publications (FIPS PUBS) in planning and developing an automated information system. However, HHS has not actively ensured compliance with this manual and HCFA has, at least with CWF, not followed it. FIPSPUBS 38 and 64 identify three phases in an automated system’s life cycle-initiation, development, and operation -and stress the impor- tance of documenting these phases as a project progresses. The initiation phase establishes the objectives and general definition of the require- ments for the software, including feasibility studies and cost/benefit analyses. During the development phase, the specific requirements for the software are determined and the software is then defined, specified, ‘Department of Health and Human Services Information Resources Management Manual (Nov. 1, 1985). Page4 GAO/JMTEG9O-63HCFAFailstoFollowGuidelinesonADPSyst.em __-~ B-239339.1 programmed, and tested. In the operational phase, the software is main- tained, evaluated, and changed as additional requirements are identi- fied. At the beginning of 1990, CWF was in the development phase, moving into the operational phase. Federal Guidelines Also We found that HCFA had not adequately documented system initiation Not Followed During CWF and development. In starting the project, for example, HCFA did not clearly define the need for CWF, or perform a feasibility study to identify Development possible alternative solutions and their costs and benefits. The agency likewise did not prepare a risk analysis during the initiation phase to identify system vulnerabilities that could endanger either the system or its program data. This analysis is to be reviewed and revised, as necessary, during each phase of the system development life cycle to assure that appropriate security measures are installed. Federal standards provide that, at the onset of the development phase, potential users provide input to system planning so that, to the extent possible, the system will serve their needs. We met with officials of HCFA'S Health Standards and Quality Bureau and HCFA'S Bureau of Data Management and Strategy. These organizations use Medicare data in planning or policy-making. According to these users, they had little opportunity to influence decisions about what type of information would be generated by CWF or how that information would be made available to them. These users expressed concern that if CWF did not provide sufficient information in a usable format, they would be unable to fully monitor health care trends. We also met with the Physician Payment Review Commission, which advises the Congress on physician payment under Medicare. Officials were uncertain if information critical to them, such as physicians’ iden- tifier numbers and zip codes, would be available from CWF. These users expressed concern that without such information, some of their statis- tical studies and trend analyses would either be inaccurate or could not be conducted at all. Although they have relayed this concern to CWF pro- ject managers, they do not know how it will be resolved. HCFA officials responsible for managing this project acknowledge that these procedures had not been followed and the recommended docu- ments not prepared. These officials are primarily responsible for man- aging claims processing and are relatively unfamiliar with the standards for developing automated information systems. Once we informed them Page 5 GAO/lMTECXO-53HCFA Fails to Follow Guidelines on ADP Systems ___. .~ Bz39339.1 of the standards, they still did not believe that the guidelines applied to this system because the system’s primary purpose was to process claims. The objective of the guidelines, however, is not to distinguish systems according to their purposes, but to ensure that all systems are effectively designed and implemented. HHS guidelines provide that the cost for an automated information Cost of CWF’ Was Not system be estimated during the initiation phase of a project’s develop- Initially Established ment. HCFA, however, did not estimate project development costs before and Continues to initiating CWF. Instead, HCFA budgeted annually for CWF as a productivity improvement item within the Medicare contractor section of the Increase agency’s annual budget. Through interviews with HCFA project officials and reviews of the budget documentation, we found that CWF, from ini- tial project funding in fiscal year 1987 through its expected completion in fiscal year 1991, will cost about $103.8 million. CWF costs have continued to increase because of system enhancements and telecommunications. When HCFA submitted its 1990 budget it did not contain reference to additional development costs, and indicated that CWF would be fully implemented in fiscal year 1990. However HCFA cur- rently plans to spend $11 million in fiscal year 1991 on system enhance- ments, which include software changes designed to correct system operational problems and to provide for planned benefits. These software changes were apparently not anticipated by HCFA one year ear- lier. In addition, the expected cost of telecommunications has risen dra- matically. In March 1989 HCFA estimated that CWF telecommunications would cost $2.6 million per year. In fiscal year 1990 HCFA revised this estimate upward. As of May 1990, HCFA estimates CWF telecommunica- tions will cost $5.8 million in fiscal year 1990 and $9 million in fiscal year 1991. Table 1 presents CWF'S costs during fiscal years 1987-1991. Page 6 GAO/IMTECXO-53HCFA FaiJsto Follow Guidelines on ALIP Systems B-239339.1 Table 1: CWF Costs From Fiscal Years 1987 Through 1991a Dollars in millions Fiscal Year cost Category of Expenditures 1987 $4.6 Development of CWF software, pilot operations 1988 7.6 Development of CWF software, pilot operations 1989 Host operations, maintenance, system conversions, 23.9 telecommunications 1990 34.6 Host operations, maintenance, system conversions 1991 Host operations, enhancements, system conversions, 33.1 telecommunications Total $103.8 Tests for fiscal years 1987-1989 are actual. Costs for fiscal years 1990 and 1991 are HCFA estimates Although HCFA has justified CWF on the basis of the system’s expected HCFA Could Not financial savings, the agency could not provide any documentation sup Document Expected porting these savings. In addition, HCFA did not develop any actual data Savings on CWF savings during pilot testing. This absence of evidence calls into question the validity of HCFA'S savings estimates. Estimated Savings From HCFA estimates that CWF will provide about $145 million in annual Medi- care program savings and administrative cost savings. According to CWF HCFA, the existing process cannot identify all payment errors. The largest portion of estimated savings-about $72 million-is expected to be derived from reducing payment errors by comparing claims sub- mitted by the contractors with combined Medicare part A and part B history files. Other projected savings include better identification of pri- vate insurance coverage to ensure that Medicare pays last in the Medi- care secondary payer program; elimination of overpayments written off because of their small size; interest on overpayments to beneficiaries; improved medical review based on more complete beneficiary data; and better identification of payments for the duplicate purchase of certain medical equipment and other services. HCFA officials also project that CWF will provide $20 million in annual savings because of a reduction in contractors’ administrative costs. Table 2 summarizes the expected annual savings attributable to CWF. Page 7 GAO/IMTEG90-63 HCFA Fails to Follow Guidelines on ADP Systems __ ..-- B-239339.1 Table 2: HCFA’s Estimate of Annual CWF Savings Dollars in millions Activity Amount Parts A/B data exchange $72 Improved medical review 12 Eliminated write-off of small overpayments 15 Improved secondary payer identification 15 Lost interest on overpavments 7 Identifying duplicate payments 4 Program savings $125 Administrative savings 20 Total annual savinas $145 HCFA Lacks Data to HCFA could not provide data to show that combining part A and part B data, through a common file, will produce the estimated program sav- Support the Projected ings. Although federal guidelines recommend that a system be thor- o,.,,:“.r(e LXLV 1rtg;a oughly tested, and that test results and findings be documented, HCFA did not follow these guidelines. HCFA'S tests did not document the system’s ability to generate savings. As a result, HCFA is unable to accu- rately estimate what savings will result. At the time that CWF was pilot-tested in Maryland, the system did not include software to identify and gather data on actual program savings. Similarly, HCFA did not gather actual savings data during the pilot test at Blue Cross and Blue Shield of Texas. HCFA believed that, because Texas already used a combined part A and part B system before CWF, savings attributable to CWF may have been difficult to measure if, in fact, they existed at all. HCFA plans to implement a project this year to gather national data on CWF savings. As of May 1990, however, the agency had not yet taken action. In addition, HCFA estimates that the annual cost of operating CWF will be about $30 million. This amount more than negates the $20 million in administrative savings HCFA expects from the system’s implementation. HCFA decided to implement CWF before it considered alternatives, esti- Conclusions mated costs, or documented savings. This decision is inconsistent with HHS and governmentwide procedures for developing automated informa- tion systems, and also with good management practices. These proce- dures-embodied in federal and HHS guidelines and regulations-are not Page 8 GAO/IMTEG90-53 HCFAFails to Follow Guidelines on ADP Systems 5239339.1 intended to suggest doing paper exercises or documenting decisions already made. Instead, they should be an integral part of, and influence on, the decision-making process. Ultimately, they should be used to decide whether or not to make an investment. Without proper planning and documentation, the acquisitions of com- plex automated systems may encounter problems. HHS management acknowledges that HCFA should have followed the Department’s gui- dance when developing CWF. HCFA'S failure to follow government procedures for developing informa- tion systems leaves the agency unable to determine whether CWF is cost- effective, or whether it will achieve its objectives. At this time, procure- ment and deployment of CWF has essentially been completed. All that HCFA can do now is gather CWF savings data and assure itself that the system will achieve savings at least equal to its estimated annual oper- ating cost of $30 million. If not, HCFA is faced with the decision of whether to continue CWF.More importantly, HCFA can learn from this experience with CWF, and avoid getting into such situations in the future. Properly following governmentwide and HHS system develop- ment procedures would help ensure that funds for future systems are spent in the most effective and efficient manner. In order to assure that the CWF is cost-effective and to limit the risks Recommendations inherent in developing information systems, we recommend that the Sec- retary of Health and Human Services direct the Administrator, Health Care Financing Administration, to evaluate the cost-effectiveness of continuing to operate the CWF system, and if the system is found not to be cost-effective to determine what, if any, alternatives exist; and follow federal information system development standards in all future system modifications and acquisitions, seeing to it that (1) all phases of a project’s development are adequately documented, (2) system costs and benefits are fully identified and justified, and (3) the system is ade- quately tested. The Department of Health and Human Services generally concurred Agency Comments and with our recommendation that HCFA needs to establish the cost-effective- Our Evaluation ness of CWF or determine what alternatives, if any, exist. The Depart- ment stated, however, that it did not believe that cost-effectiveness Page 9 GAO/IMTEC90-63 HCFA Fails to Follow Guidelines on ADP Systems - BZ39339.1 should be a sole criterion for building or retaining a system. The Depart- ment indicated that HCFA is conducting a study of CWF’S cost-effective- ness and that it will use these results to enhance the system. Also, the Health and Human Services Inspector General is conducting an evalua- tion of alternatives for the future. The Department plans to use this study to develop requirements for CWF’Seventual successor. We believe that not paying sufficient attention to the cost/benefit anal- ysis can result in agencies acquiring ineffective and inefficient auto- mated information systems. Federal guidelines, such as the National Institute of Standards and Technology Federal Information Processing Standards, emphasize the importance of the cost/benefit analysis in making the critical decision to begin the development and implementa- tion of an automated information system. While not the sole criterion, the cost/benefit analysis allows managers to make more informed deci- sions on whether to commit scarce resources to a project. The guidelines further emphasize the need to consider the cost/benefit of several alter- natives before initiating a project. The Department further concurred that HCFA should follow federal information system development standards in all future automated information system modifications and acquisitions. It noted, however, that when CWF was initiated in 1986, HCFA viewed CWF as a contractor system and, as such, not subject to information resource management guidelines. The Department indicated that this position is now being reexamined in light of recent events, such as a prior report we issued on automated information system cost reporting.” We believe that the above federal guidelines should have been applied to CWF from the StarL CWF iS HCFA’S SySk!In, not a CO~traCtOI’ SySkIn; HCFA has controlled CWF from its initiation and continues to maintain total control of the software. Given these facts, we disagree with the conten- tion that CWF development was ever exempted from the application of federal guidelines. Detailed Department of Health and Human Services comments and our evaluation are contained in appendix II. “information Technology:Health CareFinancingAdministration’s BudgetProcessNeedsImprove- ment (GAO/IMTm’;gS-31, Aug. 11,1989). Page10 GAO/IMTEGW HCFAFailstoFollowGuidelinesonADPSystems We are sending copies of this report to the Chairmen of the House Com- mittee on Appropriations, Senate Committee on Governmental Affairs, and House Committee on Government Operations; the Secretary of Health and Human Services; and the Administrator, Health Care Financing Administration. We will also make copies available to other interested parties upon request. This report was prepared under the direction of Frank Reilly, Director, Human Resources Information Systems, who can be reached at (202) 275-3462. Other major contributors are listed in appendix III. Sincerely yours, Ralph V. Carlone Assistant Comptroller General Page 11 GAO/IMTEC90-53 HCFA Fails to Follow Guidelines on ADP Systems Contents Letter 1 Appendix I 14 Objectives, Scope, and Methodology Appendix II 16 Agency Comments and GAOco~ents 22 Our Evaluation Appendix III 24 Major Contributors to This Report Abbreviations BDMS Bureau of Data Management and Strategy CWF Common Working File FIPS PUB Federal Information Processing Standards Publications GAO General Accounting Office HCFA Health Care Financing Administration HHS Department of Health and Human Services IMTEC Information Management and Technology Division Page 12 GAO/JMTJ3C9O-53 HCFA Fails to Follow Guidelines on ADP Systems -- Page 13 GAO/IlWEG9O-53 HCFA Fails to Follow Guidelines on ADP Systems Appendix I Objectives,Scope,and Methodology We initiated our review of the CWF project because it represented a major change to the processing of Medicare claims. On January 22,1990, we received a request to review the progress, performance, and costs of CWFfrom the Chairman, Subcommittee on Labor, Health and Human Services, Education, and Related Agencies, Senate Committee on Appro- priations. Our objectives were to assess (1) CWF'S performance, benefits, and costs; and (2) HCFA'S management of the project’s development. To obtain information on the CWFproject, we met with Bureau of Pro- gram Operations officials at HCFA headquarters in Baltimore, Maryland, and reviewed acquisition planning documentation. In addition, we met with officials of selected contractors. We interviewed officials of Blue Cross and Blue Shield of Maryland and Blue Cross and Blue Shield of Texas because these sites were chosen by HCFA to pilot-test the CWF pro- ject. We interviewed officials of Empire Blue Cross and Blue Shield in New York because Empire is a large host contractor and because HCFA chose Empire to be the CWFmaintenance contractor. We met with offi- cials of Blue Cross and Blue Shield of Massachusetts to discuss a shared processing arrangement that the contractor had entered into with other New England Blue Cross plans to process their claims, a preliminary step to implementing CWF,and to identify how this arrangement would affect CWF.We further discussed this arrangement with officials from the HCFA Boston Regional Office. We also met with officials of Aetna Life and Casualty Insurance Company in Connecticut because Aetna is a multistate contractor covered by five cw~ host sectors, and we wanted to determine how CWF would operate in this environment. To determine the requirements for effectively managing an automated information system, we reviewed Publications 38 and 64 of the National Institute of Standards and Technology Federal Information Processing Standards, and the Dewrtment of Health and Human Services’ Informa- tion Resource Management Manual guidelines for initiating and devel- oping an automated system. To evaluate the responsibilities of senior information resources management officials, we also reviewed the Paperwork Reduction Act of 1980, the Paperwork Reduction Reauthorization Act of 1986, and Federal Information Resources Man- agement Regulations. In addition, we contacted the Health and Human Services’ information resource management official responsible for HCFA expenditures on automated information systems to obtain his opinions of whether HCFA had adequately followed the Department’s information resource manual guidelines. Page 14 GAO/IMTECr9o-63HCFA Fails to Follow Guidelines on ADP Systems Appendix 1 Objectives, Scope, and Methodology We also incorporated Department of Health and Human Services and HCFA’S comments obtained in July 1990. Our review was conducted in accordance with generally accepted government auditing standards, from July 1989 through May 1990. Page 15 GAO/lM’R$C~ HCFA Fails to Follow Guidelines on ADP Systems Agency Comments and Our Evaluation suppienentmg those In the report text appear at the end of this appendix DWARTMENT OF HEALTH& HUMAN SERVICES Olllc* 01 Yspaan ouluti JUL 21990 Rr. Ralph V. Carlone Assistant Comptroller General United States General Accounting office Washington, D.C. 20584 Dear Rr. Carlone: Enclosed are the Department'e comments on your draft report, 'ADP Systema: ?iCFAgs Failure to Follow Guidelinrs Rakes System Effectiveness Uncertain." The comments represent the tentative position of the Department and are subject to reevaluation when the final version of this report is received. The Department appreciates the opportunity to comment on this draft report before its publication. sincerely yours. Richard P. Xuoserow Inspector General Enclosure 15 Page16 GAO/lMTEG9lM3HCFAFailstoFoUow Guidelines onADPSystems AppendixII AgencyComments andoUr Evaluation Comment8 partment of the De on the General Accounting Office Draft Report, "ADPSystems: Health Care Financing tiini8tration's (HCFA’s) Failure to Follow Guidelines Makes System FffeCtiVelM?SS Uncertain” We are pleased to cement on the subject report. We \aderstand the GAO concern for procedural safeguards, but believe the actual result of the Comon Working File (CWFI initiative speaks for the system's careful design, testing and implementation. We retain our conviction that 134~ represents a major advance in the way HCFA conducts business. We would first like to make clear why HCFA implemented CWF, and the benefits and opportunities it provides in the volatile Medicare environment. In 1987, major components of HCFA's central claims operations were 15-20 years old, cunberscme and inefficient. Elaborate and expensive postpayment error handling processes were required to maintain the system. Data were incomplete and took months ta assemble. CWFsimplified operation8 by establishing a standard prepayment claims authorization process utilizing complete beneficiary entitlement, eligibility and utilization data. Thernovementto prepayment authorization is perhaps the most significant change in claim Operation8 since the inception of Medicare. HCFAcan nw effectively control claims processing and en8ure uniform benefit administration. Beyond this, CWFfacilitates policy development via its database, increases service to contractors and beneficiaries and strategically positions HCFA to deal with inevitable program changes directed by Congress. The system was consciously designed not just to deal with known requirements, but also to be able to hsndle the unknowable. This concept grew out of close coordination with potential users of the system and a comitment to be responsive to their needs. We believe its response to the challenges of Catastro&ic Health Insurance Coversge and physician payment Reform validates the HCFAapproach. The report establishes a standard of effectiveness based only on COSt- savings. While HCFAbelieves significant savings are obtainable, the decision to pumue the project looked beyond classical return on investment calculations which attempt to recover all expenditures from direct savings. The capital investment in CWFis viewed as ah investment in better quality, higher levels of service and greater flexibility to deal with the future. Page17 GAO/lMTEC9063 HCPAFailstoFoUowGuidelineson ADPSystem Appendix II Agency Commentsand Our Evaluation Page 2 GAORecomaendation In order to ensure that the CWFis cost-effective and to limit the risks inherent in developing information systems, we recognnend that the Swretarv of Health and HumanServices direct the Administrator, HCFX. to: -- evaluate the cost-effectiveness of continuing to onerate the cWF system, and if the system is found not to be cost-effective to determine what, if any, alternatives exist: and Department Cknmaent We concur in part with the recommendation. HCPAconducts an on-going program of cost-effectiveness reviews and will certainly include CWPin those processes. In addition, the HHSInspector General is currently engaged in a study of CWFwhich includes an evaluation of alternatives for the future. We will use the results of those studies to enhance CWFand develop requirements for its eventual successor. We note that CWPhas never exceeded its annual budget. Moreover, CWFis l~lw the linchpin for all contractor claims processing operations and is the only system’ available which supports the dynamic Medicare benefit administration environment. As noted above, CWFalready provides HCFAwith significant benefits which are quantitative, qualitative and strategic. We do not agree with the premise that administrative cost-effectiveness is the sole criterion for building or retaining a system. GAORecomnendation -- follow Federal information system develounent standards in all fut system modifications and acouisitions, seeing to it that (1) all phases of a project’s develorxnent are adeauately documented, (2) system costs and benefits are fully identified and justified, and (3) the system is adeauately tested. Department &nnent Weconcur in the GM recommendation to base the future development of cWF on established formal procedures. When the CWPproject was initiated in 1986, HCFAwas sure that, as a contractor benefit administration system, CwFwas not subject to Infomtion Resource Management (IPkl) guidelines. More recent events, such as the GAOreview of HCFAADPcost reporting, Page 18 GAO/IMTEGSO-53HCFAFails to Follow Guidelines on ADP Systems Appendix II Agency Commentsand Our Evaluation page3 planning for drug bill processing under the Medicare Catastrophic legislation and this report , indicate that a clear dichotomy between Federal Systems and Medicare contractor systems laay no longer exist. HCFA will, therefore, reexamine its Medicare systems in relation to the IR4 requirements to determine to what extent and in which instances IF@l guideline8 will apply. Where the guideline8 do apply, we will certainly follow the recammndations. Other Conrnenta See comment 1 page3 - (%F was not developed to "create a data base for research and policy development." Rather, HCFAcapture8 claim8 data as they are processed through the CWFhost sites and uses this by- product to develop a centrally held claims database which supports those functions. See comment 2 page6- We disagree that benefit8 were not docueented. We furnished the review team with our docMent.ation and statements of the methodology u8ed to estimate savings where actual data were unobtainable. The chart on paOe 12 of the report reflects HCFA's conservative estimate of benefits. The benefits of CWF were certainly identified and documented a8 best we could. We reiterate our belief that benefits mean much more than cost savings alone. See comment 3 Pape7- HCFA's Bureau of Data Msnagement and Strategy (EMI had ample opportunity to influence data decisions for '%P. ~ was actively involved from the outset in the developnt snd design of c!G. BIBS worked closely with HCPA's Wwsu of program Operations to ensure that the data being gathered was proper, both in terms of its definition and fotmst. Furthermore, BEM staff is directly involved in the design, development and maintenance of over forty core progranrning modules now utilized by CWFsites. BUMSis the gatekeeper for all HCFAprogtwmnatic data and will play a major role in the systems use and evolution. See comment 4 page8- 'Ihe paraoraph on the pE%Cshould be deleted. CWFprovides 100 percent of all claims data intact to the HCFA statistical systems. Page19 GAO/IMTEG99-63 HCFAFails to Follow Guidelines on ADP Systems Appendix II Agency Commentsand Our Evaluation PBge 4 See ‘. r. f,-’ 5 Page 9 - HCFAha8 been highly successful using competitive processes (for both CVF maintenance snd host operations) to achieve operational cost efficiencies. Telecammmications costs for CWFdo exceed estimates. This is not, however, an operational problem in CWF. ‘Ihe combination of moving from SSAbased telecomnunications to the HCFAData Center and the major expansion of the part B record to better support initiatives such a8 effectiveness studies 80 changed the HCFA telecossnunicstions environment that our working asswsptions for calculating future costs were flawed. The report’s position on CWFenhancements seems to be based on a belief that “operational” means ” in its final configuration. *’ This wm8never our intent. The system is already operational, processing well over 1 million claims daily. We always planned to incrementally e.xp8nd CWFfunctionality. Further, it is incorrect to report that HCFAdid not anticipate certain enhancements. Nsjor changes such as host-to-host telecommunications and database restructuring have long been part of the system plan. They are being phased-in to meet changing needs, such a8 transaction growth, in a msnner that will not er&nger the system’s overriding priority of claims payment. Finally, such enhancements compete with other Medicare requirements for funding each year. See comment 6 Page 10 - CWFcost data for FYs 87 snd 88 include costs for pilot operation8 in Maryland and Texas. HCFAbelieves the software development and maintenance costs for a system of this complexity are reasonable based on our experience with other Medicare claims processing systems. GAOwas provided with savings estimates from the pilot tests. See comment 7 Page 12 - The lack of savings data is in no way related to testing deficiencies. CWFwa8 elaborately tested; i ,e., tested before installation at Harylsnd, tested in pilot production at Maryland, further tested in beta production at Texas and tested in a remote environment at Arizona. The problem in docmntine savings comes from the structure of CWFitself. It is a transaction processing system. It authorizes or rejects payment, but keeps records only of authorized transactions. Page20 GAO/IMTEG90-63 HCFA Fails to Follow Guidelines on ADP Systems Agency Comments and Our Evaluation Savings are actualized at the individual contrsctors. HCFAbelieved that the case for program savings was so obvious that incurring significant costs to document them in advance could not bs justified. See comment 8 Page 13 - The report contends that the $30 million CWFcost "negates" the $20 million administrative savings, Thus, the current incremental cost of the system is established at approximately $10 million. We point out that cost/savings is rather volatile in Medicare. The savings would have been $33 million until the repeal of the Catastrophic Coverage tit of 1988 (CCA) eliminated the need to track and inform beneficiaries of progress toward the Part B cap. 'Ihe physicisn payment reform requirement to track volune perfoxmsnce standards wss inconceivable prior to CWF. We have not even attempted to price the cost of administering payment refono'without CWF, 80 as to claim savings. Again we assert the real value of benefits which are not quantifiable. WhenCCA was repealed, the reestablishment of Part A spell of illness processing was so difficult that it would have taken until tiy 1990 to properly process inpatient bills without CWF. The cost to HCFA had those bills been held up for months is incalculable. These exsmples reinforce our position, stated above, that the flexibility built by design into CWFis its great strength. See comment 9 Psge 14 - We believe that we have a highly functional, robust claims authorization system which supplies terabytes of timely, detailed program data for use by Congress, the Administration and the pPRCin directing the future of Medicare and which appears to be a suitable base for sustaining operations into the next century. P8ge 21 G.40/IMTEG90-62 HCFA Fails to Follow Guidelines on ADP Systems .- Appendix Jl Agency Commentsand Our Evaluation 1. We agree that the language suggested by the Department is more pre- GAO Comments cise than the language we used; accordingly, we have modified our draft. 2. We believe that the documentation of CWF'S benefits should have included supporting evidence that benefits can be achieved. We reviewed HCFA files and interviewed HCFA program officials but were unable to obtain this type of support for the purported benefits. HCFA officials indicated that they could not measure actual savings resulting from CWF. We have no way of knowing, therefore, whether HCFA'S esti- mate of benefits is conservative or inflated. 3. We discussed CWF development with the Bureau of Data Management and Strategy (BDMS) information resource management officials and do not agree that HCFA'S BDMS was sufficiently involved in the identification of CWF requirements and its design and development. While BDMS appar- ently had more interaction with BP0 in later stages of CWF's develop- ment and implementation, BDMS officials indicated that they had little opportunity to provide input during CWF'S design. 4. The Physician Payment Review Commission is an important user of information collected by HCFA from the claims process, and was identi- fied by the CWF project staff as a user involved in developing CWF. We disagree that this paragraph should be deleted. At issue is not whether data are provided, but whether they meet the needs of users. 5. We believe that continued improvements to CWF are acceptable. How- ever, we believe that a significant portion of the cost increase shown in HCFA'S fiscal year 1991 budget is for projects to make CWF operational rather than to improve it. 6. Table 1 has been changed to show that the costs HCFA incurred for CWF in fiscal years 1987 and 1988 also include CWF pilot operations. 7. The most basic element of testing is to determine if a system will per- form as intended. A major indicator of satisfactory performance for CWF is program savings, which was HCFA'S basis for implementing the system. HCFA officials told us, however, that the testing performed was intended to determine only if CWF would function. They acknowledged that the test was not intended to determine if CWF achieved anticipated savings, 8. We agree that the benefits resulting from a systems development effort may not be easily quantified. However, that does not negate the Page 22 GAO/IMTEG9@33HCFA Fails to Follow Guidelines on ADP Systems Appendix II Agency Comments and Our Evaluation need to assess potential benefits to determine the direction and level of investment in a systems development effort. HCFA'Sjustification for CWF was limited to cost savings and did not include any discussion of poten- tial benefits. Therefore, in reviewing HCFA'Sdocumented benefits, we were limited to a review of that information. We could not assess poten- tial benefits that HCFAdid not document. 9. We agree with HCFAthat cw is functioning and supplying informa- tion. However, whether CWFis cost-effective, or whether it will be able to achieve its objectives, has not been established. Properly following governmentwide and HHSsystem development procedures would have provided greater assurances that CWF was the most suitable base for sustaining the future claims operations of Medicare. Page 23 GAO/IMTEG9O43 HCFA Fsils to Follow Guidelines on ADP Systemm Appendix III Major Contributors to This Report Douglas Nosik, Assistant Director Information Ronald Yucas, Senior Evaluator Management and Michael Resser, Evaluator Mary T. Marshall, Reports Analyst Technology Division, Washington, D.C. William A. Moffitt, Regional Management Representative Boston Regional Office Donald P. Benson, Evaluator-in-Charge Susan A. Fleming, Evaluator (510450) Page 24 GAO/IMTEGSO-63HCFA Fails to Follow Guidelines on ADP Systems Requests for copies of GAO reports should be sent to: U.S. General Accounting Office Post Office Box 6015 Gaithersburg, Maryland 20877 Telephone 202-275-624 1 The first five copies of each report are free. hddit ional copies are $2.00 each. There is a 254, discount on orders for 100 or more copies mailed to a single address. Orders must be prepaid by cash or bj check or money order made out to the Superintendent of Documrnt s.
ADP Systems: HCFA's Failure to Follow Guidelines Makes Systems Effectiveness Uncertain
Published by the Government Accountability Office on 1990-07-26.
Below is a raw (and likely hideous) rendition of the original report. (PDF)