United States General Accounting Office GAO Testimony Before the Committee on Veterans’ Affairs, U.S. Senate For Release on Delivery Expected at 2:30 p.m. YEAR 2000 COMPUTING Tuesday, April 20, 1999 CRISIS Key Actions Remain to Ensure Delivery of Veterans Benefits and Health Services Statement of Joel C. Willemssen Director, Civil Agencies Information Systems Accounting and Information Management Division GAO/T-AIMD-99-152 Mr. Chairman and Members of the Committee: We appreciate the opportunity to participate in today’s hearing on the Department of Veterans’ Affairs (VA) efforts to address the Year 2000 (Y2K) computer problem. We will focus on the Y2K readiness of automated systems that support the delivery of benefits and health care services, the compliance status of biomedical equipment used in patient care, and the Y2K readiness of the pharmaceutical and medical-surgical manufacturers upon which VA relies. In discussing biomedical equipment and pharmaceutical manufacturers, we will also share with you information on the Food and Drug Administration’s (FDA) Y2K efforts.1 In brief, VA continues to make progress in its Y2K readiness. However, key actions remain to be performed. For example, the Veterans Benefits Administration (VBA) and Veterans Health Administration (VHA) have not yet completed testing of their mission-critical systems to ensure that these systems can reliably accept future dates—such as January 1, 2000. Also, VHA has not completed Y2K assessments of its facility systems, which can be essential to ensuring continuing health care. In addition, neither VA nor FDA had implemented our prior recommendation to review the test results for biomedical equipment used in critical care/life support environments. Further, VHA’s pharmaceutical operations are at risk because the automated systems supporting its consolidated mail outpatient pharmacies are not Y2K compliant. Finally, VHA does not know if its medical facilities will have a sufficient supply of pharmaceutical and medical-surgical products on hand because it does not have complete information on the Y2K readiness of these manufacturers. It is critical that these concerns be addressed if VA is to continue reliably delivering benefits and health care. 1 Biomedical equipment refers to both medical devices regulated by FDA, within the Department of Health and Human Services, and scientific and research instruments, which are not subject to FDA regulation. Page 1 GAO/T-AIMD-99-152 Key Actions Remain to Like many organizations, VA faces the possibility of computer system failures at the turn of the century due to incorrect information processing Ensure That VA Can relating to dates. The reason for this is that in many systems, the year 2000 Deliver Benefits and is indistinguishable from 1900, since the year is represented only by “00.” This could make veterans who are eligible for benefits and medical care Health Care Into the appear ineligible. If this happens, the issuance of benefits and the provision Next Century of medical care that veterans rely on could be delayed or interrupted. As we reported last August,2 VBA had made progress in addressing the recommendations in our May 1997 report3 and in making its information systems Y2K compliant. For example, VBA changed its Y2K strategy from developing new systems to converting existing ones. It also reported it had renovated 75 percent of its mission-critical applications as of June 1998. At the same time, VHA reported it had assessed all and renovated the vast majority of its mission-critical information systems. Despite this progress, VBA was making limited progress in renovating two key mission-critical applications—the compensation and pension online application and the Beneficiary Identification and Record Locator Sub-System. And, except for its Insurance Service, VBA had not developed business continuity and contingency plans for its program services—Compensation and Pension (the largest), Education, Loan Guaranty, and Vocational Rehabilitation and Counseling—to ensure that they would continue to operate if Y2K failures occurred. VHA's Y2K program likewise had areas of concern. For example, although VHA’s medical facilities had hospital contingency plans, as required by the Joint Commission on Accreditation of Healthcare Organizations, they had not yet completed Y2K business continuity and contingency plans. To address these areas and to reduce the likelihood of delayed or interrupted benefits and health care services, we recommended that VA • reassess its Y2K mission-critical efforts for the compensation and pension online application and the Beneficiary Identification and Record Locator Sub-System, as well as other information technology initiatives, such as special projects, to ensure that the Y2K efforts have 2Year 2000 Computing Crisis: Progress Made in Compliance of VA Systems, But Concerns Remain (GAO/AIMD-98-237, August 21, 1998). 3 Veterans Benefits Computer Systems: Risks of VBA’s Year 2000 Efforts (GAO/AIMD-97-79, May 30, 1997). Leter Page 2 GAO/T-AIMD-99-152 adequate resources, including contract support, to achieve compliance in time; • establish critical deadlines for the preparation of business continuity and contingency plans for each core business process or program service so that mission-critical functions affecting benefits delivery can be carried out even if software applications and commercial-off-the- shelf (COTS) products fail, including a description of resources, staff roles, procedures, and timetables needed for implementation; and • ensure rapid development of business continuity and contingency plans for each medical facility so that mission-critical functions affecting patient care can be carried out if software applications, COTS products, and/or facility-related systems and equipment do not function properly, including a description of resources, staff roles, procedures, and timetables needed for implementation.4 VA Continues to Make VA has been responsive to our recommendations. For example, VBA Progress reassessed its mission-critical efforts for the compensation and pension online application and the Beneficiary Identification and Record Locator Sub-System, as well as other information technology initiatives. It also reallocated resources to ensure that the Y2K efforts had adequate resources, including contract support, to achieve compliance. In addition, VBA completed a draft business continuity and contingency plan in January 1999 for its core business processes, as well as a related planning template for its regional offices. The plan provides a high-level overview of the resources, staff roles, procedures, and timetables for its implementation. It addresses risks, including mitigation actions to reduce the impact of Y2K-induced business failures, and analyzes the effect on each business line of a number of potential Y2K disasters—such as loss of electrical power, loss of communications, loss of data processing capabilities, and failure of internal infrastructure. According to VBA, the plan, which it expects to test this August, is an evolving document, to be revised and updated periodically until January 1, 2000. VBA’s plan makes no reference to contingencies for the failure of three of VBA’s benefits payment systems—Compensation and Pension, Education, and Vocational Rehabilitation and Counseling. However, it is currently developing a payment contingency plan for these systems and expects this 4 GAO/AIMD-98-237, August 21, 1998. Page 3 GAO/T-AIMD-99-152 to be completed in May 1999. A VBA official told us that the payment contingency plan should have been referenced in VBA’s business continuity and contingency plan and will be in future versions. The current plan also does not contain the designation of an information technology security coordinator and a physical security coordinator—individuals that VBA acknowledges are essential to the agency’s Y2K efforts—with responsibility for ensuring overall security for VBA's network and web site and for backing up data storage before, during, and following January 1, 2000. This type of information will be necessary if security-related failures occur. According to VBA, it expects to designate these individuals by August 1999. VHA has also made progress in developing business continuity and contingency plans for its medical facilities. Last month, VHA issued its Patient-Focused Year 2000 Contingency Planning Guidebook to its medical facilities describing actions they can take to minimize Y2K-related disruptions to patient care. The guidebook discusses how a medical facility should develop contingency plans for each major hospital function—such as radiology, pharmacy, and laboratory—as well as for each major support function—such as telecommunications, facility systems, medical devices, and automated information systems. The guidebook also contains examples of plans, policies, and solutions for problems that a medical facility may face and provides Y2K templates describing the areas a facility should address by specific hospital function. VA provided this guidebook to the medical facilities early last month and expects the facilities to use it to prepare their individual business continuity and contingency plans, set to be completed by April 30. The guidebook stresses that these plans should be tested and suggests that the medical facilities begin testing in June. The guidebook addresses external emergency preparedness as well as internal operations. Specifically, it discusses three functions that a medical facility should perform in order to ensure that potential external hazards are considered and planned for. These are (1) performing an assessment of hazard vulnerabilities—that is, the types and kinds of Y2K problems that are anticipated within the community, (2) conducting an inventory of community resources—people, money, clinical space, supplies, and equipment—available to address these hazards, and (3) closing the gap between vulnerabilities and capabilities by putting into place measures that will mitigate potential disruptions in critical services by developing new working relationships with various government agencies, non-VA health care organizations, and vendors of critical supplies. Page 4 GAO/T-AIMD-99-152 In addition to implementing our recommendations, VA continues to make progress renovating, validating, and implementing its systems. On March 31, 1999, VA reported to the Office of Management and Budget (OMB) that the department had renovated and implemented all of the mission-critical applications supporting its 11 systems areas. As shown in table 1, VBA has six of these areas, and VHA has two. Table 1: Reported Status of VA’s Mission-Critical Computer Systems Areas and Their Applications Component/office Number of applications (number of systems) Systems areas renovated or replaced Veterans Benefits Compensation and Pension 30 Administration (6) Education 24 Insurance 3 Loan Guaranty 19 Vocational Rehabilitation 4 Administrative 27 Total 107 Veterans Health Veterans Health Information Administration (2) Systems and Technology Architecture 105 Veterans Health Administration Corporate Systems 95 Total 200 National Cemetery System Burial Operations Support (1) System/Automated Monument Application System 1 Reengineer 1 Total 2 Office of Financial Personnel and Accounting Management (2) Integrated Data 8 Financial Management System 1 Total 9 VA total 11 318a a Of this total, 316 applications were renovated and two were replaced. Source: VA. We have not independently verified this information. Page 5 GAO/T-AIMD-99-152 Testing of Mission-Critical Complete and thorough Y2K testing is essential to providing reasonable Systems Not Yet Complete assurance that new or modified systems will process dates correctly and will not jeopardize an organization’s ability to perform core business operations. Because the Y2K problem is so pervasive, potentially affecting an organization’s systems software, applications software, databases, hardware, firmware, embedded processors, telecommunications, and interfaces, the requisite testing can be extensive and expensive. Experience is showing that Y2K testing is consuming between 50 and 70 percent of a Y2K project’s time and resources. According to our Y2K guide,5 to be done effectively, testing should be planned and conducted in a structured and disciplined fashion. Our guide describes a step-by-step framework for managing Y2K testing, which includes the following key processes: • Software unit testing to verify that the smallest defined module of software (individual subprograms or procedures) continues to work as intended. • Software integration testing to verify that units of software, when combined, continue to work together as intended. Typically, integration testing focuses on ensuring that the interfaces work correctly and that the integrated software meets requirements. • System acceptance testing to verify that the complete system—that is, the full complement of application software running on the target hardware and systems software infrastructure—satisfies specific requirements and is acceptable to users. This testing can be run separately or in some combination in an operational environment (actual or simulated) and collectively verifies that the entire system performs as expected. According to VBA and VHA officials, their testing criteria were based on their software development life cycle guidance documents. They said that upon successful completion of software unit and integration testing, a system is considered Y2K compliant. They said this type of testing had been completed for all of their mission-critical systems. As of March 31, 1999, neither VBA nor VHA had completed systems acceptance testing—which requires that each system be tested, including 5 Year 2000 Computing Crisis: A Testing Guide (GAO/AIMD-10.1.21, November 1998). Page 6 GAO/T-AIMD-99-152 full future-date testing, on a compliant platform—for all their mission- critical systems. Specifically, according to VBA officials, the agency had completed systems acceptance testing for half of its mission-critical systems—Insurance, Loan Guaranty, and Vocational Rehabilitation and Counseling. According to VBA’s Hines, Illinois, data center Y2K coordinator, systems acceptance testing of the Compensation and Pension systems just started on April 14, 1999. According to a VBA official, one of the reasons for the late systems testing was that the new IBM compiler6 at its Hines data center was not available for use until February 1999. According to VBA, the Compensation and Pension and most of the Education systems will be future-date tested throughout April. VHA also plans to begin system acceptance testing of its mission-critical systems this month and complete it this June. According to VHA officials, they could not perform this type of testing before March of this year because VHA did not have a separate Y2K-compliant test environment to isolate the testing from the hospital systems in use. In addition to testing of individual systems, end-to-end testing of multiple systems is also critical. End-to-end testing, as defined in our test guide, verifies that a defined set of interrelated systems, which collectively support an organizational core business area or function, continues to work as intended in an operational environment, either actual or simulated. For example, in order to successfully process a compensation benefit payment to a veteran, VBA’s Compensation and Pension System must work correctly with its Beneficiary Identification and Records Locator Sub- System, Treasury’s Financial Management System, the Federal Reserve System, and financial institution systems. VBA and VHA plan to conduct end-to-end testing between now and this July. VBA is defining end-to-end testing as verification that core mission- critical business functions, including benefits payments and vendor and payroll payments, process correctly. The interfaces between VBA’s benefits systems and Treasury’s Financial Management System are to be tested in May. VBA also plans to test transactions that interface with VHA systems, such as information related to veteran eligibility. VHA is defining end-to- end testing as verification that core mission-critical business functions, including patient-care transactions and vendor and payroll payments, 6 A compiler is a computer program that converts human-readable source code into a sequence of machine instructions that the computer can run. Page 7 GAO/T-AIMD-99-152 process correctly. Once these tests are completed, VBA and VHA plan to conduct a “business process simulation” during the July 4, 1999, weekend. This simulation of day-to-day work at VA is to include users at the VBA regional offices and VHA test laboratories, who will simulate various transactions and process them through a set of interrelated systems necessary to complete a core business function. VBA expects to pretest the business process simulation during May. Assessment of VHA’s VA’s facility systems are essential to the continued delivery of health care Facility Systems Not Yet services. For example, heating, ventilating, and air conditioning equipment is used by hospitals to ensure that contaminated air is confined to a Complete specified area such as an isolation room or patient ward. If computer systems used to maintain these systems were to fail, any resulting climate fluctuations could affect patient safety. Despite their importance, VHA has not yet completed assessments of its facility systems. As of February 28, 1999, VHA medical facilities reported that they had assessed 55 percent of their facility systems. According to VHA’s Director of Safety and Technical Programs, the remaining 45 percent have not been fully assessed primarily because (1) facility systems tend to be a combination of unique elements that have to be separately assessed for compliance—a time-consuming process—and (2) VHA is still awaiting compliance status information from facility-system manufacturers. VHA has not established milestones for completing assessments and implementation of compliant facility systems. To help ensure that sufficient time remains to complete these activities, we recommend that VHA consider setting such deadlines. In the event that facility-related systems and equipment do not function properly due to Y2K problems, VHA medical facilities will need to ensure that they have business continuity and contingency plans addressing how mission-critical functions affecting patient care will be carried out. According to VHA’s Director of Safety and Technical Programs, most of VHA’s facility systems have some kind of manual override or reset that will allow them to continue functioning after a Y2K problem. The director agreed, however, with the importance of developing contingency plans that fully document continued delivery of essential services in the event of a facility system failure. VHA medical facilities expect to have individual business continuity and contingency plans completed by April 30. Page 8 GAO/T-AIMD-99-152 On April 14, 1999, VA informed us that its February 28, 1999, report contained an error. The corrected numbers for facility systems (after fixing the reporting error) at the end of February were 91 percent assessed and 9 percent not assessed. Biomedical Equipment: The question of whether biomedical equipment such as magnetic resonance imaging (MRI) systems, x-ray machines, pacemakers, and Additional Status cardiac monitoring equipment can be counted on to work reliably on and Information Available, after January 1, 2000, is also critical to VHA. To the extent that this equipment uses embedded computer chips, it is vulnerable to the Y2K But Test Results Not problem. Such vulnerability carries with it possible safety risks. This could Reviewed range from the more benign—such as incorrect formatting of a printout— to the most serious—such as incorrect operation of equipment with the potential to adversely affect the patient. The degree of risk depends in large part on the role the equipment plays in a patient’s care. Additional Biomedical Last September we testified that VHA was making progress in assessing its Equipment Status biomedical equipment, but that it did not know the full extent of the Y2K problem with this equipment because it had not received compliance Information Available information from 398 manufacturers (26.7 percent).7 According to VHA, as of March 16, 1999, the number of nonresponsive manufacturers had been reduced to 126 (8.5 percent).8 As shown in table 2, about 19 percent of the manufacturers in VHA’s database of suppliers had at least one biomedical equipment item that was either noncompliant or conditionally compliant. 7Year2000 Computing Crisis: Leadership Needed to Collect and Disseminate Critical Biomedical Equipment Information (GAO/T-AIMD-98-310, September 24, 1998). 8 According to VHA, 101 of the 126 letters requesting compliance information sent to manufacturers were marked “return to sender.” Page 9 GAO/T-AIMD-99-152 Table 2: Status of Manufacturer Responses to VHA as of March 16, 1999 Percentage of Category Number of manufacturers manufacturers Compliant manufacturersa 816 55.2 Noncompliant manufacturersb 126 8.5 Conditional-compliant manufacturersc 156 10.5 Pending manufacturersd 29 2.0 Manufacturers merged or bought out 226 15.3 Nonresponsive manufacturerse 126 8.5 Total 1,479 100.0 a For inclusion in this category, 100 percent of the manufacturer’s products had to be considered compliant. bFor inclusion in this category, only one of the manufacturer’s products had to be considered noncompliant. c For inclusion in this category, the manufacturer had to have no noncompliant equipment, no equipment pending, and at least one conditional-compliant item. d For inclusion in this category, the manufacturer had to have no noncompliant equipment and at least one equipment item pending. eFor inclusion in this category, VHA had to have no compliance information from the manufacturer. Source: VHA. We did not independently verify these data. To identify specific biomedical equipment in the inventories of VHA’s medical facilities that still require Y2K compliance status information from manufacturers, VHA’s Chief Network Officer sent a letter to the directors of VHA's 22 Veterans Integrated Service Networks (VISN). This letter requested that they (1) review VHA’s list of manufacturers that have yet to respond to VHA’s request for compliance information and compare it with a list of manufacturers from which their medical facilities still require compliance information and (2) indicate the equipment item that the facility owns for each manufacturer. According to VHA’s Y2K project director, as of mid-March—with 135 of 147 medical reporting sites— 47 biomedical equipment items involving 35 manufacturers were identified as still requiring compliance status information. The project director told us that VHA medical facilities have been instructed to replace or eliminate equipment in their inventories for which they do not know the compliance status by June 30. According to VHA's February 1999 status report on medical devices, medical facilities estimated that the total cost of renovations will be about $41 million. Page 10 GAO/T-AIMD-99-152 We have previously reported that most manufacturers citing noncompliant products listed incorrect display of date and/or time as the Y2K problem.9 According to VA, these cases do not present a risk to patient safety because health care providers, such as physicians and nurses, can work around the problem. Of more serious concern are situations in which devices depend on date calculations—the results of which can be incorrect. One manufacturer cited the example of a product used for planning delivery of radiation treatment using a radioactive isotope as the source. An error in calculating the strength of the radiation source on the day of treatment could result in a dose that is too high or too low, which could have an adverse effect on the patient. Other examples of equipment presenting risk to patient safety identified by manufacturers to FDA include hemodialysis delivery systems; therapeutic apheresis systems;10 alpha-fetoprotein kits for neural tube defects;11 various types of medical imaging equipment; and systems that store, track, and recall images in chronological order. To track the compliance status of its biomedical equipment, VHA uses a monthly status report on medical devices based on information provided by the VISNs. According to the February 1999 report, approximately 426,000 of 531,000 medical devices in VHA medical facilities are Y2K compliant. Of the remaining devices, 86,452 were identified as conditional- compliant or were not assessed for Y2K compliance because the manufacturers certified that the equipment contained no software or embedded chips, and 19,073 were reported as being noncompliant. Of the noncompliant devices identified, 15,621 are to be repaired, 1,582 are to be replaced, 757 are to be used as is, 255 are to be retired, and 858 are still awaiting a decision on the remedy. According to VHA’s Chief Biomedical Engineer, most of the noncompliant devices identified incorrectly displayed date/time. As we reported last September, FDA was also trying to determine the Y2K compliance status of biomedical equipment.12 Its goal is to provide a 9 Year 2000 Computing Crisis: Compliance Status of Many Biomedical Equipment Items Still Unknown (GAO/AIMD-98-240, September 18, 1998). 10 Such equipment allows therapeutic apheresis, which refers to the exchange or purification of blood plasma. Therapeutic apheresis is recognized as a successful treatment for more than 40 autoimmune diseases. 11Devices that use computer calculations of gestational status to help assess the risk of neural tube defects in the fetuses of pregnant women. 12 GAO/AIMD-98-240, September 18, 1998. Page 11 GAO/T-AIMD-99-152 comprehensive, centralized source of information on the Y2K compliance status of biomedical equipment used in the United States and to make this information publicly available on a web site. At the time, however, FDA had a disappointing response rate from manufacturers to its letter requesting compliance information. And, while FDA made this information available to the public, it was not detailed enough to be useful. Specifically, FDA’s list of compliant equipment lacked information on particular make and model. To provide more detailed information on the compliance status of biomedical equipment, as well as to integrate more detailed compliance information gathered by VHA, we recommended that VA and the Department of Health and Human Services (HHS) jointly develop a single data clearinghouse that provides such information to all users. We said development of the clearinghouse should involve representatives from the health-care industry, such as the Department of Defense and the Health Industry Manufacturers Association. We recommended that the clearinghouse contain such information as (1) the compliance status of all biomedical equipment by make and model and (2) the identity of manufacturers that are no longer in business. We also recommended that VHA and FDA determine what actions should be taken regarding biomedical equipment manufacturers that have not provided compliance information. In response to our recommendation, FDA—in conjunction with VHA—has established the Federal Year 2000 Biomedical Equipment Clearinghouse. With the assistance of VHA, the Department of Defense, and the Health Industry Manufacturers Association, FDA has made progress in obtaining compliance-status information from manufacturers. For example, according to FDA, 4,251 biomedical equipment manufacturers had submitted data to the clearinghouse as of April 5, 1999. As shown in figure 1, about 54 percent of the manufacturers reported having products that do not employ a date, while about 16 percent reported having date-related problems such as incorrect display of date/time. FDA is still awaiting responses from 399 manufacturers. Page 12 GAO/T-AIMD-99-152 Figure 1: Biomedical Equipment Compliance-Status Information Reported to FDA by Manufacturers as of April 5, 1999 2,500 2,299 2,000 1,500 1,000 880 669 403 500 0 ot ing te- d rte ite one p loy da po eb s d h e ts t em iant t wi lems r w uc a da cts mpl ts u s ’s od u uc prob tat urer Pr ploy rod co o d t s t em l p re Pr ated uc fac Al te a rel od u Pr man da in Note: Total number of manufacturers = 4,251. Source: FDA. FDA has also expanded the information in the clearinghouse. For example, users can now find information on manufacturers that have merged with or have been bought out by other firms. In collaboration with the National Patient Safety Partnership,13 FDA is in the process of obtaining more detailed information from manufacturers on noncompliant products, such as make and model and descriptions of the impact of the Y2K problem on products left uncorrected. For example, FDA sent a letter dated March 29, 1999, to medical device manufacturers requesting that they submit to the clearinghouse complete lists of individual product models that are Y2K compliant. 13 The National Patient Safety Partnership is a coalition of public and private health care providers, including VA, the American Medical Association, the American Hospital Association, the American Nurses Association, and the Joint Commission on Accreditation of Healthcare Organizations. Page 13 GAO/T-AIMD-99-152 Review of Biomedical We reported last September that VHA and FDA relied on manufacturers to Equipment Test Results validate, test, and certify that equipment is Y2K compliant.14 We also reported that there was no assurance that the manufacturers adequately Lacking addressed the Y2K problem for noncompliant equipment because FDA did not require medical device manufacturers to submit test results to it certifying compliance. Accordingly, we recommended that VA and HHS take prudent steps to jointly review manufacturers’ compliance test results for critical care/life support biomedical equipment. We were especially concerned that VA and FDA review test results for equipment previously determined to be noncompliant but now deemed by manufacturers to be compliant, or equipment for which concerns about compliance remain. We also recommended that VA and HHS determine what legislative, regulatory, or other changes were necessary to obtain assurances that the manufacturers’ equipment was compliant, including performing independent verification and validation of the manufacturers’ certifications. At the time, VA stated that it had no legislative or regulatory authority to implement the recommendation to review test results from manufacturers. In its response, HHS stated that it did not concur with our recommendation to review test results supporting medical device manufacturers’ certifications that their equipment is compliant. It believed that the submission of appropriate certifications of compliance was sufficient to ensure that the certifying manufacturers are in compliance. HHS also stated that it did not have the resources to undertake such a review, yet we are not aware of HHS’ requesting resources from the Congress for this purpose. More recently, VHA’s Chief Biomedical Engineer told us that VHA medical facilities are not requesting test results for critical care/life support biomedical equipment; they also are not currently reviewing the test results available on manufacturers’ web sites. He said that VHA’s priority is determining the compliance status of its biomedical equipment inventory and replacing noncompliant equipment. The director of FDA’s Division of Electronics and Computer Science likewise said FDA sees no need to question manufacturers’ certifications. 14 GAO/AIMD-98-240, September 18, 1998. Page 14 GAO/T-AIMD-99-152 In contrast to VHA’s and FDA’s positions, some hospitals in the private sector believe that testing biomedical equipment is necessary to prove that they have exercised due diligence in the protection of patient health and safety. Officials at three hospitals told us that their biomedical engineers established their own test programs for biomedical equipment and, in many cases, contacted the manufacturers for their test protocols. Several of these engineers informed us that their testing identified some noncompliant equipment that the manufacturers had earlier certified as compliant. According to these engineers, to date, the equipment found to be noncompliant all had display problems and was not critical care/life support equipment. We were told that equipment found to be incorrectly certified as compliant included a cardiac catheterization unit, a pulse oxymeter, medical imaging equipment, and ultrasound equipment. VHA, FDA, and the Emergency Care Research Institute15 continue to believe that manufacturers are best qualified to analyze embedded systems or software to determine Y2K compliance. They further believe that manufacturers are the ones with full access to all design and operating parameters contained in the internal software or embedded chips in the equipment. VHA believes that such testing can potentially cause irreparable damage to expensive health care equipment, causing it to lock up or otherwise cease functioning. Further, a number of manufacturers also have recommended that users not conduct verification and validation testing. We continue to believe that rather than relying solely on manufacturers' certifications, organizations such as VHA or FDA can provide users of medical devices with a greater level of confidence that the devices are Y2K compliant through independent reviews of manufacturers’ compliance test results. The question of whether to independently verify and validate biomedical equipment that manufacturers have certified as compliant is one that must be addressed jointly by medical facilities’ clinical staff, biomedical engineers, and corporate management. The overriding criterion should be ensuring patient health and safety. 15 An international, nonprofit health services research agency. This organization believes that superficial testing of biomedical equipment by users may provide false assurances, as well as create legal liability exposure for health care institutions. Page 15 GAO/T-AIMD-99-152 VHA Pharmaceutical Another critical component to VA’s ability to deliver health care at the turn of the century is ensuring that the automated systems supporting VHA’s Operations Also Face medical facility pharmacies and its consolidated mail outpatient Y2K Risks pharmacies (CMOP) are Y2K compliant. VHA reported that in 1998 it filled about 72 million prescriptions for 3.4 million veterans, at an estimated cost of about $2 billion. About half of the prescriptions were filled by the over 200 pharmacies located in VA’s medical centers, clinics, and nursing homes. These pharmacies rely on the pharmaceutical applications in the Veterans Health Information Systems and Technology Architecture (VISTA) for (1) drug distribution and inventory management, (2) dispensing of drugs to inpatients and outpatients, (3) patient medication information, and (4) an electronic connection between the pharmacies and the CMOPs. Y2K failures in these applications could impair the pharmacies’ ability to fill prescriptions. The remaining 50 percent of VHA’s prescriptions are filled by seven CMOPs, geographically located throughout the United States. These facilities are supported by automated systems provided by one of two contractors— SI/Baker, Inc. and Siemens ElectroCom.16 For example, the CMOP electronically receives a prescription for a veteran through the medical center. The prescription is downloaded to highly automated dispensing equipment to be filled. The filled prescription is then validated by a pharmacist who compares the medication against a computerized image of the prescribed medication. Afterward, the prescription is packaged and an automatically generated mailing label is applied for delivery to the veteran. Finally, the medical center is electronically notified that the prescription has been filled. Because of the reliance on automation, the CMOPs’ ability to fill prescriptions could be delayed or interrupted if a Y2K failure occurred. VHA has determined that the automated systems supporting its CMOPs are not Y2K compliant. Specifically, neither of the systems provided by their contractors is Y2K compliant. According to the Y2K coordinator for the SI/Baker facilities, failure to make the SI/Baker systems Y2K compliant may delay the filling of outpatient prescriptions. The SI/Baker systems are used by three of VHA’s CMOPs—Hines, Illinois; Charleston, South Carolina; and Murfreesboro, Tennessee; they handle about 58 percent of all prescriptions filled by CMOPs. In contrast to the SI/Baker systems, according to a contractor hired by the CMOPs that use these systems, 16 These include operating systems, databases, and pharmacy fulfillment application software. Page 16 GAO/T-AIMD-99-152 failure to make the Siemens ElectroCom systems Y2K compliant may result in delays in processing management reports for prescriptions filled, but not in the actual filling of prescriptions. Although the CMOPs plan to replace their noncompliant systems with compliant ones, these systems are not scheduled to be implemented until mid- to late-1999. As shown in table 3, the earliest estimated completion date for implementing a compliant system is June 30, 1999, while the latest is December 1, 1999.17 This leaves little time to address any unexpected implementation problems. Table 3: Schedule of Estimated Implementation Completion Dates and Current Daily Workload by Consolidated Mail Outpatient Pharmacies Estimated completion Current daily workload Location date (prescriptions filled) West Los Angeles, June 30, 1999 Californiaa 15,000 Bedford, Massachusettsa June 30, 1999 15,000 Dallas, Texasa June 30, 1999 14,000 Leavenworth, Kansasa July 31, 1999 16,000 Charleston, South September 1, 1999 Carolinab 23,000 Murfreesboro, Tennesseeb September 30, 1999 38,000 Hines, Illinoisb December 1, 1999 21,000 a Siemens ElectroCom automation. bSI/Baker, Inc. automation. Source: VA. Given the late schedule for implementing compliant systems, it is crucial that the CMOPs develop business continuity and contingency plans to ensure that veterans will continue to receive their medications if these systems are not implemented in time or fail to operate properly. As of March 31, VA had not completed a business continuity and contingency plan for the CMOPs. The Y2K coordinator for the Siemens ElectroCom 17 In April 15, 1999, testimony before the Subcommittee on Oversight and Investigations, Committee on Veterans’ Affairs, House of Representatives, VHA’s Y2K project director said that estimated implementation dates would be revised so that the latest date is August 30, 1999. Page 17 GAO/T-AIMD-99-152 system has been tasked with developing this plan, which is to be completed by the end of this month. Further, VA did not include the CMOP systems in its quarterly reports of mission-critical systems to OMB. According to VHA’s Y2K project director, VHA considered the CMOP systems to be COTS products and, therefore, did not report them as mission-critical systems. Given the criticality of these systems to VHA’s ability to fill prescriptions at the turn of the century, we believe VA should reassess this decision. Reporting CMOPs as mission- critical to VA top management and OMB would help ensure that necessary attention is paid and action is taken. VA Taking Action to VA, like other users of pharmaceutical and medical-surgical products, Determine Y2K Readiness of needs to know whether it will have a sufficient supply of these items for its customers. Therefore, it has taken a leadership role in the federal Pharmaceutical and government in determining whether manufacturers supplying these Medical-Surgical products to VHA are Y2K-ready. This information is essential to VHA’s Manufacturers medical facilities and CMOPs because of their “just-in-time”18 inventory policy. Accordingly, they must know whether their manufacturers’ processes, which are highly automated,19 are at risk, as well as whether the rest of the supply chain will function properly. To determine the Y2K readiness of their suppliers, on January 8, 1999, VA’s National Acquisition Center (NAC)20 sent a survey to 384 pharmaceutical firms and 459 medical-surgical firms with which it does business. The survey contained questions on the firms’ overall Y2K status and inquired about actions taken to assess, inventory, and plan for any perceived impact that the century turnover would have on their ability to operate at normal levels. In addition, the firms were asked to provide status information on progress made to become Y2K compliant and a reliable estimated date when compliance will be achieved for business processes such as (1) ordering and receipt of raw materials, (2) mixing and processing product, (3) completing final product processing, (4) packaging and 18 This term refers to maintaining a limited inventory on hand. 19Pharmaceutical manufacturers rely on automated systems for production, packaging, and distribution of their products, as well as for ordering of raw materials and supplies. 20 This organization is responsible for supporting VHA’s health care delivery system by providing an acquisition program for items such as medical, dental, and surgical supplies and equipment; pharmaceuticals; and chemicals. NAC is part of VA's Office of Acquisition and Materiel Management. Page 18 GAO/T-AIMD-99-152 labeling product, and (5) distributing finished product to distributors/ wholesalers and end customers. According to NAC officials, of the 455 firms that responded to the survey as of March 31, 1999, about 55 percent completed all or part of the survey. The remainder provided general information on their Y2K readiness status or literature21 on their efforts. As shown in table 4, more than half (52 percent) of the pharmaceutical firms surveyed responded, with just less than one-third (32 percent) of those respondents reporting that they are compliant. Among the pharmaceutical firms that had not responded as of March 31, however, were two of VA’s five largest suppliers.22 The three large pharmaceutical suppliers that did respond provided general information on their Y2K readiness status, rather than answering the survey, and estimated that they will be compliant by June 30, 1999. Table 4: Status of Companies Surveyed by VHA as of March 31, 1999 Responses Pharmaceutical Medical-surgical Y2K compliant 65 166 Will be compliant by 1/1/2000 or earliera 90 70 Provided no compliant date 50 14 Total number of responses 205 250 Non-responses 179 209 Total number of firms surveyed 384 459 a Estimated compliance status ranged from March 31, 1999 through January 1, 2000; about 71 percent of pharmaceutical firms and 80 percent of medical-surgical firms estimated they will be compliant by July 31, 1999. One firm responded that it will be compliant by January 1, 2000. Source: VA. We did not independently verify these data. Table 4 also shows that 54 percent of the medical-surgical firms surveyed responded, with about two-thirds (166) of them reporting that they are Y2K compliant. All five of VA’s largest medical-surgical suppliers have responded. Specifically, two reported being compliant, two reported they would be compliant by June 30, 1999, and the remaining supplier did not report an expected compliance date. 21This includes annual and quarterly financial reports required by the Securities and Exchange Commission for companies listed on the New York Stock Exchange. 22 On April 14, 1999, a NAC official told us that of the two suppliers that had not responded as of March 31, one responded on April 12, and the other responded on April 14. Page 19 GAO/T-AIMD-99-152 On March 17, 1999, NAC sent a second letter to its pharmaceutical and medical-surgical suppliers, informing them of VA’s plans to make Y2K readiness information previously provided to VA available to the public through a web site (www.va.gov/oa&mm/nac/y2k). VA made the survey results available on its web site on April 13, 1999. The letter also requested that manufacturers that had not previously responded provide information on their readiness. NAC’s Executive Director said that he would personally contact any major VA supplier that does not respond. On a broader level, VHA has taken a leadership role in obtaining and sharing information on the Y2K readiness of the pharmaceutical industry. Specifically, VHA chairs the Year 2000 Pharmaceuticals Acquisitions and Distributions Subcommittee, which reports to the Chair of the President’s Council on Year 2000 Conversion. The purpose of this subcommittee is to bring together federal and pharmaceutical representatives to address issues concerning supply and distribution as they relate to the year 2000. The subcommittee consists of FDA, federal health care providers, and industry trade associations such as the Pharmaceutical Research and Manufacturers of America (PhRMA), Generic Pharmaceutical Industry Association, the National Association of Chain Drug Stores, National Wholesale Druggists’ Association, and consumer advocates. Several of these trade associations have surveyed their members on their Y2K readiness and made the results available to the public. However, the information is not manufacturer- specific or as detailed as VHA's survey results. FDA’s Y2K Efforts for FDA’s oversight and regulatory responsibility for pharmaceutical and Pharmaceutical and biological products23 is to ensure that they are safe and effective for public use. Because of its concern about the Y2K impact on manufacturers of Biological Products these products, FDA has taken several actions to raise the Y2K awareness Industries Were Initially of the pharmaceutical and biological products industries. In addition, it is Focused on Awareness thinking about conducting a survey to determine the industry’s Y2K readiness. One of FDA’s actions to raise industry awareness was the January 1998 issuance of industry guidance by the Center for Biologics Evaluation and Research (CBER) on the Y2K impact of computer systems and software applications used in the manufacture of blood products. In addition, as shown in table 5, FDA has issued several letters to pharmaceutical and biological trade associations and sole-source drug manufacturers. 23 Biological products include vaccines, blood, and blood products. Page 20 GAO/T-AIMD-99-152 Table 5: FDA Letters to Manufacturers Regarding Y2K Date FDA source Recipient Purpose October Center for Pharmaceutical To relay to members FDA’s expectation that the 1998 Drug Evaluation and manufacturer trade pharmaceutical industry would (1) make resolution of Research associations Y2K a high priority, (2) ensure that production systems were fixed and tested prior to January 1, 2000, and (3) urge manufacturers to develop Y2K contingency plans. October Center for Biologics Biologics manufacturer trade Same as above. 1998 Evaluation and Research associations January 1999 Center for Sole-source drug Same as above. Also (1) noted that the impact of Y2K Drug Evaluation and manufacturers on pharmaceutical safety, efficacy, and availability Research merits special attention for firms who are the sole manufacturers of drug components, bulk ingredients, and finished products and (2) stated that pharmaceutical industry suppliers must have Y2K- compliant systems to protect against disruption in the flow of product components, packaging materials, and equipment to pharmaceutical manufacturers. Source: FDA. Further, on February 11, 1999, FDA’s director of emergency and investigation operations sent a memorandum on FDA’s interim inspection policy for the Y2K problem to the directors of FDA’s investigations branch. The policy emphasizes FDA’s Y2K awareness efforts for manufacturers. It states that FDA inspectors are to (1) inform the firm of FDA’s Y2K web page (URL http://www.fda.gov/cdrh/yr2000/year2000.html), (2) provide the firm with copies of the appropriate FDA Y2K awareness letter, (3) explain that Y2K problems could potentially affect aspects of the firm’s operations, including some areas not regulated by FDA, and that FDA anticipates that firms will take prudent steps to ensure that they are not adversely affected by Y2K, and (4) provide firms with a copy of FDA’s compliance policy guide “Year 2000 (Y2K) Computer Problems.” In addition, FDA and PhRMA jointly held a government/industry forum on the Y2K preparedness of the pharmaceutical and biotech industries on February 22, 1999. The objectives of this forum were to (1) share information on Y2K programs conducted by health care providers, pharmaceutical companies, FDA, and other federal agencies, (2) provide a vehicle for networking, and (3) raise awareness. On March 29, 1999, FDA revised its February 11, 1999, interim inspection policy. The revision states that field inspectors are now to inquire about Page 21 GAO/T-AIMD-99-152 manufacturers’ efforts to ensure that their computer-controlled or date- sensitive manufacturing processes and distribution systems are Y2K compliant. Inspectors are to include this information in their reports, along with a determination of activities that firms have completed or started to ensure that they will be Y2K compliant. Further, FDA inspectors may review documentation in cases in which firms have made changes to their computerized production or manufacturing control systems to address Y2K problems. The purpose of this review is to ensure that the changes were made in accordance with the firms’ procedures and applicable regulations. If inspectors determine that a firm has not taken steps to ensure Y2K compliance, they are to notify their district managers and the responsible FDA center. FDA’s interim policy describes steps inspectors are to take in reviewing manufacturers’ Y2K compliance. However, FDA stated that the primary focus of its inspections for the remainder of 1999 will be to ensure that products sold in the United States are safe and effective for public use and comply with federal statutes and regulations, including “good manufacturing practice” (GMP).24 FDA officials explained that the agency does not have sufficient resources to perform both regulatory oversight of the manufacturers and in-depth evaluations of firms’ Y2K compliance activities. Nevertheless, according to the March 29, 1999, memorandum, field inspectors are to note any concerns they may have with a firm’s Y2K readiness in the administrative remarks section of their inspection reports. These reports are to be reviewed by FDA district managers. If the Y2K concern appears to present a serious problem to a firm’s ability to produce safe, effective medication, the district manager can discuss this issue with FDA’s Office of Regulatory Affairs and determine a course of action. However, FDA officials have stressed that the agency cannot take any regulatory action toward the firm until a Y2K-related problem affects a pharmaceutical or biological product. Like VHA, FDA is interested in the impact of Y2K readiness of pharmaceutical and biological products on the availability of products for health care facilities and individual patients. FDA’s Acting Deputy 24 GMP requirements include federal standards for ensuring that products are high in quality and produced under sanitary conditions (21 CFR parts 210, 211). Page 22 GAO/T-AIMD-99-152 Commissioner for Policy informed us on March 24, 1999, that the agency is thinking about surveying pharmaceutical and biological products manufacturers, distributors, product repackagers, and others in the drug dispensing chain, on their Y2K readiness and contingency planning. In anticipation of a possible survey, the agency has published a notice in the March 22, 1999, Federal Register regarding this matter. The Acting Deputy Commissioner said that potential survey questions on contingency planning would include steps the manufacturers are taking to ensure an adequate supply of bulk manufacturing materials from overseas suppliers. This is a key issue because, as we reported in March 1998,25 according to FDA, as much as 80 percent of the bulk pharmaceutical chemicals used by U.S. manufacturers to produce prescription drugs is imported. In summary, VBA and VHA continue to make progress in preparing their mission-critical systems for the year 2000. However, key actions remain to be taken in the areas of mission-critical systems testing, VHA facility systems compliance, and CMOP systems compliance. We also reiterate the need for VHA and FDA to take prudent steps to ensure that the test results of critical care/life support biomedical equipment are obtained and reviewed. Finally, VHA needs information on the Y2K readiness of specific pharmaceutical and medical-surgical manufacturers. Until this information is obtained and publicized, VHA medical facilities and veterans will remain in doubt as to whether an adequate supply of pharmaceutical and biological products will be available. FDA and the pharmaceutical and biological trade associations can play key roles in helping VHA obtain this information and publicize the results in a single data clearinghouse. In carrying out this assignment, we reviewed and analyzed VA's Y2K documents and plans, comparing them against our guidance on Y2K activities. We also reviewed and analyzed FDA documentation relating to its Y2K efforts on biomedical devices and pharmaceutical manufacturers. In addition, we visited selected VHA medical centers, VA data centers, and VHA consolidated mail outpatient pharmacies to discuss their Y2K activities, and interviewed VA and FDA officials on those activities. We also interviewed officials of the Emergency Care Research Institute regarding their statements on biomedical equipment testing. Finally, we interviewed selected private hospital officials about their Y2K actions and 25 Food and Drug Administration: Improvements Needed in the Foreign Drug Inspection Program (GAO/HEHS-98-21, March 17, 1998). Page 23 GAO/T-AIMD-99-152 pharmaceutical trade associations on their Y2K readiness surveys of pharmaceutical manufacturers. Mr. Chairman, this concludes my statement. I would be pleased to respond to any questions that you or other members of the Committee may have at this time. (511748) Leter Page 24 GAO/T-AIMD-99-152 Ordering Information The first copy of each GAO report and testimony is free. Additional copies are $2 each. Orders should be sent to the following address, accompanied by a check or money order made out to the Superintendent of Documents, when necessary, VISA and MasterCard credit cards are accepted, also. Orders for 100 or more copies to be mailed to a single address are discounted 25 percent. Orders by mail: U.S. General Accounting Office P.O. Box 37050 Washington, DC 20013 or visit: Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders may also be placed by calling (202) 512-6000 or by using fax number (202) 512-6061, or TDD (202) 512-2537. Each day, GAO issues a list of newly available reports and testimony. To receive facsimile copies of the daily list or any list from the past 30 days, please call (202) 512-6000 using a touchtone phone. A recorded menu will provide information on how to obtain these lists. For information on how to access GAO reports on the INTERNET, send an e-mail message with “info” in the body to: firstname.lastname@example.org or visit GAO’s World Wide Web Home Page at: http://www.gao.gov United States Bulk Mail General Accounting Office Postage & Fees Paid Washington, D.C. 20548-0001 GAO Permit No. GI00 Official Business Penalty for Private Use $300 Address Correction Requested
Year 2000 Computing Crisis: Key Actions Remain to Ensure Delivery of Veterans Benefits and Health Services
Published by the Government Accountability Office on 1999-04-20.
Below is a raw (and likely hideous) rendition of the original report. (PDF)