United States General Accounting Office GAO Report to the Chairman and Ranking Minority Member, Subcommittee on Military Personnel, Committee on Armed Services, House of Representatives September 1999 DEFENSE HEALTH CARE Appointment Timeliness Goals Not Met; Measurement Tools Need Improvement GAO/HEHS-99-168 United States GAO General Accounting Office Washington, D.C. 20548 Health, Education, and Human Services Division B-279619 September 30, 1999 The Honorable Steve Buyer Chairman The Honorable Neil Abercrombie Ranking Minority Member Subcommittee on Military Personnel Committee on Armed Services House of Representatives Over the past decade, the Department of Defense (DOD) has faced the same challenges in delivering health care to its beneficiaries as the nation’s health care system has for the general population, including increasing costs and uneven access to care. In 1993, after years of demonstration programs designed to explore options to manage the delivery of health care more effectively, DOD restructured its health care system into TRICARE, its managed care program. Today, about 8.2 million active-duty personnel, their dependents, and retirees are eligible to receive care in this $15.6 billion-per-year health-care system. Care for eligible beneficiaries is provided mostly in military treatment facilities (MTFs), supplemented by networks of contracted civilian providers. To help ensure timely access to care, TRICARE established appointment timeliness standards and goals similar to those of private health plans for the beneficiaries who choose to enroll in TRICARE’s health maintenance organization option, called Prime. While TRICARE was designed in part to improve beneficiaries’ access to health care, beneficiaries have complained about the difficulties they encounter obtaining care, including the length of time needed to get an appointment. As you requested, this report provides information on DOD’s performance in scheduling appointments, and possible reasons why Prime enrollees might not obtain appointments within the appointment timeliness goals. We also provide information on improvements needed to DOD’s measurement tools. We conducted our work between April 1998 and June 1999 in accordance with generally accepted government auditing standards. See appendix I for our scope and methodology. After correcting definitional discrepancies in DOD data, we found DOD has Results in Brief not achieved its goal of scheduling 98 percent of acute and routine appointments within the timeliness standards it established. About 70 percent of appointments for a routine visit at MTFs were scheduled within the standard, while between 80 and 97 percent of appointments for Page 1 GAO/HEHS-99-168 Access to Defense Health Care B-279619 acute care, preventive care, or specialists were scheduled within the relevant standards. DOD’s analysis of appointment timeliness is consistent with our findings, and the Department has reported that the MTFs’ performance has fallen short of its expectations. There are several reasons why active duty members and other enrollees may not obtain appointments within the standards. For example, Prime beneficiaries sometimes request an appointment date later than the one offered that was within the standard, although DOD does not have the data needed to identify the actual number of these requests. Another factor is the extent to which MTFs provide care to nonenrolled beneficiaries. We found that about 16 percent of the appointment slots were given to nonenrolled beneficiaries. DOD permits nonenrollees, including retirees over age 65, to make appointments and obtain care in MTFs because it believes treating these beneficiaries is necessary to support medical readiness and training requirements. DOD has made no analysis, however, of the extent to which this policy adversely affects the ability of the enrolled population to obtain care and treatment or the effect of any resulting shortfall on readiness and training. Another factor affecting appointment availability is that military beneficiaries traditionally utilize health care at a higher rate than do private-sector beneficiaries. Research by the Congressional Budget Office (CBO) has shown that instituting a copayment for care provided in MTFs could reduce demand for care and improve appointment timeliness by freeing up appointments for active-duty members and other Prime enrollees. As currently configured, DOD’s data tools—its Customer Satisfaction Survey and Composite Health Care System (CHCS) appointment scheduling system—are inadequate for measuring appointment timeliness against the access standards. Survey weaknesses include reliance on the beneficiaries’ ability to correctly recall details of the appointments, a low response rate, and no analysis of the beneficiaries who do not respond—all of which affect the accuracy of the information on how well appointment standards were met. CHCS also has weaknesses. In particular, the appointment names used in the MTF’s appointment scheduling system do not directly relate to the access standards. Although DOD has some efforts under way to improve its Survey, the efforts will not overcome its inherent weaknesses, such as its reliance on beneficiary recall. DOD also has several efforts under way to improve the data contained in the CHCS appointment scheduling system, including standardizing the appointment names across the military health-care system and associating them with the timeliness standards. Once implemented, CHCS promises to become a good source of the Page 2 GAO/HEHS-99-168 Access to Defense Health Care B-279619 appointment timeliness information DOD needs to effectively manage and monitor access to care. This report makes recommendations to the Secretary of Defense to improve appointment timeliness measurement and access to care for active-duty members and other Prime enrollees. DOD’s primary medical mission is to maintain the health of 1.6 million Background active-duty service personnel and to provide health care to them during military operations. DOD additionally offers health care to 6.6 million nonactive-duty beneficiaries, including dependents of active-duty personnel, military retirees, and dependents of retirees. Under TRICARE, most care is provided in MTFs worldwide and is supplemented by civilian providers. TRICARE is a triple-option health benefit program designed to give beneficiaries a choice among a health maintenance organization (Prime), a preferred provider organization (Extra), and a fee-for-service benefit (Standard).1 TRICARE Prime is the only option for which beneficiaries must enroll; active-duty members are automatically enrolled in Prime. Active-duty family members and retirees and their dependents under age 65 are also eligible to enroll in Prime. Retirees and their dependents and survivors over age 65 are not eligible to enroll in Prime, but can still obtain care in MTFs if space or resources are available. Beneficiaries can also obtain care from civilian providers. Beneficiaries who obtain care within the MTFs, including Prime enrollees, pay nothing for their outpatient visits. However, beneficiaries obtaining care from civilian providers are subject to out-of-pocket costs ranging from 25 percent of the allowable charge for a TRICARE Standard office visit to a copayment of $6 or $12 for a Prime enrollee visiting a provider outside the MTF but in the TRICARE network.2 Under section 712 of the National Defense Authorization Act for Fiscal Year 1996 (P.L. 104-106), DOD was required to establish priorities for accessing care within MTFs.3 Under DOD’s implementing policy, active-duty personnel have highest priority, followed by active-duty family members enrolled in Prime; retirees, their family members, and survivors enrolled in Prime; nonenrolled active-duty family members; and nonenrolled retirees, their family members, and survivors. In addition, DOD policy specifies that MTF commanders have the discretion to grant exceptions to the access 1 DOD previously provided health care under the Civilian Health and Medical Program of the Uniformed Services, a fee-for-service program. 2 Dependents of lower-rank active-duty members pay $6 for an outpatient visit, while dependents of higher-rank active-duty members, and retirees and their dependents and survivors, pay $12. 3 10 U.S.C. section 1097(c). Page 3 GAO/HEHS-99-168 Access to Defense Health Care B-279619 priority rules for various reasons, such as giving groups or individuals higher priority to meet requirements of graduate medical education programs. To better ensure timely access to health care, DOD established appointment timeliness standards for Prime enrollees similar to the standards used in private-sector managed care programs. DOD’s standards, which apply to MTFs and the civilian network, established the following maximum wait times between the day a Prime enrollee requests an appointment with his or her primary-care physician and the actual date of the visit: • 1 day for acute illness care, defined as visits requiring physician intervention and urgent in nature; • 1 week for routine visits, defined as requiring physician intervention but nonurgent in nature; • 4 weeks for well visits, defined as health maintenance and prevention, and nonurgent in nature; and • 4 weeks for specialty care referrals from a primary-care physician to a specialist. In June 1998, DOD established a goal that at least 98 percent of acute and routine primary-care appointments for Prime enrollees should be scheduled within the time allowed by the standards. In March 1999, DOD lowered its 98-percent goal to what DOD considers a more achievable goal of 90 percent because most of the MTFs failed to meet the 98-percent goal. According to a DOD official, lowering the target to 90 percent provides more opportunity for MTFs to achieve DOD’s established goal. Section 713 of the Strom Thurmond National Defense Authorization Act for Fiscal Year 1999 (P.L. 105-261) established requirements for DOD to collect data on the timeliness of appointments in order to measure performance in meeting the primary-care access standards established under TRICARE.4 This requirement is consistent with the Government Performance and Results Act of 1993, which requires agencies to define their missions clearly, set goals, measure performance, and report on their accomplishments. DOD uses information from a Customer Satisfaction Survey to meet this legislative requirement. The Survey asks a sample of patients a number of questions about a specific visit with a particular medical provider in an MTF, including the severity of the need for the visit (such as whether the visit was urgent or routine), the number of days between requesting the appointment and the actual appointment date, and 4 10 U.S.C 1073. Page 4 GAO/HEHS-99-168 Access to Defense Health Care B-279619 their satisfaction with the care they received. DOD aggregates the results of selected questions as a measure of how well or poorly MTFs as a group performed in meeting the access standards. Information on appointment timeliness is also contained in the appointment-scheduling module of the CHCS system. CHCS is considered the primary health-care information system of the military health-care system, and is used by all MTFs to capture patient demographic information, schedule appointments, and to order prescriptions and ancillary services. It also contains information on the timeliness of scheduled appointments for virtually all clinics in the MTFs. Available data do not permit us or other analysts to precisely measure the DOD Experiencing extent to which DOD is meeting its access standards. However, after Difficulty Achieving correcting the DOD data for definitional discrepancies, we were able to Appointment develop an assessment of appointment timeliness. Our analysis shows that appointments obtained by Prime enrollees, including active-duty members, Timeliness Goals were not always scheduled within the timeliness standards. Furthermore, about the same percentage of appointments for nonenrolled beneficiaries were scheduled within the standards as were those for active-duty and Prime enrollees. These findings are consistent with DOD’s own analysis, which concluded that its performance in appointment timeliness has not met its expectations and goals. According to DOD officials, there are several reasons why appointments for active-duty and other Prime beneficiaries are not scheduled within the standard. These include beneficiaries turning down an appointment that was offered to them within the standard, and nonenrolled beneficiaries being scheduled for appointments that otherwise would have been available for active-duty and Prime beneficiaries. Several options exist to increase the percentage of appointments scheduled within the standards and improve access for active-duty and other enrollees. These options include DOD conducting an assessment of the extent to which medical readiness and training needs can be met without treating nonenrolled beneficiaries, and stricter enforcement of the access to care priorities based on this assessment. Also, requiring a copayment for care provided in the MTFs could reduce the traditionally higher usage of military health-care (as compared to utilization in private health plans) and help DOD achieve its appointment timeliness goals. Page 5 GAO/HEHS-99-168 Access to Defense Health Care B-279619 DOD Has Not Achieved Our analysis of Customer Satisfaction Survey and CHCS appointment data Timeliness Goal for indicates that DOD fell short of its original goal that 98 percent of acute and Active-Duty and Prime routine primary-care appointments for Prime enrollees, including active-duty members, be scheduled within the period of time set in the Enrollees standards. For example, both data sources show that only about 70 percent of routine appointments for Prime enrollees were scheduled within the required 1 week of the request for the appointment. While the 98-percent goal was in place for the time period we analyzed, the performance for scheduling acute and routine appointments was even below DOD’s lowered goal of 90 percent. Table 1 summarizes DOD’s appointment standards, goals, and the percentage of appointments within the standards for active-duty members and other Prime enrollees. Table 1: DOD Appointment Scheduling Standards, Goals, and Appointments Scheduled Within Standards for Active-Duty and Other Prime Enrollees Prime enrollee appointments Goal for Active-duty appointments scheduled within standard Appointment appointments to scheduled within standard (%) (excluding active-duty) (%) scheduling be scheduled Customer Customer standard for within standard Satisfaction Satisfaction Appointment type Prime enrollees (%) Survey dataa CHCS datab Survey dataa CHCS datab Primary care acute 1 day 98 84 91 80 92 Primary care routine 1 week 98 81 81 71 65 Primary care well 4 weeks No goal 96 91 97 81 Specialty referral 4 weeks No goal 94 96 94 91 a Data for 117 MTFs with clinics that had more than 200 visits per month for the 5-month period of May 1, 1998, to September 30, 1998. Sampling errors are no greater than +/-3 percentage points. b Data for appointments scheduled between October 1, 1997, and September 30, 1998, at five MTFs and between January 1, 1998, and December 31, 1998, at one MTF. Source: GAO analysis of DOD data. Timeliness for Prime Among the beneficiaries who obtained appointments, the percentage of Enrollees Similar to That appointments scheduled for active-duty and other Prime enrollees (those for Nonenrolled with the highest priority) within the standards was similar to the percentage of appointments within the standards for nonenrolled Beneficiaries beneficiaries (who have the lowest priority). For example, the Customer Satisfaction Survey indicates that the percentage of acute and well primary-care appointments scheduled for active-duty members within the standards (84 and 96 percent, respectively) was similar to the percentage for nonenrolled appointments (81 percent and 95 percent, respectively). Page 6 GAO/HEHS-99-168 Access to Defense Health Care B-279619 The CHCS data show that the appointment timeliness for other enrollees and the nonenrolled for all appointment types was also similar. Table 2 summarizes the appointment timeliness for active-duty members, other Prime enrollees, and nonenrolled beneficiaries. Table 2: Comparison of Appointments Scheduled Within Standards for Active-Duty, Other Prime Enrollees, and Nonenrolled Beneficiaries Prime enrollee appointments Active-duty appointments scheduled within standard Nonenrolled appointments scheduled within standard (%) (excluding active-duty) (%) scheduled within standard (%) Customer Customer Customer Satisfaction Satisfaction Satisfaction Appointment type Survey dataa CHCS datab Survey dataa CHCS datab Survey dataa CHCS datab Primary care acute 84 91 80 92 81 88 Primary care routine 81 81 71 65 69 69 Primary care well 96 91 97 81 95 78 Specialty referral 94 96 94 91 90 93 a Data for 117 MTFs with clinics that had more than 200 visits per month for the 5-month period of May 1, 1998, to September 30, 1998. Sampling errors are no greater than +/-3 percentage points. b Data for appointments scheduled between October 1, 1997, and September 30, 1998, at five MTFs and between January 1, 1998, and December 31, 1998, at one MTF. Source: GAO analysis of DOD data. While the data show similarities in the timeliness of appointments for enrolled and nonenrolled beneficiaries, it is important to note that the majority of the appointments—84 percent—were for enrolled beneficiaries. Also, there are no data showing the number of nonenrolled beneficiaries who were unable to obtain an appointment. DOD Reports Performance In October 1998, DOD reported that it had a serious problem providing in Meeting Access timely access to care, based on its analysis of Customer Satisfaction Standards Has Not Met Its Survey data for the May to July 1998 period. According to DOD, less than 15 percent of the 115 MTFs included in its analysis were able to schedule Expectations acute appointments within the standard, and DOD characterized the performance of many of the MTFs as “dismal.” Over the next 5 months, DOD said that although it had noticed some improvements, the achievement of the access standards continued to fall below its goal. In March 1999, the Executive Director of the TRICARE Management Activity stated that access must improve and tasked the Surgeons General and regional Page 7 GAO/HEHS-99-168 Access to Defense Health Care B-279619 TRICARE management offices to work with MTFs to identify access problems and make needed improvements. Reasons Why According to DOD, appointments may not be scheduled within the Appointments Are Not standards either because the beneficiary requests a later appointment for Scheduled Within the personal convenience, or because there are no appointment slots available. DOD does not have data to identify the actual number of personal Standards convenience requests, but is planning some revisions to the CHCS appointment system to capture information on whether the beneficiary accepts the first offered appointment or requests a later one. Appointment availability at the MTFs is also affected by the extent to which care is provided to nonenrolled beneficiaries. Our review of CHCS appointment data at six MTFs shows that about 16 percent of the appointments were for beneficiaries who were not enrolled in TRICARE Prime.5 According to DOD, providing medical care to other beneficiaries, including those over age 65, provides medical proficiency training that supports military medical readiness and training requirements. DOD has made no analysis of the extent to which providing care to these beneficiaries adversely affects the ability of the enrolled population to obtain care or the effect of any resulting shortfall on readiness and training. Another factor that affects the availability of appointments for active-duty and other Prime enrollees is the extent to which care in the MTFs is overutilized by beneficiaries. Studies have shown that the per-capita utilization of DOD health care services by military beneficiaries has historically been much higher than in civilian health plans, due in part to the lack of a cost-sharing requirement in MTFs. As we have previously reported, research has shown that the lack of a cost-sharing requirement leads to a higher utilization of health care.6 CBO has reported that sharing costs with beneficiaries reduces health-care utilization. In its April 1999 report, CBO concluded that requiring a copayment from beneficiaries who use MTFs would help curb excessive use.7 Furthermore, according to CBO, 5 Although we obtained appointment data from eight MTFs, we were only able to use data from six due to limitations and discrepancies in the data that could not be corrected. 6 Defense Health Care: Challenges Facing DOD in Implementing Nationwide Managed Care (GAO/T-HEHS-94-145, Apr. 19, 1994), and Addressing the Deficit: Budgetary Implications of Selected GAO Work for Fiscal Year 1998 (GAO/OCG-97-2, Mar. 14, 1997). 7 Maintaining Budgetary Discipline: Spending and Revenue Options, CBO (Washington, D.C., Apr. 1999). Page 8 GAO/HEHS-99-168 Access to Defense Health Care B-279619 concerns that increasing cost-sharing requirements could discourage beneficiaries from seeking necessary care are not well founded, especially for the military health-care beneficiaries. CBO reports that cost-sharing requirements do not prevent beneficiaries at ages and income levels typical of military beneficiaries from seeking needed care. Options to Improve Several options could improve appointment timeliness for active-duty and Appointment Timeliness other Prime enrollees. One is to more rigorously implement the access and Access priorities. Some of the MTFs we visited had procedures to give appointment priority to active-duty and other enrollees, such as specifying certain times of day for active-duty and Prime enrollees to request appointments, after which appointments were available for all beneficiaries, whether enrolled or not. However, once beneficiaries are booked into appointments, the appointment priority no longer exists. One option is to “bump” a nonenrollee who has an appointment when an enrolled beneficiary needs an appointment and none is available within the required time frame. Second, if each MTF identified what percentage of the care provided to nonenrollees was necessary to achieve their medical readiness and training requirements, the rest of the care could be reallocated to active-duty and other enrollees to improve their access. Third, establishing a beneficiary copayment for care in the MTF could reduce the demand for care in the MTF and free up more appointments for active-duty members and other Prime enrollees. A standard practice used by commercial managed care plans to bring about more appropriate utilization is requiring enrolled beneficiaries to pay a copayment for care. Commercial plan copayments for outpatient physician visits range from about $5 to $15, with most beneficiaries paying $10 per visit. DOD’s civilian copayment requirement of $6 or $12 per visit is consistent with commercial plans. While these options are intended to improve appointment timeliness and availability for enrolled beneficiaries, it is possible that the options may cause some nonenrolled beneficiaries to seek care elsewhere and experience higher out-of-pocket costs. The options may also affect the timeliness of the care they receive. However, we did not evaluate the extent to which this might occur. Page 9 GAO/HEHS-99-168 Access to Defense Health Care B-279619 While DOD has been measuring appointment timeliness, the tools it uses Weaknesses in Data have several weaknesses that limit their usefulness in providing Tools Prevent management information on the extent to which beneficiaries are Accurate Assessment obtaining appointments within the prescribed standards. For example, problems associated with the design and administration of DOD’s Customer of Appointment Satisfaction Survey, such as the accuracy of beneficiary-reported data and Timeliness the small number of visits included in the sample, prevent using the Survey to measure MTFs’ performance against the appointment timeliness standards. CHCS appointment system data can provide information on appointment timeliness at each MTF, but cannot be used to compare the data against the standards or across the military health care system unless certain modifications are made. Although DOD has efforts under way to improve the Survey, its reliance on beneficiaries’ recall of their appointment experience is an inherent weakness that fundamentally limits the Survey’s usefulness in this area. However, the efforts under way to improve CHCS should address the weaknesses and make CHCS a good source of data to measure and monitor MTF performance in scheduling appointments within the standards. Weaknesses Associated While DOD’s Customer Satisfaction Survey provides information on how With Survey Data Affect beneficiaries perceive their health care experiences, it has weaknesses Usefulness when used for measuring the performance of MTFs in meeting access standards. For example, the quality of data is entirely dependent on the beneficiary’s ability to remember the number of days it took to get a specific appointment. However, because beneficiaries can receive the survey up to 45 days after the appointment, they may have difficulty accurately recalling their experience, thus calling into question the validity of the Survey results. Another weakness is that DOD relies on the respondents to correctly classify their appointment types in their survey responses. DOD uses the respondents’ classifications to determine which access standard should be related to the appointments. The Survey asks each respondent to classify the purpose of his or her visit as one of the following: • Care for illness or injury which the patient felt required him or her to see a doctor right away; • Routine care for a nonurgent condition; • Well-patient visit for preventive care (checkup); or • Specialty care, referral visit. Page 10 GAO/HEHS-99-168 Access to Defense Health Care B-279619 However, our discussions with beneficiaries revealed that they were uncertain about the correct category for their visits. Beneficiaries did not understand the difference between a routine visit for a nonurgent condition and a well-patient visit for preventive care, and were unsure about how to categorize a follow-up visit with either a primary-care physician or specialist. Also, the design of the question intends that primary-care visits would be identified by responses to the first three categories and specialty care by selecting the fourth choice. Beneficiaries said they would select the first option if they felt that they needed care from a specialist right away. Our analysis of Survey responses confirmed the potential for this error. We found that about two-thirds of the responses from beneficiaries who received care in a specialty clinic marked one of the first three categories, and thus were misclassified as primary care. Even if beneficiaries were able to interpret the questions and report their experiences accurately, the sample size of the Survey is too small to provide precise estimates of clinic performance. Each month, DOD randomly selects 35 visits from each clinic that receives at least 200 visits per month. Given the survey response rate of 40 percent, this sample size yields about 14 responses per month for each clinic sampled. Even if data were aggregated and analyzed every 6 months, a sample size of only around 85 for the period could be expected, which would provide information only on very large changes in performance at the clinic level. For example, an increase in the appointments scheduled within the standards from 70 percent to 80 percent would not represent a statistically significant change based on a sample size of 85. The response rate of the Survey also calls into question the validity of the Survey results. While the Survey results provide information on those who responded, DOD knows little about the experiences of the 60 percent or more of surveyed beneficiaries who did not respond.8 Without conducting a nonrespondent analysis, DOD cannot determine the extent to which their health-care experiences were similar to or different from experiences of patients who did complete the survey.9 Because the group of 8 A DOD official involved with the Survey told us that the response rate has historically been 40 percent or less. 9 One way to assess the extent to which nonrespondents differ from respondents is to conduct a nonresponse analysis. A nonresponse analysis is a technique used to determine the difference between those who responded and those who did not respond to a survey, and the extent to which the respondents represent the overall population. A nonresponse analysis for a mail survey is usually conducted by administering the survey over the telephone. Page 11 GAO/HEHS-99-168 Access to Defense Health Care B-279619 nonrespondents is so large, their experiences, if different from the experiences of the respondents, could dramatically change the survey results. In regard to measuring civilian provider appointment timeliness, DOD is developing a survey modeled after the Customer Satisfaction Survey. However, the limitations of the MTF Customer Satisfaction Survey would also apply to the civilian survey. Thus, while the civilian survey might provide some general indications about beneficiaries’ experiences with civilian providers, it would not capture precise data needed to assess how well the access standards are being met in the civilian network. CHCS Appointment Data Appointment data taken directly from the CHCS appointment scheduling Need Modification to Be a system used by all MTFs potentially could be the best data DOD has Viable Measurement Tool available to measure the performance of MTFs in meeting the access standards. While CHCS data are not vulnerable to the limitations inherent in the Customer Satisfaction Survey, the CHCS has other shortcomings that limit its current usefulness as a tool to measure appointment timeliness. A critical weakness of the CHCS data for appointment-measuring purposes is that the appointment names used in the MTF’s appointment scheduling system do not directly relate the types of visits to the standards. We found that four of the eight MTFs in our study used appointment names within their scheduling systems that could not be linked to only one appointment timeliness standard. For example, at one MTF the appointment name “PRIME” was used to book acute, routine, and well primary-care appointments, which are each subject to different access standards. At another MTF, the appointment name “PACU” was used to book acute and routine appointments, while the name “ROUP” was used to book routine, follow-up, and well appointments. In these cases, more than one timeliness standard would be applicable and the MTF would not know which standard to use to measure its performance in making timely appointments. Unless MTFs link their appointment names to a single standard, they will be unable to determine the extent to which their appointments are in compliance with the appointment timeliness standards. In addition, the lack of standard appointment names among the MTFs prevents DOD from consolidating individual facility CHCS data into regional or systemwide data. Under DOD’s current procedures, each MTF has the flexibility to design a unique appointment system. This practice hampers Page 12 GAO/HEHS-99-168 Access to Defense Health Care B-279619 DOD’s ability to collect and monitor appointment data across the military health-care system. For example, DOD would have to know exactly which names were used in every MTF’s appointment system and which of the access standards applied to the appointment name. In our review of appointment names in use at eight MTFs, we found 14 different names for appointments associated with the timeliness standard for acute appointments, 18 different names for appointments associated with the timeliness standard for routine appointments, and 35 different names for appointments associated with the timeliness standard for well visits. Even though some of these MTFs could associate the appointment names they used with the applicable timeliness standard, the lack of consistent and standard appointment names across a system of more than 450 MTFs with potentially thousands of appointment names would make any effort to collect, monitor, and regularly report on systemwide appointment data a complex and complicated undertaking. DOD Attempting to DOD officials told us they recognize the weaknesses of the Survey and CHCS Resolve Data Weaknesses data and have some efforts under way or planned to address some of the weaknesses. With regard to the Survey, they acknowledged that beneficiaries are confused when trying to categorize their appointments, especially because the Survey does not define the categories. According to DOD officials, providing some general definitions with examples of appointments could help beneficiaries responding to the Survey. However, they believe it is not possible to provide sufficient examples to completely eliminate the confusion and ensure correct categorization, and are not planning any revisions to that Survey question. The officials also agreed that memory recall about the number of days it took to get an appointment was a concern. DOD officials said that sending the Survey closer to the appointment date might improve memory recall, but the administrative tasks associated with selecting the sample and mailing the Survey could not be hastened. Related to the lack of a nonrespondent analysis, DOD recently decided to conduct the first analysis of nonrespondents in fiscal year 2000. With regard to the CHCS data, DOD has efforts under way or planned that should address the critical weaknesses that affect the usefulness of the data in measuring appointment timeliness. DOD officials told us that a policy is being developed requiring MTFs to correlate their primary-care appointments to the three timeliness standards and to standardize the appointment names across the military health-care system. Other enhancements are also planned that will improve the accuracy of CHCS Page 13 GAO/HEHS-99-168 Access to Defense Health Care B-279619 appointment timeliness data. Officials estimate that CHCS data would be reliable for monitoring and measuring access in MTFs by March 2000 after these changes and improvements are tested and implemented throughout the military health-care system. If successful, DOD could rely on CHCS and cease using the Survey as a means of measuring compliance with the timeliness standards at the MTF, regional, and systemwide level. Active-duty and other Prime enrollees have not been able to obtain Conclusions appointments within the prescribed timeliness standards to the extent that DOD expected when it first established goals for TRICARE. Moreover, the performance in meeting standards is about the same for active-duty members, who have the highest priority, and nonenrolled beneficiaries, who have the lowest priority. In some cases, appointments are scheduled outside the standards due to the beneficiary’s request for a later appointment to meet personal needs. However, appointments within the standards for enrolled beneficiaries may not be available because nonenrolled beneficiaries have filled available appointment slots ahead of them. Providing care to nonenrollees, especially those who are eligible to enroll in Prime, counters the program’s intention that eligible beneficiaries enroll, and reinforces some beneficiaries’ view that they can still obtain care in the MTFs without enrolling. There are several options DOD could test to improve the availability of appointments for active-duty and other enrolled beneficiaries. These include more vigorously enforcing systemwide access priorities, to the extent of giving appointments booked for nonenrollees to enrolled beneficiaries in need of an appointment within the standard. Also, eliminating care to nonenrolled beneficiaries that exceeds medical training requirements could result in more available appointments. Lastly, instituting a copayment in the MTFs could lead to more appropriate utilization of care in MTFs, thereby opening up additional appointment slots for enrollees. While copayments could help improve appointment timeliness, potential benefits actually go well beyond this. Copayments would also serve to equalize the cost-sharing for all beneficiaries, regardless of whether they receive care from military or civilian providers, by eliminating the inherent inequity of providing more generous health benefits to those who live near an MTF. It would also allow physicians to refer beneficiaries to the most appropriate provider—whether military or civilian—without regard to the financial implications of the referral for the beneficiary. Page 14 GAO/HEHS-99-168 Access to Defense Health Care B-279619 The two data tools that provide information on appointment timeliness in MTFs have significant weaknesses that affect the accuracy and sufficiency of the data. DOD is undertaking efforts to address the weaknesses of the CHCS appointment system by requiring MTFs to associate appointment names with the access standards and by establishing standard appointment names across the system. Regarding the Customer Satisfaction Survey, we agree that it provides DOD with meaningful information on how beneficiaries feel about their health-care experiences and can be used for this purpose. Furthermore, the planned analysis of nonrespondents will further improve the data. However, two remaining weaknesses in the Survey—beneficiary categorization of appointments and reliance on memory recall—are sufficiently significant to continue to call into question the validity of the results as a measure of either civilian providers’ or MTFs’ performance against the standards. While we recognize the challenges DOD faces in obtaining comprehensive information on civilian providers’ performance, accurate and appropriate data to measure how well MTFs are meeting the standards can and should be obtained from MTF data sources, not from beneficiaries. Therefore, the CHCS system should be the primary data source for determining MTF compliance with the access standard. In our view, it is imperative that DOD implement changes to the CHCS system as soon as possible so that it can meet its responsibilities to beneficiaries and more effectively manage access to the MTFs. We recommend that the Secretary of Defense direct the Assistant Recommendations Secretary of Defense (Health Affairs) to measure and monitor beneficiaries’ access to health care in a more comprehensive and accurate manner by directing that CHCS be used in lieu of the Customer Satisfaction Survey to measure compliance with the appointment timeliness standards in the MTFs and that the necessary modifications be made to CHCS so that appointment names are linked to the appropriate access standard and standardized across the military health-care system. The Secretary should direct that the results be reported at all levels—individual facility, service- and system-wide, and by the various beneficiary categories. The Secretary should also direct a test of a policy that appointments scheduled for nonenrolled beneficiaries are subject to cancellation if an active-duty member or other Prime enrollee requests care and no other appointment is available within the access standard. This test could be implemented in those MTFs having the greatest difficulty scheduling Page 15 GAO/HEHS-99-168 Access to Defense Health Care B-279619 active-duty members and other Prime enrollees within the access standards. The Secretary should also test the option of instituting copayments within the MTFs comparable to those in the civilian networks to help bring about more appropriate utilization of military care and thus free up appointment space. We provided a draft of this report to DOD for review and comment, but DOD Agency Comments has not provided comments. As agreed with your offices, we are sending copies of this report to the Honorable William C. Cohen, Secretary of Defense, and will make copies available to others upon request. Please contact me on (202) 512-7111 or Michael T. Blair, Jr., Assistant Director, on (404) 679-1944 if you or your staff have any questions. Other major contributors to this report are listed in appendix II. Stephen P. Backhus, Director Veterans’ Affairs and Military Health Care Issues Page 16 GAO/HEHS-99-168 Access to Defense Health Care Page 17 GAO/HEHS-99-168 Access to Defense Health Care Contents Letter 1 Appendix I 20 Scope and Methodology Appendix II 24 GAO Contacts and Staff Acknowledgments Tables Table 1: DOD Appointment Scheduling Standards, Goals, and 6 Appointments Scheduled Within Standards for Active-Duty and Other Prime Enrollees Table 2: Comparison of Appointments Scheduled Within 7 Standards for Active-Duty, Other Prime Enrollees, and Nonenrolled Beneficiaries Table I.1: Primary and Specialty Care Clinics at Eight MTFs 22 Providing Appointment Data Abbreviations CBO Congressional Budget Office CHCS Composite Health Care System DOD Department of Defense MTF military treatment facility Page 18 GAO/HEHS-99-168 Access to Defense Health Care Page 19 GAO/HEHS-99-168 Access to Defense Health Care Appendix I Scope and Methodology To obtain information on the Department of Defense’s (DOD) access Scope policies and measures, we met with officials in the Office of the Assistant Secretary of Defense (Health Affairs) and the TRICARE Management Activity, who are responsible for managing the military health-care program. We also spoke with staff of the three Services’ Surgeons General and three of the TRICARE managed care support contractors. We discussed local access policies and appointment procedures with officials at 15 military treatment facilities (MTF) and visited 12 of the facilities. We also reviewed DOD standards for primary- and specialty-care appointment timeliness and DOD’s policy on priority for access to care in MTFs. We researched the access standards used by commercial managed care plans and how they measure their performance against these standards. We reviewed accreditation standards related to access to care from two health-care industry accreditation bodies—the Joint Commission on Accreditation of Healthcare Organizations and the National Committee for Quality Assurance. We also gathered information on appointment timeliness and access standards used by individual private-sector health care plans, as well as cost-sharing requirements. For the purpose of this study, we defined access as appointment Methodology timeliness—measuring the number of elapsed days between the date that the beneficiary requests an appointment and the scheduled appointment date. We selected this measure because (1) it was the access measure for which DOD had established criteria or standards against which its performance could be measured, and (2) appointment data were available throughout the military health care system from the Customer Satisfaction Survey and the Composite Health Care System (CHCS). However, because of limitations in DOD’s data from both sources, we could not use the data as they existed in DOD’s systems, and designed a methodology and analysis approach (discussed below) to minimize the effect of the limitations. Customer Satisfaction We made several adjustments to the Survey data to minimize the Survey weaknesses and correct discrepancies. Because we had concerns about whether beneficiaries had correctly classified their visits, we did not use their classification from the Survey. Instead, we used the sample selection data that provided information on the clinics the beneficiaries visited. From these data, we identified primary-care visits, which we defined as a visit to one of the four clinics DOD now considers to be a primary-care clinic throughout the military health care system—family practice, primary Page 20 GAO/HEHS-99-168 Access to Defense Health Care Appendix I Scope and Methodology care, flight medicine, or pediatrics. In consultation and agreement with DOD officials responsible for administering the Survey, we considered all appointments not associated with one of the four primary clinics to be a specialty appointment. We did not include information on visits from certain clinics for which the access standards were not applicable, such as mental health clinics and emergency departments. Our analysis covered survey responses for appointments in the 5-month period from May 1, 1998, to September 30, 1998. We selected the beginning date of May 1, 1998, so that our analysis contained only responses since the survey instrument was revised in May 1998. September 30, 1998, was the latest date for which data were available at the time of our request. To determine appointment timeliness and compliance with the primary-care standards, we relied on the beneficiaries’ response to the question asking them how many days it took to obtain the appointment and compared it to the relevant access standard determined by the clinic of their visit, as discussed above, and the beneficiaries’ categorization as to the urgency of the visit (for those determined to be primary-care visits). In analyzing the data by beneficiary category and enrollment status, we found and corrected discrepancies. We considered all active-duty respondents as enrolled in Prime, and retirees over age 65 as not enrolled in Prime, regardless of how they responded to the survey question about their enrollment status. CHCS Appointment Data As a result of our discussions with MTF officials, we confirmed that the CHCS appointment system could provide the information we needed to assess the appointment timeliness at the MTF level. This information included beneficiary category and enrollment status, the date the beneficiary requested the appointment, the date of the scheduled appointment, and appointment type or name. Our analysis of CHCS data also confirmed that the key limitation with these data was determining for each appointment name in the system which appointment timeliness standard was relevant. We asked eight MTFs to provide the appointment names used in their scheduling system that were subject to each of the access standards for their primary care.10 We also asked them to provide the same information on appointment names for selected specialty clinics. After obtaining the information on the appointment name used, we asked each of the eight MTFs to provide us with 12 months of appointment data for the identified 10 We selected the MTFs to represent the different Services, TRICARE contractors, areas of the country, and size of medical facility. Page 21 GAO/HEHS-99-168 Access to Defense Health Care Appendix I Scope and Methodology primary-care and specialty clinics. Table I.1 shows the eight MTFs and the clinics for which we obtained data. Table I.1: Primary and Specialty Care Clinics at Eight MTFs Providing Appointment Data MTF Primary care clinics Specialty care clinics Fort Benning/Martin Army Internal Medicine, Family Practice Gynecology, Internal Medicine, Optometry, Podiatry, Allergy, Community Hospital (2 clinics), Aviation Medicine Physical Therapy, Orthopedics, Ophthalmology, Dermatology, Nutrition, Obstetrics, Otorhinolaryngology, Urology Davis-Monthan AFB/355th Medical Family Practice (2 clinics) Gynecology, Internal Medicine, Optometry, Physical Therapy, Group General Surgery, Orthopedics, Dermatology Fort Hood/Darnall Army Community Family Care (3 clinics), Pediatrics Gynecology, Internal Medicine, Optometry, Podiatry, Allergy, Hospital (3 clinics) Physical Therapy, General Surgery, Orthopedics, Ophthalmology, Dermatology, Nutrition, Neurology, Obstetrics, Women’s Health, Urology Naval Hospital Oak Harbor Family Practice (3 clinics), Gynecology, Internal Medicine, Optometry, Physical Therapy, Primary Care (3 clinics), General Surgery, Obstetrics Pediatrics (3 clinics), Aviation Medicine (3 clinics) Mountain Home AFB/366th Medical Primary Care (3 clinics) Gynecology, Internal Medicine, Optometry, Physical Therapy, Group General Surgery, Obstetrics Wilford Hall Medical Center Family Medicine, Internal Gynecology, Obstetrics, Optometry, Podiatry, Allergy, Medicine, Women’s Health, Physical Therapy, Cardiology, General Surgery, General Pediatrics Ophthalmology, Dermatology, Nutrition, Neurology, Otorhinolaryngology, Orthopedics, Urology Fort Rucker/Lyster Army Aviation Medicine, Ambulatory Gynecology, Internal Medicine, Optometry, Physical Therapy, Community Hospital Care, Family Practice General Surgery, Orthopedics, Ophthalmology, Dermatology Naval Hospital Jacksonville Primary Care, Family Practice (2 Allergy, Dermatology, General Surgery, Gynecology, Internal clinics), Pediatrics Medicine, Neurology, Nutrition, Ophthalmology, Orthopedics, Otorhinolaryngology, Physical Therapy, Urology From the clinic appointment data, we calculated the number of days between the date the beneficiary requested the appointment and the date of the scheduled appointment. We analyzed these data by different variables, including beneficiary category and whether the beneficiary was enrolled in TRICARE Prime. For our analysis, we assumed that the patient’s visit to the provider was in fact for the type of visit indicated by the appointment name and timeliness standard. We could not correct the data to reflect any instances in which patients were scheduled into appointment types that were different from the type they requested. Another discrepancy we found was that in some cases active-duty personnel were recorded as not enrolled in Prime, when they are actually considered automatically enrolled. We corrected for this by recoding all active-duty as enrolled in Prime, regardless of the enrollment status field in the CHCS data. Similarly, we recoded all retirees over age 65 as not Page 22 GAO/HEHS-99-168 Access to Defense Health Care Appendix I Scope and Methodology enrolled despite the enrollment status shown in the data. We ultimately had to exclude data from two MTFs—Fort Hood and Wilford Hall Medical Center—because some of their appointment types were associated with more than one standard, which precluded comparing the data against the access standards. Page 23 GAO/HEHS-99-168 Access to Defense Health Care Appendix II GAO Contacts and Staff Acknowledgments Michael T. Blair, Jr., (404) 679-1944 GAO Contacts Nancy T. Toolan, (404) 679-1983 In addition to those mentioned above, Sylvia D. Jones, Linda S. Lootens, Staff Deborah L. Edwards, and Beverly Brooks-Hall made key contributions to Acknowledgments this report. (101613) Page 24 GAO/HEHS-99-168 Access to Defense Health Care Ordering Information The first copy of each GAO report and testimony is free. Additional copies are $2 each. Orders should be sent to the following address, accompanied by a check or money order made out to the Superintendent of Documents, when necessary. VISA and MasterCard credit cards are accepted, also. Orders for 100 or more copies to be mailed to a single address are discounted 25 percent. Orders by mail: U.S. General Accounting Office P.O. Box 37050 Washington, DC 20013 or visit: Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders may also be placed by calling (202) 512-6000 or by using fax number (202) 512-6061, or TDD (202) 512-2537. Each day, GAO issues a list of newly available reports and testimony. To receive facsimile copies of the daily list or any list from the past 30 days, please call (202) 512-6000 using a touchtone phone. A recorded menu will provide information on how to obtain these lists. For information on how to access GAO reports on the INTERNET, send an e-mail message with "info" in the body to: email@example.com or visit GAO’s World Wide Web Home Page at: http://www.gao.gov PRINTED ON RECYCLED PAPER United States Bulk Rate General Accounting Office Postage & Fees Paid Washington, D.C. 20548-0001 GAO Permit No. G100 Official Business Penalty for Private Use $300 Address Correction Requested
Defense Health Care: Appointment Timeliness Goals Not Met; Measurement Tools Need Improvement
Published by the Government Accountability Office on 1999-09-30.
Below is a raw (and likely hideous) rendition of the original report. (PDF)