United States General Accounting Office GAO Testimony Before the Subcommittee on Public Health and Safety, Committee on Labor and Human Resources, U.S. Senate For Release on Delivery Expected at 9:30 a.m. Friday, April 25, 1997 HEALTH PROFESSIONS EDUCATION Clarifying the Role of Title VII and VIII Programs Could Improve Accountability Statement for the Record by Bernice Steinhardt, Director Health Services Quality and Public Health Issues Health, Education, and Human Services Division GAO/T-HEHS-97-117 Health Professions Education: Clarifying the Role of Title VII and VIII Programs Could Improve Accountability Mr. Chairman and Members of the Subcommittee: We are pleased to contribute this statement for the Subcommittee’s deliberations on reauthorizing health professions education programs established under titles VII and VIII of the Public Health Service Act.1 In fiscal year 1996, the Congress provided nearly $300 million for about 40 programs under these titles. These programs, administered through the Department of Health and Human Services (HHS),2 provide direct student assistance, such as loans, as well as grants to institutions for expansion or maintenance of health professions education and training. When it last reauthorized titles VII and VIII in 1992, the Congress required us to report on whether these programs were effective in advancing three key purposes—increasing the numbers of health professionals, improving their distribution in locations that have a shortage of health professionals, and adding minorities to their ranks. Since our 1994 report,3 two important developments have occurred with regard to these programs. First, the Government Performance and Results Act of 1993 (GPRA), which was newly enacted when we issued our report, requires federal agencies to be more accountable for the results of their efforts. Second, both the Congress and the administration have proposed placing many existing Title VII and VIII programs into five or six “clusters” as part of efforts to streamline government. In preparing this statement, we relied on our earlier report and conducted a limited amount of follow-up work to review the current status of the Title VII and VIII programs and to place our earlier findings in the context of these new developments. Our comments will focus on (1) problems we identified in linking these programs to changes in the supply, distribution, and minority representation of health professionals and the impact of these changes on access to care; (2) the potential for implementation of GPRA to address these problems; and (3) the opportunities associated with consolidating the separate programs into program clusters. In brief, we found that the effectiveness of Title VII and VIII programs will remain difficult to measure as long as they are authorized to support a broad range of health care objectives without common goals, outcome 1 42 U.S.C. 292-298b-7 (1994). 2 HHS’ Health Resources and Services Administration (HRSA) administers these programs. 3 Health Professions Education: Role of Title VII/VIII Programs in Improving Access to Care Is Unclear (GAO/HEHS-94-164, July 8, 1994). Page 1 GAO/T-HEHS-97-117 Health Professions Education: Clarifying the Role of Title VII and VIII Programs Could Improve Accountability measures, and reporting requirements. The implementation of GPRA and the “clustering” concept offer an opportunity to address these problems by providing the framework and flexibility to clarify the role of Title VII and VIII programs and direct federal efforts to achieve desired outcomes. However, unless steps are taken with a clear eye to addressing the problems, these developments could meet with little success. In 1994, we reported that the supply of nearly all types of health Role of Programs in professionals had increased faster than the population. Moreover, the Improving Access Is number of underrepresented minorities in health professions education Unclear for which data were available4 had increased faster than the number for all races combined. For most professions, however, data were not available to demonstrate whether this increased supply and minority representation translated into more access to care in rural and underserved areas. For the two professions with the most data available—primary care physicians and general dentists—supply increased in many rural areas but not in those urban and rural areas where the greatest shortages existed. We also reported that evaluations of Title VII and VIII programs have not linked these programs to the changes in the supply, distribution, and minority representation of health professionals. HHS is not required to evaluate the effectiveness of each program, and 6 of the 23 programs established before 1990 have never been evaluated.5 Evaluations conducted on the remaining programs generally addressed the impact at individual institutions and found that the programs have assisted schools in improving or enhancing curricula, funding innovative projects, and providing seed money for starting new programs. However, the results of virtually all of these evaluations could not be generalized to determine the national impact of the programs in the three key areas. Such a relationship is difficult to establish for several reasons, including (1) the wide variety of often unrelated objectives that the programs addressed and (2) problems with the data and criteria used to measure the outcomes of what the programs were accomplishing. 4 HHS has identified African-Americans, Native Americans, and Hispanics as underrepresented in the health professions. Only data for physicians, dentists, and registered nurses were available for minority applicants, first-year enrollments, and graduates of health professions schools. 5 The Secretary of HHS is authorized by statute to set aside up to 1 percent of Public Health Service (PHS) appropriations for evaluations. We previously reported that implementation of this set-aside has been less than fully effective in providing information to the Congress on PHS programs. See Public Health Service: Evaluation Set-Aside Has Not Realized Its Potential to Inform the Congress (GAO/PEMD-93-13, Apr. 8, 1993). Page 2 GAO/T-HEHS-97-117 Health Professions Education: Clarifying the Role of Title VII and VIII Programs Could Improve Accountability Multiple Program While over the past 2 decades congressional interest in Title VII and VIII Objectives programs has focused on their utility in adding to the number of health care professionals, placing these professionals in underserved areas, and training more minority health professionals, the programs themselves have a variety of objectives. Titles VII and VIII, established in 1963 and 1964 and amended over time, authorize funding for a number of programs with diverse objectives. While most of the programs address at least one of the three key areas of improving the supply, distribution, and minority representation of health professionals, they also address other objectives as well. These other objectives, such as improving the quality of education and training, may only indirectly result in improvements to the three key areas. Furthermore, HHS officials identified some programs, including grants for chiropractic demonstration projects, that do not have objectives related to any of the three key areas. The large number and piecemeal approach of Title VII and VIII program objectives make evaluating program impact difficult. For example, one institution received a $300,590 family medicine grant to further the achievement of 12 separate objectives. One of the 12 objectives was to directly improve distribution and minority representation; none was for increasing supply. The other 11 were for various curricula improvements, such as expanding the behavioral science curriculum and maintaining the physician practice-management curriculum. While these 11 other objectives may be valuable in their own right, they represent federally funded activities that could not be directly linked to, and thus evaluated as affecting, these three key areas. Problems With Outcome Another problem hindering evaluation was that none of the Title VII and Measures VIII programs at the time of our review had established specific program outcome measures—that is, the desired results—against which to gauge their effectiveness. Establishing results-oriented measures is difficult because to set such measures, HHS must move beyond what it controls—that is, the activities—to focus on what it merely influences—the results. We found that some grantees reported on the process they established to achieve results, rather than on the results themselves. For example, a grantee reported that it instituted a recruitment activity but did not report how many students were actually recruited through federal funding of this activity. We also identified problems in the cases in which HHS had begun collecting data to measure program outcomes. For example, data provided to HHS to Page 3 GAO/T-HEHS-97-117 Health Professions Education: Clarifying the Role of Title VII and VIII Programs Could Improve Accountability qualify for a funding preference for placing graduates in medically underserved communities were not necessarily complete or comparable among schools, and HHS had not established a way to validate the data provided. Even if these data problems are resolved, other work we have conducted shows that the underlying criteria used to identify some medically underserved communities are outdated and flawed.6 For example, more than half of the locations designated as underserved may be invalid because the data are outdated or do not consider a significant number of primary care providers, such as nurse practitioners or physician assistants. Without valid criteria and data against which to measure grantee performance, it is difficult to determine whether grantee efforts under Title VII and VIII programs are needed and will be successful, or if other federal programs would be more appropriate. GPRA was intended, in part, to deal with the types of problems we Implementation of the identified and provides HHS and the Congress with an opportunity to Government address them. Concerned that federal agencies such as HHS have not Performance and always effectively managed their activities to ensure accountability, the Congress has created a legislative framework to address long-standing Results Act Provides management challenges throughout the federal government. The an Opportunity to centerpiece of this framework is GPRA. Under GPRA, every major federal agency must now ask some basic questions: What is our mission? What are Address Identified our goals and how can we achieve them? How can we measure our Problems performance? How will we use that information to improve? GPRA requires a strategic plan to be prepared in consultation with the Congress—this plan is due in September 1997. Since HHS is still finalizing its required plans, it is unclear whether implementation of GPRA for the Title VII and VIII programs will resolve the problems we identified regarding the number and variation of program goals. One unresolved issue is the degree to which Title VII and VIII program goals will be considered in relationship to the other HHS programs for health professions education and training. Because HHS’ influence on education and training involves multiple efforts spanning several of its agencies, Title VII and VIII programs should not be considered in a vacuum. HHS officials responsible for administering Title VII and VIII programs cite the influences of other, larger HHS programs on health professions education and training programs. For example, the officials said they believe the incentives for primary care education and training 6 More specifically, the designation systems we evaluated were the Health Professional Shortage Areas and Medically Underserved Areas. See Health Care Shortage Areas: Designations Not a Useful Tool for Directing Resources to the Underserved (GAO/HEHS-95-200, Sept. 8, 1995). Page 4 GAO/T-HEHS-97-117 Health Professions Education: Clarifying the Role of Title VII and VIII Programs Could Improve Accountability provided by Title VII programs are counteracted by the billions of dollars of federal funding from the Medicare program to support the training of specialists and from the National Institutes of Health to support biomedical research at medical schools. HHS officials added that goals of improving supply, distribution, or minority representation nationally are unrealistic for the few hundred million dollars that fund Title VII and VIII programs given the multibillion dollar training environment. Clarifying the intended nature and extent of the impact of Title VII and VIII programs remains an important step in overcoming the problems we identified. The consultation process under GPRA gives the Congress and HHS an opportunity to reach an understanding of what role Title VII and VIII programs should play in this broader context of programs—and what goals and desired outcomes should be set specifically for Title VII and VIII programs. In discussing the challenges of GPRA implementation, HHS officials noted that funding for Title VII and VIII programs is often intended to have impact at the margin—that is, to affect an individual institution or recipient—and therefore long-term outcomes solely attributable to funding these programs are difficult to measure. HHS officials are currently developing cross-cutting goals and indicators for the Title VII and VIII programs. Although benchmarks and specific goals against which to measure the success of these programs have not yet been established, the officials said they plan on measuring the impact on projects that receive Title VII and VIII funding, such as counting the number of enrollees at program-supported institutions, rather than measuring changes in national indicators. For successful GPRA implementation, performance information must be used to direct resources where federal intervention would have a greater impact. One area that would benefit from this process is HHS’ goal of increasing minority representation in the health professions. Although some minority groups are underrepresented in the health professions when compared with their overall percentage in the U.S. population, some groups are not underrepresented when the comparison is based more narrowly on the segment of the population with the necessary educational background to enter into health professions education and training. This may mean that federal efforts could be better spent on bringing more minority students to the point of being able to enter health professions education, instead of on helping the relatively few who already have those qualifications. GPRA, with its emphasis on targeting federal efforts on more cost-effective ways to achieve agreed-upon goals, could help to surface such considerations. Page 5 GAO/T-HEHS-97-117 Health Professions Education: Clarifying the Role of Title VII and VIII Programs Could Improve Accountability Like GPRA, recent legislative proposals provide an opportunity to focus Proposals to Cluster Title VII and VIII moneys in the most effective ways. In response to Title VII and VIII national efforts to streamline government, recent reauthorization Programs Could proposals by the Senate and the administration combine about 40 Title VII and VIII categorical programs into 5 or 6 program clusters.7 Under such an Provide Flexibility to approach, the Congress could authorize and appropriate funds for each Target Resources cluster of programs instead of authorizing and appropriating funds for each program. HHS could have the authority to fund programs within the cluster but would no longer be required to fund each individual program. This would give HHS more flexibility, in conjunction with GPRA, to determine how to spend the money to achieve stated goals. This cluster concept could provide greater flexibility to target resources to the most effective programs and to discontinue federal support when agreed-upon goals have been achieved. However, as with GPRA, achieving greater success with this approach is not automatic. Increased agency flexibility and related discretion would make it even more critical that the desired outcomes of the programs be clarified so that resources could be allocated on the basis of the need for and effectiveness of specific programs. Unless these issues are addressed, the risk continues that money will be spent without a clear idea of what is being accomplished—and whether spending it differently would produce greater results. An appropriate number and mix of health professionals are vital to Concluding ensuring that all Americans have adequate access to health care. Our work Observations points to the need to clarify the role of Title VII and VIII programs in improving the supply, distribution, and minority representation of health professionals and whether these programs are intended to affect the health professions at the national level. If these programs are to specifically improve supply, distribution, and minority representation of health professionals, federal efforts need to be directed to activities that clearly support those goals and whose results can be measured and reported in terms of those goals. Similarly, if the programs are to meet other goals, such as improving curricula to address emerging national health issues, federal efforts need to be directed to the most effective means of achieving them. Regardless of which direction is chosen, once goals are defined, performance measures and targets are critical to 7 The Senate proposal would also combine a Title III program, the National Health Service Corps, with the Title VII and VIII programs in one cluster. Page 6 GAO/T-HEHS-97-117 Health Professions Education: Clarifying the Role of Title VII and VIII Programs Could Improve Accountability determine when federal intervention is no longer required, or when federal strategies are not successful and should be redirected. The implementation of GPRA and reauthorization of the programs provide an ideal opportunity to identify where Title VII and VIII programs fit within the federal government’s overall strategy for addressing national health workforce issues. In doing so, HHS and the Congress can establish vital national goals and common outcome measures for HHS programs and allocate limited federal funds to those programs, including programs outside of Title VII and VIII, based on demonstrated effect and relative need in meeting national goals. This statement was prepared under the direction of Bernice Steinhardt, Contributors Director, Health Services Quality and Public Health Issues, who may be reached at (202) 512-7119 if there are any questions. Other key contributors include Frank Pasquier, Assistant Director, and Lacinda Baumgartner, Kim Yamane, and Stan Stenersen, Evaluators. (108326) Page 7 GAO/T-HEHS-97-117 Ordering Information The first copy of each GAO report and testimony is free. Additional copies are $2 each. Orders should be sent to the following address, accompanied by a check or money order made out to the Superintendent of Documents, when necessary. VISA and MasterCard credit cards are accepted, also. Orders for 100 or more copies to be mailed to a single address are discounted 25 percent. Orders by mail: U.S. General Accounting Office P.O. Box 6015 Gaithersburg, MD 20884-6015 or visit: Room 1100 700 4th St. NW (corner of 4th and G Sts. NW) U.S. General Accounting Office Washington, DC Orders may also be placed by calling (202) 512-6000 or by using fax number (301) 258-4066, or TDD (301) 413-0006. Each day, GAO issues a list of newly available reports and testimony. To receive facsimile copies of the daily list or any list from the past 30 days, please call (202) 512-6000 using a touchtone phone. A recorded menu will provide information on how to obtain these lists. For information on how to access GAO reports on the INTERNET, send an e-mail message with "info" in the body to: firstname.lastname@example.org or visit GAO’s World Wide Web Home Page at: http://www.gao.gov PRINTED ON RECYCLED PAPER United States Bulk Rate General Accounting Office Postage & Fees Paid Washington, D.C. 20548-0001 GAO Permit No. G100 Official Business Penalty for Private Use $300 Address Correction Requested
Health Professions Education: Clarifying the Role of Title VII and VIII Programs Could Improve Accountability
Published by the Government Accountability Office on 1997-04-25.
Below is a raw (and likely hideous) rendition of the original report. (PDF)