oversight

Health Professions Education: Clarifying the Role of Title VII and VIII Programs Could Improve Accountability

Published by the Government Accountability Office on 1997-04-25.

Below is a raw (and likely hideous) rendition of the original report. (PDF)

                          United States General Accounting Office

GAO                       Testimony
                          Before the Subcommittee on Public Health and Safety,
                          Committee on Labor and Human Resources, U.S. Senate




For Release on Delivery
Expected at 9:30 a.m.
Friday, April 25, 1997
                          HEALTH PROFESSIONS
                          EDUCATION

                          Clarifying the Role of Title
                          VII and VIII Programs Could
                          Improve Accountability
                          Statement for the Record by Bernice Steinhardt, Director
                          Health Services Quality and Public Health Issues
                          Health, Education, and Human Services Division




GAO/T-HEHS-97-117
Health Professions Education: Clarifying the
Role of Title VII and VIII Programs Could
Improve Accountability
               Mr. Chairman and Members of the Subcommittee:

               We are pleased to contribute this statement for the Subcommittee’s
               deliberations on reauthorizing health professions education programs
               established under titles VII and VIII of the Public Health Service Act.1 In
               fiscal year 1996, the Congress provided nearly $300 million for about 40
               programs under these titles. These programs, administered through the
               Department of Health and Human Services (HHS),2 provide direct student
               assistance, such as loans, as well as grants to institutions for expansion or
               maintenance of health professions education and training.

               When it last reauthorized titles VII and VIII in 1992, the Congress required
               us to report on whether these programs were effective in advancing three
               key purposes—increasing the numbers of health professionals, improving
               their distribution in locations that have a shortage of health professionals,
               and adding minorities to their ranks. Since our 1994 report,3 two important
               developments have occurred with regard to these programs. First, the
               Government Performance and Results Act of 1993 (GPRA), which was
               newly enacted when we issued our report, requires federal agencies to be
               more accountable for the results of their efforts. Second, both the
               Congress and the administration have proposed placing many existing
               Title VII and VIII programs into five or six “clusters” as part of efforts to
               streamline government. In preparing this statement, we relied on our
               earlier report and conducted a limited amount of follow-up work to review
               the current status of the Title VII and VIII programs and to place our
               earlier findings in the context of these new developments.

               Our comments will focus on (1) problems we identified in linking these
               programs to changes in the supply, distribution, and minority
               representation of health professionals and the impact of these changes on
               access to care; (2) the potential for implementation of GPRA to address
               these problems; and (3) the opportunities associated with consolidating
               the separate programs into program clusters.

               In brief, we found that the effectiveness of Title VII and VIII programs will
               remain difficult to measure as long as they are authorized to support a
               broad range of health care objectives without common goals, outcome


               1
                42 U.S.C. 292-298b-7 (1994).
               2
                HHS’ Health Resources and Services Administration (HRSA) administers these programs.
               3
                Health Professions Education: Role of Title VII/VIII Programs in Improving Access to Care Is Unclear
               (GAO/HEHS-94-164, July 8, 1994).



               Page 1                                                                         GAO/T-HEHS-97-117
                      Health Professions Education: Clarifying the
                      Role of Title VII and VIII Programs Could
                      Improve Accountability




                      measures, and reporting requirements. The implementation of GPRA and
                      the “clustering” concept offer an opportunity to address these problems by
                      providing the framework and flexibility to clarify the role of Title VII and
                      VIII programs and direct federal efforts to achieve desired outcomes.
                      However, unless steps are taken with a clear eye to addressing the
                      problems, these developments could meet with little success.


                      In 1994, we reported that the supply of nearly all types of health
Role of Programs in   professionals had increased faster than the population. Moreover, the
Improving Access Is   number of underrepresented minorities in health professions education
Unclear               for which data were available4 had increased faster than the number for all
                      races combined. For most professions, however, data were not available
                      to demonstrate whether this increased supply and minority representation
                      translated into more access to care in rural and underserved areas. For the
                      two professions with the most data available—primary care physicians
                      and general dentists—supply increased in many rural areas but not in
                      those urban and rural areas where the greatest shortages existed.

                      We also reported that evaluations of Title VII and VIII programs have not
                      linked these programs to the changes in the supply, distribution, and
                      minority representation of health professionals. HHS is not required to
                      evaluate the effectiveness of each program, and 6 of the 23 programs
                      established before 1990 have never been evaluated.5 Evaluations
                      conducted on the remaining programs generally addressed the impact at
                      individual institutions and found that the programs have assisted schools
                      in improving or enhancing curricula, funding innovative projects, and
                      providing seed money for starting new programs. However, the results of
                      virtually all of these evaluations could not be generalized to determine the
                      national impact of the programs in the three key areas. Such a relationship
                      is difficult to establish for several reasons, including (1) the wide variety of
                      often unrelated objectives that the programs addressed and (2) problems
                      with the data and criteria used to measure the outcomes of what the
                      programs were accomplishing.



                      4
                       HHS has identified African-Americans, Native Americans, and Hispanics as underrepresented in the
                      health professions. Only data for physicians, dentists, and registered nurses were available for
                      minority applicants, first-year enrollments, and graduates of health professions schools.
                      5
                       The Secretary of HHS is authorized by statute to set aside up to 1 percent of Public Health Service
                      (PHS) appropriations for evaluations. We previously reported that implementation of this set-aside has
                      been less than fully effective in providing information to the Congress on PHS programs. See Public
                      Health Service: Evaluation Set-Aside Has Not Realized Its Potential to Inform the Congress
                      (GAO/PEMD-93-13, Apr. 8, 1993).



                      Page 2                                                                         GAO/T-HEHS-97-117
                        Health Professions Education: Clarifying the
                        Role of Title VII and VIII Programs Could
                        Improve Accountability




Multiple Program        While over the past 2 decades congressional interest in Title VII and VIII
Objectives              programs has focused on their utility in adding to the number of health
                        care professionals, placing these professionals in underserved areas, and
                        training more minority health professionals, the programs themselves have
                        a variety of objectives. Titles VII and VIII, established in 1963 and 1964 and
                        amended over time, authorize funding for a number of programs with
                        diverse objectives. While most of the programs address at least one of the
                        three key areas of improving the supply, distribution, and minority
                        representation of health professionals, they also address other objectives
                        as well. These other objectives, such as improving the quality of education
                        and training, may only indirectly result in improvements to the three key
                        areas. Furthermore, HHS officials identified some programs, including
                        grants for chiropractic demonstration projects, that do not have objectives
                        related to any of the three key areas.

                        The large number and piecemeal approach of Title VII and VIII program
                        objectives make evaluating program impact difficult. For example, one
                        institution received a $300,590 family medicine grant to further the
                        achievement of 12 separate objectives. One of the 12 objectives was to
                        directly improve distribution and minority representation; none was for
                        increasing supply. The other 11 were for various curricula improvements,
                        such as expanding the behavioral science curriculum and maintaining the
                        physician practice-management curriculum. While these 11 other
                        objectives may be valuable in their own right, they represent federally
                        funded activities that could not be directly linked to, and thus evaluated as
                        affecting, these three key areas.


Problems With Outcome   Another problem hindering evaluation was that none of the Title VII and
Measures                VIII programs at the time of our review had established specific program
                        outcome measures—that is, the desired results—against which to gauge
                        their effectiveness. Establishing results-oriented measures is difficult
                        because to set such measures, HHS must move beyond what it
                        controls—that is, the activities—to focus on what it merely
                        influences—the results. We found that some grantees reported on the
                        process they established to achieve results, rather than on the results
                        themselves. For example, a grantee reported that it instituted a
                        recruitment activity but did not report how many students were actually
                        recruited through federal funding of this activity.

                        We also identified problems in the cases in which HHS had begun collecting
                        data to measure program outcomes. For example, data provided to HHS to



                        Page 3                                                      GAO/T-HEHS-97-117
                        Health Professions Education: Clarifying the
                        Role of Title VII and VIII Programs Could
                        Improve Accountability




                        qualify for a funding preference for placing graduates in medically
                        underserved communities were not necessarily complete or comparable
                        among schools, and HHS had not established a way to validate the data
                        provided. Even if these data problems are resolved, other work we have
                        conducted shows that the underlying criteria used to identify some
                        medically underserved communities are outdated and flawed.6 For
                        example, more than half of the locations designated as underserved may
                        be invalid because the data are outdated or do not consider a significant
                        number of primary care providers, such as nurse practitioners or physician
                        assistants. Without valid criteria and data against which to measure
                        grantee performance, it is difficult to determine whether grantee efforts
                        under Title VII and VIII programs are needed and will be successful, or if
                        other federal programs would be more appropriate.


                        GPRA  was intended, in part, to deal with the types of problems we
Implementation of the   identified and provides HHS and the Congress with an opportunity to
Government              address them. Concerned that federal agencies such as HHS have not
Performance and         always effectively managed their activities to ensure accountability, the
                        Congress has created a legislative framework to address long-standing
Results Act Provides    management challenges throughout the federal government. The
an Opportunity to       centerpiece of this framework is GPRA. Under GPRA, every major federal
                        agency must now ask some basic questions: What is our mission? What are
Address Identified      our goals and how can we achieve them? How can we measure our
Problems                performance? How will we use that information to improve? GPRA requires
                        a strategic plan to be prepared in consultation with the Congress—this
                        plan is due in September 1997.

                        Since HHS is still finalizing its required plans, it is unclear whether
                        implementation of GPRA for the Title VII and VIII programs will resolve the
                        problems we identified regarding the number and variation of program
                        goals. One unresolved issue is the degree to which Title VII and VIII
                        program goals will be considered in relationship to the other HHS programs
                        for health professions education and training. Because HHS’ influence on
                        education and training involves multiple efforts spanning several of its
                        agencies, Title VII and VIII programs should not be considered in a
                        vacuum. HHS officials responsible for administering Title VII and VIII
                        programs cite the influences of other, larger HHS programs on health
                        professions education and training programs. For example, the officials
                        said they believe the incentives for primary care education and training

                        6
                         More specifically, the designation systems we evaluated were the Health Professional Shortage Areas
                        and Medically Underserved Areas. See Health Care Shortage Areas: Designations Not a Useful Tool for
                        Directing Resources to the Underserved (GAO/HEHS-95-200, Sept. 8, 1995).



                        Page 4                                                                        GAO/T-HEHS-97-117
Health Professions Education: Clarifying the
Role of Title VII and VIII Programs Could
Improve Accountability




provided by Title VII programs are counteracted by the billions of dollars
of federal funding from the Medicare program to support the training of
specialists and from the National Institutes of Health to support
biomedical research at medical schools. HHS officials added that goals of
improving supply, distribution, or minority representation nationally are
unrealistic for the few hundred million dollars that fund Title VII and VIII
programs given the multibillion dollar training environment. Clarifying the
intended nature and extent of the impact of Title VII and VIII programs
remains an important step in overcoming the problems we identified.

The consultation process under GPRA gives the Congress and HHS an
opportunity to reach an understanding of what role Title VII and VIII
programs should play in this broader context of programs—and what
goals and desired outcomes should be set specifically for Title VII and VIII
programs. In discussing the challenges of GPRA implementation, HHS
officials noted that funding for Title VII and VIII programs is often
intended to have impact at the margin—that is, to affect an individual
institution or recipient—and therefore long-term outcomes solely
attributable to funding these programs are difficult to measure. HHS
officials are currently developing cross-cutting goals and indicators for the
Title VII and VIII programs. Although benchmarks and specific goals
against which to measure the success of these programs have not yet been
established, the officials said they plan on measuring the impact on
projects that receive Title VII and VIII funding, such as counting the
number of enrollees at program-supported institutions, rather than
measuring changes in national indicators.

For successful GPRA implementation, performance information must be
used to direct resources where federal intervention would have a greater
impact. One area that would benefit from this process is HHS’ goal of
increasing minority representation in the health professions. Although
some minority groups are underrepresented in the health professions
when compared with their overall percentage in the U.S. population, some
groups are not underrepresented when the comparison is based more
narrowly on the segment of the population with the necessary educational
background to enter into health professions education and training. This
may mean that federal efforts could be better spent on bringing more
minority students to the point of being able to enter health professions
education, instead of on helping the relatively few who already have those
qualifications. GPRA, with its emphasis on targeting federal efforts on more
cost-effective ways to achieve agreed-upon goals, could help to surface
such considerations.



Page 5                                                      GAO/T-HEHS-97-117
                         Health Professions Education: Clarifying the
                         Role of Title VII and VIII Programs Could
                         Improve Accountability




                         Like GPRA, recent legislative proposals provide an opportunity to focus
Proposals to Cluster     Title VII and VIII moneys in the most effective ways. In response to
Title VII and VIII       national efforts to streamline government, recent reauthorization
Programs Could           proposals by the Senate and the administration combine about 40 Title VII
                         and VIII categorical programs into 5 or 6 program clusters.7 Under such an
Provide Flexibility to   approach, the Congress could authorize and appropriate funds for each
Target Resources         cluster of programs instead of authorizing and appropriating funds for
                         each program. HHS could have the authority to fund programs within the
                         cluster but would no longer be required to fund each individual program.
                         This would give HHS more flexibility, in conjunction with GPRA, to
                         determine how to spend the money to achieve stated goals.

                         This cluster concept could provide greater flexibility to target resources to
                         the most effective programs and to discontinue federal support when
                         agreed-upon goals have been achieved. However, as with GPRA, achieving
                         greater success with this approach is not automatic. Increased agency
                         flexibility and related discretion would make it even more critical that the
                         desired outcomes of the programs be clarified so that resources could be
                         allocated on the basis of the need for and effectiveness of specific
                         programs. Unless these issues are addressed, the risk continues that
                         money will be spent without a clear idea of what is being
                         accomplished—and whether spending it differently would produce greater
                         results.


                         An appropriate number and mix of health professionals are vital to
Concluding               ensuring that all Americans have adequate access to health care. Our work
Observations             points to the need to clarify the role of Title VII and VIII programs in
                         improving the supply, distribution, and minority representation of health
                         professionals and whether these programs are intended to affect the
                         health professions at the national level. If these programs are to
                         specifically improve supply, distribution, and minority representation of
                         health professionals, federal efforts need to be directed to activities that
                         clearly support those goals and whose results can be measured and
                         reported in terms of those goals. Similarly, if the programs are to meet
                         other goals, such as improving curricula to address emerging national
                         health issues, federal efforts need to be directed to the most effective
                         means of achieving them. Regardless of which direction is chosen, once
                         goals are defined, performance measures and targets are critical to



                         7
                          The Senate proposal would also combine a Title III program, the National Health Service Corps, with
                         the Title VII and VIII programs in one cluster.



                         Page 6                                                                        GAO/T-HEHS-97-117
               Health Professions Education: Clarifying the
               Role of Title VII and VIII Programs Could
               Improve Accountability




               determine when federal intervention is no longer required, or when federal
               strategies are not successful and should be redirected.

               The implementation of GPRA and reauthorization of the programs provide
               an ideal opportunity to identify where Title VII and VIII programs fit within
               the federal government’s overall strategy for addressing national health
               workforce issues. In doing so, HHS and the Congress can establish vital
               national goals and common outcome measures for HHS programs and
               allocate limited federal funds to those programs, including programs
               outside of Title VII and VIII, based on demonstrated effect and relative
               need in meeting national goals.


               This statement was prepared under the direction of Bernice Steinhardt,
Contributors   Director, Health Services Quality and Public Health Issues, who may be
               reached at (202) 512-7119 if there are any questions. Other key
               contributors include Frank Pasquier, Assistant Director, and Lacinda
               Baumgartner, Kim Yamane, and Stan Stenersen, Evaluators.




(108326)       Page 7                                                     GAO/T-HEHS-97-117
Ordering Information

The first copy of each GAO report and testimony is free.
Additional copies are $2 each. Orders should be sent to the
following address, accompanied by a check or money order
made out to the Superintendent of Documents, when
necessary. VISA and MasterCard credit cards are accepted, also.
Orders for 100 or more copies to be mailed to a single address
are discounted 25 percent.

Orders by mail:

U.S. General Accounting Office
P.O. Box 6015
Gaithersburg, MD 20884-6015

or visit:

Room 1100
700 4th St. NW (corner of 4th and G Sts. NW)
U.S. General Accounting Office
Washington, DC

Orders may also be placed by calling (202) 512-6000
or by using fax number (301) 258-4066, or TDD (301) 413-0006.

Each day, GAO issues a list of newly available reports and
testimony. To receive facsimile copies of the daily list or any
list from the past 30 days, please call (202) 512-6000 using a
touchtone phone. A recorded menu will provide information on
how to obtain these lists.

For information on how to access GAO reports on the INTERNET,
send an e-mail message with "info" in the body to:

info@www.gao.gov

or visit GAO’s World Wide Web Home Page at:

http://www.gao.gov




PRINTED ON    RECYCLED PAPER
United States                       Bulk Rate
General Accounting Office      Postage & Fees Paid
Washington, D.C. 20548-0001           GAO
                                 Permit No. G100
Official Business
Penalty for Private Use $300

Address Correction Requested